Hadoop新手求助：Sandbox环境下HDFS命令执行失败

Hey there! I’ve seen this exact issue pop up with HDP Sandbox newbies—let’s break down why those errors are happening and how to fix them quickly.

Why You’re Seeing These Errors #

When you run hdfs dfs -ls or hdfs dfs -mkdir mayank without specifying an absolute path, Hadoop tries to operate in your default HDFS home directory (which is /user/root for the root user). If this directory doesn’t exist in HDFS, you’ll get that "No such file or directory" error every time. Also, it’s worth checking if the HDFS services are actually running first.

Step-by-Step Fixes #

1. Verify HDFS Services Are Running

First, make sure the HDFS NameNode and DataNode are up and running. Run this command to check active Hadoop processes:

jps

You should see NameNode, DataNode, and SecondaryNameNode in the output. If not, start the HDFS services. On HDP Sandbox, you can use the Ambari web UI (usually accessible at http://sandbox-hdp:8080) or run this command:

start-dfs.sh

2. Create Your Default HDFS Home Directory

Since you’re using the root user, HDFS doesn’t automatically create /user/root for you. Let’s create it and set the correct permissions:

# Create the directory (the -p flag ensures parent directories are created if missing)
hdfs dfs -mkdir -p /user/root

# Assign ownership to root user
hdfs dfs -chown root:root /user/root

3. Test Your Commands Again

Now try your original commands—they should work smoothly:

# List contents of your HDFS home directory (now exists!)
hdfs dfs -ls

# Create a directory named "mayank" in your HDFS home
hdfs dfs -mkdir mayank

# Verify it was created
hdfs dfs -ls

Pro Tip for Future Use #

Using root isn’t best practice for Hadoop operations. Consider creating a regular user (like mayank) and setting up their corresponding HDFS home directory later. But if you stick with root, just remember to create that /user/root directory first after setting up the Sandbox.

内容的提问来源于stack exchange，提问作者Mayank Kishore