Hadoop部署报错：执行mkdir命令触发UnknownHostException求助

Hey there, let's work through this UnknownHostException: NameNode error together—since you're new to Hadoop and Ubuntu, I'll break every step down so it's easy to follow.

This error pops up because Hadoop can't resolve the hostname NameNode to an actual machine on your network. The fix is straightforward: we'll update Hadoop's configuration files to use localhost (your local machine's hostname) instead.

Step 1: Navigate to Hadoop's configuration directory #

First, head to the folder where Hadoop stores its config files. By default, this is usually:

cd /usr/local/hadoop/etc/hadoop/

If you installed Hadoop in a different location, replace the path with your actual installation directory.

Step 2: Update core-site.xml #

This file handles core Hadoop settings. Open it with a text editor—nano is great for beginners since it's intuitive:

nano core-site.xml

Inside the <configuration> tags, find or add the fs.defaultFS property. Replace any mention of NameNode with localhost:

<property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
</property>

Save and exit: press Ctrl+O, hit Enter to confirm, then Ctrl+X.

Step 3: Update hdfs-site.xml #

This configures HDFS settings. Open the file:

nano hdfs-site.xml

Make sure the NameNode and SecondaryNameNode addresses point to localhost:

<property>
    <name>dfs.namenode.http-address</name>
    <value>localhost:50070</value>
</property>
<property>
    <name>dfs.namenode.secondary.http-address</name>
    <value>localhost:50090</value>
</property>

Save and exit as before.

Step 4: Update mapred-site.xml (if needed) #

If you only see a mapred-site.xml.template file, first copy it to create the actual config file:

cp mapred-site.xml.template mapred-site.xml

Then open the file:

nano mapred-site.xml

Add or update the JobTracker address to use localhost:

<property>
    <name>mapreduce.jobtracker.address</name>
    <value>localhost:8021</value>
</property>

Save and exit.

Step 5: Update yarn-site.xml (if using YARN) #

If you're using YARN for resource management, open this file:

nano yarn-site.xml

Set the ResourceManager address to localhost:

<property>
    <name>yarn.resourcemanager.address</name>
    <value>localhost:8032</value>
</property>

Save and exit.

Step 6: Re-format the NameNode #

Since we've changed core HDFS settings, we need to re-format the NameNode. Note: This will erase all existing data in HDFS—since you're setting up a new project, this is safe.

hdfs namenode -format

Step 7: Restart Hadoop services #

First stop any running Hadoop services:

stop-all.sh

Then start the HDFS and YARN services:

start-dfs.sh
start-yarn.sh

Step 8: Verify the fix #

Check if all required processes are running with the jps command:

jps

You should see NameNode, DataNode, ResourceManager, and NodeManager listed (along with other Hadoop-related processes).

Finally, test your original command again:

hadoop fs -mkdir /WordCountTutorial

This should run without the UnknownHostException error now.

Extra troubleshooting tip #

If you still run into issues, check your /etc/hosts file to ensure localhost is mapped correctly. Open it with:

sudo nano /etc/hosts

Make sure these lines are present:

127.0.0.1       localhost
::1             localhost

Save and exit, then restart your machine or network services if needed.

内容的提问来源于stack exchange，提问作者user6058265