Windows10环境Hadoop2.8.0 NodeManager启动失败求助

Hey there, let's break down why your Yarn NodeManager is failing to start on Windows 10. Looking at your error logs, the critical issue is that NullPointerException happening in FileSystem.checkPath()—this is a dead giveaway that there's a problem with how you've configured NodeManager's local storage directories.

Root Cause #

The error trace shows the failure happens when NodeManager tries to create or access local directories specified in your Yarn configuration. A null path is being passed to the file system checker, which triggers the exception. This almost always stems from invalid or missing values for the local storage/log directory properties in yarn-site.xml.

Step-by-Step Fixes #

• Validate your yarn-site.xml configuration
  Check these two key properties—they're the most likely culprits:

  • yarn.nodemanager.local-dirs: Stores temporary local files for containers.
  • yarn.nodemanager.log-dirs: Stores container logs.

  Ensure neither property is empty, and the paths are formatted correctly for Windows. Avoid spaces in paths if possible (they can cause issues with Hadoop's file system handling), and use backslashes (or escaped forward slashes) for directory separators.

• Example Correct Configuration
  Replace the values with paths that exist on your system (create the directories first if they don't):

  <property>
      <name>yarn.nodemanager.local-dirs</name>
      <value>D:\hadoop-2.8.0\yarn\local</value>
  </property>
  <property>
      <name>yarn.nodemanager.log-dirs</name>
      <value>D:\hadoop-2.8.0\yarn\logs</value>
  </property>
  

• Verify Directory Permissions & Existence
  Manually create the directories you specified in the config if they don't exist. On Windows, make sure the user running NodeManager (user1 in your logs) has read/write permissions for these folders.

• Check Winutils.exe (Windows-Specific)
  Hadoop on Windows relies on winutils.exe to handle local file system operations. Ensure you have the correct version of winutils.exe matching your Hadoop 2.8.0 release placed in your HADOOP_HOME\bin directory. Missing or mismatched winutils can cause unexpected path-related errors.

• Restart NodeManager
  After fixing the configuration and verifying directories, stop any running NodeManager processes (use Task Manager if needed) and restart the service.

内容的提问来源于stack exchange，提问作者jaseer v