Hadoop 3.x NodeManager无法启动问题求助

问题现象 #

• Hadoop版本3.x，执行jps后NameNode、SecondaryNameNode、ResourceManager进程正常，但缺少NodeManager进程
• 核心报错为java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(...) accessible: module java.base does not "opens java.lang" to unnamed module

完整报错日志 #

ERROR org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting NodeManager
java.lang.ExceptionInInitializerError
at com.google.inject.internal.cglib.reflect.$FastClassEmitter.(FastClassEmitter.java:67)
at com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:72)
at com.google.inject.internal.cglib.core.$DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at com.google.inject.internal.cglib.core.$AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at com.google.inject.internal.cglib.reflect.$FastClass$Generator.create(FastClass.java:64)
at com.google.inject.internal.BytecodeGen.newFastClass(BytecodeGen.java:204)
at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.(ProviderMethod.java:256)
at com.google.inject.internal.ProviderMethod.create(ProviderMethod.java:71)
at com.google.inject.internal.ProviderMethodsModule.createProviderMethod(ProviderMethodsModule.java:275)
at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:144)
at com.google.inject.internal.ProviderMethodsModule.configure(ProviderMethodsModule.java:123)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:349)
at com.google.inject.AbstractModule.install(AbstractModule.java:122)
at com.google.inject.servlet.ServletModule.configure(ServletModule.java:52)
at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
at com.google.inject.spi.Elements.getElements(Elements.java:110)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
at com.google.inject.Guice.createInjector(Guice.java:96)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:417)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:465)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:461)
at org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer.serviceStart(WebServer.java:125)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:122)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:963)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1042)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @58ecb515
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
at com.google.inject.internal.cglib.core.$ReflectUtils$2.run(ReflectUtils.java:56)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
at com.google.inject.internal.cglib.core.$ReflectUtils.(ReflectUtils.java:46)
... 32 more

当前配置文件 #

yarn-site.xml #

<configuration>
    <property>
        <name>yarn.resourcemanager.hostsname</name>
        <value>127.0.0.1</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.cpu-vcores</name>
        <value>8</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>8192</value>
    </property>
    <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_HOME,PATH,LANG,TZ,HADOOP_MAPRED_HOME</value>
    </property>
</configuration>

core-site.xml #

< configuration>
<property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
</property>
</configuration>

mapred-site.xml #

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.application.classpath</name>
        <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
    </property>
</configuration>

hdfs-site.xml #

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

问题分析与解决方案 #

问题根源 #

日志明确指向JDK版本冲突：尽管你设置了JAVA_HOME为JDK8，但启动NodeManager时实际调用了JDK11/17。JDK9+引入的模块化系统会限制内部API访问，而Hadoop 3.x依赖的Guice/CGLIB库在非模块化环境下无法兼容该限制，导致启动失败。同时配置文件存在拼写和格式错误，也会影响进程启动。

解决步骤 #

1. 强制切换到JDK8环境

   • 检查当前终端JDK版本：执行java -version和javac -version，确认输出为JDK8
   • 若版本不符，手动配置环境变量（以bash为例）：

     export JAVA_HOME=/path/to/your/jdk8
     export PATH=$JAVA_HOME/bin:$PATH
     

   • 验证配置：echo $JAVA_HOME确保指向JDK8目录
   • 重启Hadoop：先执行stop-all.sh，再执行start-all.sh

2. 修复配置文件错误

   • yarn-site.xml中yarn.resourcemanager.hostsname拼写错误，修正为yarn.resourcemanager.hostname：

     <property>
         <name>yarn.resourcemanager.hostname</name>
         <value>127.0.0.1</value>
     </property>
     

   • core-site.xml中< configuration>存在多余空格，修正为<configuration>

3. 验证启动结果

   • 执行jps确认NodeManager进程存在
   • 查看NodeManager日志（$HADOOP_LOG_DIR/yarn-*-nodemanager-*.log），确保无报错

补充方案 #

若上述步骤无效：

• 移除brew安装的JDK路径环境变量，避免干扰
• 在$HADOOP_HOME/etc/hadoop/hadoop-env.sh中明确指定JDK8路径：

  export JAVA_HOME=/path/to/your/jdk8
  

内容的提问来源于stack exchange，提问作者RussellXiao