MapReduce WordCount运行报错：AM容器启动异常寻求解决方案

按照Ubuntu平台的Hadoop安装指南完成部署后，运行MapReduce WordCount示例程序时出现AM Container异常及container-launch异常，程序执行失败。

当前节点进程状态 #

所有Hadoop核心进程均正常运行，进程列表如下：

6544 Jps
3041 NameNode
3842 NodeManager
3219 DataNode
3494 SecondaryNameNode
3706 ResourceManager

YARN应用状态输出 #

执行yarn app -status application_1667981786519_0006得到以下输出：

doop@contactkarim-VirtualBox:~/hadoop-3.3.1$ yarn app -status application_1667981786519_0006
2022-11-09 11:35:22,184 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /127.0.0.1:8032
2022-11-09 11:35:22,522 INFO conf.Configuration: resource-types.xml not found
2022-11-09 11:35:22,522 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
Application Report : 
    Application-Id : application_1667981786519_0006
    Application-Name : word count
    Application-Type : MAPREDUCE
    User : hdoop
    Queue : default
    Application Priority : 0
    Start-Time :  1667982679380
    Finish-Time : 1667982691120
    Progress : 0%
    State : FAILED
    Final-State : FAILED
    Tracking-URL : http://contactkm-VirtualBox:8088/cluster/app/application_1667981786519_0006
    RPC Port : -1
    AM Host : N/A
    Aggregate Resource Allocation : 20250 MB-seconds, 8 vcore-seconds
    Aggregate Resource Preempted : 0 MB-seconds, 0 vcore-seconds
    Log Aggregation Status : DISABLED
    Diagnostics : Application application_1667981786519_0006 failed 2 times due to AM Container for appattempt_1667981786519_0006_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2022-11-09 11:31:31.113]Exception from container-launch.
Container id: container_1667981786519_0006_02_000001
Exit code: 1

[2022-11-09 11:31:31.116]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


[2022-11-09 11:31:31.116]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


For more detailed output, check the application tracking page: http://contactkarim-VirtualBox:8088/cluster/app/application_1667981786519_0006 Then click on links to logs of each attempt.
. Failing the application.
    Unmanaged Application : false
    Application Node Label Expression : <Not set>
    AM container Node Label Expression : <DEFAULT_PARTITION>
    TimeoutType : LIFETIME  ExpiryTime : UNLIMITED  RemainingTime : -1seconds

解决方案 #

结合报错信息（log4j未初始化+容器退出码1）及Java版本怀疑点，按以下步骤处理：

1. 验证并修复Java版本兼容性 #

Hadoop 3.3.1仅兼容Java 8或Java 11，禁止使用Java 17及以上版本：

• 检查当前Java版本：

  java -version
  

• 若版本不符合，安装OpenJDK 8：

  sudo apt update
  sudo apt install openjdk-8-jdk
  

• 配置Hadoop的JAVA_HOME环境变量：
  编辑$HADOOP_HOME/etc/hadoop/hadoop-env.sh，添加或修改：

  export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
  

  注：路径需根据实际安装的Java目录调整，可通过update-alternatives --list java查看

2. 修复log4j配置缺失问题 #

报错显示log4j未初始化，需生成合法的配置文件：

cp $HADOOP_HOME/etc/hadoop/log4j.properties.template $HADOOP_HOME/etc/hadoop/log4j.properties

3. 重启Hadoop服务并测试 #

# 停止所有服务
stop-all.sh
# 启动所有服务
start-all.sh
# 重新运行WordCount程序
hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar wordcount /input /output

4. 额外排查点 #

若问题仍存在，检查：

• HDFS目录权限：确保hdoop用户对/input和/output目录有读写权限
• NodeManager资源配置：确认yarn-site.xml中内存、CPU分配符合机器硬件情况，避免资源不足

内容的提问来源于stack exchange，提问作者DataEater