Hadoop 3.0.0-cdh6.3.2运行Jar包报错求助：容器启动失败

环境与执行命令 #

我在安装了Hadoop 3.0.0-cdh6.3.2的机器上执行以下MapReduce任务提交命令：

hadoop jar helloworld.jar solution.CardDriver /user/vmuser/playing_cards_simple /user/vmuser/output

错误信息 #

执行后任务启动失败，报错内容如下：

Failing this attempt.Diagnostics: [2023-01-22 16:24:13.729]Exception from container-launch.
Container id: container_1674418413963_0005_02_000001
Exit code: 1

[2023-01-22 16:24:13.730]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/vmuser/appcache/application_1674418413963_0005/container_1674418413963_0005_02_000001/launch_container.sh: line 19: ${/usr/lib/hadoop-mapreduce}: bad substitution

[2023-01-22 16:24:13.731]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/vmuser/appcache/application_1674418413963_0005/container_1674418413963_0005_02_000001/launch_container.sh: line 19: ${/usr/lib/hadoop-mapreduce}: bad substitution

For more detailed output, check the application tracking page: http://localhost:8088/cluster/app/application_1674418413963_0005 Then click on links to logs of each attempt.

配置情况 #

• 修改过的配置文件：/etc/hadoop/conf/mapred-site.xml、/etc/hadoop/conf/core-site.xml
• 错误配置项：将HADOOP_MAPRED_HOME设置为${/usr/lib/hadoop-mapreduce}，尝试过用$HADOOP_HOME但报错更多
• 未修改yarn-site.xml，其中yarn.application.classpath配置如下：

<description>Classpath for typical applications.</description>
<name>yarn.application.classpath</name>
<value>
   $HADOOP_CONF_DIR,
   $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
   $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
   $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
   $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
</value>
</property>

问题分析与解决方案 #

错误原因 #

报错${/usr/lib/hadoop-mapreduce}: bad substitution是Shell语法错误：${变量名}用于引用环境变量，但这里直接把路径写在${}里，Shell会把/usr/lib/hadoop-mapreduce当成变量名，而变量名不允许包含/字符，导致替换失败。

解决步骤 #

1. 修正配置项：在mapred-site.xml和core-site.xml中，将HADOOP_MAPRED_HOME的值改为直接写路径，去掉多余的${}：

   <property>
       <name>HADOOP_MAPRED_HOME</name>
       <value>/usr/lib/hadoop-mapreduce</value>
   </property>
   

2. 重启Hadoop服务：确保配置生效，重启YARN和MapReduce相关服务
3. 重新提交任务：再次执行任务提交命令，验证问题是否解决

内容的提问来源于stack exchange，提问作者grojas123