Cloudera Hadoop VM启动脚本报错：start-all.sh/start-dfs.sh命令未找到

Hey there, let's work through this frustrating "command not found" error you're hitting when trying to start Hadoop daemons. I've dealt with this exact scenario a few times, so here's a step-by-step breakdown to get your services up:

1. First, Verify You're in the Right Directory #

Cloudera Hadoop's startup scripts are usually tucked away in either the bin or sbin subdirectories of your Hadoop installation path. If you're running these commands from a random directory, your shell won't find them unless the path is added to your system's PATH variable.

• First, locate your Hadoop installation directory. For most Cloudera CDH setups, this is typically /opt/cloudera/parcels/CDH/lib/hadoop or /usr/lib/hadoop.
• Switch to the appropriate script directory. For older versions, start-all.sh might be in bin, while newer ones put it in sbin:

  cd /opt/cloudera/parcels/CDH/lib/hadoop/sbin
  

• Now run the script with the ./ prefix (to tell your shell to look in the current directory):

  ./start-all.sh
  

  Or try ./start-dfs.sh if you just want to launch HDFS first.

2. Configure Hadoop Environment Variables (Permanent Fix) #

To avoid having to navigate to the script directory every time, add Hadoop's paths to your shell's environment variables:

• Open your user-specific bash config file with a text editor:

  nano ~/.bashrc
  

• Add these lines at the end (replace /path/to/hadoop with your actual installation path):

  export HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop
  export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
  

• Save the file (Ctrl+O, then Enter in nano) and exit (Ctrl+X). Then reload the config to apply changes immediately:

  source ~/.bashrc
  

• Now you can run start-all.sh or start-dfs.sh from any directory without the ./ prefix.

3. Check Script Existence & Permissions #

If you still get the error after navigating to the right directory, double-check that the scripts exist and have execute permissions:

• List the contents of the script directory to confirm the files are there:

  ls -l /opt/cloudera/parcels/CDH/lib/hadoop/sbin/
  

• If start-all.sh or start-dfs.sh are missing, your Cloudera Hadoop installation might be incomplete. Re-run your deployment steps to ensure all Hadoop components are installed correctly.
• If the files exist but don't have execute permissions (look for -rw-r--r-- instead of -rwxr-xr-x), fix this with:

  chmod +x /opt/cloudera/parcels/CDH/lib/hadoop/sbin/start-all.sh
  chmod +x /opt/cloudera/parcels/CDH/lib/hadoop/sbin/start-dfs.sh
  

4. Use Cloudera Manager (Recommended for Cloudera Setups) #

Cloudera actually recommends using Cloudera Manager to manage Hadoop services instead of raw shell scripts—it's more reliable and gives you better visibility into service status.

• Open a browser and go to http://your-vm-ip:7180 (default port for Cloudera Manager).
• Log in with your admin credentials, then navigate to the Clusters tab.
• Select your Hadoop cluster, then click the Actions dropdown and choose Start to launch all services (or start individual services like HDFS, YARN separately if needed).
• You can check the logs directly in Cloudera Manager if any service fails to start, which makes troubleshooting easier.

After trying these steps, run jps again to verify if daemons like NameNode, DataNode, ResourceManager, and NodeManager are running. If you still run into issues, check the Hadoop log files in $HADOOP_HOME/logs—they'll have detailed error messages to help you dig deeper.

内容的提问来源于stack exchange，提问作者Lonewolf08