HDP 2.6环境下Apache Spark 1安装后spark-shell命令未找到求助

Hey there! I've run into this exact issue when starting out with Spark on HDP, so let's get this sorted for you. The error pops up because your system doesn't know where to locate the spark-shell executable—we just need to point it in the right direction. Here are a few straightforward solutions:

1. Quick Temporary Fix (Current Session Only) #

If you want to test spark-shell right away without changing any permanent settings, use the full absolute path to the script. First, navigate to the directory where you extracted Spark, then run:

./bin/spark-shell

Or if you know the complete path (e.g., you extracted it to /home/your-username/), you can run it directly:

/home/your-username/spark-1.3.1-bin-hadoop2.6/bin/spark-shell

2. Permanent Fix (Add Spark to System PATH) #

To make spark-shell available every time you log in, we'll add Spark's bin directory to your system's PATH variable:

• Open your user's bash configuration file with a text editor:

  vi ~/.bashrc
  

• Scroll to the bottom of the file and add these two lines (replace /path/to/your/spark with the actual directory where you extracted Spark—like /home/your-username/spark-1.3.1-bin-hadoop2.6):

  export SPARK_HOME=/path/to/your/spark
  export PATH=$PATH:$SPARK_HOME/bin
  

• Save and exit the editor (in vi, press Esc, then type :wq and hit Enter).
• Load the updated configuration into your current session:

  source ~/.bashrc
  

Now try running spark-shell again—it should launch without any issues!

3. Double-Check Spark Extraction #

Just to be thorough, confirm that the spark-shell script actually exists in your Spark directory. Run this command (replace the path with your actual Spark directory):

ls /path/to/your/spark-1.3.1-bin-hadoop2.6/bin/

You should see spark-shell listed here. If it's missing, you might need to re-extract the Spark tarball (the download could have been corrupted).

Bonus: Verify Java Setup #

Since Spark relies on Java, double-check that your Java environment is properly configured. Run:

java -version

If this throws an error even though you installed Java, add the JAVA_HOME variable to your ~/.bashrc too:

export JAVA_HOME=/path/to/your/java-installation
export PATH=$PATH:$JAVA_HOME/bin

Then run source ~/.bashrc again to apply the change.

内容的提问来源于stack exchange，提问作者sreenija