PySpark连接本地MySQL失败，报JavaSparkContext绑定地址错误

Hey there! Let's break down what's going wrong here and get that MySQL connection working for you.

Core Issue Analysis #

The java.net.BindException you're hitting means Spark's driver can't bind to the default network address it's trying to use. The error message already gives a key hint—you need to explicitly set the spark.driver.bindAddress to a valid local address (like 127.0.0.1) so the driver can establish a stable connection.

Additional Fixes for Your Code #

Beyond the bind address, there are a few other small issues in your code that might be blocking the connection:

• MySQL Driver Class: For MySQL Connector/J 8.0+, the correct driver class is com.mysql.cj.jdbc.Driver (the old com.mysql.jdbc.Driver is deprecated and won't work properly with newer connector versions).
• JDBC URL: You're missing the database name in your URL. It should look like jdbc:mysql://127.0.0.1/your_db_name (replace your_db_name with the actual name of your MySQL database where the product table lives). Adding useSSL=false&serverTimezone=UTC will also help avoid common SSL and timezone-related errors.
• Spark Initialization: Using SQLContext directly is outdated. It's better to use SparkSession—the modern, recommended entry point for Spark SQL operations.

Corrected Full Code #

Here's the revised code that addresses all these issues:

from pyspark.sql import SparkSession
import os

# Initialize SparkSession with explicit bind address
spark = SparkSession.builder \
    .appName("MySQLConnection") \
    .config("spark.driver.bindAddress", "127.0.0.1") \
    .getOrCreate()

# Set up JDBC connection parameters
jdbc_url = "jdbc:mysql://127.0.0.1/your_db_name?useSSL=false&serverTimezone=UTC"
connection_props = {
    "driver": "com.mysql.cj.jdbc.Driver",
    "user": "root",
    "password": ""  # Double-check this matches your MySQL root password
}

# Read the MySQL table into a DataFrame
product_df = spark.read.jdbc(url=jdbc_url, table="product", properties=connection_props)

# Test the connection by showing sample data
product_df.show()

Extra Checks to Ensure Success #

• Confirm MySQL is Running: Make sure your local MySQL server is up and running (you can test this by running mysql -u root -p in your terminal).
• Jar File Accessibility: Ensure the MySQL Connector/J jar file is in the path you specified. If you're still having issues, you can pass the jar directly when starting PySpark:

  pyspark --jars ./mysql-connector-java-8.0.11/mysql-connector-java-8.0.11.jar
  

• MySQL Permissions: Verify the root user has access to the target database from localhost. If not, grant permissions with this SQL command:

  GRANT ALL PRIVILEGES ON your_db_name.* TO 'root'@'localhost';
  FLUSH PRIVILEGES;
  

内容的提问来源于stack exchange，提问作者lpt