Bitnami Spark容器连接PySpark遇JAVA_GATEWAY_EXITED错误求助

运行PySpark代码连接Bitnami/Spark容器时，持续出现以下错误：

Error : unable to find or load main class org.apache.spark.deploy.SparkSubmit
Caused by : java.lang.ClassNotFoundException: org.apache.spark.deploy.SparkSubmit
Traceback (most recent call last):
  File "d:\Users\azeve\Desktop\École\MT4\DEVOPS\RENDU_DEVOPS_MT4\src\main.py", line 40, in <module>
    spark = SparkSession.builder.master(f'spark://spark-master:7077').appName("SimpleApp").getOrCreate()
  File "D:\Users\azeve\Desktop\École\MT4\DEVOPS\RENDU_DEVOPS_MT4\.venv\lib\site-packages\pyspark\sql\session.py", line 497, in getOrCreate
    sc = SparkContext.getOrCreate(sparkConf)
  File "D:\Users\azeve\Desktop\École\MT4\DEVOPS\RENDU_DEVOPS_MT4\.venv\lib\site-packages\pyspark\context.py", line 515, in getOrCreate
    SparkContext(conf=conf or SparkConf())
  File "D:\Users\azeve\Desktop\École\MT4\DEVOPS\RENDU_DEVOPS_MT4\.venv\lib\site-packages\pyspark\context.py", line 201, in __init__
    SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
  File "D:\Users\azeve\Desktop\École\MT4\DEVOPS\RENDU_DEVOPS_MT4\.venv\lib\site-packages\pyspark\context.py", line 436, in _ensure_initialized
    SparkContext._gateway = gateway or launch_gateway(conf)
  File "D:\Users\azeve\Desktop\École\MT4\DEVOPS\RENDU_DEVOPS_MT4\.venv\lib\site-packages\pyspark\java_gateway.py", line 107, in launch_gateway
    raise PySparkRuntimeError(
pyspark.errors.exceptions.base.PySparkRuntimeError: [JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number.

已尝试以下方案但未解决问题：

• 更换JDK版本：从JDK8卸载后安装JDK11，已配置JAVA_HOME环境变量，java -version输出如下：

openjdk version "11.0.22" 2024-01-16 OpenJDK Runtime Environment Temurin-11.0.22+7 (build 11.0.22+7) OpenJDK 64-Bit Server VM Temurin-11.0.22+7 (build 11.0.22+7, mixed mode)

• 确保PySpark版本（3.5.0）与Bitnami Spark集群版本一致
• 确认Python版本为3.12.1，Spark Master端口8080（WebUI）、7077（服务端口）正常

运行的PySpark代码如下：

from pyspark.sql import SparkSession
from pyspark.conf import SparkConf

from pyspark.sql.types import IntegerType
from pyspark.sql.types import StructField
from pyspark.sql.types import StructType

if __name__ == "__main__":
    
    conf = SparkConf()
    conf.setAll(
        [
            (
                "spark.master",
                "spark://spark-master:7077"
            ),
            ("spark.driver.host", "local[*]"),
            ("spark.submit.deployMode", "client"),
            ("spark.driver.bindAddress", "0.0.0.0"),
            ("spark.app.name", "HelloWorld"),
        ] 
    )
    
    spark = SparkSession.builder.config(conf=conf).getOrCreate()


    df = spark.createDataFrame([("Hello World",)], ["greeting"])
    df.show()

    spark.stop()

已尝试修改spark.master为以下值：

• spark://127.0.1.1:7077
• spark://localhost:7077
• spark://spark-master:7077
• spark://host.docker.internal:7077

同时尝试修改spark.driver.host为：

• 本机内网IP
• 容器IP
• 127.0.0.1
• localhost

内容的提问来源于stack exchange，提问作者DestCom