求助：Systemd无法正常停止Apache Spark Master与Slave节点

你的Systemd服务配置使用了Type=forking，但Spark的start-master.sh/start-slave.sh脚本只是启动实际Spark进程的包装器——脚本执行完成后就会退出，导致Systemd追踪的MAINPID是脚本的PID而非真正的Spark服务进程PID。当执行stop-*.sh脚本停止Spark进程后，Systemd发现它追踪的主PID早已退出，因此会将服务标记为失败状态。

方案一：让Systemd追踪Spark原生PID文件 #

Spark的启动脚本会自动在$SPARK_HOME/logs目录生成PID文件，我们可以让Systemd直接读取这个文件来追踪实际进程，无需手动写入PID：

修改spark-master.service #

[Unit]
Description=Apache Spark Master
Wants=network-online.target
After=network-online.target

[Service]
User=spark
Group=spark
Type=forking
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3/sbin
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
# 指定Spark自动生成的PID文件路径
PIDFile=/opt/cli/spark-3.3.0-bin-hadoop3/logs/spark-spark-master-$(hostname).pid
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-master.sh
ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-master.sh

[Install]
WantedBy=multi-user.target

修改spark-slave.service #

[Unit]
Description=Apache Spark Slave
Wants=network-online.target
After=network-online.target

[Service]
User=spark
Group=spark
Type=forking
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3/sbin
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
# Spark Worker的PID文件路径
PIDFile=/opt/cli/spark-3.3.0-bin-hadoop3/logs/spark-spark-worker-$(hostname).pid
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-slave.sh spark://spark.cdn.chorke.org:7077
ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-slave.sh

[Install]
WantedBy=multi-user.target

注意：如果你的主机名包含特殊字符，或者Spark的PID文件命名规则不同，可以先手动启动一次服务，查看$SPARK_HOME/logs目录下的PID文件名，替换上述路径。

方案二：直接启动Spark主类（推荐） #

跳过Spark的包装脚本，直接用spark-class启动Master/Worker进程，让Systemd直接管理进程生命周期，无需依赖forking模式：

修改spark-master.service #

[Unit]
Description=Apache Spark Master
Wants=network-online.target
After=network-online.target

[Service]
User=spark
Group=spark
Type=simple
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
# 直接启动Spark Master主类
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/bin/spark-class org.apache.spark.deploy.master.Master
# 停止时发送TERM信号，Spark会优雅退出
ExecStop=/bin/kill -TERM $MAINPID
Restart=on-failure

[Install]
WantedBy=multi-user.target

修改spark-slave.service #

[Unit]
Description=Apache Spark Slave
Wants=network-online.target
After=network-online.target

[Service]
User=spark
Group=spark
Type=simple
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
# 直接启动Spark Worker主类，指定Master地址
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/bin/spark-class org.apache.spark.deploy.worker.Worker spark://spark.cdn.chorke.org:7077
ExecStop=/bin/kill -TERM $MAINPID
Restart=on-failure

[Install]
WantedBy=multi-user.target

修改完成后，执行以下命令让配置生效：

# 重新加载Systemd配置
systemctl daemon-reload
# 重启服务测试
systemctl restart spark-master.service
systemctl restart spark-slave.service

现在执行systemctl stop spark-master.service或systemctl stop spark-slave.service，服务会被正常停止，不再出现失败状态。

内容的提问来源于stack exchange，提问作者Śhāhēēd