PySpark与spark-shell报错：无法在/tmp创建本地目录求助

云原生阿岚

2026-5-29

Fix for Spark's "Failed to create local dir in /tmp" Error

Hey there! I’ve dealt with this exact Spark error a few times, so let’s break down what’s going on and how to fix it quickly. The error pops up because Spark can’t write temporary files to the /tmp directory—usually due to permission issues, insufficient space, or the directory being locked down by system settings.

Here are the most reliable solutions to get you back up and running:

Check and repair the /tmp directory
First, confirm if /tmp has enough space and correct permissions:
- Run df -h /tmp to check available disk space. If it’s full, delete unnecessary files (be cautious not to remove active system files!) or extend the partition if needed.
- Run ls -ld /tmp to verify permissions. The output should look like drwxrwxrwt (world-writable with sticky bit enabled). If not, fix it with this command:
```
sudo chmod 1777 /tmp
```
Configure Spark to use a custom temporary directory
If fixing /tmp isn’t feasible, tell Spark to use another directory you have write access to:
- For spark-shell, launch it with this configuration flag:
```
spark-shell --conf spark.local.dir=/path/to/your/custom/temp/folder
```
- For PySpark, use the same flag when starting the session:
```
pyspark --conf spark.local.dir=/path/to/your/custom/temp/folder
```
- To set this permanently, edit your spark-defaults.conf file (typically located in $SPARK_HOME/conf) and add:
```
spark.local.dir /path/to/your/custom/temp/folder
```
- You can also set this directly in your PySpark code:
```
from pyspark.sql import SparkSession

spark = SparkSession.builder \
    .config("spark.local.dir", "/path/to/your/custom/temp/folder") \
    .appName("YourAppName") \
    .getOrCreate()
```
Adjust tmpfs settings (if /tmp is mounted as tmpfs)
Some systems mount /tmp as a temporary filesystem (tmpfs) with a fixed size limit. Check if this applies to your system with:
```
mount | grep /tmp
```
If you see tmpfs in the output, edit /etc/fstab to increase the size (e.g., change size=2G to size=4G), then remount the directory with:
```
sudo mount -o remount /tmp
```