Linux下Python3.6中BrokenPipeError: [Errno 32]问题解决方法

First, let's break down what's happening here: The BrokenPipeError: [Errno 32] Broken pipe typically pops up when one end of a pipe (in this case, between your parent script file1.py and child script file2.py) closes before the other finishes writing data through it. Since you're using os.popen, this issue almost always relates to how the parent process handles the child's output—not the CSV read/write operations themselves (those are on regular files, not pipes).

Here are practical, actionable solutions to resolve this:

1. Read All Output From the Child Process in file1 #

When you use os.popen, the child process sends its stdout/stderr through a pipe to the parent. If the parent doesn't read all this output before moving on, the pipe can close early, causing the child to throw a BrokenPipeError when it tries to write more data.

Fix this in file1.py by explicitly reading the entire pipe output:

import os

# Open the pipe and read all output to prevent premature closure
pipe = os.popen("python file2.py")
child_output = pipe.read()  # Ensures all output is consumed before closing
exit_status = pipe.close()  # Optional: Get the child's exit code

2. Redirect Child Output to a File (If You Don't Need It in file1) #

If you don't care about capturing the child's output in file1, redirect it to a log file or discard it entirely to avoid pipe issues:

import os

# Redirect stdout and stderr to a log file
os.popen("python file2.py > file2_output.log 2>&1")

# Or discard output completely (Linux-only)
os.popen("python file2.py > /dev/null 2>&1")

3. Use the subprocess Module (Recommended) #

os.popen is deprecated in Python 3, and the subprocess module gives you far more control over child processes and pipe handling. This is the most robust long-term solution:

Option A: Capture output if you need it #

import subprocess

result = subprocess.run(
    ["python", "file2.py"],
    capture_output=True,
    text=True,
    check=False  # Set to True if you want to raise an error if the child fails
)
# Access output with result.stdout and result.stderr

Option B: Discard output entirely #

import subprocess

# Redirect stdout/stderr to DEVNULL to avoid pipe-related errors
subprocess.run(
    ["python", "file2.py"],
    stdout=subprocess.DEVNULL,
    stderr=subprocess.DEVNULL
)

4. Add Error Handling in file2.py (As a Safety Net) #

If your child script might write to stdout/stderr and you can't guarantee the parent will always read the pipe, add a safety net in file2.py to suppress the messy error message:

import sys
import pandas as pd

def main():
    try:
        # Your CSV operations
        dataset = pd.read_csv('dataset.csv', encoding='ISO-8859-1')
        # ... process your data ...
        dataset.to_csv('output.csv', index=False)
        
        # If you're printing status messages to stdout:
        print("CSV processing completed successfully")
    except BrokenPipeError:
        # Exit cleanly without spamming the error
        sys.stderr.close()
        sys.exit(0)

if __name__ == "__main__":
    main()

Why This Works #

The core issue is almost always the parent process closing the pipe before the child finishes writing to it. By either reading all output, redirecting it away from the pipe, using subprocess (which manages pipes more reliably), or adding error handling, you can eliminate the BrokenPipeError entirely.

内容的提问来源于stack exchange，提问作者Tanushree Tanu