Python与C++基于pipe进程通信无数据接收问题排查求助

• 核心原因：Python的subprocess.call在类Unix系统下默认开启close_fds=True参数，会自动关闭进程继承的除标准输入/输出/错误（0、1、2）之外的所有文件描述符。你传入C++程序的r_sub、w_sub等管道fd在subprocess启动时已经被关闭，因此读写操作都会失败。
• 次要问题：C++代码写入管道时固定写16字节，超出字符串实际长度会携带垃圾数据；写入完成后未主动关闭写端，会导致Python侧读操作一直阻塞等待EOF。

1. 修改Python代码 #

给subprocess.call添加close_fds=False参数，保留继承的管道文件描述符：

#Python
import os
import subprocess
import time
# 建立双向通信管道
r_sub, w_py = os.pipe()
r_py, w_sub = os.pipe()
# 创建子进程运行C++程序
pid = os.fork()

if pid > 0:
# 父进程（Python侧）
    os.close(r_sub)
    os.close(w_sub)

    print("[Py]Parent process is writing : r_py" + str(r_py) + " w_py" + str(w_py) +  "r_sub" + str(r_sub) + " w_sub" + str(w_sub))
    text = b"message"
    # 向C++程序写数据
    os.write(w_py,text)
    print("[Py]Written text:", text.decode())

    os.close(w_py)
    # 读取C++返回的数据
    rec = os.fdopen(r_py)
    print("[Py]Received Message " + rec.read())
else:
    # 子进程
    print("[SubP]Calling c++ .exe : r_sub" + str(r_sub) + " w_sub" + str(w_sub) + " r_py" + str(r_py) + " w_py" + str(w_py))
    # 启动C++可执行文件，关闭自动关闭fd的逻辑
    subprocess.call(["./test.out",str(r_sub),str(w_sub),str(r_py),str(w_py)], close_fds=False)

2. 修改C++代码 #

添加错误打印定位问题、调整写入长度、写入完成后关闭写端：

//c++
#include <iostream>
#include <stdio.h>
#include <unistd.h>
#include <string>
#include <string.h>
#include <errno.h>
#define MSGSIZE 16

int main(int argc, char *argv[]) {
    if(argc < 5){
        printf("[c++]File descriptors are not present");
        return -1;
    }
    printf("[c++]I received file descriptors: r_sub%s w_sub%s r_py%s w_py%s\n",argv[1],argv[2],argv[3],argv[4]);
    // 把字符串参数转成int类型的文件描述符
    int r_sub = std::stoi(argv[1]), w_sub = std::stoi(argv[2]), r_py = std::stoi(argv[3]), w_py = std::stoi(argv[4]);
    char buffer[MSGSIZE] = "";

    close(r_py);
    close(w_py);
    // 读取Python发送的内容
    if(read(r_sub,buffer,MSGSIZE) == -1) {
        printf("\nread failed: %s\n", strerror(errno));
        return -1;
    }

    printf("[c++]Received message: %s\n",buffer);

    // 向Python返回数据
    close(r_sub);
    const char* res = "message back";
    write(w_sub, res, strlen(res)+1);
    close(w_sub); // 写完主动关闭写端，通知Python侧读取结束
    std::cout << "[c++]Finish\n\n";
    return 0;
}

修复后运行结果 #

[Py]Parent process is writing : r_py5 w_py4r_sub3 w_sub6
[Py]Written text: message
[SubP]Calling c++ .exe : r_sub3 w_sub6 r_py5 w_py4
[c++]I received file descriptors: r_sub3 w_sub6 r_py5 w_py4
[c++]Received message: message
[c++]Finish

[Py]Received Message message back

内容的提问来源于stack exchange，提问作者Markos Othonos