Android将ImageProxy转byteArray经Socket发至Python服务器后图片无效求助

1. 修复ImageProxy转NV21的核心逻辑 #

CameraX返回的YUV_420_888格式中，每个平面的rowStride（行步长）可能大于实际图像宽度，直接用buffer.remaining()拷贝会包含填充的无效数据，导致YUV转JPEG失败。修改转换方法：

private byte[] imageProxyToByteArray(ImageProxy image) {
    ImageProxy.PlaneProxy[] planes = image.getPlanes();
    ByteBuffer yBuffer = planes[0].getBuffer();
    ByteBuffer uBuffer = planes[1].getBuffer();
    ByteBuffer vBuffer = planes[2].getBuffer();

    int width = image.getWidth();
    int height = image.getHeight();

    // 提取行步长和像素步长，忽略填充数据
    int yRowStride = planes[0].getRowStride();
    int uvRowStride = planes[1].getRowStride();
    int uvPixelStride = planes[1].getPixelStride();

    byte[] nv21 = new byte[width * height * 3 / 2];

    // 拷贝Y平面有效像素数据
    int yIndex = 0;
    for (int i = 0; i < height; i++) {
        yBuffer.position(i * yRowStride);
        yBuffer.get(nv21, yIndex, width);
        yIndex += width;
    }

    // 拷贝UV平面数据，按NV21格式（VU交替）排列
    int uvIndex = width * height;
    for (int i = 0; i < height / 2; i++) {
        uBuffer.position(i * uvRowStride);
        vBuffer.position(i * uvRowStride);
        for (int j = 0; j < width / 2; j++) {
            nv21[uvIndex++] = vBuffer.get(j * uvPixelStride);
            nv21[uvIndex++] = uBuffer.get(j * uvPixelStride);
        }
    }

    YuvImage yuvImage = new YuvImage(nv21, ImageFormat.NV21, width, height, null);
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    yuvImage.compressToJpeg(new Rect(0, 0, width, height), 100, out);
    return out.toByteArray();
}

2. 替换Socket分隔符方案 #

原方案用<start>/<end>字符串作为分隔符，存在与JPEG数据冲突的风险，改用长度前缀方式确保数据完整性：

Android端发送逻辑修改 #

if (imageBytes != null && imageBytes.length > 0) {
    try (Socket socket = new Socket("192.168.143.234", 65432)) {
        OutputStream outputStream = socket.getOutputStream();
        // 先发送4字节的图片长度（大端序）
        byte[] lengthBytes = ByteBuffer.allocate(4).putInt(imageBytes.length).array();
        outputStream.write(lengthBytes);
        // 发送图片数据
        outputStream.write(imageBytes);
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Python端接收逻辑修改 #

import socket
import numpy as np
import cv2

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('0.0.0.0', 65432))
server_socket.listen(1)

while True:
    client_socket, addr = server_socket.accept()
    # 读取4字节的长度信息
    length_data = client_socket.recv(4)
    if not length_data:
        break
    img_length = int.from_bytes(length_data, byteorder='big')
    # 分块读取完整图片数据
    img_array = b''
    while len(img_array) < img_length:
        chunk = client_socket.recv(min(img_length - len(img_array), 4096))
        if not chunk:
            break
        img_array += chunk
    # 解码并保存图片
    image = cv2.imdecode(np.frombuffer(img_array, dtype=np.uint8), cv2.IMREAD_COLOR)
    if image is not None:
        cv2.imwrite('received_image.jpg', image)
    client_socket.close()

3. 可选优化：保持Socket长连接 #

每次创建新Socket会增加开销，建议全局持有Socket并维护连接：

private Socket socket = null;

// 初始化连接（如在Activity onCreate中）
try {
    socket = new Socket("192.168.143.234", 65432);
} catch (IOException e) {
    e.printStackTrace();
}

// 分析器中发送数据
imageAnalysis.setAnalyzer(executor, image -> {
    byte[] imageBytes = imageProxyToByteArray(image);
    if (imageBytes != null && imageBytes.length > 0 && socket != null && socket.isConnected()) {
        try {
            OutputStream outputStream = socket.getOutputStream();
            byte[] lengthBytes = ByteBuffer.allocate(4).putInt(imageBytes.length).array();
            outputStream.write(lengthBytes);
            outputStream.write(imageBytes);
            outputStream.flush();
        } catch (Exception e) {
            e.printStackTrace();
            // 连接断开时尝试重连
            try {
                socket.close();
                socket = new Socket("192.168.143.234", 65432);
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }
    }
    try {
        Thread.sleep(1000);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
    image.close();
});

// Activity销毁时关闭连接
@Override
protected void onDestroy() {
    super.onDestroy();
    if (socket != null) {
        try {
            socket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

额外检查点 #

• 若开启setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)，可简化转换逻辑：直接将ByteBuffer转Bitmap再压缩为JPEG
• 确认Python端OpenCV版本支持JPEG解码，接收数据无截断

内容的提问来源于stack exchange，提问作者Erica