Intel D455深度摄像头Web双流双线程显示异常问题求助

问题现象 #

• 尝试通过Web流传输Intel D455深度摄像头的深度图与RGB视频，复用PyShine多视频流网页脚本
• 启动两个线程后，9000和9001端口均显示最后启动的线程画面
• 单独启动任一线程时，对应端口可正常显示对应视频

问题根源 #

1. StreamProps类属性共享覆盖：代码直接操作ps.StreamProps类的静态方法（如set_Page、set_Capture），而非创建独立实例。两个线程修改的是同一个类的属性，最后启动的线程的配置会覆盖之前的，导致两个Streamer复用同一个Capture对象，最终显示相同画面。
2. ImgCapture的isOpened方法错误：原代码中self.rs未定义，会导致运行报错。
3. 全局pipeline的帧同步问题：两个线程各自创建ImgCapture实例并调用pipeline.wait_for_frames()，会获取不同的帧序列，导致depth和color帧不同步（非当前显示重复画面的直接原因，但属于优化点）。

修改后的代码 #

import cv2
import pyshine as ps
import pyrealsense2 as rs
import numpy as np
import threading
from threading import Lock

HTML="""
<html>
<head>
<title>PyShine Live Streaming</title>
</head>

<body>
<center><h1> PyShine Live Streaming Multiple videos </h1></center>
<center><img src="10.112.33.161:9000/stream.mjpg" width='360' height='240' autoplay playsinline></center>
<center><img src="10.112.33.161:9001/stream.mjpg" width='360' height='240' autoplay playsinline></center>
</body>
</html>
"""

# 全局pipeline与锁，保证帧获取的线程安全
pipeline = rs.pipeline()
config = rs.config()
frame_lock = Lock()

# 配置摄像头流
pipeline_wrapper = rs.pipeline_wrapper(pipeline)
pipeline_profile = config.resolve(pipeline_wrapper)
device = pipeline_profile.get_device()
device_product_line = str(device.get_info(rs.camera_info.product_line))

found_rgb = False
for s in device.sensors:
    if s.get_info(rs.camera_info.name) == 'RGB Camera':
        found_rgb = True
        break
if not found_rgb:
    print("该演示需要带彩色传感器的深度摄像头")
    exit(0)

config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)

if device_product_line == 'L500':
    config.enable_stream(rs.stream.color, 960, 540, rs.format.bgr8, 30)
else:
    config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

class ImgCapture():
    def __init__(self):
        self.last_color = None
        self.last_depth = None

    def _update_frames(self):
        # 加锁保证同一时间只有一个线程获取帧
        with frame_lock:
            frames = pipeline.wait_for_frames()
            depth_frame = frames.get_depth_frame()
            color_frame = frames.get_color_frame()

            if not depth_frame or not color_frame:
                return False

            # 转换为numpy数组并处理
            depth_image = np.asanyarray(depth_frame.get_data())
            color_image = np.asanyarray(color_frame.get_data())
            depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_BONE)

            # 统一分辨率
            depth_dim = depth_colormap.shape
            color_dim = color_image.shape
            if depth_dim != color_dim:
                color_image = cv2.resize(color_image, dsize=(depth_dim[1], depth_dim[0]), interpolation=cv2.INTER_AREA)

            self.last_color = color_image
            self.last_depth = depth_colormap
            return True

    def read_color(self):
        if self._update_frames():
            return True, self.last_color
        return False, None

    def read_depth(self):
        if self._update_frames():
            return True, self.last_depth
        return False, None

    def isOpened(self):
        # 检查摄像头是否正常
        try:
            with frame_lock:
                frames = pipeline.wait_for_frames(timeout_ms=100)
                return frames.get_depth_frame() is not None and frames.get_color_frame() is not None
        except:
            return False

class ImgDepth():
    def __init__(self, cap):
        self.capture = cap

    def read(self):
        return self.capture.read_depth()

    def isOpened(self):
        return self.capture.isOpened()

class ImgColor():
    def __init__(self, cap):
        self.capture = cap

    def read(self):
        return self.capture.read_color()

    def isOpened(self):
        return self.capture.isOpened()

def color_stream():
    # 创建独立的StreamProps实例，而非操作类
    stream_props = ps.StreamProps()
    stream_props.set_Page(HTML)
    address = ('10.112.33.161', 9001)
    try:
        stream_props.set_Mode('cv2')
        capture = ImgCapture()
        color_capture = ImgColor(capture)
        stream_props.set_Capture(color_capture)
        stream_props.set_Quality(90)
        server = ps.Streamer(address, stream_props)
        print('Color服务器启动于:', 'http://'+address[0]+':'+str(address[1]))
        server.serve_forever()
    except KeyboardInterrupt:
        pipeline.stop()
        server.socket.close()

def depth_stream():
    # 创建独立的StreamProps实例
    stream_props = ps.StreamProps()
    stream_props.set_Page(HTML)
    address = ('10.112.33.161', 9000)
    try:
        stream_props.set_Mode('cv2')
        capture = ImgCapture()
        depth_capture = ImgDepth(capture)
        stream_props.set_Capture(depth_capture)
        stream_props.set_Quality(90)
        server = ps.Streamer(address, stream_props)
        print('Depth服务器启动于:', 'http://'+address[0]+':'+str(address[1]))
        server.serve_forever()
    except KeyboardInterrupt:
        pipeline.stop()
        server.socket.close()

if __name__=='__main__':
    pipeline.start(config)
    t1 = threading.Thread(target=depth_stream)
    t2 = threading.Thread(target=color_stream)
    t1.start()
    t2.start()
    t1.join()
    t2.join()

关键修改点 #

1. StreamProps实例化：每个线程创建独立的ps.StreamProps实例，避免类属性被共享覆盖，确保每个Streamer使用自己的Capture配置。
2. 帧获取线程安全：添加threading.Lock保证同一时间只有一个线程调用pipeline.wait_for_frames()，避免帧被抢占，同时保证depth和color帧同步。
3. 优化ImgCapture设计：将color和depth的帧获取逻辑合并，通过read_color和read_depth返回同一组帧的对应数据，确保画面同步。
4. 修复isOpened方法：正确检查摄像头帧是否正常获取，避免未定义变量报错。
5. 更新HTML页面：添加两个img标签，分别对应9000和9001端口的流。

内容的提问来源于stack exchange，提问作者5zigen