如何为Python多线程UDP视频流添加帧号、时间戳等自定义元数据

Hey there! Great job building that UDP video streaming app with client-server architecture and multi-threading—you’ve already got a strong base to work with. Adding metadata like frame numbers, timestamps, and video details is straightforward, and I’ll break down exactly how to implement each piece below.

Step 1: Update the Sender to Include Metadata #

First, we’ll modify the sender to capture and package all the metadata you want. We’ll use Python’s time module for timestamps, and pull video details directly from the cv2.VideoCapture object.

Here’s the updated sender code with metadata included:

import cv2
import socket
import math
import pickle
import sys
import time

max_length = 65000
host = '127.0.0.1'
port = 6000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

# Video setup
video_path = "input2.mp4"
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
    print("Error opening video file")
    sys.exit()

# Get basic video metadata
video_name = video_path.split("/")[-1]  # Extract filename from path
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
fps = cap.get(cv2.CAP_PROP_FPS)
total_duration = total_frames / fps if fps > 0 else 0  # Total video duration in seconds
start_time = time.time()  # Record when streaming starts (absolute timestamp)
frame_num = 0  # Initialize frame counter

print(f"Streaming video: {video_name}")
print(f"Total duration: {total_duration:.2f}s | FPS: {fps:.2f} | Total frames: {total_frames}")

ret, frame = cap.read()
while ret:
    frame_num += 1
    current_timestamp = time.time() - start_time  # Relative timestamp since stream started
    
    # Compress frame
    retval, buffer = cv2.imencode(".jpg", frame)
    if retval:
        buffer = buffer.tobytes()
        buffer_size = len(buffer)
        num_of_packs = math.ceil(buffer_size/max_length) if buffer_size > max_length else 1

        # Package all metadata with frame info
        frame_info = {
            "packs": num_of_packs,
            "frame_num": frame_num,
            "timestamp": current_timestamp,
            "video_name": video_name,
            "total_duration": total_duration,
            "start_time": start_time
        }

        # Send metadata first
        sock.sendto(pickle.dumps(frame_info), (host, port))
        
        # Send frame chunks
        left = 0
        right = max_length
        for i in range(num_of_packs):
            data = buffer[left:right]
            left = right
            right += max_length
            sock.sendto(data, (host, port))
    
    ret, frame = cap.read()

print("Streaming complete")
cap.release()
sock.close()

Step 2: Update the Receiver to Parse and Use Metadata #

Next, we’ll adjust the receiver to unpack the metadata and use it—whether that’s printing details, displaying them in the window title, or storing them for later use.

Here’s the updated receiver code:

import cv2
import socket
import pickle
import numpy as np
from threading import Thread

def display(threadNum, host, port, videoPlayerInfo):
    max_length = 65540
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.bind((host, port))
    frame_info = None
    buffer = None
    frame = None

    print(f"-> Waiting for stream on thread {threadNum}")
    while True:
        data, address = sock.recvfrom(max_length)
        
        # Check if data is metadata (small payload)
        if len(data) < 1000:  # Adjust threshold if needed
            frame_info = pickle.loads(data)
            if not frame_info:
                continue
            
            # Print metadata for debugging (optional)
            print(f"\nThread {threadNum} - Frame {frame_info['frame_num']}")
            print(f"Timestamp: {frame_info['timestamp']:.2f}s | Video: {frame_info['video_name']}")
            print(f"Total video duration: {frame_info['total_duration']:.2f}s")

            # Receive all frame chunks
            nums_of_packs = frame_info["packs"]
            buffer = b""
            for i in range(nums_of_packs):
                chunk, _ = sock.recvfrom(max_length)
                buffer += chunk
            
            # Reconstruct and display frame
            frame = np.frombuffer(buffer, dtype=np.uint8)
            frame = cv2.imdecode(frame, cv2.IMREAD_COLOR)
            frame = cv2.flip(frame, 1)
            
            if frame is not None:
                # Update window title with metadata
                window_title = f"{videoPlayerInfo} | {frame_info['video_name']} | Frame {frame_info['frame_num']}"
                cv2.imshow(window_title, frame)
                
                # Exit on ESC key
                if cv2.waitKey(1) == 27:
                    break
    
    print(f"Goodbye from thread {threadNum}")
    cv2.destroyAllWindows()
    sock.close()

def main():
    host = "0.0.0.0"
    ports = [7000, 7001, 7002]
    threads = []
    
    for i in range(3):
        thread = Thread(
            name=f'Thread-{i+1}',
            target=display,
            args=(f'Thread-{i+1}', host, ports[i], f'Player {i+1}')
        )
        thread.daemon = True
        threads.append(thread)
        thread.start()
    
    for thread in threads:
        thread.join()

if __name__ == "__main__":
    main()

Step 3: Middle Server Remains Unchanged #

Your middle server doesn’t need any modifications—it just forwards the raw data (metadata + frame chunks) between sender and receiver, so it can stay exactly as you wrote it.

Key Notes: #

• Metadata Packaging: We’re using pickle to serialize the metadata dictionary, which works well for simple Python objects. If you need cross-language compatibility later, you could switch to JSON (but you’d have to convert numeric types properly).
• Timestamp Options: We used a relative timestamp (time since stream start) for simplicity, but you could also use the absolute start_time to calculate wall-clock time if needed.
• Error Handling: I added a quick check to make sure the video file opens correctly—you might want to expand this with more error handling for production use.
• Threshold for Metadata: The receiver checks if the payload is small (<1000 bytes) to distinguish metadata from frame chunks. This works because metadata is tiny compared to compressed frame data, but you could add a unique identifier to the metadata dictionary if you want to be 100% sure.

That’s it! Now your UDP stream includes all the metadata you wanted, and the receiver can access and display it however you like.

内容的提问来源于stack exchange，提问作者Keshav Gambhir