基于OpenCV、YOLOv3-tiny与React JS的树莓派实时目标识别视频流网页播放方案咨询

Hey there! Let's tackle this problem step by step—since you already have a working YOLOv3-tiny + OpenCV pipeline on Python and a React app for controlling your Raspberry Pi, the key is to offload all the heavy detection work to the Python backend and stream the processed frames to your React frontend efficiently. Below are three practical, tested approaches tailored to your setup:

1. MJPEG Stream via Flask/FastAPI (Simplest Approach) #

MJPEG is a lightweight HTTP-based streaming protocol that works seamlessly with browsers. Your Python backend will process frames with YOLO, encode them as JPEGs, and serve them as a continuous stream. The React app just needs an <img> tag to consume the stream.

Backend (Python + Flask) #

First, install dependencies:

pip install flask opencv-python ultralytics  # Use ultralytics for easy YOLOv3-tiny integration

Here's the Flask code to serve the stream:

from flask import Flask, Response
import cv2
from ultralytics import YOLO

app = Flask(__name__)
model = YOLO("yolov3-tiny.pt")  # Load your pre-trained model
cap = cv2.VideoCapture(0)  # Use Pi camera or video source

def generate_frames():
    while True:
        success, frame = cap.read()
        if not success:
            break
        
        # Run YOLO detection
        results = model(frame)
        # Draw bounding boxes on the frame
        annotated_frame = results[0].plot()
        
        # Enframe to JPEG
        ret, buffer = cv2.imencode('.jpg', annotated_frame, [cv2.IMWRITE_JPEG_QUALITY, 80])
        frame_bytes = buffer.tobytes()
        
        # Yield frame in MJPEG format
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame_bytes + b'\r\n')

@app.route('/video_feed')
def video_feed():
    return Response(generate_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=False)  # Bind to all interfaces for network access

Frontend (React) #

Add an <img> component to your React app pointing to the stream URL (replace raspberrypi-ip with your Pi's local IP):

import React from 'react';

const VideoStream = () => {
  return (
    <div className="video-container">
      <img 
        src="http://raspberrypi-ip:5000/video_feed" 
        alt="YOLO Detection Stream"
        style={{ maxWidth: '100%', height: 'auto' }}
      />
    </div>
  );
};

export default VideoStream;

Pros: Super easy to set up, minimal overhead, works on all browsers.
Cons: Slightly higher latency than WebSocket/RTSP, not ideal for interactive low-latency use cases.

2. WebSocket Stream (Low-Latency + Reusable Control Connection) #

If your React app already uses WebSockets for controlling the Pi, you can reuse the same connection to stream frames. This reduces latency and avoids opening extra ports. We'll use Flask-SocketIO for the backend and socket.io-client for React.

Backend (Python + Flask-SocketIO) #

Install dependencies:

pip install flask-socketio opencv-python ultralytics

Code example:

from flask import Flask
from flask_socketio import SocketIO, emit
import cv2
from ultralytics import YOLO
import base64

app = Flask(__name__)
app.config['SECRET_KEY'] = 'your-secret-key'
socketio = SocketIO(app, cors_allowed_origins="*")  # Allow CORS for React

model = YOLO("yolov3-tiny.pt")
cap = cv2.VideoCapture(0)

def stream_frames():
    while True:
        success, frame = cap.read()
        if not success:
            break
        
        results = model(frame)
        annotated_frame = results[0].plot()
        
        # Encode to JPEG and convert to base64
        ret, buffer = cv2.imencode('.jpg', annotated_frame, [cv2.IMWRITE_JPEG_QUALITY, 80])
        base64_frame = base64.b64encode(buffer).decode('utf-8')
        
        # Emit frame to all connected clients
        socketio.emit('video_frame', {'frame': base64_frame})
        socketio.sleep(0.06)  # ~16 FPS, adjust based on Pi performance

@socketio.on('connect')
def handle_connect():
    print('Client connected')

if __name__ == '__main__':
    socketio.start_background_task(stream_frames)
    socketio.run(app, host='0.0.0.0', port=5000, debug=False)

Frontend (React) #

Install socket.io-client:

npm install socket.io-client

React component to receive and display frames:

import React, { useState, useEffect } from 'react';
import io from 'socket.io-client';

const VideoStream = () => {
  const [frameSrc, setFrameSrc] = useState('');
  const socket = io('http://raspberrypi-ip:5000');

  useEffect(() => {
    socket.on('video_frame', (data) => {
      setFrameSrc(`data:image/jpeg;base64,${data.frame}`);
    });

    return () => {
      socket.disconnect();
    };
  }, [socket]);

  return (
    <div className="video-container">
      <img 
        src={frameSrc} 
        alt="YOLO Detection Stream"
        style={{ maxWidth: '100%', height: 'auto' }}
      />
    </div>
  );
};

export default VideoStream;

Pros: Low latency, can reuse existing WebSocket connection for Pi controls, bidirectional communication support.
Cons: Slightly more setup, requires managing WebSocket connections.

3. RTSP/HTTP-FLV Stream (High Stability for Longer Streams) #

For more robust streaming (e.g., if you need to stream to multiple clients or want lower latency), use FFmpeg to convert OpenCV frames into an RTSP or HTTP-FLV stream. You'll need to install FFmpeg on your Raspberry Pi first (sudo apt install ffmpeg).

Backend (Python + FFmpeg Pipe) #

This approach pipes processed frames from OpenCV to FFmpeg, which serves the stream:

import cv2
from ultralytics import YOLO
import subprocess

model = YOLO("yolov3-tiny.pt")
cap = cv2.VideoCapture(0)

# FFmpeg command to stream as HTTP-FLV (browser-compatible)
ffmpeg_cmd = [
    'ffmpeg',
    '-y',
    '-f', 'rawvideo',
    '-vcodec', 'rawvideo',
    '-pix_fmt', 'bgr24',
    '-s', '640x480',  # Match your frame resolution
    '-r', '15',  # Frame rate
    '-i', '-',
    '-c:v', 'libx264',
    '-preset', 'ultrafast',
    '-f', 'flv',
    'http://0.0.0.0:8080/stream'
]

process = subprocess.Popen(ffmpeg_cmd, stdin=subprocess.PIPE)

while True:
    success, frame = cap.read()
    if not success:
        break
    
    results = model(frame)
    annotated_frame = results[0].plot()
    
    # Write frame to FFmpeg pipe
    process.stdin.write(annotated_frame.tobytes())

Frontend (React) #

Use react-player to play the HTTP-FLV stream:

npm install react-player

Component example:

import React from 'react';
import ReactPlayer from 'react-player';

const VideoStream = () => {
  return (
    <div className="video-container">
      <ReactPlayer
        url="http://raspberrypi-ip:8080/stream"
        playing={true}
        width="100%"
        height="auto"
        controls={false}
      />
    </div>
  );
};

export default VideoStream;

Pros: High stability, supports multiple clients, low latency.
Cons: Requires FFmpeg installation, more resource-intensive on the Pi (but yolov3-tiny + ultrafast preset should work fine).

Key Optimization Tips for Raspberry Pi #

• Use a lower resolution (e.g., 640x480) for both capture and YOLO inference to reduce CPU/GPU load.
• Set cv2.IMWRITE_JPEG_QUALITY to 70-80 to balance image quality and stream bandwidth.
• Use the ultralytics library for YOLO—it's optimized for edge devices and simplifies model loading/inference.
• If your Pi has a camera module, use picamera2 instead of cv2.VideoCapture for better performance.

内容的提问来源于stack exchange，提问作者Sudharshan R