Matlab转Python逐帧处理视频时结果数值不一致的问题排查

First off, let's tackle the numerical mismatch between your MATLAB and Python code—then we'll simplify the Python implementation for better readability and efficiency.

Why the Numerical Differences? #

Your gut check about BGR vs RGB is on point, but there are two more likely culprits here:

1. Frame Index Misalignment #

MATLAB's read(hVideo, iFrameIter) uses 1-based indexing to pull frames, while OpenCV's cap.read() reads sequentially from the current pointer position (and relies on 0-based indexing under the hood). If you didn't reset the video pointer before your loop, you might be reading entirely wrong frames compared to MATLAB.

Fix: Explicitly jump to the correct frame index in Python to mirror MATLAB's behavior:

cap = cv2.VideoCapture("your_video_file.mp4")
number_of_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))

# Reset to the first frame (0-based for OpenCV)
cap.set(cv2.CAP_PROP_POS_FRAMES, 0)

afSimilarity = np.zeros(number_of_frames - 1)
for iFrameIter in range(number_of_frames - 1):
    # Jump to the exact frame (matches MATLAB's 1-based iFrameIter)
    cap.set(cv2.CAP_PROP_POS_FRAMES, iFrameIter)
    ret, rgbCurrImage = cap.read()
    if not ret:
        break  # Handle case where frame can't be read
    
    # Rest of your processing...

2. Video Decoder Differences #

MATLAB and OpenCV use different underlying decoders (MATLAB relies on system codecs, OpenCV uses FFmpeg by default). This can lead to tiny pixel value discrepancies even for the same frame. To rule out color space confusion, convert BGR to RGB explicitly before calculating grayscale to match MATLAB's input:

# Convert OpenCV's BGR to MATLAB-style RGB
rgbCurrImage_rgb = cv2.cvtColor(rgbCurrImage, cv2.COLOR_BGR2RGB)
rgbSubSampled = np.double(rgbCurrImage_rgb[::2, ::2, :]) / 255
graySubSampled = 0.299 * rgbSubSampled[:, :, 0] + 0.587 * rgbSubSampled[:, :, 1] + 0.114 * rgbSubSampled[:, :, 2]

This makes your grayscale calculation 1:1 with MATLAB's logic. If numbers still don't match, it's almost certainly a decoder difference (normal for most use cases, but you can use a lossless video format like uncompressed AVI if precision is critical).

Cleaner Python Implementation #

You can leverage OpenCV's optimized built-in functions to simplify your code and boost performance:

Option 1: Use OpenCV's Built-in Grayscale Conversion #

OpenCV's COLOR_BGR2GRAY already implements the standard 0.299R + 0.587G + 0.114B formula, so you don't need to calculate it manually:

import cv2
import numpy as np

cap = cv2.VideoCapture("your_video_file.mp4")
number_of_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
afSimilarity = np.zeros(number_of_frames - 1)

prev_gray = None
frame_idx = 0

while cap.isOpened():
    ret, frame = cap.read()
    if not ret or frame_idx >= number_of_frames - 1:
        break
    
    # Downsample (skip every other row/column)
    downsampled_frame = frame[::2, ::2, :]
    # Convert to grayscale and normalize to 0-1 range
    gray_downsampled = cv2.cvtColor(downsampled_frame, cv2.COLOR_BGR2GRAY) / 255.0
    
    # Calculate frame difference once we have a previous frame
    if prev_gray is not None:
        # Example: Using mean absolute difference as similarity metric
        afSimilarity[frame_idx - 1] = np.mean(np.abs(gray_downsampled - prev_gray))
    
    prev_gray = gray_downsampled
    frame_idx += 1

cap.release()

Option 2: Explicit Frame Indexing (For Exact MATLAB Parity) #

If you need to stick to explicit frame indexing like MATLAB, here's a streamlined version:

import cv2
import numpy as np

cap = cv2.VideoCapture("your_video_file.mp4")
number_of_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
afSimilarity = np.zeros(number_of_frames - 1)

for i in range(number_of_frames - 1):
    cap.set(cv2.CAP_PROP_POS_FRAMES, i)
    ret, frame = cap.read()
    if not ret:
        afSimilarity[i] = 0  # Handle missing frame
        continue
    
    # Downsample and convert to normalized grayscale
    gray = cv2.cvtColor(frame[::2, ::2], cv2.COLOR_BGR2GRAY) / 255.0
    # Get next frame for comparison
    cap.set(cv2.CAP_PROP_POS_FRAMES, i + 1)
    ret_next, frame_next = cap.read()
    if ret_next:
        gray_next = cv2.cvtColor(frame_next[::2, ::2], cv2.COLOR_BGR2GRAY) / 255.0
        afSimilarity[i] = np.mean(np.abs(gray - gray_next))

cap.release()

Quick Verification Steps #

1. Check that the first frame's pixel values match between MATLAB and Python (after aligning indices and color spaces)
2. Confirm you're using the exact same video file for both tests
3. If decoder differences are still an issue, try a lossless video format to eliminate codec-related discrepancies

内容的提问来源于stack exchange，提问作者Maya Oxenhandler