基于YOLOv8 Nano的鱼类胚胎游动起止帧精准检测方案问询

背景与问题 #

已训练YOLOv8 Nano模型检测培养皿中单枚游动鱼类胚胎，模型mAP50=0.994性能优异。目标是开发软件，输入视频后提取胚胎游动阶段帧的指标（x/y坐标、游动距离、速度等），但当前硬编码算法因数据噪声（静止时bounding box中心随机偏移，噪声速度达10-20mm/s，而胚胎最小游动速度约10mm/s），无法精准检测游动结束帧，起止帧预测误差远超2-3帧的要求。

已尝试用简单指数平滑降噪：

def simple_exponential_smoothing(data, alpha):
    """
    Apply simple exponential smoothing to the data.

    Parameters:
    data (array-like): The input time series data.
    alpha (float): The smoothing factor (0 < alpha <= 1).

    Returns:
    np.ndarray: The smoothed time series data.
    """
    result = [data[0]]  # First value is same as series
    for n in range(1, len(data)):
        result.append(alpha * data[n] + (1 - alpha) * result[n-1])
    return np.array(result)

初始起止帧检测函数表现不佳：

def find_start_end_rows(df, velocity_column, filtered_velocity_column, frame_rate):
        """
        Find the start and end row indices based on a more refined approach.

        Parameters:
        df (pd.DataFrame): The dataframe to analyze.
        velocity_column (str): The name of the velocity column to search.
        filtered_velocity_column (str): The name of the filtered velocity column.

        Returns:
        tuple: A tuple containing the start row index and the end row index.
        """
        start_row = None
        end_row = None
        velocity_threshold = 20  # Minimum velocity to start swim
        filtered_velocity_threshold = 10  # Minimum filtered velocity to consider movement
        consistent_low_velocity_frames = 5  # Number of consecutive low-velocity frames to detect the end

        # Find the start row
        for i in range(len(df)):
            if df.loc[i, velocity_column] >= velocity_threshold:
                start_row = i - 1
                break

        # If start_row is still None, it means no value >= 20 was found
        if start_row is None:
            return (-1,-1)  # -1 indicates the function failed

        # Find the end row by checking for consistent low velocities after the start row
        low_velocity_count = 0
        for i in range(start_row + 2, len(df)):
            if df.loc[i, filtered_velocity_column] < filtered_velocity_threshold:
                low_velocity_count += 1
                if low_velocity_count >= consistent_low_velocity_frames:
                    end_row = i - consistent_low_velocity_frames
                    break
            else:
                low_velocity_count = 0

        # If end_row is still None, it means no consistent low-velocity frames were found
        if end_row is None:
            end_row = len(df) - 1

        return start_row, end_row

优化方案 #

1. 替换平滑算法：滑动窗口中位数过滤噪声 #

指数平滑对突变数据适应性弱，滑动窗口中位数能有效过滤随机噪声，同时保留真实的速度突变（游动开始/结束）：

from scipy.ndimage import median_filter

def sliding_window_median(data, window_size=5):
    """用滑动窗口中位数平滑速度数据，窗口大小根据帧率调整（如30fps选5-7帧）"""
    return median_filter(data, size=window_size)

2. 状态机式起止帧检测 #

用静止→启动→游动→停止的状态机逻辑，替代简单阈值判断，结合连续帧状态变化确保精度：

def find_swim_start_end(df, velocity_col):
    # 参数可根据实际数据微调
    start_threshold = 12  # 略高于噪声上限（10-20mm/s）
    stop_threshold = 8     # 略低于最小游动速度（10mm/s）
    start_consecutive = 3  # 连续3帧达标才判定启动
    stop_consecutive = 3   # 连续3帧达标才判定停止
    
    state = "idle"
    start_count = 0
    stop_count = 0
    start_row = -1
    end_row = -1
    
    for idx, row in df.iterrows():
        vel = row[velocity_col]
        
        if state == "idle":
            if vel >= start_threshold:
                start_count += 1
                if start_count >= start_consecutive:
                    start_row = idx - start_consecutive + 1  # 取首次触发的帧
                    state = "swimming"
                    start_count = 0
            else:
                start_count = 0
        
        elif state == "swimming":
            if vel < stop_threshold:
                stop_count += 1
                if stop_count >= stop_consecutive:
                    end_row = idx - stop_consecutive  # 取首次回落的帧
                    state = "stopped"
                    break
            else:
                stop_count = 0
    
    # 处理视频结束时胚胎仍在游动的情况
    if state == "swimming" and end_row == -1:
        end_row = len(df) - 1
    
    return start_row, end_row

3. 结合坐标轨迹的二阶导数（加速度）辅助判断 #

除速度外，利用坐标的二阶导数（加速度）检测游动的起始/结束：

• 游动启动时，加速度会出现明显峰值（静止→运动的突变）
• 游动停止时，加速度回落至噪声水平

先对x/y坐标做平滑，再计算加速度，结合速度阈值交叉验证，进一步提升准确率。

4. 自适应阈值调整 #

根据视频前N帧（胚胎静止阶段）的噪声分布，动态计算阈值，适配不同视频的噪声差异：

def get_adaptive_thresholds(df, velocity_col, idle_frames=50):
    """从视频前N帧（静止期）计算自适应阈值"""
    idle_velocities = df[velocity_col][:idle_frames].values
    noise_mean = idle_velocities.mean()
    noise_std = idle_velocities.std()
    
    # 启动阈值：噪声均值+3倍标准差（覆盖99.7%的噪声，避免误判）
    start_threshold = noise_mean + 3 * noise_std
    # 停止阈值：噪声均值+1倍标准差（及时检测真实停止）
    stop_threshold = noise_mean + 1 * noise_std
    
    return start_threshold, stop_threshold

内容的提问来源于stack exchange，提问作者Laconad