基于Matlab的拳头识别稳定性与性能优化技术问询

我在Matlab环境中尝试实现实时拳头边缘检测。因为无法预知图像的亮度与色彩，我先拍摄无手的背景图，后续将新图像与背景图做减除处理。接着把图像从RGB格式转换为HSV、LAB及灰度格式，将裁剪后的拳头训练图像转换为向量，提取HSV的h分量、LAB的a和b分量以及灰度图像向量，输入到GMM中，得到仅拳头为白色、其余区域为黑色的掩码图像。

示例效果 #

• 带有检测边界框的拳头
• 通过黑白阈值分割的拳头

HSV与LAB转换代码 #

lab = rgb2lab(img);

a = reshape(lab(:,:,2), [], 1);
b = reshape(lab(:,:,3), [], 1);

hsv = rgb2hsv(img);
h = reshape(hsv(:,:,1), [], 1);
gray = reshape(rgb2gray(img), [], 1);

mat1 = [h a b gray];

之后我使用detectBRISKFeatures函数检测裁剪后拳头图像的特征，并将其与后续图像的特征通过matchFeatures进行匹配，以此定位拳头区域。

特征对比效果 #

两个拳头的特征对比

我的方案可以运行，但即使图像不变，得到的定位结果也不稳定。同时算法运行速度较慢，请问如何实现更优的拳头识别，或是我哪里操作有误？

完整实现代码 #

cam = webcam('Microsoft® LifeCam HD-3000');
cam.WhiteBalanceMode = 'auto';

pause(0.1);

cam.WhiteBalanceMode = 'auto';
cam.ExposureMode = 'auto';

for i = 1:40
    ref = double(snapshot(cam))/255;
end

'screen shot - keep your hand in square untill detection start'
pause(5);

area = [150 100 300 200];

for i = 1:50
    img = double(snapshot(cam))/255;
    shape_img = insertShape(img, "rectangle", area, LineWidth = 10); % inserts circle aroud the cap
    figure(2); imshow(shape_img);
end

for i = 1:30
    img = double(snapshot(cam))/255;
    shape_img = insertShape(img, "rectangle", area, LineWidth = 10); % inserts circle aroud the cap
    imshow(shape_img);
    img = imcrop(img-ref, area);
    lab = rgb2lab(img);
    a = reshape(lab(:,:,2), [], 1);
    b = reshape(lab(:,:,3), [], 1);
    hsv = rgb2hsv(img);
    h = reshape(hsv(:,:,1), [], 1);
    gray = reshape(rgb2gray(img), [], 1);
    mat1 = [h a b gray];
    if i == 1
        mat = [mat1];
    else
        mat = [mat; mat1];
    end
end

'GMM analyze'

tic
GMModel = fitgmdist(mat, 2, 'Replicates', 2, 'CovarianceType','diagonal','SharedCovariance',true);

toc

test_img = double(snapshot(cam))/255;
test_img = imcrop(test_img-ref, area);
lab = rgb2lab(test_img);
a = reshape(lab(:,:,2), [], 1);
b = reshape(lab(:,:,3), [], 1);
hsv = rgb2hsv(test_img);
h = reshape(hsv(:,:,1), [], 1);
gray = reshape(rgb2gray(test_img), [], 1);
test_hsv = [h a b gray];
indx = cluster(GMModel, test_hsv);

N = size(hsv,1);
M = size(hsv,2);

mask1 = reshape(indx(1:M*N), N, M);

seg11 = (mask1 - 1);
se = strel('disk', 4);
BinaryMask = seg11;
BinaryMask = imdilate(imerode(seg11, se), se);
BinaryMask = imbilatfilt(BinaryMask);
BinaryMask = bwareafilt(BinaryMask > 0.7, 1);
curr_points = detectBRISKFeatures(BinaryMask); 
[curr_features, curr_valid_points] = extractHOGFeatures(BinaryMask, curr_points);
'keep still untill here!!!!!'
 
% hand recognition 
pos = [200 200];
rect = area;
count = 0;
while(true)
    count = count+1;
    hand = double(snapshot(cam))/255;
    a = rgb2gray(hand);
    if mod(count,2) == 0
        [pos, rect] = mask_hand2(hand , GMModel, pos, curr_features, curr_valid_points, rect, BinaryMask, ref);
        last_hand = rgb2gray(hand);
    end
end


function [pos_area rect] = mask_hand2(test_img, GMModel, last_pos, curr_features, curr_valid_points, rect, ~, ref)
    new_img = test_img;
    test_img = imcrop(test_img-ref, rect);
    lab = rgb2lab(test_img);
    a = reshape(lab(:,:,2), [], 1);
    b = reshape(lab(:,:,3), [], 1);
    hsv = rgb2hsv(test_img);
    h = reshape(hsv(:,:,1), [], 1);
    gray = reshape(rgb2gray(test_img), [], 1);
    test_mat = [h a b gray];
    indx = cluster(GMModel, test_mat);
    N = size(hsv,1);
    M = size(hsv,2);
    mask1 = reshape(indx(1:M*N), N, M);
    seg11 = (mask1 - 1);
    se = strel('disk', 2);
    BinaryMask = imdilate(imerode(seg11, se), se);
    BinaryMask = bwareafilt(BinaryMask > 0.7, 1); % Extract largest blob.
    points = detectBRISKFeatures(BinaryMask); 
    [features, valid_points] = extractHOGFeatures(BinaryMask, points);
    index_pairs = matchFeatures(features, curr_features);
    if isempty(index_pairs)
        rect = [0 0 size(new_img, 2) size(new_img, 1)];
        pos_area = last_pos;
    else
        matchedPtsOriginal = curr_valid_points(index_pairs(:,2));
        matchedPtsTarget = valid_points(index_pairs(:,1));
        try
            if size(find(matchedPtsTarget.Location == last_pos),1) ~= 2 
                pos_area = min(matchedPtsTarget.Location) + [rect(1) rect(2)];
            end
        end
        last_pos = pos_area;

        try
            rect = rectangle_position(pos_area);
        end
    end
end

function [rect] = rectangle_position(point)
    if point(1)-150 < 0
        x = 0;
    else
        x = point(1)-150;
    end
    if point(2)-100 < 0
        y = 0;
    else
        y = point(2)-100;
    end
    if x > 340
        x = 340;
    end
    if y > 280
        y = 280;
    end
    rect = [x y 300 200]; % [x y width height]
end

优化建议 #

1. 解决定位不稳定问题 #

• 特征匹配优化：使用matchFeatures时添加'MatchThreshold', 0.7过滤低置信度匹配对，同时设置'MaxRatio', 0.8启用比率测试，剔除歧义匹配点，减少错误定位。
• 掩码后处理增强：在形态学操作后添加bwareaopen(BinaryMask, 50)去除小噪声点，再用imclose(BinaryMask, strel('disk',3))填充掩码孔洞，避免特征检测到无效区域。
• 位置平滑：保存最近3-5帧的pos_area，取均值作为当前位置，避免单次匹配结果跳变。
• GMM聚类验证：训练后输出GMModel.Mu确认聚类中心，确保拳头区域对应正确的聚类类别，避免掩码反转错误。

2. 提升运行速度 #

• 减少重复计算：训练GMM时预先分配mat的内存（如mat = zeros(30*size(img,1)*size(img,2),4);），避免循环中动态扩容；背景图取40帧的均值，替代单帧背景，减少噪声干扰。
• 特征检测降采样：对掩码图像做imresize(BinaryMask, 0.5)降采样后再检测BRISK特征，减少特征点数量，加快匹配速度。
• 帧率与显示优化：将figure(2)的显示改为imshow(shape_img, 'Parent', ax)复用同一窗口，避免频繁创建新窗口；调整处理间隔为每3帧一次，平衡速度与实时性。
• GMM参数调整：将CovarianceType改为'spherical'，减少聚类计算量，提升cluster函数的运行速度。

3. 代码修复 #

• 补全mask_hand2函数缺失的输入参数，确保函数调用正常；
• 移除无用的a = rgb2gray(hand);语句，减少冗余计算；
• 训练GMM时增加样本多样性（如轻微移动拳头采集样本），提升模型泛化能力。

内容的提问来源于stack exchange，提问作者Yuval