如何用Python识别PNG图片中被点击的矩形？

我有一张包含矩形的PNG图片，希望展示该图片，让用户点击其中一个矩形后返回该矩形的左上角和右下角像素坐标。

示例图片：

我尝试用OpenCV的Blob检测功能，将鼠标点击坐标映射到对应Blob，但在OpenCV的安装和使用上遇到困难。请问OpenCV是实现该功能的合适库吗？

以下是我写的Blob检测代码，仅能检测出约8个Blob：

# Standard imports
import cv2
import numpy as np

# Read image
im = cv2.imread("/home/mainmeister/PycharmProjects/WarehousrLineMap/public/warehouse.png", cv2.IMREAD_GRAYSCALE)

# Set up the detector with default parameters.
parameters = cv2.SimpleBlobDetector_Params()
parameters.filterByColor = 1
parameters.blobColor = 255
#detector = cv2.SimpleBlobDetector()
detector = cv2.SimpleBlobDetector_create(parameters)

# Detect blobs.
keypoints = detector.detect(im)

# Draw detected blobs as red circles.
# cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS ensures the size of the circle corresponds to the size of blob
#im_with_keypoints = cv2.drawKeypoints(im, keypoints, np.array([]), (0,0,255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
im_with_keypoints = cv2.drawKeypoints(im, keypoints, np.array([]),(255,0,0),cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)

# Show keypoints
cv2.imshow("Keypoints", im_with_keypoints)
cv2.waitKey(0)

解答 #

OpenCV完全适合实现这个功能，你的问题出在选了不合适的检测方法：SimpleBlobDetector天生更擅长检测圆形或类圆形的 blob，对矩形这类规则多边形的检测效果很差，应该改用**轮廓检测（Contour Detection）**来定位所有矩形，再结合鼠标交互实现点击坐标映射。

改进方案代码 #

import cv2
import numpy as np

# 存储所有检测到的矩形，格式为(左上角x, 左上角y, 右下角x, 右下角y)
detected_rects = []

def on_mouse_click(event, x, y, flags, param):
    if event == cv2.EVENT_LBUTTONDOWN:
        # 遍历所有矩形，判断点击点是否在矩形范围内
        for idx, (x1, y1, x2, y2) in enumerate(detected_rects):
            if x1 <= x <= x2 and y1 <= y <= y2:
                print(f"选中矩形 {idx+1}：左上角({x1}, {y1})，右下角({x2}, {y2})")
                # 用红色框标记点击的矩形
                cv2.rectangle(display_img, (x1, y1), (x2, y2), (0, 0, 255), 2)
                cv2.imshow("仓库平面图", display_img)
                break

# 读取图像
src_img = cv2.imread("/home/mainmeister/PycharmProjects/WarehousrLineMap/public/warehouse.png")
display_img = src_img.copy()
gray_img = cv2.cvtColor(src_img, cv2.COLOR_BGR2GRAY)

# 预处理：二值化+降噪，提升轮廓检测准确性
_, binary_img = cv2.threshold(gray_img, 200, 255, cv2.THRESH_BINARY_INV)
kernel = np.ones((3, 3), np.uint8)
binary_img = cv2.morphologyEx(binary_img, cv2.MORPH_CLOSE, kernel)

# 检测图像中的外部轮廓
contours, _ = cv2.findContours(binary_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# 筛选出矩形轮廓
for cnt in contours:
    # 对轮廓进行多边形近似
    epsilon = 0.02 * cv2.arcLength(cnt, True)
    approx = cv2.approxPolyDP(cnt, epsilon, True)
    # 判断是否为矩形（4个顶点）且面积足够大（过滤小噪声）
    if len(approx) == 4 and cv2.contourArea(cnt) > 100:
        x, y, w, h = cv2.boundingRect(approx)
        rect = (x, y, x + w, y + h)
        detected_rects.append(rect)
        # 用绿色框画出所有检测到的矩形
        cv2.rectangle(display_img, (x, y), (x + w, y + h), (0, 255, 0), 1)

# 创建窗口并绑定鼠标点击事件
cv2.namedWindow("仓库平面图")
cv2.setMouseCallback("仓库平面图", on_mouse_click)

# 显示图像，等待交互
cv2.imshow("仓库平面图", display_img)
cv2.waitKey(0)
cv2.destroyAllWindows()

关键说明 #

1. 轮廓检测更适合矩形：通过cv2.findContours提取轮廓，再用approxPolyDP筛选出4个顶点的形状，精准定位所有矩形
2. 预处理提升准确性：二值化和形态学闭操作可以消除图像中的小噪声，避免误检测
3. 鼠标交互逻辑：绑定cv2.EVENT_LBUTTONDOWN事件，判断点击坐标落在哪个矩形范围内，直接输出对应坐标

内容的提问来源于stack exchange，提问作者Mainmeister