Hey there! I've got you covered on extracting desktop icon positions and their coordinates with Python. Let's walk through a few practical, actionable approaches:

This method works great if your desktop icons have a consistent look (like default Windows/macOS icons). You'll use a sample icon as a template to find all matches across your desktop.

Step 1: Install Required Libraries #

First, grab the tools you need:

pip install pyautogui opencv-python numpy

Step 2: Capture Desktop Screenshot & Prepare Template #

• Take a screenshot of one of your desktop icons (save it as icon_template.png, make sure it’s the exact size as other icons).
• Use this code to capture your full desktop and find all matching icons:

import pyautogui
import cv2
import numpy as np

# Capture the entire desktop screenshot
desktop_screenshot = pyautogui.screenshot()
# Convert screenshot to OpenCV's BGR format
screenshot_cv = cv2.cvtColor(np.array(desktop_screenshot), cv2.COLOR_RGB2BGR)

# Load your icon template (grayscale for faster matching)
template = cv2.imread("icon_template.png", 0)
screenshot_gray = cv2.cvtColor(screenshot_cv, cv2.COLOR_BGR2GRAY)

# Run template matching
match_result = cv2.matchTemplate(screenshot_gray, template, cv2.TM_CCOEFF_NORMED)
# Adjust threshold based on how strict you want matches to be (0.8 works for most cases)
threshold = 0.8
match_locations = np.where(match_result >= threshold)

# Extract center coordinates of each detected icon
icon_coords = []
for pt in zip(*match_locations[::-1]):
    # Calculate center (pt is the top-left corner of the matched template)
    center_x = pt[0] + template.shape[1] // 2
    center_y = pt[1] + template.shape[0] // 2
    icon_coords.append((center_x, center_y))
    # Optional: Draw green boxes around detected icons to verify
    cv2.rectangle(screenshot_cv, pt, (pt[0] + template.shape[1], pt[1] + template.shape[0]), (0, 255, 0), 2)

# Save the verification image
cv2.imwrite("detected_icons.png", screenshot_cv)

# Print out the coordinates
print("Detected icon center coordinates:")
for i, coord in enumerate(icon_coords, 1):
    print(f"Icon {i}: X={coord[0]}, Y={coord[1]}")

If your desktop has icons with different designs but are mostly square/rectangular, edge detection can help you find them without a template.

Step 1: Use the Same Libraries as Above #

No extra installs needed—just reuse the libraries from Approach 1.

Step 2: Detect Icons via Contours #

import pyautogui
import cv2
import numpy as np

# Capture desktop screenshot and convert to grayscale
desktop_screenshot = pyautogui.screenshot()
screenshot_cv = cv2.cvtColor(np.array(desktop_screenshot), cv2.COLOR_RGB2BGR)
screenshot_gray = cv2.cvtColor(screenshot_cv, cv2.COLOR_BGR2GRAY)

# Reduce noise and detect edges
blurred = cv2.GaussianBlur(screenshot_gray, (5, 5), 0)
edges = cv2.Canny(blurred, 50, 150)

# Find contours (shapes) in the edge image
contours, _ = cv2.findContours(edges.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

icon_coords = []
for contour in contours:
    # Approximate the contour to a polygon
    perimeter = cv2.arcLength(contour, True)
    approx_polygon = cv2.approxPolyDP(contour, 0.04 * perimeter, True)
    
    # Check if the shape is a square/rectangle and fits typical icon size
    if len(approx_polygon) == 4 and 500 < cv2.contourArea(contour) < 5000:
        x, y, w, h = cv2.boundingRect(approx_polygon)
        center_x = x + w // 2
        center_y = y + h // 2
        icon_coords.append((center_x, center_y))
        # Draw blue boxes around detected icons
        cv2.rectangle(screenshot_cv, (x, y), (x + w, y + h), (255, 0, 0), 2)

# Save verification image
cv2.imwrite("icons_detected_contours.png", screenshot_cv)

# Print coordinates
print("Detected icon center coordinates:")
for i, coord in enumerate(icon_coords, 1):
    print(f"Icon {i}: X={coord[0]}, Y={coord[1]}")

If you don’t want to work with OpenCV, PyAutoGUI has a simplified function to find all instances of an image on screen:

import pyautogui

# Locate all icons matching your template (adjust confidence as needed)
icon_locations = list(pyautogui.locateAllOnScreen("icon_template.png", confidence=0.8))

icon_coords = [pyautogui.center(loc) for loc in icon_locations]

print("Detected icon center coordinates:")
for i, coord in enumerate(icon_coords, 1):
    print(f"Icon {i}: X={coord.x}, Y={coord.y}")

• Threshold Adjustment: If you’re getting too many false positives, increase the threshold value. If icons are missing, lower it slightly.
• Template Size: Ensure your icon_template.png is exactly the same size as the icons on your desktop for accurate matches.
• Cross-Platform Notes: Desktop layouts vary between Windows, macOS, and Linux—you may need to tweak size ranges or thresholds for your specific OS.

内容的提问来源于stack exchange，提问作者Yash Raj