基于mss的低分辨率全屏录制及Emoji像素映射异常解决

需求背景 #

需要编写Python脚本，在16:9显示器上尽可能少裁剪的前提下，以70×34分辨率录制全屏。最初使用mss库结合cv2测试时，因grab函数需指定bounding box，无法直接获取低分辨率全屏画面；后续改为截图后resize至目标分辨率，但在将像素转为对应Emoji并通过WebSocket发送时，出现Emoji数量（4725）远大于目标像素数（70×34=2730）的问题。

初始测试代码 #

import cv2
from mss import mss
from PIL import Image

bounding_box = {'top': 0, 'left': 0, 'width': 500, 'height': 500}

sct = mss()

while True:
    sct_img = sct.grab(bounding_box)
    cv2.imshow('screen', np.array(sct_img))

    if (cv2.waitKey(1) & 0xFF) == ord('q'):
        cv2.destroyAllWindows()
        break

优化后代码（存在Emoji数量异常） #

import numpy as np
import cv2
from mss import mss
from PIL import Image
from math import sqrt
import asyncio
import websockets as ws

# settings:
imgWidth = 70
imgHeight = 34

# vars:
emojis = [['😃', (255, 204, 77)], ['😡', (218, 47, 71)], ['🙊', (191, 105, 82)], ['✅', (119, 178, 85)], ['❤', (187, 26, 52)], ['💙', (93, 173, 236)], ['💚', (120, 177, 89)], ['💛', (253, 203, 88)], ['💜', (170, 142, 214)], ['⚫', (0, 0, 0)], ['🌕', (204, 214, 221)], ['🐷', (244, 171, 186)], ['📀', (255, 217, 131)], ['🍊', (244, 144, 12)], ['🍪', (217, 158, 130)], ['🍑', (255, 136, 108)], ['🍐', (166, 211, 136)], ['🐙', (146, 102, 204)], ['👕', (59, 136, 195)], ['👛', (234, 89, 110)], ['💭', (189, 221, 244)], ['🍓', (204, 62, 83)], ['💰', (253, 216, 136)], ['🌳', (92, 145, 59)], ['🎀', (221, 46, 68)], ['🐘', (153, 170, 181)], ['💼', (154, 78, 28)], ['🌚', (102, 117, 127)], ['⚪', (255, 255, 255)]]
bounding_box = {'top': 0, 'left': 0, 'width': 1920, 'height': 1080}

# functions:
def closest_color(rgb): #// gets the most fitting emoji to use for the pixel to represent
    r, g, b = rgb
    color_diffs = []
    for z in emojis:
        color = z[1]
        cr, cg, cb = color
        color_diff = sqrt((r - cr)**2 + (g - cg)**2 + (b - cb)**2)
        color_diffs.append((color_diff, z[0]))
    return min(color_diffs)[1]

def turnascii(image):
    image = image.convert("RGB")
    pixels = image.getdata()
    newPixels = [closest_color(pixel) for pixel in pixels]
    newPixels = ''.join(newPixels)
    newPixels_count = len(newPixels)
    print("pixels: " + str(newPixels_count))
    asciiImage = [newPixels[index:index + imgWidth] for index in range(0, newPixels_count, imgHeight+1)]
    asciiImage = "".join(asciiImage)
    return asciiImage

# init
sct = mss()
newFrame = ""
async def echo(websocket, path):
   async for message in websocket:
        while True:
            img = sct.grab(bounding_box)
            img = cv2.resize(np.array(img),(imgWidth,imgHeight))
            img = Image.fromarray(img)
            newFrame = turnascii(img)
            print("emojis: " + str(len(newFrame)))
            await websocket.send(newFrame)
        
# start server
print("port 1874")
start_server = ws.serve(echo, "localhost", 1874)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

问题根源 #

Emoji数量异常的核心原因是turnascii函数的切片逻辑错误：

asciiImage = [newPixels[index:index + imgWidth] for index in range(0, newPixels_count, imgHeight+1)]

resize后的图像为70列×34行，总像素数2730，newPixels是按行连续排列的字符串。此处错误地使用imgHeight+1作为步长，导致每次切片的起始位置间隔仅35，远小于每行70个字符的长度，最终重复截取大量字符，使总长度远超预期。

修复后的turnascii函数 #

def turnascii(image):
    image = image.convert("RGB")
    pixels = image.getdata()
    newPixels = [closest_color(pixel) for pixel in pixels]
    newPixels = ''.join(newPixels)
    newPixels_count = len(newPixels)
    print("pixels: " + str(newPixels_count))
    # 按每行70个字符分割，步长设为imgWidth（70）
    asciiImage = [newPixels[index:index + imgWidth] for index in range(0, newPixels_count, imgWidth)]
    # 若需要每行换行，用'\n'连接；否则直接拼接成字符串
    asciiImage = "\n".join(asciiImage)
    return asciiImage

额外优化建议 #

1. 最小化裁剪适配16:9比例：目标分辨率70×34的比例为35:17，与16:9不匹配。可先截取适配区域再resize：

   # 计算16:9屏幕适配35:17比例的最小裁剪区域
   target_ratio = imgWidth / imgHeight
   screen_width, screen_height = 1920, 1080
   if screen_width / screen_height > target_ratio:
       # 屏幕更宽，裁剪左右
       new_width = int(screen_height * target_ratio)
       left = (screen_width - new_width) // 2
       bounding_box = {'top': 0, 'left': left, 'width': new_width, 'height': screen_height}
   else:
       # 屏幕更高，裁剪上下
       new_height = int(screen_width / target_ratio)
       top = (screen_height - new_height) // 2
       bounding_box = {'top': top, 'left': 0, 'width': screen_width, 'height': new_height}
   

2. 提升颜色匹配效率：比较颜色平方差即可找到最接近的Emoji，无需开根号，去掉sqrt可加快计算：

   color_diff = (r - cr)**2 + (g - cg)**2 + (b - cb)**2
   

3. 优化WebSocket发送逻辑：原代码在连接建立后无限循环发送，可添加帧率控制避免资源占用过高：

   import time
   # 在发送循环中添加间隔
   await websocket.send(newFrame)
   time.sleep(1/30) # 控制30帧每秒
   

内容的提问来源于stack exchange，提问作者Eli