Windows下剪贴板读取打印编码错误，硬编码数据正常

我通过PyAutoGui执行Ctrl+A、Ctrl+C复制亚马逊laptop搜索结果页面的全部文本，尝试用pyperclip.paste()或pd.read_clipboard()获取并处理数据，代码如下：

import pyautogui
import time
import pyperclip
import pandas as pd

keyword = 'laptop'

time.sleep(3)
pyautogui.click(x=750, y=135)
time.sleep(1)
pyautogui.write(keyword)
time.sleep(1)
pyautogui.press('enter')
time.sleep(5)
pyautogui.hotkey('ctrl', 'a')
pyautogui.hotkey('ctrl', 'c')
time.sleep(0.1)

#raw = pyperclip.paste()
raw = pd.read_clipboard()

print(raw)

Pandas读取剪贴板报错 #

使用pd.read_clipboard()时触发解析错误：

Traceback (most recent call last):
  File "c:\Users\smfah\OneDrive\Desktop\tmp\regex.py", line 32, in <module>
    raw = pd.read_clipboard()
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\clipboards.py", line 88, in read_clipboard
    return read_csv(StringIO(text), sep=sep, **kwargs)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\util\_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\util\_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\parsers\readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\parsers\readers.py", line 611, in _read
    return parser.read(nrows)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\parsers\readers.py", line 1778, in read
    ) = self._engine.read(  # type: ignore[attr-defined]
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\parsers\python_parser.py", line 282, in read
    alldata = self._rows_to_cols(content)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\parsers\python_parser.py", line 1045, in _rows_to_cols
    self._alert_malformed(msg, row_num + 1)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\io\parsers\python_parser.py", line 765, in _alert_malformed
    raise ParserError(msg)
pandas.errors.ParserError: 第726行预期4个字段，实际发现7个。错误可能是由于使用多字符分隔符时引号被忽略导致。

Pyperclip读取后打印报错 #

使用pyperclip.paste()获取数据后打印时触发编码错误：

Traceback (most recent call last):
  File "c:\Users\smfah\OneDrive\Desktop\tmp\regex.py", line 45, in <module>
    print(raw)
  File "C:\Users\smfah\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap'编解码器无法编码位置60处的字符'\u200c'：字符未映射到定义值

已知情况 #

• 在Win11的VSCode中硬编码复制的文本内容，可正常处理（比如正则匹配）
• 该问题仅出现在Windows系统，Ubuntu 22.4下无此问题

解决方案 #

1. 解决Pyperclip的Unicode编码问题

Windows控制台默认编码为cp1252，无法识别部分Unicode字符，可通过以下两种方式解决：

• 临时修改控制台输出编码为UTF-8：

import sys
sys.stdout.reconfigure(encoding='utf-8')

修改后直接执行print(raw)即可正常输出。

• 打印时手动转码：

print(raw.encode('utf-8').decode('utf-8'))

2. 解决Pandas读取剪贴板的解析错误

亚马逊页面复制的文本是非结构化内容，pd.read_clipboard()默认按CSV格式解析，必然出现字段不匹配问题。正确流程是先通过pyperclip获取原始文本，再自行处理结构化：

import pyautogui
import time
import pyperclip
import pandas as pd
import sys
import re

# 修复控制台编码问题
sys.stdout.reconfigure(encoding='utf-8')

keyword = 'laptop'

time.sleep(3)
pyautogui.click(x=750, y=135)
time.sleep(1)
pyautogui.write(keyword)
time.sleep(1)
pyautogui.press('enter')
time.sleep(5)
pyautogui.hotkey('ctrl', 'a')
pyautogui.hotkey('ctrl', 'c')
time.sleep(0.1)

# 获取原始文本
raw = pyperclip.paste()

# 示例：用正则提取商品价格
price_pattern = re.compile(r'\$\d+\.\d+')
prices = price_pattern.findall(raw)

# 转为DataFrame进行后续处理
df = pd.DataFrame({'商品价格': prices})
print(df)

内容的提问来源于stack exchange，提问作者Fahim