Python读取文本文件遇UnicodeDecodeError：无法解码0x92字节

Hey there! Let's break down exactly what's happening here and how to fix it. That 0x92 byte is the key culprit—it's not a valid UTF-8 character, but it represents something specific in another widely used encoding.

What's the 0x92 Byte? #

Byte 0x92 corresponds to the right single quotation mark (’) in the Windows-1252 (CP1252) encoding. This encoding is super common for files saved on Windows systems, even if someone thinks they saved the file as UTF-8. When you force Python to read it as UTF-8, it throws that error because 0x92 doesn't fit UTF-8's valid byte structure.

Solutions to Fix the Error #

1. Read with Windows-1252 Encoding

Since 0x92 is a valid character in Windows-1252, just switch your encoding parameter to "cp1252" (Python's identifier for this encoding):

txt = Path(text_path).read_text(encoding="cp1252")

This should decode the file correctly and eliminate the UnicodeDecodeError right away.

2. Detect the Exact File Encoding (Recommended)

If you're not 100% certain the file uses Windows-1252, use the chardet library to auto-detect the correct encoding:

• First install the library:

  pip install chardet
  

• Then run this code to detect and decode the file:

  from pathlib import Path
  import chardet
  
  # Read the file as raw byte data
  raw_file_content = Path(text_path).read_bytes()
  # Detect the encoding with confidence score
  detection_result = chardet.detect(raw_file_content)
  detected_encoding = detection_result["encoding"]
  confidence = detection_result["confidence"]
  
  print(f"Detected encoding: {detected_encoding} (confidence: {confidence:.2f})")
  # Decode using the confirmed encoding
  txt = raw_file_content.decode(detected_encoding)
  

This approach removes guesswork and ensures you're using the exact encoding the file was saved with.

3. Fallback: Ignore or Replace Invalid Bytes (Last Resort)

If you don't mind losing or replacing problematic characters (not recommended for text where accuracy matters), you can tell Python to handle errors explicitly:

• Ignore invalid bytes entirely:

  txt = Path(text_path).read_text(encoding="utf-8", errors="ignore")
  

• Replace invalid bytes with a placeholder character (�):

  txt = Path(text_path).read_text(encoding="utf-8", errors="replace")
  

About That 500 Server Log #

The [05/May/2018 03:35:45] "POST /app/ HTTP/1.1" 500 14383 log is just your server reporting an internal crash—this is directly caused by the UnicodeDecodeError breaking your code. Fix the encoding issue, and this 500 error will disappear automatically.

内容的提问来源于stack exchange，提问作者Abdul Rehman