如何让阿拉伯语URL获得与Google一致的UTF-8编码？

The core issue here is that Google applies double URL encoding to the URL, while your current implementation uses single encoding—and that's exactly why your results don't match. Let's break this down and fix it step by step:

Difference Analysis #

• Your single encoding takes Arabic characters and converts them directly to UTF-8 percent-encoded format (for example, تبويض becomes %D8%AA%D8%A8%D9%88%D9%8A%D8%B6).
• Google's result shows %25D8%25AA%25D8%25A8%25D9%2588%25D9%258A%25D8%25B6 for the same segment. Notice every % from your single encoding is replaced with %25—this means Google first does a standard UTF-8 encode, then encodes the % characters from that result a second time (since % is a reserved URL character, its encoded form is %25).

Python Implementation for Google-Style Encoding #

To replicate Google's encoding, you need to run two rounds of URL encoding. Here's a clean way to do it while preserving the full URL structure:

from urllib.parse import quote, urlsplit, urlunsplit

# Your original Arabic URL
original_url = "https://www.aljazeera.net/news/healthmedicine/2019/4/29/لحدوث الحمل أو تجنبه هكذا تختصبان أيام التبويض"

# Split the URL into components to avoid encoding the scheme (https://) and domain
scheme, netloc, path, query, fragment = urlsplit(original_url)

# First pass: Encode the Arabic path segment to standard UTF-8 percent format
first_encoded_path = quote(path, encoding='utf-8')

# Second pass: Encode the % characters from the first result to get Google's style
google_encoded_path = quote(first_encoded_path, safe='')

# Reconstruct the full URL with Google-compatible encoding
google_style_url = urlunsplit((scheme, netloc, google_encoded_path, query, fragment))

print(google_style_url)

Running this code will output the exact same encoded URL as Google's result.

Why Double Encoding? #

Double encoding is often used for server compatibility: some servers automatically decode URLs once. By encoding twice, the server's first decode step converts the double-encoded segments back to the single-encoded format, and a second decode (if needed) renders the original Arabic text. Browsers use single encoding because they handle URL decoding internally without relying on server-side pre-processing.

内容的提问来源于stack exchange，提问作者simpatico