Hey there! Let's fix that emoji formatting issue you're having. The "\ud83d\ude01" you're seeing is a Unicode surrogate pair representing the 😃 emoji (U+1F603). Here's how to convert those escape sequences (or actual emoji characters) to the clean U+XXXXXX notation you want, step by step:

Step 1: Decode Escaped Sequences (If Needed) #

First, if your tweet text is coming with escaped backslashes (like "Hello \\ud83d\\ude01 world!"), you need to decode those to get the actual emoji character. You can do this with Python's unicode_escape decoder:

escaped_text = "Hello \\ud83d\\ude01 world!"
decoded_text = escaped_text.encode('utf-8').decode('unicode_escape')
# decoded_text will now be "Hello 😃 world!"

Step 2: Replace Emojis with U+ Notation #

To replace only emojis (while keeping regular text intact), use the emoji library to detect emojis easily. First install it:

pip install emoji

Then use this function to swap emojis for their U+ notation:

import emoji

def replace_emojis_with_unicode(text):
    # Uncomment the line below if your text has escaped Unicode sequences
    # text = text.encode('utf-8').decode('unicode_escape')
    
    processed_chars = []
    for char in text:
        if emoji.is_emoji(char):
            # Get the Unicode code point and format as U+XXXXXX
            code_point = ord(char)
            processed_chars.append(f"U+{code_point:X}")
        else:
            processed_chars.append(char)
    return ''.join(processed_chars)

# Example usage:
tweet = "Just posted a new photo 😎 #Python"
processed_tweet = replace_emojis_with_unicode(tweet)
print(processed_tweet)
# Output: "Just posted a new photo U+1F60E #Python"

Step 3: Save to CSV Correctly #

When writing to CSV, make sure to use utf-8 encoding to preserve all characters. Here's a quick example using the csv module:

import csv

tweets = [
    {"text": "Hello 😃 world!"},
    {"text": "Loving this weather ☀️"}
]

with open('tweets.csv', 'w', newline='', encoding='utf-8') as f:
    writer = csv.DictWriter(f, fieldnames=["text"])
    writer.writeheader()
    for tweet in tweets:
        processed_text = replace_emojis_with_unicode(tweet["text"])
        writer.writerow({"text": processed_text})

Notes on Edge Cases #

• Combined Emojis: Some emojis (like 👨‍💻) are made of multiple Unicode code points. The function above will replace each part with its own U+ notation (e.g., U+1F468U+200DU+1F4BB), which is technically accurate since these don't have a single code point.
• No External Library: If you don't want to use the emoji library, you can check if a character falls within Unicode emoji ranges. However, this requires maintaining a list of ranges, which the emoji library handles automatically.

Hope this solves your problem! 😊

内容的提问来源于stack exchange，提问作者HARSH GUPTA