You need to enable JavaScript to run this app. 社交媒体评论情感分析中emoji转文本情感失败问题求助
优惠活动
大模型
产品
解决方案
定价
更多

社交媒体评论情感分析中emoji转文本情感失败问题求助

问题背景

采集了某产品广告对应的Facebook、Twitter评论数据,拟对评论开展情感分析工作。文本清洗环节需将emoji转换为对应的文本情感,最大化捕获评论中的全部情感信息。已尝试emoji.demojize(text)及各类公开解决方案,均无法实现转换效果,排查一周未定位问题。

已尝试的代码方案

方案1:自定义emoji映射转换

import io
import json
import re

def handleEmojis(text, keep_emoticons = False):
    global emoji_sentiment_matching
    if not 'emoji_sentiment_matching' in globals():
        with io.open('emoji.json', 'r', encoding = "UTF-8") as outfile:
            emoji_sentiment_matching = json.load(outfile)
    HASHTAG_PATTERN = re.compile(r'#\w*')
    EMOJIS_PATTERN_PLAIN_TEXT = re.compile(r"(?:X|:|;|=)(?:-)?(?:\)|\(|O|D|P|S){1,}", re.IGNORECASE)
    EMOJIS_PATTERN_SYMBOLS = re.compile(u'[\U00002600-\U000027BF]|[\U0001f300-\U0001f64F]|[\U0001f680-\U0001f6FF]')

    if keep_emoticons:
        # Replace emoji with sentiment
        for emoji in emoji_sentiment_matching:
            if emoji["emoji"] in text:
                ## Adding space if text follows right away / is right before the emoticon
                idx = text.find(emoji["emoji"])
                (space1,space2) = ("","")
                if (idx-1) >= 0 and text[idx-1] != " ":
                    space1 = " "
                if (idx+1) <= len(text) and text[idx+1] != " ":
                    space2 = " "
                ## replace emoticon with its sentiment
                text = text.replace(emoji["emoji"], "{}emoji%%{}{}".format(space1, emoji["subgroup"], space2))
    ## TO IMPLEMENT: Sentiment of other emoticons like :), :-), :-/
    else:
        for r in re.findall(EMOJIS_PATTERN_SYMBOLS,text):
            text = text.replace(r, "")
        for r in re.findall(EMOJIS_PATTERN_PLAIN_TEXT,text):
            text = text.replace(r, "")
    return text.strip()

FB_df['demojified']=FB_df['Text'] 
for i in range(len(FB_df)):
  text = FB_df.loc[i,"demojified"]
  handleEmojis(text, keep_emoticons = False)

print(FB_df)

运行后demojified列无预期转换效果。

方案2:调用emot库转换

import re
from emot.emo_unicode import UNICODE_EMO, EMOTICONS
from emoji import demojize
def convert_emojis(text):
    for emot in UNICODE_EMO:
        text = re.sub(r'('+emot+')', "_".join(UNICODE_EMO[emot].replace(",","").replace(":","").split()), text)
    return text

def convert_emoticons(text):
    for emot in EMOTICONS:
        text = re.sub(u'('+emot+')', "_".join(EMOTICONS[emot].replace(",","").split()), text)
    return text

FB_df['demojified']=FB_df['Text'] 

for row in FB_df['demojified']:
    for text in row:
        text=text
        convert_emojis(text)

FB_df.loc[:,'demojified']

方案3:结合demojize转换

import re
from emot.emo_unicode import UNICODE_EMO, EMOTICONS
import emoji
def convert_emojis(text):
    for emot in UNICODE_EMO:
        text = re.sub(r'('+emot+')',"_".join(UNICODE_EMO[emot].replace(",","").replace(":","").split()), text)
    return text

def convert_emoticons(text):
    for emot in EMOTICONS:
        text = re.sub(u'('+emot+')', "_".join(EMOTICONS[emot].replace(",","").split()), text)
    return text

FB_df['demojified']=FB_df['Text'] 
for row in FB_df['demojified']:
    for text in row:
        text=str(text)
        text = emoji.demojize(text)
问题根因与修复方案

核心错误点

  • 临时变量未回写:所有遍历过程中修改的text都是临时变量,转换完成后没有重新赋值回DataFrame对应单元格,修改结果直接被丢弃
  • 参数设置错误:方案1调用时传入keep_emoticons = False,该分支逻辑是直接删除所有emoji而非转换为情感文本,和需求完全相反
  • 循环逻辑错误:方案2、3中遍历FB_df['demojified']时嵌套遍历row的每个字符,实际是对单个字符做转换而非整行文本
  • 语法错误:部分版本的convert_emoticons函数return语句写在循环内部,第一次循环就直接返回,只处理了第一个匹配到的表情
  • 依赖兼容问题:低版本emoji、emot库对新emoji支持不足,部分库旧版本中UNICODE_EMO已改名为UNICODE_EMOJI

修复后可运行代码

import re
import emoji
import pandas as pd
from emot.emo_unicode import UNICODE_EMOJI, EMOTICONS

# 转换unicode emoji为情感文本
def convert_emojis(text):
    # 先做基础转换,再统一处理格式
    text = emoji.demojize(text, delimiters=(" ", " "))
    for emot, desc in UNICODE_EMOJI.items():
        text = text.replace(emot, desc.replace(",", "").replace(":", "").replace(" ", "_"))
    return text

# 转换颜文字为情感文本
def convert_emoticons(text):
    for emot, desc in EMOTICONS.items():
        text = re.sub(re.escape(emot), desc.replace(",", "").replace(" ", "_"), text)
    return text

# 统一处理入口
def process_emoji(text):
    text = str(text)
    text = convert_emojis(text)
    text = convert_emoticons(text)
    return text

# 用pandas apply批量处理,避免手写循环出错
FB_df['demojified'] = FB_df['Text'].apply(process_emoji)

内容的提问来源于stack exchange,提问作者Sivan

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.10.21 11:24:04