如何将对话长文本转换为含章节、发言人、角色、文本列的DataFrame
实现方案
规则匹配说明
整个实现严格对齐给定的判定要求:
- 仅当某行去除首尾空白后,和
section_list、speaker_list中的元素完全相等时,才将其识别为章节标识、发言人标识;发言内容中嵌入的同名字符串因无法满足整行完全匹配的要求,不会被误判 - 角色严格按照“发言人标识下一行单独列示”的文档规则识别,不会和普通发言内容混淆
- 跨多行的发言内容自动拼接为完整文本,不会因换行被拆分为多条记录
- 动态追踪当前所属章节、当前发言人、当前角色,碰到新的章节或发言人标识时,先把暂存的上一段发言落库,避免漏记内容
可直接运行的代码
import pandas as pd # 示例输入 all_text = """Section1 Speaker1 Role1 This is the text spoken by the first speaker. Sometimes it contains the string Speaker1, but when Speaker1 is in the words spoken section it is not on its own line. Speaker2 Role2 Now someone else is talking. Section2 Speaker2 Role2 More words Speaker1 Role1 More words Speaker2 Role2 People can speak multiple times per section.""" section_list = ["Section1", "Section2"] speaker_list = ["Speaker1", "Speaker2"] # 逐行拆分文本 lines = [line.rstrip("\n") for line in all_text.splitlines()] # 初始化追踪状态 current_section = None current_speaker = None current_role = None text_buffer = [] is_expect_role = False result = [] for line in lines: stripped_line = line.strip() # 匹配到章节行 if stripped_line in section_list: # 先将暂存的上一段发言入库 if text_buffer: result.append({ "Section": current_section, "Speaker": current_speaker, "Role": current_role, "Text": " ".join(text_buffer) }) text_buffer = [] current_section = stripped_line current_speaker = None current_role = None is_expect_role = False continue # 匹配到发言人行 if stripped_line in speaker_list: # 先将暂存的上一段发言入库 if text_buffer: result.append({ "Section": current_section, "Speaker": current_speaker, "Role": current_role, "Text": " ".join(text_buffer) }) text_buffer = [] current_speaker = stripped_line is_expect_role = True # 标记下一行是对应角色 continue # 读取下一行的角色信息 if is_expect_role: current_role = stripped_line is_expect_role = False continue # 剩余内容为发言文本,暂存到缓冲区 if current_speaker is not None: text_buffer.append(line.strip()) # 遍历结束后,将最后一段暂存的发言入库 if text_buffer: result.append({ "Section": current_section, "Speaker": current_speaker, "Role": current_role, "Text": " ".join(text_buffer) }) # 生成目标结构的DataFrame df = pd.DataFrame(result, columns=["Section", "Speaker", "Role", "Text"])
运行效果
执行代码后输出的DataFrame完全匹配预期结构,跨多行的发言内容会自动拼接,嵌入在文本中的Speaker1类字符串不会被误判为标识。
内容的提问来源于stack exchange,提问作者Nathan
相关产品推荐
相关产品推荐

