如何从标注list与词汇list中匹配问答对？

需求说明 #

现有两个列表：words 存储词汇内容，labels 对应每个词汇的标签（包含header、question、answer、other等类型），需要将question标签对应的内容与后续的answer标签内容配对，生成键值对形式的问答字典。

实现思路 #

1. 合并连续同标签文本：将词汇与标签一一对应，把连续属于同一标签的词汇合并成完整的文本块，区分出question和answer类型的块。
2. 配对问答块：遍历所有文本块，将相邻的question块与answer块配对，清理问题文本中的冗余符号后组成问答字典。

代码实现 #

def extract_qapairs(words, labels):
    # 第一步：合并连续同标签的文本块
    blocks = []
    current_label = None
    current_words = []
    for word, label in zip(words, labels):
        if label == current_label:
            current_words.append(word)
        else:
            if current_label is not None:
                blocks.append({
                    'label': current_label,
                    'text': ' '.join(current_words)
                })
            current_label = label
            current_words = [word]
    # 处理最后一个未加入的块
    if current_label is not None:
        blocks.append({
            'label': current_label,
            'text': ' '.join(current_words)
        })
    
    # 第二步：配对相邻的question和answer块
    qa_dict = {}
    for i in range(len(blocks)-1):
        current_block = blocks[i]
        next_block = blocks[i+1]
        if current_block['label'] == 'question' and next_block['label'] == 'answer':
            # 清理问题文本末尾的冒号和多余空格
            question_text = current_block['text'].rstrip(':').strip()
            qa_dict[question_text] = next_block['text'].strip()
    
    return qa_dict

# 示例数据
labels = ['header', 'header', 'header', 'header', 'header', 'other', 'other', 'other', 'header', 'header', 'header', 'header', 'header', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'question', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'other', 'header', 'question', 'question', 'question', 'question', 'answer', 'question', 'question', 'question', 'answer', 'question', 'question', 'answer', 'answer', 'question', 'question', 'answer', 'answer', 'answer', 'header', 'question', 'question', 'question', 'answer', 'answer', 'question', 'question', 'question', 'answer', 'answer', 'answer', 'question', 'question', 'question', 'answer', 'question', 'answer', 'question', 'question', 'question', 'question', 'answer', 'question', 'question', 'question', 'question', 'answer', 'answer', 'question', 'answer', 'question', 'question', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'answer', 'other', 'other', 'other', 'other', 'other', 'other', 'other']
words = ['a', 'UK', 'Visas', '&', 'Immigration', 'ANNI', '|', 'Standard', 'Document', 'checklist', 'This', 'checklist', 'shows:', '©', 'the', 'Length', 'and', 'type', 'of', 'visa', 'you', 'have', 'applied', 'for', '‘©', 'the', 'documents', 'you', 'are', 'providing', 'in', 'support', 'of', 'your', 'application', '‘©', 'what', 'you', 'need', 'to', 'do', 'next', '‘You', 'must', 'only', 'sign', 'this', 'checklist', 'when', 'you', 'go', 'to', 'your', 'appointment,', '‘Your', 'appointment', 'is', 'at', 'New', 'Delhi,', 'on', '18', 'April,', '2018', 'at', '4:45', 'PM.', 'Mezzanine', 'Floor,', 'Baba', 'Kadak', 'Singh', 'Marg.,', 'Shivali', 'Stadium', 'Metro', 'Station,', 'Connaught', 'Place,', 'New', 'Dethi,', '110001,', 'INDIA', '‘Your', 'Prime', 'Time', 'appointment', 'must', 'be', 'pald', 'for', 'before', 'you', 'attend', 'your', 'appointment.', 'You', 'can', 'do', 'this', 'on', 'our', 'visa', 'application', 'centre', "partner's", 'website', 'at:', 'https:/iwww.visglobal.co.uk/globaV/index.html.', 'Product', 'Category', 'Visitor', 'visa', 'Duration', 'month', 'Payment', 'notification', 'number:', '3070005116926', 'Total', 'fee:', '8923.00', 'INR', 'Application', 'date:', '13', 'April', '2018', 'Applicant', "Applicant's", 'full', 'name', 'Sahil', 'Sareen', 'Date', 'of', 'birth:', '29', 'August', '1983', 'Country', 'of', 'birth:', 'India', 'Nationality', 'India', 'Passport', '/', 'Travel', 'document', '3859145', 'number.', 'Contact', 'telephone', 'number:', '+0091', '9873990024', 'Email', 'sahil.sareen21@yahoo.com', 'Submitted', 'date:', '18', 'April', '2018', '07:29', 'British', '‘Summer', 'Time', '(BST)', 'Gwro47660499', 'Sahil', 'Sareen', 'Page', '|', 'of', '8']

# 提取问答对
result = extract_qapairs(words, labels)

# 格式化打印结果
import pprint
pprint.pprint(result)

输出结果 #

{'Category Visitor visa Duration': 'month',
 'Payment notification number': '3070005116926',
 'Total fee': '8923.00 INR',
 'Application date': '13 April 2018',
 "Applicant's full name": 'Sahil Sareen',
 'Date of birth': '29 August 1983',
 'Country of birth': 'India',
 'Nationality': 'India',
 'Passport / Travel document': '3859145',
 'number. Contact telephone number': '+0091 9873990024',
 'Email': 'sahil.sareen21@yahoo.com',
 'Submitted date': '18 April 2018 07:29 British ‘Summer Time (BST)'}

内容的提问来源于stack exchange，提问作者Laxmi