如何用Python高效解析数据？邮件头提取代码优化问询

Hey there! I see you're looking to optimize your email header extraction code—great call thinking about efficiency beyond small datasets. Your current approach runs a regex search for each field individually, which means scanning the entire email content multiple times (once per field). That adds up quickly with larger datasets or more fields to extract. Let's look at two better approaches to fix this:

1. Single Regex Pass to Capture All Fields #

Instead of looping through each field and running a separate search, we can build a single regex pattern that matches all your target fields in one go. This way, we only scan the email content once, which is much more efficient.

Here's how you can adjust your code:

import re

data=""" Message-ID: <1608636066635.7f830.79689714@crcvmail15.nm> Received: from 125.209.x.x (net58.219.x-x.host.lt-nn.net [91.219.x.x]) by crcvmail15.google.com with ESMTP id +844Q-zuS122aEqk5CZDZg for <test@google.com>; Received: from 125.209.x.x (net58.219.x-18.host.lt-nn.net [91.219.x.x]) by crcvmail15.google.com with ESMTP id +844Q-zuS122aEqk5CZDZg for <test@google.com>; Tue, 22 Dec 2020 11:20:58 -0000 From: "test"<from@google.com> To: test@google.com Subject:example email Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable """

# Build a regex pattern that matches any of our target fields
fields = ['From','To','Cc','Subject','Message-ID','Date','Return-Path','Reply-To']
# Escape fields that have special regex characters
escaped_fields = [re.escape(field) for field in fields]
# Use positive lookahead to stop matching at the next header field or end of string
pattern = re.compile(r'({}):\s*(.*?)(?=\s*(?:{}|$))'.format('|'.join(escaped_fields), '|'.join(escaped_fields)), re.DOTALL)

# Extract all matches in one pass
matches = pattern.findall(data)
# Convert to a dictionary for easy access
header_dict = {key.strip(): value.strip() for key, value in matches}

# Print the results
for key, value in header_dict.items():
    print(f"{key}: {value}")

Notes on this approach: #

• The positive lookahead ensures we stop capturing a field's value as soon as we hit the next header field or the end of the content.
• re.DOTALL allows the pattern to match across line breaks, handling folded header values that wrap onto multiple lines.
• The resulting dictionary makes it trivial to access any extracted field by name.

2. Use Python's Built-in email Module (Recommended) #

For even better efficiency and reliability, turn to Python's standard email module. It's purpose-built for parsing email headers, handles all the edge cases regex might miss, and is optimized for speed under the hood.

Here's how to implement it:

from email import message_from_string

data=""" Message-ID: <1608636066635.7f830.79689714@crcvmail15.nm> Received: from 125.209.x.x (net58.219.x-x.host.lt-nn.net [91.219.x.x]) by crcvmail15.google.com with ESMTP id +844Q-zuS122aEqk5CZDZg for <test@google.com>; Received: from 125.209.x.x (net58.219.x-18.host.lt-nn.net [91.219.x.x]) by crcvmail15.google.com with ESMTP id +844Q-zuS122aEqk5CZDZg for <test@google.com>; Tue, 22 Dec 2020 11:20:58 -0000 From: "test"<from@google.com> To: test@google.com Subject:example email Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable """

# Parse the email content into a Message object
msg = message_from_string(data)

# Define the fields we want to extract
target_fields = ['From','To','Cc','Subject','Message-ID','Date','Return-Path','Reply-To']

# Extract fields (automatically handles case insensitivity!)
header_dict = {}
for field in target_fields:
    value = msg.get(field)
    if value:
        header_dict[field] = value.strip()

# Print results
for key, value in header_dict.items():
    print(f"{key}: {value}")

Why this is the best choice: #

• Efficiency: The module parses the entire header in a single pass, no repeated regex scans.
• Accuracy: It correctly handles folded lines, case-insensitive field names (e.g., from vs From), and other email standard quirks.
• Maintainability: The code is clean, readable, and doesn't require complex regex debugging.

Both approaches will outperform your original loop-based method, especially as your dataset grows. The email module is the go-to for most real-world scenarios since it's designed exactly for this task.

内容的提问来源于stack exchange，提问作者user11230064