如何用Python正则表达式re提取日志文本生成指定格式列表？

Let's fix that regex for you! Your current pattern has a couple of key issues that are holding it back:

1. The greedy .* in your host_name group will match way more than just the IP address—it'll consume characters all the way until the last occurrence of - in the line, breaking your matches.
2. You're only capturing two fields, but you need to extract time and method as well.
3. It doesn't handle the case where there's an extra - (like in your second log line: 197.109.77.178 - - kertzmann3129).

Here's the optimized regex pattern #

This pattern targets each field precisely, handles both logged-in and anonymous-like entries, and captures all the data you need:

pattern = r'(?P<host_name>\d+\.\d+\.\d+\.\d+) - (?:- )?(?P<name>\w+) \[(?P<time>[^\]]+)\] "(?P<method>[^"]+)"'

Let's break down each part: #

• (?P<host_name>\d+\.\d+\.\d+\.\d+): Explicitly matches IPv4 addresses (instead of using greedy .*), so it only captures the host IP and nothing more.
• - (?:- )?: Matches the required - separator, and the optional (?:- )? handles the extra - in lines where there's an anonymous placeholder before the username. The (?:...) is a non-capturing group—we don't need to save this part, just match it.
• (?P<name>\w+): Captures the username (alphanumeric + underscores), which aligns with your sample data. If usernames might include other characters (like hyphens), swap this for (?P<name>[^\s\[]+) to match everything up to the next space or [.
• \[(?P<time>[^\]]+)\]: Captures the entire timestamp inside the square brackets—[^\]]+ matches any character except ], so it stops exactly at the closing bracket.
• "(?P<method>[^"]+)": Captures the full request string inside the quotes, stopping at the closing quote to avoid over-matching.

Full Python Code Example #

Here's how to use this pattern with your log file:

import re

# Read the log file content
with open('your_log_file.txt', 'r') as log_file:
    log_text = log_file.read()

# Use the optimized regex
regex_pattern = r'(?P<host_name>\d+\.\d+\.\d+\.\d+) - (?:- )?(?P<name>\w+) \[(?P<time>[^\]]+)\] "(?P<method>[^"]+)"'

# Generate the list of dictionaries
log_entries = []
for match in re.finditer(regex_pattern, log_text):
    log_entries.append(match.groupdict())

# Print the results to verify
for entry in log_entries:
    print(entry)

Test Output #

For your sample log lines, this will produce exactly the format you want:

{"host_name": "146.204.224.152", "name": "feest6811", "time": "21/Jun/2019:15:45:24 -0700", "method": "POST /incentivize HTTP/1.1"}
{"host_name": "197.109.77.178", "name": "kertzmann3129", "time": "21/Jun/2019:15:45:25 -0700", "method": "DELETE /virtual/solutions/target/web+services HTTP/2.0"}

Bonus: Extending for More Fields #

If you later want to capture the status code (e.g., 302) or byte count (e.g., 4622), you can extend the regex like this:

extended_pattern = r'(?P<host_name>\d+\.\d+\.\d+\.\d+) - (?:- )?(?P<name>\w+) \[(?P<time>[^\]]+)\] "(?P<method>[^"]+)" (?P<status>\d+) (?P<bytes>\d+)'

内容的提问来源于stack exchange，提问作者Yan