Python3(Windows10)下如何限制正则匹配时长避免灾难性回溯

可行方案 #

1. 触发灾难性回溯时停止匹配操作 #

使用第三方库regex替代标准库re，该库原生支持检测灾难性回溯，可通过max_backtrack参数设置允许的最大回溯次数，超出后直接抛出异常，无需修改原有正则表达式：

import regex

def my_func():
    astr = "http://www.fapiao.com/dzfp-web/pdf/download?request=6e7JGm38jfjghVrv4ILd-kEn64HcUX4qL4a4qJ4-CHLmqVnenXC692m74H5oxkjgdsYazxcUmfcOH2fAfY1Vw__%5EDadIfJgiEf"
    pattern = "^([hH][tT]{2}[pP]://|[hH][tT]{2}[pP][sS]:)(([A-Za-z0-9-~]+).)+([A-Za-z0-9-~\/])+$"
    # 设置最大回溯次数，超出则终止匹配
    reg = regex.compile(pattern, max_backtrack=10000)
    try:
        result = reg.match(astr)
        return result
    except regex.error:
        print("触发灾难性回溯，已停止匹配")
        return None

2. 限制正则匹配的时长/回溯次数，超时则抛出异常 #

方案A：使用regex库的超时参数

regex库直接支持timeout参数，设置匹配的最长耗时，到点自动终止并抛出TimeoutError，完全适配Windows平台：

import regex

def my_func():
    astr = "http://www.fapiao.com/dzfp-web/pdf/download?request=6e7JGm38jfjghVrv4ILd-kEn64HcUX4qL4a4qJ4-CHLmqVnenXC692m74H5oxkjgdsYazxcUmfcOH2fAfY1Vw__%5EDadIfJgiEf"
    pattern = "^([hH][tT]{2}[pP]://|[hH][tT]{2}[pP][sS]:)(([A-Za-z0-9-~]+).)+([A-Za-z0-9-~\/])+$"
    reg = regex.compile(pattern)
    try:
        # 设置2秒超时
        result = reg.match(astr, timeout=2)
        return result
    except regex.TimeoutError:
        print("匹配超时，已终止")
        return None

方案B：基于子进程的超时限制（不依赖第三方库）

Windows下线程无法强制中断C层面执行的正则匹配，因此用子进程隔离匹配逻辑，通过终止子进程实现超时限制：

import re
import multiprocessing

def match_worker(pattern, astr, queue):
    reg = re.compile(pattern)
    result = reg.match(astr)
    queue.put(result)

def my_func():
    astr = "http://www.fapiao.com/dzfp-web/pdf/download?request=6e7JGm38jfjghVrv4ILd-kEn64HcUX4qL4a4qJ4-CHLmqVnenXC692m74H5oxkjgdsYazxcUmfcOH2fAfY1Vw__%5EDadIfJgiEf"
    pattern = "^([hH][tT]{2}[pP]://|[hH][tT]{2}[pP][sS]:)(([A-Za-z0-9-~]+).)+([A-Za-z0-9-~\/])+$"
    
    queue = multiprocessing.Queue()
    p = multiprocessing.Process(target=match_worker, args=(pattern, astr, queue))
    p.start()
    # 设置2秒超时等待
    p.join(timeout=2)
    
    if p.is_alive():
        p.terminate()
        p.join()
        print("匹配超时，已终止进程")
        return None
    else:
        return queue.get()

3. 限制函数执行时间 #

Windows下最可靠的函数超时方案是基于子进程，因为线程中断对C扩展层的阻塞操作无效。以下是用ProcessPoolExecutor实现的简洁版本：

import re
from concurrent.futures import ProcessPoolExecutor, TimeoutError

def match_task(pattern, astr):
    reg = re.compile(pattern)
    return reg.match(astr)

def my_func():
    astr = "http://www.fapiao.com/dzfp-web/pdf/download?request=6e7JGm38jfjghVrv4ILd-kEn64HcUX4qL4a4qJ4-CHLmqVnenXC692m74H5oxkjgdsYazxcUmfcOH2fAfY1Vw__%5EDadIfJgiEf"
    pattern = "^([hH][tT]{2}[pP]://|[hH][tT]{2}[pP][sS]:)(([A-Za-z0-9-~]+).)+([A-Za-z0-9-~\/])+$"
    
    with ProcessPoolExecutor(max_workers=1) as executor:
        future = executor.submit(match_task, pattern, astr)
        try:
            # 设置2秒超时
            result = future.result(timeout=2)
            return result
        except TimeoutError:
            print("函数执行超时")
            return None

内容的提问来源于stack exchange，提问作者luckin