线程化子域名枚举器枚举完成后冻结无法退出问题求助

问题描述 #

开发线程化子域名枚举器时，完成google.com这类站点的子域名枚举后，程序会冻结且无法退出，导致无法执行最终的print("\n[+] Done enumerating")语句。即使在executor.map后添加executor.shutdown(wait=True)，问题仍未解决。原代码如下：

import requests
import validators
from concurrent.futures import ThreadPoolExecutor


def enumerate_subdomain(subdomain):
    http_url = f"http://{subdomain}.{domain}"
    https_url = f"https://{subdomain}.{domain}"
    try:
        if validators.url(https_url) == True:
            requests.get(https_url)
            print(f"[!] Subdomain found: {https_url}")
        else:
            requests.get(http_url)
            print(f"[!] Subdomain found: {http_url}")
    except requests.ConnectionError:
        pass

domain = input("[+] Domain (e.g. example.com): ")
file = input("[+] Path to wordlist file: ")
threads = int(input("[+] Amount of threads (10-50 is recommended): "))

with open(file, "r") as f:
    content = f.read()
    subdomains = content.splitlines()


if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=threads) as executor:    
        executor.map(enumerate_subdomain, subdomains)     
        executor.shutdown(wait=False)

print("\n[+] Done enumerating")

问题根源 #

1. 无超时的网络请求：requests.get默认没有超时限制，遇到无法连接的子域名时，线程会无限阻塞等待响应，导致线程池无法正常关闭
2. 冗余的线程池关闭操作：ThreadPoolExecutor的with语句会自动调用shutdown(wait=True)，手动添加shutdown(wait=False)会强制提前关闭线程池，但阻塞的线程仍在后台运行，阻止程序退出
3. 无效的URL验证：validators.url的判断逻辑冗余，requests.get本身会处理URL有效性，且该判断可能误判合法的HTTPS子域名
4. 异常捕获不全面：仅捕获ConnectionError，未处理超时、DNS解析失败等其他可能导致线程挂起的异常

修复后的代码 #

import requests
from concurrent.futures import ThreadPoolExecutor


def enumerate_subdomain(subdomain):
    http_url = f"http://{subdomain}.{domain}"
    https_url = f"https://{subdomain}.{domain}"
    # 先尝试HTTPS
    try:
        response = requests.get(https_url, timeout=5)
        # 仅当响应状态码为2xx/3xx时判定为有效子域名
        if response.status_code < 400:
            print(f"[!] Subdomain found: {https_url}")
        return
    except (requests.ConnectionError, requests.Timeout, requests.RequestException):
        pass
    # HTTPS失败后尝试HTTP
    try:
        response = requests.get(http_url, timeout=5)
        if response.status_code < 400:
            print(f"[!] Subdomain found: {http_url}")
    except (requests.ConnectionError, requests.Timeout, requests.RequestException):
        pass

domain = input("[+] Domain (e.g. example.com): ")
file = input("[+] Path to wordlist file: ")
threads = int(input("[+] Amount of threads (10-50 is recommended): "))

with open(file, "r") as f:
    subdomains = f.read().splitlines()


if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=threads) as executor:    
        executor.map(enumerate_subdomain, subdomains)

print("\n[+] Done enumerating")

关键修改说明 #

• 添加请求超时：给所有requests.get调用添加timeout=5参数，确保线程不会因无响应的请求无限阻塞
• 移除冗余的shutdown调用：依赖with语句自动管理线程池生命周期，确保所有线程完成后再关闭
• 优化子域名检测逻辑：移除validators.url判断，先尝试HTTPS，失败后再尝试HTTP，同时检查响应状态码，避免误报无法访问的子域名
• 扩展异常捕获范围：捕获Timeout和RequestException，覆盖更多可能导致线程挂起的异常场景

内容的提问来源于stack exchange，提问作者Anand Melbourne