如何在线程中使用html.render？解决无当前事件循环报错

我在将解析页面并执行html.render()的函数放入线程运行时，出现错误：Error: There is no current event loop in thread 'Thread-1 (take_proxy_us_spys_one_thread)，尝试相关实现方案后仍报错，代码如下：

import urllib3
import requests
import time
from requests_html import HTMLSession
import threading
import fake_useragent

def take_proxy_us_spys_one(urls: list=[], header:dict = None,):
    for url in urls:
        try:
            url_first = 'https://spys.one'
            r = requests.get(url_first, headers=header)
            cookies = r.cookies
            session = HTMLSession()
            r = session.post(url,
                             data={'xx00': '','xpp': '5','xf1': '0','xf2': '0','xf3': '0','xf4': '0', 'xf5': '0'},
                             headers=header,
                             cookies=cookies)
            r.html.render(reload=False,)
            print(str(r))
        except Exception as exc:
            print("Error: " + str(exc))

def take_proxy_us_spys_one_thread(event, sleeptime= 60, urls=[], lock = None):
    while event.is_set():
        try:
            user = fake_useragent.UserAgent().random
            header = {'User-Agent': user}
            lock.acquire() if lock!=None else None
            proxies_1 = take_proxy_us_spys_one(urls=urls, header=header)
            lock.release() if lock != None else None
            time.sleep(sleeptime)
        except Exception as exc:
            print("Error: " + str(exc))
            time.sleep(sleeptime)

if __name__ == '__main__':
    start_in_thread = True
    urllib3.disable_warnings()

    urls_spys_one = [
        'https://spys.one/free-proxy-list/ALL/'
    ]

    lock = threading.Lock()
    event = threading.Event()
    event.set()

    t2 = threading.Thread(target=take_proxy_us_spys_one_thread, args=(event, 10, urls_spys_one, lock),).start()

问题原因 #

requests_html的render()方法底层依赖asyncio事件循环，但Python子线程默认不会自动创建事件循环，且主线程的事件循环不会被子线程继承，因此调用render()时会提示找不到当前事件循环。

修复方案 #

为子线程手动创建并设置asyncio事件循环，推荐以下两种实现方式：

方式一：在子线程入口初始化一次事件循环（高效推荐） #

在子线程的循环逻辑前初始化事件循环，避免每次请求重复创建，提升效率：

import urllib3
import requests
import time
from requests_html import HTMLSession
import threading
import fake_useragent
import asyncio  # 新增导入

def take_proxy_us_spys_one(urls: list=[], header:dict = None,):
    for url in urls:
        try:
            url_first = 'https://spys.one'
            r = requests.get(url_first, headers=header)
            cookies = r.cookies
            session = HTMLSession()
            r = session.post(url,
                             data={'xx00': '','xpp': '5','xf1': '0','xf2': '0','xf3': '0','xf4': '0', 'xf5': '0'},
                             headers=header,
                             cookies=cookies)
            r.html.render(reload=False,)
            print(str(r))
            session.close()  # 新增：关闭session释放资源
        except Exception as exc:
            print("Error: " + str(exc))

def take_proxy_us_spys_one_thread(event, sleeptime= 60, urls=[], lock = None):
    # 子线程启动时初始化专属事件循环
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    
    while event.is_set():
        try:
            user = fake_useragent.UserAgent().random
            header = {'User-Agent': user}
            lock.acquire() if lock!=None else None
            proxies_1 = take_proxy_us_spys_one(urls=urls, header=header)
            lock.release() if lock != None else None
            time.sleep(sleeptime)
        except Exception as exc:
            print("Error: " + str(exc))
            time.sleep(sleeptime)

if __name__ == '__main__':
    start_in_thread = True
    urllib3.disable_warnings()

    urls_spys_one = [
        'https://spys.one/free-proxy-list/ALL/'
    ]

    lock = threading.Lock()
    event = threading.Event()
    event.set()

    t2 = threading.Thread(target=take_proxy_us_spys_one_thread, args=(event, 10, urls_spys_one, lock),).start()

方式二：在每次请求前初始化事件循环 #

如果需要为每个请求创建独立的事件循环，可以在请求逻辑前初始化：

import urllib3
import requests
import time
from requests_html import HTMLSession
import threading
import fake_useragent
import asyncio  # 新增导入

def take_proxy_us_spys_one(urls: list=[], header:dict = None,):
    for url in urls:
        try:
            # 为当前请求创建并设置事件循环
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
            
            url_first = 'https://spys.one'
            r = requests.get(url_first, headers=header)
            cookies = r.cookies
            session = HTMLSession()
            r = session.post(url,
                             data={'xx00': '','xpp': '5','xf1': '0','xf2': '0','xf3': '0','xf4': '0', 'xf5': '0'},
                             headers=header,
                             cookies=cookies)
            r.html.render(reload=False,)
            print(str(r))
            session.close()  # 新增：关闭session释放资源
        except Exception as exc:
            print("Error: " + str(exc))

# 其余代码同原代码...

关键修改点 #

1. 新增asyncio模块导入，用于创建和设置事件循环
2. 在子线程内初始化专属事件循环，确保HTMLSession和render()能找到可用的循环
3. 新增session.close()，避免session资源泄漏

内容的提问来源于stack exchange，提问作者rack