Python爬虫脚本遇TypeError与栈溢出问题求助

问题代码 #

from urllib.request import Request, urlopen
from bs4 import BeautifulSoup


root = "https://finviz.com/quote.ashx?t="


tickers = ['AMZN', 'GS']

news_tables = {}
for ticker in tickers:
    url = root + ticker
    req = Request(url=url, headers={'user-agent': 'dirty-30'})
    response = urlopen(req)
    #print(response)
    html = BeautifulSoup(response, 'html')
   # print(html)
    news_table = html.find(id='news-table')
    news_tables[ticker] = news_table
    

amzn_data = news_tables['AMZN']
amzn_rows = amzn_data.findALL('tr')
print(news_tables)   

报错信息 #

TypeError: 'NoneType' object is not callable

Exception in comms call get_value:

  File "C:\Users\austi\Anaconda3\lib\site-packages\spyder_kernels\comms\commbase.py", line 347, in _handle_remote_call
    self._set_call_return_value(msg_dict, return_value)

  File "C:\Users\austi\Anaconda3\lib\site-packages\spyder_kernels\comms\commbase.py", line 384, in _set_call_return_value
    self._send_message('remote_call_reply', content=content, data=data,

  File "C:\Users\austi\Anaconda3\lib\site-packages\spyder_kernels\comms\frontendcomm.py", line 109, in _send_message
    return super(FrontendComm, self)._send_message(*args, **kwargs)

  File "C:\Users\austi\Anaconda3\lib\site-packages\spyder_kernels\comms\commbase.py", line 247, in _send_message
    buffers = [cloudpickle.dumps(

  File "C:\Users\austi\Anaconda3\lib\site-packages\cloudpickle\cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)

  File "C:\Users\austi\Anaconda3\lib\site-packages\cloudpickle\cloudpickle_fast.py", line 609, in dump
    raise pickle.PicklingError(msg) from e

_pickle.PicklingError: Could not pickle object as excessively deep recursion required.

已尝试的解决操作 #

• 添加sys.setrecursionlimit(30000000)调整递归上限，未解决问题
• 重启Anaconda，确认Spyder为最新版本

核心疑问 #

1. 仅爬取两个页面，数据量极小，却触发栈溢出和PicklingError，怀疑是否存在无限循环重复收集数据的情况，但无法定位具体位置。
2. 打开dict类型的news_tables时，会收到栈溢出提示且内核重启，推测NoneType错误是栈溢出破坏变量导致字典变为空，但不确定根本原因。
3. 系统具备充足存储和RAM，排除硬件资源不足的影响。

内容的提问来源于stack exchange，提问作者oz.vegas