使用BeautifulSoup循环爬取NFL多页面时遇AttributeError问题求助
问题
我正在学习用Python BeautifulSoup做基础网页爬取,目标是遍历32个NFL球队页面,获取各队2022年的胜负数据。但代码执行到DataFrame第二次迭代时,出现了AttributeError错误。手动测试单个球队时代码能正常运行,想知道循环中使用req和soup时是不是漏了步骤,期望代码能遍历Excel文件构建的每个球队URL,输出各队每场的胜负结果。
原代码
import requests from urllib.request import Request, urlopen from bs4 import BeautifulSoup as soup import pandas as pd year = '2022' df = pd.read_excel('NFL_Team_Names.xlsx') for i, j in df.iterrows(): url_insert = j[0].replace(' ', '-') url = 'https://champsorchumps.us/team/nfl/' + url_insert + '/' + year print(url) req = Request(url , headers={'User-Agent': 'Mozilla/5.0'}) webpage = urlopen(req).read() soup = soup(webpage, "html.parser") games = 18 for x in range(1, games): insert = 'game_' + str(x) wl_count = 0 row = soup.find('tr', id=''+insert) for elements in row.find_all('td'): if(wl_count == 3): win_lose = elements.get_text().strip() print(j[0] + " " + insert + " " + win_lose[0]) wl_count = wl_count + 1
错误信息
raise AttributeError( AttributeError: ResultSet object has no attribute 'find'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?
解决方案
问题核心是变量名冲突:你将BeautifulSoup类导入时命名为soup,但在循环里又把解析后的页面对象赋值给了同名变量soup。第一次循环时,soup还是类实例能正常工作;第二次循环时,soup已经变成了解析后的ResultSet对象,再调用soup(webpage, "html.parser")就会触发错误——此时soup已经不是原本的BeautifulSoup类了。
同时,部分球队可能不足18场比赛,row找不到时直接调用find_all()也会报错,需要增加判断逻辑。
修改后的代码:
import requests from urllib.request import Request, urlopen from bs4 import BeautifulSoup as soup import pandas as pd year = '2022' df = pd.read_excel('NFL_Team_Names.xlsx') for i, j in df.iterrows(): url_insert = j[0].replace(' ', '-') url = 'https://champsorchumps.us/team/nfl/' + url_insert + '/' + year print(url) req = Request(url , headers={'User-Agent': 'Mozilla/5.0'}) webpage = urlopen(req).read() # 更换解析后页面的变量名,避免和导入的soup类冲突 page_soup = soup(webpage, "html.parser") games = 18 for x in range(1, games): insert = 'game_' + str(x) wl_count = 0 # 使用新的变量名查找元素 row = page_soup.find('tr', id=insert) # 找不到对应场次的行就跳过,避免报错 if not row: continue for elements in row.find_all('td'): if wl_count == 3: win_lose = elements.get_text().strip() print(f"{j[0]} {insert} {win_lose[0]}") wl_count += 1
内容的提问来源于stack exchange,提问作者Jesse
相关产品推荐
相关产品推荐

