Python Web爬虫无法向.txt文件写入爬取数据求助

你的代码主要有两个核心问题导致生成空文件，下面逐一说明并给出修复方案：

1. 缩进错误导致逻辑失效 #

你在遍历月度页面链接的循环中，soup = BeautifulSoup(content, "html.parser") 以及后续的数字提取代码缩进错误，跑到了for link in links:循环的外面。这意味着代码要么只处理最后一个链接的内容（如果links不为空），要么完全不执行数字提取逻辑（如果links为空）。

修复方式：将这部分代码缩进，放到循环内部：

# Scrape the winning numbers for each monthly results page
winning_numbers = []
for link in links:
    response = requests.get(link)
    content = response.content
    # 修复缩进：将下面两行放到循环内
    soup = BeautifulSoup(content, "html.parser")
    for tag in soup.find_all("div", class_="ball blue5 fcblack1"):
        numbers = tag.text.strip().split()
        winning_numbers.append(numbers)

2. 链接选择器错误导致links为空 #

你当前用soup.find_all("td", class_="td0")直接提取href，但目标网站中，链接是嵌套在<td class="td0">内部的<a>标签里，td本身并没有href属性。这会导致links列表始终为空，后续的数字提取循环根本不会执行，最终文件为空。

修复方式：调整选择逻辑，先找到目标td，再提取内部a标签的href，同时注意拼接相对路径为完整URL：

links = []
# 遍历td.td0，再找里面的a标签
for td in soup.find_all("td", class_="td0"):
    a_tag = td.find("a")
    if a_tag:
        href = a_tag.get("href")
        if href:
            # 拼接完整URL，避免相对路径请求失败
            full_url = "http://www.calotteryx.com" + href
            links.append(full_url)

额外调试与优化建议 #

• 调试时加入打印语句，比如print(f"抓取到{len(links)}个月度链接")，确认links是否有效；
• 添加请求头模拟浏览器访问，避免被网站拦截；
• 加入异常捕获，避免请求失败时静默报错。

修复后的完整代码示例：

import requests
from bs4 import BeautifulSoup

# Scrape the links to each monthly results page
url = "http://www.calotteryx.com/Fantasy-5/drawing-results-calendar.htm"
# 添加headers模拟浏览器
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
}
response = requests.get(url, headers=headers)
content = response.content
soup = BeautifulSoup(content, "html.parser")
links = []
for td in soup.find_all("td", class_="td0"):
    a_tag = td.find("a")
    if a_tag:
        href = a_tag.get("href")
        if href:
            full_url = "http://www.calotteryx.com" + href
            links.append(full_url)

print(f"抓取到{len(links)}个月度链接")

# Scrape the winning numbers for each monthly results page
winning_numbers = []
for link in links:
    try:
        response = requests.get(link, headers=headers)
        response.raise_for_status()  # 抛出请求错误
        content = response.content
        soup = BeautifulSoup(content, "html.parser")
        for tag in soup.find_all("div", class_="ball blue5 fcblack1"):
            numbers = tag.text.strip().split()
            winning_numbers.append(numbers)
    except Exception as e:
        print(f"处理链接{link}时出错：{e}")

# Write the winning numbers to a file
with open("winning_numbers.txt", "w", encoding="utf-8") as f:
    for numbers in winning_numbers:
        f.write(" ".join(numbers) + "\n")

print(f"已写入{len(winning_numbers)}组号码到文件")

内容的提问来源于stack exchange，提问作者BlackHeart