Python将爬虫获取的字典数据写入Excel仅保存单条数据问题求助

• 核心问题：你在遍历餐厅详情页的循环内，每次都执行df.to_excel('./Lagos.xlsx')写入操作，pandas的to_excel方法默认是覆盖写入模式，每爬一条数据就会清空之前的文件内容写入新数据，最终只会保留最后一次循环写入的单条数据。
• 附加问题：当前代码的列表页循环写为for url in pages[:1]，仅会爬取第一页的餐厅列表；详情页循环写为for url in rest[:2]，仅会爬取前2条餐厅详情，要爬全量数据需要删除这两个切片。

提前声明一个空列表存储所有爬取到的餐厅信息字典，等所有数据爬取完成后，统一转换为DataFrame写入Excel，避免重复覆盖文件。
修正后的参考代码如下：

from scrapingbee import ScrapingBeeClient
import requests
from bs4 import BeautifulSoup
import pandas as pd

SCRAPINGBEE_API_KEY = "bzzzz"
endpoint = "https://app.scrapingbee.com/api/v1"
pages = [
    'https://www.businesslist.com.ng/category/restaurants/1/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/2/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/3/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/4/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/5/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/6/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/7/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/8/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/9/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/10/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/11/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/12/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/13/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/14/city:lagos',
    'https://www.businesslist.com.ng/category/restaurants/15/city:lagos'
    ]

rest = []

#GET_LINKS
# 要爬全量列表页就把[:1]删掉
for url in pages[:1]:
    params = {
    'api_key': SCRAPINGBEE_API_KEY,
    'url': url}

    response = requests.get(endpoint, params=params)
    soup = BeautifulSoup(response.content, 'html.parser')

    body = soup.find_all('h4')
    for items in body:
        item = items.find('a').get('href')
        item_link = 'https://www.businesslist.com.ng' + item
        rest.append(item_link)

#GET_REST
all_rest_info = []
# 要爬全量详情就把[:2]删掉
for url in rest[:2]:
    params = {
    'api_key': SCRAPINGBEE_API_KEY,
    'url': url}
    info = {}
    response = requests.get(endpoint, params=params)
    soup = BeautifulSoup(response.content, 'html.parser')
    # 可选：加异常捕获避免某个字段缺失导致整个爬取中断
    try:
        restaraunt_name = soup.find('b', {'id':'company_name'}).text
        info['Restaraunt'] = restaraunt_name
        location = soup.find('div', {'class':'text location'}).text.split('View Map')[0]
        info['Location'] = location
        phone = soup.find('div', {'class':'text phone'}).text[:11]
        info['Phone'] = phone
        web = soup.find('div', {'class':'text weblinks'}).text
        info['web'] = web
        all_rest_info.append(info)
    except Exception as e:
        print(f"爬取{url}时出错：{e}")
        continue

# 所有数据爬完后统一写入Excel
df = pd.DataFrame(all_rest_info)
# index=False 是为了不把pandas自动生成的行索引写入文件
df.to_excel('./Lagos.xlsx', index=False)

内容的提问来源于stack exchange，提问作者Кирилл Боровой