Python爬取房产网站数据遇NameError及列表为空问题求助

• 你在循环中获取到link和address后，没有将这组数据追加到listing_details列表中，只是做了变量赋值，所以列表始终为空。
• 没有提前初始化df对象就直接调用df.append()方法，触发未定义变量的报错。

import pandas as pd
import requests
from bs4 import BeautifulSoup

headers = {'user-agent': 'Mozilla/5.0'}
url = "https://www.realestate.co.nz/residential/sale/auckland?by=latest&oad=true&pm=1&scat=1%2C7%2C2%2C4%2C6%2C5%2C3%2C50%2C53"
response = requests.get(url, headers=headers)
listings = BeautifulSoup(response.content, "lxml")

listing_details = []

details = listings.findAll('div', attrs={"data-test":"tile"})
for detail in details:
    # 获取房源链接
    links = detail.findAll('a', href=True)
    for link in links:
        link="https://www.realestate.co.nz" + link['href']
    
    # 获取房源地址
    addresses = detail.findAll('h3')
    for address in addresses:
        address=address.text.strip()
    # 新增：将当前房源的链接和地址追加到存储列表
    listing_details.append([link, address])

# 直接用填充完成的列表生成DataFrame
df = pd.DataFrame(listing_details, columns=['Link','Location'])
print(df)

因为每个房源卡片内只有1个链接和1个地址，可以去掉内层循环简化代码：

for detail in details:
    link = "https://www.realestate.co.nz" + detail.find('a', href=True)['href']
    address = detail.find('h3').text.strip()
    listing_details.append([link, address])

内容的提问来源于stack exchange，提问作者Ilovenoodles