Zillow网页爬取：如何调整代码使列表格式符合预期？

我写了一段爬取Zillow页面的代码，用来抓取卧室数量、浴室数量及房屋面积（sqft）。当前得到的列表格式如下：

['1 bd2 ba982 sqft', '3 bds2 ba1,462 sqft', etc.]

期望格式是：

['1bd 2ba 982sqft', '3bds 2ba 1,462sqft', etc.]

请问需要对以下代码做出哪些修改？

import requests
from bs4 import BeautifulSoup
# import gspread

URL = "https://www.zillow.com/san-francisco-ca/?searchQueryState=%7B%22pagination%22%3A%7B%7D%2C%22usersSearchTerm%22%3A%22San%20Francisco%2C%20CA%22%2C%22mapBounds%22%3A%7B%22west%22%3A-122.52499667529297%2C%22east%22%3A-122.34166232470703%2C%22south%22%3A37.662044543503555%2C%22north%22%3A37.88836615784793%7D%2C%22regionSelection%22%3A%5B%7B%22regionId%22%3A20330%2C%22regionType%22%3A6%7D%5D%2C%22isMapVisible%22%3Atrue%2C%22filterState%22%3A%7B%22sort%22%3A%7B%22value%22%3A%22days%22%7D%2C%22ah%22%3A%7B%22value%22%3Atrue%7D%7D%2C%22isListVisible%22%3Atrue%2C%22mapZoom%22%3A12%7D"

header = {
  "User-Agent": "YOUR AGENT",
  "Accept-Language": "YOUR LANGUAGEA"
}

response = requests.get(URL, headers=header)

web_page = response.text

soup = BeautifulSoup(web_page, 'lxml')


# Bedrooms and bathrooms
quantity_dirty = soup.find_all("ul", class_="StyledPropertyCardHomeDetailsList-c11n-8-84-3__sc-1xvdaej-0 eYPFID")
quantity_list_clean = [quantity.getText() for quantity in quantity_dirty if not quantity.getText().startswith('--')]
print(quantity_list_clean)

问题根源是直接调用getText()会把<ul>下所有<li>的文本无规则拼接，导致格式混乱。提供两种修改方案：

方案1：基于HTML结构提取（更可靠） #

直接遍历每个<ul>下的<li>子标签，提取单个属性文本后按规则拼接：

import requests
from bs4 import BeautifulSoup
# import gspread

URL = "https://www.zillow.com/san-francisco-ca/?searchQueryState=%7B%22pagination%22%3A%7B%7D%2C%22usersSearchTerm%22%3A%22San%20Francisco%2C%20CA%22%2C%22mapBounds%22%3A%7B%22west%22%3A-122.52499667529297%2C%22east%22%3A-122.34166232470703%2C%22south%22%3A37.662044543503555%2C%22north%22%3A37.88836615784793%7D%2C%22regionSelection%22%3A%5B%7B%22regionId%22%3A20330%2C%22regionType%22%3A6%7D%5D%2C%22isMapVisible%22%3Atrue%2C%22filterState%22%3A%7B%22sort%22%3A%7B%22value%22%3A%22days%22%7D%2C%22ah%22%3A%7B%22value%22%3Atrue%7D%7D%2C%22isListVisible%22%3Atrue%2C%22mapZoom%22%3A12%7D"

header = {
  "User-Agent": "YOUR AGENT",
  "Accept-Language": "YOUR LANGUAGEA"
}

response = requests.get(URL, headers=header)
web_page = response.text
soup = BeautifulSoup(web_page, 'lxml')

# Bedrooms and bathrooms
quantity_dirty = soup.find_all("ul", class_="StyledPropertyCardHomeDetailsList-c11n-8-84-3__sc-1xvdaej-0 eYPFID")
quantity_list_clean = []
for quantity in quantity_dirty:
    text = quantity.getText()
    if text.startswith('--'):
        continue
    # 提取每个li的文本，去掉内部空格后用空格分隔属性
    items = [li.getText().replace(' ', '') for li in quantity.find_all('li')]
    quantity_list_clean.append(' '.join(items))

print(quantity_list_clean)

方案2：正则表达式修正文本（快速适配） #

如果不想改动标签提取逻辑，用正则替换清理现有文本：

import requests
import re
from bs4 import BeautifulSoup
# import gspread

URL = "https://www.zillow.com/san-francisco-ca/?searchQueryState=%7B%22pagination%22%3A%7B%7D%2C%22usersSearchTerm%22%3A%22San%20Francisco%2C%20CA%22%2C%22mapBounds%22%3A%7B%22west%22%3A-122.52499667529297%2C%22east%22%3A-122.34166232470703%2C%22south%22%3A37.662044543503555%2C%22north%22%3A37.88836615784793%7D%2C%22regionSelection%22%3A%5B%7B%22regionId%22%3A20330%2C%22regionType%22%3A6%7D%5D%2C%22isMapVisible%22%3Atrue%2C%22filterState%22%3A%7B%22sort%22%3A%7B%22value%22%3A%22days%22%7D%2C%22ah%22%3A%7B%22value%22%3Atrue%7D%7D%2C%22isListVisible%22%3Atrue%2C%22mapZoom%22%3A12%7D"

header = {
  "User-Agent": "YOUR AGENT",
  "Accept-Language": "YOUR LANGUAGEA"
}

response = requests.get(URL, headers=header)
web_page = response.text
soup = BeautifulSoup(web_page, 'lxml')

# Bedrooms and bathrooms
quantity_dirty = soup.find_all("ul", class_="StyledPropertyCardHomeDetailsList-c11n-8-84-3__sc-1xvdaej-0 eYPFID")
quantity_list_clean = []
# 匹配属性前缀前的空格并替换
pattern = re.compile(r' (bd|ba|sqft)')
for quantity in quantity_dirty:
    text = quantity.getText()
    if text.startswith('--'):
        continue
    cleaned_text = pattern.sub(r'\1', text)
    quantity_list_clean.append(cleaned_text)

print(quantity_list_clean)

方案对比 #

• 方案1基于HTML结构提取，不受文本格式变化影响，稳定性更高；
• 方案2依赖当前文本格式规律，适合快速调整，但如果Zillow页面文本格式更新可能失效。

内容的提问来源于stack exchange，提问作者SolidOpt