BeautifulSoup爬取律师数据：地址清洗与姓名地址匹配

问题对应解决逻辑 #

• 地址冗余字符清洗：对提取到的地址文本分段处理，先用strip()移除每段首尾的换行、制表符、多余空格，再通过正则将文本中间连续的空白字符（多空格、制表符）替换为单个空格，过滤空内容后拼接，即可得到规整的地址文本。
• 姓名地址错位问题：放弃原有“先全页提取所有姓名、再全页提取所有地址”的逻辑，改为定位每一个律师信息的外层卡片容器，在单个容器内同步提取对应律师的姓名、地址，从根源上避免两类数据匹配错位。

修正后完整代码 #

import requests
from bs4 import BeautifulSoup
import re

url = "https://www.justia.com/lawyers/criminal-law/maine"
# 补充请求头避免被站点反爬拦截
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, "html.parser")

# 遍历所有律师信息卡片
lawyer_cards = soup.find_all("div", class_="lawyer-card")
for card in lawyer_cards:
    # 提取当前卡片对应律师姓名
    name_tag = card.find("a", class_="url main-profile-link")
    if not name_tag:
        continue
    lawyer_name = name_tag.get_text(strip=True)

    # 提取并清洗地址信息
    address_tag = card.find("span", class_="-address -hide-landscape-tablet")
    if not address_tag:
        clean_address = "未公开办公地址"
    else:
        address_parts = []
        for text_seg in address_tag.find_all(text=True):
            seg = text_seg.strip()
            if seg:
                # 替换段内连续空白为单个空格
                seg = re.sub(r"\s+", " ", seg)
                address_parts.append(seg)
        clean_address = "\n".join(address_parts)
    
    # 按指定格式输出
    print(lawyer_name)
    print(clean_address)
    print("-"*20)

输出效果 #

代码运行后单条律师信息格式完全符合预期，首位律师输出示例：

Hunter J Tzovarras
88 Hammond Street
Bangor, ME 04401

内容的提问来源于stack exchange，提问作者OrbitDuster