如何让Python识别Google Maps单结果页面并执行对应逻辑？

Google Maps在搜索结果仅1条时，会直接跳转至该地点的详情页，而非展示结果列表，因此需要先判断当前页面类型，再执行对应的抓取逻辑。以下是具体解决方案：

核心思路 #

通过检查页面元素区分两种场景：

• 结果列表页：存在div.m6QErb DxyBCb kA9KIf dS8AEf ecceSd容器，内部包含多个div.lI9IFe结果项
• 单结果详情页：无上述列表容器，直接显示地点详情，名称通常在h1.fontHeadlineLarge元素，评分在span.MW4etd元素

修改后的完整代码 #

from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
import csv
import random
import os

print(os.getcwd())

# 初始化CSV表头（仅文件不存在时写入）
csv_path = 'exported_data.csv'
if not os.path.exists(csv_path):
    with open(csv_path, 'w', newline='', encoding='utf-8') as f:
        csv_writer = csv.writer(f)
        csv_writer.writerow(['Company Name', 'Rating', 'Searched Term'])

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    with open('terms_to_search.csv') as csv_file:
        csv_reader = csv.reader(csv_file)
        for row in csv_reader:
            term_searched = row[0].strip()
            if not term_searched:
                continue

            page = browser.new_page()
            try:
                page.goto('https://www.google.com/maps/')
                # 等待搜索框加载并输入关键词
                page.wait_for_selector('//*[@id="searchboxinput"]', timeout=10000)
                page.fill('//*[@id="searchboxinput"]', term_searched)
                # 点击搜索按钮
                page.click('//*[@id="searchbox-searchbutton"]')
                # 等待页面加载完成
                page.wait_for_load_state('networkidle')

                # 仅在列表页执行滚动加载
                count = 0
                while count < 50:
                    if page.locator('div.m6QErb.DxyBCb.kA9KIf.dS8AEf.ecceSd').count() > 0:
                        page.mouse.wheel(0, 3000)
                        page.wait_for_timeout(random.randint(1000, 2000))
                        count += 1
                    else:
                        break

                # 解析页面HTML
                html = page.inner_html('html')
                soup = BeautifulSoup(html, 'html.parser')

                # 处理多结果列表场景
                results_container = soup.find('div', class_='m6QErb DxyBCb kA9KIf dS8AEf ecceSd')
                if results_container:
                    results = results_container.find_all('div', class_='lI9IFe')
                    for result in results:
                        name = result.find('div', class_='qBF1Pd fontHeadlineSmall')
                        rating = result.find('span', class_='MW4etd')
                        with open(csv_path, 'a+', encoding='utf-8', newline='') as f:
                            csv_writer = csv.writer(f)
                            csv_writer.writerow([
                                name.text.strip() if name else '',
                                rating.text.strip() if rating else '',
                                term_searched
                            ])
                else:
                    # 处理单结果详情页场景
                    name = soup.find('h1', class_='fontHeadlineLarge')
                    rating = soup.find('span', class_='MW4etd')
                    with open(csv_path, 'a+', encoding='utf-8', newline='') as f:
                        csv_writer = csv.writer(f)
                        csv_writer.writerow([
                            name.text.strip() if name else '',
                            rating.text.strip() if rating else '',
                            term_searched
                        ])

            except Exception as e:
                print(f"处理关键词 {term_searched} 时出错: {str(e)}")
            finally:
                page.close()

关键优化点 #

1. 页面类型判断：通过results_container是否存在自动切换抓取逻辑
2. 滚动逻辑优化：仅在列表页执行滚动加载，避免无效操作
3. 稳定性提升：用wait_for_selector和wait_for_load_state替代固定time.sleep，适配不同网络环境
4. 异常处理：捕获具体异常并打印，便于排查问题
5. CSV写入优化：仅在文件初始化时写入表头，避免重复写入

内容的提问来源于stack exchange，提问作者Vinícius