从Wigle导出城市全部WiFi点位仅得100个，求解决方法

我尝试从Wigle导出圣保罗的全部WiFi点位，按照教程操作后只能获取100个点位，修改first参数也无法解决问题。以下是我的代码：

import requests
from requests.auth import HTTPBasicAuth
import pandas as pd
from pandas import json_normalize
import folium
import geopandas

# SETTING WiGLE USERNAME & PASSWORD FOR API CALL:
wigle_username = 'i can share it if you need'
wigle_password = 'i can share it if you need'

city = 'Sao Paulo'
# SETTING PARAMETERS:
payload = {'first': '1', 'freenet': 'false', 'paynet': 'false', 'addresscode': city, 'api_key': (wigle_username + wigle_password).encode()}
# FETCHING JSON RESPONSE FROM WiGLE:
details = requests.get(url='https://api.wigle.net/api/v2/network/geocode', params=payload, auth=HTTPBasicAuth(wigle_username, wigle_password)).json()

# SETTING PARAMETERS:
payload = {'latrange1':'-23.7106507', 'latrange2':'-23.3906507', 'longrange1':'-46.7933824', 'longrange2':'-46.4733824', 'api_key': (wigle_username + wigle_password).encode()}
# FETCHING JSON RESPONSE FROM WiGLE:
results = requests.get(url='https://api.wigle.net/api/v2/network/search', params=payload, auth=HTTPBasicAuth(wigle_username, wigle_password)).json()

# EXTRACTING 'RESULTS' AS A PANDAS DATAFRAME TO WORK WITH:
df = json_normalize(results['results'])
# RENAMING COLUMNS FOR GEOPLOTLIB:
df = df.rename(columns={'trilat': 'lat', 'trilong': 'lon'})
cols = list(df.columns)
# PREVIEWING AVAILABLE INFORMATION:
print(f"Result obtained has {df.shape[0]} rows and {df.shape[1]} columns in it. \n\nThe list of columns include {cols}")

gdf = geopandas.GeoDataFrame(
    df, geometry=geopandas.points_from_xy(df.lon, df.lat), crs="EPSG:4326"
)

interactive_map1 = folium.Map(
    location=(-23.568399795078903, -46.6474148638533),
    zoom_start=10
)

addresses_layer = folium.features.GeoJson(gdf)
addresses_layer.add_to(interactive_map1)
interactive_map1

Wigle的network/search接口默认每页返回100条数据，且强制分页机制，单靠修改first参数无法获取全部数据，需要通过循环遍历所有分页来实现：

1. 核心逻辑说明 #

• 接口返回结果包含totalResults（总点位数量）和last（当前页最后一条数据的索引）
• 每次请求将first参数设为上一页的last + 1，直到获取结果为空，或已获取条数等于totalResults
• 必须添加请求延迟，避免触发API限流

2. 修改后的完整代码 #

import requests
from requests.auth import HTTPBasicAuth
import pandas as pd
from pandas import json_normalize
import folium
import geopandas
import time

# 配置Wigle账户信息
wigle_username = '你的Wigle用户名'
wigle_password = '你的Wigle密码'

# 搜索范围（圣保罗）
payload_base = {
    'latrange1':'-23.7106507', 
    'latrange2':'-23.3906507', 
    'longrange1':'-46.7933824', 
    'longrange2':'-46.4733824',
    'freenet': 'false',
    'paynet': 'false'
}

all_results = []
first = 1
total = None

while True:
    # 更新分页参数
    payload = payload_base.copy()
    payload['first'] = str(first)
    
    # 发送请求
    try:
        response = requests.get(
            url='https://api.wigle.net/api/v2/network/search',
            params=payload,
            auth=HTTPBasicAuth(wigle_username, wigle_password)
        )
        response.raise_for_status()
        results = response.json()
    except Exception as e:
        print(f"请求出错: {e}")
        break
    
    # 提取当前页结果
    current_page = results.get('results', [])
    if not current_page:
        break
    
    all_results.extend(current_page)
    
    # 更新分页信息
    if total is None:
        total = results.get('totalResults', len(all_results))
    
    last = results.get('last', first)
    first = last + 1
    
    # 打印进度
    print(f"已获取 {len(all_results)}/{total} 个点位")
    
    # 添加延迟，避免限流
    time.sleep(1)

# 转换为DataFrame
df = json_normalize(all_results)
df = df.rename(columns={'trilat': 'lat', 'trilong': 'lon'})
print(f"最终获取 {df.shape[0]} 条数据，共 {df.shape[1]} 列")

# 生成地图
gdf = geopandas.GeoDataFrame(
    df, geometry=geopandas.points_from_xy(df.lon, df.lat), crs="EPSG:4326"
)

interactive_map1 = folium.Map(
    location=(-23.568399795078903, -46.6474148638533),
    zoom_start=10
)

addresses_layer = folium.features.GeoJson(gdf)
addresses_layer.add_to(interactive_map1)
interactive_map1

3. 注意事项 #

• API权限限制：Wigle免费用户有请求次数和数据量限制，若无法获取全部数据，需检查账户权限或升级付费账户
• 限流处理：必须添加请求延迟，否则可能被临时封禁API访问权限
• 错误处理：代码中加入了基础异常捕获，可根据需求扩展更完善的错误处理逻辑

内容的提问来源于stack exchange，提问作者barney stinson 2