如何在GET请求的API调用中批量传递多个IMO编号？

问题背景 #

我有一组IMO编号列表：

unique_IMO = [94229,95986,96967,94731,95731,96612]

需要一次性将这些编号传入以下GET请求的API接口：

url = 'https://api.lloydslistintelligence.com/v1/aispositionhistory?output=json&vesselImo={0}&pageNumber={1}'.format(unique_IMO,1)

目前通过for循环逐个调用接口，但希望改成批量传递参数以提升调用效率。尝试过test1 = format(','.join(map(str,unique_IMO)))但报错，已通过Postman确认该接口支持多值传递。

当前可行的循环调用代码如下（vessels是存储IMO编号的DataFrame，date_list是预先生成的时间范围列表）：

df_list = []
for ind,row in vessels.iterrows():   
    vesselImo = int(row['Imo'])

    #Retrieve data from aispositionhistory endpoint          

    vessel_hist = pd.DataFrame()
    total_recs = 0

    for date_string in date_list:
        url = 'https://api.lloydslistintelligence.com/v1/aispositionhistory?output=json&vesselImo={0}&dateRange={1}&pageNumber={2}'.format(vesselImo,date_string,1)
        head = {'Authorization': '{}'.format(api_token)}
        response = requests.get(url, headers=head)

        #****DEBUGGING****
        #print("status code: ", response.status_code )

        if(response.json()['Data']['totalRecords'] != 0):
            tmp = response.json()['Data']['items']
            df = json_normalize(tmp)
            vessel_hist = vessel_hist.append(df,ignore_index=True)


        #Get reported number of records for validation
        total_recs = total_recs + response.json()['Data']['totalRecords']

        #Identify if API response is multiple pages
        if(response.json()['Data']['totalPages'] > 1):
            num_pages = response.json()['Data']['totalPages']
            #print('API pull had more than one page: ' + date_string)

            for page_no in range(2,num_pages+1):
                url = 'https://api.lloydslistintelligence.com/v1/aispositionhistory?output=json&vesselImo={0}&dateRange={1}&pageNumber={2}'.format(vesselImo,date_string,1)
                response = requests.get(url, headers=head)
                tmp = response.json()['Data']['items']
                df = json_normalize(tmp)
                vessel_hist = vessel_hist.append(df,ignore_index=True)



    # Validation based on record count
    if(total_recs != vessel_hist.shape[0]):
        print('Validation Error: reported records do not match dataframe')

    if(vessel_hist.shape[0]>0):
        #Format Dataframe       
        new_columns = ['vesselId','MMSI','PositionTimestamp','Latitude','Longitude','Speed','Course','Rot','Heading',
               'nearestPlace','nearestPlaceId','nearestCountry','Distance','Destination','Eta','Draught',
               'Dimensions','Status','Ship_type','Source']

        vessel_hist.columns = new_columns

        vessel_hist = vessel_hist[['MMSI','PositionTimestamp','Status','Latitude','Longitude','Speed','Course','Rot',
                                  'Heading','Draught','Destination','Eta','Source','Ship_type','Dimensions',
                                   'Distance','nearestCountry','nearestPlace','nearestPlaceId','vesselId']]

        vessel_hist['PositionTimestamp'] = pd.to_datetime(vessel_hist['PositionTimestamp'],dayfirst=False)
        vessel_hist.sort_values('PositionTimestamp', inplace=True)
        vessel_hist.reset_index(drop=True, inplace=True)

        df_list.append(vessel_hist)

    print('Input vessel Id: ' + str(vesselImo))
    print('Input Date Range: ' + start_input + ' - ' + end_input)        
    print('No. of AIS records: ' + str(vessel_hist.shape[0]))

df_list

其中存储IMO的DataFrame定义：

vessels = pd.DataFrame((94229,95986,96967,94731,95731,96612),columns=['Imo'])

优化方案 #

核心思路 #

既然接口支持多值传递，直接将IMO编号拼接成逗号分隔的字符串传入vesselImo参数，去掉循环单个IMO的外层逻辑，同时保留日期范围、分页处理和数据格式化的原有逻辑。

优化后代码 #

import requests
import pandas as pd
from pandas import json_normalize

# 配置参数
unique_IMO = [94229,95986,96967,94731,95731,96612]
api_token = "你的API令牌"
# 假设date_list已提前生成（示例格式：["2024-01-01,2024-01-31", ...]）
# date_list = []
start_input = "起始日期"
end_input = "结束日期"

df_list = []
# 拼接IMO编号为逗号分隔的字符串
vessel_imo_str = ','.join(map(str, unique_IMO))
vessel_hist = pd.DataFrame()
total_recs = 0

for date_string in date_list:
    # 批量传递IMO的请求URL
    url = 'https://api.lloydslistintelligence.com/v1/aispositionhistory?output=json&vesselImo={0}&dateRange={1}&pageNumber={2}'.format(vessel_imo_str, date_string, 1)
    head = {'Authorization': '{}'.format(api_token)}
    response = requests.get(url, headers=head)
    
    # 检查请求是否成功
    if response.status_code != 200:
        print(f"日期段{date_string}请求失败，状态码: {response.status_code}")
        continue
    
    data = response.json()['Data']
    # 处理当前页数据
    if data['totalRecords'] != 0:
        tmp = data['items']
        df = json_normalize(tmp)
        vessel_hist = pd.concat([vessel_hist, df], ignore_index=True)
    
    total_recs += data['totalRecords']
    
    # 处理多页数据（修复原代码分页pageNumber固定为1的bug）
    if data['totalPages'] > 1:
        num_pages = data['totalPages']
        for page_no in range(2, num_pages + 1):
            url = 'https://api.lloydslistintelligence.com/v1/aispositionhistory?output=json&vesselImo={0}&dateRange={1}&pageNumber={2}'.format(vessel_imo_str, date_string, page_no)
            response = requests.get(url, headers=head)
            if response.status_code != 200:
                print(f"日期段{date_string}分页{page_no}请求失败，状态码: {response.status_code}")
                continue
            page_data = response.json()['Data']
            tmp = page_data['items']
            df = json_normalize(tmp)
            vessel_hist = pd.concat([vessel_hist, df], ignore_index=True)

# 数据校验
if total_recs != vessel_hist.shape[0]:
    print('Validation Error: reported records do not match dataframe')

# 格式化数据并收集结果
if vessel_hist.shape[0] > 0:
    new_columns = ['vesselId','MMSI','PositionTimestamp','Latitude','Longitude','Speed','Course','Rot','Heading',
                   'nearestPlace','nearestPlaceId','nearestCountry','Distance','Destination','Eta','Draught',
                   'Dimensions','Status','Ship_type','Source']
    vessel_hist.columns = new_columns
    
    vessel_hist = vessel_hist[['MMSI','PositionTimestamp','Status','Latitude','Longitude','Speed','Course','Rot',
                              'Heading','Draught','Destination','Eta','Source','Ship_type','Dimensions',
                               'Distance','nearestCountry','nearestPlace','nearestPlaceId','vesselId']]
    
    vessel_hist['PositionTimestamp'] = pd.to_datetime(vessel_hist['PositionTimestamp'], dayfirst=False)
    vessel_hist.sort_values('PositionTimestamp', inplace=True)
    vessel_hist.reset_index(drop=True, inplace=True)
    
    df_list.append(vessel_hist)

# 输出统计信息
print('Input vessel Ids: ' + vessel_imo_str)
print('Input Date Range: ' + start_input + ' - ' + end_input)        
print('No. of AIS records: ' + str(vessel_hist.shape[0]))

df_list

关键优化点 #

1. 批量参数传递：用','.join(map(str, unique_IMO))将整数列表转换为逗号分隔的字符串，直接传入vesselImo参数，避免多次接口调用
2. 修复分页bug：原代码分页时pageNumber固定为1，优化后动态传入当前页码page_no
3. 提升数据拼接效率：用pd.concat替代DataFrame.append（后者已被Pandas标记为过时）
4. 增加错误处理：添加响应状态码检查，避免请求失败时后续逻辑报错

内容的提问来源于stack exchange，提问作者Hrithu