如何从Refinitiv Workspace提取25万+非绿色债券数据？

问题 #

需要提取25万+条非绿色债券记录，但rdp.search()单次调用最多返回10000行，现有代码无法获取完整数据集。

现有代码：

import refinitiv.dataplatform as rdp
import refinitiv.dataplatform.eikon as ek

ek.set_app_key('mykey')
df = rdp.search(
    view=rdp.SearchViews.FixedIncomeInstruments,
    filter="IsGreenBond eq false",
    select='Name, DocumentTitle, RIC, ISIN, AssetTypeDescription, MaturityDate, FaceOutstandingUSD, TrancheAmount, CouponClass, Currency, IssueDate, SeniorityTypeDescription, MaturityCorpModDuration, MaturityCorpYield, Price, SectorDescription',
    top=20000,
)

print(df)

解决方法：分页循环查询 #

Refinitiv Search API支持通过*skip*参数实现分页，每次查询一批数据（上限10000条），循环累加直到获取全部记录。

完整代码 #

import refinitiv.dataplatform as rdp
import refinitiv.dataplatform.eikon as ek
import pandas as pd
import time

ek.set_app_key('mykey')

# 查询配置
VIEW = rdp.SearchViews.FixedIncomeInstruments
FILTER = "IsGreenBond eq false"
SELECT_FIELDS = ('Name, DocumentTitle, RIC, ISIN, AssetTypeDescription, MaturityDate,'
                 'FaceOutstandingUSD, TrancheAmount, CouponClass, Currency, IssueDate,'
                 'SeniorityTypeDescription, MaturityCorpModDuration, MaturityCorpYield,'
                 'Price, SectorDescription')
BATCH_SIZE = 10000  # API单次返回上限
all_records = []

skip = 0
while True:
    # 执行分页查询
    batch_df = rdp.search(
        view=VIEW,
        filter=FILTER,
        select=SELECT_FIELDS,
        top=BATCH_SIZE,
        skip=skip
    )
    
    # 无数据返回时终止循环
    if batch_df.empty:
        break
    
    all_records.append(batch_df)
    skip += BATCH_SIZE
    print(f"已累计获取 {skip} 条记录")
    time.sleep(0.5)  # 添加短延迟避免触发API限流

# 合并所有批次数据
final_df = pd.concat(all_records, ignore_index=True)
print(f"提取完成，总记录数：{len(final_df)}")
# 保存为CSV文件
final_df.to_csv('non_green_bonds_full.csv', index=False)

关键注意点 #

• API限流：添加time.sleep()控制请求频率，避免触发平台的调用限制。如果遇到429错误，增加延迟时间。
• 内存管理：25万条数据在内存中处理无压力，若字段更多或数据量更大，可考虑每批数据直接写入文件，减少内存占用。
• 参数正确性：确保filter条件IsGreenBond eq false准确，避免筛选错误；select字段按需调整，减少不必要的数据传输。

内容的提问来源于stack exchange，提问作者Matteo De Leonardis