如何从多份CSV文件创建Intake Catalog？求最佳实践

我需要为每份CSV文件创建独立的source，以此生成Intake Catalog。单个CSV的场景已经实现：

import intake
source1 = intake.open_csv('states_1.csv')
source1.name = 'states1'
with open('catalog.yml', 'w') as f:
    f.write(str(source1.yaml()))

生成的有效YAML内容：

sources:
  states1:
    args:
      urlpath: states_1.csv
    description: ''
    driver: intake.source.csv.CSVSource
    metadata: {}

但添加第二个CSV时，直接拼接两个source的yaml会导致重复的sources条目，YAML解析失败：

import intake
source1 = intake.open_csv('states_1.csv')
source1.name = 'states1'
source2 = intake.open_csv('states_2.csv')
source2.name = 'states2'
with open('catalog.yml', 'w') as f:
    f.write(str(source1.yaml()))
    f.write(str(source2.yaml()))

生成的无效YAML：

sources:
  states1:
    args:
      urlpath: states_1.csv
    description: ''
    driver: intake.source.csv.CSVSource
    metadata: {}
sources:
  states2:
    args:
      urlpath: states_2.csv
    description: ''
    driver: intake.source.csv.CSVSource
    metadata: {}

方法一：使用Intake Catalog对象管理（推荐） #

实例化intake.Catalog对象，将多个source添加进去后再导出为YAML，这是Intake官方推荐的标准方式。

import intake
from intake.catalog import Catalog

# 创建空Catalog实例
catalog = Catalog()

# 逐个添加CSV source
source1 = intake.open_csv('states_1.csv')
catalog.add(source1, name='states1')

source2 = intake.open_csv('states_2.csv')
catalog.add(source2, name='states2')

# 将Catalog写入YAML文件
with open('catalog.yml', 'w') as f:
    f.write(catalog.yaml())

生成的有效YAML内容：

sources:
  states1:
    args:
      urlpath: states_1.csv
    description: ''
    driver: intake.source.csv.CSVSource
    metadata: {}
  states2:
    args:
      urlpath: states_2.csv
    description: ''
    driver: intake.source.csv.CSVSource
    metadata: {}

方法二：手动构建YAML字典 #

如果不想依赖Catalog对象，也可以手动构建包含所有source的字典，再通过YAML库导出：

import intake
import yaml

# 初始化sources字典
sources = {}

# 解析单个source的yaml并添加到字典
source1 = intake.open_csv('states_1.csv')
sources['states1'] = yaml.safe_load(source1.yaml())['sources']['states1']

source2 = intake.open_csv('states_2.csv')
sources['states2'] = yaml.safe_load(source2.yaml())['sources']['states2']

# 生成完整的Catalog结构并写入文件
catalog_dict = {'sources': sources}
with open('catalog.yml', 'w') as f:
    yaml.dump(catalog_dict, f, sort_keys=False)

此方法同样能生成结构合规的YAML文件。

内容的提问来源于stack exchange，提问作者Rich Signell