ruamel.yaml加载嵌套自定义Python对象时数组字段解析为空的问题

在嵌套结构中从YAML配置实例化Python对象数组时，非嵌套场景可正常执行，但YAML加载器无法递归遍历自定义对象的子节点，无论传入何种类型的数组，自定义对象from_yaml构造逻辑中读取到的数组字段始终为空。

问题复现代码 #

import yaml
import ruamel.yaml

class Person:
    def __init__(self, name: str = 'JohnDoe'):
        self.name = name

    @classmethod
    def from_yaml(cls, constructor, node):
        for m in constructor.construct_yaml_map(node):
            pass
        if 'Name' in m:
            name = m['Name']

        return cls(name=name)

    def __repr__(self):
        return f'Person(name={self.name})'


class Car:
    def __init__(self):
        self.passengers = []

    def add_person(self, person: Person = None):
        self.passengers.append(person)

    @classmethod
    def from_yaml(cls, constructor, node):
        for m in constructor.construct_yaml_map(node):
            pass

        inst = cls()

        if 'Driver' in m:
            inst.passengers = [m['Driver']]+inst.passengers

        if 'Passengers' in m:
            foo = m['Passengers']
            print(f'm["Passengers"] = {foo}')
            for person in m['Passengers']:
                inst.add_person(person)

        return inst

    def __repr__(self):
        return f'Car(passengers={self.passengers})'


if __name__ == "__main__":

    yaml = ruamel.yaml.YAML(typ='safe')
    yaml.register_class(Person)
    yaml.register_class(Car)

    data = yaml.load("""
        - !Person &0
            Name: 'Paul'

        - !Person &1
            Name: 'George'
            
        - !Person &3
            Name: 'John'

        - !Car
            Driver: *0
            Passengers: [*1]

        - !Car
            Driver: *3
            Passengers: 
                - !Person &4
                    Name: 'Ringo'

        """)

    print(f'data = {data}')

实际运行输出 #

m['Passengers'] = []
m['Passengers'] = []
data = [Person(name=Paul), Person(name=George), Person(name=John), Car(passengers=[Person(name=Paul)]), Car(passengers=[Person(name=John)])]

预期输出 #

m['Passengers'] = [Person(name=George)]
m['Passengers'] = [Person(name=Ringo)]
data = [Person(name=Paul), Person(name=George), Person(name=John), Car(passengers=[Person(name=Paul), Person(name=George)]), Car(passengers=[Person(name=John), Person(name=Ringo)])]

核心疑问 #

是否需要在from_yaml方法中手动指定构造器优先遍历剩余子节点，还是YAML加载器本身采用自底向上的递归加载逻辑？

ruamel.yaml的构造器默认采用两阶段构造逻辑处理映射、序列等集合类型：

• 第一阶段先创建集合的空实例（空字典、空列表），立即返回给上层调用方，用于解决循环引用、锚点引用的问题
• 第二阶段再递归遍历集合内的子节点，逐步填充集合内容

现有from_yaml实现中，遍历完construct_yaml_map返回的生成器后，立刻读取Passengers字段的值，此时第二阶段的子节点构造还未执行，因此拿到的是空列表。

在读取映射内的嵌套字段前，显式调用构造器的深度构造方法，递归生成所有子节点后再读取值，修改Car类的from_yaml方法如下：

@classmethod
def from_yaml(cls, constructor, node):
    # 递归构造当前节点下所有子节点，生成完整的映射对象
    m = constructor.construct_mapping(node, deep=True)
    inst = cls()

    if 'Driver' in m:
        inst.passengers = [m['Driver']] + inst.passengers

    if 'Passengers' in m:
        foo = m['Passengers']
        print(f'm["Passengers"] = {foo}')
        for person in m['Passengers']:
            inst.add_person(person)

    return inst

Person类的from_yaml方法也可以同步简化为同样的写法，去掉迭代生成器的冗余逻辑：

@classmethod
def from_yaml(cls, constructor, node):
    m = constructor.construct_mapping(node, deep=True)
    name = m.get('Name', 'JohnDoe')
    return cls(name=name)

修改后运行代码即可得到预期输出。

内容的提问来源于stack exchange，提问作者Kevan Pigott