使用PyArrow读取ORC/PARQUET文件后，嵌套数组无法正确序列化为JSON的问题求助

各位好，我最近在处理ORC和PARQUET格式的数据转换时遇到了麻烦，想请大家帮忙出出主意。

我现在的流程是用PyArrow读取ORC或PARQUET文件，然后转换成Pandas DataFrame，最终目标是把包含嵌套数组的数据转成标准的JSON格式。下面是我读取数据的代码片段：

if storage_type == "orc":
    data = orc.ORCFile(file_name).read().to_pandas()
elif storage_type == "parq":
    data = parquet.read_table(file_name).to_pandas()

但当我用json.dumps()做序列化时，发现嵌套数组根本没有被正确编码成JSON格式，而是变成了字符串形式。

我想要实现的目标JSON格式是这样的（里面包含contacts、phone、hours等多个嵌套数组）：

{
    "XXXX_code": "XXXX",
    "website": "https://www.randomwebsite.com",
    "brand": "XXXXXX",
    "contacts": [
        {
            "XXXXXXcode": "XXXXXX",
            "phone_number": "XXXXXX5054",
            "firstname": "XXXXXX",
            "code": "01016",
            "lastname": "XXXXXX",
            "title": "XXXXXX New York)",
            "source": "XXXXXX_Apr2016.xlsx",
            "source_date": "2016-04-19",
            "type": "FOM",
            "email": "scott.vanarsdale@XXXXXX"
        },
        {
            "XXXXXXcode": "1031",
            "phone_number": "609-XXXXXX",
            "firstname": "XXXXXX",
            "code": "01016",
            "lastname": "XXXXXX",
            "title": "Sr. DOM XXXXXX",
            "source": "XXXXXX.xlsx",
            "source_date": "2016-04-19",
            "type": "DOM",
            "email": "craig.salera@XXXXXX"
        }
    ],
    "lon": "-70.XXXXXX",
    "phone": [
        {
            "phone_number": "207-XXXXXX",
            "source": "XXXXXXWebServices",
            "type": "XXXXXX",
            "source_date": "2022-12-12",
            "code": "XXXXXX"
        }
    ],
    "email": "",
    "hours": [
        {
            "monday": "",
            "code": "01016",
            "tuesday": "",
            "friday": "",
            "wednesday": "",
            "thursday": "",
            "hours": "Monday-Friday: 09:00-19:00;Saturday: 09:00-18:00;Sunday: Closed",
            "source": "CoreXXXXXX",
            "sunday": "",
            "source_date": "2022-12-12",
            "type": "sales",
            "saturday": ""
        },
        {
            "monday": "",
            "code": "XXXXXX",
            "tuesday": "",
            "friday": "",
            "wednesday": "",
            "thursday": "",
            "hours": "Monday-Thursday: 07:30-18:00;Friday: 07:30-17:30;Saturday: 08:00-16:00;Sunday: Closed",
            "source": "CoreXXXXXX",
            "sunday": "",
            "source_date": "2022-12-12",
            "type": "service",
            "saturday": ""
        }
    ],
    "source": "BIRT"
}

而我目前用这段代码做转换：

res1 = json.dumps([{K:V for K,V in tup} for tup in data['json']])

得到的结果却是这样的，所有嵌套数组都变成了带方括号的字符串，完全不是标准的JSON数组结构：

{
    "hours": "[{saturday=09:00 AM - 07:00 PM}]",
    "website": "https://www.randomwebsite.com",
    "address": "[{zip=XXXXX, code=3903}]",
    "code": "XXXX",
    "lon": "-117.79XXXXX",
    "source": "XXXX",
    "phone": "[{phone_number=714-XXX-XXXX}, {phone_number=714-XXX-XXXX}]",
    "XXX_code": "XXXX",
    "geography": "[{area=LAN}]",
    "name": "TUSTIN",
    "source_contact": "XZZZ",
    "programs": "[{code=12334}]",
    "id": "",
    "employees": "[{firstname=Ramon}]",
    "fax": "714-XXXXXX",
    "brand": "XXXXX",
    "lat": "33.7XXXXXX",
    "email": null,
    "contacts": "[{Firstname=XXXXX]",
    "source_date": "2023-04-29"
}

有没有大佬能指点一下，怎么才能让这些嵌套数组正确序列化为标准的JSON格式呢？

备注：内容来源于stack exchange，提问作者Anuj Sharma