AWS Glue读取JSON时trace数组格式异常,如何还原原始格式?
AWS Glue读取JSON时
trace数组格式被转换的问题 我获取了一份ADS-B数据的JSON文件,上传至S3后使用AWS Glue读取,遇到以下问题:
原始JSON内容
{ "icao": "f3b100", "noRegData": true, "timestamp": 1690848000, "trace": [ [ 51213.77, 39.090519, -85.870361, 4500, 95, 359.4, 5, 0, { "type": "tisb_icao", "track": 359.4, "geom_rate": 0, "nic": 0, "rc": 0, "nac_p": 0, "nac_v": 0, "sil": 0, "sil_type": "unknown", "alert": 0, "spi": 0 }, "tisb_icao", null, 0, null, null, "f9adebf3" ] ] }
AWS Glue解析后的trace格式
[ [ { "double": 51213.77, "int": null, "string": null, "struct": null }, { "double": 39.090519, "int": null, "string": null, "struct": null }, { "double": -85.870361, "int": null, "string": null, "struct": null }, { "double": null, "int": 4500, "string": null, "struct": null }, { "double": 95, "int": null, "string": null, "struct": null }, { "double": 359.4, "int": null, "string": null, "struct": null }, { "double": null, "int": 5, "string": null, "struct": null }, { "double": null, "int": 0, "string": null, "struct": null }, { "double": null, "int": null, "string": null, "struct": { "type": "tisb_icao", "alt_geom": null, "track": 359.4, "geom_rate": 0, "nic": 0, "rc": 0, "nac_p": 0, "nac_v": 0, "sil": 0, "sil_type": "unknown", "alert": 0, "spi": 0 } }, { "double": null, "int": null, "string": "tisb_icao", "struct": null }, { "double": null, "int": null, "string": null, "struct": null }, { "double": null, "int": 0, "string": null, "struct": null }, { "double": null, "int": null, "string": null, "struct": null }, { "double": null, "int": null, "string": null, "struct": null }, { "double": null, "int": null, "string": "f9adebf3", "struct": null } ] ]
解析对应的Schema
array<array<struct<double:double,int:int,string:string,struct:struct<type:string,track:double,geom_rate:int,nic:int,rc:int,nac_p:int,nac_v:int,sil:int,sil_type:string,alert:int,spi:int>>>>
疑问
- 这种将原始混合类型数组转换为带类型标记结构体的行为,是AWS Glue的预期行为吗?
- 如何配置AWS Glue,才能正确读取并保留原始
trace数组的格式?
内容的提问来源于stack exchange,提问作者Hongbo Miao
相关产品推荐
相关产品推荐

