如何在Linux中使用jq批量将多JSON文件转换为CSV?
批量转换SAFECAST JSON到CSV及合并文件处理方案
一、批量转换文件夹中所有JSON文件到CSV
在Ubuntu终端执行以下bash循环命令,自动遍历当前文件夹下所有.json文件,转换为同名.csv文件:
for json_file in *.json; do csv_file="${json_file%.json}.csv" jq -r '["id", "user_id", "source url", "status", "submitted", "measurements_count", "approved", "created_at", "updated_at", "name", "description", "lines_count", "credits", "height", "orientation", "cities", "subtype", "rejected"], (.[] | [.id, .user_id, .source.url, .status, .submitted, .measurements_count, .approved, .created_at, .updated_at, .name, .description, .lines_count, .credits, .height, .orientation, .cities, .subtype, .rejected]) | @csv' "$json_file" > "$csv_file" done
二、从TXT文件加载文件名批量处理
如果你的TXT文件(比如json_filenames.txt)中每行存储一个JSON文件名,执行以下命令自动生成对应CSV:
while read -r json_file; do csv_file="${json_file%.json}.csv" jq -r '["id", "user_id", "source url", "status", "submitted", "measurements_count", "approved", "created_at", "updated_at", "name", "description", "lines_count", "credits", "height", "orientation", "cities", "subtype", "rejected"], (.[] | [.id, .user_id, .source.url, .status, .submitted, .measurements_count, .approved, .created_at, .updated_at, .name, .description, .lines_count, .credits, .height, .orientation, .cities, .subtype, .rejected]) | @csv' "$json_file" > "$csv_file" done < json_filenames.txt
如果TXT文件每行是输入JSON+输出CSV的配对(比如bgeigie_imports_001.json bgeigie_001.csv),则用以下命令:
while read -r json_file csv_file; do jq -r '["id", "user_id", "source url", "status", "submitted", "measurements_count", "approved", "created_at", "updated_at", "name", "description", "lines_count", "credits", "height", "orientation", "cities", "subtype", "rejected"], (.[] | [.id, .user_id, .source.url, .status, .submitted, .measurements_count, .approved, .created_at, .updated_at, .name, .description, .lines_count, .credits, .height, .orientation, .cities, .subtype, .rejected]) | @csv' "$json_file" > "$csv_file" done < filename_list.txt
三、合并JSON文件后的转换处理
使用jq -s '.' *.json > All_data.json合并后,原始jq命令无法工作,因为合并后的JSON是数组的数组。需要修改命令增加一层遍历:
jq -r '["id", "user_id", "source url", "status", "submitted", "measurements_count", "approved", "created_at", "updated_at", "name", "description", "lines_count", "credits", "height", "orientation", "cities", "subtype", "rejected"], (.[] | .[] | [.id, .user_id, .source.url, .status, .submitted, .measurements_count, .approved, .created_at, .updated_at, .name, .description, .lines_count, .credits, .height, .orientation, .cities, .subtype, .rejected]) | @csv' All_data.json > All_data.csv
这里的.[] | .[]会先遍历合并后的外层数组(每个原JSON文件的数组),再遍历每个数组内的具体数据项,确保所有内容都被转换。
内容的提问来源于stack exchange,提问作者Juhele
相关产品推荐
相关产品推荐

