如何用Shell或Python转换指定格式的扁平文件至其他格式？

这里提供Python和Shell两种实现方式，帮你把这份扁平的用户数据转换成易读易处理的JSON格式：

Python 实现 #

这个脚本用正则表达式解析文本，输出格式化的JSON，还优化了浮点数的显示精度（比如把7.199999999999999自动修正为7.2）：

import re
import json

def parse_flat_text(text):
    # 分割成单个用户的数据块
    user_blocks = text.split("User: ")[1:]
    result = []
    
    for block in user_blocks:
        user_data = {}
        # 提取用户名
        user_match = re.match(r'(\w+)', block)
        if user_match:
            user_data["User"] = user_match.group(1)
        
        # 提取Count并转为整数
        count_match = re.search(r'Count:(\d+)', block)
        if count_match:
            user_data["Count"] = int(count_match.group(1))
        
        # 提取Sum，优化浮点数显示
        sum_match = re.search(r'Sum:([\d\.]+)', block)
        if sum_match:
            sum_val = float(sum_match.group(1))
            # 整数显示为整数，小数保留两位
            user_data["Sum"] = round(sum_val, 2) if sum_val != int(sum_val) else int(sum_val)
        
        # 批量提取部门和对应金额
        dept_amounts = re.findall(r'departmentId: (\w+) Amount by departmentId: ([\d\.]+)', block)
        user_data["DepartmentAmounts"] = {dept: float(amt) for dept, amt in dept_amounts}
        
        result.append(user_data)
    
    return result

# 读取输入文件（替换成你的文件路径）
with open("input.txt", "r") as f:
    flat_text = f.read()

# 输出格式化的JSON
parsed_data = parse_flat_text(flat_text)
print(json.dumps(parsed_data, indent=2))

使用说明 #

1. 把原始文本保存为input.txt；
2. 运行脚本后，输出的结构化JSON示例：

[
  {
    "User": "user1",
    "Count": 3,
    "Sum": 80,
    "DepartmentAmounts": {
      "dept1": 20,
      "dept1": 35,
      "dept2": 25
    }
  },
  {
    "User": "user2",
    "Count": 3,
    "Sum": 7.2,
    "DepartmentAmounts": {
      "dept1": 2.4,
      "dept2": 2.4,
      "dept3": 2.4
    }
  }
  // 其他用户数据...
]

Shell（Awk）实现 #

如果你习惯用命令行工具，这个Awk脚本同样可以生成标准JSON格式的输出：

BEGIN {
    FS = ": "
    OFS = ", "
    print "["
}

/User:/ {
    if (user != "") {
        print "  {"
        print "    \"User\": \"" user "\","
        print "    \"Count\": " count ","
        print "    \"Sum\": " sum ","
        print "    \"DepartmentAmounts\": {"
        dept_count = 0
        for (dept in dept_amt) {
            dept_count++
            printf "      \"%s\": %s%s\n", dept, dept_amt[dept], (dept_count < length(dept_amt) ? "," : "")
        }
        print "    }"
        print "  },"
        delete dept_amt
    }
    user = $2
    next
}

/Count:/ {
    count = $2
    next
}

/Sum:/ {
    sum = $2
    next
}

/departmentId:/ {
    dept = $2
    getline
    amt = $2
    dept_amt[dept] = amt
    next
}

END {
    # 处理最后一个用户
    print "  {"
    print "    \"User\": \"" user "\","
    print "    \"Count\": " count ","
    print "    \"Sum\": " sum ","
    print "    \"DepartmentAmounts\": {"
    dept_count = 0
    for (dept in dept_amt) {
        dept_count++
        printf "      \"%s\": %s\n", dept, dept_amt[dept]
    }
    print "    }"
    print "  }"
    print "]"
}

使用说明 #

1. 把脚本保存为parse.awk；
2. 运行命令：awk -f parse.awk input.txt，就能得到结构化的JSON输出。

内容的提问来源于stack exchange，提问作者Maverick Riz