基于JSON规范解析文本文件时age字段缺失的问题排查

我正在使用Spring Boot的FileController，基于JSON规范解析定长文本文件。JSON规范定义了字段名、startIndex和length，每个文本文件对应不同的JSON规范。当前JSON规范包含firstName（startIndex:0,length:10）、lastName（startIndex:10,length:10）、age（startIndex:17,length:20），待解析文本文件有对应数据，但Postman返回结果缺失age字段。以下是相关代码、规范、文本内容及返回结果，请排查解析逻辑是否存在错误？

控制器代码 #

package com.example.justin_fulkerson_project.controllers;

import com.example.justin_fulkerson_project.config.FixedLengthFileConfiguration;
import com.example.justin_fulkerson_project.entities.FileDataEntity;
import com.example.justin_fulkerson_project.respositories.FileDataRepository;
import com.example.justin_fulkerson_project.services.FileParserService;
import com.example.justin_fulkerson_project.services.MetadataService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@RestController
@RequestMapping("/api/files")
public class FileController {

    @Autowired
    private FileParserService fileParserService;

    @Autowired
    private FileDataRepository fileDataRepository;

    @Autowired
    private FixedLengthFileConfiguration configuration;

    @Autowired
    private MetadataService metadataService;

    @PostMapping("/metadata")
    public ResponseEntity<String> uploadMetadata(@RequestParam("specFile") MultipartFile specFile,
                                                 @RequestParam("pathName") String pathName) {
        try {
            String jsonSpecFile = new String(specFile.getBytes(), StandardCharsets.UTF_8);
            metadataService.parseAndSaveMetadata(jsonSpecFile, pathName);
            return ResponseEntity.ok().body("Metadata uploaded successfully.");
        } catch (IOException e) {
            return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body("Error uploading metadata.");
        }
    }

    private void saveParsedData(List<Map<String, String>> parsedData) {
        for (Map<String, String> record : parsedData) {
            FileDataEntity mapDataEntity = new FileDataEntity();
            mapDataEntity.setData(record);
            fileDataRepository.save(mapDataEntity);
        }
    }

    @PostMapping("/parse")
    public List<Map<String, String>> parseFixedLengthFile(@RequestParam("file") MultipartFile file,
                                                          @RequestParam("spec") MultipartFile specFile) throws IOException {
        List<Map<String, String>> parsedData = new ArrayList<>();
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(file.getInputStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                parsedData.add(parseLine(line, specFile.getInputStream()));
            }
        }
        saveParsedData(parsedData);
        return parsedData;

    }

    private Map<String, String> parseLine(String line, InputStream inputStream) {
        Map<String, FixedLengthFileConfiguration.FixedLengthField> fields = configuration.getFields();
        if (fields == null) {
            throw new IllegalStateException("Fields map is null");
        }

        Map<String, String> parsedData = new HashMap<>();
        for(Map.Entry<String, FixedLengthFileConfiguration.FixedLengthField> entry : fields.entrySet()) {
            String fieldName = entry.getKey();
            FixedLengthFileConfiguration.FixedLengthField field = entry.getValue();
            int startIndex = field.getStart();
            int length = field.getLength();

            if (line.length() >= startIndex + length) {
                String fieldValue = line.substring(startIndex, startIndex + length).trim();
                parsedData.put(fieldName, fieldValue);
            }
        }
        return parsedData;
    }
}

JSON规范 #

{
  "fields": [
    { "name": "firstName", "startIndex": 0, "length": 10 },
    { "name": "lastName", "startIndex": 10, "length": 10 },
    { "name": "age", "startIndex": 17, "length": 20 }
  ]
}

待解析文本文件 #

John      Doe       25
Jane      Smith     30
Alice     Johnson   22

Postman返回结果 #

[
    {
        "firstName": "John",
        "lastName": "Doe"
    },
    {
        "firstName": "Jane",
        "lastName": "Smith"
    },
    {
        "firstName": "Alice",
        "lastName": "Johnson"
    }
]

核心问题1：JSON规范配置错误 #

你的age字段配置完全不符合文本实际格式：

• 文本中lastName占10个字符（索引10-19），age的实际起始位置是索引20左右，而规范中设置为17，指向的是lastName末尾的空格区域
• 规范中age的length设为20，而文本行总长度仅22左右，startIndex+length=37远大于行长度，触发line.length() >= startIndex + length条件不满足，导致age字段被跳过

修正后的JSON规范（根据实际文本调整）：

{
  "fields": [
    { "name": "firstName", "startIndex": 0, "length": 10 },
    { "name": "lastName", "startIndex": 10, "length": 10 },
    { "name": "age", "startIndex": 20, "length": 2 }
  ]
}

核心问题2：解析逻辑未使用传入的specFile #

当前parseLine方法直接依赖注入的全局configuration，完全忽略了parseFixedLengthFile接口传入的specFile参数，不符合“每个文本文件对应不同JSON规范”的需求，同时每次循环调用specFile.getInputStream()会重复打开流，存在资源浪费问题。

修正后的解析逻辑：

1. 新增DTO类用于解析JSON规范：

static class SpecDto {
    private List<FixedLengthField> fields;

    // Getter and Setter
    public List<FixedLengthField> getFields() {
        return fields;
    }

    public void setFields(List<FixedLengthField> fields) {
        this.fields = fields;
    }
}

static class FixedLengthField {
    private String name;
    private int startIndex;
    private int length;

    // Getters and Setters
    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getStartIndex() {
        return startIndex;
    }

    public void setStartIndex(int startIndex) {
        this.startIndex = startIndex;
    }

    public int getLength() {
        return length;
    }

    public void setLength(int length) {
        this.length = length;
    }
}

2. 修改parseFixedLengthFile和parseLine方法：

@PostMapping("/parse")
public List<Map<String, String>> parseFixedLengthFile(@RequestParam("file") MultipartFile file,
                                                      @RequestParam("spec") MultipartFile specFile) throws IOException {
    List<Map<String, String>> parsedData = new ArrayList<>();
    // 仅解析一次spec文件，避免重复读取流
    List<FixedLengthField> fields = parseSpec(specFile.getInputStream());
    
    try (BufferedReader reader = new BufferedReader(new InputStreamReader(file.getInputStream()))) {
        String line;
        while ((line = reader.readLine()) != null) {
            parsedData.add(parseLine(line, fields));
        }
    }
    saveParsedData(parsedData);
    return parsedData;
}

// 解析JSON规范为字段列表
private List<FixedLengthField> parseSpec(InputStream inputStream) throws IOException {
    com.fasterxml.jackson.databind.ObjectMapper objectMapper = new com.fasterxml.jackson.databind.ObjectMapper();
    return objectMapper.readValue(inputStream, SpecDto.class).getFields();
}

// 根据字段列表解析单行文本
private Map<String, String> parseLine(String line, List<FixedLengthField> fields) {
    Map<String, String> parsedData = new HashMap<>();
    for(FixedLengthField field : fields) {
        String fieldName = field.getName();
        int startIndex = field.getStartIndex();
        int length = field.getLength();

        // 优化判断逻辑：只要起始索引不超过行长度，就截取（超出部分取到行尾）
        if (startIndex < line.length()) {
            int endIndex = Math.min(startIndex + length, line.length());
            String fieldValue = line.substring(startIndex, endIndex).trim();
            parsedData.put(fieldName, fieldValue);
        }
    }
    return parsedData;
}

额外优化点 #

• 原判断条件line.length() >= startIndex + length过于严格，改为startIndex < line.length()并通过Math.min处理超出部分，避免有效字段被遗漏
• 移除了未使用的inputStream参数，减少无效逻辑

内容的提问来源于stack exchange，提问作者Justin Fulkerson