使用ChoETL将CSV转JSON时的字段重命名问题

我有多个CSV文件，需要用ChoETL工具转换为JSON文件，同时要重命名字段并转换数据类型，但在字段重命名环节遇到了问题。

测试输入文件：

VE NUMMER;VE NAAM;EFFECTDATUM;EINDDATUM
123;abc;20221001;
124;def;20221001;
125;ter;20221001;
126;ddf;20221001;

预期输出：

[
{
    "Id": 123,
    "Name": "abc",
    "Date": "20221001"
},
{
    "Id": 124,
    "Name": "def",
    "Date": "20221001"
}, ...
]

我已经尝试过这些方法，但都没成功：

• 使用BeforeRecordWrite事件，重命名属性时抛出错误
• 使用自定义ContractResolver，但似乎被忽略
• 使用自定义JsonConverter，但似乎被忽略

相关代码如下：

using ChoETL;
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using System.Reflection;

internal class Program
{
    private static void Main(string[] args)
    {
        QuickConversion();

    }

    private static void QuickConversion()
    {
        using var r = new ChoCSVReader("./Data./DummyFile.CSV")
               .WithFirstLineHeader()
               .WithDelimiter(";")
               .WithEOLDelimiter("\r\n")
               .MayHaveQuotedFields();

        using var w = new ChoJSONWriter(Console.Out)
           //.JsonSerializationSettings(settings =>
           //{
           //    settings.NullValueHandling = NullValueHandling.Ignore;
           //    settings.Formatting = Formatting.Indented;
           //    settings.MaxDepth = 2;
           //    settings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
           //    settings.ContractResolver = new CustomContractResolver("VE NUMMER", "ID");
           //})
           .WithJSONConverter(new CustomConverter())
        .UseJsonSerialization();

        w.Write(r);
    }
}

public class CustomConverter : JsonConverter
{
    public override bool CanConvert(Type objectType) => true;
    public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer) => throw new NotImplementedException();
    public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
    {
        serializer.ContractResolver = new CustomContractResolver("VE NUMMER", "ID");
    }
}
public class CustomContractResolver : DefaultContractResolver
{
    public string SourcePropertyName { get; set; }
    public string TargetPropertyName { get; set; }
    public CustomContractResolver(string sourcePropertyName, string targetPropertyName)
    {
        SourcePropertyName = sourcePropertyName;
        TargetPropertyName = targetPropertyName;
    }

    protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
    {
        var property = base.CreateProperty(member, memberSerialization);
        Console.WriteLine(property.PropertyName);

        if (property.PropertyName == SourcePropertyName)
        {
            property.PropertyName = TargetPropertyName;
        }
        return property;
    }
}

解决方案 #

方法1：强类型映射类（推荐） #

定义与预期输出匹配的实体类，通过ChoCSVColumn特性绑定CSV源字段，同时完成数据类型转换：

public class Record
{
    [ChoCSVColumn("VE NUMMER")]
    public int Id { get; set; }

    [ChoCSVColumn("VE NAAM")]
    public string Name { get; set; }

    [ChoCSVColumn("EFFECTDATUM")]
    public string Date { get; set; }

    // 忽略不需要的字段
    [ChoCSVColumn(Ignore = true)]
    public string EINDDATUM { get; set; }
}

修改转换代码：

private static void QuickConversion()
{
    using var r = new ChoCSVReader<Record>("./Data/DummyFile.CSV")
           .WithFirstLineHeader()
           .WithDelimiter(";")
           .WithEOLDelimiter("\r\n")
           .MayHaveQuotedFields();

    using var w = new ChoJSONWriter(Console.Out)
           .WithFormatting(true)
           .WithNullValueHandling(ChoNullValueHandling.Ignore);

    w.Write(r);
}

方法2：动态映射（无需实体类） #

直接在CSV读取阶段通过WithField指定字段映射和类型：

private static void QuickConversion()
{
    using var r = new ChoCSVReader("./Data/DummyFile.CSV")
           .WithFirstLineHeader()
           .WithDelimiter(";")
           .WithEOLDelimiter("\r\n")
           .MayHaveQuotedFields()
           .WithField("Id", fieldName: "VE NUMMER", fieldType: typeof(int))
           .WithField("Name", fieldName: "VE NAAM")
           .WithField("Date", fieldName: "EFFECTDATUM")
           .IgnoreField("EINDDATUM");

    using var w = new ChoJSONWriter(Console.Out)
           .WithFormatting(true)
           .WithNullValueHandling(ChoNullValueHandling.Ignore);

    w.Write(r);
}

方法3：LINQ投影转换 #

通过LINQ查询直接转换字段名和数据类型：

private static void QuickConversion()
{
    using var r = new ChoCSVReader("./Data/DummyFile.CSV")
           .WithFirstLineHeader()
           .WithDelimiter(";")
           .WithEOLDelimiter("\r\n")
           .MayHaveQuotedFields();

    var records = r.Select(row => new 
    {
        Id = Convert.ToInt32(row["VE NUMMER"]),
        Name = row["VE NAAM"].ToString(),
        Date = row["EFFECTDATUM"].ToString()
    });

    using var w = new ChoJSONWriter(Console.Out)
           .WithFormatting(true);

    w.Write(records);
}

原方法失效原因 #

• ContractResolver和JsonConverter被忽略：ChoETL处理动态CSV行时使用内部ChoDynamicObject，自定义JSON序列化设置无法直接作用于该类型。
• BeforeRecordWrite重命名出错：动态对象的属性集合固定，直接修改属性名会破坏内部映射逻辑。

内容的提问来源于stack exchange，提问作者Reinard