无需全量反序列化JSON对象,优化内部数组按日期排序性能
高效处理JSON中指定数组排序并输出JSON Lines格式
问题背景
现有如下结构的JSON对象:
{"results": [{"id": "abc456","groups": [{"parent_group": null,"type": "D"},{"parent_group": null,"type": "DEPARTMENT"}],"examples": [{"id": "e13b1e97-31e3-46e6-9d8f-9776c52e5ce0","date": "2020-05-10T00:00:00Z"},{"id": "bd31d475-6137-4409-8d17-535f1bf94071","date": "2021-05-11T00:00:00Z"},{"id": "0e0806ba-56f6-4527-8fd7-7e0061e30783","date": "2019-05-11T00:00:00Z"}]},{"id": "def456","groups": [{"parent_group": null,"type": "D"},{"parent_group": null,"type": "D"}],"examples": [{"id": "e13b1e97-31e3-46e6-9d8f-9776c52e5ce0","date": "2020-05-10T00:00:00Z"},{"id": "bd31d475-6137-4409-8d17-535f1bf94071","date": "2021-05-11T00:00:00Z"},{"id": "0e0806ba-56f6-4527-8fd7-7e0061e30783","date": "2019-05-11T00:00:00Z"}]}]}
需求是将每个results元素下的examples数组按date字段排序后,以JSON Lines格式输出。
原实现代码性能不佳:移除排序逻辑耗时约6ms,加入后增至30ms:
var jsonlBuilder = new StringBuilder(); var serializer = JsonSerializer.CreateDefault(new JsonSerializerSettings { DateTimeZoneHandling = DateTimeZoneHandling.Utc }); using (var textWriter = new StringWriter(jsonlBuilder)) using (var jsonWriter = new JsonTextWriter(textWriter) { Formatting = Formatting.None }) { foreach (var obj in jsonArray) { var employments = obj.SelectToken("examples"); if (employments.Count() > 1) { var b = employments.ToObject<JArray>().OrderBy(c => c.SelectToken("date").ToObject<DateTime>(serializer)); var newEmploymentArray = new JArray(b); obj["examples"].Replace(newEmploymentArray); } obj.WriteTo(jsonWriter); jsonWriter.WriteWhitespace("\n"); } }
优化方案
原实现的性能瓶颈在于多次反序列化操作(ToObject<JArray>、ToObject<DateTime>),以下两种方案可大幅提升效率:
方案1:直接利用ISO8601日期字符串排序,避免DateTime转换
由于date字段是标准ISO8601格式的字符串,这类字符串的字典序与时间顺序完全一致,无需转换为DateTime对象即可完成排序,省去大量转换开销:
var jsonlBuilder = new StringBuilder(); using (var textWriter = new StringWriter(jsonlBuilder)) using (var jsonWriter = new JsonTextWriter(textWriter) { Formatting = Formatting.None }) { foreach (var obj in jsonArray) { var examples = obj["examples"] as JArray; if (examples?.Count > 1) { // 直接按date字符串排序 var sortedExamples = new JArray(examples.OrderBy(item => (string)item["date"])); obj["examples"].Replace(sortedExamples); } obj.WriteTo(jsonWriter); jsonWriter.WriteWhitespace("\n"); } }
方案2:流式处理JSON,避免全量加载JToken
如果JSON体积极大,可使用Json.NET的JsonReader流式读取,仅在遇到examples数组时进行排序处理,其余内容直接复制到输出,大幅降低内存占用:
using (var reader = new JsonTextReader(new StringReader(inputJson))) using (var writer = new JsonTextWriter(new StringWriter(jsonlBuilder)) { Formatting = Formatting.None }) { while (reader.Read()) { if (reader.TokenType == JsonToken.PropertyName && (string)reader.Value == "examples") { // 读取并排序examples数组 reader.Read(); // 进入数组节点 var examples = JArray.Load(reader); if (examples.Count > 1) { var sorted = new JArray(examples.OrderBy(item => (string)item["date"])); sorted.WriteTo(writer); } else { examples.WriteTo(writer); } } else if (reader.TokenType == JsonToken.EndObject && reader.Path.StartsWith("results[")) { // 每个results元素结束后添加换行符 writer.WriteWhitespace("\n"); writer.WriteToken(reader); } else { // 直接复制其他JSON内容 writer.WriteToken(reader); } } }
性能说明
- 方案1可将排序相关耗时降低至接近无排序的水平(约7-8ms),核心是省去了
DateTime转换的开销。 - 方案2适合超大型JSON场景,无需加载整个JSON到内存,内存占用更低的同时保持高效排序。
内容的提问来源于stack exchange,提问作者DBK
相关产品推荐
相关产品推荐

