Zipkin接收请求返回400 Bad Request问题排查求助

场景说明 #

应用与Zipkin均部署在Docker容器中，通过Docker Compose配置在同一网络下，网络互通正常，Zipkin容器健康检查返回200 OK。但执行以下curl命令上报Span时：

docker run --rm -it --network backend appropriate/curl sh -c "curl -v -X POST http://zipkin:9411/api/v2/spans -H 'Content-Type: application/json' -d '[{\"id\":\"352bff9a74ca9ad2\",\"traceId\":\"5af7183fb1d4cf5f\",\"parentId\":\"6b221d5bc9e6496c\",\"name\":\"get /api\",\"timestamp\":1556604172355737,\"duration\":1431,\"kind\":\"SERVER\",\"localEndpoint\":{\"serviceName\":\"backend\",\"ipv4\":\"192.168.99.1\",\"port\":3306},\"remoteEndpoint\":{\"ipv4\":\"172.19.0.2\",\"port\":58648},\"tags\":{\"http.method\":\"GET\",\"http.path\":\"/api\"}}]'"

得到如下响应：

*   Trying 172.18.0.11...
* TCP_NODELAY set
* Connected to zipkin (172.18.0.11) port 9411 (#0)
> POST /api/v2/spans HTTP/1.1
> Host: zipkin:9411
> User-Agent: curl/7.59.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 342
>
* upload completely sent off: 342 out of 342 bytes
< HTTP/1.1 400 Bad Request
< content-type: text/*
< content-length: 38
< server: Armeria/1.29.4
< date: Thu, 8 Aug 2024 12:42:50 GMT
<
* Connection #0 to host zipkin left intact
Malformed reading List<Span> from json

错误原因 #

核心问题是curl命令中的JSON转义错误：

• 在sh -c的双引号包裹的命令里，-d参数用单引号包裹了JSON内容，但内部的双引号却使用了\"转义。单引号内的字符会被shell当作字面量处理，转义符不会生效，导致实际发送给Zipkin的JSON中包含\"而非正常的"，破坏了JSON格式，触发400错误。

另外，Zipkin容器日志中的警告信息与本次400错误无关，是Micrometer指标注册的配置顺序问题，不影响Span上报功能，可忽略。

修正后的命令 #

调整JSON内容的转义方式，避免多层引号冲突，推荐使用Here-doc写法更清晰：

docker run --rm -it --network backend appropriate/curl /bin/sh <<'EOF'
curl -v -X POST http://zipkin:9411/api/v2/spans \
  -H 'Content-Type: application/json' \
  -d '[
    {
      "id": "352bff9a74ca9ad2",
      "traceId": "5af7183fb1d4cf5f",
      "parentId": "6b221d5bc9e6496c",
      "name": "get /api",
      "timestamp": 1556604172355737,
      "duration": 1431,
      "kind": "SERVER",
      "localEndpoint": {
        "serviceName": "backend",
        "ipv4": "192.168.99.1",
        "port": 3306
      },
      "remoteEndpoint": {
        "ipv4": "172.19.0.2",
        "port": 58648
      },
      "tags": {
        "http.method": "GET",
        "http.path": "/api"
      }
    }
  ]'
EOF

内容的提问来源于stack exchange，提问作者Stoyan Grozdev