You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Benthos调用Overpass API报400错误，浏览器可正常访问URL

avatar
开发者阿泽
2026-6-17
问题：Benthos调用Overpass API返回400 Bad Request错误

问题背景

配置Benthos管道接收POST请求，解析请求体中的tile参数生成Overpass API查询URL。日志输出的URL可直接在浏览器正常访问，但Benthos发起请求时返回400错误；尝试转义URL后仍无法解决，改用metadata传递URL的修改也未生效。

现有配置与日志信息

初始Benthos管道配置 

input:
  http_server:
    address: ""
    path: /post
    ws_path: /post/ws
    allowed_verbs:
      - POST
    timeout: 120s
    rate_limit: ""

pipeline:
  processors:

    # Parse POST input query to get bounding box from body, ie {"tile": a line from all.json}
    - bloblang: |
        root.bbox.x_min = this.tile.split("&").index(0).split("=").index(1)
        root.bbox.x_max = this.tile.split("&").index(1).split("=").index(1)
        root.bbox.y_min = this.tile.split("&").index(2).split("=").index(1)
        root.bbox.y_max = this.tile.split("&").index(3).split("=").index(1)

    - bloblang: |
        root.url = "https://overpass-api.de/api/interpreter?data=[out:json];(node['amenity'](" + 
          this.bbox.y_min + "," + 
          this.bbox.x_min + "," + 
          this.bbox.y_max + "," + 
          this.bbox.x_max + ");" +
          "way['amenity'](" + 
          this.bbox.y_min + "," + 
          this.bbox.x_min + "," + 
          this.bbox.y_max + "," + 
          this.bbox.x_max + ");" +
          "relation['amenity'](" + 
          this.bbox.y_min + "," + 
          this.bbox.x_min + "," + 
          this.bbox.y_max + "," + 
          this.bbox.x_max + "););out 10 center;"

    # DEBUGGING
    - log: 
        level: info
        message: URL test 
        fields_mapping:
          root.url = this.url

    # Make a request to the Overpass API to fetch data
    - http: 
        url: "${!this.url}"
        verb: GET
        timeout: 60s

    # Parse the JSON response and extract relevant fields (array of elements)
    - bloblang: |
        root.batch = this.response.elements.map_each(e -> if e.tags.amenity != null && e.lat != null && e.lon != null {
          {
            "amenity": e.tags.amenity,
            "lat": e.lat,
            "lon": e.lon
          }
        }).filter(e -> e != null)

output:
  cypher:
    uri: "neo4j://localhost:7687"
    cypher: | 
      UNWIND $batch AS row
      MERGE (p:POI {amenity: row.amenity, lat: row.lat, lon: row.lon})
    # Treat array's rows as batches
    args_mapping: | 
      root.batch = this.batch 
    basic_auth:
      enabled: true
      username: "neo4j"
      password: "neotest"
    batching:
      count: 100
    max_in_flight: 64

日志输出的有效URL 

INFO URL test                                      @service=redpanda-connect label="" path=root.pipeline.processors.2 url="https://overpass-api.de/api/interpreter?data=[out:json];(node['amenity'](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095);way['amenity'](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095);relation['amenity'](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095););out 10 center;"

400错误日志 

ERRO HTTP request to '${!this.url}' failed: https://overpass-api.de/api/interpreter?data=[out:json];(node['amenity'](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095);way['amenity'](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095);relation['amenity'](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095););out 10 center;: HTTP request returned unexpected response code (400): 400 Bad Request, Error: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>400 Bad Request</title></head><body><h1>Bad Request</h1><p>Your browser sent a request that this server could not understand.<br /></p><hr><address>Apache/2.4.62 (Debian) Server at overpass-api.de Port 443</address></body></html>

错误的转义URL 

https%3A%2F%2Foverpass-api.de%2Fapi%2Finterpreter%3Fdata%3D%5Bout%3Ajson%5D%3B%28node%5B%27amenity%27%5D%2849.90146355262061%2C5.875795590895844%2C49.905976047309935%2C5.882782232769095%29%3Bway%5B%27amenity%27%5D%2849.90146355262061%2C5.875795590895844%2C49.905976047309935%2C5.882782232769095%29%3Brelation%5B%27amenity%27%5D%2849.90146355262061%2C5.875795590895844%2C49.905976047309935%2C5.882782232769095%29%3B%29%3Bout+10+center%3B

测试请求示例 

curl -X POST http://0.0.0.0:4195/post -H "Content-Type: application/json" -d '{"tile": "x_min=5.875795590895844&x_max=5.882782232769095&y_min=49.90146355262061&y_max=49.905976047309935"}'

修改后的管道配置 

pipeline:
 processors:

# Parse POST input query to get bounding box from body, ie {"tile": a line from all.json}
- bloblang: |
    root.bbox.x_min = this.tile.split("&").index(0).split("=").index(1)
    root.bbox.x_max = this.tile.split("&").index(1).split("=").index(1)
    root.bbox.y_min = this.tile.split("&").index(2).split("=").index(1)
    root.bbox.y_max = this.tile.split("&").index(3).split("=").index(1)

- bloblang: |
    root.url = "https://overpass-api.de/api/interpreter?data=[out:json];(node[%27amenity%27](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095);way[%27amenity%27](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095);relation[%27amenity%27](49.90146355262061,5.875795590895844,49.905976047309935,5.882782232769095););out%2010%20center;"

# Mapping to clear request body
- mapping: |
    meta = deleted()
    meta url = root.url

# Make a request to the Overpass API to fetch data
- http: 
    url: ${! metadata("url") }
    verb: GET
    timeout: 60s

# Parse the JSON response and extract relevant fields (array of elements)
- bloblang: |
    root.batch = this.response.elements.map_each(e -> if e.tags.amenity != null && e.lat != null && e.lon != null {
      {
        "amenity": e.tags.amenity,
        "lat": e.lat,
        "lon": e.lon
      }
    }).filter(e -> e != null)

问题原因分析

  1. URL转义范围错误：使用escape_url_query()转义了整个URL，导致协议、域名等部分被错误转义，服务器无法识别请求地址。正确的做法应该只转义data参数的值部分。
  2. 单引号未正确转义：Overpass查询中的单引号'属于URL特殊字符，浏览器会自动处理转义，但Benthos不会。直接拼接的URL中未对单引号进行转义，导致服务器解析请求参数失败。
  3. Benthos变量解析的隐式处理：直接在HTTP处理器的url字段使用${!this.url}时，可能存在隐式的字符编码或解析逻辑，导致实际发送的URL与日志输出不一致。

解决方案

方案1：正确转义data参数值

修改生成URL的Bloblang脚本，仅对Overpass查询部分进行转义，而非整个URL：

- bloblang: |
    let overpass_query = "[out:json];(node['amenity'](" + 
      this.bbox.y_min + "," + 
      this.bbox.x_min + "," + 
      this.bbox.y_max + "," + 
      this.bbox.x_max + ");" +
      "way['amenity'](" + 
      this.bbox.y_min + "," + 
      this.bbox.x_min + "," + 
      this.bbox.y_max + "," + 
      this.bbox.x_max + ");" +
      "relation['amenity'](" + 
      this.bbox.y_min + "," + 
      this.bbox.x_min + "," + 
      this.bbox.y_max + "," + 
      this.bbox.x_max + "););out 10 center;"
    root.url = "https://overpass-api.de/api/interpreter?data=" + $overpass_query.escape_url_query()

方案2：使用HTTP处理器的query参数自动转义

避免手动拼接URL，改用Benthos HTTP处理器的query字段配置参数，让框架自动处理转义逻辑：

- http: 
    url: "https://overpass-api.de/api/interpreter"
    verb: GET
    timeout: 60s
    query:
      data: |
        [out:json];(node['amenity'](${!this.bbox.y_min},${!this.bbox.x_min},${!this.bbox.y_max},${!this.bbox.x_max});way['amenity'](${!this.bbox.y_min},${!this.bbox.x_min},${!this.bbox.y_max},${!this.bbox.x_max});relation['amenity'](${!this.bbox.y_min},${!this.bbox.x_min},${!this.bbox.y_max},${!this.bbox.x_max}););out 10 center;

方案3：修复metadata传递的转义问题

如果坚持使用metadata传递URL，同样需要确保仅转义data参数值：

- bloblang: |
    let overpass_query = "[out:json];(node['amenity'](" + 
      this.bbox.y_min + "," + 
      this.bbox.x_min + "," + 
      this.bbox.y_max + "," + 
      this.bbox.x_max + ");" +
      "way['amenity'](" + 
      this.bbox.y_min + "," + 
      this.bbox.x_min + "," + 
      this.bbox.y_max + "," + 
      this.bbox.x_max + ");" +
      "relation['amenity'](" + 
      this.bbox.y_min + "," + 
      this.bbox.x_min + "," + 
      this.bbox.y_max + "," + 
      this.bbox.x_max + "););out 10 center;"
    meta.url = "https://overpass-api.de/api/interpreter?data=" + $overpass_query.escape_url_query()

内容的提问来源于stack exchange，提问作者TheArcturus

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级，最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分)， 限时 9.9 元起

产品介绍相关文档
相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级，最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分)， 限时 9.9 元起

产品介绍相关文档
火山引擎 最新活动
全面开放
Doubao-Seedance-2.5全新模型升级

模型再升级,30秒超长叙事, 模态参考扩容

HOT
方舟Codingplan

模型自由，工具不限，最新支持 Deepseek-V4 系列、GLM-5.3 系列

HOT
方舟Agent plan

超全模态模型 × Harness 升级，最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列

HOT
新用户特惠专场

大模型19元起，Al应用9.9元畅享，新人首购爆款尽享优惠

最近更新时间：2026.06.17 05:07:35