Orion-LD订阅访问MongoDB报csf/name字段缺失错误求助
Orion-LD 订阅推送MongoDB字段缺失报错排查方案
问题描述
搭建Orion-LD到Quantum Leap、CrateDB的订阅通知链路时,MongoDB访问订阅数据反复抛出两类错误,最终导致数据无法推送至Quantum Leap做后续处理:
Error (string field 'csf' is missing in BSONObjError (string field 'name' is missing in BSONObj'
容器日志报错片段如下:
... time=Friday 24 Jun 11:03:59 2022.081Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=safeMongo.cpp[145]:getStringField | msg=Runtime Error (string field 'name' is missing in BSONObj <{ _id: "urn:ngsi-ld:Subscription:62b578ea4567412cdf07306e", expiration: 2147483647.0, reference: "http://172.18.1.5:8668/v2/notify", custom: false, mimeType: "application/json", throttling: 0.0, servicePath: "/", description: "Notify me of temperature", status: "active", entities: [ { id: "", isPattern: "", type: "https://uri.fiware.org/ns/data-models#WeatherObserved", isTypePattern: false } ], attrs: [ "https://uri.fiware.org/ns/data-models#temperature" ], metadata: [], blacklist: false, ldContext: "http://172.18.1.2/datamodels.context-ngsi.jsonld", createdAt: 1656060138.058298, modifiedAt: 1656060138.058298, conditions: [ "https://uri.fiware.org/ns/data-models#temperature" ], expression: { q: "https://uri=fiware=org/ns/data-models#temperature<100", mq: "", geometry: "", coords: "", georel: "", geoproperty: "" }, format: "normalized" }> from caller setName:280) time=Friday 24 Jun 11:03:59 2022.081Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=safeMongo.cpp[145]:getStringField | msg=Runtime Error (string field 'csf' is missing in BSONObj <{ _id: "urn:ngsi-ld:Subscription:62b578ea4567412cdf07306e", expiration: 2147483647.0, reference: "http://172.18.1.5:8668/v2/notify", custom: false, mimeType: "application/json", throttling: 0.0, servicePath: "/", description: "Notify me of temperature", status: "active", entities: [ { id: "", isPattern: "", type: "https://uri.fiware.org/ns/data-models#WeatherObserved", isTypePattern: false } ], attrs: [ "https://uri.fiware.org/ns/data-models#temperature" ], metadata: [], blacklist: false, ldContext: "http://172.18.1.2/datamodels.context-ngsi.jsonld", createdAt: 1656060138.058298, modifiedAt: 1656060138.058298, conditions: [ "https://uri.fiware.org/ns/data-models#temperature" ], expression: { q: "https://uri=fiware=org/ns/data-models#temperature<100", mq: "", geometry: "", coords: "", georel: "", geoproperty: "" }, format: "normalized" }> from caller setCsf:302) ...
出现问题前已通过http://localhost:1026/ngsi-ld/v1/subscriptions/接口创建订阅,请求体如下:
{ "description": "Notify me of temperature", "type": "Subscription", "entities": [{"type": "WeatherObserved"}], "watchedAttributes": ["temperature"], "notification": { "attributes": ["temperature"], "format": "normalized", "endpoint": { "uri": "http://172.18.1.5:8668/v2/notify", "accept": "application/json" } }, "@context": "http://172.18.1.2/datamodels.context-ngsi.jsonld" }
调用查询接口验证订阅确实存在,返回片段如下:
[ { "id": "urn:ngsi-ld:Subscription:62b578ea4567412cdf07306e", "type": "Subscription", "description": "Notify me of temperature", "entities": [ { "type": "WeatherObserved" } ], "watchedAttributes": [ "temperature" ], "q": "https://uri.fiware.org/ns/data-models#temperature<100", "notification": { "attributes": [ "temperature" ], "format": "normalized", "endpoint": { "uri": "http://172.18.1.5:8668/v2/notify", "accept": "application/json" } }, "@context": "http://172.18.1.2/datamodels.context-ngsi.jsonld" }, ... ]
执行docker logs -f <orion ld container>查看日志可见上述MongoDB错误重复出现,说明Orion在主动尝试读取该订阅时触发异常。
当前部署使用的docker-compose配置如下:
version: "3.8" services: ######## # CORE # ######## # -> Orion: context broker as central component orion: labels: org.test: 'fiware' image: fiware/orion-ld:${ORION_VERSION} hostname: orion container_name: fiware-orion depends_on: - mongo-db networks: default: ipv4_address: 172.18.1.3 ports: - "${ORION_PORT}:${ORION_PORT}" command: -dbhost mongo-db -logLevel DEBUG -noCache healthcheck: test: curl --fail -s http://orion:${ORION_PORT}/version || exit 1 interval: 5s # -> Context: provide ngsi-ld context file for smart data models ld-context: labels: org.test: 'fiware' image: httpd:alpine hostname: context container_name: fiware-ld-context ports: - "3004:80" networks: default: ipv4_address: 172.18.1.2 volumes: - ./context:/usr/local/apache2/htdocs/ healthcheck: test: (wget --server-response --spider --quiet http://172.18.1.2/datamodels.context-ngsi.jsonld 2>&1 | awk 'NR==1{print $$2}'| grep -q -e "200") || exit 1 ################## # DATA MANGEMENT # ################## # Quantum Leap: is persisting Short Term History to Crate-DB quantumleap: labels: org.test: 'fiware' image: orchestracities/quantumleap:${QUANTUMLEAP_VERSION} hostname: quantumleap container_name: fiware-quantumleap depends_on: - crate-db - redis-db networks: default: ipv4_address: 172.18.1.5 ports: - "${QUANTUMLEAP_PORT}:${QUANTUMLEAP_PORT}" environment: - CRATE_HOST=crate-db - REDIS_HOST=redis-db - REDIS_PORT=${REDIS_PORT} - LOGLEVEL=DEBUG healthcheck: test: curl --fail -s http://quantumleap:${QUANTUMLEAP_PORT}/version || exit 1 ################# # VISUALIZATION # ################# # -> Grafana: Visualize Time Series data grafana: labels: org.test: 'fiware' image: grafana/grafana:6.1.6 container_name: grafana depends_on: - crate-db networks: default: ipv4_address: 172.18.1.8 ports: - "3003:3000" environment: - GF_INSTALL_PLUGINS=https://github.com/orchestracities/grafana-map-plugin/archive/master.zip;grafana-map-plugin,grafana-clock-panel,grafana-worldmap-panel volumes: - grafana:/var/lib/grafana ############# # DATABASES # ############# # -> MongoDB: database of Orion mongo-db: labels: org.test: 'fiware' image: mongo:${MONGO_DB_VERSION} hostname: mongo-db container_name: db-mongo expose: - "${MONGO_DB_PORT}" ports: - "${MONGO_DB_PORT}:${MONGO_DB_PORT}" networks: default: ipv4_address: 172.18.1.4 volumes: - mongo-db:/data healthcheck: test: | host=`hostname --ip-address || echo '127.0.0.1'`; mongo --quiet $host/test --eval 'quit(db.runCommand({ ping: 1 }).ok ? 0 : 2)' && echo 0 || echo 1 interval: 5s # -> CreateDB: database to store time-series data crate-db: labels: org.test: 'fiware' image: crate:${CRATE_VERSION} hostname: crate-db container_name: db-crate networks: default: ipv4_address: 172.18.1.6 ports: - "4200:4200" - "4300:4300" command: crate -Cauth.host_based.enabled=false -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*" environment: - CRATE_HEAP_SIZE=2g volumes: - crate-db:/data # -> Redis: Normally used to efficiently store key value pairs redis-db: labels: org.test: 'fiware' image: redis:${REDIS_VERSION} hostname: redis-db container_name: db-redis networks: default: ipv4_address: 172.18.1.7 ports: - "${REDIS_PORT}:${REDIS_PORT}" volumes: - redis-db:/data healthcheck: test: | host=`hostname -i || echo '127.0.0.1'`; ping=`redis-cli -h "$host" ping` && [ "$ping" = 'PONG' ] && echo 0 || echo 1 interval: 10s # NETWORKS networks: default: labels: org.test: 'fiware' ipam: config: - subnet: 172.18.1.0/24 # VOLUMES volumes: mongo-db: ~ context: ~ grafana: ~ crate-db: ~ redis-db: ~
对应.env配置内容如下:
# Project name COMPOSE_PROJECT_NAME=fiware # Orion variables ORION_PORT=1026 ORION_VERSION=1.0.0 # MongoDB variables MONGO_DB_PORT=27017 MONGO_DB_VERSION=4.4 # QuantumLeap Variables QUANTUMLEAP_VERSION=0.8.3 QUANTUMLEAP_PORT=8668 # CrateDB Version CRATE_VERSION=4.6 # RedisDB Version REDIS_PORT=6379 REDIS_VERSION=6
根因分析
该问题和MongoDB驱动版本无关,核心是fiware/orion-ld:1.0.0版本存在代码bug:
- 写入订阅数据到MongoDB时,序列化逻辑没有给
expression对象默认填充csf、name两个空字符串字段 - 读取订阅数据时,解析逻辑强制要求这两个字符串字段存在,缺失就直接抛出Runtime Error
- 搭配MongoDB 4.4版本时,数据库不会自动补全缺失的文档字段,因此每次Orion轮询订阅都会触发报错,导致订阅逻辑中断,无法推送数据到Quantum Leap。
解决步骤
- 升级Orion-LD版本
修改.env文件中ORION_VERSION参数为1.4.0及以上稳定版本,新版已经修复了订阅序列化缺失默认字段的问题。 - 清理旧异常订阅数据
升级后先清除MongoDB中存储的存在字段缺失的旧订阅,避免残留数据持续触发报错:- 执行
docker exec -it db-mongo mongo进入MongoDB命令行 - 执行
use orion切换到Orion默认使用的数据库 - 执行
db.csubs.remove({})清空旧订阅集合(如果存在其他正常订阅,可根据_id条件过滤删除异常条目)
- 执行
- 兼容方案(暂不升级Orion时使用)
如果短期无法升级Orion-LD,可将.env中MONGO_DB_VERSION改为3.6,旧版MongoDB对缺失字段的兼容逻辑不会触发该报错,但不推荐长期使用过旧的数据库版本。 - 重建订阅验证链路
重启所有容器后,重新调用订阅创建接口生成推送规则,观察Orion日志不再出现字段缺失报错,即可验证数据推送链路恢复正常。
内容的提问来源于stack exchange,提问作者IngMiad
相关产品推荐
相关产品推荐

