如何用jq从Kubernetes ConfigMap筛选指定URL数据并输出CSV

需求说明 #

需要从Kubernetes集群的ConfigMap中提取符合条件的数据：数据值以http://my-cluster-dns开头的URL，最终输出指定格式的CSV：

my-namespace,cluster-agent-config,DATA_2,http://my-cluster-dns.my-service:8080
her-namespace,cluster-agent-misc,DATA_1,http://my-cluster-dns.my-service:9000

原始K8s JSON数据 #

{"apiVersion": "v1","items": [{"apiVersion": "v1","data": {"DATA_1": false,"DATA_2": "http://my-cluster-dns.my-service:8080","DATA_3": 200},"kind": "ConfigMap","metadata": {"name": "cluster-agent-config","namespace": "my-namespace"}},{"apiVersion": "v1","data": {"agent-log.yaml": "log-level: INFO\nmax-filesize-mb: 50\nmax-backups: 5\nwrite-to-stdout: true"},"kind": "ConfigMap","metadata": {"name": "cluster-agent-log","namespace": "your-namespace"}},{"apiVersion": "v1","data": {"agent-monitoring.yml": "metric-collection-interval-seconds: 30\ncluster-metric-collection-interval-seconds: 60\nmetadata-collection-interval-seconds: 60\ncontainer-registration-batch-size: 5\npod-registration-batch-size: 6\nmetric-upload-retry-count: 2\nmetric-upload-retry-interval-milliseconds: 5\nmax-pods-to-register-count: 750\nmax-pod-logs-tail-lines-count: 500\ninstrumentation-max-polling-attempts: 10\npod-filter: {}\nns-to-monitor-regex: \"*.\""},"kind": "ConfigMap","metadata": {"name": "cluster-agent-mon","namespace": "his-namespace"}},{"apiVersion": "v1","data": {"DATA_1": "http://my-cluster-dns.my-service:9000","DATA_2": "something"},"kind": "ConfigMap","metadata": {"name": "cluster-agent-misc","namespace": "her-namespace"}}],"kind": "List","metadata": {"resourceVersion": ""}}

尝试过的jq命令及问题 #

• jq -r '.items[].data[] | select(contains("http://my-cluster-dns"))'：仅能获取目标值，丢失元数据（命名空间、ConfigMap名称）及对应键名
• jq -r '.items[] | . as $item | .data[] | $item.metadata.name, $item.metadata.namespace, . | select(contains("http://my-cluster-dns"))'：筛选逻辑错误，会将筛选条件应用到元数据字段上，导致输出混乱
• .items[] | . as $item | .data[] | [{ name: $item.metadata.name, namespace: $item.metadata.namespace, value: . }]：能关联元数据，但无法正确筛选出符合条件的URL值

正确的jq命令 #

jq -r '.items[] | .metadata as $meta | .data | to_entries[] | select(.value | startswith("http://my-cluster-dns")) | [$meta.namespace, $meta.name, .key, .value] | @csv'

命令拆解 #

1. .items[]：遍历所有ConfigMap条目
2. .metadata as $meta：将当前ConfigMap的元数据（命名空间、名称）保存为变量$meta，方便后续引用
3. .data | to_entries[]：将ConfigMap的data字段从键值对对象转换为{key: "...", value: "..."}的条目数组，同时获取键名和对应值
4. select(.value | startswith("http://my-cluster-dns"))：筛选出值以指定URL前缀开头的条目（startswith比contains更精准，避免匹配非开头的包含字符串）
5. [$meta.namespace, $meta.name, .key, .value]：将需要的字段按顺序组装成数组
6. @csv：将数组格式化为标准CSV格式，自动处理特殊字符（若不需要引号，可替换为join(",")，但@csv更规范）

输出结果 #

执行命令后得到符合要求的CSV：

"my-namespace","cluster-agent-config","DATA_2","http://my-cluster-dns.my-service:8080"
"her-namespace","cluster-agent-misc","DATA_1","http://my-cluster-dns.my-service:9000"

内容的提问来源于stack exchange，提问作者Thibault D