Kubernetes DaemonSet调用Node资源遇403及节点名获取问题求助

问题背景 #

写了个Python脚本加载节点资源，调用前执行了config.load_incluster_config()，但运行时先报403 Forbidden错误。脚本以DaemonSet形式部署，已经配置了ServiceAccount并绑定ClusterRole，helm install后还是报错。后来发现是ServiceAccount名称配置错误，修正后403问题解决，但又出现新问题：调用fetch_node_resource时传入的是Pod名称而非节点名称，导致查询节点资源失败。

相关代码与配置 #

Python脚本核心片段 #

def fetch_node_resource(cls, node_name: str) -> Resources:
    # config.load_kube_config()
    api_client = client.CoreV1Api()
    node = api_client.read_node(node_name)

    # 提取节点资源信息
    capacity: dict = node.status.capacity

    cpu_capacity = Maybe.from_optional(capacity.get("cpu", Nothing))
    memory_capacity = Maybe.from_optional(capacity.get("memory", Nothing))
    network_bandwidth_capacity = Maybe.from_optional(
        capacity.get("network_bandwidth", Nothing)
    )
    gpu_capacity = Maybe.from_optional(capacity.get("nvidia.com/gpu", Nothing))

    return Resources(
        cpu=cpu_capacity,
        gpu=gpu_capacity,
        memory=memory_capacity,
        net_bandwidth=network_bandwidth_capacity,
    )

报错信息 #

"node-exporter-simulator-sbcqr" is forbidden: User "system:serviceaccount:monitoring:node-exporter-simulator-account" cannot get resource "nodes" in API group

DaemonSet配置 #

{{- if .Values.nodeExporterSimulator.enabled }}
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter-simulator
  namespace: {{ .Values.namespace }}
  labels:
    app: node-exporter-simulator
spec:
  selector:
    matchLabels:
      app: node-exporter-simulator
  template:
    metadata:
      labels:
        app: node-exporter-simulator
    spec:
      serviceAccountName: node-exporter-simulator-account
      containers:
        - name: metrics-generator
          image: dev0guy/node-exporter-simulator:v0.0.5
          env:
            - name: PROMETHEUS_GATEWAY_URL
              value: "pushgateway:9091"
            - name: PUSH_INTERVAL
              value:  {{ .Values.nodeExporterSimulator.interval }}
{{- end }}

RBAC配置 #

apiVersion: v1
kind: ServiceAccount
metadata:
  name: node-exporter-simulator-account
  namespace: {{ .Values.namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: node-exporter-simulator-role
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: node-exporter-simulator-binding
  namespace: {{ .Values.namespace }}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: node-exporter-simulator-role
subjects:
- kind: ServiceAccount
  name: node-exporter-simulator-account
  namespace: {{ .Values.namespace }}

解决方案 #

已解决的403问题 #

之前的403是因为ServiceAccount名称配置不匹配，修正名称后权限验证通过。

当前节点名称获取问题 #

报错里的node-exporter-simulator-sbcqr是Pod的名称，不是节点名称，用它调用read_node自然会失败。下面两种方法可以拿到Pod所在的节点名称：

方法1：用Downward API注入节点名称到环境变量

修改DaemonSet的容器环境变量配置，添加节点名称的注入：

containers:
  - name: metrics-generator
    image: dev0guy/node-exporter-simulator:v0.0.5
    env:
      - name: PROMETHEUS_GATEWAY_URL
        value: "pushgateway:9091"
      - name: PUSH_INTERVAL
        value:  {{ .Values.nodeExporterSimulator.interval }}
      - name: NODE_NAME
        valueFrom:
          fieldRef:
            fieldPath: spec.nodeName

然后在Python脚本中读取这个环境变量，作为参数传入fetch_node_resource：

import os

# 获取注入的节点名称
node_name = os.getenv("NODE_NAME")
if node_name:
    resources = YourClass.fetch_node_resource(node_name)

方法2：通过K8s API获取当前Pod的节点信息

如果不想用Downward API，也可以通过API查询自身Pod的信息来提取节点名称。首先需要给ServiceAccount添加pods资源的get权限，修改ClusterRole的rules：

rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get"]

然后在脚本中添加获取节点名称的逻辑，同时需要注入Pod名称和命名空间的环境变量：

# 在DaemonSet的env中添加
- name: POD_NAME
  valueFrom:
    fieldRef:
      fieldPath: metadata.name
- name: POD_NAMESPACE
  valueFrom:
    fieldRef:
      fieldPath: metadata.namespace

对应的Python代码：

import os
from kubernetes import client, config

def get_current_node_name() -> str:
    config.load_incluster_config()
    v1 = client.CoreV1Api()
    
    # 从环境变量获取当前Pod的名称和命名空间
    pod_name = os.getenv("POD_NAME")
    pod_namespace = os.getenv("POD_NAMESPACE")
    if not pod_name or not pod_namespace:
        raise ValueError("未设置POD_NAME或POD_NAMESPACE环境变量")
    
    # 查询当前Pod信息
    pod = v1.read_namespaced_pod(pod_name, pod_namespace)
    return pod.spec.nodeName

# 使用示例
node_name = get_current_node_name()
resources = YourClass.fetch_node_resource(node_name)

内容的提问来源于stack exchange，提问作者Guy-Arieli