如何让GKE中多个Pod/容器共享使用同一GPU？

核心原因 #

GKE的NVIDIA设备插件只会给声明了nvidia.com/gpu: 1资源限制的容器自动注入/usr/local/nvidia/驱动目录，未声明该限制的容器不会自动获得驱动文件——这和单节点Docker环境下NVIDIA runtime的全局处理行为不同。

可行解决方案 #

1. 直接挂载节点上的NVIDIA驱动目录（最简便）

GKE节点上的NVIDIA驱动默认安装在/usr/local/nvidia/，可以通过hostPath卷把这个目录挂载到Pod内的所有容器中，让每个容器直接访问驱动文件。

修改后的Pod配置示例：

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  volumes:
    - name: x11-socket
      hostPath:
        path: /tmp/x11-socket
    # 新增NVIDIA驱动共享卷
    - name: nvidia-driver
      hostPath:
        path: /usr/local/nvidia
        type: Directory
  containers:
  - name: xserver
    image: xserver
    command: ["sleep"]
    args: ["infinity"]
    volumeMounts:
      - name: x11-socket
        mountPath: /tmp/.X11-unix
      - name: nvidia-driver
        mountPath: /usr/local/nvidia
    securityContext:
      privileged: true
    resources:
      limits:
        nvidia.com/gpu: 1
  - name: app
    image: app
    command: ["sleep"]
    args: ["infinity"]
    volumeMounts:
      - name: x11-socket
        mountPath: /tmp/.X11-unix
      - name: nvidia-driver
        mountPath: /usr/local/nvidia
    securityContext:
      privileged: true
    resources:
      limits:
        nvidia.com/gpu: 0

2. 用Init容器预复制驱动到共享卷（适配驱动路径不固定场景）

如果担心节点驱动路径有变动，可以用一个声明了GPU资源的Init容器，把/usr/local/nvidia/的内容复制到emptyDir共享卷里，再让其他容器挂载这个共享卷：

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  volumes:
    - name: x11-socket
      hostPath:
        path: /tmp/x11-socket
    # 共享驱动的emptyDir卷
    - name: shared-nvidia-driver
      emptyDir: {}
  initContainers:
  - name: copy-nvidia-driver
    image: xserver # 用带NVIDIA依赖的镜像即可
    command: ["sh", "-c", "cp -r /usr/local/nvidia/* /shared-driver/"]
    volumeMounts:
      - name: shared-nvidia-driver
        mountPath: /shared-driver
    resources:
      limits:
        nvidia.com/gpu: 1
  containers:
  - name: xserver
    image: xserver
    command: ["sleep"]
    args: ["infinity"]
    volumeMounts:
      - name: x11-socket
        mountPath: /tmp/.X11-unix
      - name: shared-nvidia-driver
        mountPath: /usr/local/nvidia
    securityContext:
      privileged: true
    resources:
      limits:
        nvidia.com/gpu: 1
  - name: app
    image: app
    command: ["sleep"]
    args: ["infinity"]
    volumeMounts:
      - name: x11-socket
        mountPath: /tmp/.X11-unix
      - name: shared-nvidia-driver
        mountPath: /usr/local/nvidia
    securityContext:
      privileged: true
    resources:
      limits:
        nvidia.com/gpu: 0

3. 适配DaemonSet部署X.Org的场景

如果X.Org用DaemonSet部署（每个节点一个X服务器），X.Org的Pod已经声明GPU资源并获得驱动。后续应用Pod只需：

• 挂载节点上的X11套接字（和X.Org DaemonSet共用同一个hostPath卷）
• 挂载节点的/usr/local/nvidia/目录到应用容器中
• 无需再声明GPU资源（GKE会自动调度应用到有GPU且X.Org运行的节点）

注意事项 #

• 确保GKE节点已安装NVIDIA设备插件（GKE GPU节点默认预装）
• 挂载hostPath依赖节点驱动版本一致性，若集群节点驱动版本不同，建议用Init容器方案或统一节点驱动版本
• 容器的securityContext.privileged: true可根据实际需求调整，若无需特权模式，需确保容器有访问驱动目录和X套接字的权限

内容的提问来源于stack exchange，提问作者Velkan