AWS EKS中Knative/KServe gRPC与Nginx Ingress配置故障求助

以下是针对当前配置问题的具体修复步骤：

1. 调整Istio入口网关Service与Gateway配置，添加h2c支持 #

Istio网关需要明确暴露h2c（明文HTTP/2）端口以处理gRPC请求，修改istio-ingressgateway Service：

apiVersion: v1
kind: Service
metadata:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /healthz/ready
    alb.ingress.kubernetes.io/healthcheck-port: "15021"
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: unknown
    istio: ingressgateway
    istio.io/rev: default
    operator.istio.io/component: IngressGateways
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
spec:
  clusterIP: 172.17.17.19
  clusterIPs:
  - 172.17.17.19
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: status-port
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: http2
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    port: 443
    protocol: TCP
    targetPort: 8443
  # 新增h2c端口用于gRPC明文传输
  - name: h2c
    port: 9080
    protocol: TCP
    targetPort: 9080
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  sessionAffinity: None
  type: ClusterIP

同时更新Istio Gateway资源，添加h2c监听规则：

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: knative-serving-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
    tls:
      httpsRedirect: true
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - "*"
    tls:
      mode: SIMPLE
      credentialName: cert-knative-wildcard
  # 新增h2c端口监听
  - port:
      number: 9080
      name: h2c
      protocol: HTTP2
    hosts:
    - "*"

2. 修正gRPC专用Ingress配置 #

当前gRPC Ingress指向Istio的80端口，需改为新添加的9080端口，并补充必要注解：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: default
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
    nginx.ingress.kubernetes.io/grpc-backend: "true"
    # 保留Host头，确保Knative能正确路由
    nginx.ingress.kubernetes.io/upstream-vhost: "$host"
  name: computing-grpc-ingress
  namespace: istio-system
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - '*.default.knative.company.com'
    secretName: cert-knative-wildcard
  rules:
  - host: '*.default.knative.company.com'
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 9080
        path: /
        pathType: Prefix

3. 启用Knative Serving的HTTP/2支持 #

检查并更新Knative Serving的config-network ConfigMap：

kubectl edit configmap config-network -n knative-serving

确保添加或修改以下配置：

data:
  http2-enabled: "true"

4. 优化InferenceService的gRPC标识 #

为InferenceService添加注解，明确标记为gRPC服务：

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  annotations:
    serving.knative.dev/protocol: grpc
    serving.knative.dev/visibility: "cluster-local"
  finalizers:
  - inferenceservice.finalizers
  name: triton-test
  namespace: default
spec:
  predictor:
    model:
      args:
      - --model-control-mode=poll
      - --repository-poll-secs=5
      - --allow-grpc=true
      - --grpc-port=9000
      - --log-verbose=0
      env:
      - name: CUDA_VISIBLE_DEVICES
        value: "0"
      - name: S3_DATA_PATH
        value: s3://mymodeldata/
      - name: S3_PARAMS
        value: --region us-east-2 --no-sign-request
      image: XXXXXXX.dkr.ecr.us-east-2.amazonaws.com/ml:mytriton
      modelFormat:
        name: triton
      name: kserve-container
      ports:
      - containerPort: 9000
        name: h2c
        protocol: TCP
      protocolVersion: v2
      resources:
        limits:
          cpu: "2"
          memory: 8Gi
          nvidia.com/gpu: "1"
        requests:
          cpu: "2"
          memory: 8Gi
          nvidia.com/gpu: "1"
      storageUri: s3://mymodeldata/
      volumeMounts:
      - mountPath: /dev/shm
        name: dshm
    nodeSelector:
      DedicatedFor: GPU
    volumes:
    - emptyDir:
        medium: Memory
        sizeLimit: 2Gi
      name: dshm

5. 测试验证 #

应用所有配置后，使用gRPC客户端测试服务连通性：

# 替换为实际模型名与域名
grpcurl -d '{"model_name": "your-model", "inputs": [...]}' triton-test.default.knative.company.com:443 inference.GRPCInferenceService/ModelInfer

# 测试服务反射
grpcurl triton-test.default.knative.company.com:443 list

内容的提问来源于stack exchange，提问作者dmitrii