如何禁用Istio集群内部通信的Proxy Protocol

当前配置的EnvoyFilter给Istio IngressGateway的所有监听端口都添加了Proxy Protocol过滤器，包括集群内部通信使用的端口。当内部VirtualService转发请求时，发送的是普通HTTP请求（不带Proxy Protocol头），而下游服务的Listener因为强制要求Proxy Protocol头，会在超时后终止连接，从而出现upstream connect error or disconnect/reset before headers报错。

推荐采用拆分网关职责的方式，让一个网关专门处理外部带Proxy Protocol的请求，另一个网关处理内部普通请求，彻底隔离两种通信场景。

方案一：创建内部专用网关（推荐） #

1. 定义内部Gateway资源 #

创建一个仅用于集群内部通信的Gateway，使用ClusterIP类型，不配置Proxy Protocol相关规则：

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: internal-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway # 复用现有IngressGateway实例，也可单独部署新的Gateway负载
  servers:
  - port:
      number: 8080
      name: http-internal
      protocol: HTTP
    hosts:
    - "*"

2. 修改原有EnvoyFilter，仅对外部端口生效 #

调整Proxy Protocol和XFF配置，只作用于外部LoadBalancer对接的端口（比如默认的80/443），不对内部端口生效：

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: proxy-protocol
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    # 仅对外部HTTP端口添加Proxy Protocol过滤器
    - applyTo: LISTENER
      match:
        listener:
          portNumber: 80 # 替换为你的外部HTTP端口
      patch:
        operation: MERGE
        value:
          listener_filters:
            - name: envoy.filters.listener.proxy_protocol
            - name: envoy.filters.listener.tls_inspector
    # 仅对外部HTTPS端口添加Proxy Protocol过滤器
    - applyTo: LISTENER
      match:
        listener:
          portNumber: 443 # 替换为你的外部HTTPS端口
      patch:
        operation: MERGE
        value:
          listener_filters:
            - name: envoy.filters.listener.proxy_protocol
            - name: envoy.filters.listener.tls_inspector
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingressgateway-settings
  namespace: istio-system
spec:
  configPatches:
    # 仅对外部HTTP端口应用XFF配置
    - applyTo: NETWORK_FILTER
      match:
        listener:
          portNumber: 80
          filterChain:
            filter:
              name: envoy.http_connection_manager
      patch:
        operation: MERGE
        value:
          name: envoy.http_connection_manager
          typed_config:
            "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
            skip_xff_append: false
            use_remote_address: true
            xff_num_trusted_hops: 1
    # 仅对外部HTTPS端口应用XFF配置
    - applyTo: NETWORK_FILTER
      match:
        listener:
          portNumber: 443
          filterChain:
            filter:
              name: envoy.http_connection_manager
      patch:
        operation: MERGE
        value:
          name: envoy.http_connection_manager
          typed_config:
            "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
            skip_xff_append: false
            use_remote_address: true
            xff_num_trusted_hops: 1

3. 调整VirtualService绑定关系 #

拆分外部和内部转发规则，外部请求绑定原app-gateway，内部转发绑定新的internal-gateway：

# 外部访问专用VirtualService
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: "app-virtual-service"
spec:
  gateways:
    - "app-gateway"
  hosts:
    - hub.example.com
  http:
    - match:
        - uri:
            prefix: "/graphql"
      rewrite:
        authority: app-api.app.svc.cluster.local
      route:
        - destination:
            host: app-api.app.svc.cluster.local
          weight: 100
    - match:
        - uri:
            prefix: "/hall"
      rewrite:
        authority: app-hall.app.svc.cluster.local
      route:
        - destination:
            host: app-hall.app.svc.cluster.local
          weight: 100
    - match:
        - uri:
            exact: /
        {{- range .Values.manager.paths }}
        - uri:
            prefix: {{ . | quote }}
        {{- end }}
      rewrite:
        authority: app-manager.app.svc.cluster.local
      route:
        - destination:
            host: app-manager.app.svc.cluster.local
          weight: 100
---
# 内部转发专用VirtualService
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: "app-internal-virtual-service"
spec:
  gateways:
    - istio-system/internal-gateway
    - knative-serving/knative-local-gateway
  hosts:
    - hub.example.com # 可替换为内部专用域名
  http:
    - match:
        - uri:
            prefix: "/graphql"
      rewrite:
        authority: app-api.app.svc.cluster.local
      route:
        - destination:
            host: app-api.app.svc.cluster.local
          weight: 100
    - match:
        - uri:
            prefix: "/hall"
      rewrite:
        authority: app-hall.app.svc.cluster.local
      route:
        - destination:
            host: app-hall.app.svc.cluster.local
          weight: 100
    - match:
        - uri:
            exact: /
        {{- range .Values.manager.paths }}
        - uri:
            prefix: {{ . | quote }}
        {{- end }}
      rewrite:
        authority: app-manager.app.svc.cluster.local
      route:
        - destination:
            host: app-manager.app.svc.cluster.local
          weight: 100

方案二：临时跳过下游服务的Proxy Protocol检查（不推荐） #

若不想拆分网关，可给下游服务的Sidecar配置EnvoyFilter，移除内部监听端口的Proxy Protocol过滤器：

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: internal-service-proxy-skip
  namespace: app # 替换为服务所在命名空间
spec:
  workloadSelector:
    labels:
      app: app-api # 替换为目标服务的标签
  configPatches:
    - applyTo: LISTENER
      match:
        listener:
          portNumber: 8080 # 替换为服务的内部监听端口
      patch:
        operation: REMOVE
        path: "/listener_filters/name:envoy.filters.listener.proxy_protocol"

此方案会降低服务的安全性，仅建议作为临时过渡方案使用。

内容的提问来源于stack exchange，提问作者blits