KEDA ScaledObject监听共享RabbitMQ队列无法拉起多Pod问题

我在Kubernetes集群中部署了基于RabbitMQ通信的Celery应用，包含三个队列：training-forecast-dev、training-forecast-solo-dev、training-forecast-prefork-dev。配置了两个Deployment实例，分别使用不同启动命令，均监听共享队列training-forecast-dev，同时各自监听专属队列。对应配置了两个KEDA ScaledObject，预期：

• 当training-forecast-dev队列有消息时，两个Pod都能被拉起
• 专属队列有消息时仅拉起对应Pod（该逻辑正常）

但实际情况是，training-forecast-dev队列有消息时仅拉起其中一个Pod，即使多任务到来也是如此。

第一个Deployment配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: worker-forecast-model-training-deploy
  labels:
    role: worker-forecast-model-training-service
spec:
  replicas: 0
  selector:
    matchLabels:
      role: worker-forecast-model-training-service
      tier: web-service
  template:
    metadata:
      labels:
        role: worker-forecast-model-training-service
        tier: web-service
    spec:
      containers:
        - name: worker-forecast-model-training
          image: prueba-celery-keda
          imagePullPolicy: IfNotPresent
          command:
            - "celery"
          args: [
            "-A",
            "app.worker",
            "worker",
            "--without-gossip",
            "--without-mingle",
            "--without-heartbeat",
            "-l",
            "info",
            "--pool",
            "solo",
            "-Q",
            "training-forecast-dev,training-forecast-solo-dev"
          ]
          env:
            - name: C_FORCE_ROOT
              value: "True"
          resources:
            requests:
              memory: "80Mi"
              cpu: "80m"
            limits:
              memory: "11000Mi"
              cpu: "2"

第二个Deployment配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: worker-forecast-model-training-prefork-deploy
  labels:
    role: worker-forecast-model-training-prefork-service
spec:
  replicas: 0
  selector:
    matchLabels:
      role: worker-forecast-model-training-prefork-service
      tier: web-service
  template:
    metadata:
      labels:
        role: worker-forecast-model-training-prefork-service
        tier: web-service
    spec:
      containers:
        - name: worker-forecast-model-training-prefork
          image: prueba-celery-keda-prefork
          imagePullPolicy: IfNotPresent
          command:
            - "celery"
          args: [
            "-A",
            "app.worker",
            "worker",
            "--without-gossip",
            "--without-mingle",
            "--without-heartbeat",
            "-l",
            "info",
            "--pool",
            "prefork",
            "-Q",
            "training-forecast-dev,training-forecast-prefork-dev"
          ]
          env:
            - name: C_FORCE_ROOT
              value: "True"
          resources:
            requests:
              memory: "80Mi"
              cpu: "80m"
            limits:
              memory: "11000Mi"
              cpu: "2"

第一个KEDA ScaledObject配置 #

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-forecast-model-deploy
spec:
  scaleTargetRef:
    name: worker-forecast-model-training-deploy
  pollingInterval: 10
  cooldownPeriod: 28800
  idleReplicaCount: 0
  minReplicaCount: 1
  maxReplicaCount: 1
  advanced:
    restoreToOriginalReplicaCount: true
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 60
          policies:
            - type: Percent
              value: 20
              periodSeconds: 1800
  triggers:
    - type: rabbitmq
      metadata:
        host: amqp://default_user:asd123@hello-world.default.svc.cluster.local:5672//
        queueName: training-forecast-dev
        mode: QueueLength
        value: "1"
    - type: rabbitmq
      metadata:
        host: amqp://default_user:asd123@hello-world.default.svc.cluster.local:5672//
        queueName: training-forecast-solo-dev
        mode: QueueLength
        value: "1"

第二个KEDA ScaledObject配置 #

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-forecast-model-training-prefork-deploy
spec:
  scaleTargetRef:
    name: worker-forecast-model-training-prefork-deploy
  pollingInterval: 10
  cooldownPeriod: 28800
  idleReplicaCount: 0
  minReplicaCount: 1
  maxReplicaCount: 1
  advanced:
    restoreToOriginalReplicaCount: true
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 60
          policies:
            - type: Percent
              value: 20
              periodSeconds: 1800
  triggers:
    - type: rabbitmq
      metadata:
        host: amqp://default_user:asd123@hello-world.default.svc.cluster.local:5672//
        queueName: training-forecast-dev
        mode: QueueLength
        value: "1"
    - type: rabbitmq
      metadata:
        host: amqp://default_user:asd123@hello-world.default.svc.cluster.local:5672//
        queueName: training-forecast-prefork-dev
        mode: QueueLength
        value: "1"

内容的提问来源于stack exchange，提问作者Diego L