Firebase模拟器在Kubernetes中重复加载旧数据原因排查

问题背景 #

我用Tilt将多个微服务部署到Docker Desktop启用的本地Kubernetes集群，其中Firebase模拟器存在以下问题：删除部署后重新执行tilt up部署时，Firestore会自动加载之前的旧数据，即使手动清空Firestore数据、执行tilt down彻底删除部署，重启后数据库仍会短时间内恢复旧数据，我需要阻止这种数据持久化行为。

相关配置 #

1. Firebase模拟器Dockerfile #

FROM node:20-alpine

WORKDIR /app

RUN apk add --no-cache openjdk11
RUN npm install -g firebase-tools

COPY . .

EXPOSE 4000 4400 4500 8085 8086 8087 9000

ENV GCP_PROJECT=my-project
ENV ENABLE_UI=true
ENV UI_EMULATOR_PORT=4000
ENV HUB_EMULATOR_PORT=4400
ENV LOGGING_EMULATOR_PORT=4500
ENV PUBSUB_EMULATOR_PORT=8085
ENV FIRESTORE_EMULATOR_PORT=8086
ENV STORAGE_EMULATOR_PORT=8087
ENV HOSTING_EMULATOR_PORT=9000
ENV SINGLE_PROJECT_MODE=true

# Prevent Firebase from attempting to use the GCP metadata server for authentication
ENV NO_AUTH=true
ENV GCE_METADATA_HOST=""

ENTRYPOINT ["firebase", "emulators:start", "--project", "my-project"]

2. Kubernetes Deployment配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"firebase-emulators","app.kubernetes.io/managed-by":"tilt"},"name":"firebase-emulators","namespace":"default"},"spec":{"selector":{"matchLabels":{"app":"firebase-emulators"}},"strategy":{},"template":{"metadata":{"labels":{"app":"firebase-emulators","app.kubernetes.io/managed-by":"tilt","tilt.dev/pod-template-hash":"f23d48fe3d7b64dfd5de"}},"spec":{"containers":[{"image":"firebase-emulators-image:tilt-40e89bb0022ca2c3","imagePullPolicy":"IfNotPresent","name":"firebase-emulators","resources":{},"startupProbe":{"failureThreshold":12,"httpGet":{"host":"host.docker.internal","path":"/","port":4000},"initialDelaySeconds":3,"periodSeconds":5,"successThreshold":1,"timeoutSeconds":1}}]}}}}
  creationTimestamp: "2024-08-06T21:05:15Z"
  generation: 1
  labels:
    app: firebase-emulators
    app.kubernetes.io/managed-by: tilt
  name: firebase-emulators
  namespace: default
  resourceVersion: "95758"
  uid: 75db218c-aea3-4c18-bcb5-634372d3a79b
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: firebase-emulators
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: firebase-emulators
        app.kubernetes.io/managed-by: tilt
        tilt.dev/pod-template-hash: f23d48fe3d7b64dfd5de
    spec:
      containers:
      - image: firebase-emulators-image:tilt-40e89bb0022ca2c3
        imagePullPolicy: IfNotPresent
        name: firebase-emulators
        resources: {}
        startupProbe:
          failureThreshold: 12
          httpGet:
            host: host.docker.internal
            path: /
            port: 4000
            scheme: HTTP
          initialDelaySeconds: 3
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 1
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2024-08-06T21:05:40Z"
    lastUpdateTime: "2024-08-06T21:05:40Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2024-08-06T21:05:15Z"
    lastUpdateTime: "2024-08-06T21:05:40Z"
    message: ReplicaSet "firebase-emulators-7fbcf868cb" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

3. firebase.json配置 #

{
  "firestore": {
    "rules": "firestore.rules",
    "host": "127.0.0.1",
    "port": 8086
  },
  "storage": {
    "rules": "storage.rules",
    "host": "127.0.0.1",
    "port": 8087
  },
  "emulators": {
    "firestore": {
      "host": "127.0.0.1",
      "port": 8086
    },
    "pubsub": {
      "host": "127.0.0.1",
      "port": 8085
    },
    "storage": {
      "host": "127.0.0.1",
      "port": 8087
    },
    "ui": {
      "enabled": true,
      "host": "127.0.0.1",
      "port": 4000
    },
    "hosting": {
      "host": "127.0.0.1",
      "port": 9000
    },
    "hub": {
      "host": "127.0.0.1",
      "port": 4400
    },
    "logging": {
      "host": "127.0.0.1",
      "port": 4500
    },
    "singleProjectMode": true
  }
}

4. .firebaserc配置 #

{}

已尝试的操作 #

• 执行kubectl delete pvc --all && kubectl delete pv --all，未找到相关资源；
• 手动删除Docker Desktop中的镜像，无效；
• 在Dockerfile的ENTRYPOINT中添加--export-on-exit=/tmp/empty，无效；
• 部署前删除所有Tilt相关卷，问题依旧。

问题原因 #

1. Firebase模拟器默认缓存机制：Firebase模拟器启动时，会默认将数据快照保存在容器内的~/.cache/firebase/emulators目录，只要该目录未被清理，重启后就会自动加载旧数据。你的Dockerfile没有启动前清理该目录的步骤，导致缓存数据持续留存。

2. Tilt增量构建镜像的特性：Tilt默认采用增量构建，只有当Dockerfile或上下文文件变化时才会重建对应镜像层。如果本地项目目录的.cache文件夹未被.dockerignore排除，构建镜像时会将旧缓存数据打包进镜像，每次部署都会加载这些数据。

3. 缺少禁用自动导出的配置：当前配置未明确禁止模拟器退出时自动导出数据，导致每次停止模拟器时，数据都会被写入缓存目录，下次启动自动恢复。

内容的提问来源于stack exchange，提问作者Felipe Machado