GAE重新部署版本时请求被路由至默认服务的Pub/Sub问题

Alright, let's break down your core issue first: When you redeploy your target microservice on App Engine, during the window where new instances are spinning up, Pub/Sub push requests get routed to your default service instead. Since your default service returns a 2xx status for _ah/push_handlers/* paths, Pub/Sub assumes the message was successfully handled and stops retrying—meaning your target service never receives those messages. That's a frustrating edge case, but there are solid, actionable fixes for it.

Here are the most effective solutions, ordered by robustness and practicality:

1. Pin Push Subscriptions to a Specific Service Version/Instance #

The cleanest long-term fix is to configure your Pub/Sub push subscription to target a specific version of your service, rather than the generic service endpoint. This ensures Pub/Sub only sends requests to that version's instances, bypassing App Engine's default routing entirely—even during deployments when traffic is being shifted.

When creating or updating your subscription, use a version-specific endpoint like:

https://[VERSION]-dot-[SERVICE]-dot-[PROJECT_ID].appspot.com/_ah/push_handlers/your-handler-path

Instead of the generic https://[SERVICE]-dot-[PROJECT_ID].appspot.com/... URL. This locks Pub/Sub's delivery to your intended service version, eliminating the chance of misrouting to the default service.

2. Adjust Default Service's _ah/push_handlers/* Logic #

If you can't use version-specific endpoints, modify your default service to reject push requests that aren't meant for it. Return a 404 Not Found or 403 Forbidden for these requests—this tells Pub/Sub the delivery failed, so it will retry until your target service's instances are ready to receive.

You can identify misdirected requests by checking the X-Goog-PubSub-Topic header. For example, in a Java Servlet:

protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    String incomingTopic = request.getHeader("X-Goog-PubSub-Topic");
    // Only process topics explicitly intended for the default service
    String defaultServiceTopic = "projects/your-project-id/topics/default-service-topic";
    if (!defaultServiceTopic.equals(incomingTopic)) {
        response.setStatus(HttpServletResponse.SC_NOT_FOUND);
        return;
    }
    // Handle default service's own Pub/Sub messages here
    // ...
}

3. Temporarily Switch to Pull Mode During Deployments (Quick Emergency Fix) #

For occasional deployments, you can pause push deliveries by switching your subscription to pull mode before deploying, then switch back once your target service is fully up and running.

Use these gcloud commands:

# Disable push mode temporarily
gcloud pubsub subscriptions update YOUR_SUBSCRIPTION_NAME --push-endpoint=""
# After deployment completes and instances are healthy, re-enable push mode
gcloud pubsub subscriptions update YOUR_SUBSCRIPTION_NAME --push-endpoint="https://your-service-dot-your-project.appspot.com/_ah/push_handlers/your-path"

This is a manual workaround, so it's not ideal for frequent deployments, but it works in a pinch to avoid message loss.

4. Optimize App Engine Deployment Flow to Reduce Routing Window #

Minimize the time window where requests might route to the default service by:

• Using traffic splitting: Roll out the new version gradually (e.g., start with 10% traffic) while keeping old instances running until the new ones are marked healthy.
• Configuring warmup requests: Add a warmup handler in your target service's app.yaml to pre-load dependencies and initialize instances before they receive traffic.
• Setting minimum idle instances: Use automatic_scaling in your app.yaml to keep a small number of idle instances running, eliminating cold-start delays during deployments.

内容的提问来源于stack exchange，提问作者S.Ivanova