部署Django应用至Google Cloud App Engine后网站持续加载无响应求助

部署完成后收到的提示信息：

Updating service [default] (this may take several minutes)...done.
Setting traffic split for service [default]...done.
Stopping version [sam-website-358409/default/20220825t151307].
Sent request to stop version [sam-website-358409/default/20220825t151307]. This operation may take some time to complete. If you would like to verify that it succeeded, run:
    $ gcloud app versions describe -s default 20220825t151307
until it shows that the version has stopped.
Deployed service [default] to [https://sam-website-358409.ew.r.appspot.com]

You can stream logs from the command line by running:
    $ gcloud app logs tail -s default

To view your application in the web browser run:
    $ gcloud app browse

执行日志查看命令：

gcloud app logs tail -s default

日志中反复出现的关键信息：

- This IS expected if you are initializing CamembertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
2022-08-25 16:33:36 default[20220825t175602]  - This IS NOT expected if you are initializing CamembertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
2022-08-25 16:33:39 default[20220825t175602]  /env/lib/python3.7/site-packages/sklearn/base.py:338: UserWarning: Trying to unpickle estimator LogisticRegression from version 0.24.2 when using version 1.0.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
2022-08-25 16:33:39 default[20220825t175602]  https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations
2022-08-25 16:33:39 default[20220825t175602]    UserWarning,
2022-08-25 16:33:39 default[20220825t175602]  /env/lib/python3.7/site-packages/spacy/util.py:865: UserWarning: [W095] Model 'fr_core_news_lg' (3.1.0) was trained with spaCy v3.1 and may not be 100% compatible with the current version (3.4.1). If you see errors or degraded performance, download a newer compatible model or retrain your custom model with the current spaCy version. For more details and available updates, run: python -m spacy validate
2022-08-25 16:33:39 default[20220825t175602]    warnings.warn(warn_msg)
2022-08-25 16:33:41 default[20220825t175602]  Some weights of the model checkpoint at camembert-base were not used when initializing CamembertModel: ['lm_head.bias', 'lm_head.dense.bias', 'lm_head.layer_norm.bias', 'lm_head.layer_norm.weight', 'lm_head.dense.weight', 'lm_head.decoder.weight']
2022-08-25 16:33:41 default[20220825t175602]  - This IS expected if you are initializing CamembertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
2022-08-25 16:33:41 default[20220825t175602]  - This IS NOT expected if you are initializing CamembertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
2022-08-25 16:33:44 default[20220825t175602]  [2022-08-25 16:33:44 +0000] [179] [INFO] Worker exiting (pid: 179)
2022-08-25 16:33:45 default[20220825t175602]  [2022-08-25 16:33:45 +0000] [191] [INFO] Booting worker with pid: 191
2022-08-25 16:33:46 default[20220825t175602]  2022-08-25 16:33:46.542163: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
2022-08-25 16:33:46 default[20220825t175602]  To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-08-25 16:33:48 default[20220825t175602]  2022-08-25 16:33:48.108067: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-08-25 16:33:48 default[20220825t175602]  2022-08-25 16:33:48.108101: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-08-25 16:33:50 default[20220825t175602]  [2022-08-25 16:33:50 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:132)
2022-08-25 16:33:50 default[20220825t175602]  [2022-08-25 16:33:50 +0000] [132] [INFO] Worker exiting (pid: 132)
2022-08-25 16:33:51 default[20220825t175602]  [2022-08-25 16:33:51 +0000] [1] [WARNING] Worker with pid 132 was terminated due to signal 9
2022-08-25 16:33:51 default[20220825t175602]  [2022-08-25 16:33:51 +0000] [144] [INFO] Booting worker with pid: 144
2022-08-25 16:33:52 default[20220825t175602]  2022-08-25 16:33:52.437860: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2022-08-25 16:33:52 default[20220825t175602]  2022-08-25 16:33:52.437899: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-08-25 16:33:52 default[20220825t175602]  2022-08-25 16:33:52.437933: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (0083c9528b79): /proc/driver/nvidia/version does not exist
2022-08-25 16:33:53 default[20220825t175602]  /env/lib/python3.7/site-packages/spacy/util.py:865: UserWarning: [W095] Model 'fr_core_news_lg' (3.1.0) was trained with spaCy v3.1 and may not be 100% compatible with the current version (3.4.1). If you see errors or degraded performance, download a newer compatible model or retrain your custom model with the current spaCy version. For more details and available updates, run: python -m spacy validate
2022-08-25 16:33:53 default[20220825t175602]    warnings.warn(warn_msg)
2022-08-25 16:33:53 default[20220825t175602]  2022-08-25 16:33:53.266169: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-08-25 16:33:53 default[20220825t175602]  2022-08-25 16:33:53.266266: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-08-25 16:33:55 default[20220825t175602]  2022-08-25 16:33:55.698315: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2022-08-25 16:33:55 default[20220825t175602]  2022-08-25 16:33:55.698350: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-08-25 16:33:55 default[20220825t175602]  2022-08-25 16:33:55.698380: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (2e597b32f6d7): /proc/driver/nvidia/version does not exist
2022-08-25 16:33:56 default[20220825t175602]  /env/lib/python3.7/site-packages/spacy/util.py:865: UserWarning: [W095] Model 'fr_core_news_lg' (3.1.0) was trained with spaCy v3.1 and may not be 100% compatible with the current version (3.4.1). If you see errors or degraded performance, download a newer compatible model or retrain your custom model with the current spaCy version. For more details and available updates, run: python -m spacy validate
2022-08-25 16:33:56 default[20220825t175602]    warnings.warn(warn_msg)
2022-08-25 16:34:10 default[20220825t175602]  Some weights of the model checkpoint at camembert-base were not used when initializing CamembertModel: ['lm_head.decoder.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.bias', 'lm_head.layer_norm.weight', 'lm_head.bias', 'lm_head.dense.weight']

当前使用的app.yaml配置：

runtime: python
env: flex
entrypoint: gunicorn -b :$PORT myteam.wsgi

beta_settings:
    cloud_sql_instances: sam-website-358409:europe-west9:sam-instance

runtime_config:
  python_version: 3.7

threadsafe: true
manual_scaling:
  instances: 2
network: {}

resources:
  cpu: 2
  memory_gb: 10
  disk_size_gb: 50
liveness_check:
  initial_delay_sec: 300
  check_interval_sec: 30
  timeout_sec: 4
  failure_threshold: 4
  success_threshold: 2
readiness_check:
  check_interval_sec: 5
  timeout_sec: 4
  failure_threshold: 2
  success_threshold: 2
  app_start_timeout_sec: 300

问题分析与解决建议 #

核心问题定位 #

日志中[CRITICAL] WORKER TIMEOUT (pid:132)和Worker with pid 132 was terminated due to signal 9是关键——Gunicorn worker进程因为超时被强制终止，导致请求无法被处理，网站持续加载。

具体修复步骤 #

1. 调整Gunicorn超时参数
   在entrypoint中添加超时配置，延长worker超时时间，避免进程被强制杀死：

   entrypoint: gunicorn -b :$PORT --timeout 300 myteam.wsgi
   

   设置300秒超时，匹配readiness_check中的app_start_timeout_sec，确保模型加载等耗时操作有足够时间完成。

2. 解决依赖版本不兼容问题

   • scikit-learn：本地训练模型用的是0.24.2，部署环境是1.0.2，版本差异可能导致模型加载异常。在requirements.txt中固定scikit-learn版本为0.24.2，或者重新用1.0.2训练模型并重新导出。
   • spaCy：fr_core_news_lg 3.1.0与当前spaCy 3.4.1不兼容，执行python -m spacy validate查看可用更新版本，在requirements.txt中指定兼容的模型版本，或者安装对应版本的spaCy。

3. 优化模型加载逻辑
   日志中反复出现CamembertModel和spaCy模型的加载信息，说明每次worker启动都在重新加载模型，这会导致启动时间过长、资源消耗大。可以：

   • 把模型加载移到Django启动时（如apps.py的ready方法），确保模型只加载一次。
   • 使用Gunicorn的--preload参数，在master进程中预加载模型，避免每个worker重复加载：

     entrypoint: gunicorn -b :$PORT --timeout 300 --preload myteam.wsgi
     

4. **GPU相关