Azure Function App中运行Python Notebook遇Kernel启动失败问题

本地运行Notebook、本地启动函数触发执行均正常，但部署到Azure Function App云端后，出现内核启动失败错误：

2024-06-16T03:50:35Z [Error] Error occurred while starting new kernel client for kernel 9e5c52cb-04d7-4065-b0e7-8c01a8f4bf80: Kernel died before replying to kernel_info

2024-06-16T03:50:35Z   [Error]   Error occurred while starting new kernel client for kernel 9e5c52cb-04d7-4065-b0e7-8c01a8f4bf80: Kernel died before replying to kernel_info
2024-06-16T03:50:35Z   [Error]   Error encountered during the execution of manually_generate_report function

Traceback (most recent call last):
File "/home/site/wwwroot/src/report_generation/main.py", line 86, in manually_generate_report
  body = execute_notebook(**parameters, notebook_file_name=notebook_file_name)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/site/wwwroot/src/report_generation/service.py", line 31, in execute_notebook
  nb = pm.execute_notebook(
        ^^^^^^^^^^^^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/papermill/execute.py", line 116, in execute_notebook
  nb = papermill_engines.execute_notebook_with_engine(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/papermill/engines.py", line 48, in execute_notebook_with_engine
  return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/papermill/engines.py", line 370, in execute_notebook
  cls.execute_managed_notebook(nb_man, kernel_name, log_output=log_output, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/papermill/engines.py", line 442, in execute_managed_notebook
  return PapermillNotebookClient(nb_man, **final_kwargs).execute()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/papermill/clientwrap.py", line 43, in execute
  with self.setup_kernel(**kwargs):
File "/usr/local/lib/python3.11/contextlib.py", line 137, in __enter__
  return next(self.gen)
          ^^^^^^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/nbclient/client.py", line 603, in setup_kernel
  self.start_new_kernel_client()
File "/home/site/wwwroot/.python_packages/lib/site-packages/jupyter_core/utils/__init__.py", line 165, in wrapped
  return loop.run_until_complete(inner)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
  return future.result()
          ^^^^^^^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/nbclient/client.py", line 566, in async_start_new_kernel_client
  await ensure_async(self.kc.wait_for_ready(timeout=self.startup_timeout))
File "/home/site/wwwroot/.python_packages/lib/site-packages/jupyter_core/utils/__init__.py", line 198, in ensure_async
  result = await obj
            ^^^^^^^^^
File "/home/site/wwwroot/.python_packages/lib/site-packages/jupyter_client/client.py", line 203, in _async_wait_for_ready
  raise RuntimeError(msg)

RuntimeError: Kernel died before replying to kernel_info

执行Notebook的代码：

import papermill as pm

nb = pm.execute_notebook(
        input_path=os.path.join(NOTEBOOK_FOLDER_PATH, f"{notebook_file_name}.ipynb"),
        output_path=None,
        parameters=dict(
            **kwargs,
        ),
    )

依赖清单requirements.txt：

azure-functions
pandas
requests
openpyxl
pymongo
jupyter
python-dotenv
numpy
matplotlib
azure-identity
azure-monitor-query
papermill
postmarker
httpx
ipykernel

• 升级Function App运行计划：
  消费计划/基础计划资源受限，Jupyter内核启动需要足够内存，建议切换到Premium计划（EP1及以上实例），并在配置中设置WEBSITE_MEMORY_LIMIT_MB=2048，避免内存不足导致内核崩溃。

• 显式配置内核与环境变量：
  在pm.execute_notebook调用中指定内核名称，并配置PYTHONPATH确保依赖被正确加载：

  nb = pm.execute_notebook(
      input_path=os.path.join(NOTEBOOK_FOLDER_PATH, f"{notebook_file_name}.ipynb"),
      output_path=None,
      parameters=dict(**kwargs),
      kernel_name="python3",
      env={
          "PYTHONPATH": "/home/site/wwwroot:/home/site/wwwroot/.python_packages/lib/site-packages"
      }
  )
  

• 固定依赖版本并注册内核：
  在requirements.txt中指定ipykernel具体版本（如ipykernel==6.29.4），避免版本兼容问题。同时在Azure门户的函数应用配置中设置启动命令：

  python -m ipykernel install --user --name python3 --display-name "Python 3"
  

  确保内核被Jupyter正确识别。

• 修正Notebook路径引用：
  检查Notebook中所有本地资源路径，使用绝对路径替代相对路径，避免云端环境下找不到文件导致内核启动失败。

• 延长内核启动超时：
  增加papermill的启动超时时间，给内核足够的初始化时间：

  nb = pm.execute_notebook(
      input_path=os.path.join(NOTEBOOK_FOLDER_PATH, f"{notebook_file_name}.ipynb"),
      output_path=None,
      parameters=dict(**kwargs),
      startup_timeout=120  # 延长至120秒
  )
  

内容的提问来源于stack exchange，提问作者Ashraf Khan