Flask应用传递**kwargs报‘Unexpected keyword argument’但测试脚本正常

问题场景 #

在Flask应用中通过Worker类的run方法调用create_archive函数时，传递**kwargs触发TypeError: create_archive() got an unexpected keyword argument 'extension'，但相同逻辑在单独的test.py脚本中能正常运行。

相关代码 #

Worker类代码

import concurrent.futures as cf

class Worker:
    """Class for multiprocessing."""

    def __init__(self, workers=1):
        self.workers = workers

    def run(self, fn, files, **kwargs):
        print(kwargs)
        num = len(files)
        with cf.ProcessPoolExecutor(max_workers=self.workers) as executor:
            results = [executor.submit(fn, f, **kwargs) for f in files]
            for i, f in enumerate(cf.as_completed(results), start=1):
                print(f'({i}/{num}): {f.result()}')

create_archive函数定义

def create_archive(folder, extension='.tar.gz', format='PAX', remove_directory=False):
   # 相关逻辑

Flask路由中的调用

worker = Worker(workers=5)
worker.run(
    create_archive,
    valid_folders,
    extension=request.json['extension'],
    format=request.json['format'],
    remove_directory=str2bool(request.json['removeDir'])
)

关键现象 #

• run方法中print(kwargs)输出为{'extension': '.tar.gz', 'format': 'pax', 'remove_directory': False}，参数正确但仍报错
• 完整错误回溯：

concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/useridhere/miniconda3/lib/python3.11/concurrent/futures/process.py", line 256, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: create_archive() got an unexpected keyword argument 'extension'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask/app.py", line 1478, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask_socketio/__init__.py", line 43, in __call__
    return super(_SocketIOMiddleware, self).__call__(environ,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/engineio/middleware.py", line 74, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask/app.py", line 1458, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask/app.py", line 1455, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask/app.py", line 869, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask/app.py", line 867, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/venv/lib/python3.11/site-packages/flask/app.py", line 852, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/app.py", line 93, in create_archive
    worker.run(
  File "/Users/useridhere/repos/myproject/api/utilities.py", line 13, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/repos/myproject/api/worker.py", line 18, in run
    print(f'({i}/{num}): {f.result()}')
                          ^^^^^^^^^^^^^
  File "/Users/useridhere/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/useridhere/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
TypeError: create_archive() got an unexpected keyword argument 'extension'

排查分析 #

问题核心是多进程间的函数序列化与模块加载不一致：

1. Flask应用运行时，ProcessPoolExecutor会通过pickle序列化函数和参数传递给子进程。如果create_archive函数所在模块被修改后未重启Flask服务，子进程会加载旧版本的函数，导致参数签名不匹配。
2. 测试脚本是独立启动的进程，加载的是最新版本的函数，因此参数匹配正常。
3. 也可能存在同名函数冲突：子进程中导入的create_archive来自其他模块，而非你定义的目标函数，导致参数签名不符。

解决方案 #

1. 重启Flask服务：确保子进程加载的是最新版本的create_archive函数，解决代码更新后模块未重新加载的问题。
2. 使用绝对路径导入函数：调用worker.run时，用绝对路径导入目标函数（如from myproject.utils import create_archive），避免子进程导入错误的同名函数。
3. 验证函数签名一致性：在run方法中添加代码，打印函数参数签名，对比主进程和子进程的函数是否一致：

   import inspect
   print(inspect.signature(fn))
   

4. 避免嵌套定义函数：如果create_archive定义在Flask路由函数内部，子进程无法正确序列化嵌套函数，需将其移到模块顶层。

内容的提问来源于stack exchange，提问作者Simon1