非默认调度器下Dask.distributed性能报告失效，如何全局注册调度器？

我尝试为基于LocalCluster的Client生成性能报告，却发现仅当将Client通过set_as_default=True设置为默认调度器时，性能报告功能才能正常工作。

代码示例 #

import dask.distributed as dd

cluster = dd.LocalCluster(n_workers=2, threads_per_worker=4, memory_limit='5GiB')
client = dd.Client(cluster, set_as_default=False)
    
with dd.performance_report(filename='dask-report.html'):
    print(client)
    print(cluster)

运行错误 #

Traceback (most recent call last):
  File “bla/test.py”, line 9, in <module>
    with dd.performance_report(filename='dask-report.html'):
  File "bla/lib/python3.10/site-packages/distributed/client.py", line 5497, in __enter__
    get_client().sync(self.__aenter__)
  File "bla/lib/python3.10/site-packages/distributed/worker.py", line 2771, in get_client
    raise ValueError("No global client found and no address provided")
ValueError: No global client found and no address provided

请问我是否遗漏了什么配置？有没有办法将调度器全局注册？

解决方案 #

原因分析 #

dd.performance_report上下文管理器默认会调用get_client()获取全局默认的Client实例，当你设置set_as_default=False时，该Client不会被注册为全局实例，导致管理器找不到可用Client而抛出错误。

方法一：显式指定Client给性能报告 #

无需修改set_as_default的设置，直接将Client对象传入performance_report的client参数即可：

import dask.distributed as dd

cluster = dd.LocalCluster(n_workers=2, threads_per_worker=4, memory_limit='5GiB')
client = dd.Client(cluster, set_as_default=False)
    
with dd.performance_report(filename='dask-report.html', client=client):
    # 执行你的Dask任务
    print(client)
    print(cluster)

方法二：手动注册全局Client #

如果希望后续所有Dask操作自动使用该Client，可手动调用client.set_as_default()将其注册为全局实例：

import dask.distributed as dd

cluster = dd.LocalCluster(n_workers=2, threads_per_worker=4, memory_limit='5GiB')
client = dd.Client(cluster, set_as_default=False)
client.set_as_default()  # 手动注册为全局Client
    
with dd.performance_report(filename='dask-report.html'):
    # 执行你的Dask任务
    print(client)
    print(cluster)

方法三：结合Client上下文管理器 #

利用Client自身的上下文管理器限定作用域，同时为性能报告指定该Client：

import dask.distributed as dd

cluster = dd.LocalCluster(n_workers=2, threads_per_worker=4, memory_limit='5GiB')
    
with dd.Client(cluster, set_as_default=False) as client:
    with dd.performance_report(filename='dask-report.html', client=client):
        # 执行你的Dask任务
        print(client)
        print(cluster)

内容的提问来源于stack exchange，提问作者Helmut