Snakemake独立工作流作为模块导入时执行失败问题求助

问题场景 #

有两个可独立运行的Snakemake工作流：

• 第一个从配置文件读取输入，生成.csv输出文件
• 第二个以该.csv文件作为输入
  尝试通过模块导入方式将二者合并到第三个Snakefile，执行snakemake --cores 1 -n时触发错误，使用Snakemake版本8.6，目录结构为combined_pipelines下包含metadata、download、run_pipe（合并工作流目录）三个子目录。

合并用Snakefile代码 #

from snakemake.utils import min_version
min_version("6.0")


module metadata:
    snakefile:
        "../../metadata/workflow/metadata.smk"
    replace_prefix: '../../metadata'
    config:
        'config/metadata.yaml'

use rule * from metadata as metadata_*

module download:
    snakefile:
        "../../download/workflow/downloads.smk"
    replace_prefix: "../../download"
    config:
        'config/download.yaml'

use rule * from download as download_*

rule all:
    input:
        rules.metadata_all.input,
        rules.download_all.input
    default_target: True

报错信息 #

Traceback (most recent call last):
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/cli.py", line 1961, in args_to_api
    dag_api = workflow_api.dag(
              ^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/api.py", line 328, in dag
    return DAGApi(
           ^^^^^^^
  File "<string>", line 6, in __init__
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/api.py", line 438, in __post_init__
    self.workflow_api._workflow.dag_settings = self.dag_settings
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/api.py", line 385, in _workflow
    workflow.include(
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/workflow.py", line 1391, in include
    exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
  File "/home/bayraktar/PycharmProjects/combined_pipelines/run_pipe/workflow/Snakefile", line 16, in <module>
    "../../download/workflow/downloads.smk"
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/workflow.py", line 2130, in decorate
    module.use_rules(
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/modules.py", line 82, in use_rules
    modifier = WorkflowModifier(
               ^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/modules.py", line 207, in __init__
    self.path_modifier = PathModifier(replace_prefix, prefix, workflow)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/path_modifier.py", line 32, in __init__
    for prefix, replacement in replace_prefix.items():
                               ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'items'

错误原因 #

Snakemake 8.x版本中，replace_prefix参数要求为字典格式（键值对，用于映射模块内的路径前缀到当前工作流的对应路径），但代码中传入的是字符串，导致程序尝试调用字符串的items()方法时触发AttributeError。

修复方案 #

将两个模块的replace_prefix修改为字典格式，键为模块内部使用的基础路径前缀（通常为.，即模块Snakefile所在目录的相对路径根），值为模块根目录相对于当前合并Snakefile的路径：

修正后的Snakefile代码 #

from snakemake.utils import min_version
min_version("6.0")


module metadata:
    snakefile:
        "../../metadata/workflow/metadata.smk"
    replace_prefix: {".": "../../metadata"}
    config:
        'config/metadata.yaml'

use rule * from metadata as metadata_*

module download:
    snakefile:
        "../../download/workflow/downloads.smk"
    replace_prefix: {".": "../../download"}
    config:
        'config/download.yaml'

use rule * from download as download_*

rule all:
    input:
        rules.metadata_all.input,
        rules.download_all.input
    default_target: True

补充说明 #

如果原工作流中使用了特定的路径前缀（比如../data），只需将字典的键替换为该前缀，值替换为当前合并工作流中对应的绝对/相对路径即可。

内容的提问来源于stack exchange，提问作者eggrep