Snakemake多通配符(id/srr)输入文件缺失问题求解

现有一段用于将文件重命名为Cell Ranger要求格式的Snakemake代码，执行时出现错误：部分任务会尝试查找不存在的{id}与{srr}组合对应的输入文件（例如tissues/A/atac_seq/SRR123/SRR123_1.fastq.gz实际仅存在于id为B的路径下）。

原代码如下：

rule all:
  input:
      expand("tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R1_001.fastq.gz", id =IDs, srr = df[df['library_type']=='Chromatin Accessibility']['Library']),
      expand("tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R2_001.fastq.gz", id =IDs, srr = df[df['library_type']=='Chromatin Accessibility']['Library']),
      expand("tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R3_001.fastq.gz", id =IDs, srr = df[df['library_type']=='Chromatin Accessibility']['Library'])

rule rename_atac_files:
    input:
      "tissues/{id}/atac_seq/{srr}/{srr}_1.fastq.gz",
      "tissues/{id}/atac_seq/{srr}/{srr}_2.fastq.gz",
      "tissues/{id}/atac_seq/{srr}/{srr}_3.fastq.gz"
    output:
      "tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R1_001.fastq.gz",
      "tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R2_001.fastq.gz",
      "tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R3_001.fastq.gz"
    shell:
      """
      mv tissues/{wildcards.id}/atac_seq/{wildcards.srr}/{wildcards.srr}_1.fastq.gz tissues/{wildcards.id}/atac_seq/{wildcards.srr}/{wildcards.srr}_S1_L001_R1_001.fastq.gz
      mv tissues/{wildcards.id}/atac_seq/{wildcards.srr}/{wildcards.srr}_2.fastq.gz tissues/{wildcards.id}/atac_seq/{wildcards.srr}/{wildcards.srr}_S1_L001_R2_001.fastq.gz
      mv tissues/{wildcards.id}/atac_seq/{wildcards.srr}/{wildcards.srr}_3.fastq.gz tissues/{wildcards.id}/atac_seq/{wildcards.srr}/{wildcards.srr}_S1_L001_R3_001.fastq.gz
      """

错误提示：

Error:Missing input files for rule rename_atac_files:

需求：让Snakemake仅执行存在的{id}和{srr}组合的任务，同时保持目录结构不变。

解决方案 #

1. 收集实际存在的有效(id, srr)配对 #

在Snakefile开头添加Python代码，遍历文件系统，筛选出同时存在三个输入fastq文件的(id, srr)组合：

import os

valid_pairs = []
# 遍历所有tissues下的id目录
for id_dir in os.listdir("tissues"):
    id_path = os.path.join("tissues", id_dir)
    if not os.path.isdir(id_path):
        continue
    # 检查当前id目录下是否有atac_seq子目录
    atac_dir = os.path.join(id_path, "atac_seq")
    if not os.path.isdir(atac_dir):
        continue
    # 遍历atac_seq下的srr目录
    for srr_dir in os.listdir(atac_dir):
        srr_path = os.path.join(atac_dir, srr_dir)
        if not os.path.isdir(srr_path):
            continue
        # 验证三个输入文件是否都存在
        r1 = os.path.join(srr_path, f"{srr_dir}_1.fastq.gz")
        r2 = os.path.join(srr_path, f"{srr_dir}_2.fastq.gz")
        r3 = os.path.join(srr_path, f"{srr_dir}_3.fastq.gz")
        if os.path.exists(r1) and os.path.exists(r2) and os.path.exists(r3):
            valid_pairs.append( (id_dir, srr_dir) )

# 拆分出对应id和srr的列表，用于后续expand
valid_ids = [pair[0] for pair in valid_pairs]
valid_srrs = [pair[1] for pair in valid_pairs]

2. 修改rule all的输入，确保仅生成有效组合 #

使用expand的zip参数，让id和srr按配对展开，避免生成所有可能的笛卡尔积：

rule all:
    input:
        expand("tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R1_001.fastq.gz", zip, id=valid_ids, srr=valid_srrs),
        expand("tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R2_001.fastq.gz", zip, id=valid_ids, srr=valid_srrs),
        expand("tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R3_001.fastq.gz", zip, id=valid_ids, srr=valid_srrs)

3. 简化shell命令（可选） #

直接引用input和output的索引，减少冗余代码，避免路径错误：

rule rename_atac_files:
    input:
        "tissues/{id}/atac_seq/{srr}/{srr}_1.fastq.gz",
        "tissues/{id}/atac_seq/{srr}/{srr}_2.fastq.gz",
        "tissues/{id}/atac_seq/{srr}/{srr}_3.fastq.gz"
    output:
        "tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R1_001.fastq.gz",
        "tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R2_001.fastq.gz",
        "tissues/{id}/atac_seq/{srr}/{srr}_S1_L001_R3_001.fastq.gz"
    shell:
        """
        mv {input[0]} {output[0]}
        mv {input[1]} {output[1]}
        mv {input[2]} {output[2]}
        """

内容的提问来源于stack exchange，提问作者user25645294