运行Snakemake调用Sortmerna遇段错误，求修复方案

测试Snakemake流程（后续需处理200余个文件，当前仅用单个文件测试）时，调用Sortmerna工具触发Segmentation fault（段错误）。已确认config_samples.json中路径引用无误，但问题仍未解决。

错误信息 #

/usr/bin/bash: line 1:  2047 Segmentation fault      sortmerna --ref /data/CEM/buetowlab/controlled_access/rtRNA/mask_HBA_rtRNAs.fasta --reads /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L007_R1_001.fastq.gz --reads /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L008_R1_001.fastq.gz --workdir /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/ --idx-dir /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/idx_CTV17_102/ --kvdb /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/kvdb_CTV17_102/ --readb /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/readb_CTV17_102/ --num_alignments 1 --fastx --aligned /scratch/ebogen/transcriptome/samples_156/sortmerna/maskedRNAs/CTV17_102/CTV17_102 --other /scratch/ebogen/transcriptome/samples_156/sortmerna/non_maskedRNAs/CTV17_102/CTV17_102 --paired_in --threads 10
[Thu May 18 14:29:10 2023]
Error in rule fastqc_analysis:
    jobid: 0
    input: /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L007_R1_001.fastq.gz, /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L008_R1_001.fastq.gz
    output: /scratch/ebogen/transcriptome/samples_156/sortmerna/maskedRNAs/CTV17_102.fq.gz, /scratch/ebogen/transcriptome/samples_156/sortmerna/non_maskedRNAs/CTV17_102.fq.gz
    shell:
        sortmerna --ref /data/CEM/buetowlab/controlled_access/rtRNA/mask_HBA_rtRNAs.fasta --reads /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L007_R1_001.fastq.gz --reads /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L008_R1_001.fastq.gz --workdir /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/ --idx-dir /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/idx_CTV17_102/ --kvdb /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/kvdb_CTV17_102/ --readb /scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/readb_CTV17_102/ --num_alignments 1 --fastx --aligned /scratch/ebogen/transcriptome/samples_156/sortmerna/maskedRNAs/CTV17_102/CTV17_102 --other /scratch/ebogen/transcriptome/samples_156/sortmerna/non_maskedRNAs/CTV17_102/CTV17_102 --paired_in --threads 10
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Snakefile代码 #

import os

configfile: "config_samples.json"

#what the final files will be called. Expand allows for the use of one line that can be filled in with all the options ie sample_name and run
#This should match the output of the final rule you have
rule all:
    input:
        expand("/scratch/ebogen/transcriptome/samples_156/sortmerna/maskedRNAs/{sample_name}.fq.gz", sample_name = config["one_names"]),
        expand("/scratch/ebogen/transcriptome/samples_156/sortmerna/non_maskedRNAs/{sample_name}.fq.gz", sample_name = config["one_names"])

rule fastqc_analysis:
    input:
        r1 = lambda wildcards: os.path.join(config[wildcards.sample_name]["fq_path"], config[wildcards.sample_name]["fq1"]),
        r2 = lambda wildcards: os.path.join(config[wildcards.sample_name]["fq_path"], config[wildcards.sample_name]["fq2"])
    output:
        "/scratch/ebogen/transcriptome/samples_156/sortmerna/maskedRNAs/{sample_name}.fq.gz",
        "/scratch/ebogen/transcriptome/samples_156/sortmerna/non_maskedRNAs/{sample_name}.fq.gz"       
    params: 
        #num_alignments = 1,
        ref_file = "/data/CEM/buetowlab/controlled_access/rtRNA/mask_HBA_rtRNAs.fasta",
        ali = "/scratch/ebogen/transcriptome/samples_156/sortmerna/maskedRNAs/{sample_name}/{sample_name}",
        oth = "/scratch/ebogen/transcriptome/samples_156/sortmerna/non_maskedRNAs/{sample_name}/{sample_name}",
        work_dir = "/scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/",
        idx_dir = "/scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/idx_{sample_name}/",
        kvdb = "/scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/kvdb_{sample_name}/",
        readb = "/scratch/ebogen/transcriptome/samples_156/sortmerna/work_dir/readb_{sample_name}/"
        
    shell:
      "sortmerna --ref {params.ref_file} --reads {input.r1} --reads {input.r2} --workdir {params.work_dir} --idx-dir {params.idx_dir} --kvdb {params.kvdb} --readb {params.readb} --num_alignments 1 --fastx --aligned {params.ali} --other {params.oth} --paired_in --threads 10"

配置文件（config_samples.json）内容 #

{

  "Comment_Sample_Info": "The following section lists the samples that are to be analyzed",
                    
  "sample_names":  ["CTV17_101", "CTV17_102"],
                    
    "CTV17_101": {
        "fq_path": "/scratch/ebogen/transcriptome/samples_156/L0078/",
        "fq1": "CTV17-101_S274_L007_R1_001.fastq.gz",
        "fq2": "CTV17-101_S274_L008_R1_001.fastq.gz"
    },
    "CTV17_102": {
        "fq_path": "/scratch/ebogen/transcriptome/samples_156/L0078/",
        "fq1": "CTV17-102_S274_L007_R1_001.fastq.gz",
        "fq2": "CTV17-102_S274_L008_R1_001.fastq.gz"
    }
}

排查与修复建议 #

• 独立运行Sortmerna命令：把错误信息中的完整Sortmerna命令复制到终端直接执行，排除Snakemake参数传递的问题。如果单独运行也报错，问题出在Sortmerna本身或输入参数/文件。
• 提前创建输出目录：Sortmerna不会自动生成嵌套子目录（比如maskedRNAs/CTV17_102/），缺失目录可能导致内存异常。修改Snakemake的shell命令，先创建所需目录：

  "mkdir -p {params.ali%/*} {params.oth%/*} {params.idx_dir} {params.kvdb} {params.readb} && sortmerna --ref {params.ref_file} --reads {input.r1} --reads {input.r2} --workdir {params.work_dir} --idx-dir {params.idx_dir} --kvdb {params.kvdb} --readb {params.readb} --num_alignments 1 --fastx --aligned {params.ali} --other {params.oth} --paired_in --threads 10"
  

• 降低线程数：当前使用10线程，部分Sortmerna版本多线程存在内存冲突，先尝试改为--threads 1，看是否仍出现段错误。
• 检查文件完整性：
  • 验证参考序列文件mask_HBA_rtRNAs.fasta格式是否正确，无截断或非法字符。
  • 用zcat查看fastq.gz文件，确认压缩包未损坏：

    zcat /scratch/ebogen/transcriptome/samples_156/L0078/CTV17-102_S275_L007_R1_001.fastq.gz | head -20
    

• 更新Sortmerna版本：旧版本Sortmerna存在已知的内存泄漏或段错误bug，升级到最新稳定版可解决部分问题。
• 调整内存资源：段错误常因内存不足触发，若在集群环境运行，申请更大内存配额；本地运行则关闭其他高内存占用程序。

内容的提问来源于stack exchange，提问作者Daisy