Snakemake
💡 Please have a look at your tutorial for beginners: tutorials/snakemake
How to deal with the dependencies?#
Overview#
Snakemake can deal with several dependencies system:
- Conda
- Module
- Singularity
- Docker (not available on the IFB Core HPC Cluster)
Example#
Requirements#
The data are available in a Zenodo record: doi.org/10.5281/zenodo.3997236
The file structure:
./
├── data
├── SRR3099585_chr18.fastq.gz
├── SRR3099586_chr18.fastq.gz
├── SRR3099587_chr18.fastq.gz
├── envs
├── fastqc-0.11.9.yml
├── multiqc-1.9.yml
├── multiqc.smk
├── [...]
The snakefile#
multiqc.smk
SAMPLES = ["SRR3099585_chr18","SRR3099586_chr18","SRR3099587_chr18"]
rule all:
input:
expand("FastQC/{sample}_fastqc.html", sample=SAMPLES),
"multiqc_report.html"
rule multiqc:
output:
"multiqc_report.html"
input:
expand("FastQC/{sample}_fastqc.zip", sample = SAMPLES)
log:
std="Logs/multiqc.std",
err="Logs/multiqc.err"
conda:
"envs/multiqc-1.9.yml"
container:
"https://depot.galaxyproject.org/singularity/multiqc:1.10.1--pyhdfd78af_1"
envmodules:
"multiqc/1.9"
shell: "multiqc {input} 1>{log.std} 2>{log.err}"
rule fastqc:
output:
"FastQC/{sample}_fastqc.zip",
"FastQC/{sample}_fastqc.html"
input:
"data/{sample}.fastq.gz"
log:
std="Logs/{sample}_fastqc.std",
err="Logs/{sample}_fastqc.err"
conda:
"envs/fastqc-0.11.9.yml"
container:
"docker://biocontainers/fastqc:v0.11.9_cv8"
envmodules:
"fastqc/0.11.9"
shell: "fastqc --outdir FastQC/ {input} 1>{log.std} 2>{log.err}"
[optional] The conda env files#
envs/fastqc-0.11.9.yml
channels:
- conda-forge
- bioconda
- default
dependencies:
- bioconda::fastqc=0.11.9
envs/multiqc-1.9.yml
channels:
- conda-forge
- bioconda
- default
dependencies:
- bioconda::multiqc=1.9
Run#
With the same Snakefile above, you can test 3 different dependencies management systems.
💡 The Env Module is the quicker solution if you are using the IFB Core HPC Cluster, but maybe not the more portable.
Use Module#
module purge; module load snakemake slurm-drmaa
# cleanup
snakemake -c 1 -s ex1_o8.smk --delete-all-output; rm -rf multiqc_*
snakemake --drmaa --jobs=3 -s ex1_o8.smk --use-envmodule
Use Conda#
module purge; module load snakemake slurm-drmaa conda
# cleanup
snakemake -c 1 -s ex1_o8.smk --delete-all-output; rm -rf multiqc_*
snakemake --drmaa --jobs=3 -s ex1_o8.smk --use-conda
Use Singularity#
module purge; module load snakemake slurm-drmaa singularity
# cleanup
snakemake -c 1 -s ex1_o8.smk --delete-all-output; rm -rf multiqc_*
snakemake --drmaa --jobs=3 -s ex1_o8.smk --use-singularity