Skip to content
Snippets Groups Projects
Commit 1fe7d8b3 authored by Gervaise Henry's avatar Gervaise Henry :cowboy:
Browse files

Use 1 less process for samtools threading and limit memto 75% of available

parent 9e121dac
2 merge requests!65Develop,!64Resolve "samtools sort: couldn't allocate memory for bam_mem"
Pipeline #9066 canceled with stages
in 2 hours, 15 minutes, and 27 seconds
......@@ -2,7 +2,7 @@
**User Facing**
**Background**
* Add memory limit per thread for samtools sort (#108)
* Add memory limit (75%) per thread for samtools sort (#108)
* Remove parsing restrictions for submitted stranded/spike/species (#105, #106)
* Pass unidentified ends instead of overwriting it as unknown
* Move fastqc process before trim to catch fastq errors (#107)
......@@ -13,7 +13,8 @@
* Handle blank submitted endness better
* Don't use file.csv from inputBag to parse manual endness, use counted from getData
* Detect malformed fastq's (#107)
* Restrict sampled alignment process to use >32GB nodes on BioHPC
* Restrict sampled alignment process to use >32GB nodes on BioHPC (#108)
* Use nproc**-1** for alignment processes (#108)
*Known Bugs*
* Override params (inputBag, fastq, species) aren't checked for integrity
......
......@@ -873,9 +873,10 @@ process alignSampleData {
# sort the bam file using Samtools
echo -e "LOG: sorting the bam file" >> ${repRID}.${ref}.alignSampleData.log
proc=\$(expr `nproc` - 1)
mem=\$(vmstat -s -S K | grep 'total memory' | grep -o '[0-9]*')
mem=\$(expr \${mem} / `nproc` \\* 85 / 100)
samtools sort -@ `nproc` -m \${mem}K -O BAM -o ${ref}.sampled.sorted.bam ${ref}.sampled.bam
mem=\$(expr \${mem} / \${proc} \\* 85 / 100)
samtools sort -@ \${proc} -m \${mem}K -O BAM -o ${ref}.sampled.sorted.bam ${ref}.sampled.bam
# index the sorted bam using Samtools
echo -e "LOG: indexing sorted bam file" >> ${repRID}.${ref}.alignSampleData.log
......@@ -1576,9 +1577,10 @@ process alignData {
# sort the bam file using Samtools
echo -e "LOG: sorting the bam file" >> ${repRID}.align.log
proc=\$(expr `nproc` - 1)
mem=\$(vmstat -s -S K | grep 'total memory' | grep -o '[0-9]*')
mem=\$(expr \${mem} / `nproc` \\* 85 / 100)
samtools sort -@ `nproc` -m \${mem}K -O BAM -o ${repRID}.sorted.bam ${repRID}.bam
mem=\$(expr \${mem} / \${proc} \\* 75 / 100)
samtools sort -@ \${proc} -m \${mem}K -O BAM -o ${repRID}.sorted.bam ${repRID}.bam
# index the sorted bam using Samtools
echo -e "LOG: indexing sorted bam file" >> ${repRID}.align.log
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment