Skip to content
Snippets Groups Projects
Commit c59b5eb4 authored by Gervaise Henry's avatar Gervaise Henry :cowboy:
Browse files

Only singularity pull 5 containers at a time

parent aa955a65
Branches
Tags
2 merge requests!76Develop,!71Seqwho
Pipeline #9432 canceled with stages
in 3 minutes and 6 seconds
......@@ -33,7 +33,7 @@ img_cache:
stage: singularity
script:
- mkdir -p ${dir}cache/
- cat nextflow.config | grep -oP "container = \K.*" | tr -d "'" | sort | uniq | xargs -P 10 -I {} singularity pull --dir ${dir} 'docker://'{} || true
- cat nextflow.config | grep -oP "container = \K.*" | tr -d "'" | sort | uniq | xargs -P 5 -I {} singularity pull --dir ${dir} 'docker://'{} || true
collect:
stage: versions
......
......@@ -38,6 +38,8 @@
* Override params (inputBag, fastq, species) aren't checked for integrity
* Authentication files and tokens must be active (active auth client) for the duration of the pipeline run (until long-lived token utilization included)
* Check for outputBag in hatrac doesn't check for any uploaded by chaise
* CI container cache will fail if cache folder is not owned by CI runner user
* CI container cache will not error if container failed to pull
<hr>
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment