Skip to content
Snippets Groups Projects
Commit c59b5eb4 authored by Gervaise Henry's avatar Gervaise Henry :cowboy:
Browse files

Only singularity pull 5 containers at a time

parent aa955a65
Branches
Tags
2 merge requests!76Develop,!71Seqwho
Pipeline #9432 canceled with stages
in 3 minutes and 6 seconds
...@@ -33,7 +33,7 @@ img_cache: ...@@ -33,7 +33,7 @@ img_cache:
stage: singularity stage: singularity
script: script:
- mkdir -p ${dir}cache/ - mkdir -p ${dir}cache/
- cat nextflow.config | grep -oP "container = \K.*" | tr -d "'" | sort | uniq | xargs -P 10 -I {} singularity pull --dir ${dir} 'docker://'{} || true - cat nextflow.config | grep -oP "container = \K.*" | tr -d "'" | sort | uniq | xargs -P 5 -I {} singularity pull --dir ${dir} 'docker://'{} || true
collect: collect:
stage: versions stage: versions
......
...@@ -38,6 +38,8 @@ ...@@ -38,6 +38,8 @@
* Override params (inputBag, fastq, species) aren't checked for integrity * Override params (inputBag, fastq, species) aren't checked for integrity
* Authentication files and tokens must be active (active auth client) for the duration of the pipeline run (until long-lived token utilization included) * Authentication files and tokens must be active (active auth client) for the duration of the pipeline run (until long-lived token utilization included)
* Check for outputBag in hatrac doesn't check for any uploaded by chaise * Check for outputBag in hatrac doesn't check for any uploaded by chaise
* CI container cache will fail if cache folder is not owned by CI runner user
* CI container cache will not error if container failed to pull
<hr> <hr>
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment