Commit 931b5ba6 authored by Peng Lian's avatar Peng Lian
Browse files

Init test version 2.0.0

parent ce376025
Pipeline #12265 failed with stages
......@@ -8,3 +8,13 @@ Initial Version.
Version 0.0.3 for astrocyte release. Tested on astrocyte live.
2021-08-04
----------
Version 0.0.9 for testing of astrocyte 1.0.0 is released. In this version singularity was enabled to run the workflow.
2022-08-09
----------
Test version 2.0.0 for testing of astrocyte 2.0.0 is released.
This version provide an example on how to pull images from customized registry, e.g. git.biohpc.swmed.edu:5050 (See the `workflow_containers` section in `astrocyte_pkg.yml`), and how to call the image with relative path in the nextflow configure file (See the `process` section of `workflow/configs/biohpc.config`).
......@@ -21,12 +21,17 @@ There is a single parameter `story`. This provides 1 or more files that the
workflow should run on.
## Containers
### Astrocyte workflow containers
The `workflow_containers` section in `astrocyte_pkg.yml` defines the URL of the contianers used by this workflow. When the `astrocyte_cli` run this workflow, it will pull the images in this section and save them to `workflow/images/singularity/` for later use.
In the workflow, the default configuration located in `workflow/configs/biohpc.config` defines how the Nextflow command to use the singularity executor for the uppercase, toLines, and wordcount processes.
### Vizapp containers
The `astrocyte_cli` version 2.0.0 supports containerized vizapp, so that vizapp becomes more independent and can be run on multiple platforms. The astrocyte workflow developers do not need to change their `vizapp_*` settings in `astrocyte_pkg.yml` to use this feature. The `shiny-prepare` command with the `--with-container` option will pull the right version of `R` container from the BioHPC "[Astrocyte Container Images](https://git.biohpc.swmed.edu/biohpc/astrocyte-container-images)" registry and save it to the `vizapp` folder for later use. Also, it will create a folder `vizapp/.rlibrary` to install the dependencies pacages.
The default configuration for this workflow is located in ./workflow/configs/biohpc.config . This configuration will cause the Nextflow command to use the singularity executor for the uppercase, toLines, and wordcount processes, using Docker.io endpoints. The biohpc_*_gitlab.config files show how to instead use images pulled from a specified GitLab repository.
## Questions
If you have any questions about this workflow example, or Astrocyte in general,
please contact the BioHPC team via biohpc-help@utsouthwestern.edu
......@@ -31,11 +31,11 @@ citation: |
# The minimum Astrocyte version that requires to run this workflow. For old pipelines, which do not have this label
# a default value of 0.3.1 will be assigned automatically. A request of minimum version less than 0.4.0 will be ignored.
minimum_astrocyte_version: '0.4.1'
minimum_astrocyte_version: '2.0.0'
# The Nextflow version that requires to run this workflow. For old pipelines, which do not have this label
# a default value of 0.31.0 will be assigned automatically. Please make sure the requested nextflow version is available
# in the module list.
nextflow_version: '21.04.1'
nextflow_version: '22.04.5'
# (Optional) The Nextflow config file to use for this workflow. If provided, the file should exist in workflow/configs
nextflow_config: 'biohpc.config'
# The container to use for this workflow, none/singularity. If omitted, the default value 'none' will be used.
......@@ -66,12 +66,16 @@ documentation_files:
# Specify versioned module names to ensure reproducability.
workflow_modules:
- Test
# A list of container images requires to run this workflow.
# Specify full path and version names to ensure reproducability.
# This keyword is required when 'container' is specified in Astrocyte 0.4.1 and above
# This keyword is required when 'container' is specified in Astrocyte 0.4.1 and above.
# Singularity supports different registries, please specify the protocol to use.
# Such as, "docker://", "shub://", "library://", etc. We encourage you to use the GitLab
# container registry of BioHPC to save and manage your container images.
workflow_containers:
- docker://ubuntu:latest
- docker://centos:centos8
- docker://git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/ubuntu:latest
- docker://git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/centos:centos8
# A list of parameters used by the workflow, defining how to present them,
# options etc in the web interface. For each parameter:
......@@ -171,7 +175,8 @@ workflow_parameters:
# The workflow must publish all final output into $baseDir
# Name of the R module that the vizapp will run against
vizapp_r_module: 'R/3.6.1-gccmkl'
# A containerized version is recommanded.
vizapp_r_module: 'R/4.1.1-img'
# List of any CRAN packages, not provided by the modules, that must be made
# available to the vizapp
......
singularity {
enabled = true
runOptions = '--disable-cache' // use this one for production
// runOptions = '--disable-cache --bind /vagrant:/vagrant' // use this one for vagrant development env only
cacheDir = "$baseDir/images/singularity"
runOptions = '' // Please do NOT use "--disable-cache" in this runOptions.
// Starting from version 2.0.0, the astrocyte_cli will clean up the cache automatically.
// runOptions = '--bind /vagrant:/vagrant' // Use this one for vagrant development env only
cacheDir = "$baseDir/images/singularity" // Singularity images specified in `workflow_containers` of astrocyte_pkg.yml will be saved to
// this folder automatically, before running the workflow. The images will be renamed as
// "NAME-TAG.img", e.g. ubuntu-latest.img, centos-centos8.img, r-4.1.1.img, etc.
}
process {
executor = 'local'
withName:parameters {
container = 'docker://ubuntu:latest'
// Path to the singularity image files. The prefix "file://images/singularity/" is the cacheDir folder
// in the singularity settings. Please keep this prefix for the path of your singularity images
container = 'file://images/singularity/ubuntu-latest.img'
}
withName:uppercase {
container = 'docker://ubuntu:latest'
container = 'file://images/singularity/ubuntu-latest.img'
}
withName:tolines {
container = 'docker://ubuntu:latest'
container = 'file://images/singularity/ubuntu-latest.img'
}
withName:wordcounts {
container = 'docker://centos:centos8'
container = 'file://images/singularity/centos-centos8.img'
}
}
docker {
enabled = true
fixOwnership = true
}
process {
executor = 'local'
withName:uppercase {
container = 'ubuntu:latest'
}
withName:tolines {
container = 'ubuntu:latest'
}
withName:wordcounts {
container = 'centos:centos8'
}
}
docker {
enabled = true
fixOwnership = true
}
process {
executor = 'local'
withName:uppercase {
container = 'git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/ubuntu:latest'
}
withName:tolines {
container = 'git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/ubuntu:latest'
}
withName:wordcounts {
container = 'git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/centos:centos8'
}
}
singularity {
enabled = true
runOptions = '--disable-cache' // use this one for production
// runOptions = '--disable-cache --bind /vagrant:/vagrant' // use this one for vagrant development env only
cacheDir = "$baseDir/images/singularity"
}
process {
executor = 'local'
withName:parameters {
container = 'git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/ubuntu:latest'
}
withName:uppercase {
container = 'docker://git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/ubuntu:latest'
}
withName:tolines {
container = 'docker://git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/ubuntu:latest'
}
withName:wordcounts {
container = 'docker://git.biohpc.swmed.edu:5050/biohpc/astrocyte_example_wordcount/centos:centos8'
}
}
/*
* Copyright (c) 2016. The University of Texas Southwestern Medical Center
* Copyright (c) 2022. The University of Texas Southwestern Medical Center
*
* This file is part of the BioHPC Workflow Platform
*
* This is a minimal test workflow package that astrocyte tests can run against.
*
* @authors
* David Trudgian, Peng Lian, Devin OKelly
* Peng Lian
*
*/
......@@ -41,6 +41,8 @@ process parameters {
echo "test_string: ${params.test_string}"
echo "test_select: ${params.test_select}"
echo "test_multiselect: ${params.test_multiselect}"
sleep 1
"""
}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment