Skip to content
Snippets Groups Projects
Commit 21c2b069 authored by David Trudgian's avatar David Trudgian
Browse files

Tidy up example

parent 010a236f
Branches
Tags
No related merge requests found
Pipeline #3815 failed with stages
# Run our tests against a specific version of astrocyte
before_script:
- module load astrocyte/0.1.0
- module load astrocyte
stages:
- check
......
Copyright © 2016. The University of Texas Southwestern Medical Center
Copyright © 2019. The University of Texas Southwestern Medical Center
5323 Harry Hines Boulevard Dallas, Texas, 75390 Telephone 214-648-3111
# Example Wordcount Package
# Example Ignite Wordcount Package
[![Build
Status](https://git.biohpc.swmed.edu/BioHPC/astrocyte_example_wordcount/badges/master/build.svg)](https://git.biohpc.swmed.edu/BioHPC/astrocyte_example_wordcount/commits/master)
[![Astrocyte](https://img.shields.io/badge/astrocyte-%E2%89%A50.1.0-blue.svg)](https://astrocyte-test.biohpc.swmed.edu/static/docs/index.html)
[![Astrocyte](https://img.shields.io/badge/astrocyte-%E2%89%A50.2.0-blue.svg)](https://astrocyte-test.biohpc.swmed.edu/static/docs/index.html)
This is an example workflow package for astrocyte. It contains a worfklow
This is a minimal test workflow package that counts the occurences of words
in a test file. It can be used as a template to develop workflows, and as to
test the astrocyte platform. This version uses the Ignite executor for
Nextflow, so that processes all execute inside a single batch job allocation.
to count the occurences of each word in a text file.
## The Workflow
The workflow `workflow/main.nf` has three processes:
- Convert all the text in the input files to uppercase
- Split the text so each word is on a separate line
- Sort, find and count the occurence of unique words
- `parameters` outputs the parameters provided to the workflow, as an example
of how workflows receive the different types of parameters that can be setup
in the `astrocyte_pkg.yml` file.
- `uppercase` converts all the text in the input files to uppercase
- `tolines` splits the text so each word is on a separate line
- `wordcount` sorts, find and count the occurence of unique words
## Parameters
There is a single parameter `story`. This provides 1 or more files that the
workflow should run on.
There is a single parameter which affects the workflow - `story`. This provides
1 or more files that the workflow should run on.
Additional parameters named `test_xxx` are included only to show how to define
the parameter type in the `astrocyte_pkg.yml` file. They are simply printed to
STDOUT by the workflow's `parameters` process.
## Ignite Execution
The workflow can be run using Nextflow + Apache Ignite using the `astrocyte_cli` tool. On the BioHPC cluster:
```sh
module add astrocyte
astrocyte ignite-cluster astrocyte_example_wordcount_ignite
```
An example Slurm sbatch script `workflow/ignite_batch.sh` has been included to
provide a template for running Nextflow workflows with Ignite outside of
astrocyte.
Both the `astrocyte_pkg.yml` and `ignite_batch.sh` are set so that the workflow
will run using 4 x 32GB nodes on the BioHPC Nucleus cluster. The `tolines`
process in the `main.nf` workflow has been artificially set to require 16 cpus,
and sleep for 300s so that you can examine the running workflow to see how
Ignite distributes jobs between nodes.
## Questions
......
......@@ -13,7 +13,7 @@ author: 'David Trudgian'
# A contact email address for questions
email: 'biohpc-help@utsouthwestern.edu'
# A more informative title for the workflow package
title: 'Example Wordcount Workflow'
title: 'Ignite Example Wordcount Workflow'
# A summary of the workflow package in plain text
description: |
This is a minimal test workflow package that counts the occurences of words
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment