Commit b4b389fc authored by Jeremy Mathews's avatar Jeremy Mathews
Browse files

Make most updates except last one

parent 3a4c7a97
Pipeline #8334 failed with stages
in 16 seconds
......@@ -4,16 +4,17 @@ before_script:
- pip install --user pytest-pythonpath==0.7.1 pytest-cov==2.5.1
- module load nextflow/20.01.0
- module load singularity/3.5.3
- mkdir -p test_data/simple1
- mkdir -p test_data/simple2
- mkdir -p test_data/simple3
- ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/* test_data/simple1/
- ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple2/* test_data/simple2/
- ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple3/* test_data/simple3/
- mkdir -p test_data/cellranger1bcl
- mkdir -p test_data/cellranger2bcl
- mkdir -p test_data/spaceranger1bcl
- ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/cellranger1bcl/* test_data/cellranger1bcl/
- ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/cellranger2bcl/* test_data/cellranger2bcl/
- ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/spaceranger1bcl/* test_data/spaceranger1bcl/
stages:
- astrocyte
- simple
- cellranger
- spaceranger
astrocyte_check:
stage: astrocyte
......@@ -26,11 +27,11 @@ astrocyte_check:
when:
- always
simple_1FC:
stage: simple
cellranger1bcl:
stage: cellranger
script:
- nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/simple1/*.tar.gz" --designFile "test_data/simple1/cellranger-tiny-bcl-simple-1_2_0.csv" --ci true
##- pytest -m simple1
- nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/cellranger1bcl/*.tar.gz" --designFile "test_data/cellranger1bcl/cellranger-tiny-bcl-simple-1_2_0.csv" --ci true
##- pytest -m cellranger1bcl
artifacts:
name: "$CI_JOB_NAME"
when: always
......@@ -43,11 +44,11 @@ simple_1FC:
when:
- always
simple_2FC:
stage: simple
cellranger2bcl:
stage: cellranger
script:
- nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/simple2/*.tar.gz" --designFile "test_data/simple2/cellranger-tiny-bcl-simple-1_2_0.csv" --ci true
##- pytest -m simple2
- nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/cellranger2bcl/*.tar.gz" --designFile "test_data/cellranger2bcl/cellranger-tiny-bcl-simple-1_2_0.csv" --ci true
##- pytest -m cellranger2bcl
artifacts:
name: "$CI_JOB_NAME"
when: always
......@@ -60,11 +61,11 @@ simple_2FC:
when:
- always
simple_3FC:
stage: simple
spaceranger1bcl:
stage: spaceranger
script:
- nextflow run workflow/main.nf -profile biohpc,cluster --ranger "spaceranger" --bcl "test_data/simple3/*.tar.gz" --designFile "test_data/simple3/spaceranger-tiny-bcl-simple-1.0.0.csv" --ci true
##- pytest -m simple3
- nextflow run workflow/main.nf -profile biohpc,cluster --ranger "spaceranger" --bcl "test_data/spaceranger1bcl/*.tar.gz" --designFile "test_data/spaceranger1bcl/spaceranger-tiny-bcl-simple-1.0.0.csv" --ci true
##- pytest -m spaceranger1bcl
artifacts:
name: "$CI_JOB_NAME"
when: always
......
......@@ -3,6 +3,7 @@
* Add flag to select cellranger vs spaceranger
* Adjust versions to take into account multiple 10x-Ranger softwares
* Remove countDesign process
* Spaceranger V1.2.0 update
**Background**
* Update scripts to reflect spaceranger or cellranger input
......
......@@ -42,19 +42,19 @@ To Run:
* Available parameters:
* **-profile**
* what environments to run on, available: `biohpc`, `local`, `cluster`, `aws`, `ondemand`, `spot`
* eg: ```-profile biohpc,cluster``` to run on BioHPC in cluster mode
* eg: ```-profile aws,ondemand``` to run on AWS on a on-demand queue
* eg: `-profile biohpc,cluster` to run on BioHPC in cluster mode
* eg: `-profile aws,ondemand` to run on AWS on a on-demand queue
* **--name**
* run name, puts outputs in a directory with this name
* eg: ```--name 'test'```
* eg: `--name 'test'`
* **--ranger**
* select the 10x ranger being run.
* eg: ```--ranger 'cellranger'``` to run cellranger mkfastq
* eg: ```--ranger 'spaceranger'``` to run spaceranger mkfastq
* eg: `--ranger 'cellranger'` to run cellranger mkfastq
* eg: `--ranger 'spaceranger'` to run spaceranger mkfastq
* **--bcl**
* base call files (tarballed [*.tar] +/- gunzipping [*.tar.gz] from a sequencing of 10x single-cell expereiment, supports pigr parallelization)
* there can be multiple basecall files, but they all will be demultiplexed by the same design file
* eg: ```--bcl '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-1_2_0.tar.gz'```
* eg: `--bcl '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-1_2_0.tar.gz'`
* **--designFile**
* path to design file (csv format) location
* column 1 = "Lane" (number of lanes to demultiplex, **\*** for all lanes)
......@@ -64,13 +64,13 @@ To Run:
* [Current sample barcode IDs](https://s3-us-west-2.amazonaws.com/10x.files/supp/cell-exp/chromium-shared-sample-indexes-plate.csv)
* can have repeated "Sample" if there are multiple fastq R1/R2 pairs for the samples
* can be downloaded [HERE](https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_mkfastq/blob/master/docs/design.csv)
* eg: ```--designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple/cellranger-tiny-bcl-simple-1_2_0.csv'```
* eg: `--designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple/cellranger-tiny-bcl-simple-1_2_0.csv'`
* **--mask**
* add a base mask if the sequencing strategy doesn't match the requirements of 10x Genomics
* eg: ```--mask '--use-bases-mask=Y\*,I8n\*,n\*,Y\*'```
* eg: `--mask '--ignore-dual-index --use-bases-mask=Y\*,I8n\*,n\*,Y\*'`
* **--outDir**
* optional output directory for run
* eg: ```--outDir 'test'```
* eg: `--outDir 'test'`
* FULL EXAMPLE:
```
nextflow run workflow/main.nf -profile biohpc,cluster --name 'test' --ranger 'cellranger' --bcl '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-1_2_0.tar.gz' --designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-simple-1_2_0.csv' --outDir 'test'
......
......@@ -76,6 +76,7 @@ process trackStart {
"sessionId": "${workflow.sessionId}", \
"pipeline": "cellranger_mkfastq", \
"pipelineVersion": "${pipelineVersion}", \
"10x-ranger": ${params.ranger}, \
"start": "${workflow.start}", \
"astrocyte": ${params.astrocyte}, \
"status": "started", \
......@@ -213,7 +214,7 @@ process spaceranger_mkfastq {
tag "${bcl.simpleName}"
publishDir "${outDir}/${task.process}", mode: 'copy', pattern: "{*/outs/**/*.fastq.gz}"
queue '128GB,256GB,256GBv1,384GB'
module 'spaceranger/1.1.0:bcl2fastq/2.19.1'
module 'spaceranger/1.2.0:bcl2fastq/2.19.1'
input:
file versions_spacerangerScript
......
......@@ -14,15 +14,15 @@ import os
test_output_path = os.path.dirname(os.path.abspath(__file__)) + \
'/../output/'
@pytest.mark.simple1
def test_simple1_mkfastq():
@pytest.mark.cellranger1bcl
def test_cellranger1bcl_mkfastq():
assert os.path.exists(os.path.join(test_output_path, 'mkfastq', 'cellranger-tiny-bcl-1_2_0', 'outs'))
@pytest.mark.simple2
def test_simple2_mkfastq():
@pytest.mark.cellranger2bcl
def test_cellranger2bcl_mkfastq():
assert os.path.exists(os.path.join(test_output_path, 'mkfastq', 'cellranger-tiny-bcl-1_2_0-1', 'outs'))
assert os.path.exists(os.path.join(test_output_path, 'mkfastq', 'cellranger-tiny-bcl-1_2_0-2', 'outs'))
@pytest.mark.simple3
def test_simple3_mkfastq():
@pytest.mark.spaceranger1bcl
def test_spaceranger1bcl_mkfastq():
assert os.path.exists(os.path.join(test_output_path, 'spaceranger_mkfastq', 'mkfastq_spaceranger-tiny-bcl-1', 'outs'))
......@@ -14,14 +14,14 @@ import os
test_output_path = os.path.dirname(os.path.abspath(__file__)) + \
'/../output/multiqc/run/'
@pytest.mark.simple1
def test_simple1_multiqc():
@pytest.mark.cellranger1bcl
def test_cellranger1bcl_multiqc():
assert os.path.exists(os.path.join(test_output_path, 'multiqc_report.html'))
@pytest.mark.simple2
def test_simple2_multiqc():
@pytest.mark.cellranger2bcl
def test_cellranger2bcl_multiqc():
assert os.path.exists(os.path.join(test_output_path, 'multiqc_report.html'))
@pytest.mark.simple3
def test_simple3_multiqc():
@pytest.mark.spaceranger1bcl
def test_spaceranger1bcl_multiqc():
assert os.path.exists(os.path.join(test_output_path, 'multiqc_report.html'))
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment