Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
C
cellranger_mkfastq
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Locked Files
Issues
7
Issues
7
List
Boards
Labels
Service Desk
Milestones
Iterations
Merge Requests
1
Merge Requests
1
Requirements
Requirements
List
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Test Cases
Security & Compliance
Security & Compliance
Dependency List
License Compliance
Operations
Operations
Incidents
Environments
Packages & Registries
Packages & Registries
Container Registry
Analytics
Analytics
CI / CD
Code Review
Insights
Issue
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
BICF
Astrocyte
cellranger_mkfastq
Commits
b4b389fc
Commit
b4b389fc
authored
Nov 10, 2020
by
Jeremy Mathews
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Make most updates except last one
parent
3a4c7a97
Pipeline
#8334
failed with stages
in 16 seconds
Changes
6
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
with
44 additions
and
41 deletions
+44
-41
.gitlab-ci.yml
.gitlab-ci.yml
+20
-19
CHANGELOG.md
CHANGELOG.md
+1
-0
README.md
README.md
+9
-9
workflow/main.nf
workflow/main.nf
+2
-1
workflow/tests/test_mkfastq.py
workflow/tests/test_mkfastq.py
+6
-6
workflow/tests/test_multiqc.py
workflow/tests/test_multiqc.py
+6
-6
No files found.
.gitlab-ci.yml
View file @
b4b389fc
...
...
@@ -4,16 +4,17 @@ before_script:
-
pip install --user pytest-pythonpath==0.7.1 pytest-cov==2.5.1
-
module load nextflow/20.01.0
-
module load singularity/3.5.3
-
mkdir -p test_data/
simple1
-
mkdir -p test_data/
simple2
-
mkdir -p test_data/s
imple3
-
ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/
simple1/* test_data/simple1
/
-
ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/
simple2/* test_data/simple2
/
-
ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/s
imple3/* test_data/simple3
/
-
mkdir -p test_data/
cellranger1bcl
-
mkdir -p test_data/
cellranger2bcl
-
mkdir -p test_data/s
paceranger1bcl
-
ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/
cellranger1bcl/* test_data/cellranger1bcl
/
-
ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/
cellranger2bcl/* test_data/cellranger2bcl
/
-
ln -sfn /project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/s
paceranger1bcl/* test_data/spaceranger1bcl
/
stages
:
-
astrocyte
-
simple
-
cellranger
-
spaceranger
astrocyte_check
:
stage
:
astrocyte
...
...
@@ -26,11 +27,11 @@ astrocyte_check:
when
:
-
always
simple_1FC
:
stage
:
simple
cellranger1bcl
:
stage
:
cellranger
script
:
-
nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/
simple1/*.tar.gz" --designFile "test_data/simple1
/cellranger-tiny-bcl-simple-1_2_0.csv" --ci
true
##- pytest -m
simple1
-
nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/
cellranger1bcl/*.tar.gz" --designFile "test_data/cellranger1bcl
/cellranger-tiny-bcl-simple-1_2_0.csv" --ci
true
##- pytest -m
cellranger1bcl
artifacts
:
name
:
"
$CI_JOB_NAME"
when
:
always
...
...
@@ -43,11 +44,11 @@ simple_1FC:
when
:
-
always
simple_2FC
:
stage
:
simple
cellranger2bcl
:
stage
:
cellranger
script
:
-
nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/
simple2/*.tar.gz" --designFile "test_data/simple2
/cellranger-tiny-bcl-simple-1_2_0.csv" --ci
true
##- pytest -m
simple2
-
nextflow run workflow/main.nf -profile biohpc,cluster --ranger "cellranger" --bcl "test_data/
cellranger2bcl/*.tar.gz" --designFile "test_data/cellranger2bcl
/cellranger-tiny-bcl-simple-1_2_0.csv" --ci
true
##- pytest -m
cellranger2bcl
artifacts
:
name
:
"
$CI_JOB_NAME"
when
:
always
...
...
@@ -60,11 +61,11 @@ simple_2FC:
when
:
-
always
s
imple_3FC
:
stage
:
s
imple
s
paceranger1bcl
:
stage
:
s
paceranger
script
:
-
nextflow run workflow/main.nf -profile biohpc,cluster --ranger "spaceranger" --bcl "test_data/s
imple3/*.tar.gz" --designFile "test_data/simple3
/spaceranger-tiny-bcl-simple-1.0.0.csv" --ci
true
##- pytest -m s
imple3
-
nextflow run workflow/main.nf -profile biohpc,cluster --ranger "spaceranger" --bcl "test_data/s
paceranger1bcl/*.tar.gz" --designFile "test_data/spaceranger1bcl
/spaceranger-tiny-bcl-simple-1.0.0.csv" --ci
true
##- pytest -m s
paceranger1bcl
artifacts
:
name
:
"
$CI_JOB_NAME"
when
:
always
...
...
CHANGELOG.md
View file @
b4b389fc
...
...
@@ -3,6 +3,7 @@
*
Add flag to select cellranger vs spaceranger
*
Adjust versions to take into account multiple 10x-Ranger softwares
*
Remove countDesign process
*
Spaceranger V1.2.0 update
**Background**
*
Update scripts to reflect spaceranger or cellranger input
...
...
README.md
View file @
b4b389fc
...
...
@@ -42,19 +42,19 @@ To Run:
*
Available parameters:
*
**-profile**
*
what environments to run on, available:
`biohpc`
,
`local`
,
`cluster`
,
`aws`
,
`ondemand`
,
`spot`
*
eg:
`
``-profile biohpc,cluster``
`
to run on BioHPC in cluster mode
*
eg:
`
``-profile aws,ondemand``
`
to run on AWS on a on-demand queue
*
eg:
`
-profile biohpc,cluster
`
to run on BioHPC in cluster mode
*
eg:
`
-profile aws,ondemand
`
to run on AWS on a on-demand queue
*
**--name**
*
run name, puts outputs in a directory with this name
*
eg:
`
``--name 'test'``
`
*
eg:
`
--name 'test'
`
*
**--ranger**
*
select the 10x ranger being run.
*
eg:
`
``--ranger 'cellranger'``
`
to run cellranger mkfastq
*
eg:
`
``--ranger 'spaceranger'``
`
to run spaceranger mkfastq
*
eg:
`
--ranger 'cellranger'
`
to run cellranger mkfastq
*
eg:
`
--ranger 'spaceranger'
`
to run spaceranger mkfastq
*
**--bcl**
*
base call files (tarballed [
*.tar] +/- gunzipping [*
.tar.gz] from a sequencing of 10x single-cell expereiment, supports pigr parallelization)
*
there can be multiple basecall files, but they all will be demultiplexed by the same design file
*
eg:
`
``--bcl '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-1_2_0.tar.gz'``
`
*
eg:
`
--bcl '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-1_2_0.tar.gz'
`
*
**--designFile**
*
path to design file (csv format) location
*
column 1 = "Lane" (number of lanes to demultiplex,
**\***
for all lanes)
...
...
@@ -64,13 +64,13 @@ To Run:
*
[
Current sample barcode IDs
](
https://s3-us-west-2.amazonaws.com/10x.files/supp/cell-exp/chromium-shared-sample-indexes-plate.csv
)
*
can have repeated "Sample" if there are multiple fastq R1/R2 pairs for the samples
*
can be downloaded
[
HERE
](
https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_mkfastq/blob/master/docs/design.csv
)
*
eg:
`
``--designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple/cellranger-tiny-bcl-simple-1_2_0.csv'``
`
*
eg:
`
--designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple/cellranger-tiny-bcl-simple-1_2_0.csv'
`
*
**--mask**
*
add a base mask if the sequencing strategy doesn't match the requirements of 10x Genomics
*
eg:
`
``--mask '--use-bases-mask=Y\*,I8n\*,n\*,Y\*'``
`
*
eg:
`
--mask '--ignore-dual-index --use-bases-mask=Y\*,I8n\*,n\*,Y\*'
`
*
**--outDir**
*
optional output directory for run
*
eg:
`
``--outDir 'test'``
`
*
eg:
`
--outDir 'test'
`
*
FULL EXAMPLE:
```
nextflow run workflow/main.nf -profile biohpc,cluster --name 'test' --ranger 'cellranger' --bcl '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-1_2_0.tar.gz' --designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_mkfastq/simple1/cellranger-tiny-bcl-simple-1_2_0.csv' --outDir 'test'
...
...
workflow/main.nf
View file @
b4b389fc
...
...
@@ -76,6 +76,7 @@ process trackStart {
"sessionId": "${workflow.sessionId}", \
"pipeline": "cellranger_mkfastq", \
"pipelineVersion": "${pipelineVersion}", \
"10x-ranger": ${params.ranger}, \
"start": "${workflow.start}", \
"astrocyte": ${params.astrocyte}, \
"status": "started", \
...
...
@@ -213,7 +214,7 @@ process spaceranger_mkfastq {
tag "${bcl.simpleName}"
publishDir "${outDir}/${task.process}", mode: 'copy', pattern: "{*/outs/**/*.fastq.gz}"
queue '128GB,256GB,256GBv1,384GB'
module 'spaceranger/1.
1
.0:bcl2fastq/2.19.1'
module 'spaceranger/1.
2
.0:bcl2fastq/2.19.1'
input:
file versions_spacerangerScript
...
...
workflow/tests/test_mkfastq.py
View file @
b4b389fc
...
...
@@ -14,15 +14,15 @@ import os
test_output_path
=
os
.
path
.
dirname
(
os
.
path
.
abspath
(
__file__
))
+
\
'/../output/'
@
pytest
.
mark
.
simple1
def
test_
simple1
_mkfastq
():
@
pytest
.
mark
.
cellranger1bcl
def
test_
cellranger1bcl
_mkfastq
():
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'mkfastq'
,
'cellranger-tiny-bcl-1_2_0'
,
'outs'
))
@
pytest
.
mark
.
simple2
def
test_
simple2
_mkfastq
():
@
pytest
.
mark
.
cellranger2bcl
def
test_
cellranger2bcl
_mkfastq
():
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'mkfastq'
,
'cellranger-tiny-bcl-1_2_0-1'
,
'outs'
))
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'mkfastq'
,
'cellranger-tiny-bcl-1_2_0-2'
,
'outs'
))
@
pytest
.
mark
.
s
imple3
def
test_s
imple3
_mkfastq
():
@
pytest
.
mark
.
s
paceranger1bcl
def
test_s
paceranger1bcl
_mkfastq
():
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'spaceranger_mkfastq'
,
'mkfastq_spaceranger-tiny-bcl-1'
,
'outs'
))
workflow/tests/test_multiqc.py
View file @
b4b389fc
...
...
@@ -14,14 +14,14 @@ import os
test_output_path
=
os
.
path
.
dirname
(
os
.
path
.
abspath
(
__file__
))
+
\
'/../output/multiqc/run/'
@
pytest
.
mark
.
simple1
def
test_
simple1
_multiqc
():
@
pytest
.
mark
.
cellranger1bcl
def
test_
cellranger1bcl
_multiqc
():
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'multiqc_report.html'
))
@
pytest
.
mark
.
simple2
def
test_
simple2
_multiqc
():
@
pytest
.
mark
.
cellranger2bcl
def
test_
cellranger2bcl
_multiqc
():
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'multiqc_report.html'
))
@
pytest
.
mark
.
s
imple3
def
test_s
imple3
_multiqc
():
@
pytest
.
mark
.
s
paceranger1bcl
def
test_s
paceranger1bcl
_multiqc
():
assert
os
.
path
.
exists
(
os
.
path
.
join
(
test_output_path
,
'multiqc_report.html'
))
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment