Skip to content
Snippets Groups Projects
Commit caecad18 authored by Gervaise Henry's avatar Gervaise Henry :cowboy:
Browse files

Merge branch 'develop' into 'master'

Develop

See merge request !57
parents 4b102a5e bedc0841
Branches
Tags
2 merge requests!64Master,!57Develop
Pipeline #4732 passed with stages
in 10 minutes and 54 seconds
Showing
with 738 additions and 102 deletions
...@@ -36,16 +36,19 @@ simple_1: ...@@ -36,16 +36,19 @@ simple_1:
- branches - branches
- tags - tags
except: except:
- develop refs:
- master - develop
- master
script: script:
- nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v3s1r500/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v3s1r500/design.csv" --genome 'GRCh38-3.0.0' --kitVersion 'three' --version '3.0.2' - nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v3s1r500/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v3s1r500/design.csv" --genome 'GRCh38-3.0.0' --kitVersion 'three' --version '3.1.0'
- pytest -m count302 - pytest -m count310
artifacts: artifacts:
name: "$CI_JOB_NAME" name: "$CI_JOB_NAME"
when: always when: always
paths: paths:
- .nextflow.log - .nextflow.log
- workflow/output/count310/sample1/outs/web_summary.html
- workflow/output/multiqc/run/multiqc_report.html
expire_in: 2 days expire_in: 2 days
retry: retry:
max: 1 max: 1
...@@ -57,17 +60,20 @@ simple_2: ...@@ -57,17 +60,20 @@ simple_2:
only: only:
- branches - branches
except: except:
- develop refs:
- master - develop
- tags - master
- tags
script: script:
- nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/mu.v3s1r500/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/mu.v3s1r500/design.csv" --genome 'mm10-3.0.0' --kitVersion 'three' --version '3.0.1' - nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/mu.v3s1r500/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/mu.v3s1r500/design.csv" --genome 'mm10-3.0.0' --kitVersion 'three' --version '3.1.0'
- pytest -m count301 - pytest -m count310
artifacts: artifacts:
name: "$CI_JOB_NAME" name: "$CI_JOB_NAME"
when: always when: always
paths: paths:
- .nextflow.log - .nextflow.log
- workflow/output/count310/sample1/outs/web_summary.html
- workflow/output/multiqc/run/multiqc_report.html
expire_in: 2 days expire_in: 2 days
retry: retry:
max: 1 max: 1
...@@ -80,36 +86,43 @@ detailed_1: ...@@ -80,36 +86,43 @@ detailed_1:
- develop - develop
- master - master
except: except:
- tags refs:
- tags
script: script:
- nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v3s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v3s2r10k/design.csv" --genome 'GRCh38-3.0.0' --kitVersion 'auto' --version '3.0.2' - nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v3s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v3s2r10k/design.csv" --genome 'GRCh38-3.0.0' --kitVersion 'auto' --version '3.1.0'
- pytest -m count302 - pytest -m count310
artifacts: artifacts:
name: "$CI_JOB_NAME" name: "$CI_JOB_NAME"
when: always when: always
paths: paths:
- .nextflow.log - .nextflow.log
- workflow/output/count310/sample1/outs/web_summary.html
- workflow/output/multiqc/run/multiqc_report.html
expire_in: 2 days expire_in: 2 days
retry: retry:
max: 1 max: 1
when: when:
- always - always
detailed_2: detailed_2:
stage: detailed stage: detailed
only: only:
- develop - develop
- master - master
except: except:
- tags refs:
- tags
script: script:
- nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/mu.v3s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/mu.v3s2r10k/design.csv" --genome 'mm10-3.0.0' --kitVersion 'three' --version '3.0.2' - nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v3s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v3s2r10k/design.csv" --genome 'GRCh38-3.0.0' --kitVersion 'auto' --version '3.0.2'
- pytest -m count302 - pytest -m count302
artifacts: artifacts:
name: "$CI_JOB_NAME" name: "$CI_JOB_NAME"
when: always when: always
paths: paths:
- .nextflow.log - .nextflow.log
- workflow/output/count302/sample1/outs/web_summary.html
- workflow/output/multiqc/run/multiqc_report.html
expire_in: 2 days expire_in: 2 days
retry: retry:
max: 1 max: 1
...@@ -122,7 +135,32 @@ detailed_3: ...@@ -122,7 +135,32 @@ detailed_3:
- develop - develop
- master - master
except: except:
- tags refs:
- tags
script:
- nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/mu.v3s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/mu.v3s2r10k/design.csv" --genome 'mm10-3.0.0' --kitVersion 'three' --version '3.0.1'
- pytest -m count301
artifacts:
name: "$CI_JOB_NAME"
when: always
paths:
- .nextflow.log
- workflow/output/count301/sample1/outs/web_summary.html
- workflow/output/multiqc/run/multiqc_report.html
expire_in: 2 days
retry:
max: 1
when:
- always
detailed_4:
stage: detailed
only:
- develop
- master
except:
refs:
- tags
script: script:
- nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v2s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v2s2r10k/design.csv" --genome 'GRCh38-1.2.0' --kitVersion 'two' --version '2.1.1' - nextflow run workflow/main.nf --fastq "$CI_PROJECT_DIR/test_data/hu.v2s2r10k/*.fastq.gz" --designFile "$CI_PROJECT_DIR/test_data/hu.v2s2r10k/design.csv" --genome 'GRCh38-1.2.0' --kitVersion 'two' --version '2.1.1'
- pytest -m count211 - pytest -m count211
...@@ -131,6 +169,8 @@ detailed_3: ...@@ -131,6 +169,8 @@ detailed_3:
when: always when: always
paths: paths:
- .nextflow.log - .nextflow.log
- workflow/output/count211/sample1/outs/web_summary.html
- workflow/output/multiqc/run/multiqc_report.html
expire_in: 2 days expire_in: 2 days
retry: retry:
max: 1 max: 1
......
# v1.1.0 (in development) # v1.2.0
**User Facing**
* Add Cellranger Version 3.1.0
* Add human/mouse farmyard reference Version 3.1.0 from 10x
* Add Vizapp (shiny)
* Fix mutiqc error
* Add MIT License
**Background**
* Add CI Artifacts
*Known Bugs*
* Vizapp does not yet work for Astrocyte
# v1.1.0
**User Facing** **User Facing**
* Make report (multiqc) for cellranger qc output, version, references * Make report (multiqc) for cellranger qc output, version, references
......
LICENSE 0 → 100644
MIT License
Copyright (c) 2019 University of Texas Southwestern Medical Center.
Contributors: Gervaise H. Henry, Jeremy Mathews, and Venkat Malladi
Department: Bioinformatic Core Facility, Department of Bioinformatics
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
...@@ -27,7 +27,7 @@ To Run: ...@@ -27,7 +27,7 @@ To Run:
* path to the fastq location * path to the fastq location
* R1 and R2 only necessary but can include I2 * R1 and R2 only necessary but can include I2
* only fastq's in designFile (see below) are used, not present will be ignored * only fastq's in designFile (see below) are used, not present will be ignored
* eg: **--fastq '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r100k/\*.fastq.gz'** * eg: **--fastq '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r10k/\*.fastq.gz'**
* **--designFile** * **--designFile**
* path to design file (csv format) location * path to design file (csv format) location
* column 1 = "Sample" * column 1 = "Sample"
...@@ -35,7 +35,7 @@ To Run: ...@@ -35,7 +35,7 @@ To Run:
* column 3 = "fastq_R2" * column 3 = "fastq_R2"
* can have repeated "Sample" if there are multiple fastq R1/R2 pairs for the samples * can have repeated "Sample" if there are multiple fastq R1/R2 pairs for the samples
* can be downloaded [HERE](https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/master/docs/design.csv) * can be downloaded [HERE](https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/master/docs/design.csv)
* eg: **--designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r100k/design.csv'** * eg: **--designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r10k/design.csv'**
* **--genome** * **--genome**
* reference genome * reference genome
* requires workflow/conf/biohpc.config to work * requires workflow/conf/biohpc.config to work
...@@ -46,8 +46,9 @@ To Run: ...@@ -46,8 +46,9 @@ To Run:
* *'hg19-1.2.0'* = Human GRCh37 (hg19) release 84 * *'hg19-1.2.0'* = Human GRCh37 (hg19) release 84
* *'mm10-3.0.0'* = Mouse GRCm38 (mm10) release 93 * *'mm10-3.0.0'* = Mouse GRCm38 (mm10) release 93
* *'mm10-3.0.0'* = Mouse GRCm38 (mm10) release 84 * *'mm10-3.0.0'* = Mouse GRCm38 (mm10) release 84
* *'hg19_and_mm10-3.0.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm19) release 93 * *'GRCh38_and_mm10-3.1.0'* = Human GRCh38 + Mouse GRCm38 (mm10) release 93
* *'hg19_and_mm10-1.2.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm19) release 84 * *'hg19_and_mm10-3.0.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm10) release 93
* *'hg19_and_mm10-1.2.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm10) release 84
* *'ercc92-1.2.0'* = ERCC.92 Spike-In * *'ercc92-1.2.0'* = ERCC.92 Spike-In
* if --genome is used then --genomeLocationFull is not necessary * if --genome is used then --genomeLocationFull is not necessary
* eg: **--genome 'GRCh38-3.0.0'** * eg: **--genome 'GRCh38-3.0.0'**
...@@ -83,16 +84,17 @@ To Run: ...@@ -83,16 +84,17 @@ To Run:
* cellranger version * cellranger version
* --version (cellranger version) 2.1.1 can only read --kitVersion of two (2) * --version (cellranger version) 2.1.1 can only read --kitVersion of two (2)
* options: * options:
* *'3.1.0'*
* *'3.0.2'* * *'3.0.2'*
* *'3.0.1'* * *'3.0.1'*
* *'2.1.1'* * *'2.1.1'*
* eg: **--version '3.0.2'** * eg: **--version '3.1.0'**
* **--outDir** * **--outDir**
* optional output directory for run * optional output directory for run
* eg: **--outDir 'test'** * eg: **--outDir 'test'**
* FULL EXAMPLE: * FULL EXAMPLE:
``` ```
nextflow run workflow/main.nf --fastq '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r100k/*.fastq.gz' --designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r100k/design.csv' --genome 'GRCh38-3.0.0' --kitVersion 'three' --version '3.0.2' --outDir 'test' nextflow run workflow/main.nf --fastq '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r10k/*.fastq.gz' --designFile '/project/shared/bicf_workflow_ref/workflow_testdata/cellranger/cellranger_count/hu.v3s2r10k/design.csv' --genome 'GRCh38-3.0.0' --kitVersion 'three' --version '3.1.0' --outDir 'test'
``` ```
* Design example: * Design example:
......
...@@ -43,6 +43,7 @@ workflow_modules: ...@@ -43,6 +43,7 @@ workflow_modules:
- 'cellranger/2.1.1' - 'cellranger/2.1.1'
- 'cellranger/3.0.1' - 'cellranger/3.0.1'
- 'cellranger/3.0.2' - 'cellranger/3.0.2'
- 'cellranger/3.1.0'
- 'bcl2fastq/2.17.1.14' - 'bcl2fastq/2.17.1.14'
- 'multiqc/1.7' - 'multiqc/1.7'
...@@ -104,8 +105,9 @@ workflow_parameters: ...@@ -104,8 +105,9 @@ workflow_parameters:
- ['hg19-1.2.0', 'Human GRCh37 (hg19) release 84'] - ['hg19-1.2.0', 'Human GRCh37 (hg19) release 84']
- ['mm10-3.0.0', 'Mouse GRCm38 (mm10) release 93'] - ['mm10-3.0.0', 'Mouse GRCm38 (mm10) release 93']
- ['mm10-1.2.0', 'Mouse GRCm38 (mm10) release 84'] - ['mm10-1.2.0', 'Mouse GRCm38 (mm10) release 84']
- ['hg19_and_mm10-3.0.0', 'Human GRCh37 (hg19) + Mouse GRCm38 (mm19) release 93'] - ['GRCh38_and_mm10-3.1.0', 'Human GRCh38 + Mouse GRCm38 (mm10) release 93']
- ['hg19_and_mm10-1.2.0', 'Human GRCh37 (hg19) + Mouse GRCm38 (mm19) release 84'] - ['hg19_and_mm10-3.0.0', 'Human GRCh37 (hg19) + Mouse GRCm38 (mm10) release 93']
- ['hg19_and_mm10-1.2.0', 'Human GRCh37 (hg19) + Mouse GRCm38 (mm10) release 84']
- ['ercc92-1.2.0', 'ERCC.92 Spike-In'] - ['ercc92-1.2.0', 'ERCC.92 Spike-In']
required: true required: true
description: | description: |
...@@ -142,8 +144,9 @@ workflow_parameters: ...@@ -142,8 +144,9 @@ workflow_parameters:
- id: version - id: version
type: select type: select
default: '3.0.2' default: '3.1.0'
choices: choices:
- ['3.1.0', '3.1.0']
- ['3.0.2', '3.0.2'] - ['3.0.2', '3.0.2']
- ['3.0.1', '3.0.1'] - ['3.0.1', '3.0.1']
- ['2.1.1', '2.1.1'] - ['2.1.1', '2.1.1']
......
...@@ -36,8 +36,9 @@ To Run: ...@@ -36,8 +36,9 @@ To Run:
* *'hg19-1.2.0'* = Human GRCh37 (hg19) release 84 * *'hg19-1.2.0'* = Human GRCh37 (hg19) release 84
* *'mm10-3.0.0'* = Human GRCm38 (mm10) release 93 * *'mm10-3.0.0'* = Human GRCm38 (mm10) release 93
* *'mm10-3.0.0'* = Human GRCm38 (mm10) release 84 * *'mm10-3.0.0'* = Human GRCm38 (mm10) release 84
* *'hg19_and_mm10-3.0.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm19) release 93 * *'GRCh38-and-mm10-3.1.0'* = Human GRCh38 + Mouse GRCm38 (mm10) release 93
* *'hg19_and_mm10-1.2.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm19) release 84 * *'hg19_and_mm10-3.0.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm10) release 93
* *'hg19_and_mm10-1.2.0'* = Human GRCh37 (hg19) + Mouse GRCm38 (mm10) release 84
* *'ercc92-1.2.0'* = ERCC.92 Spike-In * *'ercc92-1.2.0'* = ERCC.92 Spike-In
* **expect cells** * **expect cells**
* Expected number of recovered cells. * Expected number of recovered cells.
......
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
* Anaconda (Anaconda Software Distribution, [https://anaconda.com](https://anaconda.com)) * Anaconda (Anaconda Software Distribution, [https://anaconda.com](https://anaconda.com))
2. **cellranger** 2. **cellranger**
* Cellranger mkfastq [https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/mkfastq](https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/mkfastq) * Cellranger count [https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/count](https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/count)
3. **MultiQc**: 3. **MultiQc**:
* Ewels P., Magnusson M., Lundin S. and Käller M. 2016. MultiQC: Summarize analysis results for multiple tools and samples in a single report. Bioinformatics 32(19): 3047–3048. doi:[10.1093/bioinformatics/btw354](https://dx.doi.org/10.1093/bioinformatics/btw354) * Ewels P., Magnusson M., Lundin S. and Käller M. 2016. MultiQC: Summarize analysis results for multiple tools and samples in a single report. Bioinformatics 32(19): 3047–3048. doi:[10.1093/bioinformatics/btw354](https://dx.doi.org/10.1093/bioinformatics/btw354)
......
LoadData <- function(dim=c("pca","tsne","umap"),dir,samp){
exp <- readMM(paste0(dir,samp,"/outs/filtered_feature_bc_matrix/matrix.mtx.gz"))
features <- read.table(paste0(dir,samp,"/outs/filtered_feature_bc_matrix/features.tsv.gz"),quote="\t")
barcodes <- read.table(paste0(dir,samp,"/outs/filtered_feature_bc_matrix/barcodes.tsv.gz"),quote="\t")
exp.raw <- readMM(paste0(dir,samp,"/outs/raw_feature_bc_matrix/matrix.mtx.gz"))
features.raw <- read.table(paste0(dir,samp,"/outs/filtered_feature_bc_matrix/features.tsv.gz"),quote="\t")
barcodes.raw <- read.table(paste0(dir,samp,"/outs/raw_feature_bc_matrix/barcodes.tsv.gz"),quote="\t")
dr <- list()
for (i in dim){
if (i=="pca"){
dr[[i]] <- read.csv(paste0(dir,samp,"/outs/analysis/",i,"/10_components/projection.csv"))
dr[[i]] <- dr[[i]][,1:3]
} else {
dr[[i]] <- read.csv(paste0(dir,samp,"/outs/analysis/",i,"/2_components/projection.csv"))
}
}
rm(i)
cl <- c("graphclust",paste0("kmeans_",2:10,"_clusters"))
cluster <- list()
deg <- list()
for (i in cl){
cluster[[i]] <- read.csv(paste0(dir,samp,"/outs/analysis/clustering/",i,"/clusters.csv"))
deg[[i]] <- read.csv(paste0(dir,samp,"/outs/analysis/diffexp/",i,"/differential_expression.csv"))
}
rm(i)
qc <- read.csv(paste0(dir,samp,"/outs/metrics_summary.csv"))
qc <- data.frame(t(qc))
qc[,2] <- qc[,1]
qc[,1] <- rownames(qc)
qc <- qc[-1,]
colnames(qc) <- c("Metric","Value")
results <- list(
sample=samp,
exp=exp,
features=features,
barcodes=barcodes,
exp.raw=exp.raw,
features.raw=features.raw,
barcodes.raw=barcodes.raw,
dr=dr,
cluster=cluster,
deg=deg,
qc=qc
)
return(results)
}
Plot.cluster <- function(dr,cluster){
pl <-merge(dr,cluster,by="Barcode",all.x=TRUE)
axis.labs <- colnames(pl)[2:3]
colnames(pl)[2:3] <- c("dim1","dim2")
plot.out <- ggplot(pl,aes(x=dim1,y=dim2,col=factor(Cluster)))+geom_point()+scale_color_viridis(discrete=TRUE)+xlab(axis.labs[1])+ylab(axis.labs[2])+labs(col="Cluster")+theme_cowplot()
return(plot.out)
}
Plot.feature <- function(dr,ft){
dr$exp <- ft
axis.labs <- colnames(dr)[2:3]
colnames(dr)[2:3] <- c("dim1","dim2")
if (sum(dr$exp) != 0){
plot.out <- ggplot(dr,aes(x=dim1,y=dim2,col=exp))+geom_point()+scale_color_viridis(option="inferno")+xlab(axis.labs[1])+ylab(axis.labs[2])+labs(col="Expression")+theme_cowplot()
} else {
plot.out <- ggplot(dr,aes(x=dim1,y=dim2,col=exp))+geom_point()+scale_color_gradient(low="black",high="black")+xlab(axis.labs[1])+ylab(axis.labs[2])+labs(col="Expression")+theme_cowplot()
}
return(plot.out)
}
Plot.violinbox <- function(ft,cluster){
cluster$exp <- ft
cluster$Cluster <- factor(cluster$Cluster)
plot.out <- ggplot(cluster,aes(x=Cluster,y=exp,fill=Cluster))+geom_violin(scale="width",trim=TRUE)+geom_boxplot(width=0.25,fill="white",outlier.shape=NA)+scale_fill_viridis(discrete=TRUE)+ylab("Expression")+theme_cowplot()+theme(axis.text.x=element_text(angle=45))
return(plot.out)
}
Plot.cliffknee <- function(barcodes.raw,exp.raw,barcodes){
umi <- matrix(nrow=length(barcodes.raw[,1]),ncol=4)
umi[,1] <- t(colSums(exp.raw))
umi[,2] <- as.character(barcodes.raw[,1])
umi <- as.data.frame(umi)
umi[,1] <- as.numeric(levels(umi[,1]))[umi[,1]]
colnames(umi) <- c("nUMI","barcodes","rank","Cell")
umi <- umi[order(umi$nUMI,decreasing=TRUE),]
umi$rank <- 1:nrow(umi)
umi$Cell <- factor(as.character(umi$barcodes %in% barcodes[,1]))
umi$Cell <- factor(umi$Cell,levels(umi$Cell)[c(2,1)])
plot.out <- ggplot(umi,aes(x=rank,y=nUMI,col=Cell))+geom_point()+scale_color_manual(values=c("darkgreen","darkred"))+
scale_x_log10(breaks=trans_breaks("log10",function(x) 10^x),labels=trans_format("log10",math_format(10^.x)))+
scale_y_log10(breaks=trans_breaks("log10",function(x) 10^x),labels=trans_format("log10",math_format(10^.x)))+
annotation_logticks()+xlab("Barcode Rank")+ylab("nUMI")+theme_cowplot()
return(plot.out)
}
library(shiny)
library(shinythemes)
library(Matrix)
library(ggplot2)
library(RColorBrewer)
library(viridis)
library(cowplot)
library(scales)
setwd("../")
source("./vizapp/functions.R")
#load data
dir.shared <- "/work/BICF/s189701/cellranger_count/workflow/output/count310/"
samples <- list.dirs(dir.shared,full.names=FALSE,recursive=FALSE)
results <- LoadData(dim=c("umap","tsne","pca"),dir=dir.shared,samp=samples[1])
# This example implements a simple file browser for accessing results.
library(shiny) library(shiny)
library(shinyFiles) library(shinythemes)
library(Matrix)
# Results are available in the directory specified by the outputDir environment library(ggplot2)
# variable, red by Sys.getenv library(RColorBrewer)
library(viridis)
rootdir <- Sys.getenv('outputDir') library(cowplot)
library(scales)
shinyServer(function(input, output, session) {
# The backend for a simple file chooser, restricted to the
# rootdir we obtained above.
# See https://github.com/thomasp85/shinyFiles
shinyFileChoose(input, 'files', roots=c('workflow'=rootdir), filetypes=c('', 'bed', 'xls','wig'), session=session)
shinyServer(function(input,output,session){
#reactive inputs
values <- reactiveValues(
results=results,
dr.c="",cluster.c="",
dr.f="",feature.f="",
cluster.vb="",feature.vb="",
cluster.deg="",cluster.1.deg="",alpha.deg="",fc.deg="",direction.deg=""
)
values$results <- eventReactive(input$go.d,{
LoadData(dim=c("umap","tsne","pca"),dir=dir.shared,samp=input$sample)
})
output$loaded <- eventReactive(input$go.d,{
paste0("Sample ",values$results()$sample)
})
output$nav.analysis <- renderUI({
req(values$results()$sample)
tabsetPanel(type="tabs",
tabPanel("Cluster Plot",
sidebarPanel(
h3(textOutput("lab.sample.c")),
uiOutput("dr.c"),
uiOutput("cluster.c"),
actionButton("go.c","Submit")
),
mainPanel(
h3(textOutput("lab.dr.c")),
h4(textOutput("lab.cluster.c")),
plotOutput("plot.cluster.c"),
tags$br(),
uiOutput("button.download.c")
)
),
tabPanel("Feature Plot",
sidebarPanel(
h3(textOutput("lab.sample.f")),
uiOutput("dr.f"),
uiOutput("feature.f"),
actionButton("go.f","Submit")
),
mainPanel(
h3(textOutput("lab.feature.f")),
h4(textOutput("lab.dr.f")),
plotOutput("plot.feature"),
tags$br(),
uiOutput("button.download.f")
)
),
tabPanel("ViolinBox Plot",
sidebarPanel(
h3(textOutput("lab.sample.vb")),
uiOutput("cluster.vb"),
uiOutput("feature.vb"),
actionButton("go.vb","Submit"),
br(),
br(),
plotOutput("plot.cluster.vb")
),
mainPanel(
h3(textOutput("lab.feature.vb")),
h4(textOutput("lab.cluster.vb")),
plotOutput("plot.violinbox"),
tags$br(),
uiOutput("button.download.vb"),
tags$br(),
tags$br(),
tableOutput("table.expression.vb")
)
),
tabPanel("Differentially Expressed Features",
sidebarPanel(
h3(textOutput("lab.sample.deg")),
uiOutput("cluster.deg"),
uiOutput("cluster.1.deg"),
numericInput("alpha.deg","Maximum p-value",min=0,max =1,value=0.05,step=0.001),
numericInput("fc.deg","Minimum Fold Change",min=0,max =50,value=0,step=0.5),
radioButtons("direction.deg","Fold Change Direction",c("Positively Expressed"="up","Negatively Expressed"="down","Both"="both"),selected="up"),
actionButton("go.deg","Submit"),
br(),
br(),
plotOutput("plot.cluster.deg")
),
mainPanel(
h3(textOutput("lab.cluster.1.deg")),
h4(textOutput("lab.cluster.deg")),
h5(textOutput("lab.alpha.deg")),
h5(textOutput("lab.fc.deg")),
uiOutput("download.deg.button"),
tags$br(),
tags$br(),
tableOutput("table.deg")
)
)
)
})
output$nav.qc <- renderUI({
req(values$results()$sample)
tabPanel("Output",
sidebarPanel(
h3(textOutput("lab.sample.qc")),
h4(textOutput("lab.cell.count")),
br(),
tableOutput("qc")
),
mainPanel(
plotOutput("plot.cliffknee")
)
)
})
#sidebar input/outputs
output$lab.sample.c <- reactive({paste0("Sample ",values$results()$sample)})
output$dr.c <- renderUI({selectInput("dr.c","Dimentionality Reduction",names(values$results()$dr))})
output$cluster.c <- renderUI({selectInput("cluster.c","Clustering",names(values$results()$cluster))})
output$lab.sample.f <- reactive({paste0("Sample ",values$results()$sample)})
output$dr.f <- renderUI({selectInput("dr.f","Dimentionality Reduction",names(values$results()$dr))})
output$feature.f <- renderUI({textInput("feature.f","Feature",levels(values$results()$features[,2])[1])})
output$lab.sample.vb <- reactive({paste0("Sample ",values$results()$sample)})
output$cluster.vb <- renderUI({selectInput("cluster.vb","Clustering",names(values$results()$cluster))})
output$feature.vb <- renderUI({textInput("feature.vb","Feature",levels(values$results()$features[,2])[1])})
output$plot.cluster.vb <- renderPlot({
req(input$go.vb)
print(plot.cluster.vb())
})
output$lab.sample.deg <- reactive({paste0("Sample ",values$results()$sample)})
output$cluster.deg <- renderUI({selectInput("cluster.deg","Clustering",names(values$results()$cluster))})
output$cluster.1.deg <- renderUI({
clust <- levels(factor(values$results()$cluster[[req(input$cluster.deg)]]$Cluster))
selectInput("cluster.1.deg", "Specific Cluster",clust,clust[1])
})
observe({
if (!is.numeric(input$alpha.deg)) {
updateNumericInput(session,"alpha","Maximum p-value",min=0,max =1,value=0.05,step=0.001)
}
})
observe({
if (!is.numeric(input$fc.deg)) {
updateNumericInput(session,"fc","Minimum Fold Change",min=0,max =50,value=0,step=0.5)
}
})
output$lab.sample.qc <- reactive({paste0("Sample ",values$results()$sample)})
output$lab.cell.count <- renderText({paste0(nrow(values$results()$barcodes)," Cells Detected")})
output$qc <- renderTable(values$results()$qc)
#main input/outputs
output$lab.dr.c <- eventReactive(input$go.c,{
req(input$go.c)
paste0(toupper(req(input$dr.c))," Reduction")
},ignoreNULL=FALSE)
output$lab.cluster.c <- eventReactive(input$go.c,{
req(input$go.c)
paste0(toupper(req(input$cluster.c))," Clustering")
},ignoreNULL=FALSE)
output$plot.cluster.c <- renderPlot({
req(input$go.c)
print(plot.cluster.c())
})
output$lab.feature.f <- eventReactive(input$go.f,{
req(input$go.f)
if(toupper(req(input$feature.f)) %in% as.character(values$results()$features[,2])){
toupper(req(input$feature.f))
} else {
paste0(req(input$feature.f)," NOT PRESENT")
}
},ignoreNULL=FALSE)
output$lab.dr.f <- eventReactive(input$go.f,{
req(input$go.f)
paste0(toupper(req(input$dr.f))," Reduction")
},ignoreNULL=FALSE)
output$plot.feature <- renderPlot({
req(input$go.f)
print(plot.feature())
})
output$lab.feature.vb <- eventReactive(input$go.vb,{
req(input$go.vb)
if(toupper(req(input$feature.vb)) %in% as.character(values$results()$features[,2])){
toupper(req(input$feature.vb))
} else {
paste0(req(input$feature.vb)," NOT PRESENT")
}
},ignoreNULL=FALSE)
output$lab.cluster.vb <- eventReactive(input$go.vb,{
req(input$go.vb)
paste0(toupper(req(input$cluster.vb))," Clustering")
},ignoreNULL=FALSE)
output$plot.violinbox <- renderPlot({
req(input$go.vb)
print(plot.violinbox())
})
output$table.expression.vb <- renderTable({
req(input$go.vb)
table.expression.vb()
})
output$lab.cluster.1.deg <- eventReactive(input$go.deg,{
req(input$go.deg)
paste0("Cluster ",req(input$cluster.1.deg)," DEGs")
},ignoreNULL=FALSE)
output$lab.cluster.deg <- eventReactive(input$go.deg,{
req(input$go.deg)
paste0(toupper(req(input$cluster.deg))," Clustering")
},ignoreNULL=FALSE)
output$lab.alpha.deg <- eventReactive(input$go.deg,{
req(input$go.deg)
paste0("Maximum p-value <- ",req(input$alpha.deg))
},ignoreNULL=FALSE)
output$lab.fc.deg <- eventReactive(input$go.deg,{
req(input$go.deg)
if (req(input$direction.deg) == "up"){
paste0("Minimum fold-change difference <- ",req(input$fc.deg)," above the other cells")
} else if (req(input$direction.deg) == "down") {
paste0("Minimum fold-change difference <- ",req(input$fc.deg)," below the other cells")
} else {
paste0("Minimum fold-change difference <- ",req(input$fc.deg)," above & below the other cells")
}
},ignoreNULL=FALSE)
output$table.deg <- renderTable({
req(input$go.deg)
table.deg()
})
output$plot.cluster.deg <- renderPlot({
req(input$go.deg)
print(plot.cluster.deg())
})
output$download.deg.button <- renderUI({
req(input$go.deg)
downloadButton("download.deg","Download Table")
})
output$plot.cliffknee <- renderPlot({print(Plot.cliffknee(barcodes.raw=values$results()$barcodes.raw,exp.raw=values$results()$exp.raw,barcodes=values$results()$barcodes))})
#reactive download buttons
output$button.download.c <- renderUI({
req(input$go.c)
downloadButton("download.c","Download Figure")
})
output$button.download.f <- renderUI({
req(input$go.f)
downloadButton("download.f","Download Figure")
})
output$button.download.vb <- renderUI({
req(input$go.vb)
downloadButton("download.vb","Download Figure")
})
output$button.download.deg <- renderUI({
req(input$go.deg)
downloadButton("download.deg","Download Table")
})
#save current parameters
values$dr.c <- eventReactive(input$go.c,{input$dr.c})
values$cluster.c <- eventReactive(input$go.c,{input$cluster.c})
values$dr.f <- eventReactive(input$go.f,{input$dr.f})
values$feature.f <- eventReactive(input$go.f,{toupper(input$feature.f)})
values$cluster.vb <- eventReactive(input$go.vb,{input$cluster.vb})
values$feature.vb <- eventReactive(input$go.vb,{toupper(input$feature.vb)})
values$cluster.deg <- eventReactive(input$go.deg,{input$cluster.deg})
values$cluster.1.deg <- eventReactive(input$go.deg,{input$cluster.1.deg})
values$alpha.deg <- eventReactive(input$go.deg,{input$alpha.deg})
values$fc.deg <- eventReactive(input$go.deg,{input$fc.deg})
values$direction.deg <- eventReactive(input$go.deg,{input$direction.deg})
#plot functions
plot.cluster.c <- eventReactive(input$go.c,{
Plot.cluster(
values$results()$dr[[req(input$dr.c)]],values$results()$cluster[[req(input$cluster.c)]])
},ignoreNULL=FALSE)
plot.feature <- eventReactive(input$go.f,{
if(toupper(req(input$feature.f)) %in% as.character(values$results()$features[,2])){
Plot.feature(values$results()$dr[[req(input$dr.f)]],values$results()$exp[as.numeric(rownames(values$results()$features[values$results()$features[,2]==toupper(req(input$feature.f)),])),])
} else {
plot.new()
}
},ignoreNULL=FALSE)
plot.violinbox <- eventReactive(input$go.vb,{
if(toupper(req(input$feature.vb)) %in% as.character(values$results()$features[,2])){
Plot.violinbox(values$results()$exp[as.numeric(rownames(values$results()$features[values$results()$features[,2]==toupper(req(input$feature.vb)),])),],values$results()$cluster[[req(input$cluster.vb)]])
} else {
plot.new()
}
},ignoreNULL=FALSE)
plot.cluster.vb <- eventReactive(input$go.vb,{
Plot.cluster(
values$results()$dr[["umap"]],values$results()$cluster[[req(input$cluster.vb)]])
},ignoreNULL=FALSE)
plot.cluster.deg <- eventReactive(input$go.deg,{
req(input$cluster.1.deg)
Plot.cluster(
values$results()$dr[["umap"]],values$results()$cluster[[input$cluster.deg]])
},ignoreNULL=FALSE)
#table functions
table.expression.vb <- eventReactive(input$go.vb,{
if(toupper(req(input$feature.vb)) %in% as.character(values$results()$features[,2])){
tab <- values$results()$deg[[req(input$cluster.vb)]][values$results()$deg[[req(input$cluster.vb)]][,2]==toupper(req(input$feature.vb)),c(2,grep("Mean.Counts",names(values$results()$deg[[req(input$cluster.vb)]])))]
colnames(tab) <- c("Feature",gsub(".Mean.Counts","",names(tab[,-1])))
tab <- data.frame(t(tab))
tab[,2] <- tab[,1]
tab[,1] <- rownames(tab)
tab <- tab[-1,]
colnames(tab) <- c("Cluster",paste0("Ave ",req(input$feature.vb)," Expression"))
tab
} else {
""
}
},ignoreNULL=FALSE)
table.deg <- eventReactive(input$go.deg,{
tab <- values$results()$deg[[input$cluster.deg]]
tab <- tab[tab[paste0("Cluster.",req(input$cluster.1.deg),".Adjusted.p.value")]<=input$alpha.deg,]
if (input$fc.deg == 0){
fc <- 0
} else {
fc <- log2(input$fc.deg)
}
if (input$direction.deg=="up"){
tab <- tab[tab[paste0("Cluster.",req(input$cluster.1.deg),".Log2.fold.change")]>=fc,]
} else if (input$direction.deg=="down"){
tab <- tab[tab[paste0("Cluster.",req(input$cluster.1.deg),".Log2.fold.change")]<=-fc,]
} else {
tab <- tab[tab[paste0("Cluster.",req(input$cluster.1.deg),".Log2.fold.change")]>=fc | tab[paste0("Cluster.",req(input$cluster.1.deg),".Log2.fold.change")]<=-fc,]
}
tab <- tab[,c(colnames(values$results()$deg[[input$cluster.deg]])[2],paste0("Cluster.",req(input$cluster.1.deg),".Mean.Counts"),paste0("Cluster.",req(input$cluster.1.deg),".Log2.fold.change"),paste0("Cluster.",req(input$cluster.1.deg),".Adjusted.p.value"))]
tab <- tab[order(abs(tab[paste0("Cluster.",req(input$cluster.1.deg),".Log2.fold.change")]),decreasing=TRUE),]
tab
},ignoreNULL=FALSE)
#download
output$download.c <- downloadHandler(
filename <- function(){paste0("clusters_",values$dr.c(),"_",values$cluster.c(),".pdf")},
content <- function(file){
pdf(file,onefile=FALSE)
print(plot.cluster.c())
dev.off()
}
)
output$download.f <- downloadHandler(
filename <- function(){paste0("feature_",values$dr.f(),"_",toupper(values$feature.f()),".pdf")},
content <- function(file){
pdf(file,onefile=FALSE)
print(plot.feature())
dev.off()
}
)
output$download.vb <- downloadHandler(
filename <- function(){paste0("violinbox_",values$cluster.vb(),"_",toupper(values$feature.vb()),".pdf")},
content <- function(file){
pdf(file,onefile=FALSE)
print(plot.violinbox())
dev.off()
}
)
output$download.deg <- downloadHandler(
filename <- function(){paste0("deg_",values$cluster.deg(),"_cluster.",values$cluster.1.deg(),"_p",values$alpha.deg(),"_fc",values$fc.deg(),values$direction.deg(),".csv")},
content <- function(file){
write.csv(table.deg(),file,row.names=FALSE,quote=FALSE)
}
)
}) })
library(shiny) navbarPage(theme=shinytheme("slate"),"BICF Cellranger Count Analysis (cellranger 3.1.0 version)",
library(shinyFiles) tabPanel("Select Data",
mainPanel(
selectInput("sample","Available Samples",samples),
shinyUI(fluidPage( actionButton("go.d","Load"),
br(),
verticalLayout( br(),
h4("Please whait until the sample has been loaded and a message below displays this, it may take a few minutes"),
# Application title h3(textOutput("loaded"))
titlePanel("Astrocyte Example"),
wellPanel(
helpText("This is a minimal example, demonstrating how
a Shiny visualization application can access the output of a workflow.
Here we provide a file browser using the shinyFiles package. Real
Astrocyte vizapps would provide custom methods to access and visualize
output."),
helpText("The workflow output is in the directory set in the
outputDir environment variable. this can be retrieved in R with the
command Sys.getenv('outputDir')"),
# A simple file browser within the workflow output directory
# See https://github.com/thomasp85/shinyFiles
shinyFilesButton('files', label='Browse workflow output', title='Please select a file', multiple=FALSE)
) )
),
tabPanel("Analysis",
uiOutput("nav.analysis")
),
tabPanel("QC",
uiOutput("nav.qc")
) )
)) )
...@@ -18,6 +18,10 @@ process { ...@@ -18,6 +18,10 @@ process {
module = ['cellranger/3.0.2'] module = ['cellranger/3.0.2']
queue = '128GB,256GB,256GBv1,384GB' queue = '128GB,256GB,256GBv1,384GB'
} }
withLabel: count310 {
module = ['cellranger/3.1.0']
queue = '128GB,256GB,256GBv1,384GB'
}
withLabel: versions { withLabel: versions {
module = ['python/3.6.1-2-anaconda','pandoc/2.7','multiqc/1.7'] module = ['python/3.6.1-2-anaconda','pandoc/2.7','multiqc/1.7']
executor = 'local' executor = 'local'
...@@ -49,6 +53,9 @@ params { ...@@ -49,6 +53,9 @@ params {
'mm10-1.2.0' { 'mm10-1.2.0' {
loc = '/project/apps_database/cellranger/refdata-cellranger-' loc = '/project/apps_database/cellranger/refdata-cellranger-'
} }
'GRCh38_and_mm10-3.1.0' {
loc = '/project/apps_database/cellranger/refdata-cellranger-'
}
'hg19_and_mm10-3.0.0' { 'hg19_and_mm10-3.0.0' {
loc = '/project/apps_database/cellranger/refdata-cellranger-' loc = '/project/apps_database/cellranger/refdata-cellranger-'
} }
......
#!/usr/bin/env nextflow #!/usr/bin/env nextflow
/*
// Path to an input file, or a pattern for multiple inputs main.nf
// Note - $baseDir is the location of this workflow file main.nf *
* --------------------------------------------------------------------------
* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
* --------------------------------------------------------------------------
*
*/
// Define Input variables // Define Input variables
params.name = "run" params.name = "run"
...@@ -11,7 +16,7 @@ params.genome = 'GRCh38-3.0.0' ...@@ -11,7 +16,7 @@ params.genome = 'GRCh38-3.0.0'
params.expectCells = 10000 params.expectCells = 10000
params.forceCells = 0 params.forceCells = 0
params.kitVersion = 'three' params.kitVersion = 'three'
params.version = '3.0.2' params.version = '3.1.0'
params.astrocyte = false params.astrocyte = false
params.outDir = "${baseDir}/output" params.outDir = "${baseDir}/output"
params.multiqcConf = "${baseDir}/conf/multiqc_config.yaml" params.multiqcConf = "${baseDir}/conf/multiqc_config.yaml"
...@@ -64,6 +69,7 @@ outDir = params.outDir ...@@ -64,6 +69,7 @@ outDir = params.outDir
multiqcConf = params.multiqcConf multiqcConf = params.multiqcConf
references = params.references references = params.references
process checkDesignFile { process checkDesignFile {
tag "${name}" tag "${name}"
...@@ -100,20 +106,25 @@ samples.into { ...@@ -100,20 +106,25 @@ samples.into {
samples211 samples211
samples301 samples301
samples302 samples302
samples310
} }
refLocation.into { refLocation.into {
refLocation211 refLocation211
refLocation301 refLocation301
refLocation302 refLocation302
refLocation310
} }
expectCells211 = expectCells expectCells211 = expectCells
expectCells301 = expectCells expectCells301 = expectCells
expectCells302 = expectCells expectCells302 = expectCells
expectCells310 = expectCells
forceCells211 = forceCells forceCells211 = forceCells
forceCells301 = forceCells forceCells301 = forceCells
forceCells302 = forceCells forceCells302 = forceCells
forceCells310 = forceCells
chemistryParam301 = chemistryParam chemistryParam301 = chemistryParam
chemistryParam302 = chemistryParam chemistryParam302 = chemistryParam
chemistryParam310 = chemistryParam
process count211 { process count211 {
...@@ -143,7 +154,7 @@ process count211 { ...@@ -143,7 +154,7 @@ process count211 {
ulimit -a ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref} bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells211} cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells211}
sed -E 's/("([^"]*)")?,/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
""" """
} }
else { else {
...@@ -152,7 +163,7 @@ process count211 { ...@@ -152,7 +163,7 @@ process count211 {
ulimit -a ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref} bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells211} cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells211}
sed -E 's/("([^"]*)")?,/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
""" """
} }
...@@ -187,7 +198,7 @@ process count301 { ...@@ -187,7 +198,7 @@ process count301 {
ulimit -a ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref} bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells301} --chemistry=${chemistryParam301} cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells301} --chemistry=${chemistryParam301}
sed -E 's/("([^"]*)")?,/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
""" """
} }
else { else {
...@@ -196,7 +207,7 @@ process count301 { ...@@ -196,7 +207,7 @@ process count301 {
ulimit -a ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref} bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells301} --chemistry=${chemistryParam301} cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells301} --chemistry=${chemistryParam301}
sed -E 's/("([^"]*)")?,/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
""" """
} }
...@@ -231,7 +242,7 @@ process count302 { ...@@ -231,7 +242,7 @@ process count302 {
ulimit -a ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref} bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells302} --chemistry=${chemistryParam302} cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells302} --chemistry=${chemistryParam302}
sed -E 's/("([^"]*)")?,/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
""" """
} }
else { else {
...@@ -240,7 +251,51 @@ process count302 { ...@@ -240,7 +251,51 @@ process count302 {
ulimit -a ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref} bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells302} --chemistry=${chemistryParam302} cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells302} --chemistry=${chemistryParam302}
sed -E 's/("([^"]*)")?,/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
"""
}
}
process count310 {
queue '128GB,256GB,256GBv1,384GB'
tag "${sample}"
publishDir "${outDir}/${task.process}", mode: 'copy'
module 'cellranger/3.1.0'
input:
set sample, file("${sample}_S?_L001_R1_001.fastq.gz"), file("${sample}_S?_L001_R2_001.fastq.gz") from samples310
file ref from refLocation310.first()
expectCells310
forceCells310
chemistryParam310
output:
file("**/outs/**") into outPaths310
file("*_metrics_summary.tsv") into metricsSummary310
when:
version == '3.1.0'
script:
if (forceCells310 == 0) {
"""
hostname
ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --expect-cells=${expectCells310} --chemistry=${chemistryParam310}
sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
"""
}
else {
"""
hostname
ulimit -a
bash ${baseDir}/scripts/filename_check.sh -r ${ref}
cellranger count --id=${sample} --transcriptome=./${ref} --fastqs=. --sample=${sample} --force-cells=${forceCells310} --chemistry=${chemistryParam310}
sed -E 's/("([^"]*)")?(,|\$)/\\2\t/g' ${sample}/outs/metrics_summary.csv | tr -d "," | sed "s/^/${sample}\t/" > ${sample}_metrics_summary.tsv
""" """
} }
...@@ -272,10 +327,9 @@ process versions { ...@@ -272,10 +327,9 @@ process versions {
} }
metricsSummary = metricsSummary211.mix(metricsSummary301, metricsSummary302) metricsSummary = metricsSummary211.mix(metricsSummary301, metricsSummary302, metricsSummary310)
// Generate MultiQC Report
process multiqc { process multiqc {
tag "${name}" tag "${name}"
...@@ -299,4 +353,4 @@ process multiqc { ...@@ -299,4 +353,4 @@ process multiqc {
multiqc -c ${multiqcConf} . multiqc -c ${multiqcConf} .
""" """
} }
\ No newline at end of file
#!/usr/bin/env python3 #!/usr/bin/env python3
#check_design.py
'''Check if design file is correctly formatted and matches files list.''' #*
#* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
import argparse import argparse
import logging import logging
...@@ -11,8 +15,6 @@ For more details: ...@@ -11,8 +15,6 @@ For more details:
%(prog)s --help %(prog)s --help
''' '''
# SETTINGS
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler()) logger.addHandler(logging.NullHandler())
logger.propagate = False logger.propagate = False
...@@ -61,6 +63,7 @@ def check_design_headers(design): ...@@ -61,6 +63,7 @@ def check_design_headers(design):
return design return design
def check_files(design, fastq): def check_files(design, fastq):
'''Check if design file has the files found.''' '''Check if design file has the files found.'''
...@@ -104,4 +107,4 @@ def main(): ...@@ -104,4 +107,4 @@ def main():
new_design_df.to_csv('design.checked.csv', header=True, sep=',', index=False) new_design_df.to_csv('design.checked.csv', header=True, sep=',', index=False)
if __name__ == '__main__': if __name__ == '__main__':
main() main()
\ No newline at end of file
#!/bin/bash #!/bin/bash
#filename_check.sh #filename_check.sh
#*
#* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
usage() { usage() {
echo "-r --ref file" echo "-r --ref file"
...@@ -28,4 +33,4 @@ if [ $(echo "${ref}" | tr -d ' ') != "${ref}" ]; then ...@@ -28,4 +33,4 @@ if [ $(echo "${ref}" | tr -d ' ') != "${ref}" ]; then
echo "Error: Spaces found in Reference Files" echo "Error: Spaces found in Reference Files"
echo ${ref} echo ${ref}
exit 21 exit 21
fi fi
\ No newline at end of file
#!/usr/bin/env python3 #!/usr/bin/env python3
#generate_references.py
'''Make header for HTML of references.''' #*
#* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
import argparse import argparse
import subprocess import subprocess
...@@ -12,8 +16,6 @@ For more details: ...@@ -12,8 +16,6 @@ For more details:
%(prog)s --help %(prog)s --help
''' '''
# SETTINGS
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler()) logger.addHandler(logging.NullHandler())
logger.propagate = False logger.propagate = False
...@@ -65,4 +67,4 @@ def main(): ...@@ -65,4 +67,4 @@ def main():
if __name__ == '__main__': if __name__ == '__main__':
main() main()
\ No newline at end of file
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- #generate_versions.py
#*
'''Make YAML of software versions.''' #* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
from __future__ import print_function from __future__ import print_function
from collections import OrderedDict from collections import OrderedDict
...@@ -15,7 +18,6 @@ For more details: ...@@ -15,7 +18,6 @@ For more details:
%(prog)s --help %(prog)s --help
''' '''
# SETTINGS
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler()) logger.addHandler(logging.NullHandler())
logger.propagate = False logger.propagate = False
...@@ -104,4 +106,4 @@ def main(): ...@@ -104,4 +106,4 @@ def main():
if __name__ == '__main__': if __name__ == '__main__':
main() main()
\ No newline at end of file
#!/usr/bin/env python3 #!/usr/bin/env python3
#test_check_design.py
#*
#* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
import pytest import pytest
import pandas as pd import pandas as pd
...@@ -19,3 +25,7 @@ def test_count301_design(): ...@@ -19,3 +25,7 @@ def test_count301_design():
@pytest.mark.count302 @pytest.mark.count302
def test_count302_design(): def test_count302_design():
assert os.path.exists(os.path.join(test_output_path, 'design.checked.csv')) assert os.path.exists(os.path.join(test_output_path, 'design.checked.csv'))
@pytest.mark.count310
def test_count310_design():
assert os.path.exists(os.path.join(test_output_path, 'design.checked.csv'))
\ No newline at end of file
#!/usr/bin/env python3 #!/usr/bin/env python3
#test_count.py
#*
#* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
import pytest import pytest
import pandas as pd import pandas as pd
...@@ -23,3 +29,8 @@ def test_count301_count(): ...@@ -23,3 +29,8 @@ def test_count301_count():
def test_count302_count(): def test_count302_count():
assert os.path.exists(os.path.join(test_output_path, 'count302', 'sample1_metrics_summary.tsv')) assert os.path.exists(os.path.join(test_output_path, 'count302', 'sample1_metrics_summary.tsv'))
assert os.path.exists(os.path.join(test_output_path, 'count302', 'sample1', 'outs')) assert os.path.exists(os.path.join(test_output_path, 'count302', 'sample1', 'outs'))
@pytest.mark.count310
def test_count310_count():
assert os.path.exists(os.path.join(test_output_path, 'count310', 'sample1_metrics_summary.tsv'))
assert os.path.exists(os.path.join(test_output_path, 'count310', 'sample1', 'outs'))
\ No newline at end of file
#!/usr/bin/env python3 #!/usr/bin/env python3
#test_multiqc.py
#*
#* --------------------------------------------------------------------------
#* Licensed under MIT (https://git.biohpc.swmed.edu/BICF/Astrocyte/cellranger_count/blob/develop/LICENSE)
#* --------------------------------------------------------------------------
#*
import pytest import pytest
import pandas as pd import pandas as pd
...@@ -19,3 +25,7 @@ def test_count301_multiqc(): ...@@ -19,3 +25,7 @@ def test_count301_multiqc():
@pytest.mark.count302 @pytest.mark.count302
def test_count302_multiqc(): def test_count302_multiqc():
assert os.path.exists(os.path.join(test_output_path, 'multiqc_report.html')) assert os.path.exists(os.path.join(test_output_path, 'multiqc_report.html'))
@pytest.mark.count310
def test_count310_multiqc():
assert os.path.exists(os.path.join(test_output_path, 'multiqc_report.html'))
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment