BUG: thread error on Astrocyte

Gervaise Henry
2020-04-20T18:31:32-05:00

Detected by Lavanya (Gupta Lab)...

Exceeding thread limit on Astrocyte intermittently

Appears to be localized to 256v1 nodes (see below, 1574863-1590532)
```
1049334 nf-count3+ FAILED Nucleus024 128GB
1363646 nf-count3+ FAILED Nucleus156 256GBv1
1432212 nf-count3+ FAILED Nucleus010 128GB
1492005 nf-count3+ NODE_FAIL Nucleus025 128GB
1503055 nf-count3+ FAILED Nucleus059 256GB
1503183 nf-count3+ FAILED Nucleus059 256GB
1574863 nf-count3+ FAILED Nucleus142 256GBv1
1581543 nf-count3+ FAILED Nucleus132 256GBv1
1581544 nf-count3+ FAILED Nucleus193 256GBv1
1581550 nf-count3+ FAILED Nucleus132 256GBv1
1582398 nf-count3+ FAILED Nucleus152 256GBv1
1585588 nf-count3+ FAILED Nucleus140 256GBv1
1590532 nf-count3+ FAILED Nucleus140 256GBv1
```

Where ulimit -u is being reduced for an unknown reason
```
Nucleus140
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 1031252
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
```

**FIX**
Add this code to the cellranger process start:
`ulimit -u 16384`