Gitlab CI overridding concurrent pipelines

added bug confirmed critical labels

https://gitlab.com/gitlab-org/gitlab-runner/issues/3688#note_308716493

1. .gitlab-runner/config.toml:

[runners.custom_build_dir]
    enabled = true

2. in .gitlab-ci.yml:

variables:
  GIT_DEPTH: 5
  GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_RUNNER_ID/$CI_CONCURRENT_ID/$CI_PROJECT_NAME/$CI_CONCURRENT_PROJECT_ID

Linked to atacseq_analysis#21 (closed)

Another try:

variables:
  GIT_DEPTH: 5
  GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_RUNNER_SHORT_TOKEN/$CI_CONCURRENT_ID/$CI_PROJECT_NAME/$CI_PIPELINE_ID

Combine with cleanup if all succeeds and this will work.

1. Clone for each job

variables:
  GIT_DEPTH: 5
  GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_RUNNER_SHORT_TOKEN/$CI_PROJECT_NAME/$CI_PIPELINE_ID/$CI_CONCURRENT_ID/$CI_JOB_ID/

2. Remove work after each job succeds to reduce space. Doesn't work, always runs

 after_script:
      - rm -rf work/

3. Remove pipeline directory if it all completed

cleanup_job:
  stage: cleanup
  script:
    - cd $CI_BUILDS_DIR/$CI_RUNNER_SHORT_TOKEN/$CI_PROJECT_NAME
    - rm -fr $CI_PIPELINE_ID/

4. Remove failed pipelines using a cron job that removes directories after 7 days

https://git.biohpc.swmed.edu/snippets/28

mentioned in merge request !68 (merged)

closed via merge request !68 (merged)

mentioned in commit df7ed9bf