BioHPC currently uses a customized Singularity version of Singularity - ''%%singularity/2.4.2-utsw-2018010%%''
BioHPC currently uses a customized Singularity version of Singularity - `singularity/2.4.2-utsw-2018010`
The codebase from this version tracks singularity/release-2.4 on GitHub, but includes 2 PRs developed by DCT that have not yet been merged into mainstream Singularity release:
* PR 1181 - Docker aufs whiteout handling
* PR 1124 - Workaround to inject bind points in to docker containers on systems without overlayfs available (RHEL6.x)
==== Detail of PR Customizations ====
## Detail of PR Customizations
=== PR 1181 ===
### PR 1181
This PR to Singularity implements correct handling of whiteout files when extracting complex Docker containers. When a file is removed in a Dockerfile build script, which was created by a parent layer, the current layer will contain a ''%%.wh.%%'' marker file noting that the parent file should be removed as the current layer is extracted. Prior to PR1181 this was not implemented correctly in Singularity, so some Docker containers such as ''%%nvidia/digits:6.0%%'' would not run correctly. The PR corrects the behaviour to follow Docker.
This PR to Singularity implements correct handling of whiteout files when extracting complex Docker containers. When a file is removed in a Dockerfile build script, which was created by a parent layer, the current layer will contain a `.wh.` marker file noting that the parent file should be removed as the current layer is extracted. Prior to PR1181 this was not implemented correctly in Singularity, so some Docker containers such as `nvidia/digits:6.0` would not run correctly. The PR corrects the behaviour to follow Docker.
PR1181 will be merged into the main Singularity codebase for 3.0.
=== PR1124 ===
### PR1124
BioHPC runs RHEL6.9, which does not include support for OverlayFS. When running a container, Singularity will attempt to bind cluster filesystems into the container, so that tools inside the container can act on data on cluster filesystems. Without overlayfs, the target of the bind must exist in the container. Docker containers have e.g. a ''%%/home%%'', but not a ''%%/home2%%'', so Singularity cannot bind user home directories on BioHPC.
BioHPC runs RHEL6.9, which does not include support for OverlayFS. When running a container, Singularity will attempt to bind cluster filesystems into the container, so that tools inside the container can act on data on cluster filesystems. Without overlayfs, the target of the bind must exist in the container. Docker containers have e.g. a `/home`, but not a `/home2`, so Singularity cannot bind user home directories on BioHPC.
This PR implements a ''%%docker inject binds%%'' feature, that will inject a layer containing bind points defined in ''%%singularity.conf%%'' into any Docker container that is pulled or run using Singularity. This ensures containers work as users expect, without requiring overlayfs
This PR implements a `docker inject binds` feature, that will inject a layer containing bind points defined in `singularity.conf` into any Docker container that is pulled or run using Singularity. This ensures containers work as users expect, without requiring overlayfs
PR1124 may not be merged into Singularity 3.0 - there has not been a lot of interest in this feature in the Singularity community.
==== Singularity Install Procedure ====
## Singularity Install Procedure
Singularity uses setuid binaries for bind mounting and some other small steps, so cannot be installed entirely from the usual moduleadmin account.
module-whatis "Description: Singularity is a container platform focused on supporting Mobility of Compute. Mobility of Compute encapsulates the development to compute model where developers can work in an environment of their choosing and creation and when the developer needs additional compute resources, this environment can easily be copied and executed on other platforms. Additionally as the primary use case for Singularity is targeted towards computational portability, many of the barriers to entry of other container solutions do not apply to Singularity making it an ideal solution for users (both computational and non-computational) and HPC centers."
