Containers
Containers allow users to package an application and all of the software dependencies needed to run it into a single, sandboxed piece of software. Containers enable an application to run quickly and reliably from one computing environment to another.
What is a Container?
A container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings. Users do not need to rely on host system libraries, except for interconnect/MPI libraries, to achieve best performance in internode communication.
Container images become containers at runtime and will always run the same, regardless of the infrastructure. Containers isolate software from its environment and ensure that it works uniformly despite differences for instance between development and staging. Containers significantly improve reproducibility and portability of a scientific workflow.
- Reproducibility
- By using the same software stack, things like library and compiler versions play less of a role in job variability.
- Portability
- Users can build a single container for their workflow and run it on multiple HPC systems with no need to recompile or relink software.
- Collaboration
- Software packaged in a container can be shared with collaborators and other researchers; the only requirement for them to use it is to have a container engine installed on their system.
- Productivity
- If a software package has a complex tree of dependencies that require lots of time to install, solve this issue once in a container and it will be solved for good.
What is Singularity?
To build and run container workloads on Pawsey systems, users can use the open-source software Singularity, which is our preferred software for running container images.
Singularity is a free, cross-platform and open-source computer program that performs containerisation. One of the main uses of Singularity is to bring containers and reproducibility to scientific computing and the high-performance computing (HPC) world.
- Performance
- Singularity allows native execution of GPU-enabled containers.
- Configuration of MPI and high-speed interconnect is also supported on Singularity.
- Security
- Singularity allows users to run container workflows with the same privileges and permissions they would normally have on Pawsey systems.
Related pages
The following pages discuss containers, running Singularity, and some specific and more complex use cases.
- Use with Singularity — Singularity is a container platform: it lets you create and run containers that package up pieces of software in a way that is portable and reproducible. With a few basic commands, you can set up a workflow to run on Pawsey systems using Singularity. This page introduces how to get or build a Singularity container image and run that image on HPC systems.
- SHPC (Singularity Registry HPC) — SHPC is a utility that allows the installation of software containers in the form of container modules.
- Pawsey-Provided Container Images — Pawsey develops and supports a limited number of images on public repositories, to assist researchers in building and running optimally performing containers.