If you are familiar with Docker containers, Singularity/Apptainer containers are essentially the same thing, but are better suited for multi-user HPC systems such as LUMI. The main benefit of using a container is that it provides an isolated software environment for each application, which makes it easier to install and manage complex applications.
This page provides guidance on preparing your Singularity/Apptainer containers for use with LUMI. Please consult the container jobs page for guidance on running your container on LUMI.
There are two major providers of the
singularity runtime, namely
Singularity CE and Apptainer, with the latter
being a fork of the former. For most cases these should be fully compatible.
LUMI provides a Singularity CE runtime.
Pulling container images from a registry¶
Singularity allows pulling existing container images (Singularity or Docker)
from container registries such as DockerHub or AMD Infinity
Hub. Pulling container images from registries can be done on
LUMI. For instance, the Ubuntu image
ubuntu:22.04 can be pulled from
DockerHub with the following command:
This will create the Singularity image file
ubuntu_22.04.sif in the directory
where the command was run. Once the image has been pulled, the container can be
run. Instructions for running the container may be found on the container jobs
Take care when pulling container images
Please take care to only use images uploaded from reputable sources as these images can easily be a source of security vulnerabilities or even contain malicious code.
Set cache directories when using Docker containers
When pulling or building from Docker containers using
conversion can be quite heavy. Speed up the conversion and avoid leaving
behind temporary files by using the in-memory filesystem on
/tmp as the
Singularity cache directory, i.e.
Building Apptainer/Singularity SIF containers¶
Building your own container on LUMI is, unfortunately, not in general possible.
singularity build command, in general, requires some level of root
fakeroot, which are disabled on LUMI for security
reasons. Thus, in order to build your own Singularity/Apptainer container for
LUMI, you have two options:
- Use the cotainr tool to build containers on LUMI (only for certain use cases).
- Build your own container on your local hardware, e.g. your laptop.
Building containers using the cotainr tool¶
cotainr is available in the LUMI central software
stack and may be loaded using
When building containers using
cotainr build, you may either specify a base
image for the container yourself (using the
--base-image option) or you may
--system option to use the recommended base images for LUMI. To list
the available systems, run
As an example, you may then use
cotainr build to create a container for
LUMI-G containing a Conda/pip environment by running
my_script.py is your Python script.
Make sure your Conda environment supports the hardware in LUMI
In order to take advantage of e.g. the GPUs in LUMI-G, the
packages you specify in your Conda environment must be compatible with
LUMI-G, i.e. built against ROCm. Similarly, in order to take full advantage
of the Slingshot 11 interconnect when running MPI jobs, you
must make sure your packages are built against Cray MPICH. Cotainr does
not do any magic conversion of the packages specified in the Conda
environment to make sure they fit the hardware of LUMI. It simply installs
the packages exactly as listed in the
cotainr to build a container from a Conda/pip environment is
different from wrapping a Conda/pip environment using the LUMI container
wrapper. Each serve their own purpose. See
the Python installation guide for an overview of
differences and this GitHub issue for a detailed
discussion of the differences.
See the cotainr documentation for more details about
Building containers on local hardware¶
You may also build a Singularity/Apptainer container for LUMI on your local hardware and transfer it to LUMI.
As an example, consider building a container that is compatible with the MPI stack on LUMI.
For MPI-enabled containers, the application inside the container must be dynamically linked to an MPI version that is ABI-compatible with the host MPI.
The following Singularity definition file
mpi_osu.def, installs MPICH-3.1.4, which is ABI-compatible with the
Cray-MPICH found on LUMI. That MPICH will be used to compile the OSU
microbenchmarks. Finally, the OSU point to point bandwidth test
is set as the "runscript" of the image.
bootstrap: docker from: ubuntu:21.04 %post # Install software apt-get update apt-get install -y file g++ gcc gfortran make gdb strace wget ca-certificates --no-install-recommends # Install mpich wget -q http://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz tar xf mpich-3.1.4.tar.gz cd mpich-3.1.4 ./configure --disable-fortran --enable-fast=all,O3 --prefix=/usr make -j$(nproc) make install ldconfig # Build osu benchmarks wget -q http://mvapich.cse.ohio-state.edu/download/mvapich/osu-micro-benchmarks-5.3.2.tar.gz tar xf osu-micro-benchmarks-5.3.2.tar.gz cd osu-micro-benchmarks-5.3.2 ./configure --prefix=/usr/local CC=$(which mpicc) CFLAGS=-O3 make make install cd .. rm -rf osu-micro-benchmarks-5.3.2 rm osu-micro-benchmarks-5.3.2.tar.gz %runscript /usr/local/libexec/osu-micro-benchmarks/mpi/pt2pt/osu_bw
The image can be built on your local hardware (not on LUMI) with