Installing software with EasyBuild¶
Most software in the central LUMI software stacks is installed through EasyBuild. The central software stack is kept as compact as possible to ease maintenance and to avoid user confusion. E.g., packages for which users request special customisations will never be installed in the central software stack. Moreover, due to the technical implementation of the software stack on a system the size of LUMI, installing software can be disruptive so new software is mostly made available during maintenance intervals.
This however does not mean that you may have to wait for weeks before you can get the software you need for your project in LUMI. We made it very easy to install additional software in your home or project directories (where the latter is the better choice as you can then share with the other people in your project) and use that software. It requires not much more than loading a module that configure EasyBuild for local installations and running EasyBuild with a few recipes that can be supplied by the LUMI User Support Team or your national support team or that you may write yourself. And this software is then build in exactly the same way as it would be in a central installation.
Beginner's guide to installing software on LUMI¶
We support installing software with EasyBuild only in the LUMI software stacks, not in CrayEnv.
By default our EasyBuild setup will install software in
this location can be changed by pointing the environment variable
to the directory where you want to create the software installation. In most cases
a subdirectory in your
/project/project_* directory is the best location to install
software as that directory is both permanent for the duration of your project and
shared with all users in your project so that everybody can use the software. It is
a very good idea to set this environment variable in your
Tip for users with multiple projects
If you participate
in multiple projects, you'll have to either have only a very personal software
setup in your home directory which noone else can use, or a setup in each of the
project directories as sharing of project directories across projects is not possible.
Our modules can also support only one user software setup at a time. However, you can
always switch to a different setup by changing the value of the
environment variable, but you should only do so when no modules are loaded,
not even the
LUMI module. Hence you should always do a
EBU_USER_PREFIX. If you fail to do so, the old user module directories will not be removed from the module search path, not even if you reload the
LUMImodule, and you may get very unexpected results from module load operations.
From now on you will also see the software that you have installed yourself for the
selected version of the LUMI software stack and partition when you do
module spider will also search those directories.
The second step is to ensure that the right version of the software stack is loaded.
Assume that we want to install software in the
LUMI/22.06 stack, then one needs to
partitionmodule for the part of LUMI you are on. See also the page on the software stacks.
Issue: Only partition/L and partition/C are currently fully supported
Note that in the initial versions of the software stack, only
partition/C are supported. The
partition/EAP module is for the Early
Access Platform based on the MI100 GPUs. This early access platform is meant for
developers and the LUMI User Support Team does not do any software installations
there beyond the basic build tools provided there.
partition/G module is for all MI250X nodes, wether in the regular LUMI-G
partition or in the temporary second early access platform, snd similarly is only
meant for users who install their own software and not supported by the LUMI
User Support Team except for basic build tools until after the LUMI-G pilot phase.
Though it is technically possible to cross-compile software for a different partition, it may not be without problems as not all install scripts that come with software support cross-compiling and as tests may fail when compiling for a CPU with instructions that the host CPU does not support.
The next step to install software in the directory you have just indicated, is to load
Now an EasyBuild build recipe is a file with a name that consists of different components. Consider, e.g., the build recipe
GROMACS, is the name of the package. The second part of the name,
2021.4is the version of GROMACS, in this case the initial 2021 release. The next part,
cpeGNU-22.06, denotes the so-called toolchain used for the build. The
cpeGNUtoolchain uses the
PrgEnv-gnuprogramming environment, the
PrgEnv-aoccenvironment and the
PrgEnv-amdenvironment. The version of the toolchain should match the version of the LUMI software stack or the installation will fail. (In fact, it is not just the version in the file name that should match but the version of the toolchain that is used in the recipe file.) The next part of the name,
-PLUMED-2.7.4-CPU, is called the version suffix. Version suffixes are typically used to distinguish different builds of the same version of the package. In this case, it indicates that it is a build of the 2021.4 version purely for CPU and also includes PLUMED as we have also builds without PLUMED (which is not compatible with every GROMACS version).
EasyBuild is configured so that it searches in the user repository and two repositories on the system. The current directory is not part of the default search path but is easily added with a command line option. By default, EasyBuild will not install dependencies of a package and fail instead if one or more of the dependencies cannot be found, but that is also easily changed on the command line. If all needed EasyBuild recipes are in one of those repository or in the current directory, all you need to do to install the package is to run
-rtells EasyBuild to also install dependencies that may not yet be installed, and with the dot added to it, to also add the current directory to the front of the search path. The
-rflags should be omitted if you want full control and install dependency by dependency before installing the package (which may be very useful if building right away fails).
If you now type
module avail you should see the
Advanced guide to EasyBuild on LUMI¶
Toolchains on Cray¶
Toolchains in EasyBuild contains at least a compiler, but can also contains an MPI library and a number of mathematical libraries (BLAS, LAPACK, ScaLAPACK and a FFT library). Programs compiled with different toolchains cannot be loaded together (though the module system will not always prevent this on LUMI).
The toolchains on LUMI are different of what you may be used from non-Cray systems. On most systems, EasyBuild uses its own toolchains installed from within EasyBuild, but on LUMI we use toolchains that are based on the Cray Programming Environment. Three toolchains are currently implemented
cpeGNUis the equivalent of the Cray
cpeCrayis the equivalent of the Cray
cpeAOCCis the equivalent of the Cray
cpeAMDis the equivalent of the Cray
All four toolchains use
cray-mpich over the Open Fabric Interface library
craype-network-ofi) and Cray LibSci for the mathematical libraries, with the
releases taken from the Cray PE release that corresponds to the version number of the
cpeGNU/Cray/AOCC/AMD and PrgEnv-gnu/cray/aocc/amd
cpeAOCC modules don't load the corresponding
PrgEnv-* modules nor the
cpe/<version> modules. This is because in the current
setup of LUMI both modules have their problems and the result of loading those
modules is not always as intended.
If you want to compile software that uses modules from the LUMI stack, it is best
to use one of the
cpeAOCC modules to load the compiler
and libraries rather than the matching
PrgEnv-* modules as those may
not always load all modules in the correct version.
Since the LUMI software stack does not support the EasyBuild common toolchains (such
as the EasyBuild intel and foss toolchains) one cannot use the default EasyBuild build
recipes without modifying them. Hence they are not included in the robot search path
of EasyBuild so that you don't accidentally try to install them (and also removed
from the search path for
eb -S or
eb --search to avoid any confustion that they
Building your own EasyBuild repository¶
We advise users to maintain their own repository of EasyConfig files which they installed in their personal or project space. This may help to rebuild your environment for a later project on LUMI. It may even be a good idea to keep the repository on a personal GitHub or other version control service.
The repository is created automatically the first time
EasyBuild-user is loaded.
The directory is called
UserRepo and is in
the default location
$HOME/EasyBuild if you don't set the environment variable).
It must be structured similarly to
the main EasyBuild EasyConfig repository.
The EasyBuild recipes (
.eb files) should be in a subdirectory
leaving room for personal EasyBlocks also (which would then go in the
easybuild/easyblocks subdirectory) and even personal configuration files that overwrite
some system options. This setup also guarantees compatibility with some EasyBuild features
for very advanced users that go way beyond this page.
To store this repository on GitHub, you can follow the GitHub documentation, and in particular the page "Adding an existing project to GitHub using the command line.
Technical documentation on the toolchains on LUMI and the directory structure of EasyBuild can be found in the documentation of the LUMI-SoftwareStack GitHub repository.
If you want to get more familiar with EasyBuild and develop your own EasyBuild recipes, we suggest the following sources of information:
- EasyBuild manual on ReadtheDocs
- EasyBuild tutorials
- The EasyBuild YouTube channel contains recordings of a four-session tutorial given for the LUMI User Support Team by Kenneth Hoste (UGent), the lead developer of EasyBuild and Luca Marsella (CSCS)
- Technical documentation on our setup for developers
- LUMI EasyBuild recipes
- Main LUMI software stack GitHub repository
contains the full EasyBuild setup for LUMI, including the EasyBuild recipes
that we use for the central software stack and many others that we fully support
and consider of good quality. The clone on the system is automatically searched
- LUMI contributed EasyBuild recipes GitHub repository contains contributed EasyBuild recipes and other recipes developed by LUST that haven't been as thoroughly checked or are deemed not appropriate for central installation at this point. However, they are fully compatible with the setup on LUMI, with correct dependency versions etc.
- Main LUMI software stack GitHub repository contains the full EasyBuild setup for LUMI, including the EasyBuild recipes that we use for the central software stack and many others that we fully support and consider of good quality. The clone on the system is automatically searched by the
- Other EasyBuild recipes for the Cray Programming Environment
- CSCS GitHub repository. Most of the recipes are for Piz Daint which uses slightly different toolchains. Moreover dependencies typically need updating as the software installation on LUMI is not in sync with the CSCS installation. The repository is particularly useful for CPU-only programs as the GPUs in their system are not compatible with those in LUMI.
- EasyBuild recipes that are not compatible with the Cray Programming Environment but that may sometimes be a good source to start developing compatible ones (if you're an EasyBuild expert):