Building Software

This section describes how to build and use your own software on JUWELS.

Compiled Languages

Compiled languages such as C, C++, and Fortran have to be translated from human readable source code into machine code by a compiler ahead of time. In addition to a compiler, this often involves a build system.

Compilers

The software modules on JUWELS provide toolchains for C, C++, and Fortran based on several compilers, paired with different MPI libraries, see Available compilers. Once a module containing a compiler has been loaded, the compiler programs can be used in the usual way, e.g. to compile a single file C program with the GNU compiler:

$ gcc program.c -o program

For more detailed instructions, please see the manual pages included with the compilers, e.g.:

$ man icc

to learn more about the Intel C compiler. The module descriptions should include links to further information as well, e.g.:

$ module help NVHPC
[...]
More information
================
 - Homepage: https://developer.nvidia.com/hpc-sdk/

MPI Compiler Wrappers

MPI libraries often ship compiler wrappers which pass appropriate command line arguments to an underlying compiler. On JUWELS these become available after loading a compiler module such as GCC and an MPI module such as ParaStationMPI. The compiler wrappers accept the same command line arguments as the underlying compiler, e.g.:

$ mpicc mpi_program.c -o mpi_program

CUDA

CUDA programs can be compiled with the nvcc compiler included in the CUDA module:

$ nvcc cuda_program.cu -o cuda_program

Build Systems

Except for very simple cases, compilers are not used directly. Larger software projects use build systems to invoke the correct compilers with appropriate command line arguments in a suitable order. Popular choices for build systems are GNU Make, GNU Autotools, CMake, Bazel, etc. Many of these build systems are available as modules, for a basic build of a project using CMake:

$ module load CMake
$ cd <project source directory>
$ mkdir build
$ cd build
$ cmake ..
$ make

Exotic Languages

Compilers and build tools for programming languages more exotic than C, C++, and Fortran are available. These include Rust, Go, and Intel’s ISPC compiler. module spider can be used to find out more.

Interpreted Languages

Interpreted languages are typically executed from the source code or some intermediate code that is not machine code by passing the code to an interpreter program.

Python

The Python interpreter is available through the Python software module. Several popular extensions are available as further modules, such as SciPy-Stack, mpi4py, numba, etc.

Julia

The Julia interpreter is available through the Julia software module. Some additional packages are available as further modules, such as Julia.MPI and Julia.CUDA.

Pre-Compiled Binaries

Many package managers exist which deliver pre-compiled binaries. When using these managers, each user needs to ensure that they are allowed to use the software on JUWELS. Also, it is not a given that this pre-compiled software has been built specifically for the HPC environment on JUWELS; rather, generic compilation flags may have been chosen. For these reason, we are careful in supporting these third-party package managers.

Conda/Mamba

A famous and frequently-used third-party package manager is Conda. While strictly open source software, default usage of Conda may not be free: When using the default packages channel hosted by Anaconda, a license may be required. For example, usage at an organization with more than 200 employees already requires at least a Business license.

For JUWELS, usage of the default Conda channel is disabled by blockage of the respective server name / IP.

We recommend using free, publicly available community channels like conda-forge, possibly with the alternative open source client *Mamba* (together set up easiest with miniforge). See also the write-up by FZJ’s JuRSE community.