.. include:: system.rst .. _building_software: Building Software ================= This section describes how to build and use your own software on |SYSTEM_NAME|. Compiled Languages ------------------ Compiled languages such as C, C++, and Fortran have to be translated from human readable source code into machine code by a compiler ahead of time. In addition to a compiler, this often involves a build system. Compilers ^^^^^^^^^ The software modules on |SYSTEM_NAME| provide toolchains for C, C++, and Fortran based on several compilers, paired with different MPI libraries, see :ref:`software_modules_available_toolchains`. Once a module containing a compiler has been loaded, the compiler programs can be used in the usual way, e.g. to compile a single file C program with the GNU compiler: .. code-block:: none $ gcc program.c -o program For more detailed instructions, please see the manual pages included with the compilers, e.g.: .. code-block:: none $ man icc to learn more about the Intel C compiler. The module descriptions should include links to further information as well, e.g.: .. code-block:: none $ module help NVHPC [...] More information ================ - Homepage: https://developer.nvidia.com/hpc-sdk/ MPI Compiler Wrappers ^^^^^^^^^^^^^^^^^^^^^ MPI libraries often ship compiler wrappers which pass appropriate command line arguments to an underlying compiler. On |SYSTEM_NAME| these become available after loading a compiler module such as `GCC` and an MPI module such as `ParaStationMPI`. The compiler wrappers accept the same command line arguments as the underlying compiler, e.g.: .. code-block:: none $ mpicc mpi_program.c -o mpi_program CUDA ^^^^ CUDA programs can be compiled with the ``nvcc`` compiler included in the ``CUDA`` module: .. code-block:: none $ nvcc cuda_program.cu -o cuda_program Build Systems ^^^^^^^^^^^^^ Except for very simple cases, compilers are not used directly. Larger software projects use build systems to invoke the correct compilers with appropriate command line arguments in a suitable order. Popular choices for build systems are GNU Make, GNU Autotools, CMake, Bazel, etc. Many of these build systems are available as modules, for a basic build of a project using CMake: .. code-block:: none $ module load CMake $ cd $ mkdir build $ cd build $ cmake .. $ make .. warning:: Please be aware of the number of parallel build processes that build systems spawn, as they are a common way to cause high-load issues on nodes and affect other users. Consider limiting them to a small number, e.g. with the usage of ``make -j 12`` or ``ninja -j 12``. The command ``make -j`` alone is not sufficient, as this does not limit the number of ``make`` processes that run simultaneously. Exotic Languages ^^^^^^^^^^^^^^^^ Compilers and build tools for programming languages more exotic than C, C++, and Fortran are available. These include Rust, Go, and Intel's ISPC compiler. ``module spider`` can be used to find out more. Interpreted Languages --------------------- Interpreted languages are typically executed from the source code or some intermediate code that is not machine code by passing the code to an interpreter program. Python ^^^^^^ The Python interpreter is available through the ``Python`` software module. Several popular extensions are available as further modules, such as ``SciPy-Stack``, ``mpi4py``, ``numba``, etc. Julia ^^^^^ The Julia interpreter is available through the ``Julia`` software module. Additional packages, such as ``CUDA.jl`` and ``MPI.jl``, can be added using Julia’s built-in package manager, ``Pkg``. Pre-Compiled Binaries --------------------- Many package managers exist which deliver pre-compiled binaries. When using these managers, each user needs to ensure that they are allowed to use the software on |SYSTEM_NAME|. Also, it is not a given that this pre-compiled software has been built specifically for the HPC environment on |SYSTEM_NAME|; rather, generic compilation flags may have been chosen. For these reason, we are careful in supporting these third-party package managers. Conda/Mamba ^^^^^^^^^^^ A famous and frequently-used third-party package manager is `Conda `_. While strictly open source software, `default usage of Conda may not be free `_: When using the default packages channel hosted by Anaconda, a license may be required. For example, usage at an organization with more than 200 employees already requires at least a Business license. For |SYSTEM_NAME|, usage of the default Conda channel is disabled by blockage of the respective server name / IP. We recommend using free, publicly available community channels like `conda-forge `_, possibly with the alternative open source client `*Mamba* `_ (together set up easiest with `miniforge `_). See also `the write-up by FZJ's JuRSE community `_.