.. include:: system.rst

.. _building_software:

Building Software
=================

This section describes how to build and use your own software on |SYSTEM_NAME|.

Compiled Languages
------------------

Compiled languages such as C, C++, and Fortran have to be translated from human readable source code into machine code by a compiler ahead of time.
In addition to a compiler, this often involves a build system.

Compilers
^^^^^^^^^

The software modules on |SYSTEM_NAME| provide toolchains for C, C++, and Fortran based on several compilers, paired with different MPI libraries, see :ref:`software_modules_available_toolchains`.
Once a module containing a compiler has been loaded, the compiler programs can be used in the usual way, e.g. to compile a single file C program with the GNU compiler:

.. code-block:: none

   $ gcc program.c -o program

For more detailed instructions, please see the manual pages included with the compilers, e.g.:

.. code-block:: none

   $ man icc

to learn more about the Intel C compiler.
The module descriptions should include links to further information as well, e.g.:

.. code-block:: none

   $ module help NVHPC
   [...]
   More information
   ================
    - Homepage: https://developer.nvidia.com/hpc-sdk/

MPI Compiler Wrappers
^^^^^^^^^^^^^^^^^^^^^

MPI libraries often ship compiler wrappers which pass appropriate command line arguments to an underlying compiler.
On |SYSTEM_NAME| these become available after loading a compiler module such as `GCC` and an MPI module such as `ParaStationMPI`.
The compiler wrappers accept the same command line arguments as the underlying compiler, e.g.:

.. code-block:: none

   $ mpicc mpi_program.c -o mpi_program

CUDA
^^^^

CUDA programs can be compiled with the ``nvcc`` compiler included in the ``CUDA`` module:

.. code-block:: none

   $ nvcc cuda_program.cu -o cuda_program

Build Systems
^^^^^^^^^^^^^

Except for very simple cases, compilers are not used directly.
Larger software projects use build systems to invoke the correct compilers with appropriate command line arguments in a suitable order.
Popular choices for build systems are GNU Make, GNU Autotools, CMake, Bazel, etc.
Many of these build systems are available as modules, for a basic build of a project using CMake:

.. code-block:: none

   $ module load CMake
   $ cd <project source directory>
   $ mkdir build
   $ cd build
   $ cmake ..
   $ make

.. warning::
   Please be aware of the number of parallel build processes that build systems spawn, as they are 
   a common way to cause high-load issues on nodes and affect other users. Consider limiting them to a 
   small number, e.g. with the usage of ``make -j 12`` or ``ninja -j 12``. 

   The command ``make -j`` alone is not sufficient, as this does not limit the number of ``make`` processes
   that run simultaneously.

Exotic Languages
^^^^^^^^^^^^^^^^

Compilers and build tools for programming languages more exotic than C, C++, and Fortran are available.
These include Rust, Go, and Intel's ISPC compiler.
``module spider`` can be used to find out more.

Interpreted Languages
---------------------

Interpreted languages are typically executed from the source code or some intermediate code that is not machine code by passing the code to an interpreter program.

Python
^^^^^^

The Python interpreter is available through the ``Python`` software module.
Several popular extensions are available as further modules, such as ``SciPy-Stack``, ``mpi4py``, ``numba``, etc.

Julia
^^^^^

The Julia interpreter is available through the ``Julia`` software module.
Additional packages, such as ``CUDA.jl`` and ``MPI.jl``, can be added using Julia’s built-in package manager, ``Pkg``.

Pre-Compiled Binaries
---------------------

Many package managers exist which deliver pre-compiled binaries. When using these managers, each user needs to ensure that they are allowed to use the software on |SYSTEM_NAME|. Also, it is not a given that this pre-compiled software has been built specifically for the HPC environment on |SYSTEM_NAME|; rather, generic compilation flags may have been chosen. For these reason, we are careful in supporting these third-party package managers.

Conda/Mamba
^^^^^^^^^^^

A famous and frequently-used third-party package manager is `Conda <https://anaconda.org/anaconda/conda>`_. While strictly open source software, `default usage of Conda may not be free <https://www.anaconda.com/pricing/terms-of-service-faqs>`_: When using the default packages channel hosted by Anaconda, a license may be required. For example, usage at an organization with more than 200 employees already requires at least a Business license.

For |SYSTEM_NAME|, usage of the default Conda channel is disabled by blockage of the respective server name / IP.

We recommend using free, publicly available community channels like `conda-forge <https://conda-forge.org/>`_, possibly with the alternative open source client `*Mamba* <https://github.com/mamba-org/mamba>`_ (together set up easiest with `miniforge <https://conda-forge.org/miniforge/>`_). See also `the write-up by FZJ's JuRSE community <https://www.fz-juelich.de/en/rse/the_latest/the-anaconda-is-squeezing-us>`_.