System Changelog

Current state

Installed software

Software

Version

Description

Rocky Linux

9.6

Kernel Version

5.14.0-570.42.2.el9_6

NVIDIA GPU Driver

580.95.05

OFED

25.04-OFED.25.04.0.6.0.1

Slurm

24.11.5-1.20250602git2ed9014

ParaStation Management

6.4.1

GPFS

5.2.3-2

Apptainer

1.3.6-1

PMIx

5.0.8

Default Software Stage

2025

Changelog entries

2025-10-16 Software update

Update type: OS Packages

OS Packages

  • NVIDIA GPU Driver has been updated to 580.95.05 (from 580.65.06)

2025-09-22 Software update

Update type: OS Packages

OS Packages

  • Kernel Version has been updated to 5.14.0-570.42.2.el9_6 (from 5.14.0-570.32.1.el9_6)

2025-09-22 Update UCX

Update type: SW Modules

  • UCX has been changed to 1.18.1 from 1.17.0

2025-09-09 Software update

Update type: SW Modules

UCX-settings

  • UCX_CUDA_COPY_DMABUF=no has been removed for the UCX-settings/[RC,UD,DC]-CUDA modules, since it is no longer necessary to prevent crashes, and it actually causes a performance regression with the latest OFED and NVIDIA driver

2025-08-28 Software update

Update type: OS Packages

OS Packages

  • Rocky Linux has been updated to 9.6 (from 9.5)

  • Kernel Version has been updated to 5.14.0-570.32.1.el9_6 (from 5.14.0-503.40.1.el9_5)

  • NVIDIA GPU Driver has been updated to 580.65.06 (from 570.133.20)

  • ParaStation Management has been updated to 6.4.1 (from 6.3.0)

  • GPFS has been updated to 5.2.3-2 (from 5.2.2-1)

2025-07-17 Software update

Update type: OS Packages

OS Packages

  • ParaStation Management has been updated to 6.3.0 (from 6.2.4)

2025-07-01 Software update

Update type: OS Packages

OS Packages

  • ParaStation Management has been updated to 6.2.4 (from 6.2.3)

2025-06-03 Software update

Update type: OS Packages

OS Packages

  • Kernel Version has been updated to 5.14.0-503.40.1.el9_5 (from 5.14.0-503.38.1.el9_5)

  • OFED has been updated to 25.04-OFED.25.04.0.6.0.1 (from 25.01-OFED.25.01.0.6.0.1)

  • Slurm has been updated to 24.11.5-1.20250602git2ed9014 (from 23.11.10-1.20240920git20c5755)

  • ParaStation Management has been updated to 6.2.3 (from 6.1.1)

  • PMIx has been updated to 5.0.8 (from 5.0.6)

2025-05-05 Software update

Update type: OS Packages

OS Packages

  • Compute nodes updated (from partition: batch and scraper)

    • Kernel Version has been updated to 5.14.0-503.38.1.el9_5 (from 5.14.0-503.23.1.el9_5)

2025-04-29 Software update

Update type: OS Packages

OS Packages

  • Systems with NVIDIA GPUs have been updated (compute nodes in partitions gpus and develgpus):

    • Kernel Version has been updated to 5.14.0-503.38.1.el9_5 (from 5.14.0-503.23.1.el9_5)

    • NVIDIA GPU Driver has been updated to 570.133.20 (from 570.86.15)

    • gpumond is removed

2025-03-06 Software update

Update type: OS Packages

OS Packages

  • Rocky Linux has been updated to 9.5 (from 9.4)

  • Kernel Version has been updated to 5.14.0-503.23.1.el9_5 (from 5.14.0-427.33.1.el9_4)

  • NVIDIA GPU Driver has been updated to 570.86.15 (from 560.35.03)

  • OFED has been updated to 25.01-OFED.25.01.0.6.0.1 (from 24.07-OFED.24.07.0.6.1.1)

  • GPFS has been updated to 5.2.2-1 (from 5.1.9-4)

2025-02-27 MemoryMax

Update type: Login nodes

  • MemoryMax has been set to 25% on individual user slices on login nodes

2025-02-05 Change MPI-settings for OpenMPI

Update type: SW Modules

  • As of 2025 romio321 is not working, so we have disabled the selection of romio321 in the MPI-settings, giving OpenMPI the freedom to choose and prioritize, currently ompio is selected.

2025-01-29 Default UCX-settings module

Update type: SW Modules

  • RC has been made the default module for UCX-settings in the 2025 stage. Until now it was UD by mistake.

2024-12-11 Software update

Update type: OS Packages

OS Packages

  • Slurm has been updated to 23.11.10-1.20240920git20c5755 (from 23.02.7-1.20240328git405c820)

  • Apptainer has been updated to 1.3.6-1 (from 1.3.2-1)

2024-09-18 Software update

Update type: OS Packages

OS Packages

  • Rocky Linux has been updated to 9.4 (from 8.10)

  • Kernel Version has been updated to 5.14.0-427.33.1.el9_4 (from 4.18.0-553.el8_10)

  • NVIDIA GPU Driver has been updated to 560.35.03 (from 550.54.15)

  • OFED has been updated to 24.07-OFED.24.07.0.6.1.1 (from 24.04-OFED.24.04.0.6.6.1)

  • GPFS has been updated to 5.1.9-4 (from 5.1.9-3)

2024-07-15 Software update

Update type: OS Packages

OS Packages

  • Slurm has been updated to 23.02.7-1.20240328git405c820 (from 22.05.11-1.20231215gitc756517)

2024-06-18 Software update

Update type: OS Packages

OS Packages

  • Rocky Linux has been updated to 8.10 (from 8.9)

  • Kernel Version has been updated to 4.18.0-553.el8_10 (from 4.18.0-513.18.1.el8_9)

  • NVIDIA GPU Driver has been updated to 550.54.15 (from 535.154.05)

  • OFED has been updated to 24.04-OFED.24.04.0.6.6.1 (from 23.10-OFED.23.10.1.1.9.1)

  • GPFS has been updated to 5.1.9-3 (from 5.1.9-1)

  • Apptainer has been updated to 1.3.2-1 (from 1.2.3-1)

2024-03-11 Software update

Update type: OS Packages

OS Packages:

  • Kernel 4.18.0-513.18.1.el8_9 (from 4.18.0-513.11.1.el8_9)

2023-12-14 Software update

Update type: OS Packages, Batch system, SW Modules

OS Packages:

  • SLURM has been updated to 22.05.10-2.20231203gitae058ea to address newly-discovered security issues

  • psmgmt has been updated to 5.1.58-1

Software stack

  • netCDF in the 2024 stage has been rebuilt to add support for extra compression libraries

  • GCC in the 2024 stage has been recompiled to patch some bugs that appeared in combination with PyTorch

2023-10-27 PMIx update

Update type: SW Modules

Configuration:

  • All OpenMPI installations have been rebuilt to include a patch necessary for the new PMIx

2023-10-19 Software update

Update type: OS Packages, Batch system

Packages:

  • Kernel 4.18.0-477.27.1.el8_8.x86_64

  • NVIDIA OFED 23.07-0.5.1.1

  • NVIDIA GPU drivers 535.104.12

  • GPFS 5.1.8-2

  • Apptainer 1.2.3-1

  • DDN IME 1.5.2-152128

  • psmgmt-5.1.57-1

  • IB Switch firmware 27.2012.1010

Configuration:

  • ssh now rejects RSA keys

  • All OpenMPI installations rely now on a user-space provided PMIx

  • Cgroups v2 were enabled

2023-08-30 UCX-settings update

Update type: SW Modules

The UCX-settings/*CUDA modules also set UCX_RNDV_FRAG_MEM_TYPE=cuda. This enables the GPU to initiate transfers of CUDA managed buffers. This can have a large speed-up in case Unified Memory (cudaMallocManaged()) is used, as staging of data is avoided.

2023-08-03 # General maintenance/update

Update type: OS Packages, General configuration, Storage, Network, Other

Compute and login update

The nodes are updated to:

  • Rocky 8.8 (from 8.7)

  • MOFED 23.04-OFED.23.04.1.1.3.0 (from 5.8-OFED.5.8.2.0.3.0)

  • GPFS 5.1.8-1 (from 5.1.7-1.5)

  • NVIDIA driver 535.54.03 (from 525.105.17)

  • psmgmt 5.1.56-2 (from 5.1.56-1)

2022-09-06 System software update

JUSUF was updated to Rocky Linux 8.6 and drivers were updated to:

  • OFED 5.7-1.0.2.0

  • NVIDIA GPU 515.65.01

  • GPFS 5.1.4-1

2022-03-08 Change in user installations

Update type: Announcement, SW Modules

Change in user installations

  • The module structure has been changed so $MODULEPATH is not expanded depending on the existence of the $PROJECT variable. Now the variable used is $USERINSTALLATIONS, so the project software is not automatically activated when using jutil.

2022-02-07 Enable user slices

Per-slice-limits enabled for users. This means, even with multiple sessions a user is limited to the given amount of cpu time slots.

The value was set to 2000%. The value is given in % of a single core, i.e. 2000% means 20 full cores all of the time or 40 core half of the time, …

2022-02-01 Default Software Stage Update

During the maintenance on 2022-02-01 the software stage was updated from 2020 to 2022. If you wish to keep using the old stage please make sure you load Stages/2020 in your jobs.

2022-02-01 Rocky update

Rocky update was finalized on 2022-02-01, the final two nodes (jsfm01,jsfm02) along with the hosted containers were upgraded to Rocky 8.5

2022-01-18 Rocky update

During the maintenance on 2021-01-18 majority of the nodes (computes, logins, admins) were upgraded to Rocky 8.5

2022-01-18 IB switch FW update

Switched had their FW updated to 27.2010.1310

2022-01-18 HCA FW update

HCAs on all nodes had their FW update to 20.32.1010

2022-01-18 PSMGMT update

PSMGMT was updated to 5.1.45-4