The Juelich Storage cluster (JUST) renewal

In Q2/2024 the current Juelich Storage Cluster JUST will be replaced by it’s 6th generation. We will perform the migration of all user data with minimal impact on the production. We use this opportunity to consolidate some relicts and to change the overall system design in the background, to enhance the overall resilience.

Besides this migration we also have to deal with the eXtended Capacity Storage Tier (XCST) system, which is currently storing the $DATA file systems. This storage system will be going out of service in two phases (2024 and 2026), which requires data migration to JUST6.

Maintenance JUNE 10-12, 2024

We currently target for a 3 day maintenance starting on Monday 10th of June 2024, which might be moved in case of delays.

In this maintenance we will replace all connections from the HPC systems to the old storage cluster by new ones to JUST6. Furthermore the user environment will be adjusted to map our design changes and prepare the merge of /p/largedata, /p/largedata1 and /p/fastdata to one $DATA. The /p/largedata will be migrated to /p/data1. In addition each user will get a new primary group which is unique. The old primary group jusers will be a added as a secondary group to all users.

Most time will be spend in finalizing the data migration, which is currently running completely in the background.

**User Environment Changes**
Storage Layer/Variable	Change
`$SCRATCH`	Every compute project will get a new empty directory. The old data can still be accessed by using the temporary variable `$OLDSCRATCH` which is pointing to the old file system. The `$OLDSCRATCH` is set to read only and will be available for 90 days. Be aware of the automatic deletion process on both scratch file systems.
`$FASTDATA`	new variable name: `$DATA`, the path will be renamed: `/p/fastdata/...` to `/p/data1/...` `/p/largedata/...` will be moved to `/p/data1/...`
`$DATA`	new variable name: `$LARGEDATA` (pointing to `/p/largedata2`)
`$PROJECT`	path rename: `/p/project/...` to `/p/project1/...`
`$ARCHIVE`	path rename: `/arch[2]/...` to `/p/arch[12]/...`

Backup/Archive Maintenance in JUNE/JULY 2024

$ARCHIVE unavailable for three weeks (17.06. - 5.07.2024):: Due to constrains in the design changes a special converting procedure must be performed. While this task is ongoing no user access to $ARCHIVE is allowed.
Archive/Backup infrastructure migration 1.-5.07.2024:: Backup and restore of files will be temporary not possible. The file system snapshots have to cover this gap.

Maintenance in Q4/2026

/p/largedata1 migration to $DATA:: Affected projects will loose $LARGEDATA and get $DATA for the new location of their data.