JEDI user documentation

This documentation explains various aspects of using the JEDI system at Jülich Supercomputing Centre. It contains both introductory and advanced material.

First time users might want to start at the beginning and work through the first few chapters, skipping those sub-chapters that contain advanced material. This way, you will learn how to log in to JEDI, familiarize yourself with the environment, find pre-installed software, compile your own programs, and run them on JEDI.

Advanced users can peruse the table of contents below to find out about particular topics they are interested in.

Warning

JEDI is a preparatory system for JUPITER and its configuration is therefore under active development and enhancement. There are multiple differences compared to other HPC systems at JSC.

• The amount of EasyBuild modules will evolve over time and is currently focussing on core packages.

• The usage agreement deviates from the standard JSC agreement, to account for the pre-production nature of JEDI. Please acknowledge it and pay special attention to the guidelines for publications.

Contents:

• Access
  • Prerequisites
  • SSH Login
  • MFA with TOTP
  • Login Nodes
  • Alternative Login Methods
• Configuration
  • Hardware Configuration
  • Software Overview
• Environment
  • Shell
  • Active project
  • Available file systems
  • Machine identification file
  • Transferring files with scp, rsync, etc.
  • Using Git on JEDI
• Building Software
  • Compiled Languages
  • Interpreted Languages
  • Pre-Compiled Binaries
• Batch system
  • Slurm Partitions
  • Internet Access
  • Allocations, Jobs and Job Steps
  • Writing a Batch Script
  • Generic Resources, Features and Topology-aware Allocations
  • Job Steps
  • Dependency Chains
  • Interactive Sessions
  • Hold and Release Batch Jobs
  • Slurm commands
  • Summary of sbatch and srun Options
• Processor Affinity
  • Slurm options
  • Affinity visualization tool
  • Affinity examples
• MPMD: Multiple Program Multiple Data Execution Model
• GPU Computing
  • JEDI GPU Nodes
  • GPU Visibility/Affinity
  • Setting GPU Clock Rates
  • NVIDIA Profiling Tools and Clock Speed
  • CUDA MPS: Multiple MPI Ranks per GPU
  • Job Script Examples
• GPFS File Systems in the Jülich Environment
  • Best practice notes
• HPC Data Rules for GPFS
  • Structure
  • File system and group/project-directories
  • User $HOME Directory
  • User data
• Data Transfer to and from JEDI
• Known Issues on JEDI
  • Open Issues
  • Recently Resolved and Closed Issues
• FAQ
  • General FAQ
  • FAQ about Data Management
• Maintenance
• Accounting
  • Accounting Mode
  • Command description
• JSC Tools
  • JuDoor
  • Status Page
  • LLview Job Reporting
  • Pinning Tool
  • JUBE Benchmarking Environment
  • User tool jutil
  • UNICORE access
  • BLABLADOR documentation
• Container Runtime on JEDI
  • What Containers Provide
  • Getting Access
  • Apptainer on JEDI
  • Container Build System
  • Apptainer image building
• Parallel Debugging and Performance Analysis
  • Debugging with TotalView on JEDI
  • Using TotalView
  • Memory Debugging with TotalView
• Support
• Jacamar CI Runners
  • Getting Access
  • Runners Tags on JEDI
  • Jacamar Example .gitlab-ci.yml File