Configuration
Usage Model
JSC Cloud can be accessed via the JSC Cloud OpenStack Dashboard or via its REST APIs mentioned here in a programatic approach using http libraries/wrappers (e.g. curl or the python-openstackclient Python libraries).
The APIs can be used to spawn virtual machines (VMs), which are accessible via SSH and run on datacenter resources. Within those virtual machines, users can gain root access and do whatever they need to provide their scientific services.
Virtual Machine Types
A number of different virtual machine configurations, so-called flavors, are available.
Name
VCPUs
RAM
Disk
Ephemeral
Is Public
Hardware
SCS-1L:1:20n
1
1024
20
0
True
SCS-1L:2:20n
1
2048
20
0
True
SCS-1L:4:20n
1
4096
20
0
True
SCS-2L:2:20n
2
2048
20
0
True
SCS-2L:4:20n
2
4096
20
0
True
SCS-2L:8:20n
2
8192
20
0
True
SCS-4L:4:20n
4
4096
20
0
True
SCS-4L:8:20n
4
8192
20
0
True
SCS-4L:16:20n
4
16384
20
0
True
SCS-8L:8:20n
8
8192
20
0
True
SCS-8L:16:20n
8
16384
20
0
True
SCS-8L:24:20n
8
24576
20
0
True
SCS-16L:16:20n
16
16384
20
0
True
SCS-16L:32:20n
16
32768
20
0
True
SCS-16L:48:20n
16
49152
20
0
True
SCS-16L:64:20n
16
65536
20
0
True
SCS-16L:128:20n
16
131072
20
0
True
SCS-4L:16:20n-z2-nvme
4
16384
20
0
True
1 TB NVMe
SCS-16L:64:20n-z2-nvme
16
65536
20
0
True
1 TB NVMe
SCS-4L:16:20n-z3-nvme
4
16384
20
0
True
1 TB NVMe
SCS-16L:64:20n-z3-nvme
16
65536
20
0
True
1 TB NVMe
SCS-16L:64:20n-z2-GNv:80
16
65536
20
0
False
NVIDIA V100 16GB
SCS-16L:64:20n-z3-GNa:108
16
65536
20
0
False
NVIDIA A100 80GB
Storage Layers
Virtual machines spawned within the JSC Cloud can have access to four different kind of storages.
Type
Livetime
Access
Quantity
Description
Root File system
Ephemeral
blockdevice
10s of GB
Auto-created at VM spawn
NVMe
Ephemeral
PCI-Passthrough
894 GB
Manual mount/cleanup! OpenStack Flavor
Dedicated Volume
Persistent
blockdevice
100s of GB - 10s of TB
Create/delete via OpenStack Cinder
DATA
Persistent
NFS mount
100s of GB - 10s of TB
Manual procedure for access.
The blockdevice for the root filesystem is placed on JUSTCOM, an external storage system which is part of JUST.
Access to specific parts of the DATA filesystem from our JUST storage system is a manual process and is only granted based on case-by-case decisions.
Hardware Overview
Hardware Configuration
- 8 Cloud compute nodes
2× AMD EPYC 7513, 2× 32 cores, 2.6 GHz
512 (16× 32) GB DDR4, 3200 MHz
1x Connect-X6 HDR100 (configured in Ethernet mode)
local disk for operating system
1 TB NVMe
- 4 accelerated Cloud nodes
2× AMD EPYC 7513, 2× 32 cores, 2.6 GHz
1024 (32× 32) GB DDR4, 3200 MHz
1x Connect-X6 HDR100 (configured in Ethernet mode)
local disk for operating system
3× NVIDIA A100 GPU with 80 GB HBM2e
- 8 Cloud compute nodes
2× AMD EPYC 7742, 2× 64 cores, 2.25 GHz
256 (16× 16) GB DDR4, 3200 MHz
1x Connect-X6 HDR100 (configured in Ethernet mode)
local disk for operating system
1 TB NVMe
- 16 accelerated Cloud nodes
2× AMD EPYC 7742, 2× 64 cores, 2.25 GHz
256 (16× 16) GB DDR4, 3200 MHz
InfiniBand HDR100 (Connect-X6)
local disk for operating system
1 TB NVMe
1× NVIDIA V100 GPU with 16 GB HBM2e
- 3 Cloud manager nodes
2× AMD EPYC 7313, 2× 16 cores, 3.0 GHz
256 (16× 16) GB DDR4, 3200 MHz
2x Connect-X6 HDR100 (configured in Ethernet mode)
7,680 CPU threads, 14 TB memory
Note
The Cloud nodes do NOT have Infiniband. The NVMe and GPUs in the Cloud are available via PCIe passthrough. Note that the GPU VM flavors are private and need to be requested first.
Software Overview
The software is based on kolla-ansible. For further information see the OpenStack documentation.