Configuration

Usage Model

JSC Cloud can be accessed via the JSC Cloud OpenStack Dashboard or via its REST APIs mentioned here in a programatic approach using http libraries/wrappers (e.g. curl or the python-openstackclient Python libraries).

The APIs can be used to spawn virtual machines (VMs), which are accessible via SSH and run on datacenter resources. Within those virtual machines, users can gain root access and do whatever they need to provide their scientific services.

Virtual Machine Types

A number of different virtual machine configurations, so-called flavors, are available.

Name

VCPUs

RAM

Disk

Ephemeral

Is Public

Hardware

SCS-1L:1:20n

1

1024

20

0

True

SCS-1L:2:20n

1

2048

20

0

True

SCS-1L:4:20n

1

4096

20

0

True

SCS-2L:2:20n

2

2048

20

0

True

SCS-2L:4:20n

2

4096

20

0

True

SCS-2L:8:20n

2

8192

20

0

True

SCS-2L:16:20n

4

16384

20

0

True

SCS-4L:4:20n

4

4096

20

0

True

SCS-4L:8:20n

4

8192

20

0

True

SCS-4L:16:20n

4

16384

20

0

True

SCS-4L:32:20n

4

32768

20

0

True

SCS-16L:16:20n

16

16384

20

0

True

SCS-16L:32:20n

16

32768

20

0

True

SCS-16L:64:20n

16

65536

20

0

True

SCS-16L:128:20n

16

131072

20

0

True

SCS-4L:16:20n-z3-nvme

4

16384

20

0

True

NVMe

SCS-16L:64:20n-z3-nvme

16

65536

20

0

True

NVMe

Storage Layers

Virtual machines spawned within the JSC Cloud can have access to four different kind of storages.

Type

Livetime

Access

Quantity

Description

Root File system

Ephemeral

blockdevice

10s of GB

Auto-created at VM spawn

NVMe

Ephemeral

PCI-Passthrough

894 GB

Manual mount/cleanup! OpenStack Flavor

Dedicated Volume

Persistent

blockdevice

100s of GB - 10s of TB

Create/delete via OpenStack Cinder

DATA

Persistent

NFS mount

100s of GB - 10s of TB

Manual procedure for access.

The blockdevice for the root filesystem is placed on JUSTCOM, an external storage system which is part of JUST.

Access to specific parts of the DATA filesystem from our JUST storage system is a manual process and is only granted based on case-by-case decisions.

Hardware Overview

Hardware Configuration

  • 8 Cloud compute nodes

    • 2× AMD EPYC 7513, 2× 32 cores, 2.6 GHz

    • 512 (16× 32) GB DDR4, 3200 MHz

    • 1x Connect-X6 HDR100 (configured in Ethernet mode)

    • local disk for operating system

    • 1 TB NVMe

  • 4 accelerated Cloud nodes

    • 2× AMD EPYC 7513, 2× 32 cores, 2.6 GHz

    • 1024 (32× 32) GB DDR4, 3200 MHz

    • 1x Connect-X6 HDR100 (configured in Ethernet mode)

    • local disk for operating system

    • 3× NVIDIA A100 GPU with 80 GB HBM2e

  • 8 Cloud compute nodes

    • 2× AMD EPYC 7742, 2× 64 cores, 2.25 GHz

    • 256 (16× 16) GB DDR4, 3200 MHz

    • 1x Connect-X6 HDR100 (configured in Ethernet mode)

    • local disk for operating system

    • 1 TB NVMe

  • 16 accelerated Cloud nodes

    • 2× AMD EPYC 7742, 2× 64 cores, 2.25 GHz

    • 256 (16× 16) GB DDR4, 3200 MHz

    • InfiniBand HDR100 (Connect-X6)

    • local disk for operating system

    • 1 TB NVMe

    • 1× NVIDIA V100 GPU with 16 GB HBM2e

  • 3 Cloud manager nodes

    • 2× AMD EPYC 7313, 2× 16 cores, 3.0 GHz

    • 256 (16× 16) GB DDR4, 3200 MHz

    • 2x Connect-X6 HDR100 (configured in Ethernet mode)

  • 7,680 CPU threads, 14 TB memory

Note

The Cloud nodes do NOT have Infiniband. The GPUs in the Cloud are partitioned in different vGPU types. You can select the different vGPU types by the flavors.

Software Overview

The software is based on kolla-ansible. For further information see the OpenStack documentation.