Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table 1. Setonix Node Overview

Type

N. NodesNozdes

CPU

Cores Per Node

RAM Per Node

GPUs Per Node

Login

9

AMD Milan2x 64256GBn/a

CPU computing

1592

AMD Milan (2.45GHz, 280W)

2x 64

256GBn/a
CPU high memory8AMD Milan (2.45GHz, 280W)2x 641TBn/a
GPU computing154AMD Trento1 x 64256GB8 GCDs (from 4x "AMD MI250X" cards, each card with 2 GCDs)
GPU high memory38AMD Trento1 x 64512GB8 GCDs (from 4x "AMD MI250X" cards, each card with 2 GCDs)
Data movement11

AMD 7502P

1x 32128Gbn/a

...

For Setonix GPU nodes, the charge rate is 512 SU per node hour, based on the difference in energy consumption between the CPU and GPU node architectures. Since there are fewer GPU nodes than CPU nodes, these GPU nodes are to be used solely for GPU-enabled codes. Thus, resource requests on GPU nodes are slightly different to CPU nodes as all requests are in units of GCDs, with 1 GCD = 1 Slurm GPU. Requests cannot be made based on memory but must be based on the number of GPUs to be used. 

Worked Examples of the Accounting Model

Let’s work through some CPU examples.

Example 1: Memory heavy

You request the following: 

ResourceRequest
Cores32
Memory128GB
Nodes1
Hours4

On a CPU node, there are 128 cores per node. Core Proportion is calculated as requested cores/ total cores, which is 32/128=0.25
On a normal CPU node (i.e. not the highmem nodes), there is 256GB of memory. Memory Proportion is calculated as requested memory/ total memory, which is 128/256=0.5

Which of these is bigger? It’s the memory proportion in this case, so that’s what we will use for the equation. You always use whichever is the biggest proportion. Also remember that the Partition Charge Rate never changes. 

128 x 0.5 x 1 x 4 = 256SU

Partition charge rate x memory proportion x nodes x hours = 256SU

  

Example 2: CPU heavy

You request the following: 

ResourceRequest
Cores128
Memory64GB
Nodes1
Hours4

On a CPU node, there are 128 cores per node. Core Proportion is calculated as requested cores/ total cores, which is 128/128=1
On a normal CPU node (i.e. not the highmem nodes), there is 256GB of memory. Memory Proportion is calculated as requested memory/ total memory, which is 64/256=0.25

Which of these is bigger? It’s the memory proportion in this case, so that’s what we will use for the equation. You always use whichever is the biggest proportion. Also remember that the Partition Charge Rate never changes. 

128 x 1  x 1 x 4 = 512SU

Partition charge rate x  CPU proportion x nodes x hours = 512SU

Example 3: GPU 

ResourceRequest
GCDs3
MemoryN/A
Nodes1
Hours4

On a GPU node, there are 8 GCDs per node. The GPU proportion is calculated as requested GCDs/total GCDs which is 3/8=0.375

You don't need to worry about how much memory to request as this is automatically allocated for you. 


512 x 0.375 x 1 x 4 = 768SU

Partition charge rate x memory proportion x nodes x hours = 768SU

Maintenance

Due to the cutting-edge nature of Setonix, regular and frequent updates of the software stack is expected during the first year of Setonix's operation as further optimisations and improvements are made available.

...