UAntwerp Tier-2 Clusters#

Login infrastructure#

You can log in to the CalcUA infrastructure using SSH via login.hpc.uantwerpen.be.

Alternatively, you can also log in directly to the login nodes of the individual clusters using one of the following hostnames.

Cluster	Generic login name	Individual login node
Vaughan	login-vaughan.hpc.uantwerpen.be	login1-vaughan.hpc.uantwerpen.be login1-vaughan.hpc.uantwerpen.be
Leibniz	login-leibniz.hpc.uantwerpen.be login.hpc.uantwerpen.be	login1-leibniz.hpc.uantwerpen.be login2-leibniz.hpc.uantwerpen.be
Visualization	viz1-leibniz.hpc.uantwerpen.be
Breniac	login-breniac.hpc.uantwerpen.be

Note

Direct login is possible to all login nodes and to the visualization node from within Belgium only. From outside of Belgium, a VPN connection to the UAntwerp network is required.

Compute clusters#

The CalcUA infrastructure contains 3 compute clusters. Partitions in bold are the default partition for the corresponding cluster.

Vaughan #

Partition	Nodes	CPU-GPU	Memory	Maximum wall time
zen2	152	2x 32-core AMD EPYC 7452	256 GB	3 days
zen3	24	2x 32-core AMD EPYC 7543	256 GB	3 days
zen3_512	16	2x 32-core AMD EPYC 7543	512 GB	3 days
ampere_gpu	1	2x 32-core AMD EPYC 7452 4x NVIDIA A100 (Ampere) 40 GB SXM4	256 GB	1 day
arcturus_gpu	2	2x 32-core AMD EPYC 7452 2x AMD Instinct MI100 (Arcturus) 32 GB HBM2	256 GB	1 day

Leibniz #

Partition	Nodes	CPU-GPU	Memory	Maximum wall time
broadwell	144	2x 14-core Intel Xeon E5-2680v4	128 GB	3 days
broadwell_256	8	2x 14-core Intel Xeon E5-2680v4	256 GB	3 days
pascal_gpu	2	2x 14-core Intel Xeon E5-2680v4 2x NVIDIA Tesla P100 (Pascal) 16 GB HBM2	128 GB	1 day

Breniac #

Partition	Nodes	CPU	Memory	Maximum wall time
skylake	23	2x 14-core Intel Xeon Gold 6132	192 GB	7 days

Storage infrastructure#

The storage is organised according to the VSC storage guidelines.

Environment variable	Type	Access	Backup	Default quota	Total capacity
$VSC_HOME	NFS/XFS	VSC	YES	3 GB, 20k files	3.5 TB
$VSC_DATA	NFS/XFS	VSC	YES	25 GB, 100k files	60 TB
$VSC_SCRATCH $VSC_SCRATCH_SITE	BeeGFS	Site	NO	50 GB, 100k files	0.6 PB
$VSC_SCRATCH_NODE	ext4	Node	NO		node-specific

For node-specific details, see the hardware information for Vaughan, Leibniz and Breniac.

Software instructions#

Searching for installed software#

After logging in, the calcua/all module will be loaded automatically. This will let you search through all (centrally) installed software using the module command’s search capabilities.

Recommended Once you have found the specific software module you would like to use, load the corresponding version-specific calcua module. This will also restrict your next searches and module loads to modules in the same version of the software stack, which avoids compatibility errors.

Example Loading a module

To load the GCC module, you can execute the following commands.

module purge

module load calcua/2023a
module load GCC/12.3.0

There are three commands to search for a module:

Search in the list of activated modules
Activated modules are modules that can be loaded without first loading another calcua module.
module av Python
will search for an activated module whose name or version string contains Python.
Search in the list of all available modules
This search also includes modules that require loading other modules first.
module spider Python
If you need more information about the module, including which modules you might need to load, you can also use module spider using a specific module version.

The explanation given about the module(s) that need to be loaded, can be a bit confusing. You have to take only one line from the output and load all modules on that line.
Example Using module spider with a version
If you want more specific information on Python/3.11.3-GCCcore-12.3.0, you can use

$ module spider Python 3.11.3-GCCcore-12.3.0 ----------------------------------------------------------------- Python: Python/3.11.3-GCCcore-12.3.0 ----------------------------------------------------------------- Description: Python is a programming language that lets you work more quickly and integrate your systems more effectively. You will need to load all module(s) on any one of the lines below before the "Python/3.11.3-GCCcore-12.3.0" module is available to load. calcua/2023a calcua/all

In this case, it suffices to first load either calcua/2023a, or calcua/all: it is not required to load both.
Search through the whatis-information
Each module also contains a limited amount of additional information that can be shown with the module whatis command, e.g.
module whatis Python/3.11.3-GCCcore-12.3.0
You can search through this ‘whatis’-information of installed modules using module keyword
module keyword CMake

If you did not find the software you want to use, please do not hesitate to contact the UAntwerp support team at hpc@uantwerpen.be.

Installation directories#

Additional packages or software distributions should be installed on VSC_DATA or VSC_SCRATCH, depending on the circumstances.

UAntwerp Users with a vsc2xxxx account

When working on the UAntwerp clusters, VSC_DATA is a local file system for you. We expect that for Python packages (and similarly for R and Perl), VSC_DATA will give better performance than VSC_SCRATCH due to the amount of file metadata accesses that occur when running Python.

Example To install the sphinx Python package, you can use the following command
```
pip install --prefix=${VSC_DATA}/python_lib sphinx
```
VUB UGent KU Leuven Users with a different home institution

VSC_DATA is a file system at your local institution. Due to the distance between our clusters and your home institution, file access to VSC_DATA will have a high latency.

Installing the packages that you need locally on VSC_SCRATCH will give much better performance.

Conda#

The preferred method of installing additional Python packages on the UAntwerp cluster is using pip, easy_install or python setup, depending on what the package supports. This does require that all non-Python and in some cases Python dependencies are already installed. However, it makes maximum use of what is already installed on the systems, with a minimal number of additional files.

Warning

Like with Conda, when you install from binaries available on PyPi, they will likely not be optimized for the specific CPUs on our system. Moreover, not all binary wheels are compatible with the Linux version that we use. The CalcUA support team always tries to compile packages from source using up-to-date compilers and only uses binary wheels when nothing else works in a reasonable time.

We discourage the use of Conda-variants for various reasons. It should only be used if nothing else works.

Conda installations avoid using libraries already present on the system, effectively installing their own Linux distribution in the Conda directories. As such they consume a lot of disk space and can put a high load on the file system. Expect slower performance just because of that already.
As Conda effectively installs its own upper layers of a Linux/GNU-system and doesn’t use security-sensitive libraries from our system, it is up to you to keep it secure by frequently updating. This is particularly important for those packages that make connections of the internet. If you’re not using any of these, this does not need to be a big concern.
The Conda repositories contain a mix of very well optimized binary packages and packages that are not optimized for modern CPUs. In some cases, multiple versions are available, but as a Conda user you need to be well aware of where to find these. As an example, the Intel Python distribution with properly optimized NumPy, SciPy and a few other performance-critical packages is also available via Conda.

The generic CPU that is used for binaries that should run on everything is usually an ancient Pentium 4 or Core CPU. For some code, e.g., dense linear algebra and FFT, using the newer instructions of more recent processors can give a big speed boost for those routines, up to a factor 4 on Leibniz and Vaughan and up to a factor of 7 or so on the Skylake partition of the previous Tier-1 cluster BrENIAC.