2.1. UAntwerp Tier-2 Infrastructure#
2.1.1. Login infrastructure#
You can log in to the CalcUA infrastructure using SSH via login.hpc.uantwerpen.be
.
Alternatively, you can also log in directly to the login nodes of the individual clusters using one of the following hostnames.
Cluster |
Generic login name |
Individual login node |
---|---|---|
login-vaughan.hpc.uantwerpen.be |
login1-vaughan.hpc.uantwerpen.be
login1-vaughan.hpc.uantwerpen.be
|
|
login-leibniz.hpc.uantwerpen.be
login.hpc.uantwerpen.be
|
login1-leibniz.hpc.uantwerpen.be
login2-leibniz.hpc.uantwerpen.be
|
|
viz1-leibniz.hpc.uantwerpen.be |
||
login-breniac.hpc.uantwerpen.be |
Note
Direct login is possible to all login nodes and to the visualization node from within Belgium only. From outside of Belgium, a VPN connection to the UAntwerp network is required.
2.1.2. Compute clusters#
The CalcUA infrastructure contains 3 compute clusters. Partitions in bold are the default partition for the corresponding cluster.
Vaughan#
Partition |
Nodes |
CPU-GPU |
Memory |
Maximum wall time |
---|---|---|---|---|
zen2 |
152 |
2x 32-core AMD Epyc 7452 |
256 GB |
3 days |
zen3 |
24 |
2x 32-core AMD Epyc 7543 |
256 GB |
3 days |
zen3_512 |
16 |
2x 32-core AMD Epyc 7543 |
512 GB |
3 days |
ampere_gpu |
1 |
2x 32-core AMD Epyc 7452
4x NVIDIA A100 (Ampere) 40 GB SXM4
|
256 GB |
1 day |
arcturus_gpu |
2 |
2x 32-core AMD Epyc 7452
2x AMD MI100 (Arcturus) 32 GB HBM2
|
256 GB |
1 day |
Leibniz#
Partition |
Nodes |
CPU-GPU |
Memory |
Maximum wall time |
---|---|---|---|---|
broadwell |
144 |
2x 14-core Intel Xeon E5-2680v4 |
128 GB |
3 days |
broadwell_256 |
8 |
2x 14-core Intel Xeon E5-2680v4 |
256 GB |
3 days |
pascal_gpu |
2 |
2x 14-core Intel Xeon E5-2680v4
2x NVIDIA P100 (Pascal) 16 GB HBM2
|
128 GB |
1 day |
Breniac#
Partition |
Nodes |
CPU |
Memory |
Maximum wall time |
---|---|---|---|---|
skylake |
23 |
2x 14-core Intel Xeon Gold 6132 |
192 GB |
7 days |
2.1.3. Storage infrastructure#
The storage is organised according to the VSC storage guidelines.
Environment variable |
Type |
Access |
Backup |
Default quota |
Total capacity |
---|---|---|---|---|---|
$VSC_HOME |
NFS/XFS |
VSC |
YES |
3 GB, 20k files |
3.5 TB |
$VSC_DATA |
NFS/XFS |
VSC |
YES |
25 GB, 100k files |
60 TB |
$VSC_SCRATCH
$VSC_SCRATCH_SITE
|
BeeGFS |
Site |
NO |
50 GB, 100k files |
0.6 PB |
$VSC_SCRATCH_NODE |
ext4 |
Node |
NO |
node-specific |
For node-specific details, see the hardware information for Vaughan, Leibniz and Breniac.
See also
For more information on the file systems, please see the UAntwerp storage page.
2.1.4. Software instructions#
Searching for installed software#
After logging in, the calcua/all
module will be loaded automatically.
This will let you search through all (centrally) installed software using the
module
command’s search capabilities.
Recommended Once you have found the specific software module you would like
to use, load the corresponding version-specific calcua
module.
This will also restrict your next searches and module loads to modules in the same
version of the software stack, which avoids compatibility errors.
- Example Loading a module
To load the GCC module, you can execute the following commands.
module purge module load calcua/2023a module load GCC/12.3.0
There are three commands to search for a module:
- Search in the list of activated modules
Activated modules are modules that can be loaded without first loading another
calcua
module.module av Python
will search for an activated module whose name or version string contains
Python
.
- Search in the list of all available modules
This search also includes modules that require loading other modules first.
module spider Python
If you need more information about the module, including which modules you might need to load, you can also use
module spider
using a specific module version.The explanation given about the module(s) that need to be loaded, can be a bit confusing. You have to take only one line from the output and load all modules on that line.
- Example Using
module spider
with a version If you want more specific information on
Python/3.11.3-GCCcore-12.3.0
, you can use$ module spider Python 3.11.3-GCCcore-12.3.0 ----------------------------------------------------------------- Python: Python/3.11.3-GCCcore-12.3.0 ----------------------------------------------------------------- Description: Python is a programming language that lets you work more quickly and integrate your systems more effectively. You will need to load all module(s) on any one of the lines below before the "Python/3.11.3-GCCcore-12.3.0" module is available to load. calcua/2023a calcua/all
In this case, it suffices to first load either
calcua/2023a
, orcalcua/all
: it is not required to load both.
- Example Using
- Search through the whatis-information
Each module also contains a limited amount of additional information that can be shown with the
module whatis
command, e.g.module whatis Python/3.11.3-GCCcore-12.3.0
You can search through this ‘whatis’-information of installed modules using
module keyword
module keyword CMake
If you did not find the software you want to use, please do not hesitate to contact the UAntwerp support team at hpc@uantwerpen.be.
Installation directories#
Additional packages or software distributions should be installed on VSC_DATA or VSC_SCRATCH, depending on the circumstances.
UAntwerp Users with a vsc2xxxx account
When working on the UAntwerp clusters, VSC_DATA is a local file system for you. We expect that for Python packages (and similarly for R and Perl), VSC_DATA will give better performance than VSC_SCRATCH due to the amount of file metadata accesses that occur when running Python.
Example To install the
sphinx
Python package, you can use the following commandpip install --prefix=${VSC_DATA}/python_lib sphinx
VUB UGent KU Leuven Users with a different home institution
VSC_DATA is a file system at your local institution. Due to the distance between our clusters and your home institution, file access to VSC_DATA will have a high latency.
Installing the packages that you need locally on VSC_SCRATCH will give much better performance.
Conda#
The preferred method of installing additional Python packages on the UAntwerp cluster is using pip
, easy_install
or python setup
, depending on what the package supports. This does require that all non-Python and in
some cases Python dependencies are already installed. However, it makes maximum use of what is already
installed on the systems, with a minimal number of additional files.
Warning
Like with Conda, when you install from binaries available on PyPi, they will likely not be optimized for the specific CPUs on our system. Moreover, not all binary wheels are compatible with the Linux version that we use. The CalcUA support team always tries to compile packages from source using up-to-date compilers and only uses binary wheels when nothing else works in a reasonable time.
We discourage the use of Conda-variants for various reasons. It should only be used if nothing else works.
Conda installations avoid using libraries already present on the system, effectively installing their own Linux distribution in the Conda directories. As such they consume a lot of disk space and can put a high load on the file system. Expect slower performance just because of that already.
As Conda effectively installs its own upper layers of a Linux/GNU-system and doesn’t use security-sensitive libraries from our system, it is up to you to keep it secure by frequently updating. This is particularly important for those packages that make connections of the internet. If you’re not using any of these, this does not need to be a big concern.
The Conda repositories contain a mix of very well optimized binary packages and packages that are not optimized for modern CPUs. In some cases, multiple versions are available, but as a Conda user you need to be well aware of where to find these. As an example, the Intel Python distribution with properly optimized NumPy, SciPy and a few other performance-critical packages is also available via Conda.
The generic CPU that is used for binaries that should run on everything is usually an ancient Pentium 4 or Core CPU. For some code, e.g., dense linear algebra and FFT, using the newer instructions of more recent processors can give a big speed boost for those routines, up to a factor 4 on Leibniz and Vaughan and up to a factor of 7 or so on the Skylake partition of the previous Tier-1 cluster BrENIAC.