Supercomputing for Beginners: Cheatsheets for Queuing System Quick Reference

High Performance Computing (HPC) typically involves connecting to very large computing systems elsewhere in the world.

These other systems can be used to do work that would either be impossible or much slower or smaller systems.

The standard method of interacting with such systems is via a command line interface called Bash.

An HPC system is a set of networked machines.

HPC systems typically provides login nodes and a set of worker nodes.

The resources found on independent (worker) nodes can vary in volume and type (amount of RAM, processor architecture, availability of network mounted file systems, etc.).

Files saved on one node are available on all nodes.

The scheduler handles how compute resources are shared between users.

Everything you do should be run through the scheduler.

A job is just a shell script.

If in doubt, request more resources than you will need.

The home directory is the default place to store data.

The scratch directory is a larger space for temporary files.

On Hawk in Cardiff home is backed up but is also a slower disk.

Quotas on home are much smaller than scratch.

Discover available software with module avail

Load software with module load softwareName

Unload software with module purge

The module system handles software versioning and package conflicts for you automatically.

wget downloads a file from the internet.

scp transfer files to and from your computer.

You can use an SFTP client like FileZilla to transfer files through a GUI.

The smaller your job, the faster it will schedule.

Be careful how you use the login node.

Your data on the system is your responsibility.

Plan and test large data transfers.

It is often best to convert many files to a single archive file before transferring.

Again, don’t run stuff on the login node.

Don’t be a bad person and run stuff on the login node.