Getting Started:
Access to the Sonic HPC Cluster is through an SSH command line interface on the Sonic login node - login.ucd.ie on port 22
ssh username@login.ucd.ie
If you are using windows, you'll need an SSH client such as Putty to SSH to the login node. You will also need to you a transfer client such as WinSCP. Mac OSX or Linux users can use any Terminal application.
The Sonic Cluster is a Linux environment. Users not familiar with working in a Linux environment should familiarise themselves with the basic file and directory operations of the Linux command line interface. There are a good series of introductory tutorials available on the University of Surrey's Electronic Engineering Department website
Once you are logged in to the cluster, you will find yourself in your home directory. On Sonic, your home directory is at /home/people/username (where usernameis your UCD Connect username).
Each shared group user has a disk quota of about 50GB. Within your home directory you will see a linked directory called "scratch" - this is an area that users can use to temporarily store working data. As this Please note that this data is liable to be deleted at short notice and as such is not suitable for long term storage. Users of a contributory group have larger quotas
File Storage:
Each shared group user has a home directory at /home/people/username which can be used to store approximately 50GB of data - this storage area is intended only for Research Data relevant to the users cluster usage and should not be used as a private data store.
Additionally, users have access to extra storage via the 'scratch/' directory/symlink within your home directory. This storage area is for temporary short-term storage of data required for input into, or output from, your currently running jobs. This is not a long term stoage platform and files not modified for 6 months will be deleted
Submitting Jobs:
To submit a job to the Sonic Cluster, you will need to use the sbatch command to queue your job. sbatch is part of SLURM which controls the queues on Sonic.
Your job will first need to be saved in a script which will need to pass certain parameters to Slurm. These parameters should be specified at the beginning of your jobs script.
An example of a script using some of these parameters is shown below:
#!/bin/bash -l
# Set the number of nodes
#SBATCH -N 1
# Set the number of tasks/cores per node required
#SBATCH -n 3
# Set the walltime of the job to 1 hour (format is hh:mm:ss)
#SBATCH -t 01:00:00
# E-mail on begin (b), abort (a) and end (e) of job
#SBATCH --mail-type=ALL
# E-mail address of recipient
#SBATCH --mail-user=myemailaddress@ucd.ie
# Specifies the jobname
#SBATCH --job-name=myjob
# Change working directory to current directory
cd $SLURM_SUBMIT_DIR
# Your code here! (This example just prints the current date and exits!)
date;
To run the script, type:
$sbatch myjob.sh
This will then return something like:
Submitted batch job 331
where 331 is your job ID.
See here for more detail on sbatch parameters. There are also two example scripts present in your home folder: samplejob.sh and samplempi.sh
To view your own queued jobs, you can use the 'squeue' command, which will show you your job id, the name of the job,the jobs current status, and the queue (called a partition in slurm) and the length of time in which your jobs is running.
sacct -j jobid will give detailed of your job once it is finished
You can delete a job from the queue by using "scancel <jobid>".
Specifying the GPU nodes
If you wish to avail of the GPU enabled worker nodes (equipped with 2 Nvidia Telsa V100's each). To request a GPU server and all its cores you should submit to the GPU queue/partion with this command . There is a sample GPU job in your home directory called samlegpu.sh which has all the setting you to need to use a GPU nodes and that you should amend as required. To submit enter the following
sbatch --partition=gpu samplegpu.sh If you are a computer science user please use this command
sbatch --partition=csgpu samplegpu.sh
Specifying the High Memory nodes
If you wish to avail of the high memory server sonicmem2 equipped with 768 GB of RAM and sonicmem3 equipped with 1.5TB of RAM you should add this line to your job submission script
sbatch --nodelist=sonicmem3 cluster
Queues:
For shared queues there is one standard queue which allows job to run up to a maximum of 10 days
As contributors to the cluster users from the School of Computer Science jobs are sent to the cs queue
Interactive Jobs
If your jobs require user intervention then you need to run an interactive job which requests an bash shell on a worker node. The syntax for submitting an interactive job is
srun --time=03:00:00 -c6 --pty bash
--time referes to the length of time you require the shell prompt
-c6 means the number of cores you require
Graphical Interactive Jobs
To run a graphical interactive job you must forward your display to sonic and you must submit an interactive job to the graphical queue. To forward your display to sonic on linux use the following command ssh -X username@sonic.ucd.ie . For windows you need to run a service such as Xming and select "Enable X11 forwarding " on your putty session which is on the left hand side under SSH and X11. Once on sonic you run this command to request a shell session on a node that has graphical components installed . For
srun --partition=graphical --time=01:00:00 --x11 --pty bash
Once you run this command the job will queue and will appear to "hang" after the message waiting on resources returns, this is normal typically you will wait about 30 seconds before getting a prompt. As these are interactive jobs if your session ends the job will also end as well so please save output as you go along .
How many Jobs can I run at time (This information is subject to change)
Currently each standard shared user is allowed to use up to 48 cores at any one time. When the cluster is busy this number is affected by the amount of resources availble. Jobs with a smaller number of cores requested typically start earlier than those jobs wishing to reserve large amount of cores. When the cluster is quiet, a hard limit will allow users requesting more than 48 cores access of up to 260 cores, to utilise any free resources.
Contributory uses are allowed to utilise more cores at any one time that shared users, the amount of maximum.
What Software is installed
The best way to find out what centrally installed software is availble is with the module avail command.
Is there an easy way to load environment variables into my path
Yes, using the modules package you can load environmental for different packages into your path. More information on this can be found in the "Sonic Software" section below.
Access off Campus
There is no direct access to the new cluster externally . If you have a @ucd.ie email address you can apply for a UCD VPN account using the VPN request form on VPN webpage and access the cluster using this. Please ensure you are logged into UCD Connect to access the form
If you have a @ucdconnect.ie email address you can apply for a bastion host account by emailing ithelpdesk@ucd.ie .
How do I use the Bastion Host ?
Windows User Guide for Bastion Host
User Guide for Bastion Host for Linux Clients
