VACC User Guide

Logging In | Submitting Jobs to the Cluster | MPI Jobs | Common Commands for Monitoring Your Jobs | Caveats

Advanced VACC User Documentation

Instructions on Compiling MATLAB code

Logging In

There are two machines accessible to VACC users: bluemoon-user1.uvm.edu and bluemoon-user2.uvm.edu. Use ssh to get access to the cluster.

It is highly recommended to change your shell to bash (In order to do this, log in to zoo, run shupdate to get new versions of shell profiles for both bash and tcsh, and then run zoochsh to change your shell to bash.)before using bluemoon. Note that it takes a sync from DCE-LDAP before bluemoon sees that your shell has changed - currently that happens overnight between 2 and 5 AM.

Submitting Jobs to the Cluster

We are currenty using PBS portable batch system for scheduling jobs.

These are pretty easy, and if you have lots of them, PBS will farm them out to the different processors that are available, so you can queue up many jobs and they will be started as soon as processors become available. However, keep in mind that there is some overhead in scheduling jobs, so if you have many jobs (hundreds or thousands) that each take less than 10 seconds or so to run, you should think about aggregating them.

For each job you want to run, create a job script. In this case we'll call it myjob.script:

Submit the job by running: qsub myjob.script

Return to Top

MPI Jobs

For jobs that take advantage of multiple processors, PBS will help communicate to your program which nodes it should run on. You just need to make use of the right variables.

Example Scriptfile:

Return to Top

Common Commands, for Monitoring Your Jobs

showq

showq will show what the currently queued and running jobs are. It will also show the remaining walltime allocated to each job. showq - u userid shows status of your jobs.

qstat

qstat will print out the currently queued, running, and recently exited jobs.

You can see from the above that there are 3 jobs in the queue, and all three are running (S=R).
qstat -r will give slightly more detail (but only for jobs that are currently running):

This shows the number of nodes assigned to each job, and the elapsed runtime (wall clock time, not CPU time.) To get more detailed job status, use qstat -f. This will give details such as the nodes your job is currently running on, the environment variables set, the amount of resources used so far, or if your job has not started yet, the reason why.

qstat - u userid shows status of only your jobs.

Return to Top

Caveats

While we consider research computing central to the University's mission, bluemoon is not considered a business-critical service. Its architecture has been chosen for performance, as most HPC clusters are, rather than resiliency or reliability. This means that if a component of the cluster fails, we may not respond in the same timeliness that we would if, say, www.uvm.edu were down. If a node or two were to fail, it will probably be several days before it is repaired, since it is presumed the cluster will continue to operate. In the event of multiple failures of critical systems across campus, we will attend to business-critical services before we get to fixing problems with bluemoon.

Also note that we have not invested any money in data backups for the cluster. In the event of data failure, the cluster configuration would likely be recoverable, but research data will not be regularly backed up, as it would be on zoo.uvm.edu. At this time, users are responsible for backing up their own data.