Skip to content

Data and Storage

Storage specific to the VACC

/gpfs1 is the main pool of storage used by the VACC. We provide daily backups, copied to a data recovery location, and are generally able to retain the last 7 days, 4 weeks of weekly backups, and 3 months of monthly backups, although there is some fluctuation based on overall usage. If you need files recovered from backups, please email us at vacchelp@uvm.edu with details about what files or directories you need recovered and from what time.

/gpfs2 is considered scratch space. That means there are NO BACKUPS taken of /gpfs2, which is linked to a scratch folder in your home directory. Disk quotas are higher in /gpfs2/scratch, so it’s great for large temporary or intermediate files.

/gpfs3 is available as scratch space and consists of 192TB of NVMe SSD, replicated to provide a 96TB filesystem. This is scratch space and has NO BACKUPS. Old data may be flushed occasionally.

GPFS disk quotas

The amount of gpfs storage is your disk quota. Disk quotas are determined by the VACC account owner’s tier — the level of support the principal investigator (PI) is providing to the VACC.

All members of a PI-group share the group’s disk quota. A “PI-group” includes the PI and any sponsored users under that account. If you have no sponsored users, the PI-group is just you. Check the disk usage of your PI-group with the command groupquota.

GPFS disk storage & file limits

In the table below, “block” is the total size of your group’s disk storage. “Files” is the total number of files allowed to be written to disk.

gpfs1

TIER BLOCK
Soft Limit / Hard Limit
FILES
Soft Limit / Hard Limit
1 2TB / 4TB 1M / 1.5M
2 8TB / 13TB 3M / 6M
3 20TB / 25TB 6M / 12M

gpfs2

TIER BLOCK
Soft Limit / Hard Limit
FILES
Soft Limit / Hard Limit
1 4TB / 8TB 1.5M / 3M
2 15TB / 25TB 4M / 8M
3 35TB / 45TB 8M / 16M

Soft Limit vs. Hard Limit

If you look at the table above, you will see two types of limits:  soft and hard.

  • Soft limits are the amount of resources you are allocated based on your tier and the filesystem you are using.
  • Hard limits are the buffer of resources we allow you because we understand that sometimes resources are unintentionally exceeded.
  • You have a grace period of 7 days to reduce your usage of resources back to your soft limit.

For example, if you unintentionally exceed the soft limit of resources allocated to your tier/filesystem, your work will not be terminated and deleted for 7 days (the grace period). However, once the grace period of 7 days expires, your work will be terminated and files will be deleted to the soft limit you are allocated based on your tier/filesystem.

Research storage (Netfiles)

Do NOT use the VACC for for general file storage. The storage space on the GPFS file servers is for data you are currently using on the cluster. Please do not save/store general data or data that will not be used for active work there. For data storage, see Research Storage.

Research Storage can be requested that is accessible from the VACC cluster. When that is created, all Netfiles 'shares' will show up under the directory /netfiles. The specific server that is included in the message sent when shares are created is NOT used.

Transferring data to the VACC cluster

There are multiple ways to transfer data to and from the VACC cluster.

Using a command line terminal

On MacOS, Linux, and on UVM administered Windows machines, there should be a Terminal application. On Windows 10 and 11, the Command Prompt or PowerShell application can be used.

From the terminal prompt, you can copy files using the scp command. The scp command needs to know the name of the cluster host, the user on that host, the name of the file(s) being copied, and the location to which they should be copied. In the following examples, substitute your own NetID for netid in the text.

Note that the colon (:) character is used to separate the name of the host from the directory path. Anything after the : that does not begin with a / is assumed to be a file or directory in your home directory.

Copying a file or directory to the VACC cluster

Copying a file called some_file.txt from the current directory on your computer to your home directory on the VACC, you would use the first example below. To copy a file called iris.R to the scripts directory under your home directory on the VACC, use the second example. Note that the file will have the same name when copied since no alternate filename is given.

# Copy some_file.txt to ~/some_file.txt
scp some_file.txt netid@login.vacc.uvm.edu:

# Copy iris.R to the VACC into the directory ~/scripts; will not create ~scripts
scp iris.R netid@login.vacc.uvm.edu:scripts

# Copy the directory and contents of scripts to your home directory on the VACC
scp -r scripts netid@login.vacc.uvm.edu:

Copying a file or directory from the VACC cluster

All of these examples presume you want to copy to the current directory on your own computer, specified by ./.

# Copy some_file.txt from the VACC to the current directory
scp netid@login.vacc.uvm.edu:some_file.txt ./

# Copy iris.R to the VACC into the directory ~/scripts; will not create ~scripts
scp netid@login.vacc.uvm.edu:scripts/iris.R ./

# Copy the scripts directory from the VACC to your computer
scp -r netid@login.vacc.uvm.edu:scripts ./

Using a graphical file transfer program

There are several graphical file transfer programs available for both MacOS and Windows. We will use FileZilla as an example. All such programs typically need the same information, which we will provide here. These will often want to use sftp instead of scp. The results are the same.

Once you have installed and started FileZilla, open a new connection with the following connection settings.

Field label Value
File protocol: SFTP
Host name: login.vacc.uvm.edu
User name: netid
Password: your UVM NetID password
Port: 22

We recommend that you NOT save your password, but save the rest for easy, future use.

Click on Connect to make the connection. FileZilla should present to panels with files and folders, one will be your computer and the other will be the VACC cluster. You should be able to drag and drop files or folders from one to the other.

Using Globus

Globus is a web-based data management tool that enables researchers to easily share and receive large data sets with collaborators from other institutions.

Globus has been configured to work with data stored on Netfiles shares, VACC accounts, and Microsoft OneDrive – allowing UVM researchers to easily share and receive enormous data transfers using existing research storage solutions.

Please see the page at the Globus Research Computing knowlegebase for more information about using Globus.