Skip to content

Sharing Large-Language Models (LLMs)

If a user/group is finding an LLM model useful and feels other users/groups may find it useful as well, there are two ways that the sharing of models can happen on the cluster: 1) The user can themselves share access to the model that resides in their group’s file space, or 2) The user can request of the VACC admins that the model be shared in the central model library located at /gpfs1/llm.

Sharing access yourself

Sharing access to any file or directory with other users/groups on the VACC can be done using Access Control Lists (ACLs).

An example:

To share models with another group on the VACC, a user can create a directory in their group’s shared PI space expressly for this purpose and set a default ACL on that directory so that all files and directories added to the directory will be accessible by the group you specify:

$ setfacl -R -m g:<newgroup>:rx /gpfs1/pi/<pi-netid>/<share-dir-name> (replacing <newgroup> with the group name, ex: pi-pbrochu, and <share-dir-name> with the new directory name)

You’ll also want to allow that group to reach that shared folder, so you’ll set an ACL to that effect on your PI folder (set this way, the other group won’t be able to see what else is in the PI folder, they will only be able to navigate through it to the shared directory):

$ setfacl g:<newgroup>:x /gpfs1/pi/<pi-netid> (replacing <pi-netid> with the netid of the PI of the group)

If instead of sharing to a single other group you would like to share the directory of models to all VACC users, replace g:<newgroup> in both commands above with o.

Requesting a model be added to the central library

The process by which an LLM model is added to the central library (currently at /gpfs1/llm):

User opens a ticket (by emailing vacchelp@uvm.edu) to request a model be considered for central hosting; information to include:

  • Name of model
  • Links to model source and documentation
  • Format(s) requested (original format, HuggingFace safetensors, Ollama GGUF, etc)
  • Description of why the model is useful/valuable

VACC admins approve the model based upon these criteria (among others):

  • Resource efficiency
  • Models using fewer resources (GPU, CPU, RAM, etc) will be prioritized
  • Security, conformity to UVM/ETS policies
  • Number of users/groups using or being interested in using the model (models with more interest from users/groups will be prioritized)

If a model passes the approval process, VACC admins will share the model to /gpfs1/llm and add an entry to the LLM Central Library page.

VACC admins will monitor usage of the model and delete the model if it has not been accessed in one year.