Below please find the software and hardware available on our two clusters, Bluemoon and DeepGreen.
Last updated: January 2020
Bluemoon
Software
Operating System
The operating system is RedHat Enterprise Linux 7 (64-bit) with the GNU compilers (gcc, f77).
Cluster Management
- Resources Manager: TORQUE 6.1.3 (TORQUE is an extension of PBS)
- Workload Scheduler: Moab 9.1.3
- Package Manager: Spack 0.11.2
Installed Software
We offer a variety of software, community codes and tools. Science areas include chemistry, molecular dynamics and image processing. An assortment of compilers, debuggers and performance tools are also available. Specific information is available in our VACC User Documentation (UVM NetID login required).
Hardware
- 32 dual-processor, 12-core (Intel E5-2650 v4) Dell PowerEdge R430 nodes, with 64 GB each, 10Gbit/s Ethernet-connected.
- 8 dual-processor, 12-core (Intel E5-2650 v4) Dell PowerEdge R430 nodes, with 256 GB each, 10Gbit/s Ethernet-connected.
- 32 dual-processor, 10-core (Intel E5-2650 v3) Dell PowerEdge R630 nodes, with 64 GB each, Infiniband 4XFDR (56Gbit/s)-connected. (Reserved for jobs that use IB.)
- 8 dual-processor, 10-core (Intel E5-2650 v3) Dell PowerEdge R630 nodes, with 64 GB each, Infiniband 4XFDR (56Gbit/s)-connected. (Reserved for jobs that use IB.)
- 3 dual-processor, 10-core (Intel E5-2650 v3) Dell PowerEdge R630 nodes, with 256 GB each, Ethernet-connected.
- 22 dual-processor, 6-core (Intel E5-2630) IBM dx360m4 nodes, with 32GB each, Infiniband 4XFDR (56Gbit/s)-connected. (Reserved for jobs that use IB.)
- 130 dual-processor, 6-core (Intel X5650) IBM dx360m3 nodes, with 24GB each, Ethernet-connected.
- 2 dual-processor, 12-core (Intel E5-2650 v4) Dell R730, with 1TB.
- 1 dual-processor, 8-core (Intel E7-8837) IBM x3690 x5, with 512GB.
- 2 dual-processor, 12-core (Intel E5-2650 v4) Dell R730 GPU nodes, each with 2 NVidia Tesla P100 GPUs and 128GB RAM. (Each GPU has 3584 CUDA cores and 16GB RAM.)
- 2 user nodes (2 x Dell R430s, each with 2X 12-core Intel E5-2650 and 64GB RAM.)
- 4 I/O nodes (2 Dell R430s, 10G ethernet connected, and 2 IBM x3655s, 10G ethernet connected) connected to:
- 1 IBM DS4800 providing 260 terabytes of raw storage to GPFS (roughly 197TB usable).
- 1 IBM DS4700 providing 104 terabytes of raw storage (roughly 76TB usable).
- 1 IBM DCS3850 providing 240 terabytes of raw storage to GPFS (roughly 164TB usable).
- 1 Dell MD3460 providing 357.5 terabytes of raw storage to GPFS (roughly 260.5TB usable), and 43 terabytes of solid-state disk to GPFS (for fast random-access data & metadata, roughly 27.5 TB usable.)
- 1 IBM V3700 providing 10 terabytes of solid-state disk to GPFS (for fast random-access data & metadata.)
- 2 Flash-storage GPFS Metadata nodes (IBM x3655s, 10G ethernet connected)
DeepGreen
Software
Operating System
The operating system is RedHat Enterprise Linux 7 (64-bit) with the GNU compilers (gcc, f77).
Cluster Management
- Resources Manager: Slurm 19.05
- Package Manager: Spack 0.11.2
Installed Software
We offer a variety of software, community codes and tools. Science areas include chemistry, molecular dynamics and image processing. An assortment of compilers, debuggers and performance tools are also available. Specific information is available in our VACC User Documentation (UVM NetID login required).
Hardware
- 10 GPU nodes (Penguin Relion XE4118GTS):
- 2 Intel(R) Xeon(R) Gold 6130 CPU@2.10GHz (32 cores)
- 768GB RAM (256GB for GPFS pagepool)
- 8 NVIDIA Tesla V100s with 32GB RAM, NVLink
- 4 2-lane HDR(100Gb/s, so 400Gb/s/node) Infiniband links to QM8700 switch
- 2 NVMe nodes, each with 96TB NVMe devices (12x8TB), replicated to provide 96TB filesystem
- Mellanox QM8700 switch running at HDR speeds