It wasn’t long ago that scientists downloading high volumes of data from the Atmospheric Radiation Measurement (ARM) user facility worked with that data locally using their own computing resources. However, with data volume and complexity escalating, user options for large-scale projects were needed.
In 2017, ARM introduced a new capability for its scientific users—computation clusters to the ARM Data Services that provide important resources where the data reside and eliminate the need for individual large-volume downloads.
“It is important to meet the needs of scientific users who use large volumes of ARM data, which significantly reduces the data processing time for tasks like large-scale radar processing, complex synthesis product developments, and advanced analytics and visualizations,” says ARM Technical Director Jim Mather. “The goal is to make it simple for users to migrate computational scripts from single workstations to the new computation clusters.”
There are no formal application deadlines to use these high-performance computing resources. Requests for the use of ARM computation clusters are reviewed quarterly and are subject to scientific peer review. Requests must:
- describe how the use of cluster resources will support the advancement of atmospheric research
- describe the scientific objectives of the work
- describe the ARM data sets to be analyzed
- explain why ARM computation cluster resources are the most appropriate computing resources for achieving the science goals.
The first proposals were awarded last year. According to Giri Prakash, ARM Data Services manager at Oak Ridge National Laboratory in Tennessee, that pioneering group of ARM users tapped into the high-performance computation clusters with great success. The group includes:
- Hagen Telg, Cooperative Institute for Research in Environmental Sciences (CIRES)
- Christopher Williams, University of Colorado, Boulder; CIRES
- Christine Chiu, Colorado State University
- Jeremy McGibbon, University of Washington
- David Turner, National Oceanic and Atmospheric Administration
- Maximilian Maahn, CIRES
- Thijs Heus, Cleveland State University.
“The ARM Data Center has integrated two computation clusters called Stratus and Cumulus,” says Prakash. “They support model simulations, petascale data storage, and big-data analytics for successful ARM science research.”
Stratus Cluster – for Big Data
The Stratus cluster is a midscale Cray cluster that consists of 1,080 processing cores with 256 gigabytes of memory per node. It offers two tiers of data storage, including a 2-terabyte solid state drive per node as fast read/write scratch storage and a 100-terabyte Lustre parallel file server. This cluster allows science users to develop and run computer-intensive retrieval algorithms and perform large-scale ARM data analysis and extractions with direct access to ARM data.
In addition to these research applications, ARM staff use Stratus for a range of ARM infrastructure processes, including routine radar processing, large-scale reprocessing, complex value-added product operations, NoSQL-based advanced visualizations, long-term data quality analysis, and bulk data plot generations.
Cumulus Cluster – for High-End Modeling
The Cumulus cluster is a midrange Cray system and consists of 4,032 processing cores with a 2-petabyte General Parallel File System. This cluster is used for routine operations of the Large-Eddy Simulation (LES) ARM Symbiotic Simulation and Observation (LASSO) workflow. LASSO combines observational data and LES modeling to support the study of atmospheric processes, improvement of observational retrievals, and representations of clouds and radiation in earth system models.
The initial LASSO implementation on the Cumulus cluster was at ARM’s Southern Great Plains atmospheric observatory in Oklahoma and focused on high-resolution model simulations of shallow clouds driven by ensembles of forcing inputs.
Need Other Computing Resources?
For something beyond Stratus and Cumulus, other resources are available. For large-scale simulations and data analytics applications that do not require high volumes of ARM data, go to the U.S. Department of Energy’s (DOE) National Energy Research Scientific Computing Center or other DOE leadership computing user facilities, such as the Oak Ridge Leadership Computing Facility and Argonne Leadership Computing Facility. Allocations for these facilities should be requested directly from the individual facilities.
# # #The ARM Climate Research Facility is a DOE Office of Science user facility. The ARM Facility is operated by nine DOE national laboratories, including Oak Ridge National Laboratory.