ARM Priorities

The Air-ARM project is procuring a replacement aircraft for the AAF G-1. Air-ARM does not include onboard scientific instrumentation. This ENG is designed to capture the replacement and development activities for the said onboard scientific instrumentation.
Over 50 AAF owned and operated instruments have served the community during the G-1 deployments. AAF aims to maintain/replicate the current AAF instrumentation capability while (where possible) also achieving improvements that serve the broader science community.

Part a (by January 30): SN upgrades to address delayed tickets: the objective of this task is to capture the reason for any ticket that cannot be automatically acted on and closed. We expect this to be straightforward, adding a few key features and automatic reporting within SN.

Part b (by April 30): routine instrument metadata workflow automation (a follow-on/phase 2 to the INST process) will include integration of existing capabilities (e.g., PCM processes list), and will automate a check of database (DB) tables, and automated/programmed cloning where appropriate. Metadata Service (MDS)->Service Now(SN) connections will allow MC coordinators tp review the automated metadata. This saves manual clones within MDS, and minimizes potential human error (e.g., cloning wrong data level). Other automation will include integrated checking of Data Discovery (DD) (link generation) within SN.

Part c (by September 30): guest instrument metadata and data archival workflow modernization will include development of an Online Metadata Editor (OME)<->ServiceNow(SN)<->Metadata Service (MDS). In addition to providing helpful automation for data submitters and metadata team, this capability will also allow us to provide key features (e.g., a "coming soon" button on field campaign pages) and allow for better tracking of PI-provided dataset submissions (e.g., a SN dashboard) for transparency to management.

Benefit to sponsor/program: decreased overhead with a modern metadata and data flow (increased efficiency) and better tracking (e.g., dashboard) of datasets expected and received; increased transparency in what datasets will be included in a campaign for the user community can push up user metrics (e.g., data use, publications citing data).

Motivation The vast ARM data archive contains an extensive resource of atmospheric measurements spanning over 25 years. User feedback suggests that it can be difficult to find high-quality data of scientific interest. ARM has begun a number of activities to help improve accessibility of ARM data, such as upgrades to the Data Discovery Tool and updating the list of recommended datastreams. To complement these activities, ARM has developed the concept of data epochs (defined as time periods with high quality data of scientific interest) and tagged descriptors (e.g., dimension of quality, other atmospheric variables in use by other teams, hierarchical terms) to enhance existing metadata. For a given datastream, ARM metadata currently provides many descriptors such as location, time, data source (i.e. instrument, VAP, external source, campaign, PI product), type of instrument, types of measurements, and amount of processing (data level). The introduction of the new tagging to ARM data holdings adds another layer of metadata that will enrich the user experience.

This engineering change is to refine the scope and guidelines for defining data epochs, identify appropriate metadata tags in accordance with Dublin Core key principles, develop necessary database tables to hold this information, and metadata tools to assign and edit these tags. Finally, search capabilities on the web site and Data Discovery tool will be enabled for users to filter and order data of interest. These activities will be in coordination with activities under ENG 4341 (2021), which focuses on an initial set of tagged descriptors aimed at recommended data and collected during the rationale statement process.

During the 2021 Developers Meeting, participants of the Data Tagging breakout session provided ample feedback and suggestions for tagged descriptors and data epochs. We will use this information to develop the design document and guidelines.

Purpose of this ENG is to design, and deploy a new Cumulus-2 cluster within the ARM Computing Environment. Cumulus-1 has served the needs of ARM's LASSO modeling, and other research and data processing efforts over the last few years. As Cumulus-1 comes close to end of life, a replacement cluster needs to be designed and deployed to ensure uninterrupted availability of computational capacity for LASSO model simulations and operational data processing. This overarching ENG would capture all stages of requirements gathering, design and deployment thru child ENG/EWO/TSKs.

Target user groups: - - LASSO development and operations team. -- VAP developers. -- Operational data processing team. -- ARM/ASR Research community.

The Eppley radiometers (NIP, 8-48, PSP) are in need of replacement. This ENG is to track the requirements, testing, procurements and deployment of the new radiometers.

Introduction To obtain retrievals of fractional sky cover over its atmospheric observatories, the ARM user facility uses Total Sky Imagers (TSI), which provide real-time processing of hemispheric visible images of daytime sky conditions. For a continuous representation of cloud life-cycles, an Infrared Sky Imager is also operating at the Southern Great Plains facility that captures infrared images of the sky, during both the day and night.

Background TSIs were initially installed at the ARM sites from 2000 to 2003. The instruments were re-engineered to increase their performance in the harsh environments of the remote sites (ENG0000378) and later modified to increase imager resolution (ENG0000674), provide site-specific imager calibration (ENG0000644), and improve autonomous control (ENG0003147). However, the TSIs are aging and many of the parts have become unserviceable and/or are no-longer in production.

Objective The purpose of this request is to consider options for the replacement of the ARM TSIs with newer, commercially-available, technology. I will present the current status of the TSIs, results of the ARM Sky Imager Measurement Needs survey conducted in May, and identified instrument replacement options.

Reference Morris V.R. 06/11/2019. "Fractional Sky Cover Measurement for the ARM User Facility." Presented by Victor Morris at ARM/ASR PI Meeting, Rockville, MD. (https://asr.science.energy.gov/meetings/stm/posters/abstract/2197)

The high temporal resolution convective cell tracking measurements effectuated with the second generation C-SAPR2 during the CACTI field campaign in Argentina, revealed the necessity for the development of an automated adaptive scanning system for the radar. For the upcoming TRACER campaign in Houston, Texas, we propose to develop and implement on the C-SAPR2 a convective cell tracking algorithm for the high resolution polarimetric scans. We would prefer to implement two scenarios: a) The C-SAPR2 will perform low elevation surveillance scans for cell identification, based on the reflectivity threshold limits. Once the cell is identified, the radar will follow the selected cell until it satisfies the tracking parameters. b) To increase the temporal resolution, when it is possible, the radar will use information from an external radar or network, and will do only the cell tracking. The C-SAPR2 will accept the tracking parameters (azimuth and range of the center of the cell to be followed), and an identifier for the scanning task.

To purchase 2 additional DL units to replace older units at SGP EFs and deploy new systems to the supplementary sites at Southeast US AMF3. ARM has procured 2 additional DLs in FY21 for the same purpose. Instruments will be initially shipped to SGP.

Procurement of TSI SMPS 3938 requested for deployment in ENA for the ambient particle size measurement from 10 to 500 nm. Categorized as a baseline measurement for size distribution, the instrument will also serve as a reference measurement for ACSM volume closure comparison and a potential calibration unit for size and count dependent measurements in ENA.

Planetary boundary layer (PBL) depth is important to a wide range of atmospheric processes including cloud formation, aerosol mixing and transport, and chemical mixing and transport. The CAPI working group has proposed development of an ARM PBL height VAP to make this important value available to the ARM and broader scientific community.

Numerous instruments and algorithms may be used for PBL height detection, each with their own strengths and weaknesses. We plan to implement several different algorithms for PBL height detection using different instruments as the comparisons between the different methods will provide useful information about the reliability of the PBL height retrievals. We will develop this VAP in a phased approach, in which with we start with relatively simple algorithms and successively add new instruments/algorithms. Additionally, we will initially focus on the convective daytime boundary layer, as it is more easily detected and algorithms are more reliable than for the stable nighttime boundary layer.

In the first phase of this VAP (covered by this ECR), we will focus on implementing methods for radiosondes, ceilometer, and MPL as these instruments exist at all ARM sites. Two radiosonde methods (Heffter and bulk Richardson) and three ceilometer/MPL algorithms (1D gradient, wavelet, combined 2D gradient and wavelet) have been identified. More details on these methods are given in the implementation plan. Code for these algorithms will be provided by ARM scientists, and will be implemented in ISDE. In the next stage of development, which will be defined in more detail at a later date, we will explore implementing retrievals for more advanced instrumentation (AERI, Raman Lidar, Doppler Lidar, Radar wind profilers) that do not exist at all sites.

Initial evaluation of the VAP will focus on several IOP periods: MC3E, Azores, CARES, and AMIE. As part of the VAP development, we will also develop a database of PI PBL height retrievals for these IOP periods. We do not plan to run a formal inter-comparison, but will use this database to 1) evaluate the implementation of the VAP methods, 2) identify promising methods to implement in the next state of development, 3) identify areas of uncertainty and methods of flagging PBL height retrievals, and 4) encourage science PIs to submit their retrievals as formal ARM PI products.

A draft PBL height implementation plan has been developed and is available from Sally upon request.

At the moment users can only subset ordering using data stream name, start date and end date. Given the huge volume of radar data users need to be able to subset the data based on radar volume contents. This will create a summary data point for each radar data volume CMAC2.0 touches and save it to a Cassandra database within the ADC for future use by the ADC in data discovery and ordering.

The goal of this ENG is to revive the CCN Profile VAP that has been developed, update it, and get it running in production. A technical report on this VAP is already available and much of the development is already done. The technical report is available at : https://www.arm.gov/capabilities/vaps/ccnprof At the time the VAP was developed, the CCN and RL data needed for the VAP were not sufficiently high quality to generate a reliable and high-quality VAP. The data quality has dramatically improved to the point where this VAP can be re-visited.

This workscope will use CCNAVG, CCNSPECTRA, and SMPS datastreams to generate a CCNKappa VAP product. Kappa is a parameterized representation of an aerosol particle's hygroscopicity and is widely used in models to calculate CCN concentrations and cloud properties. The critical dry diameter at various supersaturation (SS) setpoints in the CCN instrument will be calculated by assuming particles greater than critical diameter are CCN-active and reconciling CCN number and particle size distributions integrated to that diameter. Kappa-Köhler theory (Petters and Kreidenweis, 2007) will be used to generate the time-series of kappa values as a function of SS.

The ARM TBS and AAF teams have collected almost 350 hours of flight data at SGP and AMF3 from 2018-2020. Data collected during these TBS deployments are available on Data Discovery for the following datastreams: tbsground (surface temperature, pressure, wind speed, gust wind speed, wind direction, and relative humidity), tbspops (aerosol number concentration and size distribution from 140 nm – 3 µm), tbscpc (aerosol number concentration from 10 nm – 1 µm), tbsimet and tbsimetxq (airborne temperature, pressure, relative humidity, and altitude), tbswind (airborne wind speed and gust wind speed), tbsslwc (airborne supercooled liquid water content), and tbsdts (airborne high-resolution temperature). The data currently exist on Data Discovery as unique datasets, and a user would require familiarity with TBS operations to request and interpret data from multiple instruments flown simultaneously. The ARM user community would benefit from the creation of integrated datasets and visualizations of TBS flight data, to aid in the identification of flight data collected during atmospheric conditions of research interest, determining when multiple instruments of interest were operating, promoting a broader community understanding of data available from TBS flights, and improving the ease of use of TBS datasets. To achieve these goals we will merge individual TBS datastreams into an integrated dataset and create visualizations of the integrated data. Data from the surface-based ceilometer will also be added to allow the user to determine cloud base height and if the flight occurred under clear or cloudy conditions. A process will be developed to allow this integrated dataset and visualization to be generated during future ARM TBS data ingests.

Cloud thermodynamic phase identification is important to understand many cloud processes such as ice particle production, precipitation formation, and cloud lifecycle evolution. The CAPI working group has proposed development of an ARM cloud phase VAP to make this important value available to the ARM and broader scientific community.

Cloud thermodynamic phase classification can be determined from remote sensing measurements. Specifically, active remote sensing instruments such as lidar and radar provide vertically resolved cloud thermodynamic phase identification. Due to different wavelengths, lidar backscattered signal is more sensitive to small particles with high number concentrations such as liquid droplets. Lidar depolarization measurements can be used to identify the nonspherical particles such as ice crystals. However, lidar signal is quickly attenuated by cloud droplets. Radar signal is dominated by large particles such as ice crystals and is susceptible to attenuation only in case of strong precipitation. In addition, microwave radiometer measurements can provide constraints on the presence of liquid layers. Since no one instrument can provide sufficient level of information to cover from small droplets to large ice particles and snow, we use the multisensor method developed by Shupe (2007) to classify cloud phase. Similar to the ARM Ka band Zenith Radar (KAZR) Active Remote Sensing of Clouds (KAZR-ARSCL) VAP, the multisensor approach uses coincident ARM Micropulse lidar (MPL) with depolarization, Ka-band ARM Zenith Radar (KAZR), and microwave radiometer measurements, together with atmospheric thermodynamic profiles from radiosonde measurements to determine cloud thermodynamic phases.

For the initial implementation, the VAP will be implemented at the ARM North Slope of Alaska (NSA) atmospheric observatory central facility at Utqiaġvik. An HSRL system was deployed and has been acquiring measurements at the NSA Utqiaġvik site since April 2011. The Ka-band ARM zenith radar (KAZR) replaced the millimeter-wavelength cloud radar system starting in May 2011, providing measurements with high temporal and vertical resolution and reduced spectral artifacts. These measurements provide high quality inputs for the VAP. Over the NSA Utqiaġvik site, persistent stratiform mixed-phase clouds are frequently observed during the transition seasons such as in spring and fall. Liquid clouds are often observed during summer while ice clouds in winter. These seasonal variations of major cloud thermodynamic phase provide a good opportunity to apply the cloud phase identification methods.

A draft cloud phase identification implementation plan has been developed and uploaded.

The goal of this ENG is to produce a merged size distribution VAP that will combine data from the SMPS and APS into a single, continuous size distribution. We have nearly finished the harmonization of the size distribution instruments and data are now available in a consistent format and with consistent naming conventions. There has been a longstanding request from users of the aerosol data to produce a single size distribution from multiple instruments. The challenge is that some instruments use different measurement principles and therefore don't measure on an identical diameter basis. As a result, the process is not a simple file merge. Specifically, the APS measures an aerodynamic diameter, the SMPS measures a mobility diameter, and the UHSAS measure an optical equivalent diameter based on calibration with PSLs.

An initial review of the data was conducted in collaboration with the instrument mentors at BNL. We agreed that a logical start would be to merge the SMPS and APS data at the SGP site. Therefore, the initial goal is to convert the APS aerodynamic diameter to a mobility diameter and re-bin the APS data to the SMPS bin widths. We will largely follow the Beddows et al method described in the attached manuscript.

Background: Accurate simulation of aerosols, clouds, radiation, and precipitation is an ongoing challenge for current state-of-art climate models. Developing and implementing metrics for aerosol-cloud interactions (ACIs) in the current ARM-DIAGS is particularly important for boundary layer clouds and mixed-phase clouds. The University of Arizona (UA) group led by Prof. Xiquan Dong has been working on classifying and investigating aerosol properties and interactions with clouds for decades, using remote sensing and in-situ measurements, as well as model simulations. With their expertise in the field of ACI and experience in retrieving and processing ARM data, in this collaboration work, we propose to implement standardized analysis tools and associated ARM-site data sets for ACI metrics into ARM-DIAGS, as a new set of process-oriented diagnostics suite.

Tasks: The development and implementation of the ACI metrics into current infrastructure of ARM-DIAGS involves following tasks: • Write Python scripts to pre-process variables from raw ARM instrument data required to generate the necessary statistics. • Create an observationally based database for ARM-DIAGS by following the data format defined by CMIP conventions • Write Python modules to compute performance metrics for CMIP models

The UA group will work closely with the LLNL ARM team to interface the data and Python modules with existing ARM-DIAGS framework and document the data products and the diagnostics package at the end of the funding period.

Timelines: October 1, 2021: Project starts. March 31, 2022: Complete Phase 1: Implement basic metrics on ACI, which includes the climatology of cloud and aerosol properties, and the magnitude of an index describing the ACI process. Preprocess variables such as the aerosol optical depth (AOD), surface cloud condensation nuclei (CCN), cloud-top height, cloud-base height, cloud fraction etc., which are required for the metrics from various data sources collected at multiple ARM sites. September 30, 2022: Complete Phase 2: Implement process-oriented diagnostics on ACI at selected ARM sites, such as SGP, ENA and NSA.

This ENG encompasses the LASSO work related to CACTI during fiscal year 2022. Each milestone will be included as sub-component of this ENG.

This ENG encompasses tasks related to LASSO-ENA work in fiscal year 2022. Each milestone will be included as sub-component of this ENG.