ARM Priorities

ArcticShark SGP Routine Flights over SGP

After getting approval of the IMB, AAF is embarking on routine flights over the SGP site on FY23. It is expected that we will deploy 3 times during FY23 namely in March, June and August 2023.

ADC Infrastructure Upgrades and Future Planning

ADC Infrastructure Upgrades and Migration Paths

The infrastructure at the ADC includes systems and applications from both the old Archive, Data Management Facility, and External Data Center. These systems, since they're relocation to ORNL, have remained separated (by network, systems management procedures, patching, etc.) and makes the operations/management of these systems challenging . The goal of this ENG is to outline paths forward for providing maximum consistency in systems management, patching, naming, etc. as possible and increasing system stability, accessibility, and availability across the ADC which is expected to greatly improve the overall operational efficiency. The goal in this is to configure the infrastructure in such a way that there are clear upgrade paths and consistent expectations for users, developers, and administrators. This is an overarching ENG and items below will be child ENGs. Stakeholder requirements will be gathered and potential impacts to operations will be documented.

1. Isolation of development, staging, and production environments as well as public and back-end systems

Currently, many systems are able to communicate with hosts in what should be different environments (e.g. development systems/applications/databases can talk to development and prod systems/applications/databases). From both an operations and cyber security perspective, it is best practice to isolate each of these environments from one another. By doing so, we will remove the possibility of production systems/applications/data being affected by staging/development systems. This will also allow us to implement better testing/release policies as they can be tested on staging systems prior to being released to production. Formal policies will have to be developed with input from various stakeholders with regards to data replication/synchronization from production to staging and development (e.g. how often will the staging database be sync'd from production?). It is also desirable to ensure that public facing systems are segregated from those that are not. A strategy for isolating zones and policies for replicating data/information will be presented in a design review. Once this is in place, we will have clearly defined policies and procedures for testing and releasing applications.

2. Centralized system management and automation with Ansible

RedHat 8 has been released. RH8 and Ansible are very tightly integrated. It is necessary to plan out Ansible playbooks for all areas of the ADC (web servers, processing nodes, operations systems, etc.). Once these playbooks have been developed, a design review will be held prior to them being deployed. This will greatly simply management of the systems and ensure consistent configurations across all systems.

3. Utilize centralized ARM LDAP authentication for system level access

Currently, no hosts utilize LDAP directly for authentication. Accounts are synchronized to the systems on a schedule. It would be prudent to update the systems to utilize ARM LDAP as their source of authentication. This will allow us to implement RBAC (Role Based Access Control) by permitting access to hosts based on LDAP group memberships. This will also require us to implement password expirations, which are recommended. This will affect the configuration of the LDAP servers. Of course any changes will be fully tested by stakeholders prior to being moved into production.

4. Patch/Update management

Currently, RH Satellite is used to push updates to the systems that were part of the Archive. Updates are still manually run on the systems inherited from the DMF and XDC. It is desirable to have all systems managed by a single patch/update management system, such as RH Satellite or Katello. Updates should first be pushed out to and tested on dev and staging systems. Once it has been confirmed that deploying updates to production will not cause outages, then they would be deployed accordingly. A design review will be held to outline the schedules and expectations for deploying system and application updates. This will allow for a consistent patching strategy across all systems.

5. Implementation of consistent naming strategy (e.g. get rid of xdc.arm.gov, archive.arm.gov, dmf.arm.gov, domains)

There are a number of hosts that still contain the dmf.arm.gov, xdc.arm.gov, and archive.arm.gov domains in their name. These will be renamed to adc.arm.gov. The old addresses will continue to be active for a while to help with the transition of processes that are currently using these names.

The implementation strategy will include: items 1, 3, and 5 to be done concurrently. I would recommend waiting on 2 and 4 until 1 is being deployed. They are tightly coupled. If we move forward on 2 and 4 too soon, we may have to repeat effort to make them work as desired after item 1 is complete.

Development of a new Calibration System Phase I

The current calibration database is housed in OSS which is being replaced by the asset management tool. OSS will remain until we have a new calibration system. Requirements have been gathered through 3 feedback sessions with ARM mentors, site operations, and infrastructure in December along with information solicited from science users through email and discussions with a UEC subgroup. The detailed requirements are housed in the google doc with the higher level requirements mapping an priorities in the google sheet for broader review and approval

Doc: https://docs.google.com/document/d/1a6-O8O0tGsI6nJz5YKMhij2GJGrcyMWK80XC7zfx5fk/edit?usp=sharing

Sheet: https://docs.google.com/spreadsheets/d/1AylvpmP_wvOzJvDre0CFflX4dShRH-Ty72fPbq8lqOY/edit?usp=sharing

In talking with the science users, it was clear that there was a large array of needs when it comes to the dissemination of calibration information. There are three main schools of though which I think we need to account for, 1.) those that want to see the calibration records and have them delivered with their data, 2.) those that want to be able to view the records in a web-base viewer, and 3.) those that just want to know that their data are calibrated through a mechanism in the data files. 1 and 2 will be addressed in phase 1 and 3 will require a larger effort that includes modifications to the ingest processes in phase 2.

ARM Data Workbench Phase 2

This ENG is the second of three ENGs to support the release of the ARM Data Workbench (ADW) identified in the ARM decadal vision and based on the requirements gathered during phase I. The ARM data, computing, and software ecosystem is designed to offer users a unified and seamless access experience, facilitating data analysis with ARM and externally collected data. The purpose of this effort is to develop additional capabilities and services to address the requirements along with providing interfaces to interact with recently deployed ADW capabilities. Data Discovery will be extended to provide support for conditional querying, managing subsets of data and plotting using the ADW ecosystem. This phase will include the development of these capabilities along with refining the guidelines for how to contribute/integrate with ADW. The attached diagram shows our planned activity for Phase II and Phase III.

This ENG will have the following tasks:

- Develop mocks of how this new UI would look and integrate into Data Discovery.

- Develop and integrate a user interface extension into Data Discovery that provides a seamless transition from finding data to analyzing data.

- Develop an interface for managing subsets of data and DOIs.

- Evaluate various backend data solutions to identify the best solution to support the ADW requirements that were collected from the users and stakeholders.

- Implement the chosen backend data solution and preload the most used data as a starting point.

- Develop pipelines and workflows to stage and retrieve data to/from the chosen data solution.

- Develop new services and integrate existing ones to support needed capabilities. This is how many of the requirements will be integrated. Many requirements will need a corresponding service developed and then be integrated into the rest of the ADW.

- Refine the integration/contribution guidelines for other software packages to be included in the ADW.

- Develop the ARM Data Workbench hub/landing page where users can find all resources/services related to the ARM Data Workbench.

- Add beta user support to Data Discovery

- Design a workflow to develop or contribute Jupyter notebook templates for popular ARM Datastreams. These Jupyter notebooks will be managed and accessed via the arm git repository (ENG0004668) and will be consumed within the ADW.

This effort will be split into incremental releases. The functionality will be released to beta users as it becomes available and we will gather continuous feedback throughout the development. The entire feature set of the second phase of the ARM Data Workbench will likely be released throughout FY24 into FY25.

AMF3 BNF: Developing a Network Node for Spatially-Distributed Aerosol Measurements for Deployment of the AMF3 to the Southeast United States

Within a global climate grid cell, aerosols exhibit high spatial variability due to their disparate sources, short atmospheric lifetimes, strong coupling to the land-surface heterogeneity via surface heat, mass, and momentum fluxes, as well as the nonlinear, interdependent physical and chemical processes that control their properties. Land-surface controls on aerosol sources (e.g., vegetative BVOC emissions, anthropogenic SO2/H2SO4 emissions), sinks (e.g., wet/dry deposition), and transformations (e.g., aerosol uptake of atmospheric water vapor) tend to vary significantly over short distances, leading to complex interactions between atmospheric aerosol evolution and the dominant spatial patterns of relevant surface fluxes. Aerosol advection further obscures characterization of land-surface aerosol controls due to simultaneous mixing and aerosol processing during transport. Land-surface controls on aerosol present a significant observational and modeling challenge, limiting our ability to understand and predict aerosol effects on radiation, precipitation, and thus weather and climate in heterogeneous regions, as exemplified by the agricultural, forest, rural, and urban atmospheric environments impacting the deployment of the AMF3 in northern Alabama. Historically, complex measurements of the range of aerosol properties have been made at a single point, which informs our understanding of the relationship among aerosol physical, chemical, and optical properties. To address the current knowledge gaps however, spatially distributed aerosol measurements are critically needed to understand land-surface controls on the variability in aerosol-climate impacts, particularly in the Southeast United States.

While heavily-instrumented "point" measurements (e.g., AOS) have proven immensely useful for process-level science, they may not necessarily be representative of the larger atmospheric environment. However, it will be challenging both financially and operationally to deploy an AOS network due to the costs associated with such comprehensive instrumentation, infrastructure, and labor, as well as the operational complexities associated with siting and installing an AOS (e.g., power, communications, land-use agreements). Instead, we propose to develop an aerosol measurement network that meets both the measurement requirements needed to characterize aerosol-climate impacts, as well as the operational requirements needed to support a lower-cost, lower-complexity system.

Replace current TSIs with newer technology all-sky imager

The ARM Program currently operates four all-sky imagers (visible) at three fixed sites, SGP (C1 and E42), NSA, ENA and has six additional systems that serve as spares or are designated for each of the three AMF sites. These systems are Total Sky Imager (TSI) that were manufactured by Yankee Environmental Systems and purchased between 2000 and 2013. The original systems are two different models (660 and 880) that have undergone significantly reengineering and upgrades by ARM and BNL (ENG0000378) as well as software upgrades in collaboration with NOAA (ENG0000674). The systems are aging and many of the parts have become unserviceable and/or are no-longer in production. Additionally, the TSI design relies on a rotating mirror that tracks the sun. This design element has been the source of significant maintenance and repair costs because of alignment issues as well as failure of the gear and motor systems that move the mirror.

At least two options for the replacement of the current ARM TSIs with newer, commercially available, technology have been identified and reviewed. These systems do not rely on moving parts or a large occulter to obscure the sun, but instead consist of a high-resolution camera with a 180° field of view covered by an anti-reflective coated glass dome that is housed in a weather protected enclosure. The required specification of these two systems or other similar systems will be attached to this ENG but basic requirements for the procurement include:

• Provide Primary measurement: fractional sky coverage

• Provide Secondary measurements: visible sky image, cloud decision image, daily movie of sky images, system status, opaque and thin cloud discrimination

• Visible spectrum camera CCD or CMOS imager (fisheye) with 180-degree FOV

• Imager resolution: 4-5 MP, 12 or 24-bit color

• Temporal resolution: minimal 30s, programmable for longer sampling

• The software should (1) mask the sun in processed images, (2) allow the user to manually select up to two FOVs and record simultaneously, (3) center image alignment, (4) provide clear sky calibration, and (5) set parameters to distinguish two or more cloud types.

• Size and Weight: portable and weight should not exceed 25kg (55 lbs.)

• Able to operate in a wide range of environmental conditions

Procure third generation C-band precipitation radar

The CSAPR2 was obtained to provide a deployable precipitation measurement capability (ENG0000906). The radar was originally designed to operate on a trailer and without a radome; however, as use cases were explored, it became clear that this configuration would not permit operation in kind of conditions (rain, wind) that were of most interest. Therefore, a system was reconfigured to include a radome and a platform that could support the pedestal+radome configuration. That system was first deployed to Argentina for CACTI and is currently deployed to Houston for TRACER for its second mission. Both of these campaigns have a strong focus on deep convection and associated precipitation, and the CSAPR2 is a great tool for probing those systems. The next deployment for the CSAPR2 will be an extended-term deployment to the southeast United States. This deployment is expected to last at least five years so the CSAPR2 will not be available for deployment with other AMF1 or 2 deployments for the foreseeable future.

The purpose for this ENG is to purchase a new deployable radar that can take the place of the CSAPR2 for upcoming mobile deployments and that could also potentially be deployed at the SGP or elsewhere.

AMF3 BNF: Aerosol Flux Measurements on the Tower - CPCs

As part of the AMF3 tower measurements, there is a need from the science team to measure aerosol fluxes at multiple heights. The science team has proposed deploying 4 CPCs at different heights (Top, Above, Below, 10,) to measure aerosol concentrations and combine them with 3D-winds from the TOWERMET system to produce an aerosol flux product. This aerosol flux product will be handled in a separate ENG at a later time once the process is refined. An inlet and housing for the CPC will need to be designed for implementation on the tower.

For each CPC, we will collect and process it like a normal CPC datastream to a1 and b1 levels which will result in 4 sets of datastreams with facilities of S10, S11, S12, S13.

The Center for Aerosol Measurement Science (CAMS)

Summary. Accurate measurements of atmospheric aerosol properties require thorough understanding of the science behind the measurements, consistent instrument operations, and well-defined instrument calibration procedures. A framework to coordinate the development and application of best practices in instrument operation and calibration is currently lacking in the U.S. To fill this gap, the Environmental and Climate Sciences Department at Brookhaven National Laboratory (BNL) is developing a dedicated facility, the Center for Aerosol Measurement Science (CAMS), which is envisioned as a nation-wide calibration and measurement science center for aerosol instruments, providing traceable measurement services to DOE's Atmospheric Radiation Measurement (ARM) program and its users, research organizations, and other interested parties for data quality assurance in aerosol measurements that facilitate high-quality and internally consistent measurement networks

For more details, see the attached Whitepaper.

Objectives.

● Establishing and maintaining gold-standard instrumentation (i.e., the highest order of reference instrument against which other instruments will be periodically calibrated) for key aerosol measurements (number, size, composition, hygroscopicity, optical properties),

● Providing regular, traceable calibrations for field instruments using gold-standard instrumentation,

● Developing and testing uniform instrument calibration protocols, best practices for instrument operation, and data processing algorithms to provide guidance for PIs in maintaining data uniformity across deployments and improving data quality across all ARM aerosol measurements,

● Characterizing existing and new measurement infrastructure components (e.g., individual sample inlets) or whole measurement platforms such as the AOS under controlled conditions to improve or develop necessary corrections to be applied to measurement data,

● Performing closure experiments among measurements of related aerosol properties to validate and build trust in the measurements and to develop additional data products, and

● Testing new sampling strategies (such as mitigating water condensation or particle losses) to provide an understanding of expected performance and to implement any necessary changes before deployments.

Procure SO2 instrument for the AMF3 BNF deployment

Measurements of aerosol precursors are needed at the AMF3 BNF deployment to help understand the role of new particle formation and growth as a source of boundary layer CCN. SO2 is an important precursor measurement and will complement the other planned trace gas measurements. This instrument will be deployed to the main site.

Instruments to fill in the gaps of aerosol measurements at Utqiagvik (NSA)

The NOAA Federated Aerosol Network has been performing aerosol measurements at Barrow (Utqiagvik) since 1973. The properties they have been monitoring are particle number concentration, scattering, and absorption. Other aerosol measurements in this location have been performed by the Leibniz Institute for Tropospheric Research -TROPOS (particle size distribution, fine fraction, since 2008), the National Institute of Polar Research (black carbon, since 2012), and more recently, University of California -San Diego (CCN, filter chemical analysis, and took over the TROPOS SMPS, through a 5-y NSF grant). See more details about the existing measurements in Table 1 (attached.

There are still gaps in the aerosol measurements at Utqiagvik and these are the ones the ARM AOS team proposes to address at this stage through this ENG. The gaps are for the full particle size distribution (coarse mode), aerosol chemical composition, and aerosol hygroscopicity. To fill in those gaps, the plan is to deploy the APS, ACSM, SP2, and Humidigraph at the NOAA facility, alongside the existing aerosol instruments. Please see Table 1 (attached) for the details regarding the proposed instrumentation, including the cost of the mentorships. There could be a second phase where we could also take over the SMPS and CCN measurements once the UCSD grant reaches an end, or when ARM decides that it is the right time for this.

While the instruments are deployed at the NOAA facility, we expect to contract with NOAA for day-to-day maintenance of the instruments, but also expect ARM to have access to the instruments as needed. We will periodically review the suitability of this arrangement with the expectation that at some point, we may build an independent AOS at our NSA site – but that independent AOS is beyond the scope of this ENG. The scope of this ENG is to procure the listed gap instruments, deploy them at the NOAA facility, and to establish a technical support arrangement with NOAA – but note that finally, the instruments will be fully owned and mentored by ARM.

Data Management Plan:

NOAA will allow instrument PCs and a collector system on their network. Access to the instrument systems will be permitted through inbound https from access.arm.gov. Access to the collector system will be permitted though inbound ssh access from ssh.vsn.arm.gov, portal.arm.gov, and portal2.arm.gov. The collector system will pull data via ftp from the instruments, in the same way data is collected at the measurement facilities. The collector system will transfer collected data to the ADC using the site transfer daemon (standard for ARM). Data transfer to the ADC will require bi-directional tcp/1992 between the collector and prod-transfer2.adc.arm.gov.

AMF3 BNF Procure tower and modify CSAPR2 mounting for deployment on the tower

The CSAPR2 will be deployed to AMF3 BNF. Due to the height of the tree canopy, the CSAPR2 will need to be installed on top of a tower to avoid the tree canopy. This ENG covers the design and procurement of the tower and modifications to the CSAPR2 to be able to mount it on the tower.

The CSAPR will be modified to be placed on top of a 20' tower being built by Baron. The tower will consist of 2 sets of staircases and 1 landing. See picture attached.

As suggested by Baron for the tower:

HVAC: the insulated duct work could come up the middle supported by the horizontal cross members or look at using some mini splits without ductwork. With either option, mounting provisions can be detailed on the tower.

Pricing:

PE Stamped drawings – tower steel and a foundation per TIA-222-H standard and analysis by an AL PE and construction drawings ($15-20K)

20' Tower with top plate: ~$100k (Subtract $25k if we supply the top plate)

Construction of the tower, foundation, grounding, etc would be separate. Baron estimates that a local contractor could run around $50k for the concrete foundation and they are open to discussing the remainder of the installation items.

Development of new PBL Height VAP

Planetary boundary layer (PBL) depth is important to a wide range of atmospheric processes including cloud formation, aerosol mixing and transport, and chemical mixing and transport. The CAPI working group has proposed development of an ARM PBL height VAP to make this important value available to the ARM and broader scientific community.

Numerous instruments and algorithms may be used for PBL height detection, each with their own strengths and weaknesses. We plan to implement several different algorithms for PBL height detection using different instruments as the comparisons between the different methods will provide useful information about the reliability of the PBL height retrievals. We will develop this VAP in a phased approach, in which with we start with relatively simple algorithms and successively add new instruments/algorithms. Additionally, we will initially focus on the convective daytime boundary layer, as it is more easily detected and algorithms are more reliable than for the stable nighttime boundary layer.

In the first phase of this VAP (covered by this ECR), we will focus on implementing methods for radiosondes, ceilometer, and MPL as these instruments exist at all ARM sites. Two radiosonde methods (Heffter and bulk Richardson) and three ceilometer/MPL algorithms (1D gradient, wavelet, combined 2D gradient and wavelet) have been identified. More details on these methods are given in the implementation plan. Code for these algorithms will be provided by ARM scientists, and will be implemented in ISDE. In the next stage of development, which will be defined in more detail at a later date, we will explore implementing retrievals for more advanced instrumentation (AERI, Raman Lidar, Doppler Lidar, Radar wind profilers) that do not exist at all sites.

Initial evaluation of the VAP will focus on several IOP periods: MC3E, Azores, CARES, and AMIE. As part of the VAP development, we will also develop a database of PI PBL height retrievals for these IOP periods. We do not plan to run a formal inter-comparison, but will use this database to 1) evaluate the implementation of the VAP methods, 2) identify promising methods to implement in the next state of development, 3) identify areas of uncertainty and methods of flagging PBL height retrievals, and 4) encourage science PIs to submit their retrievals as formal ARM PI products.

A draft PBL height implementation plan has been developed and is available from Sally upon request.

Update CCN Profile VAP

The goal of this ENG is to revive the CCN Profile VAP that has been developed, update it, and get it running in production. A technical report on this VAP is already available and much of the development is already done. The technical report is available at : https://www.arm.gov/capabilities/vaps/ccnprof

At the time the VAP was developed, the CCN and RL data needed for the VAP were not sufficiently high quality to generate a reliable and high-quality VAP. The data quality has dramatically improved to the point where this VAP can be re-visited.

Development of new cloud phase VAP

Cloud thermodynamic phase identification is important to understand many cloud processes such as ice particle production, precipitation formation, and cloud lifecycle evolution. The CAPI working group has proposed development of an ARM cloud phase VAP to make this important value available to the ARM and broader scientific community.

Cloud thermodynamic phase classification can be determined from remote sensing measurements. Specifically, active remote sensing instruments such as lidar and radar provide vertically resolved cloud thermodynamic phase identification. Due to different wavelengths, lidar backscattered signal is more sensitive to small particles with high number concentrations such as liquid droplets. Lidar depolarization measurements can be used to identify the nonspherical particles such as ice crystals. However, lidar signal is quickly attenuated by cloud droplets. Radar signal is dominated by large particles such as ice crystals and is susceptible to attenuation only in case of strong precipitation. In addition, microwave radiometer measurements can provide constraints on the presence of liquid layers. Since no one instrument can provide sufficient level of information to cover from small droplets to large ice particles and snow, we use the multisensor method developed by Shupe (2007) to classify cloud phase. Similar to the ARM Ka band Zenith Radar (KAZR) Active Remote Sensing of Clouds (KAZR-ARSCL) VAP, the multisensor approach uses coincident ARM Micropulse lidar (MPL) with depolarization, Ka-band ARM Zenith Radar (KAZR), and microwave radiometer measurements, together with atmospheric thermodynamic profiles from radiosonde measurements to determine cloud thermodynamic phases.

For the initial implementation, the VAP will be implemented at the ARM North Slope of Alaska (NSA) atmospheric observatory central facility at Utqiaġvik. An HSRL system was deployed and has been acquiring measurements at the NSA Utqiaġvik site since April 2011. The Ka-band ARM zenith radar (KAZR) replaced the millimeter-wavelength cloud radar system starting in May 2011, providing measurements with high temporal and vertical resolution and reduced spectral artifacts. These measurements provide high quality inputs for the VAP. Over the NSA Utqiaġvik site, persistent stratiform mixed-phase clouds are frequently observed during the transition seasons such as in spring and fall. Liquid clouds are often observed during summer while ice clouds in winter. These seasonal variations of major cloud thermodynamic phase provide a good opportunity to apply the cloud phase identification methods.

A draft cloud phase identification implementation plan has been developed and uploaded.

Merged Aerosol Size Distribution VAP

The goal of this ENG is to produce a merged size distribution VAP that will combine data from the SMPS and APS into a single, continuous size distribution. We have nearly finished the harmonization of the size distribution instruments and data are now available in a consistent format and with consistent naming conventions. There has been a longstanding request from users of the aerosol data to produce a single size distribution from multiple instruments. The challenge is that some instruments use different measurement principles and therefore don't measure on an identical diameter basis. As a result, the process is not a simple file merge. Specifically, the APS measures an aerodynamic diameter, the SMPS measures a mobility diameter, and the UHSAS measure an optical equivalent diameter based on calibration with PSLs.

An initial review of the data was conducted in collaboration with the instrument mentors at BNL. We agreed that a logical start would be to merge the SMPS and APS data at the SGP site. Therefore, the initial goal is to convert the APS aerodynamic diameter to a mobility diameter and re-bin the APS data to the SMPS bin widths. We will largely follow the Beddows et al method described in the attached manuscript.

Produce CMAC2.0 for radar data produced at GUC by the CSU radar at SAIL

Take files, both individual PPI scans and RHI scans, value add, combine and add metadata to create radar volumes suitable for users. Ensure metadata meets ARM standards. In addition monitor output from the radar and provide weekly reports using the ARM Campaign Dashboard for SAIL. Report any issues seen in a timely fashion to both CSU and ARM staff managing the contract. If needed report out to the SAIL science team.

LASSO-ENA

This ENG encompasses tasks related to LASSO-ENA work in fiscal year 2022. Each milestone will be included as sub-component of this ENG.

Develop snowfall rate (LWE) retrievals for SAIL

(note, text taken from SOW which will be attached)

As part of the SAIL field campaign ARM deployed a X-Band radar system owned and operated by Colorado State University. This system produces radar moments and dual polarimetric observations from hydrometeors and other scatterers.

The system is located approximately 8km to the southeast of the main site. Due to the complexity of microphysical habits of snowfall (dendrites, rods, plates and vast varieties of hybrids and aggregates of the aforementioned) relating radar measurements to snowfall rates (liquid equivalent rates, the rates in mm/hr if the snowflakes were all melted) is challenging. Matrosov et al 2009 nicely shows the range of uncertainties in reflectivity-based snowfall retrieval of the form Z=ASB. There are many other papers on snowfall retrieval (rarer at X-Band than, say, S-Band) our aim here is to start simple first and then add more sophisticated retrievals, including using other measurements, as we establish verification procedures.

ARSCL Unification and Upgrade

The ARSCL family of VAPs is one of the ARM Program's most downloaded products. This VAP chain and family of VAPs includes KAZR*COR, ARSCL, KAZR*ARSCL, and WACRARSCL VAPs. It is important that this product chain continue to improve in order to provide the best cloud location and characterization possible. Over the last few years, some new and useful sources of cloud data have been added to ARM's Archive. Also in recent years, ARM infrastructure has been increasingly adopting Python as the programming language of choice, as well as the open source paradigm, where appropriate. As a result of the new data sources and updated software engineering practices, the time has come for KAZRCOR, KAZR-ARSCL and WACR-ARSCL upgrades. Accordingly, the following upgrades are suggested to take full advantage of new ARM Program capabilities and advances in software engineering practices:

KAZRCOR:

-- Convert to Python

-- Accept either KAZR, (M)WACR, possibly SACR VTP scans as input

-- Improve computational efficiency

-- Open source appropriate portions

ARSCL:

-- Convert to Python

-- Accept corrected and/or calibrated KAZR, (M)WACR, possibly SACR VTP scans as input

-- Add additional data sources: MPLCMASKML, RWP observations, COGS where available

-- Explore use of spectra-based decluttering techniques

-- Eliminate manual clutter height review

-- Open source appropriate portions

The above is only a rough outline of the proposed project. Please see attached overview document for additional details.

Development of aerosol vertical profile retrievals using lidar measurements

Aerosol microphysical properties, especially their vertical distributions, are important to understand aerosol radiative effects and aerosol-cloud-interactions, which represents the largest uncertainty in future climate projections according to the IPCC report. To make this important information available to the ARM and scientific communities, the CAPI working group proposes to develop an aerosol microphysics profile VAP.

While passive remote sensing can be used to retrieve column-averaged microphysical properties, lidar measurements can provide retrievals of vertically-resolved aerosol microphysical properties. By combining lidar-measured aerosol extinction at both 355 and 532 nm and backscatter at 355, 532, and 1064 nm (3β+2α), it is possible to retrieve profiles of aerosol microphysical properties, such as effective radius and concentration, as shown by Burton et al. (2016) and Ferrare et al. (2017). We plan to use the Tikhonov Advanced Regularization Algorithm (TiARA) developed by the NASA LaRC group, which has been successfully applied in previous studies (Ferrare et al., 2017; Müller et al. 2019), to retrieve vertically-resolved aerosol microphysics. The input data for TiARA will be particle backscatter coefficient profiles at 532 nm and 1064 nm and extinction coefficient profiles at 532 nm from the two-wavelengths high-spectral-resolution lidar (HSRL), as well as particle backscatter and extinction coefficient profiles at 355 nm from the Raman lidar RLPROF-FEX VAP. The retrieved microphysical properties will include aerosol effective radius, number concentration, surface area, volume concentration, and fine-mode fraction.

To implement the VAP, we will start at the ARM Southern Great Plains (SGP) atmospheric observatory in Lamont, Oklahoma. The TiARA algorithm was developed using data from the Combined HSRL and Raman lidar Measurement Study (CHARMS) IOP, which took place from July to September 2015 at SGP. Initially, we will set up and test the retrieval system using CHARMS measurements and compare the results with those of Ferrare et al. (2017) to ensure proper performance of the algorithm. Once the initial implementation and testing are successful, we will apply the retrieval to other time periods when both HSRL and Raman lidar measurements are available, such as during August 2020 to May 2021 at SGP. Additionally, starting in September 2023, ARM plans to deploy AMF3 to the Southeastern US, where collocated HSRL and Raman lidar measurements over forest canopy will provide a good opportunity to apply the TiARA retrieval method.

An implementation plan will be developed and uploaded.

References

Burton, S. P., Chemyakin, E., Liu, X., Knobelspiesse, K., Stamnes, S., Sawamura, P., et al. (2016). Information content and sensitivity of the 3β + 2α lidar measurement system for aerosol microphysical retrievals. Atmospheric Measurement Techniques, 9(11), 5555–5574. https://doi.org/10.5194/amt-9-5555-2016

Ferrare, R., Thorsen, T., Clayton, M., Muller, D., Chemyakin, E., Burton, S., et al. (2017). Vertically Resolved Retrievals of Aerosol Concentrations and Effective Radii from the DOE Combined HSRL and Raman lidar Measurement Study (CHARMS) Merged High-Spectral-Resolution Lidar-Raman Lidar Data Set.

Müller, D., Chemyakin, E., Kolgotin, A., Ferrare, R. A., Hostetler, C. A., & Romanov, A. (2019). Automated, unsupervised inversion of multiwavelength lidar data with TiARA: assessment of retrieval performance of microphysical parameters using simulated data. Applied Optics, 58(18), 4981. https://doi.org/10.1364/ao.58.004981