In response to community feedback, ARM is committed to making its calibration processes more transparent
The Atmospheric Radiation Measurement (ARM) user facility has a robust calibration ecosystem for its instruments. Regular calibration to established standards or reference instruments and ongoing calibration checks (verifications) help ARM ensure that it is providing the highest-quality measurements possible for scientists to conduct their research.
Within ARM, responsibility for supervising calibration protocols and maintaining related documentation lies with a group of scientists and engineers known as instrument mentors. In total, mentors oversee the operations of about 430 ground-based and aerial instrument systems around ARM.
Did you know?
ARM instrument mentors oversee the operations of about 430 ground-based and aerial instrument systems around the user facility.
ARM mentors have calibration plans for their instruments that they along with site operations personnel carry out at defined intervals. Site operations staff also perform regular instrument checks to determine if calibrations are warranted. If an instrument needs to be calibrated, the mentor, site operations staff, or vendor might do the work, depending on the instrument.
Anyone interested in learning more about the calibration process for a specific ARM instrument can check out the Calibration section in the instrument handbook or contact the mentor(s) assigned to that instrument.
In response to feedback from its 2020 Triennial Review, ARM is working to make calibration information more openly available to people who want to better understand how and when calibrations and calibration checks are being performed.
“Giving the community confidence that our data is reliable, that is the goal,” says ARM Director Jim Mather.
One Stop for Calibration
Calibration is going on all the time across ARM, but it can be a logistical challenge to check and calibrate groups of instruments that are spread out around the world.
Having a single location for some calibrations can be helpful, which is the case for ARM’s radiometer calibrations.
The time between mid-April and the end of August every year is the Broadband Outdoor Radiometer Calibration (BORCAL) season at ARM’s Southern Great Plains (SGP) atmospheric observatory in Oklahoma. During this period, all of the user facility’s pyrheliometers, pyranometers, and pyrgeometers come to the SGP for calibration activities to help ensure that ARM’s broadband solar radiation measurements remain accurate and traceable to the International System of Units through the World Radiometric Reference for shortwave radiation. Correspondingly, infrared radiation measurements are traceable to the interim World Infrared Standard Group.
There is even a dedicated building for this calibration work: the Radiometer Calibration Facility (RCF), which has been around for more than 20 years.
During the BORCAL season, SGP technicians set up over 100 radiometers at a time on the RCF roof to collect data for ARM’s broadband radiometer mentors at the National Renewable Energy Laboratory in Colorado. Once the mentors receive the data set, they will analyze it to determine if there are enough data to conclude the BORCAL event. They will then inform the technicians about which of the radiometers have been successfully calibrated and can be removed from the roof as well as which, if any, need to stay for the next session. In addition, the mentors will produce a calibration report and calibration stickers, which technicians will attach to each radiometer. Then another 100-plus radiometers are set up for the next event, and the process is repeated until all of ARM’s broadband radiometers have been calibrated.
“We need a calibration center here in the United States. We have our calibration protocols, but there is no facility here with dedicated reference instruments against which others can be calibrated.”
Aerosol instruments will soon have their own dedicated facility for calibration as well.
Aerosol mentors will calibrate ARM aerosol instruments to reference instruments at the new Center for Aerosol Measurement Science (CAMS), which is located at Brookhaven National Laboratory in New York.
CAMS Director Olga Mayol-Bracero is an atmospheric scientist at Brookhaven Lab and the lead mentor for ARM’s ground-based Aerosol Observing Systems (AOS).
“We need a calibration center here in the United States,” said Mayol-Bracero in a 2023 Brookhaven Lab article about CAMS. “We have our calibration protocols, but there is no facility here with dedicated reference instruments against which others can be calibrated.”
CAMS scientists will regularly check the center’s instruments against reference instruments located in Leipzig, Germany, at the World Calibration Centre for Aerosol Physics.
Information on calibration plans for the aerosol instruments at each ARM site can be found in the ARM Fiscal Year 2024 (FY2024) Aerosol Operations Plan.
Fixed Versus Mobile Instruments
ARM operates three fixed-location atmospheric observatories in Oklahoma, Alaska, and the Azores, as well as three mobile facilities.
When it comes to calibration, ARM Mobile Facilities have considerations that are different from fixed sites because the instruments move around the world periodically and operate in varying climate regimes. Instruments are calibrated before, after, and sometimes during a mobile facility deployment, depending on the instrument.
For instruments such as those in the AOS, mentors have typically traveled to the deployment site to set up and calibrate their instruments. ARM’s current aerosol operations plan documents the FY2024 timelines for AOS calibrations.
Should an instrument need to go out for calibration during a field campaign, some instruments have spares onsite that can rotate into operations. If a spare is unavailable, ARM staff will coordinate to make sure they can send out the instrument for calibration with as little impact to the campaign as possible.
While some calibrations are relatively straightforward, it can be more complex for instruments such as radars.
For example, ARM radar engineers calibrate radars onsite before a campaign to make sure that measurements are accurate and repeatable. This process includes comparing data from the radars and cross-calibrating them.
During a campaign, a radar component can drift and cause offsets in the data, such as in reflectivity. Radars are set to perform internal calibration measurements at least once a day, depending on the instrument. ARM radar data mentors and the Data Quality Office regularly monitor the data, making corrections immediately if needed.
After the campaign, ARM radar data mentors cross-compare the data from the radars and other instruments to calculate final offsets. These offsets are applied in a post-processed data set—“b1-level” in ARM parlance, signifying that it has undergone calibrations, corrections, and quality control processes beyond ARM’s standard quality checks and corrections.
Some campaigns can present challenges during the b1-level data processing because of their location or environmental factors.
In 2019 and 2020, ARM radars operated on a research icebreaker in the central Arctic during the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition. Because of the campaign’s remote nature, there were limited options for intercomparisons or absolute calibration of the ARM radars.
After discovering a large difference in reflectivity between the Ka-Band ARM Zenith Radar (KAZR), Marine W-Band Cloud Radar (MWACR), and Ka-Band ARM Scanning Cloud Radar (KASACR), the data team compared the MOSAiC KAZR general mode (GE) data with wintertime data from the KAZR at ARM’s North Slope of Alaska observatory. The reflectivities between the two KAZRs were found to be similar to within a few decibels. This led the data team to use the KAZR GE data as the relative truth for the reflectivity offsets of the MWACR and KASACR.
The technical report on the MOSAiC b1-level radar data processing is available on ARM.gov.
Help to Lighten the Load
The beginnings and endings of ARM Mobile Facility deployments are always busy times for ARM’s instrument mentors, but FY2024 is proving to be particularly eventful. Mentors are working through calibration processes for three deployments at once while continuing to monitor instruments at fixed sites.
Calibrations are underway for several instruments that operated in La Jolla, California, during the Eastern Pacific Cloud Aerosol Precipitation Experiment (EPCAPE). A small set of instruments is being calibrated in Tasmania for the Cloud And Precipitation Experiment at kennaook (CAPE-k). Some mentors also will go to Alabama to make sure instruments are correctly installed and properly calibrated before the opening of ARM’s Bankhead National Forest (BNF) atmospheric observatory.
The mentors are getting some help from onsite colleagues, who play an important role in the calibration of many instruments.
“Over the past few years, we’ve been working toward training site operations (staff) more and more on how to do these calibrations to lessen the burden on the mentor team, but also to give them more insight and responsibility and oversight of the instruments,” says ARM Instrument Operations Manager Adam Theisen.
To that end, CAPE-k site operators recently visited CAMS, where they received training from AOS mentors on calibration and maintenance for three aerosol instruments that will be part of the campaign. The AOS team plans similar training sessions this year for site operators supporting the SGP, BNF, and upcoming Coast-Urban-Rural Atmospheric Gradient Experiment (CoURAGE) in and around Baltimore, Maryland.
Improving Calibration Resources
ARM is listening to input on how best to communicate and present its calibration activities to the scientific community.
ARM’s User Executive Committee (UEC) has a subgroup focused on improving the communication of measurement uncertainty and instrument calibration information to users. The subgroup’s recommendations to ARM during the 2021–2022 UEC term included streamlining communication, promoting instrument publications, and surveying community needs inside and outside of ARM.
During a UEC workshop in October 2023, the subgroup’s current chair, Maria Zawadowicz from Brookhaven Lab, presented a new set of proposed recommendations to ARM. Those recommendations are now formally included in a workshop summary report and are discussed further in this ARM blog post from Zawadowicz.
In response to the feedback from its 2020 Triennial Review, ARM intends to roll out a new public-facing system that will allow users to view instrument calibration records and mentor plans for future calibrations.
The system will also address feedback from ARM staff who want easier access to calibration information. Mentors will be able to assign calibration plans to ARM instruments, schedule calibration events, and send notifications to staff about those events.
Unlike ARM’s Common Calibration Database, released in 2010, the new system will provide greater flexibility for entering information.
“Calibration data is so diverse. It could be a single coefficient; it could be a multidimensional parameter. It’s not one size fits all. We have not had a system that could accommodate the breadth of information that we need.”
“Calibration data is so diverse,” says Mather. “It could be a single coefficient; it could be a multidimensional parameter. It’s not one size fits all. We have not had a system that could accommodate the breadth of information that we need.”
The first phase of the system has been released for mentors to enter their calibration plans.
ARM expects to populate the database with calibration records and engage with users about the new system by the end of FY2024.
ARM also plans to make calibration information available to users through ARM’s Data Discovery browser and as part of data downloads.
“Hopefully, it will give users more confidence once they see these actual calibration records along the way and that they can see it with their data when things are calibrated,” says Theisen.
To make its calibration processes even more transparent to the scientific community, ARM will begin a periodic series of blog posts by instrument mentors. In these blogs, mentors will share more about the calibration processes for their instruments.
# # #ARM is a DOE Office of Science user facility operated by nine DOE national laboratories.