From Numbers to Data: Advancing Calibration and Uncertainty Communication
Editor’s note: In 2021, the ARM User Executive Committee (UEC) created four subgroups to continue previous work and start new efforts to broaden community outreach. In a four-part series of blogs, each subgroup chair will introduce their subgroup, share their subgroup’s ideas and recommendations to ARM, and ways in which the ARM community can engage with subgroup members. Here, Daniel Feldman introduces the Measurement Uncertainty and Communicating Calibrations subgroup.
One of the most unsung features of the ARM community is its attention to the details, big and small, that enable scientists far and wide to search, order, and download ARM data, and develop insights with the petabytes (and growing by the day!) of data that ARM collects.
With observations from the ends of the Earth and many places in between, from hundreds of instruments and thousands of datastreams, ARM data have the potential to reveal the processes—at microscopic and macroscopic levels—that underpin how and why the shifty fluid known as our Earth’s atmosphere changes in space and time. Together, the observations coupled with follow-on advanced analyses provide benchmarks to the modeling community of behaviors they need to reproduce to achieve accurate weather and climate predictions.
Countless scientists have sat down at their workstations, run the numbers comparing ARM observations with model results, and found that the two data sets simply don’t agree. There’s a conflict that must be settled by the scientist and/or the scientific community.
But who’s right, and who needs to pour some more coffee? Who goes back out to the field to make a better measurement, or who gets back to the whiteboard or the computer and builds a better model? Deciding who is right in this age-old conflict is exactly why the details matter so much. While observationalists and modelers have been duking it out throughout the history of science and collaboratively advanced scientific understanding and predictions along the way, evaluation and improvement of models are only as good as the accuracy of measurements, and the path toward ensuring that measurements are robust is arduous.
The mundane details of how instruments are calibrated against known standards, how uncertainty in a measurement is estimated and verified, how the theory of an instrument’s data collection interfaces with the practice of making measurements in the uncontrolled conditions of the field, and how a stream of numbers coming out of an instrument is turned into physically interpretable data … getting these details right allows scientists to tell the modeler that a coffee refill may be in order and that a better model is possible.
The User Executive Committee Subgroup on Measurement Uncertainty and Communicating Calibrations is an all-volunteer effort of UEC members who are passionate about ensuring that ARM provides and communicates these details to the scientific community. The subgroup recognized that it is especially important to organize and communicate measurement calibrations and uncertainty estimates, both for the community of users familiar with ARM data and those who are new to them.
It’s also a lot easier said than done. With such a wide range of dizzyingly high-tech instruments, the general principles of how one should estimate uncertainty and communicate the calibration procedures in a perfect world are quickly confronted by the real, messy world, where the perfect cannot be the enemy of the good.
The subgroup therefore met monthly for over a year, communicated interim findings to the entire UEC, and developed a set of recommendations that vary in implementation difficulty and/or effort level, some of which seek to pick off low-hanging fruit and some of which swing for the fences.
Recommendations to ARM
Streamlining communication, promoting publications on instruments, and surveying community needs (both inside and outside of ARM) are some of the low-hanging fruit that the subgroup identified, because it recognized that the strategies, approaches, and needs of today may not reflect those of tomorrow.
On the more ambitious side, expanding the community of ARM science translators and enabling more frequent calibrations by expanding the pool of people who can perform such calibrations would increase confidence in ARM’s data performance. This would enhance adoption of the data by the modeling community and increase the impact of those data on the model development cycle.
The subgroup is excited to continue working with ARM to support scientists as they face the model-observation disagreement question. The subgroup and the UEC recognize that when ARM helps scientists move from measurement-model disagreement to a deeper insight into the cause(s) of the disagreement, that’s when scientific progress happens, and when ARM’s updated Decadal Vision is realized. By clearly establishing measurement uncertainty and calibration approaches, ARM data have been and can be the standard to which models can be compared.
Zooming out, the subgroup also wants to enable novel and unexpected uses of ARM data. When ARM ensures that uncertainty and calibration are well characterized, the future scientific community can use ARM data to advance science in ways that the subgroup, UEC, and even the current community can scarcely imagine.
ARM Community Engagement
This subgroup is interested in hearing from the ARM data user community about their experiences with measurement uncertainty and/or instrument calibration. Please reach out to us! The subgroup looks forward to communicating findings from the large community to the entire UEC.