Diverse cloud detection algorithms have been developed and applied to atmospheric lidar data to identify cloud boundaries and produce cloud masks. The general approach that most algorithms use reflects the complex physics of interactions between laser light and atmospheric particles to detect and distinguish clouds from other features (atmospheric and non-atmospheric) in the data.
This type of cloud detection algorithm was implemented to provide cloud boundaries from the Atmospheric Radiation Measurement (ARM) user facility’s micropulse lidars (MPLs) in the original Micropulse Lidar Cloud Mask value-added product (MPLCMASK VAP).
While these algorithms can identify a wide variety of cloud types and conditions, it is often observed that the algorithms can still fail to accurately detect clouds that are readily discernable when visually inspecting lidar images. Based on this observation, an alternative approach for cloud detection is to take advantage of machine learning capabilities and the trained eye as an interpreter of lidar images, and in turn, train a neural network to recognize the desired features in the lidar data.
This new approach has been applied to data from ARM MPLs to develop a machine learning model that can produce pixel-to-pixel predictions of clouds in lidar images (Cromwell and Flynn 2019). These predictions are available in ARM’s new Micropulse Lidar Cloud Mask Machine Learning (MPLCMASKML) VAP. This VAP also gives users the cloud mask generated from the prediction output, the number of cloud layers, and the cloud layer boundaries.
The new VAP uses the model to produce a cloud mask from the MPL attenuated backscatter and linear depolarization ratio. MPLCMASKML reduces cloud misclassification by almost half compared with the MPLCMASK cloud mask when evaluated against hand-labeled cloud masks created by a lidar expert.
MPLCMASKML can capture clouds below 500 meters and prevent cloud layer merging. In addition, the new VAP captures finer details of the cloud layers.
Evaluation data from the new MPLCMASKML VAP are now available for the following ARM sites and date ranges:
- the Southern Great Plains Central Facility near Lamont, Oklahoma, from January 1, 2013, to May 31, 2021
- the Eastern North Atlantic atmospheric observatory from October 3, 2013, to May 31, 2021
- the North Slope of Alaska atmospheric observatory at Utqiaġvik (formerly Barrow) from January 1, 2015, to December 23, 2017
- Oliktok Point, Alaska, from September 13, 2013, to May 31, 2021
- McMurdo Station, Antarctica, from December 1, 2015, to January 2, 2017, during the ARM West Antarctic Radiation Experiment (AWARE)
- Córdoba, Argentina, from September 27, 2018, to April 30, 2019, during the Cloud, Aerosol, and Complex Terrain Interactions (CACTI) deployment.
Future data will come from the 2019–2020 Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition.
Eventually, all available ARM fast-switching polarized MPL data will be processed with MPLCMASKML.
More information about MPLCMASKML is available on the VAP web page.
To share your experience using the evaluation data or to ask questions, contact ARM translator Damao Zhang, assistant translator Donna Flynn, or VAP developer Erol Cromwell.
Access the data in the ARM Data Center. (Go here to create an account to download the data.)
To cite the MPLCMASKML data, please use doi:10.5439/1637940.
Reference: Cromwell E and D Flynn. 2019. “Lidar Cloud Detection With Fully Convolutional Networks.” In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), 619-627, doi:10.1109/WACV.2019.00071.
# # #ARM is a DOE Office of Science user facility operated by nine DOE national laboratories.