ClimateBenchPress (v1.0): a benchmark for lossy compression of climate data
Abstract:
<jats:p>Abstract. The rapidly growing volume of weather and climate data, both from models and observations, is increasing the pressure on data centers, restricting scientific analysis, and data distribution. For example, kilometre-scale climate models can generate petabytes of data per simulated month, making it generally infeasible to store all output. To address this challenge, numerous novel compression techniques have been proposed to ease data storage requirements. However, there exist no well-defined benchmarks for rigorously evaluating and comparing the performance of these compressors, including their impact on the data's properties. The lack of benchmarks makes it difficult to design and standardize compressors for weather and climate data, and for scientists to trust that compression errors have no significant impact on their analysis. Here, we address this gap by presenting ClimateBenchPress, a benchmark suite for lossy compression of climate data, which defines both data sets and evaluation techniques. The benchmark covers climate variables following various statistical distributions at medium to very high resolution in time and space, from both numerical models and satellite observations. To ensure a fair comparison between different compressors, each variable comes with a set of maximum error bound checks that the lossy compressors need to pass. By evaluating an initial set of baseline compressors on the benchmark, we gather practical insights for effective application of lossy compression. Our benchmark is open source and extensible: users can easily add new compressors, data sources, and evaluation metrics depending on their own specific use cases.</jats:p>From stable online coupling to decade-long climate simulations: A machine learning parameterization for cloud microphysics in ICON
CloudFlow: a flow matching model to generate high-resolution cloud structures
Abstract:
Our limited understanding of clouds is the dominant source of uncertainty in future climate predictions. Understanding how changing atmospheric environmental conditions constrain cloud organisational patterns and their radiative effects is key to understanding their impact on future climate change. We present CloudFlow, a flow matching model that is able to generate high-resolution cloud structures conditioned on coarse-scale atmospheric conditions. Our model generates realistic cloud structures that match the spectra and distributions of the original high-resolution scenes. CloudFlow introduces a new modeling regime to study how atmospheric environmental conditions impact cloud morphologies which will contribute to an improved understanding of cloud feedbacks, the cloud response to a changing climate and its effect on climate itself.