Hannah Christensen (she/her)

Abstract:

Abstract Subgrid machine‐learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher‐resolution physics without incurring the prohibitive computational cost associated with more explicit physics‐based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long‐term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low‐resolution real‐geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture‐agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle‐inspired architectures achieve state‐of‐the‐art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics‐AI climate simulation. Plain Language Summary Future climate models may use machine learning (ML) to replace small‐scale physical processes that are otherwise too costly to simulate directly over long timescales. Such “hybrid” physics–ML models could improve predictions by reducing uncertainties from current approximations. But making them run reliably in full climate simulations has been a major challenge. To speed progress, scientists created an open data set, benchmarking framework, and global competition to drive improvement for these ML components. This paper follows up on that competition by testing ideas from the winning teams within hybrid climate models. For the first time, we show that stable hybrid simulation is now reproducible across a range of diverse ML architectures. We find that different architectures share similar patterns of errors both before and after coupling, although their responses to added training inputs can differ. Finally, some competition‐inspired designs achieve state‐of‐the‐art scores on individual performance measures, but no single approach beats the previous benchmark (Hu et al., 2025, https://doi.org/10.1029/2024ms004618 ) on every metric. Key Points Online stability in the low‐resolution real‐geography setting is reproducibly achievable across diverse architectures Offline and online zonal mean biases are near‐identical across architectures; online runs underestimate tropical precipitable water An expanded variable list is universally beneficial offline but has diverging, architecture‐dependent effects online

Abstract:

Abstract. Fluvial floods pose severe socioeconomic and environmental risks and are projected to change in frequency and severity in future decades. Estimating the magnitude of extreme floods remains challenging, particularly for sparse tail events. This motivates the need to identify predictors across catchments and time. Synoptic-scale weather patterns (WPs) are often more temporally persistent and predictable than local meteorological variables, such as precipitation. However, the value of weather patterns as predictors for flood magnitude estimation is not well established. This study introduces a feature incorporation machine learning framework to quantify the relative contribution of synoptic, meteorological, and catchment controls on winter peak-over-threshold (POT) flood magnitudes (≥99th percentile) in near-natural catchments across the United Kingdom (UK) benchmark network. We train Random Forest regression models for a pooled national sample and for multiple hydro-climatic regional samples. Model interpretability was examined using Shapley Additive Explanations (SHAP). Additionally, we analyze the conditional probabilities of the WPs co-occurring with flood magnitudes. Our results show that WPs associated with cyclonic low-pressure systems frequently coincide with flood magnitudes but add minimal value to their estimation. Model skill is dominated by static catchment attributes such as aridity and event-day precipitation in the UK model, with regional model variability in feature importance reflecting hydro-climatic contrasts. Our findings highlight the variability in model outcomes depending on the model structure and the choice of features. This study also offers methodological guidance for developing large-sample machine learning models for flood estimation that integrate atmospheric predictors with traditional hydro-meteorological and geographical variables across a feature incorporation framework.

Abstract:

Machine-learnt weather prediction (MLWP) models are now well established as being competitive with conventional numerical weather prediction (NWP) models in the medium range. However, there is still much uncertainty as to how this performance extends to longer timescales, where interactions with slower components of the earth system become important. We take GenCast, a state-of-the-art probabilistic MLWP model, and apply it to the task of seasonal forecasting with prescribed sea surface temperature (SST), by providing anomalies persisted over climatology (GenCast-Persisted) or forcing with observed SSTs (GenCastForced). The forecasts are compared to the European Centre for Medium-Range Weather Forecasts seasonal forecasting system, SEAS5. Our results indicate that, despite being trained at short timescales, GenCast-Persisted produces much of the correct precipitation patterns in response to El Ni˜no and La Ni˜na events, with several erroneous patterns in GenCast-Persisted corrected with GenCast-Forced. The uncertainty in precipitation response, as represented by the ensemble, compares favourably to SEAS5. Whilst SEAS5 achieves superior skill in the tropics for 2-metre temperature and mean sea level pressure (MSLP), GenCast-Persisted achieves higher skill in some areas in higher latitudes, including mountainous areas, with notable improvements for MSLP in particular; this is reflected in a slightly higher correlation with the observed NAO index. Reliability diagrams indicate that GenCast-Persisted has little skill relative to climatology, whilst GenCast-Forced produces forecasts with reliability comparable to SEAS5. These results provide an indication of the potential of MLWP models similar to GenCast for the ‘full’ seasonal forecasting problem, where the atmospheric model is coupled to ocean, land and cryosphere models.

Abstract:

Data driven parameterisations for sub-grid processes unlocks the ability to surpass the current computational constraints of Earth system models. However, machine learning (ML) can be brittle. State-of-the-art ML approaches can reliably perform on in-distribution data, exceeding human ability across a diverse range of tasks. Yet, when faced with shifts in data distribution, performance degrades. In climate modelling, when the task is predicting the state of a non-stationary system, this is evidently a substantial issue. We illustrate this with the ClimSim dataset, forming spatio-temporal groups and quantitatively show how even small shifts in distribution affect performance.Next, we use the theory of compositional generalisation to build models which are less susceptible to these shifts in distribution. Compositional generalisation is the formation of novel combinations of observed elementary components. That is, the ability to decompose data into building blocks that are reused across both the in- and shifted-domains, such that a model can capture a domain shifted state through a set of in-domain, learnt abstractions. Inspired by these concepts we propose various architectural and regularisation changes to standard ML parameterisations to improve generalisation. Preliminary results in sub-grid process emulators suggest new insights into if and how CG can reduce model sensitivity to domain shifts.