Earth system modelling – and particularly weather forecasting, is in the middle of a disruptive event. The last two years have seen deep learning forecasting methods go from being a promising idea, to keeping up with state-of-the-art approaches to weather forecasting. The new models might be computationally expensive to train, but can be hundreds or even thousands of times less expensive at the time a forecast is run.

There’s huge potential in developing the methods for climate timescales. The savings in computational expense could lead to a lowering of barriers to access to climate modelling and climate projection data. It could lead to massive advances in uncertainty quantification, downscaling, or even the provision of highly tailored climate impacts information.

But weather and climate are not the same – weather is primarily an initial value problem, and climate is a boundary value problem (albeit with higher systemic uncertainty). There are huge, important parts of the climate system (the deep ocean, the future) where we don’t have much (or any) data. The approaches to applying deep learning methods will necessarily be very different, and the widespread use of more traditional process models will surely linger longer.

I’m really keen to get into this field – there is a huge overlap with the emulation and UQ work I’ve been doing these last 15 years. I’m excited to be running the office’s scoping of data-driven models of the climate. As such, here are some of the foundational papers that I think are really important in this space.

Models/emulators

ACE is the AI2 Climate emulator. Its a Spherical Fourier Neural Operator (SFNO)
architecture neural net that emulates the output of a climate model, at around 100 times less computational cost.

Aurora is a transformer-based foundation model. That means it’s a pre-trained 3D model of the atmosphere, that can be fine-tuned using data applicable to a particular problem. They use high resolution and atmospheric chemistry data in the paper to make a high-res forecast and a global air pollution forecast.

ClimaX is another vision transformer-based foundation model. This one is used to make some subseasonal-to-seaonal predictions, and to reproduce the outputs of ClimateBench.

NeuralGCM is a really interesting hybrid: a traditional dynamical core with a machine-learned wrapper. The whole thing is written in JAX, so it’s automatically differentiable. It’s probably the thing that will look most familiar to climate scientists as a “climate model”, but the training costs are high.

Benchmarking/data/workflows

The OG of benchmarks for ML in the climate space is ClimateBench, which is based on WeatherBench (2). ClimateBench is also great, because as well as setting out a prediction challenge and suggested training data set, it produces and tests a number of statistical and ML benchmark solutions to that prediction challenge. Since then, there has been ClimSim, ClimateLearn, ClimateSet and even things like OceanBench.

This literature is expanding massively, and very quickly, so I’m bound to have missed things in the climate sphere. In the next blog post, I’ll have a look at some useful commentaries on ML/DL and climate, and look at some survey papers that round up recent literature more broadly than I have here.

Comment if you have a interesting paper I should look at!