This weekend, Ed pointed to the paper “Retrospective Prediction of the Global Warming Slowdown in the Past Decade”, in Nature SomethingOrOther*.
Colour me slightly underwhelmed by the actual paper. The topic is important, and I’m sure they’re on to something. These things are often much more impressive if you have a real feel for the model. I’d like to see much more detail about the dynamics that the models are simulating, including some spatial patterns (are they doing the right thing for the right reasons?), and I’d like a bigger pair of ensembles for comparison (are we really sure that the models are doing the right thing?)**.
Also, how hard is it to directly label your graphs? What did blue mean again?
My biggest complaint is the language. I’m going to go right ahead and suggest that we reserve the word “prediction” for situations where you don’t actually know the answer. I have no problems at all with using the word “prediction” for past events; a natural stance in a subjective Bayesian framework. I always thought the great (Bohr?) quote – “prediction is very difficult, especially about the future” – probably makes more sense straight up, rather than in the jokey way that it is often used.
There is a fundamental difference between a prediction and a hindcast – which is the word that I would use for what they do in the paper. I would say that the team has demonstrated that the model more successfully simulates the evolving trajectory of temperatures, when initialised with the known state of the world. Hindcasts are a really good thing, and give us lots of information, but not as much information as predictions. Adding the modifier “retrospective” to “prediction” is not good enough – in fact I would say it was a contradiction in terms.
You could argue that this version of the climate model doesn’t see the “right” answer a priori, but any prediction system is a combination of model and a human components. The opportunities for subtle biases to creep in if information about the “right” answer is available are numerous. Would the paper have been published if the model didn’t get the “right” answer?
I think that the team that wrote this paper are actually calibrating the prediction system. Great, but let’s preserve special status of true predictions by reserving the word.
* ht @micefearboggis for the journal name
** UPDATE To be fair, some of this is in the supplementary information