Skill and uncertainty in climate models

Opinion

Julia C. Hargreaves

Published Online: Jun 24 2010

DOI: 10.1002/wcc.58

How to cite this article
Download citation
Contact the editor about this article
Authors: submit updates and notes
Comment on this article
Share

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Analyses of skill are widely used for assessing weather predictions, but the time scale and lack of validation data mean that it is not generally possible to investigate the predictive skill of today's climate models on the multidecadal time scale. The predictions made with early climate models can, however, be analyzed, and here we show that one such forecast did have skill. It seems reasonable to expect that predictions based on today's more advanced models will be at least as skillful. In general, assessments of predictions based on today's climate models should use Bayesian methods, in which the inevitable subjective decisions are made explicit. For the AR4, the Intergovernmental Panel on Climate Change (IPCC) recommended the Bayesian paradigm for making estimates of uncertainty and probabilistic statements, and here we analyze the way in which uncertainty was actually addressed in the report. Analysis of the ensemble of general circulation models (GCMs) used in the last IPCC report suggests there is little evidence to support the popular notion that the multimodel ensemble is underdispersive, which would imply that the spread of the ensemble may be a reasonable starting point for estimating uncertainty. It is important that the field of uncertainty estimation is developed in order that the best use is made of current scientific knowledge in making predictions of future climate. At the same time, it is only by better understanding the processes and inclusion of these processes in the models, the best estimates of future climate will be closer to the truth. Copyright © 2010 John Wiley & Sons, Ltd.

Figure 1.

Forecast of Hansen et al.2 (blue line) evaluated against observational data (black). Twenty‐year trends of forecast and observations are indicated with thick lines, as is the null hypothesis of zero trend (red).

[ Normal View 31K | Magnified View 46K ]

References

1 Glickman, TS. Glossary of Meteorology. Boston, Mass: American Meteorological Society; 2000.
2 Hansen, J, Fung, I, Lacis, A, Rind, D, Lebedeff, S, Ruedy, R, Russell, G, P Stone,. Global climate changes as forecast by Goddard Institute for Space Studies three‐dimensional model. J Geophys Res‐Atmos 1988; 93(D8): 9341–9364.
3 Hansen, J, Sato, M, Ruedy, R, Lo, K, Lea, DW, Medina‐Elizade, M. Global temperature change. Proc Natl Acad Sci 2006; 103(39): 14288.
4 Rahmstorf, S, Cazenave, A, Church, JA, Hansen, JE, Keeling, RF, Parker, DE, Somerville, RCJ. Recent climate observations compared to projections. Science 2007; 316(5825): 709.
5 Brohan, P, Kennedy, JJ, Harris, I, Tett, SFB, Jones, PD. Uncertainty estimates in regional and global observed temperature changes: a new dataset from 1850. J Geophys Res 2006; 111: D12–D1206.
6 Hegerl, GC, Zwiers, FW, Braconnot, P, Gillett, NP, Luo, Y, Marengo Orsini, JA, Nicholls, N, Penner, JE, Stott, PA. %22Understanding and attributing climate change%22. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press 2007.
7 Weber, SL. The utility of earth‐system models of intermediate complexity (EMICS). WIRES Clim Change 2009; 1: 243–252.
8 Murphy, JM, Sexton, DMH, Barnett, DN, Jones, GS, Webb, MJ, Collins, M, Stainforth, DA. Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 2004; 430: 768–772.
9 Stainforth, DA, Aina, T, Christensen, C, Collins, M, Faull, N, Frame, DJ, Kettleborough, JA, Knight, S, Martin, A, Murphy, JM, et al. Uncertainty in predictions of the climate response to rising levels of greenhouse gases. Nature 2005; 433: 403–406.
10 Annan, JD, Hargreaves, JC, Ohgaito, R, Abe‐Ouchi, A, Emori, S. Efficiently constraining climate sensitivity with paleoclimate simulations. SOLA 2005; 1: 181–184.
11 Bernardo, JM, Smith, AFM. Bayesian Theory Chichester: Wiley; 1994.
12 Christensen, JH, Hewitson, B, Busuioc, A, Chen, A, Gao, X, Held, I, Jones, R, Kolli, RK, Kwon, W‐T, Laprise, R, et al. %22Regional climate projections%22. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press; 2007.
13 Solomon, S, Qin, D, Manning, M, Chen, Z, Marquis, M, Averyt, KB, Tignor, M, Miller, HL. Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press; 2007.
14 Moss, RH, Schneider, SH. Uncertainties in the IPCC TAR: recommendations to lead authors for more consistent assessment and reporting. Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC; 2000; 33–51.
15 IPCC. Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties. IPCC; 2005.
16 Randall, DA, Wood, RA, Bony, S, Colman, R, Fichefet, T, Fyfe, J, Kattsov, V, Pitman, A, Shukla, J, Srinivasan, J, et al. %22Cilmate models and their evaluation%22. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press; 2007.
17 Meehl, GA, Stocker, TF, Collins, WD, Friedlingstein, P, Gaye, AT, Gregory, JM, Kitoh, A, Knutti, R, Murphy, JM, Noda, A, Raper, SCB, Watterson, IG, Weaver, AJ, Zhao, Z‐C. %22Global climate projections%22. In Climate Change 2007: The Physical Science Basis. Contribution of the Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press; 2007.
18 Denman, KL, Brasseur, G, Chidthaisong, A, Ciais, P, Cox, PM, Dickinson, RE, Hauglustaine, D, Heinze, C, Holland, E, Jacob, D, et al. %22Couplings between changes in the climate system and biogeochemistry%22. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press; 2007.
19 Moorcroft, PR. How close are we to a predictive science of the biosphere? Trends Ecol Evol 2006; 21(7): 400–407.
20 Talagrand, O, Vautard, R, Strauss, B. Evaluation of probabilistic prediction systems. In: Proceedings of ECMWF Workshop on Predictability. ECMWF, Shinfield Park: Reading RG2‐9AX; 1997; 1–26.
21 Annan, JD, Hargreaves, JC. Reliability of the CMIP3 ensemble. Geophys Res Lett 2010; 37(2): L02703.
22 Oreskes, N, Shrader‐Frechette, K, Belitz, K. Verification, validation, and confirmation of numerical models in the earth sciences. Science 1994; 263(5147): 641.
23 McWilliams, JC. Irreducible imprecision in atmospheric and oceanic simulations. Proc Natl Acad Sci 2007; 104(21): 8709.
24 Smith, LA. What might we learn from climate forecasts? Proc Natl Acad Sci 2002; 99(Suppl. 1): 2487.
25 Yokohata, T, Webb, MJ, Collins, M, Williams, KD, Yoshimori, M, Hargreaves, JC, Annan, JD. Structural similarities and differences in climate responses to co2 increase between two perturbed physics ensembles by general circulation models. J Clim 2009; 23: 1392–1410.
26 Hargreaves, J, Annan, J. On the importance of paleoclimate modelling for improving predictions of future. Clim Past 2009; 5: 803–814.
27 Held, IM. The gap between simulation and understanding in climate modeling. Bull Am Meteor Soc 2005; 86(11): 1609–1614.
28 Rougier, JC. Probabilistic inference for future climate using an ensemble of simulator evaluations. Clim Change 2007; doi: 10.1007/s10584‐006‐9156‐9.
29 Holden, PB, Edwards, NR, Oliver, KIC, Lenton, TM, Wilkinson, RD. A probabilistic calibration of climate sensitivity and terrestrial carbon change in genie‐1. Clim Dyn 2009; 1–22. doi: 10.1007/s00382‐009‐0630‐8.
30 Tachiiri, K, Hargreaves, JC, Annan, JD, Oka, A, Abe‐Ouchi, A, Kawamiya, K. Development of a system emulating the global carbon cycle in Earth system models. Geosci Model Dev Discus 2010; 3(1): 99–180.
31 Whetton, P, Macadam, I, Bathols, J, O`Grady, J. Assessment of the use of current climate patterns to evaluate regional enhanced greenhouse response patterns of climate models. Geophys Res Lett 2007; 34(14): L14701. doi: 10.1029/2007GL030025.
32 Abe, M, Shiogama, H, Hargreaves, JC, Annan, JD, Nozawa, T, Emori, S. Correlation between inter‐model similarities in spatial pattern for present and projected future mean climate. SOLA 2009; 5: 133–136.
33 Räisänen, J, Ruokolainen, L, Ylhäisi, J. Weighting of model results for improving best estimates of climate change. Clim Dyn 2009: 1–16. doi: 10.1007/s00382‐009‐0659‐8.
34 Tebaldi, C, Smith, RL, Nychka, D, Mearns, LO. Quantifying uncertainty in projections of regional climate change: a Bayesian approach to the analysis of multimodel ensembles. J Clim 2005; 18(10): 1524–1540.
35 Tebaldi, C, Knutti, R. The use of the multi‐model ensemble in probabilistic climate projections. Philos Trans Royal Soc A: Math Phys Eng Sci 2007; 365(1857): 2053.
36 Allen, MR, Stott, PA, Mitchell, JFB, Schnur, R, Delworth, TL. Quantifying the uncertainty in forecasts of anthropogenic climate change. Nature 2000; 407(6804): 617–620.
37 Boé, J, Hall, A, Qu, X. September sea‐ice cover in the Arctic Ocean projected to vanish by 2100. Nature Geoscience 2009, 2: 341–343.
38 Lipton, P. Testing hypotheses: prediction and prejudice. Science 2005; 307: 219–221.
39 Crucifix, M. Does the last glacial maximum constrain climate sensitivity? Geophys Res Lett 2006; 33: L18701. doi: 10.1029/2006GL027137.
40 Otto‐Bliesner, BL, Schneider, R, Brady, EC, Kucera, M, Abe‐Ouchi, A, Bard, E, Braconnot, P, Crucifix, M, Hewitt, CD, Kageyama, M, et al. A comparison of PMIP2 model simulations and the MARGO proxy reconstruction for tropical sea surface temperatures at last glacial maximum. Clim Dyn 2009; 32: 799–815. doi: 10.1007/s00382‐008‐0509‐0.
41 Hargreaves, JC, Abe‐Ouchi, A, Annan, JD. Linking glacial and future climates through an ensemble of GCM simulations. Clim Past 2007; 3(1): 77–87.
42 Waelbroeck, C, Paul, A, Kucera, M, Rosell‐Melé, A, Weinelt, M, Schneider, R, Mix, AC, Abelmann, A, L Armand, Bard, E, et al. Constraints on the magnitude and patterns of ocean cooling at the last glacial maximum. Nature Geosci 2009; 2: 127–132.
43 Haywood, AM, Dowsett, HJ, Otto‐Bliesner, B, Chandler, MA, Dolan, AM, Hill, DJ, Lunt, DJ, Robinson, MM, Rosenbloom, N, Slazmann, U, et al. Pliocene model intercomparison project (PlioMIP): experimental design and boundary conditions (experiment 1). Geosci Model Dev Discus 2010; 3: 227–242.
44 Saltzman, B, Verbitsky, M. Predicting the Vostok CO2 curve. Nature 1995; 377: 690.
45 Wolff, EW, Chappellaz, J, Fischer, H, Kull, C, Miller, H, Stocker, TF, Watson, AJ. The epica challenge to the earth system modeling community. EOS Trans Am Geophys Union 2004; 85(35): 363.
46 Forest, CE, Stone, PH, Sokolov, AP, Allen, MR, Webster, MD. Quantifying uncertainties in climate system properties with the use of recent climate observations. Science 2002; 295(5552): 113–117.
47 Knutti, R, Stocker, TF, Joos, F, Plattner, G‐K. Constraints on radiative forcing and future climate change from observations and climate model ensembles. Nature 2002; 416: 719–723.
48 Hargreaves, JC, Annan, JD, Edwards, NR, Marsh, R. An efficient climate forecasting method using an intermediate complexity Earth system model and the ensemble Kalman filter. Clim Dyn 2004; 23(7–8): 745–760.
49 Ridgwell, A, Hargreaves, JC, Edwards, NR, Annan, JD, Lenton, TM, Marsh, R, Yool, A, Watson, A. Marine geochemical data assimilation in an efficient earth system model of global biogeochemical cycling. Biogeosciences 2007; 4(1): 87–104.
50 Hargreaves, JC, Annan, JD. Assimilation of paleo‐data in a simple Earth system model. Clim Dyn 2002; 19(5–6): 371–381.
51 Annan, JD, Hargreaves, JC. Using multiple observationally‐based constraints to estimate climate sensitivity. Geophys Res Lett 2006; 33(L06704). doi: 10.1029/2005GL025259.
52 Hegerl, GC, Crowley, TJ, Hyde, WT, Frame, DJ. Climate sensitivity constrained by temperature reconstructions over the past seven centuries. Nature 2006; 440: 1029–1032.