Scientific Motivation
While the first ocean-atmosphere coupled general circulation models date back to 1969, transient simulations attempting to reproduce the historical record and project future climate changes were not available until the late 1980s. These first-generation GCMs included only atmospheric and oceanic components, were run at nominal resolutions of 3º-10º, and omitted many important sub-grid scale processes whose parameterizations are now common-place. Since then, climate model development has continued, with pushes towards higher resolution, the inclusion of other components of the Earth System, and a flurry of new and/or improved parameterizations. While several studies have documented the improvements in model skill reaped by these model developments, there has been no comprehensive study of climate model skill that spans all the way from the first-generation models of the late 1980s to the state-of-the-art CMIP6 ensemble.
Below: change in climate model performance across CMIP1, CMIP2, and CMIP3 from Reichler et al. 2008
Proposed Hacking
Overall Goal
Compute variable-specific and general model performance metrics (e.g. normalized area-weighted root-mean square; pattern correlations; area-weighted absolute bias; etc) across several model generations, including CMIP6.
Methods
Preliminary results
For some context on the old simulations and a preliminary result which compares model skill between the IPCC Second Assessment Report multi-model mean and the CMIP5 multi-model mean, see our recent pre-print at EarthArXiv (rejected but being revised for resubmission).
Below: example skill metric for temperature trends from IPCC Second Assessment Report ensemble and CMIP5 multi-model mean
Anticipated Data Needs
Monthly-mean values of a number of common variables for CMIP6 historical (e.g. 1800–2019) simulations, ~1000 years of control simulations, and 1% per year CO2 runs (if available).
Variables of interest (based on model skill metric in Reichler 2008):
- sea level pressure
- air temperature
- 2-m air temperature
- zonal and meridional wind
- precipitation
- specific and/or relative humidity
- snow fraction
- sea ice fraction
Ideally I would have access to ERA5 for all of these variables as well – does this exist somewhere on the PANGEO cloud?
The early generations of climate models (pre-CMIP) had such coarse grids that I only need a few GB somewhere to dump the data for comparing them against CMIP6!
Anticipated Software Tools
Hopefully we would build off of and contribute to existing packages (like esmtools and xskillscore) that leverage xarray and already feature tools for handling model ensembles and computing model performance metrics.
Desired Collaborators
Anyone! In particular, experience with the relevant software tools, model evaluation in general, handling large CMIP ensembles, and / or reanlysis products like ERA5 would be helpful.