Statistical downscaling and post-processing have been a part of both weather forecasting and climate modeling applications for decades. These statistical techniques are used to correct persistent biases in atmospheric model outputs. While many methods for statistical downscaling have been proposed over the years, few have been made open source. This fact has made it difficult to systematically explore the performance of different methods in the context of the broad range of applications their output is use for. Furthermore, reproducing previous downscaling efforts has proved impossible in most cases and has lead to man reinventions of the downscaling wheel.
The recent growth in popularity of machine learning, and more specifically deep learning, is leading to the development of new downscaling methods that apply machine learning techniques to downscale climate model output. These approaches are appealing in part because they have the potential to make higher-order connections between modeled and observed climate. Additionally, the deep learning community has developed a productive culture of benchmarking approaches, leading to demonstrated improvements in target ML problems. We hope to do the same in the downscaling space.
Proposed Hacking
We plan to focus on three areas of development:
Development of end-to-end pipelines for applying downscaling methods to CMIP6 data
Development of methods for comparing downscaled data against other downscaled climate data products.
Development of metrics to help inform the selection of different methods
Implementation of new and existing downscaling methods in a common framework (this work is underway already)
Anticipated Data Needs
We plan to focus on downscaling the traditional target variables first (i.e. daily precipitation and temperature). However, if collaborators are interested in downscaling other variables, the tool set we hope to develop should support that. We have made data requests to have daily precipitation/temperature/wind/humidity/etc made available before the workshop.
Note: Xsd is a new tool we are developing to provide a common scikit-learn-like API on top of common downscaling approaches (e.g. BCSD, Analog-Regression, Quantile-Mapping). We are actively looking for collaborators on the development of this project. More information on the xsd roadmap is available here
Desired Collaborators
Anyone interested in statistical downscaling, machine learning, or climate applications.
@jhamman I like this project idea, and the proposed hacking would be very useful for the community! I am particularly interested in downscaling precipitation and wind speed/direction, thus I would be happy to contribute.
I’m not so familiar with the time investment required to make the proposed hacking happen, but it seems ambitious for 3 days? Do you have specific goals for the hackathon?
Hi @eric_keenan. Glad to hear this seems interesting to you. My personal goals are to establish some momentum around the xsd project and to produce a few sample pipelines using relatively simple downscaling methods. I don’t expect to walk away from the event with all of CMIP6 downscaled but I hope we can show some significant progress in this area. Some specific hacking activities that may interest people:
build out a model evaluation tool for evaluating multiple downscaling methods. Hopefully this takes the form of something like sklearn.metrics.
develop a benchmark problem to help the community measure the effectiveness of individual models
(re) implement a traditional downscaling method (e.g. BCSD, ARRM, QM) in a sklearn like framework
develop a new downscaling pipeline using sklearn. Something like a RandomForestRegressor would be a great place to start.
All of these are fairly well contained efforts and significant progress could be made on them in a few days. I expect any complete downscaling efforts will happen after the hack though.
I just created a slack channel and I’ve added this project to the signup spreadsheet.
What may be useful at this point is to hear from others what their specific downscaling applications look like. I’ll provide a brief description of what I’m going for:
Target variables: daily temperature and precip (but may just start off with temp)
Target region: global
Obs data: ERA-5
Methods: BCSD with possible mods at the tails
A great outcome for me would also be an end-to-end workflow for one method (BCSD) that possibly also shows differences at the tails depending on the type of quantile mapping used.
I have an interest in downscaling CMIP6. This is specifically to forecast a river follow and examine water-energy-food nexus in one of the basins within Ethiopia. Would you please let me know the status of your research project and its relevance for WEF Nexus analysis (forecast)?
Hi Mulugeta, we’re now working on an open source package for climate downscaling called scikit-downscale. You can find more details on the package and its potential applications here: https://github.com/jhamman/scikit-downscale.
Hi @jhamman. This project sounds very interesting and I would like to contribute if possible. My interest is statistical downscaling for the ocean and I am currently implementing the quantile mapping approach used for daily atmospheric variables in the ISIMIP3BASD project, but targeting monthly ocean variables. It would be great to be more involved in your project.
@trondkr - great. Would love to get your involvement in the project. We don’t have the exact ISIMIP3BASD implementation in the package yet, but it seems possible using mostly off the shelf components. If you don’t mind opening a feature request issue on github, we can discuss more there.