WxData 3.0 Released on conda-forge and PyPi
WxData 3.0 brings the WxData clients into the shell where you can download and process over 20+ Weather Models, Observed Soundings, NWS Forecasts/SPC Outlooks, NEXRAD II Radar Data straight from the command-line. Users can now use the WxData Command-Line Interface tool to build advanced ETL pipelines in shell scripts that can bulk download the data from various source servers (i.e. NCEP/NOMADS or ECMWF Open-Data vs. Amazon Web Services and Google Cloud) (and you can use a proxy server if necessary) → subset the data to a bounding box → decode the GRIB2 variable keys into plain-language → convert from GRIB2 to NETCDF → export the end result as a NETCDF (.nc) file.
TLDR: WxData 3.0 allows you to build automated pipelines in shell scripts (and native Python too) for large scale weather model data downloads, clean the data and export to a NETCDF (.nc) file. Download → Clean → Decode → Convert GRIB2 to NETCDF → Export to NETCDF.
WxData 3.0 introduces several new real-time ECMWF data clients and significant performance improvements to the existing ECMWF data clients.
Finally, WxData 3.0 introduces several new clients that retrieve and process archived weather model data and NEXRAD II Radar data. This can be useful to researchers who want archived weather model data for a case study or ongoing research.
Here are some helpful links to help you get started building your ETL pipelines
- Full WxData Documentation (I just created a new GitHub web page for this)
- Full WxData 3.0 Release Notes
- WxData GitHub Repository
- WxData Jupyter Notebook Repository
- WxData Shell Scripting Repository
Get started by either:
conda install -c conda-forge wxdata
OR
pip install wxdata
WxData 3.0 requires Python >= 3.10.