Would be nice to have a topic where people can suggest modules that would like to have on the cloud, and then people can upvote them. If there are enough upvotes the module can get added to the default environment.
1 Like
Gibbs Sea Water (gsw) bug should be fixed.
Thanks for opening this.
One thing to keep in mind, some of these modules may be OK just keeping on the singleuser jupyterlab environment, and not adding to the full image which is also used on the workers (things like ffmpeg and xmovie).
To what extent would this be solved by installing these modules in a persistent location in your user directory? Something like
pip install --user cmocean
Will install into /home/jovyan/.local/.... It will persist across sessions and is already on sys.path so the usual import cmocean will work.
Thanks Tom, this is very helpful.
I was not aware that there is a possibility to install modules that persist. The information on the information sheet (http://pangeo.io/cloud.html#software-environment) should be updated in accordance.
Oh that is indeed awesome! Thanks Tom.
Is there a way to request/add packages to be installed on the workers on a user basis aswell? Or would such libraries have to be added to the general environment?
For worker environments, the best bet is something like https://github.com/dask/distributed/pull/3216. In the meantime, you can install them manually with client.run and a function that invokes pip. But that doesn’t play well with adapative scaling.
1 Like