# How about creating a pangeo-hpc github repo?

**URL:** <https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150>\
**Category:** HPC\
**Created:** [September 19, 2019, 7:38pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150 "2019-09-19T19:38:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![geynard](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/geynard/32/2924_2.png) [@geynard](https://discourse.pangeo.io/u/geynard)\
**Post date:** [September 19, 2019, 7:38pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150/1 "2019-09-19T19:38:15Z")

</div>

The idea (that originaly comes from @odakat from Ifremer) would be to share some configuration files, script or some documentation on how Pangeo has been deployed on various supercomputers.

Some thing I could personaly share from CNES Pangeo instance:

- `jupyterhub_config.py` config file, obfuscated from any IP, but that still contains things like ProfileSpawner or BatchSpawner configuration,
- Systemctl compliant juyterhub.service file,
- Not sure about the complete ansible script that deploys the hub, maybe too tightly linked with our infrastructure,
- Environment.yml files to deploy a complete Pangeo conda environement,
- Environment.yml file that can be used to launch a notebook server with useful widgets.
- Some docs on how to setup or configure this or that…

---

<div class="post-metadata">

**Author:** ![rabernat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rabernat/32/22_2.png) [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Post date:** [September 19, 2019, 8:52pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150/2 "2019-09-19T20:52:02Z")

</div>

Great idea! This was discussed at the Seattle meeting, and Will Barnes got a start on it already!

> **[pangeo-data/pangeo-for-hpc](https://github.com/pangeo-data/pangeo-for-hpc)**
>
> Instructions and boilerplate for running Pangeo on HPC platforms - pangeo-data/pangeo-for-hpc

We also tried doing it in the past. What is left of that is [http://pangeo.io/setup\_guides/hpc.html](http://pangeo.io/setup_guides/hpc.html).

We used to host some specific configurations in the main pangeo repo, but we ended up removing them. Matt encouraged us to migrate the configurations to [https://jobqueue.dask.org/en/latest/examples.html](https://jobqueue.dask.org/en/latest/examples.html).

I personally would love to see [hpc.pangeo.io](http://hpc.pangeo.io), full of detailed configurations for deploying on HPC. The challenge of course is keeping if fresh in a rapidly changing software environment. If you and Will wanted to take this one, many people would benefit.

---

<div class="post-metadata">

**Author:** ![geynard](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/geynard/32/2924_2.png) [@geynard](https://discourse.pangeo.io/u/geynard)\
**Post date:** [September 20, 2019, 8:04pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150/3 "2019-09-20T20:04:59Z")

</div>

Great! I was not aware of this github repo.

I guess I now just have to fill it with our stuff! I’ll try to share some of the things I listed above soon.

---

<div class="post-metadata">

**Author:** ![wtbarnes](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/wtbarnes/32/220_2.png) [@wtbarnes](https://discourse.pangeo.io/u/wtbarnes)\
**Post date:** [September 26, 2019, 8:50pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150/4 "2019-09-26T20:50:09Z")

</div>

Sorry I’m a bit late to the party. I was out of the office last week and have not used discourse before!

As Ryan mentioned, David Shean and I created the pangeo-for-hpc repo at the Seattle meeting. However, as you can probably see, not much has been done there outside of the PR I opened during the meeting. This is a good home for configuration files, but not necessarily the most ideal place for hosting docs, i.e. we probably want something more than a README.

It would be great to migrate all HPC-related docs to something like [hpc.pangeo.io](http://hpc.pangeo.io). Could we also store the source for such a webpage in the same repo and deploy to that URL?

---

<div class="post-metadata">

**Author:** ![rabernat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/rabernat/32/22_2.png) [@rabernat](https://discourse.pangeo.io/u/rabernat)\
**Post date:** [September 27, 2019, 4:04pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150/5 "2019-09-27T16:04:16Z")

</div>

That sounds like a great plan @wtbarnes!

We have a pangeo [sphinx theme](https://github.com/pangeo-data/sphinx_pangeo_theme/) which makes it easy to deploy a pangeo-style sphinx site ([example](https://github.com/pangeo-data/pangeo-datastore/tree/master/docs)).

I recommend you use the existing pangeo-for-hpc repo for this. Once you have a github pages site working there, we can create the DNS entry for [hpc.pangeo.io](http://hpc.pangeo.io).

---

<div class="post-metadata">

**Author:** ![wtbarnes](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.pangeo.io/wtbarnes/32/220_2.png) [@wtbarnes](https://discourse.pangeo.io/u/wtbarnes)\
**Post date:** [September 27, 2019, 5:41pm UTC](https://discourse.pangeo.io/t/how-about-creating-a-pangeo-hpc-github-repo/150/6 "2019-09-27T17:41:56Z")

</div>

Great! Thanks for the links to the examples. I’ve just created [an issue on the repo](https://github.com/pangeo-data/pangeo-for-hpc/issues/3) for this.