Hey,
I am going to start working on making Terraform (https://www.terraform.io/) modules for deploying Pangeo on Google Clouds GKE Kubernetes at work and have a green light to contribute to open source. I had a few questions and wanted to open this up to discussion.
Is there anyone currently working on Terraform modules?
What would be the process after I have a good module built up to contribute it to the Pangeo comunity?
Thanks,
Justin Minsk
Hi @Justin_Minsk,
A first attempt at making terraform script can be find in this discussion. Not sure why we did not persist longer. I guess that Pangeo’s main Kubernetes cluster are sufficiently long lived so we did not feel the need to use something like Terraform, but this does not mean it won’t be useful!
And about the contribution, just build a good module on your github namespace, and we could then put it in the pangeo namespace once it’s somehow mature enough!
Cheers,
Guillaume.
Hey all,
@scottyhq and I are planning to work more on Terraform deployments on AWS in the coming months. We have a small start on this repo. The plan is to have Terraform modules that can be taken by anyone to launch a JupyterHub-ready cluster.
Currently, we have found (in the case of AWS) that the eksctl tool is much easier to use than Terraform for some AWS infrastructure (mostly EKS). A lot of our Terraform learning is going to be finding what eksctl does for us in the background that we need to explicitly tell Terraform to do.
To your point @geynard, I think Terraform is changing rapidly enough that it didn’t have enough functionality to be practical compared to eksctl and other tools, especially over a year ago. Thus, Pangeo’s main Kubernetes clusters aren’t managed with it.
We can keep you posted on our Terraform work and would love to hear your progress as well!
I have played with eksctl and found it to be similar to the out of the box experience of GCPs GKE command line tool. I have some GKE infra. already working that spawn/maintains a cluster and resources and also installs a helm deployment. I had to do some hacks to get helm on terraform to behave I should have an example up in a week or two (more than likely by end of the week) which might be able to help with some of the AWS EKS deployment. Let me know if you have any questions or want me to look over anything I have access to AWS at work also.