Would like to solicit feedback on this so re-posting this here.
Summary:
I recently proposed formatting changes to a single cookbook.
- I wanted to first see if others accept this change before doing mass updates
- I was wondering whether updates to the upstream template will propagate downstream
My main concern is that if auto-propagation is not implemented and the cookbook template is updated with say, package deprecation or important security updates, then all the existing cookbooks will fall behind and someone has to manually comb through them to update one by one.
Thus, I propose implementing a tool to propagate changes downstream before more cookbooks are added.
As an example, at Prefect, cruft is used to maintain consistency across multiple Collections, e.g. prefect-great-expectations/.cruft.json at main · PrefectHQ/prefect-great-expectations · GitHub and PrefectHQ/prefect-great-expectations#30)
2 Likes
We don’t currently have any mechanism to auto-propagate changes from the template to individual cookbooks. It’s something we’ve discussed (Automation for Cookbook repos · Issue #86 · ProjectPythia/cookbook-gallery · GitHub), and the need will definitely grow as the cookbook collection grows. We have not identified a good solution yet.
I can imagine a solution akin to the “migrators” that conda forge uses to keep recipes up to date. Minting a new release of the cookbook repo could automatically trigger PRs on every cookbook repo. From there, individual cookbook maintainers would have to take over and do any necessary tweaking of the bot-generated PR.
Haven’t heard of cruft before, I’ll take a look!
actions-template-sync looks really promising! Thanks for that suggestion. I will try to take a close look. I’ll mention this in our GitHub issue as well.
1 Like
I took a look at both actions-template-sync and cruft today.
I think cruft is promising for a future iteration, and I opened an issue for that here. It would require that we pivot from using GitHub templates to a cookiecutter approach instead, but I see a lot of value in that.
actions-template-sync looks like the right tool – it is designed to work with GitHub templates. But I did not find a way to configure it to deal sensibly with merge conflicts between a Cookbook repo and the Template repo. I left some notes about that here.
So after some hours of investigating, unfortunately I don’t have a recommendation for a good way to propagate changes in the template.
1 Like
Thanks for diving deep into it; I agree that cruft is likely the way to go!
The question is, once we go there, I wonder what the plan is to migrate existing cookbooks?
I imagine it would have to be a tedious manual migration. We would need to convince ourselves that the pain is worth the gain.