How should Project Pythia review Cookbook submissions?

What’s the problem?

Pythia needs a clearer and more sustainable process for Cookbook submission and review as submissions grow. The goal of this discussion is not establishing rigorous scientific review, but more for clearly documenting and communicating the list of criteria to meet before a Cookbook is “published” to, and balancing responsibility across maintainers for keeping Cookbook submissions moving toward this goal.

How can you help?

This Google Doc has the current (in-development) list of criteria we will build the process on. Read the intro of that document, and suggest criteria, steps to take, and other considerations & concerns as comments in that Google Doc or here in this Discourse thread. You can also email your considerations.

These criteria will be reviewed and distilled to their most important pieces at the upcoming Pythia Education Working Group Meeting [1] next Monday, 31 July 2023, 1:00 pm MDT / 1900 GMT. From there, we will discuss options for our review process to evaluate these criteria for Cookbook submissions moving forward. Join us to participate in this discussion and implement this process!

Thanks for your time and help.

[1]: see Calendar section of for more meeting dates and information


Changing the topic slightly, is there any guidance on the mechanics of performing a cookbook review? Since the repo for the cookbook is transferred by the author to Pythia, there is no PR on that repo that would enable a “conventional” GitHub review. What to do? Should reviewers instead open issues on the transferred repo with requested changes, or should changes be communicated through the PR that results for cookbook gallery submission?