From ba685bea65e2f1c2ab7d1d728ca4e6b008a54a12 Mon Sep 17 00:00:00 2001 From: Gregor Sturm Date: Mon, 29 Jun 2026 20:49:14 +0200 Subject: [PATCH 1/2] Add reviewer guide --- docs/reviewer_guide.md | 57 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 docs/reviewer_guide.md diff --git a/docs/reviewer_guide.md b/docs/reviewer_guide.md new file mode 100644 index 0000000..4f0cac6 --- /dev/null +++ b/docs/reviewer_guide.md @@ -0,0 +1,57 @@ +# Reviewer guide + +The purpose of this document is to describe how to review ecosystem packages. + +Users add pull requests to request to be added to the scverse ecosystem package listing. +To be added, the the packages must fulfil the criteria listed in the ecosystems-package [README](https://github.com/scverse/ecosystem-packages#checklist-for-adding-packages). +Users are asked as part of their PR to add the checklist to the main PR comment and to fill it faithfully. +Your job as reviewer is to assess if the all checklist criteria are fulfilled. +Additionally, in case you notice anything else that could be improved (technically, or scientifically) +you can point that out during the review, but highlight that this is independnet of the criteria required +for package approval. + +Check the [package schema](https://github.com/scverse/ecosystem-packages/blob/main/scripts/src/ecosystem_scripts/schema.json) for additional context. + +## 1. Check if the checklist exists and is the same as the original one in the README + +Some users modify the checklist or leave out points or do not provide any checklist at all. +All "mandatory" points from the original checklist must be fulfilled. + +## 2. Check the license + +- license declared in the meta.yaml matches the license in the repo `LICENSE` file and in metadata fields of the package (e.g. pyproject.toml). +- A `LICENSE` file is mandatory +- license is indeed OSI-approved + +## 3. Check releases + +- releases are available on PyPI or any other approved registry. +- The repo makes use of github releases, or at least uses git tags that are consistent with the releases on the registry. + +## 4. Automated tests + +- Check the source code of the package if automated tests cover essential functions of the package and a reasonable range of inputs and conditions. +- At least all user-facing (public) functions should be covered by tests, but there can be exceptions, e.g. if a function is trivial or there are good reasons why it can't be tested automatically (e.g. slow and doesn't work with toy data) +- Minimal tests ensure that the function does not fail on an example data set. Ideally, tests also ensure the correctness of the results, e.g. by comparing against a snapshot. Provide feedback on how good the test quality is. + +## 5. Continuous integration + +- Check the repo that CI scripts exist that execute the automated tests from (4). +- Check that the CI checks are actually executed (on commits to main and PRs) +- Typically, this is done in github actions, but other CI systems are allowed, too. + +## 6. API docs + +- API documentation must exist. API documentation is an overview of all public functions provided by the package with a documentation of their parameters. +- Typically this is done via sphinx-autodocs, but it is ok to have it in a README or any other documentation system, as long as it's comprehensive. + +## 7. data structures + +- The package must be interoperable with other scverse packages by using its data structures +- AnnData for unimodal omics data +- MuData for multimodal omics data +- SpatialData for spatially resolved omics data. + +## 8. author approval + +- The author must have ticked the checkbox "I am an author or maintainer of the tool and agree on listing the package on the scverse website" From fb12eac143bff054fa73ea929a1a846a389214ba Mon Sep 17 00:00:00 2001 From: Gregor Sturm Date: Tue, 30 Jun 2026 16:00:24 +0200 Subject: [PATCH 2/2] Draft reviewer's guide --- docs/reviewer_guide.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/reviewer_guide.md b/docs/reviewer_guide.md index 4f0cac6..af034ca 100644 --- a/docs/reviewer_guide.md +++ b/docs/reviewer_guide.md @@ -17,41 +17,48 @@ Check the [package schema](https://github.com/scverse/ecosystem-packages/blob/ma Some users modify the checklist or leave out points or do not provide any checklist at all. All "mandatory" points from the original checklist must be fulfilled. -## 2. Check the license +## 2. check the meta.yaml + +- The meta.yaml must follow the [package schema](https://github.com/scverse/ecosystem-packages/blob/main/scripts/src/ecosystem_scripts/schema.json). +- This is already checked by github actions -- if there are failures, point them out to the user before continuing to review. +- The only category users can submit to is `ecosystem` (`core` packages require a special approval process). + +## 3. Check the license - license declared in the meta.yaml matches the license in the repo `LICENSE` file and in metadata fields of the package (e.g. pyproject.toml). - A `LICENSE` file is mandatory - license is indeed OSI-approved -## 3. Check releases +## 4. Check releases - releases are available on PyPI or any other approved registry. - The repo makes use of github releases, or at least uses git tags that are consistent with the releases on the registry. +- The `version` field in `meta.yaml` must match the latest version of the package at the time of submission of the PR. -## 4. Automated tests +## 5. Automated tests - Check the source code of the package if automated tests cover essential functions of the package and a reasonable range of inputs and conditions. - At least all user-facing (public) functions should be covered by tests, but there can be exceptions, e.g. if a function is trivial or there are good reasons why it can't be tested automatically (e.g. slow and doesn't work with toy data) - Minimal tests ensure that the function does not fail on an example data set. Ideally, tests also ensure the correctness of the results, e.g. by comparing against a snapshot. Provide feedback on how good the test quality is. -## 5. Continuous integration +## 6. Continuous integration - Check the repo that CI scripts exist that execute the automated tests from (4). - Check that the CI checks are actually executed (on commits to main and PRs) - Typically, this is done in github actions, but other CI systems are allowed, too. -## 6. API docs +## 7. API docs - API documentation must exist. API documentation is an overview of all public functions provided by the package with a documentation of their parameters. - Typically this is done via sphinx-autodocs, but it is ok to have it in a README or any other documentation system, as long as it's comprehensive. -## 7. data structures +## 8. data structures - The package must be interoperable with other scverse packages by using its data structures - AnnData for unimodal omics data - MuData for multimodal omics data - SpatialData for spatially resolved omics data. -## 8. author approval +## 9. author approval - The author must have ticked the checkbox "I am an author or maintainer of the tool and agree on listing the package on the scverse website"