-
Notifications
You must be signed in to change notification settings - Fork 68
start policy updates on ai tool usage #1006
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 18 commits
ee99798
c1a156e
63bd811
408eff0
45a08f1
8986330
0d78e12
3c9df0d
993339a
3a21983
aec399b
79a85dd
c70e6c4
2de4038
f28a7c2
155d143
f3c41e3
873d91e
c0e29dc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,6 +37,27 @@ This concise guide presents the software peer review process for you as a packag | |
| - Your README file should strive to explain your package's functionality and aims, assuming readers have little to no domain knowledge. All technical tems, including references to other software, should be clarified. | ||
| - Your package will continue to evolve after review, the chapter on *Package evolution* [provides guidance about the topic](#evolution). | ||
|
|
||
| ### Use of generative AI tools | ||
|
|
||
| - The use of generative AI tools is acceptable in packages submitted for peer review, as described in [our initial blog post](https://ropensci.org/blog/2026/02/26/ropensci-ai-policy/). | ||
| - As stated in [our general policies](#policies-ai), we require all submitting authors to describe how such tools may have been used, and to include links to any relevant aspects of repositories. | ||
| - A very general rule-of-thumb is the greater the use of generative AI tools in an rOpenSci submission, the more documentation we will expect. | ||
|
|
||
| #### How to use generative AI tools in preparing software for review | ||
|
|
||
| Software review must be able to focus on design decisions and design history. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why history? What does it mean in practice for a reviewer? I'm not playing devil's advocate, I just think that as a reviewer I wonder. How about creating an example fake submission to exemplify what you have in mind?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like the idea of having a toy example. I read this, and to be honest, I don't know what to do as an author. Many people who send their packages to our peer review are not experienced developers or even have formal training in developing software. |
||
| Generative AI tools should be used in ways that best enable that. | ||
|
|
||
| - We encourage the use of tools which produce systematic records of design decisions and histories, such as [GitHub's "spec-kit"](https://github.com/github/spec-kit) or [our own experimental 'designlens' tool](https://github.com/ropensci-review-tools/designlens). | ||
| - Both of these create an additional `specs/` directory intended to be committed within a repository, and that records individual development phases, generally under sequentially-numbered sub-directories, with each containing several documents like "plan.md", "tasks.md", and "report.md". | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. aren't they _very_verbose? The one I've tried out for a hackathon once is https://github.com/Fission-AI/OpenSpec/
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, they're verbose, but the outputs really are very useful to understanding human decision-making processes behind software. In my opinion, this kind of verbosity is currently the best way to ensure something human is retained amidst the machine-generated noise. And it all adds only one additional sub-dir within a repo, which ain't that much.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. but part of the verbosity comes from the IA not the humans describing their thoughts?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Trying to understand the process here: I developed a package, at some point I use AI, then I decide to send the package to peer-review, and then I read this policy and learn about the existence of designlens and the GitHub tool. Can I use it at this stage? Or do these tools need to be used from the start of the development to capture the design decisions and history? |
||
| - A `specs/` folder can also be produced and maintained by hand, or by directly instructing a generative AI tool, but however produced it should contain a comprehensive history of software design, and provide an informative basis for review. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I really think we need an example package of some sort.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed, but see comment above - I prefer to wait until we have some good submissions to link to. I expect we'll be iterating on these phrases quite frequently, and really, strongly hope to be able to have good ones to link to. They're just not there yet. |
||
| - For alternative approaches, consider JOSS's requirement for statements of [_Scope and significance_](https://joss.readthedocs.io/en/latest/submitting.html#scope-and-significance), and see [recently accepted papers](https://joss.theoj.org/papers/published) for examples of such statements. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nice that we have some examples! |
||
|
|
||
| #### The importance of Git logs | ||
|
|
||
| - The Git log remains the ultimate source of insight into design history, and a minimal requirement for all submissions remains an ability to grasp design history through a sufficiently extensive Git log. | ||
| - While we place no current minimal limits on Git log size, we do note that [JOSS now requires at least six months of public commit history](https://joss.readthedocs.io/en/latest/submitting.html#:~:text=six%20months), and will defer to such requirements when considered necessary. | ||
|
|
||
| ## Preparing for Submission {#preparing-for-submission} | ||
|
|
||
| ### Asking for help | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -37,6 +37,15 @@ Esta gu铆a condensa el proceso de revisi贸n por pares desde el punto de vista de | |||||
| - El archivo *README* debe esforzarse por explicar las funcionalidades y los objetivos de tu paquete asumiendo poco o ning煤n conocimiento del dominio. Adem谩s, debe aclarar todos los temas t茅cnicos, incluidas las referencias a otros software. | ||||||
| - Tu paquete seguir谩 evolucionando despu茅s de la revisi贸n, el cap铆tulo sobre *Evoluci贸n de paquetes* [proporciona orientaci贸n sobre el tema](#evolution). | ||||||
|
|
||||||
| ### Uso de herramientas de IA generativa | ||||||
|
|
||||||
| El uso de herramientas de IA generativa es aceptable en los paquetes enviados para revisi贸n por pares, como se describe en [nuestro art铆culo en el blog](https://ropensci.org/blog/2026/02/26/ropensci-ai-policy/). | ||||||
| Como se indica en [nuestras pol铆ticas generales](#policies-ai) exigimos a todas las personas autoras que env铆en propuestas que describan c贸mo se han utilizado dichas herramientas, y que incluyan enlaces a cualquier aspecto relevante de los repositorios. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add examples of what a relevant aspect is ? As a potential author sending a package I don't know what a relevant aspect is...
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @yabellini Unfortunately, not yet. We don't yet have any good examples to link to. We just need to get these updates out first, then hope to have actual examples in as quickly as possible after then. |
||||||
| Una regla general es que, cuanto m谩s se utilicen herramientas de IA generativa en en un paquete enviado a rOpenSci, m谩s documentaci贸n esperaremos recibir. Adem谩s esperaremos que dichas herramientas se hayan utilizado *sistem谩ticamente*. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It said more documentation we wait to get, this documentation is about the use of AI? It can be confused with package documentation, at least in the Spanish version, and also what kind of documentation do we expect to receive? We really need examples here. |
||||||
| El uso sistem谩tico de herramientas incluye cualquier enfoque que detalle de forma transparente y progresiva la contribuci贸n de las herramientas de IA generativa a la producci贸n de software. | ||||||
| Los ejemplos de herramientas sistem谩ticas de IA generativa van desde [el "spec-kit" de GitHub para el desarrollo basado en especificaciones](https://github.com/github/spec-kit) a [nuestra propia herramienta experimental para documentar las decisiones de dise帽o de software](https://github.com/ropensci-review-tools/designlens). | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have some problems understanding what I'm supposed to do if I use AI in my package to meet the requirements outlined here. I need to check the examples linked in more deeply. I'm wondering if we can link to good examples from a package under review or other packages that have the information as we would like to see. |
||||||
| Todas ellas comparten el patr贸n com煤n de producir artefactos adicionales que documentan el *proceso* de desarrollo junto con el propio software. | ||||||
|
|
||||||
| ## Preparaci贸n para el env铆o {#preparing-for-submission} | ||||||
|
|
||||||
| ### Pedir ayuda | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -99,6 +99,14 @@ Esto | |||||
|
|
||||||
| Ten en cuenta que si el paquete no se env铆a a trav茅s de la *branch* principal o por defecto, tienes que cambiar a la *branch* usada para enviar antes de comenzar tu revisi贸n. | ||||||
|
|
||||||
| ### Uso de herramientas de IA generativa | ||||||
|
|
||||||
| El uso de herramientas de IA generativa es aceptable tanto para los autores como para los revisores, pero los revisores no pueden utilizarlas para fundamentar directamente sus decisiones o recomendaciones. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| Por lo general, los revisores deben solicitar la aprobaci贸n de los autores y editores antes de utilizar herramientas de IA generativa, y hacerlo p煤blicamente en el hilo de la revisi贸n. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I removed "En general" because it looks like there are some cases where you don't need to ask for permission, or you don't need to make the use public on the thread. |
||||||
| De la misma manera que les solicitamos a las personas autoras de los paquetes, pedimos a las personas que revisan, que informen cualquier uso de herramientas de IA generativa con el mayor detalle posible. | ||||||
| Algunos ejemplos de uso aceptable son la generaci贸n de una visi贸n general de la funcionalidad del paquete, o la iteraci贸n a trav茅s de un historial Git para proporcionar una visi贸n general de las decisiones de desarrollo. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we link to some good examples? Or how can you actually do that? |
||||||
| Pedimos a quienes revisan que sean lo m谩s transparentes posible. | ||||||
|
|
||||||
| ### Devoluciones sobre el proceso {#feedback-on-the-process} | ||||||
|
|
||||||
| Te animamos a que preguntes y opines sobre el proceso de revisi贸n en https://github.com/ropensci/software-review-meta/issues | ||||||
|
|
@@ -115,5 +123,3 @@ Te animamos a que preguntes y opines sobre el proceso de revisi贸n en https://gi | |||||
| Las personas que enviaron el paquete a revis贸n deben responder en un plazo de 2 semanas con sus cambios en el paquete en respuesta a tu revisi贸n. En esta fase, te pedimos que respondas si los cambios abordan suficientemente las cuestiones planteadas en tu revisi贸n. Fomentamos el debate continuo entre quienes enviaron el paquete y quienes lo revisan, y tambi茅n puedes pedir que quienes est谩n a cargo de la edici贸n aclaren cosas en el mismo hilo. | ||||||
|
|
||||||
| Utilizar谩s la [plantilla de aprobaci贸n](#approval2template). | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -34,6 +34,17 @@ El paquete incluye todos los siguientes tipos de documentaci贸n: | |||||
| - [ ] **Pruebas autom谩ticas:** Hay *tests* unitarios que cubren las funciones esenciales dentro del paquete con un rango razonable de entradas y condiciones. Todas las pruebas corren correctamente en la computadora local. | ||||||
| - [ ] **Directrices de empaquetado:** El paquete cumple las directrices de empaquetado de rOpenSci. | ||||||
|
|
||||||
|
|
||||||
| #### Uso de herramientas de IA generativa | ||||||
|
|
||||||
|
|
||||||
| - [ ] Se utilizaron herramientas de IA generativa para realizar esta revisi贸n | ||||||
|
|
||||||
| Solo si marca esa casilla: explique la contribuci贸n de dichas herramientas a su revisi贸n. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| --- | ||||||
|
|
||||||
|
|
||||||
| Estimaci贸n de horas dedicadas a la revisi贸n: | ||||||
|
|
||||||
| - [ ] Si quienes son responsables del paquete lo consideran apropiado, estoy de acuerdo en que se me reconozca en el archivo *DESCRIPTION* del paquete con rol de revisi贸n ("rev"). | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
details on such links? Example of a GitHub permalink?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No great ones yet - definitely planned, but nothing to date is in a state that would be a good reference there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't clear enough: I think this should explain the format you expect for such links, probably permalinks.