Request: using llm issues in an open, pre-registered coding-agent trial

[aravkek]

Hi Simon,

I'm Arav, one of the people behind Quorus (https://github.com/Quorus-dev/Quorus), an open-source tool that runs coding agents from different vendors in one repo.

We're preparing a pre-registered, open trial of coding agents (Claude Code, Codex and opencode) on 30 real issues filed after the models' training cutoffs, and we'd like to ask whether we could use issues from llm.

What it would involve for you, only if you're interested:

• You approve the task list and one acceptance test per issue before anything runs. The tests stay hidden from the agents.
• You review anonymised patches (no author or branch info) and pick at most one per task. Only that one would be opened as a PR, and only if you want it.
• Every patch and number gets published, including failures, with the method fixed in advance so nobody can tune it afterwards.

You can say no, or pull out at any time, and we'll credit the repo or not, as you prefer. If this isn't a good fit, no reply needed. Thanks for building llm.

Arav