Skip to content

feat: support concurrent user-program QIS calls - #222

Draft
doug-q wants to merge 5 commits into
0.3-seriesfrom
doug/runtime-thread-safety-spec
Draft

doug-q wants to merge 5 commits into
0.3-seriesfrom
doug/runtime-thread-safety-spec

Conversation

@doug-q

@doug-q doug-q commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Support concurrent user-program QIS calls through thread-safe runtime interfaces and a single batch consumer. Runtime handles become const pointers, every Rust runtime method takes &self, and interfaces/factories require Send + Sync. Implementations synchronize their own state; FFI adapters borrow shared references without UnsafeCell.

The bundled and example runtimes use separate scheduling and result locks. The loaded-plugin wrapper excludes lifecycle/metric calls from other calls and serializes batch retrieval. One host consumer still executes batches in order; forcing a measurement guarantees resolution after draining and publishing its result.

Bump the runtime API to 0.4.0 and reject older plugins, regenerate the C header, and document the contract in the spec and API docs. Also fix leakage-measurement forcing and the example's barrier indices. The branch includes the previously committed spec and development-environment updates (devenv nixpkgs input and Rust components).

Code organization:

  • emulator/consumer.rs now owns simulator/error-model creation, batch retrieval and execution, result publication, shot lifecycle calls, and metric collection, moved out of emulator.rs and selene_instance/metadata.rs. These run on one consumer thread, so simulator/error-model interfaces need no new thread-safety requirements.
  • emulator.rs keeps QIS submissions on the calling threads and sends explicit drain requests to the consumer; there is no polling.
  • ffi_interface.rs borrows shared instance references for QIS calls. SeleneInstance and event hooks synchronize mutable state, with existing and new locks using parking_lot.

Validation:

  • Before the FFI follow-up, Linux wheel CI rebuilt the runtime plugins and passed all 140 Python tests.

  • 24 native tests passed across selene-core, selene-sim, both bundled runtimes, and the runtime example, including concurrent QIS calls, output serialization, shutdown, and runtime access guards.

  • Threaded Sol and Helios C QIS smoke tests each passed eight callers across three shots (768 measurements), with metrics and tracing enabled.

  • Clippy with -D warnings passed for selene-core, both bundled runtimes, and selene-sim.

  • Formatting and whitespace checks passed; Rust documentation and generated-header/C/C++ checks were also validated during implementation.

Draft follow-up:

  • Bound concurrency-test waits so deadlock regressions fail instead of hanging.

CI follow-up: macOS wheel jobs now use a fresh per-job RUSTUP_HOME to avoid the observed Clippy component-file conflict in the runner toolchain.

@doug-q

doug-q commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

@jake-arkinstall what do you think?

run: |
# Avoid component-file conflicts in the runner's preinstalled toolchains.
rustup_dir="$(mktemp -d "$RUNNER_TEMP/selene-rustup.XXXXXX")"
echo "RUSTUP_HOME=$rustup_dir" >> "$GITHUB_ENV"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macOS ARM wheel job failed before compiling Selene: when cargo metadata triggered installation of the components listed in rust-toolchain.toml, rustup reported clippy-preview-aarch64-apple-darwin conflicting with an existing bin/cargo-clippy.

This gives each macOS job a fresh toolchain directory and exports it before dtolnay/rust-toolchain runs. Both installation and the later wheel build therefore use the same isolated rustup state, while retaining the pinned Rust version and requested components. It avoids deleting or repairing the runner's existing toolchains.

Failing job · rustup's RUSTUP_HOME documentation

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks robot buddy!
@jake-arkinstall This seems helpful happy to split into separate PR if you think it might be good

Run batch retrieval, simulation, and result publication on one consumer thread while allowing concurrent runtime submissions. Synchronize FFI output, event hooks, PRNG access, and the shared time cursor without changing simulator or error-model plugin interfaces.

Use parking_lot for the new and existing locks, document lifecycle exclusion, and add concurrency and shutdown regression coverage.
Remove consumer polling and the active-shot flag. QIS calls already request draining after submission, so block on the request channel between calls and align the test runtime with that contract.
@doug-q doug-q changed the title feat: make runtime plugin interfaces thread-safe feat: support concurrent user-program QIS calls Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant