Resolve per-test actions concurrently during plan construction#1773
Open
inju2403 wants to merge 1 commit into
Open
Resolve per-test actions concurrently during plan construction#1773inju2403 wants to merge 1 commit into
inju2403 wants to merge 1 commit into
Conversation
`_constructStepGraph` resolves every test's action — running its traits' `prepare(for:)` and evaluating its test cases — serially, before any test runs. Because the run phase is already parallel, this serial pre-phase can become the wall-clock bottleneck for suites with many async-evaluated traits or conditions (for example `.enabled(if: await …)`). Resolve each test's action concurrently in a task group, then apply the resolved tests and actions back into the graphs afterwards. Mutating the shared `actionGraph` is the reason the original loop was serial, so it is hoisted out of the concurrent region; nothing else changes. The serial path is preserved when parallelization is disabled (`--no-parallel`). Plan construction runs once, before any test, so it is intentionally not throttled by the execution-time `maximumParallelizationWidth`. This is a no-op for suites without async trait work. It does not resolve the existing intra-test FIXME in `_determineAction` (that parallelizes a single test's traits); this is the orthogonal cross-test axis.
Contributor
|
Thanks for the PR! The code owners are going to review it soon. In the mean time, please let us know if you used an LLM/agent for any part of the work. Thanks! |
Contributor
Author
|
@grynspan If the project has a policy on AI-assisted contributions — or would rather not take them — just let me know and I'll respect that. Thanks! |
This was referenced Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Test execution is already parallel, but plan construction isn't:
_constructStepGraphresolves each test's action — running its traits'prepare(for:)and evaluating its test cases — serially, before any test runs. For typical suites that's immaterial, since planning is already sub-millisecond. But when many tests have async-evaluated conditions or traits (for example.enabled(if: await …)), this serial pre-phase runs every async evaluation back-to-back before the first test starts — and because execution is already parallel, that pre-phase can become the dominant cost of the run.This PR resolves those actions concurrently, so the work that the run phase already parallelizes is no longer serialized one step earlier. It's a no-op for suites without async trait work, and it's distinct from the existing
// FIXMEin_determineAction(which is about parallelizing a single test's traits; this is the cross-test axis). How common the async-condition regime is in practice is a call I'd defer to you.Modifications
Gather the non-placeholder tests, resolve each test's action concurrently in a
withTaskGroup, then apply the resolved tests and actions back intotestGraph/actionGraph. Mutating the sharedactionGraphis the only reason the originalawait testGraph.mapValues { … }loop had to be serial, so that mutation is hoisted out of the concurrent region; everything else is unchanged. Behavior is equivalent to the serial version — the same tests, actions, and graph structure result, and actions are applied serially after collection, so the outcome is independent of completion order. The existing suite passes andswift buildsucceeds.--no-parallel: the serial path is taken whenisParallelizationEnabledisfalse.maximumParallelizationWidth(which bounds how many test cases run at once), since this is one-time planning work rather than execution — happy to gate it on the width instead if you'd prefer.prepare(for:)and parameterized-argument closures now run concurrently across tests. Traits areSendableand scope providers already run concurrently during execution, so well-behaved traits are unaffected; the only case this could surprise is an@unchecked Sendabletrait whose synchronization assumed serial planning, which isn't a documented guarantee — flagging it explicitly.Test.cancel()) is still isolated by_determineAction's inner task group and does not affect siblings. Added a regression test that builds an eight-test plan where one test cancels in a trait and asserts the other seven still run.withTaskGroup— the same primitive already used in_determineActionand the run loop — so no new unstructured tasks. Non-Darwin (Embedded / WASI / Windows) compilation is left to CI.Measurements
A synthetic measurement to size the mechanism — deliberately a worst case, not a real-world figure. 1000 free-function tests, each with a trait whose
prepare(for:)awaits ~1 ms (modeling an async.enabled(if: await …)condition) and with empty bodies, measured by togglingisParallelizationEnabledon the same build:The fraction is this large only because the bodies are empty, so plan construction is essentially the whole run, and the condition is pure suspension. With real test bodies the fraction shrinks accordingly, and suites with no async trait work see no change at all. The only claim here is that the serial phase is not inherently sub-millisecond once trait preparation is async — which is the case this PR addresses.
Checklist: