Repository navigation
Add an in-process TRX generation mode - #12318
Open
Rolf Bjarne Kvinge (rolfbjarne) wants to merge 2 commits into
Open
Rolf Bjarne Kvinge (rolfbjarne) wants to merge 2 commits into
Rolf Bjarne Kvinge (rolfbjarne) wants to merge 2 commits into
Conversation
Let test launchers report results from a separate app without restarting the launcher. Keep controller-backed recovery as the default where supported. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The platform documentation omits Windows/Linux defaults and Android/browser/WASI fallback behavior and out-of-process restrictions, leaving users unclear about supported modes for their target.
1 open issue
What changed in this PR
Adds an opt-in in-process mode to Microsoft.Testing.Platform’s TRX extension for launchers that do not need controller-backed recovery, while preserving existing defaults.
Changes:
- Adds
--trx-modevalidation and shares mode selection across controller activation and TRX communication. - Extends unit and acceptance coverage for mode selection, process counts, report contents, and help output.
- Updates package documentation, resource catalogs, and internal API tracking.
Test-value evidence: The description reports four failing mode cases when mode selection was disabled, followed by all eight passing after restoration. Execution artifacts and exact revision identifiers were not supplied, so this remains author-reported rather than independently verified evidence.
| File | Description |
|---|---|
| test/UnitTests/Microsoft.Testing.Extensions.UnitTests/TrxReportGeneratorCommandLineTests.cs | Tests mode values and report-option dependency. |
| test/UnitTests/Microsoft.Testing.Extensions.UnitTests/TrxModeHelpersTests.cs | Tests controller and pipe selection. |
| test/UnitTests/Microsoft.Testing.Extensions.UnitTests/RetryTrxRegistrationBehaviorTests.cs | Updates registered-option expectations. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/TrxTests.cs | Checks process counts and generated report contents. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/TrxSkippedTestTests.cs | Adds in-process skipped-test coverage. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/TrxFailingTestTests.cs | Adds in-process failure-report coverage. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs | Updates help and info expectations. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxProcessLifetimeHandler.cs | Makes controller participation mode-dependent. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxModeHelpers.cs | Centralizes mode-dependent controller decisions. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxEnvironmentVariableProvider.cs | Applies mode selection to environment setup. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxCommandLine.cs | Registers and validates the new option. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.zh-Hant.xlf | Adds Traditional Chinese catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.zh-Hans.xlf | Adds Simplified Chinese catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.tr.xlf | Adds Turkish catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.ru.xlf | Adds Russian catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.pt-BR.xlf | Adds Brazilian Portuguese catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.pl.xlf | Adds Polish catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.ko.xlf | Adds Korean catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.ja.xlf | Adds Japanese catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.it.xlf | Adds Italian catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.fr.xlf | Adds French catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.es.xlf | Adds Spanish catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.de.xlf | Adds German catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/xlf/ExtensionResources.cs.xlf | Adds Czech catalog entries. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/Resources/ExtensionResources.resx | Defines mode help and validation messages. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/PACKAGE.md | Documents modes and recovery trade-offs. |
| src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt | Tracks new constants, overrides, and helper. |
Review effort: Balanced
| Enable TRX report generation via the `--report-trx` command line option. | ||
|
|
||
| On platforms that can launch a test-host process (all except Android, browser, iOS, tvOS, and WASI), TRX uses controller-backed recovery by default: the test host still streams results and generates the report during normal execution, but a surviving controller process can recover completed results into a TRX report if the test host crashes, hangs, or is stopped by `--timeout`. Android, browser, iOS, tvOS, and WASI cannot launch a test-host process, so TRX automatically falls back to its original in-process implementation there — no controller-backed recovery is attempted, and no configuration is required to get this fallback. | ||
| On macOS, TRX uses a controller process by default. The test host generates the report during normal execution; the controller can recover completed results if the host crashes, hangs, or is stopped by `--timeout`. On iOS and tvOS, TRX runs in-process by default, without controller-backed recovery. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds a
--trx-modeoption to the TRX report extension:--trx-mode in-processmakes the test host write the TRX report itself. It doesn't start a separate controller process.--trx-mode out-of-processuses the controller. This is the current behavior.The default doesn't change. The option requires
--report-trx, and it accepts the--trx-mode:value,--trx-mode=valueand--trx-mode valueforms.Our use case is a launcher that runs tests in a separate app, for example on an iOS device or simulator. The test host is only a helper that starts and watches that app. Restarting the helper under a controller doesn't add any isolation, because the tests already run in another process. It only slows down startup and makes the launcher harder to debug and control.
Design and decision record
TrxModeHelpersdecides whether TRX generation uses the controller. The environment variable provider and the process lifetime handler both use this decision.--crashdumpor--timeout, can still start a controller. In that case, the test host still writes the TRX report itself.out-of-processis rejected on platforms that don't support a controller (iOS and tvOS). Those platforms already write the report in-process.PACKAGE.mddocuments both modes and the trade-off.Demonstration
--report-trx(default)--trx-mode:in-process--trx-mode out-of-process--trx-mode=in-process --crashdumpValidation and regression proof
Trx_ModeControlsProcessIsolationAndReportGenerationtest and new in-process rows inTrxFailingTestTestsandTrxSkippedTestTests.--report-trx, unsupported platforms) and the mode decision matrix inTrxModeHelpersTests.HelpInfoAllExtensionsTests.Test-value evidence
To check that the new acceptance test catches regressions, I temporarily changed the helper to ignore the mode. 4 of the 8 process-mode cases then failed at the expected startup-count and report assertions. After restoring the helper, all 8 passed.
Note
This pull request description was drafted with GitHub Copilot.