test: reset intentservice state between tests - #2125
Open
lorenzozanee wants to merge 1 commit into
Open
Conversation
intentservice keeps its state in module globals and nothing resets them between tests, so any test calling load() leaked a loaded service into every test that ran after it. test_invalid_intents_rejected only passed because of that leak - on its own it raised GarakException instead of the expected ValueError. Add an autouse clear_intentservice_state fixture in tests/conftest.py mirroring clear_langprovider_state, make test_invalid_intents_rejected load the service itself, and add a regression pair in tests/cas/test_intentservice_state_reset.py proving the reset. Closes NVIDIA#2124 Signed-off-by: lorenzozanee <wyz0707@proton.me>
jmartin-tech
requested changes
Aug 28, 2026
Collaborator
There was a problem hiding this comment.
There is no reason to have a test file to test testing only functionality.
At best test_intentservice_load_populates_state asserts are functionality that should be added to test_load_intentservice in test_intentservice.py.
| reload() | ||
|
|
||
|
|
||
| @pytest.fixture(autouse=True) |
Collaborator
There was a problem hiding this comment.
Setting this to autouse at the root of the test path will cause it to run for every test. Is that really what it should do?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
intentservicekeeps its state in module globals (is_loaded,intent_typology,intent_detectors,intents_active) and nothing resets them between tests. The suite runs in a single process, so whichever test last calledload()decides what every later test sees. This is the same problemlangservicehad, already fixed byclear_langprovider_stateintests/langservice/conftest.py;intentservicestill needs the equivalent.Concrete symptom:
test_invalid_intents_rejectednever loads the service itself, so it only passes because an earlier test leaves the service loaded — run alone it raisesGarakException: get_intent_stubs called on non-loaded intentserviceinstead of the expectedValueError. Intent-probe translation tests intests/langservice/probes/test_probes_base.pyalso flip their skip reason depending on whether an earlier test loaded the service. (The issue's example usesprobes.tap.TAPIntent, which no longer exists onmain; the same flip reproduces with the remainingIntentProbe,probes.grandma.GrandmaIntent.)Fix
clear_intentservice_statefixture intests/conftest.py, mirroringclear_langprovider_state: resets the four module globals after every testtest_invalid_intents_rejectednow loads the service itself instead of relying on leaked statetests/cas/test_intentservice_state_reset.py: the first test loads the service and asserts it is populated, the second asserts the state was reset in between — the second test fails onmainwithout the fixtureI searched the open PRs and none address intentservice test state; the only intentservice-flavoured open PR (#2030) is about harness intent handling.
Verification
Closes #2124