test: re-enable four tests skipped during past tooling migrations#3517
Open
PaulHax wants to merge 1 commit into
Open
test: re-enable four tests skipped during past tooling migrations#3517PaulHax wants to merge 1 commit into
PaulHax wants to merge 1 commit into
Conversation
Four tests were skipped over the years, each during a test-harness or build migration rather than for a genuine feature failure. A skipped test is green in CI, so none produced a signal and all silently rotted. Re-enable them. testLabelmapOutlineManyRenderer Skipped 2022-03-11 in commit 50ace42 (PR Kitware#2308, "Make test harnesses the same for release and PRs"); the commit gave no reason. While skipped the code rotted: createLabelPipeline called backgroundImageData.getDimensionsByReference(), which is not a function on vtkImageData. Use getDimensions(). Passes against the existing baseline, so no baseline change. testEdgeVisibility Skipped 2021-11-12 in commit 13b9be1 (PR Kitware#2161, "Update test framework"), flagged in a pixelmatch-threshold-0 audit (PR Kitware#2098, closed unmerged) as a "bad image test" (issue Kitware#2104). The 2017 baseline predates commit a8873e5 (2021-02-19, "feat(vtkopenglpolydatamapper): add support for backface property"), which changed the edge draw path from lit (ambient = getAmbient(), diffuse = getDiffuse()) to flat (ambient = 1.0, diffuse = 0.0): edges now render at the full configured edge color, the intended behavior. The render has been stable since at least 2021-06 (a steady 1.80% from the 2017 baseline at 2021-06, 2022, and 2024). Refresh the baseline; the tolerance is unchanged at 1% (chromium ~0%, firefox passes). testAverageIntensityProjection Skipped 2021-11-12 in commit 13b9be1 (PR Kitware#2161), tracked by issue Kitware#2102. It did not drift or regress. Before the 2021 test-framework work the test was effectively a no-op: the old compareImages passed its threshold argument (here 1.5) straight to pixelmatch, whose threshold range is 0 to 1, so pixelmatch reported zero differing pixels for any image and the test passed vacuously. The pixelmatch-threshold-0 audit (PR Kitware#2098, closed unmerged) exposed the real comparison, and the test then "failed" because its baseline had never actually been checked. The committed baseline is the original 2019 image from aae2f55; commit 8a50448 only renamed it OpenGL/ to Core/ (100% rename). It is a near-blank volume the real render never matched. Rendering against old Sources shows a stable, correct head volume back to 2021-06 (~1.5% from the corrected baseline, ~0% on current master). Replace the baseline with the actual render (chromium ~0%, firefox 0.01%, within the 1% tolerance) and re-enable the test. Also release resources via .finally; gc.releaseResources had been passed as an unused 5th argument to compareImages. testSetTable Skipped 2026-05-21 by the Vite/Vitest migration (c0d9887), which rewrote test.onlyIfWebGL(...) as it.skip(...) instead of the it.skipIf(__VTK_TEST_NO_WEBGL__)(...) form used for every other migrated WebGL test, silently disabling a passing test. Restore the intended WebGL-conditional skip. Closes Kitware#2102
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four tests were skipped over the years, each during a test-harness or build migration rather than for a genuine feature failure. A skipped test is green in CI, so none produced a signal and all silently rotted. Re-enable them.
testLabelmapOutlineManyRenderer
Skipped 2022-03-11 in commit 50ace42 (PR #2308, "Make test
harnesses the same for release and PRs"); the commit gave no
reason. While skipped the code rotted: createLabelPipeline called
backgroundImageData.getDimensionsByReference(), which is not a
function on vtkImageData. Use getDimensions(). Passes against the
existing baseline, so no baseline change.
testEdgeVisibility
Skipped 2021-11-12 in commit 13b9be1 (PR #2161, "Update test
framework"), flagged in a pixelmatch-threshold-0 audit (PR #2098,
closed unmerged) as a "bad image test" (issue #2104). The 2017
baseline predates commit a8873e5 (2021-02-19,
"feat(vtkopenglpolydatamapper): add support for backface
property"), which changed the edge draw path from lit
(ambient = getAmbient(), diffuse = getDiffuse()) to flat
(ambient = 1.0, diffuse = 0.0): edges now render at the full
configured edge color, the intended behavior. The render has been
stable since at least 2021-06 (a steady 1.80% from the 2017
baseline at 2021-06, 2022, and 2024). Refresh the baseline; the
tolerance is unchanged at 1% (chromium ~0%, firefox passes).
testAverageIntensityProjection
Skipped 2021-11-12 in commit 13b9be1 (PR #2161), tracked by
issue #2102. It did not drift or regress. Before the 2021
test-framework work the test was effectively a no-op: the old
compareImages passed its threshold argument (here 1.5) straight to
pixelmatch, whose threshold range is 0 to 1, so pixelmatch reported
zero differing pixels for any image and the test passed vacuously.
The pixelmatch-threshold-0 audit (PR #2098, closed unmerged)
exposed the real comparison, and the test then "failed" because its
baseline had never actually been checked.
The committed baseline is the original 2019 image from aae2f55;
commit 8a50448 only renamed it OpenGL/ to Core/ (100% rename).
It is a near-blank volume the real render never matched. Rendering
against old Sources shows a stable, correct head volume back to
2021-06 (~1.5% from the corrected baseline, ~0% on current master).
Replace the baseline with the actual render (chromium ~0%, firefox
0.01%, within the 1% tolerance) and re-enable the test. Also
release resources via .finally; gc.releaseResources had been passed
as an unused 5th argument to compareImages.
testSetTable
Skipped 2026-05-21 by the Vite/Vitest migration (c0d9887),
which rewrote test.onlyIfWebGL(...) as it.skip(...) instead of the
it.skipIf(VTK_TEST_NO_WEBGL)(...) form used for every other
migrated WebGL test, silently disabling a passing test. Restore the
intended WebGL-conditional skip.
Closes #2102