-
Notifications
You must be signed in to change notification settings - Fork 20
Create sanitizer tests #69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
ajanczew
wants to merge
9
commits into
main
Choose a base branch
from
ajanczew-sanitizers
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+411
−14
Draft
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
81c23ad
Created sanitizer tests
35e4d63
Trigger sanitizers
ajanczew 308925f
Added cleanup
ajanczew 4508ede
Fixed undefined issue
ajanczew 569fbd7
Changed log upload
ajanczew 01c56a9
Added sanitizer app tests
ajanczew 9be7c15
Added conformance tests to sanitizer
ajanczew 675a4ae
Added tests summary
ajanczew 2300a6a
Changed the workflow runner
ajanczew File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # UBSan/IntSan suppression file for known false positives in libstdc++ headers. | ||
| # These are intentional unsigned arithmetic patterns in the C++ standard library | ||
| # that cannot be fixed in project code. | ||
|
|
||
| # std::string::npos is size_t(-1); comparisons against it use intentional unsigned wrap | ||
| unsigned-integer-overflow:*bits/basic_string.h* | ||
|
|
||
| # Mersenne Twister PRNG uses deliberate unsigned modular arithmetic | ||
| unsigned-integer-overflow:*bits/random.tcc* | ||
|
|
||
| # uniform_int_distribution rejection sampling uses unsigned negation (-range) | ||
| unsigned-integer-overflow:*bits/uniform_int_dist.h* | ||
| implicit-integer-sign-change:*bits/uniform_int_dist.h* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,233 @@ | ||
| name: Sanitizer Tests | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 18 * * *' # Daily 18:00 UTC (20:00 CEST / 19:00 CET) | ||
| workflow_dispatch: | ||
| push: | ||
| branches: ['ajanczew-sanitizers'] # TODO: remove before merge | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: bash | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| JOBS_NUM: 16 | ||
|
|
||
| jobs: | ||
| # ── Address / Thread / Undefined / Integer sanitizers ────────────────────── | ||
| # Release build with clang; runs unit tests (parrallelUT.sh) + conformance | ||
| # (ParallelAllTests.sh) non-halting, then gates once via the summary step. | ||
| sanitizer-tests: | ||
| name: "Sanitizer: ${{ matrix.sanitizer }}" | ||
| runs-on: ['self-hosted', 'linux', 'x64', 'docker', 'kubernetes', 'valgrind', 'jpeg-perf'] | ||
| timeout-minutes: 180 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - sanitizer: address | ||
| - sanitizer: thread | ||
| - sanitizer: undefined | ||
| - sanitizer: integer | ||
| steps: | ||
| - name: 'Harden Runner' | ||
| uses: step-security/harden-runner@17d0e2bd7d51742c71671bd19fa12bdc9d40a3d6 # v2.8.1 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - name: 'Setup: Cleanup workspace' | ||
| run: | | ||
| rm -rf ./* || true | ||
| rm -rf ./.??* || true | ||
|
|
||
| - name: 'Checkout repository' | ||
| uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 | ||
|
|
||
| - name: 'Setup: Install dependencies' | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get -y install cmake nasm clang llvm make | ||
|
|
||
| - name: 'Fix workspace permissions' | ||
| if: always() | ||
| run: | | ||
| sudo chown -R "$(id -u):$(id -g)" "$GITHUB_WORKSPACE" || true | ||
| chmod -R u+rwX "$GITHUB_WORKSPACE" || true | ||
|
|
||
| - name: 'Build: release with ${{ matrix.sanitizer }} sanitizer' | ||
| working-directory: Build/linux | ||
| run: | | ||
| ./build.sh release test sanitizer=${{ matrix.sanitizer }} cc=clang cxx=clang++ jobs=$(nproc) | ||
|
|
||
| - name: 'Unit tests: ${{ matrix.sanitizer }} sanitizer (release)' | ||
| # continue-on-error: collect all findings; the summary step is the single gate | ||
| continue-on-error: true | ||
| env: | ||
| # Non-halting: one pass collects every finding; the summary step gates. | ||
| ASAN_OPTIONS: "halt_on_error=0:exitcode=0:print_stacktrace=1" | ||
| TSAN_OPTIONS: "halt_on_error=0:exitcode=0:history_size=4" | ||
| UBSAN_OPTIONS: "suppressions=${{ github.workspace }}/.github/config/ubsan_suppressions.txt:print_stacktrace=1:halt_on_error=0:exitcode=0" | ||
| run: | | ||
| cd Bin/Release/ | ||
| chmod +x ${{ github.workspace }}/tests/scripts/parrallelUT.sh | ||
| chmod +x ./* | ||
| export LD_LIBRARY_PATH=$(pwd) | ||
| mkdir -p "${{ github.workspace }}/sanitizer-logs" | ||
| # pipefail so a failing shard is not masked by tee | ||
| set -o pipefail | ||
| ${{ github.workspace }}/tests/scripts/parrallelUT.sh ./SvtJpegxsUnitTests ${{ env.JOBS_NUM }} \ | ||
| 2>&1 | tee "${{ github.workspace }}/sanitizer-logs/unit-tests-${{ matrix.sanitizer }}.log" | ||
|
|
||
| - name: 'Conformance tests: ${{ matrix.sanitizer }} sanitizer (release)' | ||
| # continue-on-error: collect all findings; the summary step is the single gate | ||
| continue-on-error: true | ||
| working-directory: ${{ github.workspace }}/tests/scripts | ||
| env: | ||
| # Non-halting: one pass collects every finding; the summary step gates. | ||
| ASAN_OPTIONS: "halt_on_error=0:exitcode=0:print_stacktrace=1" | ||
| TSAN_OPTIONS: "halt_on_error=0:exitcode=0:history_size=4" | ||
| UBSAN_OPTIONS: "suppressions=${{ github.workspace }}/.github/config/ubsan_suppressions.txt:print_stacktrace=1:halt_on_error=0:exitcode=0" | ||
| DEC_BIN_DIR: ${{ github.workspace }}/Bin/Release | ||
| INPUT_FILES_PATH: /opt/samples | ||
| run: | | ||
| mkdir -p "${{ github.workspace }}/sanitizer-logs" | ||
| chmod +x ./*.sh | ||
| chmod +x "$DEC_BIN_DIR"/* | ||
| export LD_LIBRARY_PATH="$DEC_BIN_DIR" | ||
| # pipefail so a failing conformance shard is not masked by tee | ||
| set -o pipefail | ||
| ./ParallelAllTests.sh ${{ env.JOBS_NUM }} "$INPUT_FILES_PATH" "$DEC_BIN_DIR/SvtJpegxsDecApp" \ | ||
| 2>&1 | tee "${{ github.workspace }}/sanitizer-logs/conformance-tests-${{ matrix.sanitizer }}.log" | ||
|
|
||
| - name: 'Summarize: ${{ matrix.sanitizer }} sanitizer findings' | ||
| if: always() | ||
| working-directory: ${{ github.workspace }} | ||
| run: | | ||
| chmod +x tests/scripts/sanitizer_summary.sh | ||
| # integer is report-only (does not gate on findings); every other sanitizer fails on findings. | ||
| # A non-zero test-harness exit (broken run) fails the job for ALL sanitizers, integer included. | ||
| tests/scripts/sanitizer_summary.sh ${{ matrix.sanitizer == 'integer' && '--no-gate ' || '' }}"${{ matrix.sanitizer }}" \ | ||
| sanitizer-logs/unit-tests-${{ matrix.sanitizer }}.log \ | ||
| sanitizer-logs/conformance-tests-${{ matrix.sanitizer }}.log | ||
|
|
||
| - name: 'Archive: upload binary and logs on failure' | ||
| if: failure() | ||
| uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3 | ||
| with: | ||
| name: sanitizer-${{ matrix.sanitizer }}-failure-${{ github.run_number }} | ||
| retention-days: 7 | ||
| path: | | ||
| Bin/Release/SvtJpegxsUnitTests | ||
| Bin/Release/SvtJpegxsEncApp | ||
| Bin/Release/SvtJpegxsDecApp | ||
| sanitizer-logs/unit-tests-${{ matrix.sanitizer }}.log | ||
| sanitizer-logs/conformance-tests-${{ matrix.sanitizer }}.log | ||
|
|
||
| # Integer sanitizer runs report-only (halt_on_error=0), so the job stays | ||
| # green; always archive its logs so the findings are not lost. | ||
| - name: 'Archive: upload integer sanitizer report' | ||
| if: always() && matrix.sanitizer == 'integer' | ||
| uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3 | ||
| with: | ||
| name: sanitizer-integer-report-${{ github.run_number }} | ||
| retention-days: 7 | ||
| path: | | ||
| sanitizer-logs/unit-tests-integer.log | ||
| sanitizer-logs/conformance-tests-integer.log | ||
|
|
||
| - name: 'Cleanup: Remove workspace artifacts' | ||
| if: always() | ||
| run: | | ||
| rm -rf Bin/ Build/linux/Release/ || true | ||
|
|
||
| # ── Memory sanitizer ──────────────────────────────────────────────────────── | ||
| # Must be Debug (release does not build with MSan). | ||
| # Unit tests can't run under MSan: gtest aborts during global init. | ||
| # Use the C encoder/decoder apps instead. | ||
| sanitizer-memory: | ||
| name: "Sanitizer: memory" | ||
| runs-on: ['self-hosted', 'linux', 'x64', 'docker', 'kubernetes', 'valgrind', 'jpeg-perf'] | ||
| timeout-minutes: 180 | ||
| steps: | ||
| - name: 'Harden Runner' | ||
| uses: step-security/harden-runner@17d0e2bd7d51742c71671bd19fa12bdc9d40a3d6 # v2.8.1 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - name: 'Setup: Cleanup workspace' | ||
| run: | | ||
| rm -rf ./* || true | ||
| rm -rf ./.??* || true | ||
|
|
||
| - name: 'Checkout repository' | ||
| uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 | ||
|
|
||
| - name: 'Setup: Install dependencies' | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get -y install cmake nasm clang llvm make | ||
|
|
||
| - name: 'Fix workspace permissions' | ||
| if: always() | ||
| run: | | ||
| sudo chown -R "$(id -u):$(id -g)" "$GITHUB_WORKSPACE" || true | ||
| chmod -R u+rwX "$GITHUB_WORKSPACE" || true | ||
|
|
||
| - name: 'Build: debug with memory sanitizer' | ||
| working-directory: Build/linux | ||
| run: | | ||
| ./build.sh debug sanitizer=memory cc=clang cxx=clang++ jobs=$(nproc) | ||
|
|
||
| - name: 'Conformance tests: memory sanitizer (debug)' | ||
| # continue-on-error: collect all findings; the summary step is the single gate | ||
| continue-on-error: true | ||
| working-directory: ${{ github.workspace }}/tests/scripts | ||
| env: | ||
| DEC_BIN_DIR: ${{ github.workspace }}/Bin/Debug | ||
| INPUT_FILES_PATH: /opt/samples | ||
| # Non-halting: collect findings in one pass; the summary step gates. | ||
| MSAN_OPTIONS: "halt_on_error=0:exitcode=0" | ||
| # avx2: AVX-512 paths produce MSan false positives | ||
| SANITIZER_ASM: avx2 | ||
| run: | | ||
| mkdir -p "${{ github.workspace }}/sanitizer-logs" | ||
| chmod +x ./*.sh | ||
| chmod +x "$DEC_BIN_DIR"/* | ||
| export LD_LIBRARY_PATH="$DEC_BIN_DIR" | ||
| # pipefail so a failing conformance shard is not masked by tee | ||
| set -o pipefail | ||
| # fast: reduce the matrix (MSan is very slow); SANITIZER_ASM pins avx2 | ||
| ./ParallelAllTests.sh ${{ env.JOBS_NUM }} "$INPUT_FILES_PATH" "$DEC_BIN_DIR/SvtJpegxsDecApp" fast \ | ||
| 2>&1 | tee "${{ github.workspace }}/sanitizer-logs/memory-conformance-tests.log" | ||
|
|
||
| - name: 'Summarize: memory sanitizer findings' | ||
| if: always() | ||
| working-directory: ${{ github.workspace }} | ||
| run: | | ||
| chmod +x tests/scripts/sanitizer_summary.sh | ||
| tests/scripts/sanitizer_summary.sh "memory" \ | ||
| sanitizer-logs/memory-conformance-tests.log | ||
|
|
||
| - name: 'Archive: upload binaries and logs on failure' | ||
| if: failure() | ||
| uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3 | ||
| with: | ||
| name: sanitizer-memory-failure-${{ github.run_number }} | ||
| retention-days: 7 | ||
| path: | | ||
| Bin/Debug/SvtJpegxsEncApp | ||
| Bin/Debug/SvtJpegxsDecApp | ||
| sanitizer-logs/memory-conformance-tests.log | ||
|
|
||
| - name: 'Cleanup: Remove workspace artifacts' | ||
| if: always() | ||
| run: | | ||
| rm -rf Bin/ Build/linux/Debug/ || true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.