Skip to content

fix(defish0r,c0rners): avoid out-of-bounds read on frames smaller than the interpolator window - #332

Open
hextheshadow wants to merge 1 commit into
dyne:masterfrom
hextheshadow:fix/interp-small-frame-oob
Open

hextheshadow wants to merge 1 commit into
dyne:masterfrom
hextheshadow:fix/interp-small-frame-oob

Conversation

@hextheshadow

Copy link
Copy Markdown

Fixes the out-of-bounds read (CWE-125) reported to security@dyne.org, opened as a PR at the maintainers' request.

Cause: the 4-, 6- and 16-tap interpolators in interp.h clamp their
window start to the frame, but when the frame is smaller than the window
the clamp produces a negative start index (and reads past the buffer
vertically). Clamping the index to 0 does not help, because the window
simply does not fit in the frame.

Fix: set_intp() falls back to bilinear when the frame is smaller
than the selected interpolator's window (4px for bicubic/spline4, 6px for
spline6, 16px for Lanczos). One spot per plugin; c0rners ships its own
copy of interp.h with the same pattern, so it is fixed too.

Impact: requires a non-default interpolator and a frame under
16px (or 4/6px). The default (bilinear) is unaffected.

Testing:

  • ASan/UBSan: all 7 interpolators x 225 frame sizes, 0 failures in defish0r
    (previously 50-178 per interpolator for interpolators 2-6)
  • Output is byte-identical to the previous code for every frame size that
    fits the interpolator window, including the exact boundaries (4, 6, 16)
  • Full clang-14 build + ctest (202/202) and native bundle contract tests pass

…n the interpolator window

The bicubic, spline 4x4/6x6 and Lanczos (16-tap) interpolators read a window of 4, 6 or 16 pixels around the sample point. interp.h clamps the window start with `if (m<0) m=0; if ((m+N+1)>w) m=w-N;`, which yields a negative start index when the frame is narrower than the window, and reads past the buffer when it is shorter.

Select bilinear in set_intp() when the frame is smaller than the chosen interpolator's window. set_intp() only runs from the constructor and the parameter setters, so there is no per-frame cost, and output for frames that fit the window is unchanged.

Reproduce: 8x8 frame, Lanczos interpolator, under AddressSanitizer.
@jaromil
jaromil force-pushed the fix/interp-small-frame-oob branch from f70b988 to 52ea1d7 Compare November 16, 2026 12:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant