Repository navigation
fix(defish0r,c0rners): avoid out-of-bounds read on frames smaller than the interpolator window - #332
Open
hextheshadow wants to merge 1 commit into
Open
hextheshadow wants to merge 1 commit into
hextheshadow wants to merge 1 commit into
Conversation
…n the interpolator window The bicubic, spline 4x4/6x6 and Lanczos (16-tap) interpolators read a window of 4, 6 or 16 pixels around the sample point. interp.h clamps the window start with `if (m<0) m=0; if ((m+N+1)>w) m=w-N;`, which yields a negative start index when the frame is narrower than the window, and reads past the buffer when it is shorter. Select bilinear in set_intp() when the frame is smaller than the chosen interpolator's window. set_intp() only runs from the constructor and the parameter setters, so there is no per-frame cost, and output for frames that fit the window is unchanged. Reproduce: 8x8 frame, Lanczos interpolator, under AddressSanitizer.
jaromil
force-pushed
the
fix/interp-small-frame-oob
branch
from
November 16, 2026 12:49
f70b988 to
52ea1d7
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the out-of-bounds read (CWE-125) reported to security@dyne.org, opened as a PR at the maintainers' request.
Cause: the 4-, 6- and 16-tap interpolators in
interp.hclamp theirwindow start to the frame, but when the frame is smaller than the window
the clamp produces a negative start index (and reads past the buffer
vertically). Clamping the index to 0 does not help, because the window
simply does not fit in the frame.
Fix:
set_intp()falls back to bilinear when the frame is smallerthan the selected interpolator's window (4px for bicubic/spline4, 6px for
spline6, 16px for Lanczos). One spot per plugin;
c0rnersships its owncopy of
interp.hwith the same pattern, so it is fixed too.Impact: requires a non-default interpolator and a frame under
16px (or 4/6px). The default (bilinear) is unaffected.
Testing:
(previously 50-178 per interpolator for interpolators 2-6)
fits the interpolator window, including the exact boundaries (4, 6, 16)