Repository navigation
feat(motiondiff0r): add temporal frame differencing filter - #320
Conversation
✅ Conventional Commits check passedAll commit messages in this PR now follow the Conventional Commits specification. 🎉 |
Compares the current frame with a delayed one taken from a short, timestamp-based history (0-2 s, 0 = previous frame). Three modes: negative overlay (inverted delayed frame blended over the current one, static areas turn mid gray), absolute difference (static areas black) and motion mask (amount of change written to alpha). Parameters for blend, contrast, noise threshold and luminance-only comparison. In-place rendering is supported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XnoxvoRbSLkQ3sTCkxgiyi
Checks the three modes, frame selection by timestamp, threshold, seeking backwards, same-time and in-place rendering, and a long run at the maximum offset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XnoxvoRbSLkQ3sTCkxgiyi
0b56aa9 to
49fa9fa
Compare
|
nifty old trick :^) I had it made in assembler inside early freej versions, called absdiff, works well when combined with other filters. |
jaromil
left a comment
There was a problem hiding this comment.
Reviewed all five changed files at 49fa9fa. The existing Test and Website workflows are green. A local Clang 19 core-bundle build succeeds, and all 185 selected native CTests pass, including motiondiff0r-contract and the bundle symbol audit. Optional OpenCV/Cairo/GAVL plugins and the two profiles requiring their development packages were excluded locally.
I found one blocking resource-handling defect, detailed inline. Failure injection against the real shared module makes a frame allocation throw through f0r_update, then the next update segfaults even after allocations recover. The existing tests exercise normal allocation paths but not this failure/recovery path. No implementation changes have been made.
| history.push_back(Frame()); | ||
| Frame& f = history.back(); | ||
| f.time = time; | ||
| if (!pool.empty()) { | ||
| f.pixels.swap(pool.back()); | ||
| pool.pop_back(); | ||
| } | ||
| f.pixels.assign(in, in + n); |
There was a problem hiding this comment.
[P1] Make frame allocation failure exception-safe and recoverable
The Frame is inserted into history before its pixel storage is allocated. If pixels.assign throws std::bad_alloc, the exception escapes f0r_update into the C host, and the new history entry remains present with an empty pixels vector. Even a host that catches the exception cannot safely continue: the next update may use that empty entry as history.front(), and process dereferences an invalid delayed-frame pointer.
I reproduced this against the built shared module using valid 8×8 frames: inject failure of the first 256-byte frame allocation, catch bad_alloc around f0r_update at t=0, restore normal allocations, then call f0r_update at t=1/30. The first output is left untouched; the second update terminates with SIGSEGV. No actual memory exhaustion is needed for the reproduction. Frame history makes allocation pressure realistic at larger resolutions and offsets.
Please allocate/populate a candidate frame before committing it to history, handle allocation failures from the history/pool operations as well, and produce defined fallback output without letting exceptions cross the plugin API. Add a regression that fails an allocation and then renders another frame successfully.
Copy the incoming frame into a local buffer before touching the history, so a failed allocation can no longer leave an empty entry behind that a later update dereferences. Recycling keeps a single spare buffer and no longer allocates, and update() catches every exception so none crosses the plugin API: when memory runs out the frame is compared with itself, which gives the neutral "no motion" output, and the next frames render from the intact history. Add a regression test that replaces the global operator new to fail the frame copy (first frame, with history, in place) and checks the fallback output and the recovery. Without the fix it reproduces the reported segfault. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XnoxvoRbSLkQ3sTCkxgiyi
Limit each instance to 128 frame slots and 128 MiB of pixel storage, allocated during construction. Reuse a timestamped ring without allocating during update, and shorten the effective offset when history does not fit. Allow the C++ helper to reject failed constructor resources, return NULL on allocation failure, and restore parameter metadata after failed construction. Add memory-budget and allocation-free-rendering coverage, and adapt the existing allocation regression to constructor preallocation. Verified 187 selected native tests, four focused GCC and ASan checks, and a no-exceptions module build with its rendering contract.
Thanks for the review and for pushing 937ebcd — preallocating the history and keeping update() allocation-free is a cleaner fix than my try/catch fallback, which it supersedes. I pulled it and rebuilt locally: the full suite passes (194/194), and motiondiff0r-alloc and motiondiff0r-memory-contract are also clean under ASan/UBSan. One note for users: with the 128 MiB cap the reachable offset depends on resolution (about 1.4 s at 720p, 0.6 s at 1080p, 0.12 s at 4K at 25 fps). That’s fine for the motion-extraction use case, which mostly uses short offsets. Would it make sense to size the ring from the selected offset (still capped at 128 MiB), so small offsets don’t reserve the full budget up front? Happy to look into it if you think it’s worth it. |
Keep the bounded constructor allocation while decoupling chronological history entries from physical pixel slots. Reuse released slots first so short offsets do not cycle through and touch the entire reservation. Add working-set regression coverage for zero and positive offsets, history wraparound, seeks, and repeated timestamps.
Update the parameter description and allocation contract, including acceptance at the two-frame boundary and rejection just beyond it. Keep the 128-frame limit and reuse of released buffers.
Summary
Adds
motiondiff0r, a new filter that reveals motion by comparing the current frame with a delayed copy of the stream. The delayed frame is inverted and laid over the current one: everything that stays still cancels out to neutral gray, while moving or changing regions show up as light and dark (positive / negative) shapes.The idea comes from this video, which shows the trick of duplicating footage, inverting one copy, overlaying it and offsetting it in time: https://www.youtube.com/watch?v=NSS6yAMZF78
Modes
0(1 - blend) * F(t) + blend * (255 - F(t - dt)); with blend 0.5 static areas become mid gray0.5abs(F(t) - F(t - dt)); static areas become black1Parameters
Implementation notes
in == out) is supported.frei0r.hpp, RGBA8888, no new dependencies.Testing
test/motiondiff0r-contract.c(registered in ctest) covering the three modes, frame selection by timestamp, threshold, seeking backwards, same-time and in-place rendering, and a long run at the maximum offset.test/motiondiff0r-memory-contract.cppverifies byte/frame limits (including the exact 1 GiB two-frame boundary), constructor allocation failures, existing-instance metadata after failure, dense timestamp histories, rendering with heap allocations disabled, and reuse of only two buffers at zero offset or five at 125 ms / 32 fps across wraparound, seeks, and repeated timestamps.cpplintwith the CI filters: no findings.ffplay -vf "frei0r=motiondiff0r:0.1|0|0.5|0|0|n"on macOS.