Repository navigation
feat(motiondiff0r): add temporal frame differencing filter #320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+851
−1
Merged
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
cde0da8
feat(motiondiff0r): add temporal frame differencing filter
phoebus-84 49fa9fa
test(motiondiff0r): add contract test
phoebus-84 75c87c9
fix(motiondiff0r): recover from frame allocation failures
phoebus-84 937ebcd
fix(motiondiff0r): preallocate bounded frame history
jaromil e74c3b8
fix(tests): compile shared bundle fixture as PIC
jaromil 25732b5
fix(motiondiff0r): reuse released history buffers
jaromil f22267d
fix(motiondiff0r): raise history memory cap to 1 GiB
jaromil File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| set (SOURCES motiondiff0r.cpp) | ||
| set (TARGET motiondiff0r) | ||
|
|
||
| if (MSVC) | ||
| set (SOURCES ${SOURCES} ${FREI0R_DEF}) | ||
| endif (MSVC) | ||
|
|
||
| add_library (${TARGET} MODULE ${SOURCES}) | ||
| set_target_properties (${TARGET} PROPERTIES PREFIX "") | ||
|
|
||
| install (TARGETS ${TARGET} LIBRARY DESTINATION ${LIBDIR}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,226 @@ | ||
| /* | ||
| * motiondiff0r - temporal frame differencing | ||
| * Copyright (C) 2026 phoebus-84 | ||
| * | ||
| * The current frame is compared with a frame taken from the recent past: | ||
| * the delayed frame is inverted and laid over the current one, so that | ||
| * everything that stays still turns into neutral gray while moving or | ||
| * changing regions show up as light and dark (positive / negative) shapes. | ||
| * | ||
| * Modes: | ||
| * - Negative overlay: R = (1 - b) * F(t) + b * (255 - F(t - dt)) | ||
| * with b = 0.5 static areas become mid gray | ||
| * - Absolute difference: R = |F(t) - F(t - dt)|, static areas are black | ||
| * - Motion mask: current frame with the amount of change written | ||
| * into the alpha channel, for use with other filters | ||
| * | ||
| * The delayed frame is chosen from a small history using the timestamps | ||
| * given by the host, so the time offset is independent of the frame rate. | ||
| * Memory use grows with the time offset: one frame buffer is kept for every | ||
| * frame received during the offset interval. | ||
| * | ||
| * This source code is free software; you can redistribute it and/or | ||
| * modify it under the terms of the GNU Public License as published by | ||
| * the Free Software Foundation; either version 2 of the License, or | ||
| * (at your option) any later version. | ||
| * | ||
| * This source code is distributed in the hope that it will be useful, | ||
| * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Please refer | ||
| * to the GNU Public License for more details. | ||
| */ | ||
|
|
||
| #include "frei0r.hpp" | ||
|
|
||
| #include <algorithm> | ||
| #include <cmath> | ||
| #include <cstdlib> | ||
| #include <deque> | ||
| #include <vector> | ||
|
|
||
| namespace { | ||
|
|
||
| const double MAX_OFFSET_SECONDS = 2.0; | ||
| const double MAX_GAIN = 16.0; | ||
| // Safety limit for hosts sending very dense timestamps. | ||
| const size_t MAX_HISTORY_FRAMES = 512; | ||
|
|
||
| enum Mode { MODE_OVERLAY = 0, MODE_ABSDIFF = 1, MODE_MASK = 2 }; | ||
|
|
||
| inline uint8_t clamp8(double v) | ||
| { | ||
| if (v <= 0.0) return 0; | ||
| if (v >= 255.0) return 255; | ||
| return (uint8_t)(v + 0.5); | ||
| } | ||
|
|
||
| inline int luma(const uint8_t* p) | ||
| { | ||
| // Rec. 601 weights on R, G, B (RGBA8888 byte order) | ||
| return (299 * p[0] + 587 * p[1] + 114 * p[2] + 500) / 1000; | ||
| } | ||
|
|
||
| struct Frame { | ||
| double time; | ||
| std::vector<uint32_t> pixels; | ||
| }; | ||
|
|
||
| class motiondiff0r : public frei0r::filter | ||
| { | ||
| public: | ||
| motiondiff0r(unsigned int, unsigned int) | ||
| { | ||
| offset = 0.1; // 200 ms | ||
| mode = 0.0; // negative overlay | ||
| blend = 0.5; | ||
| contrast = 0.0; // gain 1x | ||
| threshold = 0.0; | ||
| monochrome = false; | ||
|
|
||
| register_param(offset, "Time Offset", | ||
| "Time between the two frames (0..1 = 0..2 seconds; 0 = previous frame)"); | ||
| register_param(mode, "Mode", | ||
| "0: negative overlay, 0.5: absolute difference, 1: motion mask"); | ||
| register_param(blend, "Blend", | ||
| "Opacity of the inverted delayed frame in negative overlay mode"); | ||
| register_param(contrast, "Contrast", | ||
| "Amplification of the differences (0..1 = 1x..16x)"); | ||
| register_param(threshold, "Threshold", | ||
| "Ignore differences smaller than this to suppress noise"); | ||
| register_param(monochrome, "Monochrome", | ||
| "Compare luminance only and produce a grayscale result"); | ||
| } | ||
|
|
||
| virtual void update(double time, uint32_t* out, const uint32_t* in) | ||
| { | ||
| const size_t n = size; | ||
| const double target = time - std::min(std::max(offset, 0.0), 1.0) * MAX_OFFSET_SECONDS; | ||
|
|
||
| // Forget frames at or after the current time: the host seeked backwards | ||
| // or is rendering the same time again. | ||
| while (!history.empty() && history.back().time >= time) | ||
| recycle_back(); | ||
|
|
||
| // Keep the newest frame not later than the target as the oldest one. | ||
| while (history.size() >= 2 && history[1].time <= target) | ||
| recycle_front(); | ||
| while (history.size() >= MAX_HISTORY_FRAMES) | ||
| recycle_front(); | ||
|
|
||
| // Store a copy of the current frame before rendering, so that the | ||
| // history stays intact when the host renders in place (in == out). | ||
| history.push_back(Frame()); | ||
| Frame& f = history.back(); | ||
| f.time = time; | ||
| if (!pool.empty()) { | ||
| f.pixels.swap(pool.back()); | ||
| pool.pop_back(); | ||
| } | ||
| f.pixels.assign(in, in + n); | ||
|
|
||
| // With no older frame yet, the current frame is compared with itself. | ||
| process(&f.pixels[0], &history.front().pixels[0], out, n); | ||
| } | ||
|
|
||
| private: | ||
| double offset; | ||
| double mode; | ||
| double blend; | ||
| double contrast; | ||
| double threshold; | ||
| bool monochrome; | ||
|
|
||
| std::deque<Frame> history; | ||
| std::vector< std::vector<uint32_t> > pool; | ||
|
|
||
| void recycle_front() | ||
| { | ||
| pool.push_back(std::vector<uint32_t>()); | ||
| pool.back().swap(history.front().pixels); | ||
| history.pop_front(); | ||
| trim_pool(); | ||
| } | ||
|
|
||
| void recycle_back() | ||
| { | ||
| pool.push_back(std::vector<uint32_t>()); | ||
| pool.back().swap(history.back().pixels); | ||
| history.pop_back(); | ||
| trim_pool(); | ||
| } | ||
|
|
||
| void trim_pool() | ||
| { | ||
| // A couple of spare buffers is enough to avoid reallocating every frame. | ||
| while (pool.size() > 2) | ||
| pool.pop_back(); | ||
| } | ||
|
|
||
| int current_mode() const | ||
| { | ||
| if (mode < 1.0 / 3.0) return MODE_OVERLAY; | ||
| if (mode < 2.0 / 3.0) return MODE_ABSDIFF; | ||
| return MODE_MASK; | ||
| } | ||
|
|
||
| void process(const uint32_t* in, const uint32_t* delayed, uint32_t* out, size_t n) const | ||
| { | ||
| const int m = current_mode(); | ||
| const double b = std::min(std::max(blend, 0.0), 1.0); | ||
| const double gain = std::pow(MAX_GAIN, std::min(std::max(contrast, 0.0), 1.0)); | ||
| const int thr = (int)(std::min(std::max(threshold, 0.0), 1.0) * 255.0 + 0.5); | ||
|
|
||
| const uint8_t* src = reinterpret_cast<const uint8_t*>(in); | ||
| const uint8_t* old = reinterpret_cast<const uint8_t*>(delayed); | ||
| uint8_t* dst = reinterpret_cast<uint8_t*>(out); | ||
|
|
||
| for (size_t i = 0; i < n; ++i, src += 4, old += 4, dst += 4) { | ||
| int cur[3], d[3]; | ||
| if (monochrome) { | ||
| const int lc = luma(src); | ||
| const int ld = luma(old); | ||
| cur[0] = cur[1] = cur[2] = lc; | ||
| d[0] = d[1] = d[2] = lc - ld; | ||
| } else { | ||
| for (int c = 0; c < 3; ++c) { | ||
| cur[c] = src[c]; | ||
| d[c] = src[c] - old[c]; | ||
| } | ||
| } | ||
|
|
||
| int mag = std::max(std::abs(d[0]), std::max(std::abs(d[1]), std::abs(d[2]))); | ||
| if (mag < thr) { | ||
| d[0] = d[1] = d[2] = 0; | ||
| mag = 0; | ||
| } | ||
|
|
||
| switch (m) { | ||
| case MODE_OVERLAY: | ||
| // (1 - b) * current + b * inverted delayed frame, where the | ||
| // delayed frame is current - gain * difference. | ||
| for (int c = 0; c < 3; ++c) | ||
| dst[c] = clamp8((1.0 - b) * cur[c] + b * (255.0 - cur[c] + gain * d[c])); | ||
| dst[3] = src[3]; | ||
| break; | ||
| case MODE_ABSDIFF: | ||
| for (int c = 0; c < 3; ++c) | ||
| dst[c] = clamp8(gain * std::abs(d[c])); | ||
| dst[3] = src[3]; | ||
| break; | ||
| default: // MODE_MASK | ||
| for (int c = 0; c < 3; ++c) | ||
| dst[c] = (uint8_t)cur[c]; | ||
| dst[3] = clamp8(gain * mag * src[3] / 255.0); | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| }; | ||
|
|
||
| } // namespace | ||
|
|
||
| static frei0r::construct<motiondiff0r> plugin("motiondiff0r", | ||
| "Temporal frame differencing: overlays an inverted delayed frame to reveal motion", | ||
| "phoebus-84", | ||
| 0, 1, | ||
| F0R_COLOR_MODEL_RGBA8888); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Make frame allocation failure exception-safe and recoverable
The Frame is inserted into history before its pixel storage is allocated. If pixels.assign throws std::bad_alloc, the exception escapes f0r_update into the C host, and the new history entry remains present with an empty pixels vector. Even a host that catches the exception cannot safely continue: the next update may use that empty entry as history.front(), and process dereferences an invalid delayed-frame pointer.
I reproduced this against the built shared module using valid 8×8 frames: inject failure of the first 256-byte frame allocation, catch bad_alloc around f0r_update at t=0, restore normal allocations, then call f0r_update at t=1/30. The first output is left untouched; the second update terminates with SIGSEGV. No actual memory exhaustion is needed for the reproduction. Frame history makes allocation pressure realistic at larger resolutions and offsets.
Please allocate/populate a candidate frame before committing it to history, handle allocation failures from the history/pool operations as well, and produce defined fallback output without letting exceptions cross the plugin API. Add a regression that fails an allocation and then renders another frame successfully.