Add auto-alignment for HDR bracket merging#21420
Conversation
|
Are the settings in preferences pretty much set and forget (don't need to be messed with) or useful to be able to adjust without having to go to preferences? I created an HDR, then adjusted the settings and tried to create another HDR using the same images as the previous. The second HDR overwrote the first HDR. Probably should use the on conflict naming. EDIT: In my very quick test, the images looked fine. No alignment errors were observed. |
I believe some of them might be removed after more testing, or we remove them from the GUI and keep them as expert settings in dartablerc only.
This behaviour exists since day 1 of the HDR merge feature and is not scope of this PR. I plan to improving the HDR merge feature after this PR, as I discovered a lot of bad behaviours with the actual HDR merge functionality. I want to make it closer to the hdrmerge algorithm, which - by the way - was also recently adopted in vkdt. It prevents ghosting much better as the naive merge algorithm in darktable.
Perfect! Can you try some really challenging handheld exposure brackets? |
At that time there was only 1 possible output. This PR introduces settings, making multiple outputs possible. Since settings are available, users will try and tune the HDR output to find the best settings. You could probably steal the conflict code from the exporter or just do a simple test if the file exists and then append
Everything I have can be aligned by simple x y alignments. |
a5a803c to
938a8f3
Compare
I prefer to do this in another PR, I don't want to change too many things at a time. EDIT: Okay, the more I think about this the more sense it makes, especially for testing the feature with the ability to create multiple new HDR DNG files with different settings will be much easier with that filename numbering schema. If we settle on a specific set of parameter values, we can also remove this again. |
|
FYI: I did not test the auto-alignment with X-Trans (Fuji) RAW files. It would be great if anybody could cover this case or provide exposure brackets for testing. Thanks! |
Implements automatic image registration in the lighttable merge HDR
path so handheld/tripod brackets can be merged without ghosting, via OpenCV.
Key pieces:
- src/common/hdr_alignment.{h,c}: pure-C layer — CFA luma proxy
build, percentile normalisation, proxy→full-res homography rescale,
CFA-aware same-color warp (Bayer exact, X-Trans approx),
reliability/sanity gate, per-frame orchestration
- src/common/hdr_alignment_cv.cc: OpenCV C++ backend — SIFT detect,
FLANN kNN + Lowe ratio + mutual-NN, spatial keypoint balancing
(5000-each), findHomography RANSAC, affine fallback, cluster-
degradation to translation-only, spatial match subsampling
- control_jobs.c: wire alignment state into dt_control_merge_hdr_t;
set_reference on first frame, align_frame on each subsequent frame
- Optional USE_OPENCV CMake flag (default ON); without OpenCV the
feature compiles to a no-op and the merge is unchanged
- Seven runtime prefs in darktableconfig.xml.in (auto_align,
auto_reference, proxy_scale, feature_gamma, clahe_clip,
sift_keypoints, debug_images)
- DT_DEBUG_HDR_MERGE channel (-d hdr_merge), debug image dump
- cmocka unit tests: synthetic homography round-trip + probe ranking
- CI/CD: add OpenCV per-module packages to Dockerfile, Brewfile,
ci.yml, nightly.yml; README + Windows packaging docs updated
Sync documentation with implementation again. Software quality: - control_jobs: restore the authoritative per-frame decoded-buffer guard (filters/channels/datatype) that the flags-only pre-check cannot replace (monochrome raws carry DT_IMAGE_RAW yet decode to filters==0; HDR/mono flags can lag the decode). Also reject monochrome in the up-front validator. - hdr_alignment: the X-Trans degenerate resample fallback now searches outward for the nearest same-colour photosite instead of returning any colour, so a warped cell never gets a wrong-colour sample (preserves mosaic phase). - Remove the dead motion-model parameter/enum from dt_hdr_alignment_new(). - Auto-reference probe pre-pass is now cancellable and advances the progress bar (was uninterruptible and looked hung on large brackets). - Report the cluster-degraded translation model accurately in the metrics log (new used_translation stat) instead of mislabelling it "homography". - _resolve_debug_dir creates the DT_HDR_DEBUG_IMAGE_DIR override if missing; dt_hdr_cv_count_features drops a no-op CLAHE call and a defensive clone; rename the moving-image cv::Mat i -> mov_img; write the reference debug visuals once instead of per frame. Tests: - test_hdr_alignment: CFA-modulated Bayer alignment test (per-channel gains, so a colour-blind warp would introduce crosstalk) — the headline property. - test_hdr_alignment_internal: X-Trans same-colour isolation test exercising the border fallback (sampling colour c only ever reads colour-c photosites).
- Cache the reference FLANN index. The reference (template) descriptors are fixed for the whole merge, but the image->template KD-tree was rebuilt for every one of the N-1 moving frames. Train it once in dt_hdr_cv_features_create() and store it on RefFeatures; only the template->image direction still builds a per-frame index over the moving descriptors. Match semantics (queryIdx/trainIdx) are unchanged. - Auto-reference probe builds its proxy at the probe resolution directly. SIFT in the probe runs at <= AUTO_REFERENCE_PROBE_DIM (1500px) anyway, so building the full DT_HDR_PROXY_SCALE proxy and letting OpenCV downscale it wasted work: for a 24-100MP raw the intermediate proxy was 2.5-4.9x larger than needed. Now the proxy is built (and percentile-normalized) directly at probe scale; the ranking stays self-consistent because all frames share dimensions. - Cap the FLANN parallel-sections team to num_threads(2): only two sections do work, so there is no reason to wake the full OMP team. - Skip the redundant full-frame copy for a static frame. When the warp is a reliable no-op (sub-pixel corner motion) or identity, align_frame now returns FALSE and leaves `out` untouched, so the caller accumulates its own source buffer instead of a byte-identical copy (~w*h*4 bytes saved per static frame). The header contract is updated: `out` is only guaranteed populated on TRUE. - Free the moving float proxy right after the u8 proxy is built (it is only an intermediate), trimming peak footprint during the SIFT stage. Not done: the affine/translation warp fast path (incremental source-coordinate stepping when H[6]==H[7]==0). It is gated behind collapse(2)/parallelization changes for a minority of per-pixel warp cost while SIFT dominates wall time, so it is left as profile-gated backlog per the review.
b88115d to
927c845
Compare
Implements automatic image registration in the lighttable merge HDR path so handheld/tripod brackets can be merged without ghosting, via OpenCV.
Auto-alignment processing steps:
estimateAffine2Dis tried as a fallback. If inliers cluster into ≤ 2 grid cells the model is further degraded to a pure translation to avoid wild extrapolation.A more detailed design overview can be found here:
https://github.com/da-phil/darktable/blob/a5a803c5447894faa208a9507a5c82e5fb38e260/dev-doc/HDR_Alignment_Design.md
Key changes
App images can be found here:
https://github.com/da-phil/darktable/actions/runs/28748026727#artifacts
Disclaimer: this work has been co-created with Claude.
Fixes: #17326