Add auto-alignment for HDR bracket merging

[da-phil]

Implements automatic image registration in the lighttable merge HDR path so handheld/tripod brackets can be merged without ghosting, via OpenCV.

Auto-alignment processing steps:

• Each bracket frame is collapsed from its raw CFA mosaic to a CFA-free luma proxy (~0.625× resolution) by averaging stride-1 2×2 windows, then percentile-normalised and gamma-encoded to an 8-bit image.
• SIFT keypoints are detected on the 8-bit proxy, filtered to a 6 px scale floor, then spatially balanced to 5000 per frame across a 6×6 grid so neither frame dominates the matcher.
• Descriptors are matched with FLANN kNN, filtered by Lowe ratio (0.75) and mutual-NN consistency, then spatially subsampled to 1800 matches before RANSAC to avoid bias toward dense bright regions.
• RANSAC estimates an 8-DOF homography; if it yields fewer than 50 inliers or below 40 % inlier ratio, estimateAffine2D is tried as a fallback. If inliers cluster into ≤ 2 grid cells the model is further degraded to a pure translation to avoid wild extrapolation.
• The result is sanity-checked (translation < 30 % of the image diagonal, scale within 0.5–2×); frames that fail are accumulated unwarped, so the output is never worse than today's alignment-free merge.
• The homography is rescaled from proxy to full-sensor coordinates and applied with a CFA-aware resampler that only reads same-color photosites, preserving the Bayer/X-Trans mosaic phase so the warped frame drops directly into the existing weighted accumulator.

A more detailed design overview can be found here:
https://github.com/da-phil/darktable/blob/a5a803c5447894faa208a9507a5c82e5fb38e260/dev-doc/HDR_Alignment_Design.md

Key changes

• src/common/hdr_alignment.{h,c}: pure-C layer — CFA luma proxy build, percentile normalisation, proxy→full-res homography rescale, CFA-aware same-color warp (Bayer exact, X-Trans approx), reliability/sanity gate, per-frame orchestration
• src/common/hdr_alignment_cv.cc: OpenCV C++ backend — SIFT detect, FLANN kNN + Lowe ratio + mutual-NN, spatial keypoint balancing (5000-each), findHomography RANSAC, affine fallback, cluster- degradation to translation-only, spatial match subsampling
• control_jobs.c: wire alignment state into dt_control_merge_hdr_t; set_reference on first frame, align_frame on each subsequent frame
• Optional USE_OPENCV CMake flag (default ON); without OpenCV the feature compiles to a no-op and the merge is unchanged
• Seven runtime prefs in darktableconfig.xml.in (auto_align, auto_reference, proxy_scale, feature_gamma, clahe_clip, sift_keypoints, debug_images)
• DT_DEBUG_HDR_MERGE channel (-d hdr_merge), debug image dump
• cmocka unit tests: synthetic homography round-trip + probe ranking
• CI/CD: add OpenCV per-module packages to Dockerfile, Brewfile, ci.yml, nightly.yml; README + Windows packaging docs updated

App images can be found here:
https://github.com/da-phil/darktable/actions/runs/31011476951#artifacts

If you want to build yourself, you need install the OpenCV ≥ 4.4 (or 5.x) lib first (no opencv_contrib needed):

• macOS (Homebrew): brew install opencv
• Debian / Ubuntu: sudo apt install libopencv-core-dev libopencv-imgproc-dev libopencv-features2d-dev libopencv-flann-dev libopencv-calib3d-dev
• Fedora / RHEL: sudo dnf install opencv-devel
• Arch: sudo pacman -S opencv
• openSUSE: sudo zypper install opencv-devel
• Alpine: sudo apk add opencv-dev
• Windows: pacman -S mingw-w64-ucrt-x86_64-opencv

Disclaimer: this work has been co-created with Claude.

Fixes: #17326

[wpferguson]

Are the settings in preferences pretty much set and forget (don't need to be messed with) or useful to be able to adjust without having to go to preferences?

I created an HDR, then adjusted the settings and tried to create another HDR using the same images as the previous. The second HDR overwrote the first HDR. Probably should use the on conflict naming.

EDIT: In my very quick test, the images looked fine. No alignment errors were observed.

[da-phil]

Are the settings in preferences pretty much set and forget (don't need to be messed with) or useful to be able to adjust without having to go to preferences?

I believe some of them might be removed after more testing, or we remove them from the GUI and keep them as expert settings in dartablerc only.

The second HDR overwrote the first HDR. Probably should use the on conflict naming.

This behaviour exists since day 1 of the HDR merge feature and is not scope of this PR. I plan to improving the HDR merge feature after this PR, as I discovered a lot of bad behaviours with the actual HDR merge functionality. I want to make it closer to the hdrmerge algorithm, which - by the way - was also recently adopted in vkdt. It prevents ghosting much better as the naive merge algorithm in darktable.
Then we can also talk about other improvements of the actual HDR merge behaviour, such as how do we deal with existing merged DNG files.

EDIT: In my very quick test, the images looked fine. No alignment errors were observed.

Perfect! Can you try some really challenging handheld exposure brackets?

[wpferguson]

This behaviour exists since day 1 of the HDR merge feature

At that time there was only 1 possible output. This PR introduces settings, making multiple outputs possible. Since settings are available, users will try and tune the HDR output to find the best settings. You could probably steal the conflict code from the exporter or just do a simple test if the file exists and then append _xx to the filename where xx is a 2 digit sequence number.

Can you try some really challenging handheld exposure brackets?

Everything I have can be aligned by simple x y alignments.

[da-phil]

At that time there was only 1 possible output. This PR introduces settings, making multiple outputs possible. Since settings are available, users will try and tune the HDR output to find the best settings. You could probably steal the conflict code from the exporter or just do a simple test if the file exists and then append _xx to the filename where xx is a 2 digit sequence number.

I prefer to do this in another PR, I don't want to change too many things at a time.
This PR is only about auto-alginment for the HDR merge functionality and getting feedback from folks who requested the feature.

EDIT: Okay, the more I think about this the more sense it makes, especially for testing the feature with the ability to create multiple new HDR DNG files with different settings will be much easier with that filename numbering schema. If we settle on a specific set of parameter values, we can also remove this again.
Will push another commit soon.

[da-phil]

FYI: I did not test the auto-alignment with X-Trans (Fuji) RAW files. It would be great if anybody could cover this case or provide exposure brackets for testing. Thanks!

[da-phil]

In case anybody is on macOS and frequently stacks hands-free exposure brackets, please give this PR a shot. Pre-built AppImages are mentioned in the PR description and instructions how to install the new dependency (OpenCV) for all relevant operation systems are provided.

CC: @masterpiga @MStraeten @zisoft

[OuraN2O]

Hello @da-phil, I would really like to test your PR (on linux). The appimages at the place you mentionned seems to be expired, where do I need to go to download your PR ?

[da-phil]

Hello @da-phil, I would really like to test your PR (on linux). The appimages at the place you mentionned seems to be expired, where do I need to go to download your PR ?

Thanks for the hint, I just kicked off an artifact build 2 days ago and they're already expired...
I'll just kick off another run in the meantime: https://github.com/da-phil/darktable/actions/runs/31009524615#artifacts
Should be ready in about 15-20min.

[da-phil]

I think I need to temporarily update the retention-days within .github/workflows/nightly.yml to not have artifacts expire after one day already....

[da-phil]

Remove before merge!

[da-phil]

Remove before merge!

[da-phil]

Remove before merge!

[da-phil]

I changed the retention time to 21 days and did another artifact build here:
https://github.com/da-phil/darktable/actions/runs/31011476951#artifacts

[OuraN2O]

Thanks ! I've tested the merge with some of my pictures (shot on Sony A7II) without messing with the default settings, it works really well. The only time I encountered a problem was because one of the photo was blurry to begin with, so nothing to report.

Thank you a lot for the work. I'd be happy to test further if you make other changes.

One thing I could point out is to have the settings for the merge in a popup window when clicking on the merge button, or alternatively another button "HDR settings" just next to the "Create HDR" one. Would be less cluttered than having it amongst all the other settings

[da-phil]

Thanks ! I've tested the merge with some of my pictures (shot on Sony A7II) without messing with the default settings, it works really well. The only time I encountered a problem was because one of the photo was blurry to begin with, so nothing to report.

Thank you a lot for the work. I'd be happy to test further if you make other changes.

Cool, thanks for your feedback! How many images did you merge at once and how long did the merge take rougly?
I'd be interested in the platform you're using:

• OS (and if Linux, which distro and version)
• GPU

One thing I could point out is to have the settings for the merge in a popup window when clicking on the merge button, or alternatively another button "HDR settings" just next to the "Create HDR" one. Would be less cluttered than having it amongst all the other settings

Hmmm, this would be a breaking change in our UX/UI in such a case. If HDR merge would be a separate "operation group" in the lighttable UI, I think this would make sense, but not in the current design where HDR merge functionality is provided by a mere action button within the "actions on selection" group.

[OuraN2O]

I merged 3 exposures at 2 stops intervals and it took roughly 6 seconds to process.

• I use Gentoo Linux
• I have an AMD Vega 56, but I have deactivated opencl acceleration inside darktable, it is unstable on my machine.
• I have a Ryzen 5 5600 for the CPU

Hmmm, this would be a breaking change in our UX/UI in such a case. If HDR merge would be a separate "operation group" in the lighttable UI, I think this would make sense, but not in the current design where HDR merge functionality is provided by a mere action button within the "actions on selection" group.

I see, then I'd prefer to have it as a separate tab (as you'd see it if you use the lua script HDRmerge) rather than in global settings. I agree that having a single button for HDR amongst other actions is quite bizarre.
Although, it would still be great if it's merged as is, don't get me wrong

[wpferguson]

Hmmm, this would be a breaking change in our UX/UI in such a case.

The scope of this PR is to replace the existing functionality with an aligned version while leaving the UI unchanged.

If a more extensive UI is needed for HDR merging I would suggest that it be a separate PR rather than having this PR start feature creeping.

[piccolbo]

FYI: I did not test the auto-alignment with X-Trans (Fuji) RAW files. It would be great if anybody could cover this case or provide exposure brackets for testing. Thanks!

Just tested on X-Trans. Looking good so far. Not challenging, three-shot handheld brackets kept as steady as possible, up to 85mm equiv focal, but they won't merge correctly in stock darktable. I can provide RAF files but can't attach them here.

[da-phil]

FYI: I did not test the auto-alignment with X-Trans (Fuji) RAW files. It would be great if anybody could cover this case or provide exposure brackets for testing. Thanks!

Just tested on X-Trans. Looking good so far. Not challenging, three-shot handheld brackets kept as steady as possible, up to 85mm equiv focal, but they won't merge correctly in stock darktable. I can provide RAF files but can't attach them here.

I'm not sure I understand you correctly. Did you mean to say that while stock darktable was not able to merge the images correctly, darktable with the changes in this PR was able to deliver a good HDR merge result (which I really hope 😄)?

[da-phil]

I merged 3 exposures at 2 stops intervals and it took roughly 6 seconds to process.

I believe this is pretty good for the hardware you listed below! So I think that performance seems not to be a bottleneck.

I see, then I'd prefer to have it as a separate tab (as you'd see it if you use the lua script HDRmerge) rather than in global settings. I agree that having a single button for HDR amongst other actions is quite bizarre. Although, it would still be great if it's merged as is, don't get me wrong

Yeah, let's talk about UI changes after this PR was merged, but you're right, the current solution isn't great and was mainly provided for debugging reasons while this PR is open. I'm not even sure if I want to keep the settings eventually, as long as the default settings work well enough for every possible use-case.

[piccolbo]

FYI: I did not test the auto-alignment with X-Trans (Fuji) RAW files. It would be great if anybody could cover this case or provide exposure brackets for testing. Thanks!

Just tested on X-Trans. Looking good so far. Not challenging, three-shot handheld brackets kept as steady as possible, up to 85mm equiv focal, but they won't merge correctly in stock darktable. I can provide RAF files but can't attach them here.

I'm not sure I understand you correctly. Did you mean to say that while stock darktable was not able to merge the images correctly, darktable with the changes in this PR was able to deliver a good HDR merge result (which I really hope 😄)?

Correct, while approximately aligned they are not pixel-level aligned and had to be aligned in an external program.

[TurboGit]

@da-phil : I don't have images to test this feature as I have never done HDR :) Do you have some images to share for testing purpose? Otherwise I'll take some in the coming days. TIA.

[TurboGit]

BTW, sounds like a very nice feature.

[da-phil]

@da-phil : I don't have images to test this feature as I have never done HDR :) Do you have some images to share for testing purpose? Otherwise I'll take some in the coming days. TIA.

All images I have tested so far are part of my portfolio, so I'm hesitant to share them, I need to look if I can find some handheld exposure bracket test shots which I can share...

[OuraN2O]

@TurboGit I can share some of my shots for testing if you want. Tell me how you want me to send them to you

[TurboGit]

@OuraN2O : Send me a link from any upload service to pascal@obry.net. TIA.

[TurboGit]

I had not all the OpenCV lib installed the auto-alignment was OFF (the HDR had ghosting images) but I still had the HDR alignment options in the preferences. Should not be there if the support is not active.

[TurboGit]

After installing the libraries it works fine on my side.