rawspeed: add pin-reproduction and upstream-split tooling - #1
Draft
MaykThewessen wants to merge 1 commit into
Draft
rawspeed: add pin-reproduction and upstream-split tooling#1MaykThewessen wants to merge 1 commit into
MaykThewessen wants to merge 1 commit into
Conversation
The rawspeed submodule is pinned to 2c3dfc5779, which carries the LJpeg predictor-mode work from darktable-org/rawspeed#963 plus a local JPEG XL (DNG 1.7) decompressor. That commit is not reachable from any branch of darktable-org/rawspeed, so a fresh clone of this fork cannot initialise the submodule. Commit e89eb1c noted the problem but left no way to recover. Add tools/rawspeed_proraw/ with two aids, neither of which is part of the darktable build: pin/ A 10-patch series plus rebuild_pin.sh that reconstructs the pinned tree from public refs only: upstream refs/heads/stable (4c511d61) plus the series. Verified to produce tree 099b577e, byte-for-byte identical to the pinned commit's tree; the script fails loudly if it does not. The rebuilt commit SHA differs, since committer identity and timestamps differ, so the README also documents pushing to a fork and repointing the submodule as the real fix. upstream_split/ The same work split into five series that each apply to upstream develop (c835b05a) with zero conflicts and each compile and link on their own: predictor modes, the UniqueCameraModel DNG tag, the 12-bit lossy-JPEG error message, the non-LJpeg analyzer fixes, and JPEG XL. Original authorship is preserved in every patch. Records one trap found while splitting: upstream cf87137 cherry-picks onto develop without conflict but does not compile there, because one hunk validates a predictorMode member that only the predictor-mode work adds ("'predictorMode' was not declared in this scope"). A naive commit-by-commit split looks clean and is broken, so that hunk is folded into the predictor series. Series 05 builds against libjxl 0.7.0, so JPEG XL does not require the 0.11.2 recorded in the earlier bump notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NUwVtz47jcvsuPy17tLUQu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The rawspeed submodule is pinned to
2c3dfc5779, which carries the LJpeg predictor-mode work from darktable-org/rawspeed#963 plus a local JPEG XL (DNG 1.7) decompressor.That commit is not reachable from any branch of darktable-org/rawspeed, so a fresh clone of this fork cannot run
git submodule update --init src/external/rawspeed. Commit e89eb1c noted the problem but left no way to recover from it.Separately, darktable-org#963 is stalled upstream: the code has been stable for over a month and the only red CI is pre-existing infra noise, but the maintainer is focused on a Rust port and is not currently merging features. Splitting it into smaller reviews is the most actionable path.
This PR adds
tools/rawspeed_proraw/to address both. Nothing here is part of the darktable build; these are maintenance aids.What changed
pin/: reproduce the pinned tree from public refs onlyA 10-patch series plus
rebuild_pin.sh, which rebuilds from upstreamrefs/heads/stable(4c511d61, published) plus the series.099b577e, byte-for-byte identical to the pinned commit's tree, andgit diffagainst the pin is empty.--initwork for a stranger.upstream_split/: darktable-org#963 carved into five independently reviewable seriesEach applies to upstream
develop(c835b05a) with zero conflicts, and each compiles and links on its own. Original authorship (Philipp Lutz) is preserved in every patch.01_ljpeg_predictor_modes02_dng_unique_camera_modelUniqueCameraModelExif tag for DNG03_dng_12bit_jpeg_errmsg04_analyzer_hardening05_jpegxl_dng17Series 02 to 04 are single small commits. Landing those first shrinks darktable-org#963 to the part that genuinely needs decoder expertise. Series 05 is not part of darktable-org#963; it is separate local work listed because it shares the pin.
One trap worth flagging upstream
Upstream
cf87137("Address clang-tidy, sanitizer and fuzzer findings") looks self-contained, but one hunk validates apredictorModemember that only the predictor-mode work introduces.LJpegDecompressorisfinaland does not inheritAbstractLJpegDecoder, so that member does not exist ondevelop.It cherry-picks onto
developwithout conflict and then fails to compile:So a naive commit-by-commit split of darktable-org#963 looks clean and is broken. This split folds that hunk into the predictor series and keeps the other three hunks, which touch no LJpeg code, in
04_analyzer_hardening.Verification
Configured with CMake + Ninja,
Release, testing/benchmarking/tools/fuzzers off, built to completion:-DWITH_JPEGXL=ONAlso confirmed the negative case above actually fails to compile, rather than assuming it.
Series 05 builds against libjxl 0.7.0, so JPEG XL does not require the 0.11.2 recorded in the earlier bump notes.
Not covered: no raw sample corpus was decoded here, so this verifies compilation and separability, not decode correctness. Decode correctness for predictor modes 1 to 7 and for ProRAW was checked separately on real iPhone 12 / 15 Pro Max files and reported in the darktable-org#963 thread.
Note
The submodule pin itself is unchanged by this PR.
🤖 Generated with Claude Code
https://claude.ai/code/session_01NUwVtz47jcvsuPy17tLUQu
Generated by Claude Code