diff --git a/tools/rawspeed_proraw/README.md b/tools/rawspeed_proraw/README.md new file mode 100644 index 000000000000..110fa8c85151 --- /dev/null +++ b/tools/rawspeed_proraw/README.md @@ -0,0 +1,81 @@ +# rawspeed ProRAW / predictor-mode support + +This directory holds two related things: + +1. `pin/`: how to reproduce the exact rawspeed source tree that + `src/external/rawspeed` is pinned to, using only refs that + darktable-org/rawspeed publishes. +2. `upstream_split/`: the same work carved into five independent series that + can each go upstream as a small pull request. + +Nothing here is built as part of darktable. These are maintenance aids. + +## Background + +darktable's rawspeed submodule is pinned to a commit that carries: + +- darktable-org/rawspeed#963 (LJpeg predictor modes 2 to 7), which unblocks + Apple ProRAW DNGs and DJI / Blackmagic files that use predictor mode 6. +- A libjxl-backed JPEG XL decompressor (DNG 1.7, compression 52546) for + iPhone 16 ProRAW. This is local work, not part of #963. + +Upstream #963 is still open. The maintainer is focused on a Rust port and is +not currently merging features, so the pin is expected to live here a while. + +## 1. Reproducing the pin (`pin/`) + +### The problem + +The pinned commit `2c3dfc5779b604b647956ef2f4c292e943ab1d79` is not reachable +from any branch of darktable-org/rawspeed. It was built locally. So a fresh +clone of this fork cannot do: + +``` +git submodule update --init src/external/rawspeed +``` + +The commit that `git submodule` wants simply is not published anywhere. + +### The fix + +`pin/rebuild_pin.sh` reconstructs the tree from public refs only: + +- base: `refs/heads/stable` = `4c511d611c1beee9aa97a2ec50b0838c6c7be52e` + (published by upstream) +- plus the 10 patches in `pin/`, applied with `git am` + +``` +tools/rawspeed_proraw/pin/rebuild_pin.sh +``` + +The rebuilt commit SHA will not equal the recorded pin, because committer +identity and timestamps differ. The *tree* is byte-for-byte identical, and +that is what the compiler sees. The script verifies this and fails loudly if +the tree hash does not come out as +`099b577e836a53d8e5de87149aa0ea13b20130a1`. + +Verified: applying the series to public `stable` yields exactly the pinned +tree, and `git diff` against the pinned commit is empty. + +### Making it reproducible for everyone + +The script gets a *contributor* to a working tree. It does not make +`git submodule update --init` work for a stranger, because the pinned SHA is +still unpublished. To close that gap properly, push the rebuilt commit to a +fork you control and repoint the submodule: + +``` +# in src/external/rawspeed, after running rebuild_pin.sh +git push git@github.com:/rawspeed.git HEAD:refs/heads/proraw_predictor7_on_v3.6 +``` + +Then update `.gitmodules` to that fork's URL and record the pushed SHA with +`git add src/external/rawspeed`. After that a fresh clone initialises without +any of this. + +Until then, note that the recorded pin is only resolvable by someone who +already has the objects, which is the gap this directory documents. + +## 2. Upstream split (`upstream_split/`) + +See `upstream_split/README.md`. diff --git a/tools/rawspeed_proraw/pin/0001-LJpeg-support-predictor-modes-2-7.patch b/tools/rawspeed_proraw/pin/0001-LJpeg-support-predictor-modes-2-7.patch new file mode 100644 index 000000000000..1a3571ed8330 --- /dev/null +++ b/tools/rawspeed_proraw/pin/0001-LJpeg-support-predictor-modes-2-7.patch @@ -0,0 +1,396 @@ +From 946aa890a95999e836a6985f19028b3499221d55 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 11:43:10 +0200 +Subject: [PATCH 01/10] =?UTF-8?q?LJpeg:=20support=20predictor=20modes=202?= + =?UTF-8?q?=E2=80=937?= +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Summary: + +rawspeed's LJpeg decoder previously supported only predictor mode 1 (left neighbor). ITU-T T.81 defines seven predictor modes, and several camera manufacturers — notably DJI (Mavic 3S, Mavic 3 Pro, etc.) and Blackmagic — use mode 6 in their DNG files. This change adds support for all seven modes and handles the associated tile geometry those cameras use. + +Predictor modes 2–7 + +* Added a computePrediction(mode, Ra, Rb, Rc) helper that computes the seven ITU-T T.81 predictions. Arithmetic is done in int32_t to avoid overflow in modes 4–6, where Ra + Rb - Rc can transiently exceed the 16-bit range; the final result wraps via uint16_t cast per the standard. +* decodeRowN() is extended to accept predMode and prevStripe. For mode 1, the existing fast path (no memory access to the previous row) is preserved. For modes 2–7, the three 2D neighbors (Ra = left, Rb = above, Rc = above-left) are looked up from prevStripe; at the start of a row the above-left neighbor is not available, so Rc = Rb per the JPEG specification. +* decodeN() now tracks isFirstRow per restart interval. The first row of each interval always uses predictor mode 1 (per T.81: the standard mandates horizontal prediction for the first row). For subsequent rows, prevStripe is a CroppedArray2DRef into the already-decoded portion of the output image. No extra allocation is needed. +Inverted tile reshape (LJpegDecoder) + +Inverted tile reshape + +DJI DNGs present tiles in a non-standard geometry: the JPEG SOF3 frame is wider than the declared tile (e.g. 8000×1500 for a 4000×3000 tile). Each JPEG row encodes two tile rows concatenated side-by-side — a widthPack = jpegFrameDim.x / tileW packing. This is the inverse of the Adobe-style reshape rawspeed already handles (MCU > 1×1). + +Design trade-offs: + +Separate decode-then-deinterleave vs. in-place: Reusing the existing LJpegDecompressor in-place would require threading the tile geometry inversion through all MCU addressing logic. Instead, the inverted-reshape path decodes into a temporary RawImage at the JPEG frame dimensions, then copies/deinterleaves rows into the real output image. The extra allocation is bounded by a single tile's worth of data and avoids touching the hot-path decompressor. +widthPack validation: The factor is computed from the ratio jpegFrameDim.x / maxRes.x and validated against widthPack * jpegFrameDim.y == maxRes.y before decoding begins. An upper bound of 4 is enforced as a sanity check. +The inverted reshape is restricted to single-component LJpeg (N_COMP == 1), which is the only case seen in practice for this layout. +Fuzz harness + +The LJpegDecompressor fuzz entry point is updated to source a predictorMode byte from the fuzzer input, covering the new code paths. + +(cherry picked from commit cfa4429965054d698c003eca721dad9af05b9b2d) +--- + .../decompressors/LJpegDecompressor.cpp | 3 +- + .../decompressors/LJpegDecoder.cpp | 115 +++++++++++++++--- + .../decompressors/LJpegDecompressor.cpp | 87 ++++++++++++- + .../decompressors/LJpegDecompressor.h | 3 + + 4 files changed, 189 insertions(+), 19 deletions(-) + +diff --git a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +index aa7b7154..83e406ab 100644 +--- a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -88,10 +88,11 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) { + }); + + const int numLJpegRowsPerRestartInterval = bs.getI32(); ++ const int predictorMode = bs.getByte(); + + rawspeed::LJpegDecompressor d( + mRaw, rawspeed::iRectangle2D(mRaw->dim.x, mRaw->dim.y), frame, rec, +- numLJpegRowsPerRestartInterval, ++ numLJpegRowsPerRestartInterval, predictorMode, + bs.getSubStream(/*offset=*/0).peekRemainingBuffer().getAsArray1DRef()); + mRaw->createData(); + (void)d.decode(); +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index a4efc03b..b4fad06b 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -32,8 +32,10 @@ + #include + #include + #include ++#include + #include + #include ++#include + #include + + using std::copy_n; +@@ -104,7 +106,7 @@ void LJpegDecoder::decode(uint32_t offsetX, uint32_t offsetY, uint32_t width, + Buffer::size_type LJpegDecoder::decodeScan() { + invariant(frame.cps > 0); + +- if (predictorMode != 1) ++ if (predictorMode < 1 || predictorMode > 7) + ThrowRDE("Unsupported predictor mode: %u", predictorMode); + + for (uint32_t i = 0; i < frame.cps; i++) +@@ -123,9 +125,6 @@ Buffer::size_type LJpegDecoder::decodeScan() { + return {*hts[i], initPred[i]}; + }); + +- const iRectangle2D imgFrame = { +- {static_cast(offX), static_cast(offY)}, +- {static_cast(w), static_cast(h)}}; + const auto jpegFrameDim = iPoint2D(frame.w, frame.h); + + if (implicit_cast(maxDim.x) * implicit_cast(mRaw->getCpp()) > +@@ -137,31 +136,119 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (maxRes.area() != N_COMP * jpegFrameDim.area()) + ThrowRDE("LJpeg frame area does not match maximal tile area"); + +- if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) +- ThrowRDE("Maximal output tile size is not a multiple of LJpeg frame size"); ++ // Detect whether the JPEG frame uses an inverted reshape (e.g. DJI/Blackmagic ++ // CinemaDNG): JPEG frame is wider than tile and shorter, with packed rows. ++ // Standard (Adobe): maxRes.x >= jpegFrameDim.x (tile is wider/equal) ++ // Inverted (DJI): jpegFrameDim.x > maxRes.x (JPEG frame is wider) ++ bool invertedReshape = (jpegFrameDim.x > maxRes.x); ++ ++ if (!invertedReshape) { ++ // Standard case: tile width is a multiple of JPEG frame width. ++ if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) ++ ThrowRDE( ++ "Maximal output tile size is not a multiple of LJpeg frame size"); ++ ++ auto MCUSize = ++ iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; ++ if (MCUSize.area() != implicit_cast(N_COMP)) ++ ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); ++ ++ const iRectangle2D imgFrame = { ++ {static_cast(offX), static_cast(offY)}, ++ {static_cast(w), static_cast(h)}}; ++ const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; ++ ++ int numLJpegRowsPerRestartInterval; ++ if (numMCUsPerRestartInterval == 0) { ++ numLJpegRowsPerRestartInterval = jpegFrameDim.y; ++ } else { ++ const int numMCUsPerRow = jpegFrameDim.x; ++ if (numMCUsPerRestartInterval % numMCUsPerRow != 0) ++ ThrowRDE("Restart interval is not a multiple of frame row size"); ++ numLJpegRowsPerRestartInterval = ++ numMCUsPerRestartInterval / numMCUsPerRow; ++ } ++ ++ LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, ++ numLJpegRowsPerRestartInterval, ++ implicit_cast(predictorMode), ++ input.peekRemainingBuffer().getAsArray1DRef()); ++ return d.decode(); ++ } ++ ++ // Inverted reshape case (DJI/Blackmagic CinemaDNG): ++ // JPEG frame is wider than tile, e.g. JPEG=8000x1500 1-comp, tile=4000x3000. ++ // Each JPEG row contains 'widthPack' tile rows concatenated. ++ if (N_COMP != 1) ++ ThrowRDE("Inverted reshape only supported for single-component LJpeg"); ++ ++ if (jpegFrameDim.x % maxRes.x != 0) ++ ThrowRDE("LJpeg frame width is not a multiple of tile width"); ++ if (maxRes.y % jpegFrameDim.y != 0) ++ ThrowRDE("Tile height is not a multiple of LJpeg frame height"); ++ ++ const int widthPack = jpegFrameDim.x / maxRes.x; ++ if (widthPack * jpegFrameDim.y != maxRes.y) ++ ThrowRDE("Inverted reshape dimensions mismatch"); + +- auto MCUSize = iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; +- if (MCUSize.area() != implicit_cast(N_COMP)) +- ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); ++ if (widthPack < 1 || widthPack > 4) ++ ThrowRDE("Unexpected row packing factor: %d", widthPack); + ++ // Decode into a temporary buffer at JPEG frame dimensions. ++ // MCU is {1,1} since we have a single component. ++ const auto MCUSize = iPoint2D{1, 1}; ++ ++ // Create a temporary raw image to decode the JPEG into. ++ // RawImage::create with dimensions already calls createData() internally. ++ RawImage tmpRaw = RawImage::create( ++ iPoint2D(jpegFrameDim.x, jpegFrameDim.y), RawImageType::UINT16, 1); ++ ++ const iRectangle2D tmpFrame = { ++ {0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; + const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; + + int numLJpegRowsPerRestartInterval; + if (numMCUsPerRestartInterval == 0) { +- // Restart interval not enabled, so all of the rows +- // are contained in the first (implicit) restart interval. + numLJpegRowsPerRestartInterval = jpegFrameDim.y; + } else { + const int numMCUsPerRow = jpegFrameDim.x; + if (numMCUsPerRestartInterval % numMCUsPerRow != 0) + ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = numMCUsPerRestartInterval / numMCUsPerRow; ++ numLJpegRowsPerRestartInterval = ++ numMCUsPerRestartInterval / numMCUsPerRow; + } + +- LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, ++ LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, + numLJpegRowsPerRestartInterval, ++ implicit_cast(predictorMode), + input.peekRemainingBuffer().getAsArray1DRef()); +- return d.decode(); ++ auto consumed = d.decode(); ++ ++ // Deinterleave: each JPEG row of width (widthPack * tileW) maps to ++ // widthPack consecutive tile rows of width tileW. ++ const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); ++ const auto outData = mRaw->getU16DataAsUncroppedArray2DRef(); ++ ++ const int tileW = implicit_cast(w); ++ const int cpp = implicit_cast(mRaw->getCpp()); ++ const int outRowPixels = cpp * tileW; ++ ++ for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { ++ for (int pack = 0; pack < widthPack; ++pack) { ++ const int tileRow = ++ implicit_cast(offY) + jpegRow * widthPack + pack; ++ if (tileRow >= mRaw->dim.y) ++ continue; ++ const int srcCol = pack * outRowPixels; ++ const int dstCol = cpp * implicit_cast(offX); ++ for (int col = 0; col < outRowPixels && (srcCol + col) < jpegFrameDim.x; ++ ++col) { ++ outData(tileRow, dstCol + col) = tmpData(jpegRow, srcCol + col); ++ } ++ } ++ } ++ ++ return consumed; + } + + } // namespace rawspeed +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index c1361829..33716870 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -53,10 +53,12 @@ LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + Frame frame_, + std::vector rec_, + int numLJpegRowsPerRestartInterval_, ++ int predictorMode_, + Array1DRef input_) + : mRaw(std::move(img)), input(input_), imgFrame(imgFrame_), + frame(std::move(frame_)), rec(std::move(rec_)), +- numLJpegRowsPerRestartInterval(numLJpegRowsPerRestartInterval_) { ++ numLJpegRowsPerRestartInterval(numLJpegRowsPerRestartInterval_), ++ predictorMode(predictorMode_) { + + if (mRaw->getDataType() != RawImageType::UINT16) + ThrowRDE("Unexpected data type (%u)", +@@ -181,9 +183,39 @@ constexpr iPoint2D MCU = {MCUWidth, MCUHeight}; + + } // namespace + ++namespace { ++ ++// Compute the LJpeg prediction value given predictor mode and neighbor values. ++// Ra = left, Rb = above, Rc = above-left. ++// All arithmetic done in int32_t to avoid overflow in modes 4-6. ++// Result is modulo 2^16 per ITU-T T.81. ++inline int computePrediction(int predMode, int Ra, int Rb, int Rc) { ++ switch (predMode) { ++ case 1: ++ return Ra; ++ case 2: ++ return Rb; ++ case 3: ++ return Rc; ++ case 4: ++ return Ra + Rb - Rc; ++ case 5: ++ return Ra + ((Rb - Rc) >> 1); ++ case 6: ++ return Rb + ((Ra - Rc) >> 1); ++ case 7: ++ return (Ra + Rb) >> 1; ++ default: ++ __builtin_unreachable(); ++ } ++} ++ ++} // namespace ++ + template + void LJpegDecompressor::decodeRowN( + Array2DRef outStripe, Array2DRef pred, ++ int predMode, Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const { + invariant(MCUSize.area() == N_COMP); +@@ -207,7 +239,21 @@ void LJpegDecompressor::decodeRowN( + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { + int c = (MCUSize.x * MCURow) + MCUСol; +- int prediction = pred(MCURow, MCUСol); ++ int prediction; ++ if (predMode == 1) { ++ // Fast path for the common case (mode 1 = left neighbor). ++ prediction = pred(MCURow, MCUСol); ++ } else { ++ // For modes 2-7, compute Ra, Rb, Rc. ++ int Ra = pred(MCURow, MCUСol); // left neighbor ++ int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ int stripeRow = MCURow; ++ int Rb = prevStripe(stripeRow, stripeCol); ++ int Rc = (stripeCol >= MCUSize.x) ++ ? prevStripe(stripeRow, stripeCol - MCUSize.x) ++ : Rb; // First column: Rc = Rb ++ prediction = computePrediction(predMode, Ra, Rb, Rc); ++ } + int diff = (static_cast&>(ht[c])) + .decodeDifference(bs); + int pix = prediction + diff; +@@ -230,7 +276,21 @@ void LJpegDecompressor::decodeRowN( + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { + int c = (MCUSize.x * MCURow) + MCUСol; +- int prediction = pred(MCURow, MCUСol); ++ int prediction; ++ if (predMode == 1) { ++ prediction = pred(MCURow, MCUСol); ++ } else { ++ int Ra = pred(MCURow, MCUСol); ++ int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ int stripeRow = MCURow; ++ int Rb = (stripeCol < prevStripe.width()) ++ ? prevStripe(stripeRow, stripeCol) ++ : Ra; ++ int Rc = (stripeCol >= MCUSize.x && stripeCol < prevStripe.width()) ++ ? prevStripe(stripeRow, stripeCol - MCUSize.x) ++ : Rb; ++ prediction = computePrediction(predMode, Ra, Rb, Rc); ++ } + int diff = (static_cast&>(ht[c])) + .decodeDifference(bs); + int pix = prediction + diff; +@@ -284,6 +344,7 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + restartIntervalIndex != numRestartIntervals; ++restartIntervalIndex) { + auto predStorage = getInitialPreds(); + auto pred = Array2DRef(predStorage.data(), MCU.x, MCU.y); ++ bool isFirstRow = true; + + if (restartIntervalIndex != 0) { + auto marker = peekMarker(inputStream); +@@ -321,7 +382,24 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + /*croppedHeight=*/frame.mcu.y) + .getAsArray2DRef(); + +- decodeRowN(outStripe, pred, ht, bs); ++ // For predictor modes 2-7, we need the previous row (stripe). ++ // For the first row of each restart interval, use predictor mode 1 ++ // (per ITU-T T.81: first row always uses horizontal prediction). ++ // For the first row, prevStripe points to outStripe itself (unused ++ // since predMode will be 1). ++ const int predMode = isFirstRow ? 1 : predictorMode; ++ const Array2DRef prevStripe = ++ isFirstRow ++ ? Array2DRef(outStripe) ++ : CroppedArray2DRef( ++ img, ++ /*offsetCols=*/0, ++ /*offsetRows=*/row - frame.mcu.y, ++ /*croppedWidth=*/img.width(), ++ /*croppedHeight=*/frame.mcu.y) ++ .getAsArray2DRef(); ++ ++ decodeRowN(outStripe, pred, predMode, prevStripe, ht, bs); + + // The predictor for the next line is the start of this line. + pred = CroppedArray2DRef(outStripe, +@@ -330,6 +408,7 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + /*croppedWidth=*/MCU.x, + /*croppedHeight=*/MCU.y) + .getAsArray2DRef(); ++ isFirstRow = false; + } + + inputStream.skipBytes(bs.getStreamPosition()); +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index 69c77739..3c1f4352 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -59,6 +59,7 @@ private: + const Frame frame; + const std::vector rec; + const int numLJpegRowsPerRestartInterval; ++ const int predictorMode; + + int numFullMCUs = 0; + int trailingPixels = 0; +@@ -79,6 +80,7 @@ private: + template + __attribute__((always_inline)) inline void decodeRowN( + Array2DRef outStripe, Array2DRef pred, ++ int predMode, Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const; + +@@ -89,6 +91,7 @@ public: + LJpegDecompressor(RawImage img, iRectangle2D imgFrame, Frame frame, + std::vector rec, + int numLJpegRowsPerRestartInterval_, ++ int predictorMode_, + Array1DRef input); + + [[nodiscard]] ByteStream::size_type decode() const; diff --git a/tools/rawspeed_proraw/pin/0002-LJpeg-eliminate-per-pixel-branch.patch b/tools/rawspeed_proraw/pin/0002-LJpeg-eliminate-per-pixel-branch.patch new file mode 100644 index 000000000000..5928dc951bd7 --- /dev/null +++ b/tools/rawspeed_proraw/pin/0002-LJpeg-eliminate-per-pixel-branch.patch @@ -0,0 +1,229 @@ +From 528551a68c47e5511d8ebc8e241349b098b9b106 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 12:00:15 +0200 +Subject: [PATCH 02/10] LJpeg: eliminate per-pixel branch +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Eliminate per-pixel predMode branch (performance) + +decodeRowN gains a bool Use2DPred template parameter. The mode-1 (left-neighbor) path and the 2D-predictor path are now separate compile-time instantiations selected with if constexpr. The runtime if (predMode == 1) that previously executed for every pixel in every tile is gone entirely. Mode-1 files — the overwhelming majority of existing DNG content — are completely unaffected at the generated-code level; the compiler produces the same tight loop as before. For modes 2–7, the computePrediction switch is also folded away since predictorMode is now only evaluated once at the per-row dispatch in decodeN(), not per-pixel. + +The first-column Rc check is also simplified: stripeCol >= MCUSize.x (computed per-pixel) is replaced with mcuIdx > 0 (a loop-index comparison available for free), and the redundant stripeRow alias is removed. + +Replace scalar deinterleave with memcpy (performance) + +In the inverted-reshape path (LJpegDecoder), the per-pixel operator() copy loop is replaced by a single std::memcpy per row-segment. The previous loop also carried a data-dependent bounds check (srcCol + col < jpegFrameDim.x) that blocked auto-vectorisation; the check is provably redundant given the dimension validation already performed, so it is removed. memcpy allows the compiler to emit an optimal vector store. + +(cherry picked from commit 1d3a262388dfb51f79d94b423cc61428bed2c32e) +--- + .../decompressors/LJpegDecoder.cpp | 19 +++-- + .../decompressors/LJpegDecompressor.cpp | 73 +++++++++---------- + .../decompressors/LJpegDecompressor.h | 4 +- + 3 files changed, 46 insertions(+), 50 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index b4fad06b..b8b2145c 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -113,7 +113,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (frame.compInfo[i].superH != 1 || frame.compInfo[i].superV != 1) + ThrowRDE("Unsupported subsampling"); + +- int N_COMP = frame.cps; ++ const int N_COMP = frame.cps; + + std::vector rec; + rec.reserve(N_COMP); +@@ -131,7 +131,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + std::numeric_limits::max()) + ThrowRDE("Maximal output tile is too large"); + +- auto maxRes = ++ const auto maxRes = + iPoint2D(implicit_cast(mRaw->getCpp()) * maxDim.x, maxDim.y); + if (maxRes.area() != N_COMP * jpegFrameDim.area()) + ThrowRDE("LJpeg frame area does not match maximal tile area"); +@@ -140,15 +140,14 @@ Buffer::size_type LJpegDecoder::decodeScan() { + // CinemaDNG): JPEG frame is wider than tile and shorter, with packed rows. + // Standard (Adobe): maxRes.x >= jpegFrameDim.x (tile is wider/equal) + // Inverted (DJI): jpegFrameDim.x > maxRes.x (JPEG frame is wider) +- bool invertedReshape = (jpegFrameDim.x > maxRes.x); +- ++ const bool invertedReshape = (jpegFrameDim.x > maxRes.x); + if (!invertedReshape) { + // Standard case: tile width is a multiple of JPEG frame width. + if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) + ThrowRDE( + "Maximal output tile size is not a multiple of LJpeg frame size"); + +- auto MCUSize = ++ const auto MCUSize = + iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; + if (MCUSize.area() != implicit_cast(N_COMP)) + ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); +@@ -222,7 +221,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + numLJpegRowsPerRestartInterval, + implicit_cast(predictorMode), + input.peekRemainingBuffer().getAsArray1DRef()); +- auto consumed = d.decode(); ++ const auto consumed = d.decode(); + + // Deinterleave: each JPEG row of width (widthPack * tileW) maps to + // widthPack consecutive tile rows of width tileW. +@@ -241,10 +240,10 @@ Buffer::size_type LJpegDecoder::decodeScan() { + continue; + const int srcCol = pack * outRowPixels; + const int dstCol = cpp * implicit_cast(offX); +- for (int col = 0; col < outRowPixels && (srcCol + col) < jpegFrameDim.x; +- ++col) { +- outData(tileRow, dstCol + col) = tmpData(jpegRow, srcCol + col); +- } ++ // Contiguous row-segment copy. Bounds guaranteed by validation: ++ // srcCol + outRowPixels <= widthPack * outRowPixels <= jpegFrameDim.x ++ std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), ++ sizeof(uint16_t) * outRowPixels); + } + } + +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index 33716870..b5f2bcb1 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -212,10 +212,10 @@ inline int computePrediction(int predMode, int Ra, int Rb, int Rc) { + + } // namespace + +-template ++template + void LJpegDecompressor::decodeRowN( + Array2DRef outStripe, Array2DRef pred, +- int predMode, Array2DRef prevStripe, ++ Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const { + invariant(MCUSize.area() == N_COMP); +@@ -238,25 +238,22 @@ void LJpegDecompressor::decodeRowN( + .getAsArray2DRef(); + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { +- int c = (MCUSize.x * MCURow) + MCUСol; ++ const int c = (MCUSize.x * MCURow) + MCUСol; + int prediction; +- if (predMode == 1) { +- // Fast path for the common case (mode 1 = left neighbor). ++ if constexpr (!Use2DPred) { + prediction = pred(MCURow, MCUСol); + } else { +- // For modes 2-7, compute Ra, Rb, Rc. +- int Ra = pred(MCURow, MCUСol); // left neighbor +- int stripeCol = MCUSize.x * mcuIdx + MCUСol; +- int stripeRow = MCURow; +- int Rb = prevStripe(stripeRow, stripeCol); +- int Rc = (stripeCol >= MCUSize.x) +- ? prevStripe(stripeRow, stripeCol - MCUSize.x) +- : Rb; // First column: Rc = Rb +- prediction = computePrediction(predMode, Ra, Rb, Rc); ++ const int Ra = pred(MCURow, MCUСol); ++ const int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ const int Rb = prevStripe(MCURow, stripeCol); ++ const int Rc = (mcuIdx > 0) ++ ? prevStripe(MCURow, stripeCol - MCUSize.x) ++ : Rb; ++ prediction = computePrediction(predictorMode, Ra, Rb, Rc); + } +- int diff = (static_cast&>(ht[c])) +- .decodeDifference(bs); +- int pix = prediction + diff; ++ const int diff = (static_cast&>(ht[c])) ++ .decodeDifference(bs); ++ const int pix = prediction + diff; + outTile(MCURow, MCUСol) = uint16_t(pix); + } + } +@@ -275,29 +272,27 @@ void LJpegDecompressor::decodeRowN( + // We may end up needing just part of last N_COMP pixels. + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { +- int c = (MCUSize.x * MCURow) + MCUСol; ++ const int c = (MCUSize.x * MCURow) + MCUСol; + int prediction; +- if (predMode == 1) { ++ if constexpr (!Use2DPred) { + prediction = pred(MCURow, MCUСol); + } else { +- int Ra = pred(MCURow, MCUСol); +- int stripeCol = MCUSize.x * mcuIdx + MCUСol; +- int stripeRow = MCURow; +- int Rb = (stripeCol < prevStripe.width()) +- ? prevStripe(stripeRow, stripeCol) +- : Ra; +- int Rc = (stripeCol >= MCUSize.x && stripeCol < prevStripe.width()) +- ? prevStripe(stripeRow, stripeCol - MCUSize.x) +- : Rb; +- prediction = computePrediction(predMode, Ra, Rb, Rc); ++ const int Ra = pred(MCURow, MCUСol); ++ const int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ const int Rb = (stripeCol < prevStripe.width()) ++ ? prevStripe(MCURow, stripeCol) ++ : Ra; ++ const int Rc = (mcuIdx > 0 && stripeCol < prevStripe.width()) ++ ? prevStripe(MCURow, stripeCol - MCUSize.x) ++ : Rb; ++ prediction = computePrediction(predictorMode, Ra, Rb, Rc); + } +- int diff = (static_cast&>(ht[c])) +- .decodeDifference(bs); +- int pix = prediction + diff; +- int stripeRow = MCURow; +- int stripeCol = (MCUSize.x * mcuIdx) + MCUСol; ++ const int diff = (static_cast&>(ht[c])) ++ .decodeDifference(bs); ++ const int pix = prediction + diff; ++ const int stripeCol = (MCUSize.x * mcuIdx) + MCUСol; + if (stripeCol < outStripe.width()) +- outStripe(stripeRow, stripeCol) = uint16_t(pix); ++ outStripe(MCURow, stripeCol) = uint16_t(pix); + } + } + ++mcuIdx; // We did just process one more MCU. +@@ -386,8 +381,7 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + // For the first row of each restart interval, use predictor mode 1 + // (per ITU-T T.81: first row always uses horizontal prediction). + // For the first row, prevStripe points to outStripe itself (unused +- // since predMode will be 1). +- const int predMode = isFirstRow ? 1 : predictorMode; ++ // since Use2DPred will be false). + const Array2DRef prevStripe = + isFirstRow + ? Array2DRef(outStripe) +@@ -399,7 +393,10 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + /*croppedHeight=*/frame.mcu.y) + .getAsArray2DRef(); + +- decodeRowN(outStripe, pred, predMode, prevStripe, ht, bs); ++ if (!isFirstRow && predictorMode != 1) ++ decodeRowN(outStripe, pred, prevStripe, ht, bs); ++ else ++ decodeRowN(outStripe, pred, prevStripe, ht, bs); + + // The predictor for the next line is the start of this line. + pred = CroppedArray2DRef(outStripe, +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index 3c1f4352..a35dd79c 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -77,10 +77,10 @@ private: + template + [[nodiscard]] std::array getInitialPreds() const; + +- template ++ template + __attribute__((always_inline)) inline void decodeRowN( + Array2DRef outStripe, Array2DRef pred, +- int predMode, Array2DRef prevStripe, ++ Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const; + diff --git a/tools/rawspeed_proraw/pin/0003-Apply-clang-format.patch b/tools/rawspeed_proraw/pin/0003-Apply-clang-format.patch new file mode 100644 index 000000000000..0482a6e99b8b --- /dev/null +++ b/tools/rawspeed_proraw/pin/0003-Apply-clang-format.patch @@ -0,0 +1,105 @@ +From 6bed940a3bbe5a1685caee10bc98e36b8f7154de Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 12:07:09 +0200 +Subject: [PATCH 03/10] Apply clang-format + +(cherry picked from commit fb89fea813f3ae5d931635c2d0a6c0e5679da471) +--- + .../decompressors/LJpegDecoder.cpp | 13 +++++------ + .../decompressors/LJpegDecompressor.cpp | 22 +++++++++---------- + .../decompressors/LJpegDecompressor.h | 3 +-- + 3 files changed, 16 insertions(+), 22 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index b8b2145c..a42aa8d9 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -199,11 +199,10 @@ Buffer::size_type LJpegDecoder::decodeScan() { + + // Create a temporary raw image to decode the JPEG into. + // RawImage::create with dimensions already calls createData() internally. +- RawImage tmpRaw = RawImage::create( +- iPoint2D(jpegFrameDim.x, jpegFrameDim.y), RawImageType::UINT16, 1); ++ RawImage tmpRaw = RawImage::create(iPoint2D(jpegFrameDim.x, jpegFrameDim.y), ++ RawImageType::UINT16, 1); + +- const iRectangle2D tmpFrame = { +- {0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; ++ const iRectangle2D tmpFrame = {{0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; + const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; + + int numLJpegRowsPerRestartInterval; +@@ -213,8 +212,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + const int numMCUsPerRow = jpegFrameDim.x; + if (numMCUsPerRestartInterval % numMCUsPerRow != 0) + ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = +- numMCUsPerRestartInterval / numMCUsPerRow; ++ numLJpegRowsPerRestartInterval = numMCUsPerRestartInterval / numMCUsPerRow; + } + + LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, +@@ -234,8 +232,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + + for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { + for (int pack = 0; pack < widthPack; ++pack) { +- const int tileRow = +- implicit_cast(offY) + jpegRow * widthPack + pack; ++ const int tileRow = implicit_cast(offY) + jpegRow * widthPack + pack; + if (tileRow >= mRaw->dim.y) + continue; + const int srcCol = pack * outRowPixels; +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index b5f2bcb1..ee389916 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -246,9 +246,8 @@ void LJpegDecompressor::decodeRowN( + const int Ra = pred(MCURow, MCUСol); + const int stripeCol = MCUSize.x * mcuIdx + MCUСol; + const int Rb = prevStripe(MCURow, stripeCol); +- const int Rc = (mcuIdx > 0) +- ? prevStripe(MCURow, stripeCol - MCUSize.x) +- : Rb; ++ const int Rc = ++ (mcuIdx > 0) ? prevStripe(MCURow, stripeCol - MCUSize.x) : Rb; + prediction = computePrediction(predictorMode, Ra, Rb, Rc); + } + const int diff = (static_cast&>(ht[c])) +@@ -383,15 +382,14 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + // For the first row, prevStripe points to outStripe itself (unused + // since Use2DPred will be false). + const Array2DRef prevStripe = +- isFirstRow +- ? Array2DRef(outStripe) +- : CroppedArray2DRef( +- img, +- /*offsetCols=*/0, +- /*offsetRows=*/row - frame.mcu.y, +- /*croppedWidth=*/img.width(), +- /*croppedHeight=*/frame.mcu.y) +- .getAsArray2DRef(); ++ isFirstRow ? Array2DRef(outStripe) ++ : CroppedArray2DRef( ++ img, ++ /*offsetCols=*/0, ++ /*offsetRows=*/row - frame.mcu.y, ++ /*croppedWidth=*/img.width(), ++ /*croppedHeight=*/frame.mcu.y) ++ .getAsArray2DRef(); + + if (!isFirstRow && predictorMode != 1) + decodeRowN(outStripe, pred, prevStripe, ht, bs); +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index a35dd79c..672249c2 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -90,8 +90,7 @@ private: + public: + LJpegDecompressor(RawImage img, iRectangle2D imgFrame, Frame frame, + std::vector rec, +- int numLJpegRowsPerRestartInterval_, +- int predictorMode_, ++ int numLJpegRowsPerRestartInterval_, int predictorMode_, + Array1DRef input); + + [[nodiscard]] ByteStream::size_type decode() const; diff --git a/tools/rawspeed_proraw/pin/0004-Fix-MCU-1-2-layout-decoding.patch b/tools/rawspeed_proraw/pin/0004-Fix-MCU-1-2-layout-decoding.patch new file mode 100644 index 000000000000..7cfb15e2a1b5 --- /dev/null +++ b/tools/rawspeed_proraw/pin/0004-Fix-MCU-1-2-layout-decoding.patch @@ -0,0 +1,188 @@ +From ad2ade3248012a7e3f7b306cf4a97e885e942bff Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 13:27:01 +0200 +Subject: [PATCH 04/10] Fix MCU{1,2} layout decoding +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Some DNG files encode 2-component tiles (e.g., 592×158) as JPEG frames with the component dimension packed horizontally into a narrower, taller JPEG (592×79, 2 components). The JPEG SOF reports half the tile height, with each row encoding two consecutive output rows' worth of data — components are always interleaved horizontally, so the effective decoded width per JPEG row is jpegFrameDim.x × N_COMP = 1184. + +LJpegDecompressor.cpp — allow MCU{1,2} construction and dispatch + +The constructor's MCU allowlist and decode()'s dispatch table gain {1,2}. This is needed so the decompressor can be instantiated when the caller passes MCU{1,2} (it will never be used for actual pixel output after the decoder fix below, but the validation and dispatch must not reject it at an early stage). + +LJpegDecoder.cpp — detect vertical MCU and route to the inverted reshape path + +Two complementary changes: + +* In the standard (non-inverted) path, after computing MCUSize = maxRes / jpegFrameDim, gate the direct-decode branch on MCUSize.x >= MCUSize.y. When MCUSize is purely vertical (e.g., {1,2}), fall through instead of decoding in-place. + +* Generalize the inverted reshape path to handle N_COMP > 1: + * effectiveJpegWidth = jpegFrameDim.x × N_COMP replaces jpegFrameDim.x everywhere in width calculations. + * The temporary decode buffer is sized effectiveJpegWidth × jpegFrameDim.y. + * MCU is set to {N_COMP, 1} (horizontal interleaving), not hardcoded {1,1}. + * The N_COMP != 1 guard is removed. + * widthPack is derived from effectiveJpegWidth / maxRes.x, so for the 2-comp example: 1184 / 592 = 2, matching the tile's 158 / 79 = 2× height ratio. + +This correctly reconstructs the full-width, full-height tile from the narrower, shorter JPEG frame, yielding proper CFA Bayer output instead of the full-image purple/orange vertical stripe pattern that occurred when the two components were written to separate rows. + +(cherry picked from commit d8577b1229438acac1afb0c4937e44cf7f04f6a5) +--- + .../decompressors/LJpegDecoder.cpp | 87 +++++++++++-------- + .../decompressors/LJpegDecompressor.cpp | 7 +- + 2 files changed, 54 insertions(+), 40 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index a42aa8d9..082b20e0 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -152,57 +152,68 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (MCUSize.area() != implicit_cast(N_COMP)) + ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); + +- const iRectangle2D imgFrame = { +- {static_cast(offX), static_cast(offY)}, +- {static_cast(w), static_cast(h)}}; +- const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; +- +- int numLJpegRowsPerRestartInterval; +- if (numMCUsPerRestartInterval == 0) { +- numLJpegRowsPerRestartInterval = jpegFrameDim.y; +- } else { +- const int numMCUsPerRow = jpegFrameDim.x; +- if (numMCUsPerRestartInterval % numMCUsPerRow != 0) +- ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = +- numMCUsPerRestartInterval / numMCUsPerRow; ++ // Standard MCU layouts have MCU.x >= MCU.y: {1,1}, {2,1}, {3,1}, ++ // {4,1}, {2,2}. If the MCU is purely vertical (e.g., {1,2}), the ++ // encoder uses horizontal component interleaving with wider effective ++ // JPEG rows that must be reshaped. Fall through to inverted reshape. ++ if (MCUSize.x >= MCUSize.y) { ++ const iRectangle2D imgFrame = { ++ {static_cast(offX), static_cast(offY)}, ++ {static_cast(w), static_cast(h)}}; ++ const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; ++ ++ int numLJpegRowsPerRestartInterval; ++ if (numMCUsPerRestartInterval == 0) { ++ numLJpegRowsPerRestartInterval = jpegFrameDim.y; ++ } else { ++ const int numMCUsPerRow = jpegFrameDim.x; ++ if (numMCUsPerRestartInterval % numMCUsPerRow != 0) ++ ThrowRDE("Restart interval is not a multiple of frame row size"); ++ numLJpegRowsPerRestartInterval = ++ numMCUsPerRestartInterval / numMCUsPerRow; ++ } ++ ++ LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, ++ numLJpegRowsPerRestartInterval, ++ implicit_cast(predictorMode), ++ input.peekRemainingBuffer().getAsArray1DRef()); ++ return d.decode(); + } +- +- LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, +- numLJpegRowsPerRestartInterval, +- implicit_cast(predictorMode), +- input.peekRemainingBuffer().getAsArray1DRef()); +- return d.decode(); + } + +- // Inverted reshape case (DJI/Blackmagic CinemaDNG): +- // JPEG frame is wider than tile, e.g. JPEG=8000x1500 1-comp, tile=4000x3000. +- // Each JPEG row contains 'widthPack' tile rows concatenated. +- if (N_COMP != 1) +- ThrowRDE("Inverted reshape only supported for single-component LJpeg"); +- +- if (jpegFrameDim.x % maxRes.x != 0) +- ThrowRDE("LJpeg frame width is not a multiple of tile width"); ++ // Inverted reshape case: ++ // The effective decoded width per JPEG row exceeds the tile width. ++ // This occurs when: ++ // (a) The JPEG frame is wider than the tile (DJI/Blackmagic CinemaDNG): ++ // e.g., JPEG=8000x1500 1-comp, tile=4000x3000. ++ // (b) Multi-component JPEG with vertical MCU ratio: ++ // e.g., JPEG=592x79 2-comp, tile=592x158 (effectiveWidth=1184). ++ // In both cases, components are interleaved horizontally, and each JPEG row ++ // contains 'widthPack' tile rows of pixel data concatenated. ++ const int effectiveJpegWidth = jpegFrameDim.x * N_COMP; ++ ++ if (effectiveJpegWidth % maxRes.x != 0) ++ ThrowRDE("Effective JPEG width is not a multiple of tile width"); + if (maxRes.y % jpegFrameDim.y != 0) + ThrowRDE("Tile height is not a multiple of LJpeg frame height"); + +- const int widthPack = jpegFrameDim.x / maxRes.x; ++ const int widthPack = effectiveJpegWidth / maxRes.x; + if (widthPack * jpegFrameDim.y != maxRes.y) + ThrowRDE("Inverted reshape dimensions mismatch"); + + if (widthPack < 1 || widthPack > 4) + ThrowRDE("Unexpected row packing factor: %d", widthPack); + +- // Decode into a temporary buffer at JPEG frame dimensions. +- // MCU is {1,1} since we have a single component. +- const auto MCUSize = iPoint2D{1, 1}; ++ // Decode into a temporary buffer at effective decoded dimensions. ++ // Components are always interleaved horizontally: MCU{N_COMP, 1}. ++ const auto MCUSize = iPoint2D{N_COMP, 1}; + + // Create a temporary raw image to decode the JPEG into. + // RawImage::create with dimensions already calls createData() internally. +- RawImage tmpRaw = RawImage::create(iPoint2D(jpegFrameDim.x, jpegFrameDim.y), +- RawImageType::UINT16, 1); ++ RawImage tmpRaw = RawImage::create( ++ iPoint2D(effectiveJpegWidth, jpegFrameDim.y), RawImageType::UINT16, 1); + +- const iRectangle2D tmpFrame = {{0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; ++ const iRectangle2D tmpFrame = {{0, 0}, {effectiveJpegWidth, jpegFrameDim.y}}; + const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; + + int numLJpegRowsPerRestartInterval; +@@ -226,8 +237,8 @@ Buffer::size_type LJpegDecoder::decodeScan() { + const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); + const auto outData = mRaw->getU16DataAsUncroppedArray2DRef(); + +- const int tileW = implicit_cast(w); +- const int cpp = implicit_cast(mRaw->getCpp()); ++ const auto tileW = implicit_cast(w); ++ const auto cpp = implicit_cast(mRaw->getCpp()); + const int outRowPixels = cpp * tileW; + + for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { +@@ -238,7 +249,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + const int srcCol = pack * outRowPixels; + const int dstCol = cpp * implicit_cast(offX); + // Contiguous row-segment copy. Bounds guaranteed by validation: +- // srcCol + outRowPixels <= widthPack * outRowPixels <= jpegFrameDim.x ++ // srcCol + outRowPixels <= widthPack * outRowPixels <= effectiveJpegWidth + std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), + sizeof(uint16_t) * outRowPixels); + } +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index ee389916..5ef2d66f 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -102,8 +102,8 @@ LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + ThrowRDE("Frame has zero size"); + + if (iPoint2D{1, 1} != frame.mcu && iPoint2D{2, 1} != frame.mcu && +- iPoint2D{3, 1} != frame.mcu && iPoint2D{4, 1} != frame.mcu && +- iPoint2D{2, 2} != frame.mcu) ++ iPoint2D{1, 2} != frame.mcu && iPoint2D{3, 1} != frame.mcu && ++ iPoint2D{4, 1} != frame.mcu && iPoint2D{2, 2} != frame.mcu) + ThrowRDE("Unexpected MCU size: {%i, %i}", frame.mcu.x, frame.mcu.y); + + if (rec.size() != static_cast(frame.mcu.area())) +@@ -423,6 +423,9 @@ ByteStream::size_type LJpegDecompressor::decode() const { + if (frame.mcu == MCU<2, 1>) { + return decodeN>(); + } ++ if (frame.mcu == MCU<1, 2>) { ++ return decodeN>(); ++ } + break; + case 3: + if (frame.mcu == MCU<3, 1>) { diff --git a/tools/rawspeed_proraw/pin/0005-Refactor-clean-up-to-satisfy-clang-tidy.patch b/tools/rawspeed_proraw/pin/0005-Refactor-clean-up-to-satisfy-clang-tidy.patch new file mode 100644 index 000000000000..0f36bb199841 --- /dev/null +++ b/tools/rawspeed_proraw/pin/0005-Refactor-clean-up-to-satisfy-clang-tidy.patch @@ -0,0 +1,398 @@ +From 9dde76f09480d3f49cffd4c473b46c5c573be39a Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 13:13:56 +0200 +Subject: [PATCH 05/10] Refactor & clean-up to satisfy clang-tidy + +(cherry picked from commit aeb171913a3d980cd32cd35fbecc395dae7360ed) +--- + .../decompressors/LJpegDecompressor.cpp | 4 +- + .../decompressors/LJpegDecoder.cpp | 289 +++++++++--------- + .../decompressors/LJpegDecompressor.cpp | 7 +- + .../decompressors/LJpegDecompressor.h | 7 +- + 4 files changed, 160 insertions(+), 147 deletions(-) + +diff --git a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +index 83e406ab..2d1b1a63 100644 +--- a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -90,9 +90,11 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) { + const int numLJpegRowsPerRestartInterval = bs.getI32(); + const int predictorMode = bs.getByte(); + ++ const rawspeed::LJpegDecompressor::DecodeSettings settings{ ++ numLJpegRowsPerRestartInterval, predictorMode}; + rawspeed::LJpegDecompressor d( + mRaw, rawspeed::iRectangle2D(mRaw->dim.x, mRaw->dim.y), frame, rec, +- numLJpegRowsPerRestartInterval, predictorMode, ++ settings, + bs.getSubStream(/*offset=*/0).peekRemainingBuffer().getAsArray1DRef()); + mRaw->createData(); + (void)d.decode(); +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index 082b20e0..1e3deb08 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -30,18 +30,139 @@ + #include "io/Buffer.h" + #include "io/ByteStream.h" + #include +-#include + #include + #include + #include + #include +-#include + #include + +-using std::copy_n; +- + namespace rawspeed { + ++namespace { ++ ++using PerCompRecipeVec = std::vector; ++ ++struct ScanSettings final { ++ RawImage raw; ++ iRectangle2D imgFrame; ++ iPoint2D jpegFrameDim; ++ iPoint2D maxRes; ++ LJpegDecompressor::DecodeSettings decode; ++ Array1DRef input; ++}; ++ ++[[nodiscard]] int ++getNumLJpegRowsPerRestartInterval(uint32_t numMCUsPerRestartInterval, ++ iPoint2D jpegFrameDim) { ++ if (numMCUsPerRestartInterval == 0) ++ return jpegFrameDim.y; ++ ++ const int numMCUsPerRow = jpegFrameDim.x; ++ if (numMCUsPerRestartInterval % numMCUsPerRow != 0) ++ ThrowRDE("Restart interval is not a multiple of frame row size"); ++ return implicit_cast(numMCUsPerRestartInterval) / numMCUsPerRow; ++} ++ ++[[nodiscard]] iPoint2D getMaxResolution(const RawImage& raw, iPoint2D maxDim, ++ int numComponents, ++ iPoint2D jpegFrameDim) { ++ if (implicit_cast(maxDim.x) * implicit_cast(raw->getCpp()) > ++ std::numeric_limits::max()) ++ ThrowRDE("Maximal output tile is too large"); ++ ++ const auto maxRes = ++ iPoint2D(implicit_cast(raw->getCpp()) * maxDim.x, maxDim.y); ++ if (maxRes.area() != numComponents * jpegFrameDim.area()) ++ ThrowRDE("LJpeg frame area does not match maximal tile area"); ++ ++ return maxRes; ++} ++ ++[[nodiscard]] iPoint2D ++getStandardMCUSize(int numComponents, iPoint2D jpegFrameDim, iPoint2D maxRes) { ++ if (jpegFrameDim.x > maxRes.x) ++ return {}; ++ ++ if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) ++ ThrowRDE("Maximal output tile size is not a multiple of LJpeg frame size"); ++ ++ const auto mcuSize = ++ iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; ++ if (mcuSize.area() != implicit_cast(numComponents)) ++ ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); ++ ++ if (mcuSize.x < mcuSize.y) ++ return {}; ++ ++ return mcuSize; ++} ++ ++[[nodiscard]] ByteStream::size_type ++decodeStandardScan(const ScanSettings& settings, iPoint2D mcuSize, ++ const PerCompRecipeVec& rec) { ++ const LJpegDecompressor::Frame jpegFrame = {mcuSize, settings.jpegFrameDim}; ++ LJpegDecompressor d(settings.raw, settings.imgFrame, jpegFrame, rec, ++ settings.decode, settings.input); ++ return d.decode(); ++} ++ ++void copyDeinterleavedRows(const RawImage& raw, RawImage tmpRaw, uint32_t offX, ++ uint32_t offY, uint32_t tileWidth, int widthPack) { ++ const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); ++ const auto outData = raw->getU16DataAsUncroppedArray2DRef(); ++ ++ const auto cpp = implicit_cast(raw->getCpp()); ++ const int outRowPixels = cpp * implicit_cast(tileWidth); ++ ++ for (int jpegRow = 0; jpegRow < tmpRaw->dim.y; ++jpegRow) { ++ for (int pack = 0; pack < widthPack; ++pack) { ++ const int tileRow = implicit_cast(offY) + jpegRow * widthPack + pack; ++ if (tileRow >= raw->dim.y) ++ continue; ++ ++ const int srcCol = pack * outRowPixels; ++ const int dstCol = cpp * implicit_cast(offX); ++ std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), ++ sizeof(uint16_t) * outRowPixels); ++ } ++ } ++} ++ ++[[nodiscard]] ByteStream::size_type ++decodeInvertedScan(const ScanSettings& settings, int numComponents, ++ uint32_t offX, uint32_t offY, uint32_t tileWidth, ++ const PerCompRecipeVec& rec) { ++ const int effectiveJpegWidth = settings.jpegFrameDim.x * numComponents; ++ ++ if (effectiveJpegWidth % settings.maxRes.x != 0) ++ ThrowRDE("Effective JPEG width is not a multiple of tile width"); ++ if (settings.maxRes.y % settings.jpegFrameDim.y != 0) ++ ThrowRDE("Tile height is not a multiple of LJpeg frame height"); ++ ++ const int widthPack = effectiveJpegWidth / settings.maxRes.x; ++ if (widthPack * settings.jpegFrameDim.y != settings.maxRes.y) ++ ThrowRDE("Inverted reshape dimensions mismatch"); ++ if (widthPack < 1 || widthPack > 4) ++ ThrowRDE("Unexpected row packing factor: %d", widthPack); ++ ++ const auto mcuSize = iPoint2D{numComponents, 1}; ++ RawImage tmpRaw = ++ RawImage::create(iPoint2D(effectiveJpegWidth, settings.jpegFrameDim.y), ++ RawImageType::UINT16, 1); ++ const iRectangle2D tmpFrame = {{0, 0}, ++ {effectiveJpegWidth, settings.jpegFrameDim.y}}; ++ const LJpegDecompressor::Frame jpegFrame = {mcuSize, settings.jpegFrameDim}; ++ ++ LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, settings.decode, ++ settings.input); ++ const auto consumed = d.decode(); ++ ++ copyDeinterleavedRows(settings.raw, tmpRaw, offX, offY, tileWidth, widthPack); ++ return consumed; ++} ++ ++} // namespace ++ + LJpegDecoder::LJpegDecoder(ByteStream bs, const RawImage& img) + : AbstractLJpegDecoder(bs, img) { + if (mRaw->getDataType() != RawImageType::UINT16) +@@ -113,149 +234,37 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (frame.compInfo[i].superH != 1 || frame.compInfo[i].superV != 1) + ThrowRDE("Unsupported subsampling"); + +- const int N_COMP = frame.cps; ++ const int numComponents = frame.cps; + +- std::vector rec; +- rec.reserve(N_COMP); +- std::generate_n(std::back_inserter(rec), N_COMP, +- [&rec, hts = getPrefixCodeDecoders(N_COMP), +- initPred = getInitialPredictors( +- N_COMP)]() -> LJpegDecompressor::PerComponentRecipe { ++ PerCompRecipeVec rec; ++ rec.reserve(numComponents); ++ std::generate_n(std::back_inserter(rec), numComponents, ++ [&rec, hts = getPrefixCodeDecoders(numComponents), ++ initPred = getInitialPredictors(numComponents)]() ++ -> LJpegDecompressor::PerComponentRecipe { + const auto i = implicit_cast(rec.size()); + return {*hts[i], initPred[i]}; + }); + + const auto jpegFrameDim = iPoint2D(frame.w, frame.h); +- +- if (implicit_cast(maxDim.x) * implicit_cast(mRaw->getCpp()) > +- std::numeric_limits::max()) +- ThrowRDE("Maximal output tile is too large"); +- + const auto maxRes = +- iPoint2D(implicit_cast(mRaw->getCpp()) * maxDim.x, maxDim.y); +- if (maxRes.area() != N_COMP * jpegFrameDim.area()) +- ThrowRDE("LJpeg frame area does not match maximal tile area"); +- +- // Detect whether the JPEG frame uses an inverted reshape (e.g. DJI/Blackmagic +- // CinemaDNG): JPEG frame is wider than tile and shorter, with packed rows. +- // Standard (Adobe): maxRes.x >= jpegFrameDim.x (tile is wider/equal) +- // Inverted (DJI): jpegFrameDim.x > maxRes.x (JPEG frame is wider) +- const bool invertedReshape = (jpegFrameDim.x > maxRes.x); +- if (!invertedReshape) { +- // Standard case: tile width is a multiple of JPEG frame width. +- if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) +- ThrowRDE( +- "Maximal output tile size is not a multiple of LJpeg frame size"); +- +- const auto MCUSize = +- iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; +- if (MCUSize.area() != implicit_cast(N_COMP)) +- ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); +- +- // Standard MCU layouts have MCU.x >= MCU.y: {1,1}, {2,1}, {3,1}, +- // {4,1}, {2,2}. If the MCU is purely vertical (e.g., {1,2}), the +- // encoder uses horizontal component interleaving with wider effective +- // JPEG rows that must be reshaped. Fall through to inverted reshape. +- if (MCUSize.x >= MCUSize.y) { +- const iRectangle2D imgFrame = { +- {static_cast(offX), static_cast(offY)}, +- {static_cast(w), static_cast(h)}}; +- const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; +- +- int numLJpegRowsPerRestartInterval; +- if (numMCUsPerRestartInterval == 0) { +- numLJpegRowsPerRestartInterval = jpegFrameDim.y; +- } else { +- const int numMCUsPerRow = jpegFrameDim.x; +- if (numMCUsPerRestartInterval % numMCUsPerRow != 0) +- ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = +- numMCUsPerRestartInterval / numMCUsPerRow; +- } +- +- LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, +- numLJpegRowsPerRestartInterval, +- implicit_cast(predictorMode), +- input.peekRemainingBuffer().getAsArray1DRef()); +- return d.decode(); +- } +- } +- +- // Inverted reshape case: +- // The effective decoded width per JPEG row exceeds the tile width. +- // This occurs when: +- // (a) The JPEG frame is wider than the tile (DJI/Blackmagic CinemaDNG): +- // e.g., JPEG=8000x1500 1-comp, tile=4000x3000. +- // (b) Multi-component JPEG with vertical MCU ratio: +- // e.g., JPEG=592x79 2-comp, tile=592x158 (effectiveWidth=1184). +- // In both cases, components are interleaved horizontally, and each JPEG row +- // contains 'widthPack' tile rows of pixel data concatenated. +- const int effectiveJpegWidth = jpegFrameDim.x * N_COMP; +- +- if (effectiveJpegWidth % maxRes.x != 0) +- ThrowRDE("Effective JPEG width is not a multiple of tile width"); +- if (maxRes.y % jpegFrameDim.y != 0) +- ThrowRDE("Tile height is not a multiple of LJpeg frame height"); +- +- const int widthPack = effectiveJpegWidth / maxRes.x; +- if (widthPack * jpegFrameDim.y != maxRes.y) +- ThrowRDE("Inverted reshape dimensions mismatch"); +- +- if (widthPack < 1 || widthPack > 4) +- ThrowRDE("Unexpected row packing factor: %d", widthPack); +- +- // Decode into a temporary buffer at effective decoded dimensions. +- // Components are always interleaved horizontally: MCU{N_COMP, 1}. +- const auto MCUSize = iPoint2D{N_COMP, 1}; +- +- // Create a temporary raw image to decode the JPEG into. +- // RawImage::create with dimensions already calls createData() internally. +- RawImage tmpRaw = RawImage::create( +- iPoint2D(effectiveJpegWidth, jpegFrameDim.y), RawImageType::UINT16, 1); +- +- const iRectangle2D tmpFrame = {{0, 0}, {effectiveJpegWidth, jpegFrameDim.y}}; +- const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; +- +- int numLJpegRowsPerRestartInterval; +- if (numMCUsPerRestartInterval == 0) { +- numLJpegRowsPerRestartInterval = jpegFrameDim.y; +- } else { +- const int numMCUsPerRow = jpegFrameDim.x; +- if (numMCUsPerRestartInterval % numMCUsPerRow != 0) +- ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = numMCUsPerRestartInterval / numMCUsPerRow; +- } +- +- LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, +- numLJpegRowsPerRestartInterval, +- implicit_cast(predictorMode), +- input.peekRemainingBuffer().getAsArray1DRef()); +- const auto consumed = d.decode(); +- +- // Deinterleave: each JPEG row of width (widthPack * tileW) maps to +- // widthPack consecutive tile rows of width tileW. +- const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); +- const auto outData = mRaw->getU16DataAsUncroppedArray2DRef(); +- +- const auto tileW = implicit_cast(w); +- const auto cpp = implicit_cast(mRaw->getCpp()); +- const int outRowPixels = cpp * tileW; +- +- for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { +- for (int pack = 0; pack < widthPack; ++pack) { +- const int tileRow = implicit_cast(offY) + jpegRow * widthPack + pack; +- if (tileRow >= mRaw->dim.y) +- continue; +- const int srcCol = pack * outRowPixels; +- const int dstCol = cpp * implicit_cast(offX); +- // Contiguous row-segment copy. Bounds guaranteed by validation: +- // srcCol + outRowPixels <= widthPack * outRowPixels <= effectiveJpegWidth +- std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), +- sizeof(uint16_t) * outRowPixels); +- } +- } +- +- return consumed; ++ getMaxResolution(mRaw, maxDim, numComponents, jpegFrameDim); ++ const ScanSettings settings{mRaw, ++ {{static_cast(offX), static_cast(offY)}, ++ {static_cast(w), static_cast(h)}}, ++ jpegFrameDim, ++ maxRes, ++ {getNumLJpegRowsPerRestartInterval( ++ numMCUsPerRestartInterval, jpegFrameDim), ++ implicit_cast(predictorMode)}, ++ input.peekRemainingBuffer().getAsArray1DRef()}; ++ ++ if (const auto mcuSize = ++ getStandardMCUSize(numComponents, jpegFrameDim, maxRes); ++ mcuSize.hasPositiveArea()) ++ return decodeStandardScan(settings, mcuSize, rec); ++ ++ return decodeInvertedScan(settings, numComponents, offX, offY, w, rec); + } + + } // namespace rawspeed +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index 5ef2d66f..376ddd3a 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -52,13 +52,12 @@ namespace rawspeed { + LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + Frame frame_, + std::vector rec_, +- int numLJpegRowsPerRestartInterval_, +- int predictorMode_, ++ DecodeSettings settings, + Array1DRef input_) + : mRaw(std::move(img)), input(input_), imgFrame(imgFrame_), + frame(std::move(frame_)), rec(std::move(rec_)), +- numLJpegRowsPerRestartInterval(numLJpegRowsPerRestartInterval_), +- predictorMode(predictorMode_) { ++ numLJpegRowsPerRestartInterval(settings.numLJpegRowsPerRestartInterval), ++ predictorMode(settings.predictorMode) { + + if (mRaw->getDataType() != RawImageType::UINT16) + ThrowRDE("Unexpected data type (%u)", +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index 672249c2..11caad31 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -45,6 +45,10 @@ public: + const iPoint2D mcu; + const iPoint2D dim; + }; ++ struct DecodeSettings final { ++ const int numLJpegRowsPerRestartInterval; ++ const int predictorMode; ++ }; + struct PerComponentRecipe final { + const PrefixCodeDecoder<>& ht; + const uint16_t initPred; +@@ -90,8 +94,7 @@ private: + public: + LJpegDecompressor(RawImage img, iRectangle2D imgFrame, Frame frame, + std::vector rec, +- int numLJpegRowsPerRestartInterval_, int predictorMode_, +- Array1DRef input); ++ DecodeSettings settings, Array1DRef input); + + [[nodiscard]] ByteStream::size_type decode() const; + }; diff --git a/tools/rawspeed_proraw/pin/0006-Support-UniqueCameraModel-Exif-tag-for-DNG.patch b/tools/rawspeed_proraw/pin/0006-Support-UniqueCameraModel-Exif-tag-for-DNG.patch new file mode 100644 index 000000000000..78b867edabce --- /dev/null +++ b/tools/rawspeed_proraw/pin/0006-Support-UniqueCameraModel-Exif-tag-for-DNG.patch @@ -0,0 +1,47 @@ +From 11921a6519258d9fedd3313fc968b2247faf6368 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 14:19:22 +0200 +Subject: [PATCH 06/10] Support UniqueCameraModel Exif tag for DNG +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Blackmagic CinemaDNG files don't include TIFF Make/Model tags — they only have UniqueCameraModel (tag 50708), which is valid per the DNG spec. In decodeMetaDataInternal(), the code called getID() unconditionally. + +Instead of calling getID() in a try/catch (which logged a spurious error), check for MAKE+MODEL existence first. If present, call getID() as before. If absent, fall back to UNIQUECAMERAMODEL for both make and model fields — mirroring the approach already used in checkSupportInternal(). + +(cherry picked from commit a9dbfad873923c802a7c156e83899d5a5df19019) +--- + src/librawspeed/decoders/DngDecoder.cpp | 18 +++++++++++++----- + 1 file changed, 13 insertions(+), 5 deletions(-) + +diff --git a/src/librawspeed/decoders/DngDecoder.cpp b/src/librawspeed/decoders/DngDecoder.cpp +index ecd11989..ae972d2a 100644 +--- a/src/librawspeed/decoders/DngDecoder.cpp ++++ b/src/librawspeed/decoders/DngDecoder.cpp +@@ -699,12 +699,20 @@ void DngDecoder::decodeMetaDataInternal(const CameraMetaData* meta) { + + TiffID id; + +- try { ++ if (mRootIFD->hasEntryRecursive(TiffTag::MAKE) && ++ mRootIFD->hasEntryRecursive(TiffTag::MODEL)) { + id = mRootIFD->getID(); +- } catch (const RawspeedException& e) { +- mRaw->setError(e.what()); +- // not all dngs have MAKE/MODEL entries, +- // will be dealt with by using UNIQUECAMERAMODEL below ++ } else if (mRootIFD->hasEntryRecursive(TiffTag::UNIQUECAMERAMODEL)) { ++ // Not all DNGs have MAKE/MODEL entries (e.g. Blackmagic CinemaDNG). ++ // Fall back to UNIQUECAMERAMODEL for identification. ++ std::string unique = ++ mRootIFD->getEntryRecursive(TiffTag::UNIQUECAMERAMODEL)->getString(); ++ if (unique.empty()) ++ ThrowRDE("UNIQUECAMERAMODEL is empty"); ++ id.make = unique; ++ id.model = unique; ++ } else { ++ ThrowRDE("DNG has neither MAKE/MODEL nor UNIQUECAMERAMODEL"); + } + + // Set the make and model diff --git a/tools/rawspeed_proraw/pin/0007-Address-codeChecker-findings.patch b/tools/rawspeed_proraw/pin/0007-Address-codeChecker-findings.patch new file mode 100644 index 000000000000..67e4ddf88a35 --- /dev/null +++ b/tools/rawspeed_proraw/pin/0007-Address-codeChecker-findings.patch @@ -0,0 +1,34 @@ +From ed6e667346b1bf41ca1e76bac9b0ee8230ee09b9 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 14:34:47 +0200 +Subject: [PATCH 07/10] Address codeChecker findings + +(cherry picked from commit a167a44f3a5034725be51b941bff7e00e7f751aa) +--- + src/librawspeed/decompressors/LJpegDecoder.cpp | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index 1e3deb08..31d3b870 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -20,6 +20,7 @@ + */ + + #include "decompressors/LJpegDecoder.h" ++#include "adt/Array1DRef.h" + #include "adt/Casts.h" + #include "adt/Invariant.h" + #include "adt/Point.h" +@@ -106,8 +107,9 @@ decodeStandardScan(const ScanSettings& settings, iPoint2D mcuSize, + return d.decode(); + } + +-void copyDeinterleavedRows(const RawImage& raw, RawImage tmpRaw, uint32_t offX, +- uint32_t offY, uint32_t tileWidth, int widthPack) { ++void copyDeinterleavedRows(const RawImage& raw, const RawImage& tmpRaw, ++ uint32_t offX, uint32_t offY, uint32_t tileWidth, ++ int widthPack) { + const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); + const auto outData = raw->getU16DataAsUncroppedArray2DRef(); + diff --git a/tools/rawspeed_proraw/pin/0008-Add-more-meaningful-error-message-for-12-bit-JPG-fil.patch b/tools/rawspeed_proraw/pin/0008-Add-more-meaningful-error-message-for-12-bit-JPG-fil.patch new file mode 100644 index 000000000000..f590a7253cef --- /dev/null +++ b/tools/rawspeed_proraw/pin/0008-Add-more-meaningful-error-message-for-12-bit-JPG-fil.patch @@ -0,0 +1,117 @@ +From 2840416e1715a295ba11757e87d997031c4591be Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 16:25:44 +0200 +Subject: [PATCH 08/10] Add more meaningful error message for 12-bit JPG files + +Some Blackmagic CinemaDNG files (Pocket Cinema Camera 4K files (3)/(4), Micro Cinema Camera (1)) use SOF1 (Extended Sequential DCT) at 12-bit precision but label tiles as TIFF compression=7 (lossless JPEG). The lossless JPEG decoder hit a DQT marker and threw "Not a valid RAW file." + +On libjpeg-turbo 2.1.5, the error is now "Unsupported JPEG data precision 12" (much clearer). + +Future work: on systems with libjpeg-turbo 3.0+, we can enable full 12-bit lossy JPEG decoding. + +(cherry picked from commit 8e10cee704b021a6a092da142cf0215ea7aab38c) +--- + .../decompressors/AbstractDngDecompressor.cpp | 67 +++++++++++++++++++ + .../decompressors/JpegDecompressor.cpp | 4 ++ + 2 files changed, 71 insertions(+) + +diff --git a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +index b828a4fe..c68044d9 100644 +--- a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp ++++ b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +@@ -51,6 +51,64 @@ + + namespace rawspeed { + ++namespace { ++ ++// Some DNG files (e.g. Blackmagic CinemaDNG) use TIFF compression=7 ++// (lossless JPEG) but the actual tile data contains lossy DCT JPEG ++// (SOF0/SOF1/SOF2 with DQT). Detect this by scanning the first few ++// JPEG markers in the tile stream. ++[[nodiscard]] bool tileContainsLossyJpeg(const ByteStream& bs) { ++ const auto remaining = bs.getRemainSize(); ++ if (remaining < 4) ++ return false; ++ ++ // Must start with JPEG SOI marker ++ if (bs.peekByte(0) != 0xFF || bs.peekByte(1) != 0xD8) ++ return false; ++ ++ // Scan markers after SOI. Stop after a reasonable number of bytes. ++ const auto limit = ++ std::min(remaining, static_cast(1024)); ++ ByteStream::size_type pos = 2; ++ ++ while (pos + 3 < limit) { ++ if (bs.peekByte(pos) != 0xFF) ++ return false; // Invalid marker - stop scanning ++ ++ const uint8_t marker = bs.peekByte(pos + 1); ++ ++ // DQT (quantization table) is definitive proof of lossy JPEG ++ if (marker == 0xDB) // DQT ++ return true; ++ ++ // SOF0/SOF1/SOF2 = lossy DCT-based JPEG ++ if (marker == 0xC0 || marker == 0xC1 || marker == 0xC2) ++ return true; ++ ++ // SOF3 = lossless - this is what compression=7 should be ++ if (marker == 0xC3) ++ return false; ++ ++ // SOS = start of scan data - stop scanning ++ if (marker == 0xDA) ++ return false; ++ ++ // Skip this marker segment ++ if (pos + 4 > remaining) ++ return false; ++ const auto segLen = static_cast( ++ (static_cast(bs.peekByte(pos + 2)) << 8) | ++ static_cast(bs.peekByte(pos + 3))); ++ if (segLen < 2) ++ return false; ++ pos += 2 + segLen; ++ } ++ ++ return false; ++} ++ ++} // namespace ++ + template <> void AbstractDngDecompressor::decompressThread<1>() const noexcept { + #ifdef HAVE_OPENMP + #pragma omp for schedule(static) +@@ -116,6 +174,15 @@ template <> void AbstractDngDecompressor::decompressThread<7>() const noexcept { + for (const auto& e : + Array1DRef(slices.data(), implicit_cast(slices.size()))) { + try { ++#ifdef HAVE_JPEG ++ // Some cameras (e.g. Blackmagic CinemaDNG) mislabel lossy DCT JPEG ++ // tiles as compression=7 (lossless JPEG). Detect and redirect. ++ if (tileContainsLossyJpeg(e.bs)) { ++ JpegDecompressor j(e.bs.peekBuffer(e.bs.getRemainSize()), mRaw); ++ j.decode(e.offX, e.offY); ++ continue; ++ } ++#endif + LJpegDecoder d(e.bs, mRaw); + d.decode(e.offX, e.offY, e.width, e.height, + iPoint2D(e.dsc.tileW, e.dsc.tileH), mFixLjpeg); +diff --git a/src/librawspeed/decompressors/JpegDecompressor.cpp b/src/librawspeed/decompressors/JpegDecompressor.cpp +index 569bb037..a0a4da39 100644 +--- a/src/librawspeed/decompressors/JpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/JpegDecompressor.cpp +@@ -139,6 +139,10 @@ void JpegDecompressor::decode(uint32_t offX, + if (JPEG_HEADER_OK != jpeg_read_header(&dinfo, static_cast(true))) + ThrowRDE("Unable to read JPEG header"); + ++ if (dinfo.data_precision != 8) ++ ThrowRDE("Lossy JPEG tiles with %d-bit precision are not yet supported.", ++ dinfo.data_precision); ++ + jpeg_start_decompress(&dinfo); + if (dinfo.output_components != static_cast(mRaw->getCpp())) + ThrowRDE("Component count doesn't match"); diff --git a/tools/rawspeed_proraw/pin/0009-Backport-rawspeed-963-cf87137-clang-tidy-sanitizer-f.patch b/tools/rawspeed_proraw/pin/0009-Backport-rawspeed-963-cf87137-clang-tidy-sanitizer-f.patch new file mode 100644 index 000000000000..ecbbd0fcbc51 --- /dev/null +++ b/tools/rawspeed_proraw/pin/0009-Backport-rawspeed-963-cf87137-clang-tidy-sanitizer-f.patch @@ -0,0 +1,83 @@ +From a15b9a2ebca98fff395a2a9fb94a65770d01f025 Mon Sep 17 00:00:00 2001 +From: Mayk Thewessen +Date: Sun, 21 Jun 2026 22:15:14 +0200 +Subject: [PATCH 09/10] Backport rawspeed#963 cf87137: + clang-tidy/sanitizer/fuzzer fixes + +Manually applies upstream PR darktable-org/rawspeed#963 commit cf87137 +onto this v3.6 backport (a raw cherry-pick conflicts: different base +plus local JPEG XL WIP in AbstractDngDecompressor.cpp). + +- LJpegDecompressor.cpp: reject predictor modes outside 1-7 (fuzzer + hardening for the widened predictor set) +- SimpleTiffDecoder.h: initialize raw/width/height/off/c2 in ctor +- AbstractDngDecompressor.cpp: add missing include +- FileReader.cpp: typed fn-pointer deleter plus codechecker + false-positive marker for unix.Stream +--- + src/librawspeed/decoders/SimpleTiffDecoder.h | 3 ++- + .../decompressors/AbstractDngDecompressor.cpp | 1 + + src/librawspeed/decompressors/LJpegDecompressor.cpp | 3 +++ + src/librawspeed/io/FileReader.cpp | 9 ++++++--- + 4 files changed, 12 insertions(+), 4 deletions(-) + +diff --git a/src/librawspeed/decoders/SimpleTiffDecoder.h b/src/librawspeed/decoders/SimpleTiffDecoder.h +index 1fa79fdc..e08b5e31 100644 +--- a/src/librawspeed/decoders/SimpleTiffDecoder.h ++++ b/src/librawspeed/decoders/SimpleTiffDecoder.h +@@ -39,7 +39,8 @@ class SimpleTiffDecoder : public AbstractTiffDecoder { + + public: + SimpleTiffDecoder(TiffRootIFDOwner&& root, Buffer file) +- : AbstractTiffDecoder(std::move(root), file) {} ++ : AbstractTiffDecoder(std::move(root), file), raw(nullptr), width(0), ++ height(0), off(0), c2(0) {} + + void prepareForRawDecoding(); + +diff --git a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +index c68044d9..6ba745b6 100644 +--- a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp ++++ b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +@@ -35,6 +35,7 @@ + #include "io/ByteStream.h" + #include "io/Endianness.h" + #include "io/IOException.h" ++#include + #include + #include + #include +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index 376ddd3a..57b36f7c 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -116,6 +116,9 @@ LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + if (numLJpegRowsPerRestartInterval < 1) + ThrowRDE("Number of rows per restart interval must be positives"); + ++ if (predictorMode < 1 || predictorMode > 7) ++ ThrowRDE("Unsupported predictor mode: %i", predictorMode); ++ + if (static_cast(frame.mcu.x) * frame.dim.x > + std::numeric_limits::max() || + static_cast(frame.mcu.y) * frame.dim.y > +diff --git a/src/librawspeed/io/FileReader.cpp b/src/librawspeed/io/FileReader.cpp +index 2378ba4c..79d0f18f 100644 +--- a/src/librawspeed/io/FileReader.cpp ++++ b/src/librawspeed/io/FileReader.cpp +@@ -53,9 +53,12 @@ FileReader::readFile() const { + size_t fileSize = 0; + + #if defined(__unix__) || defined(__APPLE__) +- auto fclose = [](std::FILE* fp) { std::fclose(fp); }; +- using file_ptr = std::unique_ptr; +- file_ptr file(fopen(fileName, "rb"), fclose); ++ using file_ptr = std::unique_ptr; ++ // The opened stream is owned by `file`, whose deleter (std::fclose) closes it ++ // on every exit path. The static analyzer does not model the unique_ptr ++ // destructor, so it wrongly reports a leak here. ++ // codechecker_false_positive [unix.Stream] close done by unique_ptr deleter ++ file_ptr file(std::fopen(fileName, "rb"), &std::fclose); + + if (file == nullptr) + ThrowFIE("Could not open file \"%s\".", fileName); diff --git a/tools/rawspeed_proraw/pin/0010-Add-JPEG-XL-DNG-1.7-Compression-52546-decompressor.patch b/tools/rawspeed_proraw/pin/0010-Add-JPEG-XL-DNG-1.7-Compression-52546-decompressor.patch new file mode 100644 index 000000000000..8676ac7c03be --- /dev/null +++ b/tools/rawspeed_proraw/pin/0010-Add-JPEG-XL-DNG-1.7-Compression-52546-decompressor.patch @@ -0,0 +1,410 @@ +From 2c3dfc5779b604b647956ef2f4c292e943ab1d79 Mon Sep 17 00:00:00 2001 +From: Mayk Thewessen +Date: Sun, 21 Jun 2026 22:19:05 +0200 +Subject: [PATCH 10/10] Add JPEG XL (DNG 1.7 / Compression 52546) decompressor + +Adds libjxl-backed decoding of DNG tiles compressed as JPEG XL +(TIFF Compression 52546, DNG 1.7), as used by Apple ProRAW on the +iPhone 16. Structured like the existing lossy-JPEG path: + +- cmake: WITH_JPEGXL option + pkg-config libjxl detection -> HAVE_JPEGXL +- config.h.in: HAVE_JPEGXL define +- JpegXlDecompressor: RAII-guarded libjxl decode loop, validates channel + count and dimensions, clipped copy of the decoded tile into the raw buffer +- DngDecoder: accept compression 52546 in dropUnsupportedChunks +- AbstractDngDecompressor: dispatch compression 52546 to the JPEG XL decoder + +Gated behind HAVE_JPEGXL; when disabled a #pragma message warns and the +compression is reported unsupported. +--- + CMakeLists.txt | 1 + + cmake/src-dependencies.cmake | 23 +++ + src/config.h.in | 2 + + src/librawspeed/decoders/DngDecoder.cpp | 12 ++ + .../decompressors/AbstractDngDecompressor.cpp | 35 +++++ + src/librawspeed/decompressors/CMakeLists.txt | 6 + + .../decompressors/JpegXlDecompressor.cpp | 140 ++++++++++++++++++ + .../decompressors/JpegXlDecompressor.h | 56 +++++++ + 8 files changed, 275 insertions(+) + create mode 100644 src/librawspeed/decompressors/JpegXlDecompressor.cpp + create mode 100644 src/librawspeed/decompressors/JpegXlDecompressor.h + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index 2346db8d..af15a818 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -80,6 +80,7 @@ else() + set(ALLOW_DOWNLOADING_PUGIXML OFF CACHE BOOL "If pugixml src tree is not found in location specified by PUGIXML_PATH, do fetch the archive from internet" FORCE) + endif() + option(WITH_JPEG "Enable JPEG support for DNG Lossy JPEG support" ON) ++option(WITH_JPEGXL "Enable JPEG XL support for DNG 1.7 JPEG XL compression" ON) + option(WITH_ZLIB "Enable ZLIB support for DNG deflate support" ON) + if(WITH_ZLIB) + option(USE_BUNDLED_ZLIB "Build and use zlib in-tree" OFF) +diff --git a/cmake/src-dependencies.cmake b/cmake/src-dependencies.cmake +index a9a887e6..59619ea8 100644 +--- a/cmake/src-dependencies.cmake ++++ b/cmake/src-dependencies.cmake +@@ -184,6 +184,29 @@ else() + endif() + add_feature_info("Lossy JPEG decoding" HAVE_JPEG "used for DNG Lossy JPEG compression decoding") + ++unset(HAVE_JPEGXL) ++if(WITH_JPEGXL) ++ message(STATUS "Looking for JPEG XL (libjxl)") ++ find_package(PkgConfig QUIET) ++ if(PkgConfig_FOUND) ++ pkg_check_modules(libjxl IMPORTED_TARGET libjxl) ++ endif() ++ if(NOT libjxl_FOUND) ++ message(SEND_ERROR "Did not find libjxl! Either install jpeg-xl, or pass -DWITH_JPEGXL=OFF to disable JPEG XL.") ++ else() ++ message(STATUS "Looking for JPEG XL - found (${libjxl_VERSION})") ++ set(HAVE_JPEGXL 1) ++ target_link_libraries(rawspeed PRIVATE PkgConfig::libjxl) ++ set_package_properties(libjxl PROPERTIES ++ TYPE RECOMMENDED ++ DESCRIPTION "JPEG XL reference codec library" ++ PURPOSE "Used for decoding DNG JPEG XL (DNG 1.7) compression") ++ endif() ++else() ++ message(STATUS "JPEG XL is disabled, DNG JPEG XL (DNG 1.7) support won't be available.") ++endif() ++add_feature_info("JPEG XL decoding" HAVE_JPEGXL "used for DNG JPEG XL (DNG 1.7) compression decoding") ++ + unset(HAVE_ZLIB) + if (WITH_ZLIB) + message(STATUS "Looking for ZLIB") +diff --git a/src/config.h.in b/src/config.h.in +index 623d1417..0e42746e 100644 +--- a/src/config.h.in ++++ b/src/config.h.in +@@ -65,6 +65,8 @@ static_assert(RAWSPEED_LARGEPAGESIZE >= RAWSPEED_PAGESIZE, + #cmakedefine HAVE_JPEG + #cmakedefine HAVE_JPEG_MEM_SRC + ++#cmakedefine HAVE_JPEGXL ++ + #cmakedefine HAVE_CXX_THREAD_LOCAL + #cmakedefine HAVE_GCC_THREAD_LOCAL + +diff --git a/src/librawspeed/decoders/DngDecoder.cpp b/src/librawspeed/decoders/DngDecoder.cpp +index ae972d2a..49004c4a 100644 +--- a/src/librawspeed/decoders/DngDecoder.cpp ++++ b/src/librawspeed/decoders/DngDecoder.cpp +@@ -119,6 +119,9 @@ void DngDecoder::dropUnsuportedChunks(std::vector* data) { + case 9: // VC-5 as used by GoPro + #ifdef HAVE_JPEG + case 0x884c: // lossy JPEG ++#endif ++#ifdef HAVE_JPEGXL ++ case 52546: // JPEG XL (DNG 1.7) + #endif + // no change, if supported, then is still supported. + break; +@@ -140,6 +143,15 @@ void DngDecoder::dropUnsuportedChunks(std::vector* data) { + "chunk, but the jpeg support was " + "disabled at build!"); + [[clang::fallthrough]]; ++#endif ++#ifndef HAVE_JPEGXL ++ case 52546: // JPEG XL (DNG 1.7) ++#pragma message \ ++ "JPEG XL is not present! DNG JPEG XL compression will not be supported!" ++ writeLog(DEBUG_PRIO::WARNING, "DNG Decoder: found JPEG XL-encoded " ++ "chunk, but JPEG XL support was " ++ "disabled at build!"); ++ [[clang::fallthrough]]; + #endif + default: + supported = false; +diff --git a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +index 6ba745b6..a6254013 100644 +--- a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp ++++ b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +@@ -50,6 +50,10 @@ + #include "decompressors/JpegDecompressor.h" + #endif + ++#ifdef HAVE_JPEGXL ++#include "decompressors/JpegXlDecompressor.h" ++#endif ++ + namespace rawspeed { + + namespace { +@@ -269,6 +273,29 @@ void AbstractDngDecompressor::decompressThread<0x884c>() const noexcept { + } + #endif + ++#ifdef HAVE_JPEGXL ++template <> ++void AbstractDngDecompressor::decompressThread<52546>() const noexcept { ++#ifdef HAVE_OPENMP ++#pragma omp for schedule(static) ++#endif ++ for (const auto& e : ++ Array1DRef(slices.data(), implicit_cast(slices.size()))) { ++ try { ++ JpegXlDecompressor j(e.bs.peekBuffer(e.bs.getRemainSize()), mRaw); ++ j.decode(e.offX, e.offY); ++ } catch (const RawDecoderException& err) { ++ mRaw->setError(err.what()); ++ } catch (const IOException& err) { ++ mRaw->setError(err.what()); ++ } catch (...) { ++ // We should not get any other exception type here. ++ __builtin_unreachable(); ++ } ++ } ++} ++#endif ++ + void AbstractDngDecompressor::decompressThread() const noexcept { + invariant(mRaw->dim.x > 0); + invariant(mRaw->dim.y > 0); +@@ -300,6 +327,14 @@ void AbstractDngDecompressor::decompressThread() const noexcept { + #else + #pragma message "JPEG is not present! Lossy JPEG DNG will not be supported!" + mRaw->setError("jpeg support is disabled."); ++#endif ++ } else if (compression == 52546) { ++ /* JPEG XL (DNG 1.7) */ ++#ifdef HAVE_JPEGXL ++ decompressThread<52546>(); ++#else ++#pragma message "JPEG XL is not present! DNG JPEG XL will not be supported!" ++ mRaw->setError("JPEG XL support is disabled."); + #endif + } else { + mRaw->setError("AbstractDngDecompressor: Unknown compression"); +diff --git a/src/librawspeed/decompressors/CMakeLists.txt b/src/librawspeed/decompressors/CMakeLists.txt +index 8933a3ad..20627c34 100644 +--- a/src/librawspeed/decompressors/CMakeLists.txt ++++ b/src/librawspeed/decompressors/CMakeLists.txt +@@ -26,6 +26,8 @@ FILE(GLOB SOURCES + "JpegDecompressor.cpp" + "JpegDecompressor.h" + "JpegMarkers.h" ++ "JpegXlDecompressor.cpp" ++ "JpegXlDecompressor.h" + "KodakDecompressor.cpp" + "KodakDecompressor.h" + "LJpegDecoder.cpp" +@@ -82,4 +84,8 @@ if(WITH_JPEG AND TARGET JPEG::JPEG) + target_link_libraries(rawspeed_decompressors PUBLIC JPEG::JPEG) + endif() + ++if(WITH_JPEGXL AND TARGET PkgConfig::libjxl) ++ target_link_libraries(rawspeed_decompressors PUBLIC PkgConfig::libjxl) ++endif() ++ + target_link_libraries(rawspeed PRIVATE rawspeed_decompressors) +diff --git a/src/librawspeed/decompressors/JpegXlDecompressor.cpp b/src/librawspeed/decompressors/JpegXlDecompressor.cpp +new file mode 100644 +index 00000000..7b7b4702 +--- /dev/null ++++ b/src/librawspeed/decompressors/JpegXlDecompressor.cpp +@@ -0,0 +1,140 @@ ++/* ++ RawSpeed - RAW file decoder. ++ ++ Copyright (C) 2026 darktable developers ++ ++ This library is free software; you can redistribute it and/or ++ modify it under the terms of the GNU Lesser General Public ++ License as published by the Free Software Foundation; either ++ version 2 of the License, or (at your option) any later version. ++ ++ This library is distributed in the hope that it will be useful, ++ but WITHOUT ANY WARRANTY; without even the implied warranty of ++ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ++ Lesser General Public License for more details. ++ ++ You should have received a copy of the GNU Lesser General Public ++ License along with this library; if not, write to the Free Software ++ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ++*/ ++ ++#include "rawspeedconfig.h" // IWYU pragma: keep ++ ++#ifdef HAVE_JPEGXL ++ ++#include "adt/Array2DRef.h" ++#include "adt/Point.h" ++#include "decoders/RawDecoderException.h" ++#include "decompressors/JpegXlDecompressor.h" ++#include ++#include ++#include ++#include ++#include ++ ++using std::min; ++ ++namespace rawspeed { ++ ++namespace { ++// RAII wrapper so JxlDecoderDestroy runs on every exit path (incl. throws). ++struct JxlDecoderGuard final { ++ JxlDecoder* dec; ++ explicit JxlDecoderGuard(JxlDecoder* d) : dec(d) {} ++ JxlDecoderGuard(const JxlDecoderGuard&) = delete; ++ JxlDecoderGuard(JxlDecoderGuard&&) = delete; ++ JxlDecoderGuard& operator=(const JxlDecoderGuard&) = delete; ++ JxlDecoderGuard& operator=(JxlDecoderGuard&&) = delete; ++ ~JxlDecoderGuard() { JxlDecoderDestroy(dec); } ++}; ++} // namespace ++ ++void JpegXlDecompressor::decode(uint32_t offX, uint32_t offY) { ++ JxlDecoder* dec = JxlDecoderCreate(nullptr); ++ if (dec == nullptr) ++ ThrowRDE("JXL: JxlDecoderCreate failed"); ++ JxlDecoderGuard guard(dec); ++ ++ if (JXL_DEC_SUCCESS != ++ JxlDecoderSubscribeEvents(dec, JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE)) ++ ThrowRDE("JXL: JxlDecoderSubscribeEvents failed"); ++ ++ // rawspeed/darktable handle orientation themselves; do not auto-rotate. ++ if (JXL_DEC_SUCCESS != JxlDecoderSetKeepOrientation(dec, JXL_TRUE)) ++ ThrowRDE("JXL: JxlDecoderSetKeepOrientation failed"); ++ ++ if (JXL_DEC_SUCCESS != ++ JxlDecoderSetInput(dec, input.begin(), input.getSize())) ++ ThrowRDE("JXL: JxlDecoderSetInput failed"); ++ JxlDecoderCloseInput(dec); ++ ++ const uint32_t cpp = mRaw->getCpp(); ++ const JxlPixelFormat fmt = {/*num_channels=*/cpp, ++ /*data_type=*/JXL_TYPE_UINT16, ++ /*endianness=*/JXL_LITTLE_ENDIAN, ++ /*align=*/0}; ++ ++ JxlBasicInfo info = {}; ++ uint32_t jxl_w = 0; ++ uint32_t jxl_h = 0; ++ std::vector pixels; ++ ++ for (;;) { ++ const JxlDecoderStatus status = JxlDecoderProcessInput(dec); ++ if (status == JXL_DEC_ERROR) ++ ThrowRDE("JXL: decoding error"); ++ if (status == JXL_DEC_NEED_MORE_INPUT) ++ ThrowRDE("JXL: needs more input (truncated tile?)"); ++ if (status == JXL_DEC_BASIC_INFO) { ++ if (JXL_DEC_SUCCESS != JxlDecoderGetBasicInfo(dec, &info)) ++ ThrowRDE("JXL: JxlDecoderGetBasicInfo failed"); ++ jxl_w = info.xsize; ++ jxl_h = info.ysize; ++ if (info.num_color_channels != cpp) ++ ThrowRDE("JXL: color channel count %u does not match cpp %u", ++ info.num_color_channels, cpp); ++ continue; ++ } ++ if (status == JXL_DEC_NEED_IMAGE_OUT_BUFFER) { ++ size_t buf_size = 0; ++ if (JXL_DEC_SUCCESS != JxlDecoderImageOutBufferSize(dec, &fmt, &buf_size)) ++ ThrowRDE("JXL: JxlDecoderImageOutBufferSize failed"); ++ pixels.resize(buf_size / sizeof(uint16_t)); ++ if (JXL_DEC_SUCCESS != ++ JxlDecoderSetImageOutBuffer(dec, &fmt, pixels.data(), buf_size)) ++ ThrowRDE("JXL: JxlDecoderSetImageOutBuffer failed"); ++ continue; ++ } ++ if (status == JXL_DEC_FULL_IMAGE) ++ continue; // image now in `pixels` ++ if (status == JXL_DEC_SUCCESS) ++ break; ++ ThrowRDE("JXL: unexpected decoder status %d", static_cast(status)); ++ } ++ ++ if (pixels.empty() || jxl_w == 0 || jxl_h == 0) ++ ThrowRDE("JXL: no pixel data decoded"); ++ ++ // Copy the decoded interleaved uint16 tile into the raw buffer at ++ // (offX,offY). ++ const uint32_t copy_w = min(static_cast(mRaw->dim.x) - offX, jxl_w); ++ const uint32_t copy_h = min(static_cast(mRaw->dim.y) - offY, jxl_h); ++ ++ const Array2DRef out(mRaw->getU16DataAsUncroppedArray2DRef()); ++ for (uint32_t row = 0; row < copy_h; ++row) { ++ for (uint32_t col = 0; col < cpp * copy_w; ++col) { ++ out(static_cast(row + offY), static_cast(cpp * offX + col)) = ++ pixels[(static_cast(row) * jxl_w * cpp) + col]; ++ } ++ } ++} ++ ++} // namespace rawspeed ++ ++#else ++ ++#pragma message \ ++ "JPEG XL is not present! DNG JPEG XL (DNG 1.7) compression will not be " \ ++ "supported!" ++ ++#endif +diff --git a/src/librawspeed/decompressors/JpegXlDecompressor.h b/src/librawspeed/decompressors/JpegXlDecompressor.h +new file mode 100644 +index 00000000..7cbcef08 +--- /dev/null ++++ b/src/librawspeed/decompressors/JpegXlDecompressor.h +@@ -0,0 +1,56 @@ ++/* ++ RawSpeed - RAW file decoder. ++ ++ Copyright (C) 2026 darktable developers ++ ++ This library is free software; you can redistribute it and/or ++ modify it under the terms of the GNU Lesser General Public ++ License as published by the Free Software Foundation; either ++ version 2 of the License, or (at your option) any later version. ++ ++ This library is distributed in the hope that it will be useful, ++ but WITHOUT ANY WARRANTY; without even the implied warranty of ++ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ++ Lesser General Public License for more details. ++ ++ You should have received a copy of the GNU Lesser General Public ++ License along with this library; if not, write to the Free Software ++ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ++*/ ++ ++#pragma once ++ ++#include "rawspeedconfig.h" ++ ++#ifdef HAVE_JPEGXL ++ ++#include "common/RawImage.h" ++#include "decompressors/AbstractDecompressor.h" ++#include "io/Buffer.h" ++#include ++#include ++ ++namespace rawspeed { ++ ++// Decodes a single DNG tile whose data is a self-contained JPEG XL codestream ++// (TIFF Compression tag 52546, as used by Apple ProRAW on iPhone 16 / DNG 1.7). ++class JpegXlDecompressor final : public AbstractDecompressor { ++ Buffer input; ++ RawImage mRaw; ++ ++public: ++ JpegXlDecompressor(Buffer bs, RawImage img) ++ : input(bs), mRaw(std::move(img)) {} ++ ++ void decode(uint32_t offsetX, uint32_t offsetY); ++}; ++ ++} // namespace rawspeed ++ ++#else ++ ++#pragma message \ ++ "JPEG XL is not present! DNG JPEG XL (DNG 1.7) compression will not be " \ ++ "supported!" ++ ++#endif diff --git a/tools/rawspeed_proraw/pin/rebuild_pin.sh b/tools/rawspeed_proraw/pin/rebuild_pin.sh new file mode 100755 index 000000000000..041d0dc15fcf --- /dev/null +++ b/tools/rawspeed_proraw/pin/rebuild_pin.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# +# Rebuild the rawspeed commit that src/external/rawspeed is pinned to, using +# only refs that darktable-org/rawspeed publishes. +# +# Why this exists: the pinned commit (see PINNED_COMMIT below) is not reachable +# from any branch of darktable-org/rawspeed, so a fresh clone of this fork +# cannot "git submodule update --init" it. This script reconstructs the same +# source tree from the public 'stable' branch plus the patch series next to it. +# +# The rebuilt commit's SHA will NOT equal PINNED_COMMIT -- committer identity +# and timestamps differ -- but its *tree* is byte-for-byte identical, which is +# what the build consumes. The script verifies that and refuses to finish if +# the tree does not match. +# +# Usage: +# tools/rawspeed_proraw/pin/rebuild_pin.sh [] +# +# With no argument it operates on src/external/rawspeed relative to the repo +# root. Afterwards, point the submodule at the rebuilt commit it prints. + +set -euo pipefail + +UPSTREAM="https://github.com/darktable-org/rawspeed.git" +BASE_REF="refs/heads/stable" +BASE_COMMIT="4c511d611c1beee9aa97a2ec50b0838c6c7be52e" +PINNED_COMMIT="2c3dfc5779b604b647956ef2f4c292e943ab1d79" +EXPECTED_TREE="099b577e836a53d8e5de87149aa0ea13b20130a1" + +script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +repo_root="$(git -C "$script_dir" rev-parse --show-toplevel)" +target="${1:-$repo_root/src/external/rawspeed}" + +shopt -s nullglob +patches=("$script_dir"/[0-9]*.patch) +shopt -u nullglob +if [ ${#patches[@]} -eq 0 ]; then + echo "error: no patches found next to $0" >&2 + exit 1 +fi +echo "Using ${#patches[@]} patches from $script_dir" + +if [ ! -d "$target/.git" ]; then + echo "Cloning $UPSTREAM into $target" + mkdir -p "$target" + git clone --quiet "$UPSTREAM" "$target" +fi + +cd "$target" + +echo "Fetching $BASE_REF from upstream" +git fetch --quiet "$UPSTREAM" "$BASE_REF" + +if ! git cat-file -e "${BASE_COMMIT}^{commit}" 2>/dev/null; then + echo "error: base commit $BASE_COMMIT not found after fetching $BASE_REF." >&2 + echo " Upstream may have moved 'stable'; see ../README.md." >&2 + exit 1 +fi + +# Refuse to clobber uncommitted work. +if ! git diff --quiet || ! git diff --cached --quiet; then + echo "error: $target has uncommitted changes; refusing to continue." >&2 + exit 1 +fi + +git am --abort >/dev/null 2>&1 || true +echo "Checking out base $BASE_COMMIT" +git checkout --quiet --detach "$BASE_COMMIT" + +echo "Applying patch series" +if ! git am "${patches[@]}"; then + echo "error: 'git am' failed. Resolve, then run 'git am --continue'." >&2 + exit 1 +fi + +rebuilt_commit="$(git rev-parse HEAD)" +rebuilt_tree="$(git rev-parse 'HEAD^{tree}')" + +echo +if [ "$rebuilt_tree" != "$EXPECTED_TREE" ]; then + echo "FAILED: rebuilt tree $rebuilt_tree != expected $EXPECTED_TREE" >&2 + exit 1 +fi + +echo "OK: rebuilt tree matches the pinned tree ($EXPECTED_TREE)" +echo " rebuilt commit : $rebuilt_commit" +echo " recorded pin : $PINNED_COMMIT (content-identical, different SHA)" +echo +echo "The working tree now matches the pin and is ready to build." +echo "To make this reproducible for others, push the rebuilt commit to a fork" +echo "you control and repoint the submodule -- see ../README.md." diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0001-LJpeg-support-predictor-modes-2-7.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0001-LJpeg-support-predictor-modes-2-7.patch new file mode 100644 index 000000000000..01e4db910215 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0001-LJpeg-support-predictor-modes-2-7.patch @@ -0,0 +1,397 @@ +From 92cf98322facd53a019028e2b858170a1c75912d Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 11:43:10 +0200 +Subject: [PATCH 1/7] =?UTF-8?q?LJpeg:=20support=20predictor=20modes=202?= + =?UTF-8?q?=E2=80=937?= +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Summary: + +rawspeed's LJpeg decoder previously supported only predictor mode 1 (left neighbor). ITU-T T.81 defines seven predictor modes, and several camera manufacturers — notably DJI (Mavic 3S, Mavic 3 Pro, etc.) and Blackmagic — use mode 6 in their DNG files. This change adds support for all seven modes and handles the associated tile geometry those cameras use. + +Predictor modes 2–7 + +* Added a computePrediction(mode, Ra, Rb, Rc) helper that computes the seven ITU-T T.81 predictions. Arithmetic is done in int32_t to avoid overflow in modes 4–6, where Ra + Rb - Rc can transiently exceed the 16-bit range; the final result wraps via uint16_t cast per the standard. +* decodeRowN() is extended to accept predMode and prevStripe. For mode 1, the existing fast path (no memory access to the previous row) is preserved. For modes 2–7, the three 2D neighbors (Ra = left, Rb = above, Rc = above-left) are looked up from prevStripe; at the start of a row the above-left neighbor is not available, so Rc = Rb per the JPEG specification. +* decodeN() now tracks isFirstRow per restart interval. The first row of each interval always uses predictor mode 1 (per T.81: the standard mandates horizontal prediction for the first row). For subsequent rows, prevStripe is a CroppedArray2DRef into the already-decoded portion of the output image. No extra allocation is needed. +Inverted tile reshape (LJpegDecoder) + +Inverted tile reshape + +DJI DNGs present tiles in a non-standard geometry: the JPEG SOF3 frame is wider than the declared tile (e.g. 8000×1500 for a 4000×3000 tile). Each JPEG row encodes two tile rows concatenated side-by-side — a widthPack = jpegFrameDim.x / tileW packing. This is the inverse of the Adobe-style reshape rawspeed already handles (MCU > 1×1). + +Design trade-offs: + +Separate decode-then-deinterleave vs. in-place: Reusing the existing LJpegDecompressor in-place would require threading the tile geometry inversion through all MCU addressing logic. Instead, the inverted-reshape path decodes into a temporary RawImage at the JPEG frame dimensions, then copies/deinterleaves rows into the real output image. The extra allocation is bounded by a single tile's worth of data and avoids touching the hot-path decompressor. +widthPack validation: The factor is computed from the ratio jpegFrameDim.x / maxRes.x and validated against widthPack * jpegFrameDim.y == maxRes.y before decoding begins. An upper bound of 4 is enforced as a sanity check. +The inverted reshape is restricted to single-component LJpeg (N_COMP == 1), which is the only case seen in practice for this layout. +Fuzz harness + +The LJpegDecompressor fuzz entry point is updated to source a predictorMode byte from the fuzzer input, covering the new code paths. + +(cherry picked from commit cfa4429965054d698c003eca721dad9af05b9b2d) +(cherry picked from commit 946aa890a95999e836a6985f19028b3499221d55) +--- + .../decompressors/LJpegDecompressor.cpp | 3 +- + .../decompressors/LJpegDecoder.cpp | 115 +++++++++++++++--- + .../decompressors/LJpegDecompressor.cpp | 87 ++++++++++++- + .../decompressors/LJpegDecompressor.h | 3 + + 4 files changed, 189 insertions(+), 19 deletions(-) + +diff --git a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +index aa7b7154..83e406ab 100644 +--- a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -88,10 +88,11 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) { + }); + + const int numLJpegRowsPerRestartInterval = bs.getI32(); ++ const int predictorMode = bs.getByte(); + + rawspeed::LJpegDecompressor d( + mRaw, rawspeed::iRectangle2D(mRaw->dim.x, mRaw->dim.y), frame, rec, +- numLJpegRowsPerRestartInterval, ++ numLJpegRowsPerRestartInterval, predictorMode, + bs.getSubStream(/*offset=*/0).peekRemainingBuffer().getAsArray1DRef()); + mRaw->createData(); + (void)d.decode(); +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index 74db4109..c9c93aab 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -32,8 +32,10 @@ + #include + #include + #include ++#include + #include + #include ++#include + #include + + using std::copy_n; +@@ -104,7 +106,7 @@ void LJpegDecoder::decode(uint32_t offsetX, uint32_t offsetY, uint32_t width, + Buffer::size_type LJpegDecoder::decodeScan() { + invariant(frame.cps > 0); + +- if (predictorMode != 1) ++ if (predictorMode < 1 || predictorMode > 7) + ThrowRDE("Unsupported predictor mode: %u", predictorMode); + + for (uint32_t i = 0; i < frame.cps; i++) +@@ -123,9 +125,6 @@ Buffer::size_type LJpegDecoder::decodeScan() { + return {*hts[i], initPred[i]}; + }); + +- const iRectangle2D imgFrame = { +- {static_cast(offX), static_cast(offY)}, +- {static_cast(w), static_cast(h)}}; + const auto jpegFrameDim = iPoint2D(frame.w, frame.h); + + if (implicit_cast(maxDim.x) * implicit_cast(mRaw->getCpp()) > +@@ -137,31 +136,119 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (maxRes.area() != N_COMP * jpegFrameDim.area()) + ThrowRDE("LJpeg frame area does not match maximal tile area"); + +- if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) +- ThrowRDE("Maximal output tile size is not a multiple of LJpeg frame size"); ++ // Detect whether the JPEG frame uses an inverted reshape (e.g. DJI/Blackmagic ++ // CinemaDNG): JPEG frame is wider than tile and shorter, with packed rows. ++ // Standard (Adobe): maxRes.x >= jpegFrameDim.x (tile is wider/equal) ++ // Inverted (DJI): jpegFrameDim.x > maxRes.x (JPEG frame is wider) ++ bool invertedReshape = (jpegFrameDim.x > maxRes.x); ++ ++ if (!invertedReshape) { ++ // Standard case: tile width is a multiple of JPEG frame width. ++ if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) ++ ThrowRDE( ++ "Maximal output tile size is not a multiple of LJpeg frame size"); ++ ++ auto MCUSize = ++ iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; ++ if (MCUSize.area() != implicit_cast(N_COMP)) ++ ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); ++ ++ const iRectangle2D imgFrame = { ++ {static_cast(offX), static_cast(offY)}, ++ {static_cast(w), static_cast(h)}}; ++ const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; ++ ++ int numLJpegRowsPerRestartInterval; ++ if (numMCUsPerRestartInterval == 0) { ++ numLJpegRowsPerRestartInterval = jpegFrameDim.y; ++ } else { ++ const int numMCUsPerRow = jpegFrameDim.x; ++ if (numMCUsPerRestartInterval % numMCUsPerRow != 0) ++ ThrowRDE("Restart interval is not a multiple of frame row size"); ++ numLJpegRowsPerRestartInterval = ++ numMCUsPerRestartInterval / numMCUsPerRow; ++ } ++ ++ LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, ++ numLJpegRowsPerRestartInterval, ++ implicit_cast(predictorMode), ++ input.peekRemainingBuffer().getAsArray1DRef()); ++ return d.decode(); ++ } ++ ++ // Inverted reshape case (DJI/Blackmagic CinemaDNG): ++ // JPEG frame is wider than tile, e.g. JPEG=8000x1500 1-comp, tile=4000x3000. ++ // Each JPEG row contains 'widthPack' tile rows concatenated. ++ if (N_COMP != 1) ++ ThrowRDE("Inverted reshape only supported for single-component LJpeg"); ++ ++ if (jpegFrameDim.x % maxRes.x != 0) ++ ThrowRDE("LJpeg frame width is not a multiple of tile width"); ++ if (maxRes.y % jpegFrameDim.y != 0) ++ ThrowRDE("Tile height is not a multiple of LJpeg frame height"); ++ ++ const int widthPack = jpegFrameDim.x / maxRes.x; ++ if (widthPack * jpegFrameDim.y != maxRes.y) ++ ThrowRDE("Inverted reshape dimensions mismatch"); + +- auto MCUSize = iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; +- if (MCUSize.area() != implicit_cast(N_COMP)) +- ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); ++ if (widthPack < 1 || widthPack > 4) ++ ThrowRDE("Unexpected row packing factor: %d", widthPack); + ++ // Decode into a temporary buffer at JPEG frame dimensions. ++ // MCU is {1,1} since we have a single component. ++ const auto MCUSize = iPoint2D{1, 1}; ++ ++ // Create a temporary raw image to decode the JPEG into. ++ // RawImage::create with dimensions already calls createData() internally. ++ RawImage tmpRaw = RawImage::create( ++ iPoint2D(jpegFrameDim.x, jpegFrameDim.y), RawImageType::UINT16, 1); ++ ++ const iRectangle2D tmpFrame = { ++ {0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; + const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; + + int numLJpegRowsPerRestartInterval; + if (numMCUsPerRestartInterval == 0) { +- // Restart interval not enabled, so all of the rows +- // are contained in the first (implicit) restart interval. + numLJpegRowsPerRestartInterval = jpegFrameDim.y; + } else { + const int numMCUsPerRow = jpegFrameDim.x; + if (numMCUsPerRestartInterval % numMCUsPerRow != 0) + ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = numMCUsPerRestartInterval / numMCUsPerRow; ++ numLJpegRowsPerRestartInterval = ++ numMCUsPerRestartInterval / numMCUsPerRow; + } + +- LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, ++ LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, + numLJpegRowsPerRestartInterval, ++ implicit_cast(predictorMode), + input.peekRemainingBuffer().getAsArray1DRef()); +- return d.decode(); ++ auto consumed = d.decode(); ++ ++ // Deinterleave: each JPEG row of width (widthPack * tileW) maps to ++ // widthPack consecutive tile rows of width tileW. ++ const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); ++ const auto outData = mRaw->getU16DataAsUncroppedArray2DRef(); ++ ++ const int tileW = implicit_cast(w); ++ const int cpp = implicit_cast(mRaw->getCpp()); ++ const int outRowPixels = cpp * tileW; ++ ++ for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { ++ for (int pack = 0; pack < widthPack; ++pack) { ++ const int tileRow = ++ implicit_cast(offY) + jpegRow * widthPack + pack; ++ if (tileRow >= mRaw->dim.y) ++ continue; ++ const int srcCol = pack * outRowPixels; ++ const int dstCol = cpp * implicit_cast(offX); ++ for (int col = 0; col < outRowPixels && (srcCol + col) < jpegFrameDim.x; ++ ++col) { ++ outData(tileRow, dstCol + col) = tmpData(jpegRow, srcCol + col); ++ } ++ } ++ } ++ ++ return consumed; + } + + } // namespace rawspeed +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index b42ba706..9060fd44 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -53,10 +53,12 @@ LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + Frame frame_, + std::vector rec_, + int numLJpegRowsPerRestartInterval_, ++ int predictorMode_, + Array1DRef input_) + : mRaw(std::move(img)), input(input_), imgFrame(imgFrame_), + frame(std::move(frame_)), rec(std::move(rec_)), +- numLJpegRowsPerRestartInterval(numLJpegRowsPerRestartInterval_) { ++ numLJpegRowsPerRestartInterval(numLJpegRowsPerRestartInterval_), ++ predictorMode(predictorMode_) { + + if (mRaw->getDataType() != RawImageType::UINT16) + ThrowRDE("Unexpected data type (%u)", +@@ -181,9 +183,39 @@ constexpr iPoint2D MCU = {MCUWidth, MCUHeight}; + + } // namespace + ++namespace { ++ ++// Compute the LJpeg prediction value given predictor mode and neighbor values. ++// Ra = left, Rb = above, Rc = above-left. ++// All arithmetic done in int32_t to avoid overflow in modes 4-6. ++// Result is modulo 2^16 per ITU-T T.81. ++inline int computePrediction(int predMode, int Ra, int Rb, int Rc) { ++ switch (predMode) { ++ case 1: ++ return Ra; ++ case 2: ++ return Rb; ++ case 3: ++ return Rc; ++ case 4: ++ return Ra + Rb - Rc; ++ case 5: ++ return Ra + ((Rb - Rc) >> 1); ++ case 6: ++ return Rb + ((Ra - Rc) >> 1); ++ case 7: ++ return (Ra + Rb) >> 1; ++ default: ++ __builtin_unreachable(); ++ } ++} ++ ++} // namespace ++ + template + void LJpegDecompressor::decodeRowN( + Array2DRef outStripe, Array2DRef pred, ++ int predMode, Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const { + invariant(MCUSize.area() == N_COMP); +@@ -207,7 +239,21 @@ void LJpegDecompressor::decodeRowN( + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { + int c = (MCUSize.x * MCURow) + MCUСol; +- int prediction = pred(MCURow, MCUСol); ++ int prediction; ++ if (predMode == 1) { ++ // Fast path for the common case (mode 1 = left neighbor). ++ prediction = pred(MCURow, MCUСol); ++ } else { ++ // For modes 2-7, compute Ra, Rb, Rc. ++ int Ra = pred(MCURow, MCUСol); // left neighbor ++ int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ int stripeRow = MCURow; ++ int Rb = prevStripe(stripeRow, stripeCol); ++ int Rc = (stripeCol >= MCUSize.x) ++ ? prevStripe(stripeRow, stripeCol - MCUSize.x) ++ : Rb; // First column: Rc = Rb ++ prediction = computePrediction(predMode, Ra, Rb, Rc); ++ } + int diff = (static_cast&>(ht[c])) + .decodeDifference(bs); + int pix = prediction + diff; +@@ -230,7 +276,21 @@ void LJpegDecompressor::decodeRowN( + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { + int c = (MCUSize.x * MCURow) + MCUСol; +- int prediction = pred(MCURow, MCUСol); ++ int prediction; ++ if (predMode == 1) { ++ prediction = pred(MCURow, MCUСol); ++ } else { ++ int Ra = pred(MCURow, MCUСol); ++ int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ int stripeRow = MCURow; ++ int Rb = (stripeCol < prevStripe.width()) ++ ? prevStripe(stripeRow, stripeCol) ++ : Ra; ++ int Rc = (stripeCol >= MCUSize.x && stripeCol < prevStripe.width()) ++ ? prevStripe(stripeRow, stripeCol - MCUSize.x) ++ : Rb; ++ prediction = computePrediction(predMode, Ra, Rb, Rc); ++ } + int diff = (static_cast&>(ht[c])) + .decodeDifference(bs); + int pix = prediction + diff; +@@ -284,6 +344,7 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + restartIntervalIndex != numRestartIntervals; ++restartIntervalIndex) { + auto predStorage = getInitialPreds(); + auto pred = Array2DRef(predStorage.data(), MCU.x, MCU.y); ++ bool isFirstRow = true; + + if (restartIntervalIndex != 0) { + auto marker = peekMarker(inputStream); +@@ -321,7 +382,24 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + /*croppedHeight=*/frame.mcu.y) + .getAsArray2DRef(); + +- decodeRowN(outStripe, pred, ht, bs); ++ // For predictor modes 2-7, we need the previous row (stripe). ++ // For the first row of each restart interval, use predictor mode 1 ++ // (per ITU-T T.81: first row always uses horizontal prediction). ++ // For the first row, prevStripe points to outStripe itself (unused ++ // since predMode will be 1). ++ const int predMode = isFirstRow ? 1 : predictorMode; ++ const Array2DRef prevStripe = ++ isFirstRow ++ ? Array2DRef(outStripe) ++ : CroppedArray2DRef( ++ img, ++ /*offsetCols=*/0, ++ /*offsetRows=*/row - frame.mcu.y, ++ /*croppedWidth=*/img.width(), ++ /*croppedHeight=*/frame.mcu.y) ++ .getAsArray2DRef(); ++ ++ decodeRowN(outStripe, pred, predMode, prevStripe, ht, bs); + + // The predictor for the next line is the start of this line. + pred = CroppedArray2DRef(outStripe, +@@ -330,6 +408,7 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + /*croppedWidth=*/MCU.x, + /*croppedHeight=*/MCU.y) + .getAsArray2DRef(); ++ isFirstRow = false; + } + + inputStream.skipBytes(bs.getStreamPosition()); +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index 69c77739..3c1f4352 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -59,6 +59,7 @@ private: + const Frame frame; + const std::vector rec; + const int numLJpegRowsPerRestartInterval; ++ const int predictorMode; + + int numFullMCUs = 0; + int trailingPixels = 0; +@@ -79,6 +80,7 @@ private: + template + __attribute__((always_inline)) inline void decodeRowN( + Array2DRef outStripe, Array2DRef pred, ++ int predMode, Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const; + +@@ -89,6 +91,7 @@ public: + LJpegDecompressor(RawImage img, iRectangle2D imgFrame, Frame frame, + std::vector rec, + int numLJpegRowsPerRestartInterval_, ++ int predictorMode_, + Array1DRef input); + + [[nodiscard]] ByteStream::size_type decode() const; diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0002-LJpeg-eliminate-per-pixel-branch.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0002-LJpeg-eliminate-per-pixel-branch.patch new file mode 100644 index 000000000000..00317938e84f --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0002-LJpeg-eliminate-per-pixel-branch.patch @@ -0,0 +1,230 @@ +From a9388ba26b8989dee7444ae3a7f8b77480f5db8a Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 12:00:15 +0200 +Subject: [PATCH 2/7] LJpeg: eliminate per-pixel branch +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Eliminate per-pixel predMode branch (performance) + +decodeRowN gains a bool Use2DPred template parameter. The mode-1 (left-neighbor) path and the 2D-predictor path are now separate compile-time instantiations selected with if constexpr. The runtime if (predMode == 1) that previously executed for every pixel in every tile is gone entirely. Mode-1 files — the overwhelming majority of existing DNG content — are completely unaffected at the generated-code level; the compiler produces the same tight loop as before. For modes 2–7, the computePrediction switch is also folded away since predictorMode is now only evaluated once at the per-row dispatch in decodeN(), not per-pixel. + +The first-column Rc check is also simplified: stripeCol >= MCUSize.x (computed per-pixel) is replaced with mcuIdx > 0 (a loop-index comparison available for free), and the redundant stripeRow alias is removed. + +Replace scalar deinterleave with memcpy (performance) + +In the inverted-reshape path (LJpegDecoder), the per-pixel operator() copy loop is replaced by a single std::memcpy per row-segment. The previous loop also carried a data-dependent bounds check (srcCol + col < jpegFrameDim.x) that blocked auto-vectorisation; the check is provably redundant given the dimension validation already performed, so it is removed. memcpy allows the compiler to emit an optimal vector store. + +(cherry picked from commit 1d3a262388dfb51f79d94b423cc61428bed2c32e) +(cherry picked from commit 528551a68c47e5511d8ebc8e241349b098b9b106) +--- + .../decompressors/LJpegDecoder.cpp | 19 +++-- + .../decompressors/LJpegDecompressor.cpp | 73 +++++++++---------- + .../decompressors/LJpegDecompressor.h | 4 +- + 3 files changed, 46 insertions(+), 50 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index c9c93aab..0c2bd001 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -113,7 +113,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (frame.compInfo[i].superH != 1 || frame.compInfo[i].superV != 1) + ThrowRDE("Unsupported subsampling"); + +- int N_COMP = frame.cps; ++ const int N_COMP = frame.cps; + + std::vector rec; + rec.reserve(N_COMP); +@@ -131,7 +131,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + std::numeric_limits::max()) + ThrowRDE("Maximal output tile is too large"); + +- auto maxRes = ++ const auto maxRes = + iPoint2D(implicit_cast(mRaw->getCpp()) * maxDim.x, maxDim.y); + if (maxRes.area() != N_COMP * jpegFrameDim.area()) + ThrowRDE("LJpeg frame area does not match maximal tile area"); +@@ -140,15 +140,14 @@ Buffer::size_type LJpegDecoder::decodeScan() { + // CinemaDNG): JPEG frame is wider than tile and shorter, with packed rows. + // Standard (Adobe): maxRes.x >= jpegFrameDim.x (tile is wider/equal) + // Inverted (DJI): jpegFrameDim.x > maxRes.x (JPEG frame is wider) +- bool invertedReshape = (jpegFrameDim.x > maxRes.x); +- ++ const bool invertedReshape = (jpegFrameDim.x > maxRes.x); + if (!invertedReshape) { + // Standard case: tile width is a multiple of JPEG frame width. + if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) + ThrowRDE( + "Maximal output tile size is not a multiple of LJpeg frame size"); + +- auto MCUSize = ++ const auto MCUSize = + iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; + if (MCUSize.area() != implicit_cast(N_COMP)) + ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); +@@ -222,7 +221,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + numLJpegRowsPerRestartInterval, + implicit_cast(predictorMode), + input.peekRemainingBuffer().getAsArray1DRef()); +- auto consumed = d.decode(); ++ const auto consumed = d.decode(); + + // Deinterleave: each JPEG row of width (widthPack * tileW) maps to + // widthPack consecutive tile rows of width tileW. +@@ -241,10 +240,10 @@ Buffer::size_type LJpegDecoder::decodeScan() { + continue; + const int srcCol = pack * outRowPixels; + const int dstCol = cpp * implicit_cast(offX); +- for (int col = 0; col < outRowPixels && (srcCol + col) < jpegFrameDim.x; +- ++col) { +- outData(tileRow, dstCol + col) = tmpData(jpegRow, srcCol + col); +- } ++ // Contiguous row-segment copy. Bounds guaranteed by validation: ++ // srcCol + outRowPixels <= widthPack * outRowPixels <= jpegFrameDim.x ++ std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), ++ sizeof(uint16_t) * outRowPixels); + } + } + +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index 9060fd44..93ac87eb 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -212,10 +212,10 @@ inline int computePrediction(int predMode, int Ra, int Rb, int Rc) { + + } // namespace + +-template ++template + void LJpegDecompressor::decodeRowN( + Array2DRef outStripe, Array2DRef pred, +- int predMode, Array2DRef prevStripe, ++ Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const { + invariant(MCUSize.area() == N_COMP); +@@ -238,25 +238,22 @@ void LJpegDecompressor::decodeRowN( + .getAsArray2DRef(); + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { +- int c = (MCUSize.x * MCURow) + MCUСol; ++ const int c = (MCUSize.x * MCURow) + MCUСol; + int prediction; +- if (predMode == 1) { +- // Fast path for the common case (mode 1 = left neighbor). ++ if constexpr (!Use2DPred) { + prediction = pred(MCURow, MCUСol); + } else { +- // For modes 2-7, compute Ra, Rb, Rc. +- int Ra = pred(MCURow, MCUСol); // left neighbor +- int stripeCol = MCUSize.x * mcuIdx + MCUСol; +- int stripeRow = MCURow; +- int Rb = prevStripe(stripeRow, stripeCol); +- int Rc = (stripeCol >= MCUSize.x) +- ? prevStripe(stripeRow, stripeCol - MCUSize.x) +- : Rb; // First column: Rc = Rb +- prediction = computePrediction(predMode, Ra, Rb, Rc); ++ const int Ra = pred(MCURow, MCUСol); ++ const int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ const int Rb = prevStripe(MCURow, stripeCol); ++ const int Rc = (mcuIdx > 0) ++ ? prevStripe(MCURow, stripeCol - MCUSize.x) ++ : Rb; ++ prediction = computePrediction(predictorMode, Ra, Rb, Rc); + } +- int diff = (static_cast&>(ht[c])) +- .decodeDifference(bs); +- int pix = prediction + diff; ++ const int diff = (static_cast&>(ht[c])) ++ .decodeDifference(bs); ++ const int pix = prediction + diff; + outTile(MCURow, MCUСol) = uint16_t(pix); + } + } +@@ -275,29 +272,27 @@ void LJpegDecompressor::decodeRowN( + // We may end up needing just part of last N_COMP pixels. + for (int MCURow = 0; MCURow != MCUSize.y; ++MCURow) { + for (int MCUСol = 0; MCUСol != MCUSize.x; ++MCUСol) { +- int c = (MCUSize.x * MCURow) + MCUСol; ++ const int c = (MCUSize.x * MCURow) + MCUСol; + int prediction; +- if (predMode == 1) { ++ if constexpr (!Use2DPred) { + prediction = pred(MCURow, MCUСol); + } else { +- int Ra = pred(MCURow, MCUСol); +- int stripeCol = MCUSize.x * mcuIdx + MCUСol; +- int stripeRow = MCURow; +- int Rb = (stripeCol < prevStripe.width()) +- ? prevStripe(stripeRow, stripeCol) +- : Ra; +- int Rc = (stripeCol >= MCUSize.x && stripeCol < prevStripe.width()) +- ? prevStripe(stripeRow, stripeCol - MCUSize.x) +- : Rb; +- prediction = computePrediction(predMode, Ra, Rb, Rc); ++ const int Ra = pred(MCURow, MCUСol); ++ const int stripeCol = MCUSize.x * mcuIdx + MCUСol; ++ const int Rb = (stripeCol < prevStripe.width()) ++ ? prevStripe(MCURow, stripeCol) ++ : Ra; ++ const int Rc = (mcuIdx > 0 && stripeCol < prevStripe.width()) ++ ? prevStripe(MCURow, stripeCol - MCUSize.x) ++ : Rb; ++ prediction = computePrediction(predictorMode, Ra, Rb, Rc); + } +- int diff = (static_cast&>(ht[c])) +- .decodeDifference(bs); +- int pix = prediction + diff; +- int stripeRow = MCURow; +- int stripeCol = (MCUSize.x * mcuIdx) + MCUСol; ++ const int diff = (static_cast&>(ht[c])) ++ .decodeDifference(bs); ++ const int pix = prediction + diff; ++ const int stripeCol = (MCUSize.x * mcuIdx) + MCUСol; + if (stripeCol < outStripe.width()) +- outStripe(stripeRow, stripeCol) = uint16_t(pix); ++ outStripe(MCURow, stripeCol) = uint16_t(pix); + } + } + ++mcuIdx; // We did just process one more MCU. +@@ -386,8 +381,7 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + // For the first row of each restart interval, use predictor mode 1 + // (per ITU-T T.81: first row always uses horizontal prediction). + // For the first row, prevStripe points to outStripe itself (unused +- // since predMode will be 1). +- const int predMode = isFirstRow ? 1 : predictorMode; ++ // since Use2DPred will be false). + const Array2DRef prevStripe = + isFirstRow + ? Array2DRef(outStripe) +@@ -399,7 +393,10 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + /*croppedHeight=*/frame.mcu.y) + .getAsArray2DRef(); + +- decodeRowN(outStripe, pred, predMode, prevStripe, ht, bs); ++ if (!isFirstRow && predictorMode != 1) ++ decodeRowN(outStripe, pred, prevStripe, ht, bs); ++ else ++ decodeRowN(outStripe, pred, prevStripe, ht, bs); + + // The predictor for the next line is the start of this line. + pred = CroppedArray2DRef(outStripe, +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index 3c1f4352..a35dd79c 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -77,10 +77,10 @@ private: + template + [[nodiscard]] std::array getInitialPreds() const; + +- template ++ template + __attribute__((always_inline)) inline void decodeRowN( + Array2DRef outStripe, Array2DRef pred, +- int predMode, Array2DRef prevStripe, ++ Array2DRef prevStripe, + std::array>, N_COMP> ht, + BitStreamerJPEG& bs) const; + diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0003-Apply-clang-format.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0003-Apply-clang-format.patch new file mode 100644 index 000000000000..5ecc90855909 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0003-Apply-clang-format.patch @@ -0,0 +1,106 @@ +From 44207a28281cf0a1721154b323ccd7edac824279 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 12:07:09 +0200 +Subject: [PATCH 3/7] Apply clang-format + +(cherry picked from commit fb89fea813f3ae5d931635c2d0a6c0e5679da471) +(cherry picked from commit 6bed940a3bbe5a1685caee10bc98e36b8f7154de) +--- + .../decompressors/LJpegDecoder.cpp | 13 +++++------ + .../decompressors/LJpegDecompressor.cpp | 22 +++++++++---------- + .../decompressors/LJpegDecompressor.h | 3 +-- + 3 files changed, 16 insertions(+), 22 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index 0c2bd001..4c781090 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -199,11 +199,10 @@ Buffer::size_type LJpegDecoder::decodeScan() { + + // Create a temporary raw image to decode the JPEG into. + // RawImage::create with dimensions already calls createData() internally. +- RawImage tmpRaw = RawImage::create( +- iPoint2D(jpegFrameDim.x, jpegFrameDim.y), RawImageType::UINT16, 1); ++ RawImage tmpRaw = RawImage::create(iPoint2D(jpegFrameDim.x, jpegFrameDim.y), ++ RawImageType::UINT16, 1); + +- const iRectangle2D tmpFrame = { +- {0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; ++ const iRectangle2D tmpFrame = {{0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; + const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; + + int numLJpegRowsPerRestartInterval; +@@ -213,8 +212,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + const int numMCUsPerRow = jpegFrameDim.x; + if (numMCUsPerRestartInterval % numMCUsPerRow != 0) + ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = +- numMCUsPerRestartInterval / numMCUsPerRow; ++ numLJpegRowsPerRestartInterval = numMCUsPerRestartInterval / numMCUsPerRow; + } + + LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, +@@ -234,8 +232,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + + for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { + for (int pack = 0; pack < widthPack; ++pack) { +- const int tileRow = +- implicit_cast(offY) + jpegRow * widthPack + pack; ++ const int tileRow = implicit_cast(offY) + jpegRow * widthPack + pack; + if (tileRow >= mRaw->dim.y) + continue; + const int srcCol = pack * outRowPixels; +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index 93ac87eb..e31184e4 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -246,9 +246,8 @@ void LJpegDecompressor::decodeRowN( + const int Ra = pred(MCURow, MCUСol); + const int stripeCol = MCUSize.x * mcuIdx + MCUСol; + const int Rb = prevStripe(MCURow, stripeCol); +- const int Rc = (mcuIdx > 0) +- ? prevStripe(MCURow, stripeCol - MCUSize.x) +- : Rb; ++ const int Rc = ++ (mcuIdx > 0) ? prevStripe(MCURow, stripeCol - MCUSize.x) : Rb; + prediction = computePrediction(predictorMode, Ra, Rb, Rc); + } + const int diff = (static_cast&>(ht[c])) +@@ -383,15 +382,14 @@ ByteStream::size_type LJpegDecompressor::decodeN() const { + // For the first row, prevStripe points to outStripe itself (unused + // since Use2DPred will be false). + const Array2DRef prevStripe = +- isFirstRow +- ? Array2DRef(outStripe) +- : CroppedArray2DRef( +- img, +- /*offsetCols=*/0, +- /*offsetRows=*/row - frame.mcu.y, +- /*croppedWidth=*/img.width(), +- /*croppedHeight=*/frame.mcu.y) +- .getAsArray2DRef(); ++ isFirstRow ? Array2DRef(outStripe) ++ : CroppedArray2DRef( ++ img, ++ /*offsetCols=*/0, ++ /*offsetRows=*/row - frame.mcu.y, ++ /*croppedWidth=*/img.width(), ++ /*croppedHeight=*/frame.mcu.y) ++ .getAsArray2DRef(); + + if (!isFirstRow && predictorMode != 1) + decodeRowN(outStripe, pred, prevStripe, ht, bs); +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index a35dd79c..672249c2 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -90,8 +90,7 @@ private: + public: + LJpegDecompressor(RawImage img, iRectangle2D imgFrame, Frame frame, + std::vector rec, +- int numLJpegRowsPerRestartInterval_, +- int predictorMode_, ++ int numLJpegRowsPerRestartInterval_, int predictorMode_, + Array1DRef input); + + [[nodiscard]] ByteStream::size_type decode() const; diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0004-Fix-MCU-1-2-layout-decoding.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0004-Fix-MCU-1-2-layout-decoding.patch new file mode 100644 index 000000000000..da719f8dc7a2 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0004-Fix-MCU-1-2-layout-decoding.patch @@ -0,0 +1,189 @@ +From 8481bfc5e82c8ab314fd5629e59d2498d63734b7 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sat, 18 Apr 2026 13:27:01 +0200 +Subject: [PATCH 4/7] Fix MCU{1,2} layout decoding +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Some DNG files encode 2-component tiles (e.g., 592×158) as JPEG frames with the component dimension packed horizontally into a narrower, taller JPEG (592×79, 2 components). The JPEG SOF reports half the tile height, with each row encoding two consecutive output rows' worth of data — components are always interleaved horizontally, so the effective decoded width per JPEG row is jpegFrameDim.x × N_COMP = 1184. + +LJpegDecompressor.cpp — allow MCU{1,2} construction and dispatch + +The constructor's MCU allowlist and decode()'s dispatch table gain {1,2}. This is needed so the decompressor can be instantiated when the caller passes MCU{1,2} (it will never be used for actual pixel output after the decoder fix below, but the validation and dispatch must not reject it at an early stage). + +LJpegDecoder.cpp — detect vertical MCU and route to the inverted reshape path + +Two complementary changes: + +* In the standard (non-inverted) path, after computing MCUSize = maxRes / jpegFrameDim, gate the direct-decode branch on MCUSize.x >= MCUSize.y. When MCUSize is purely vertical (e.g., {1,2}), fall through instead of decoding in-place. + +* Generalize the inverted reshape path to handle N_COMP > 1: + * effectiveJpegWidth = jpegFrameDim.x × N_COMP replaces jpegFrameDim.x everywhere in width calculations. + * The temporary decode buffer is sized effectiveJpegWidth × jpegFrameDim.y. + * MCU is set to {N_COMP, 1} (horizontal interleaving), not hardcoded {1,1}. + * The N_COMP != 1 guard is removed. + * widthPack is derived from effectiveJpegWidth / maxRes.x, so for the 2-comp example: 1184 / 592 = 2, matching the tile's 158 / 79 = 2× height ratio. + +This correctly reconstructs the full-width, full-height tile from the narrower, shorter JPEG frame, yielding proper CFA Bayer output instead of the full-image purple/orange vertical stripe pattern that occurred when the two components were written to separate rows. + +(cherry picked from commit d8577b1229438acac1afb0c4937e44cf7f04f6a5) +(cherry picked from commit ad2ade3248012a7e3f7b306cf4a97e885e942bff) +--- + .../decompressors/LJpegDecoder.cpp | 87 +++++++++++-------- + .../decompressors/LJpegDecompressor.cpp | 7 +- + 2 files changed, 54 insertions(+), 40 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index 4c781090..92d25a52 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -152,57 +152,68 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (MCUSize.area() != implicit_cast(N_COMP)) + ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); + +- const iRectangle2D imgFrame = { +- {static_cast(offX), static_cast(offY)}, +- {static_cast(w), static_cast(h)}}; +- const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; +- +- int numLJpegRowsPerRestartInterval; +- if (numMCUsPerRestartInterval == 0) { +- numLJpegRowsPerRestartInterval = jpegFrameDim.y; +- } else { +- const int numMCUsPerRow = jpegFrameDim.x; +- if (numMCUsPerRestartInterval % numMCUsPerRow != 0) +- ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = +- numMCUsPerRestartInterval / numMCUsPerRow; ++ // Standard MCU layouts have MCU.x >= MCU.y: {1,1}, {2,1}, {3,1}, ++ // {4,1}, {2,2}. If the MCU is purely vertical (e.g., {1,2}), the ++ // encoder uses horizontal component interleaving with wider effective ++ // JPEG rows that must be reshaped. Fall through to inverted reshape. ++ if (MCUSize.x >= MCUSize.y) { ++ const iRectangle2D imgFrame = { ++ {static_cast(offX), static_cast(offY)}, ++ {static_cast(w), static_cast(h)}}; ++ const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; ++ ++ int numLJpegRowsPerRestartInterval; ++ if (numMCUsPerRestartInterval == 0) { ++ numLJpegRowsPerRestartInterval = jpegFrameDim.y; ++ } else { ++ const int numMCUsPerRow = jpegFrameDim.x; ++ if (numMCUsPerRestartInterval % numMCUsPerRow != 0) ++ ThrowRDE("Restart interval is not a multiple of frame row size"); ++ numLJpegRowsPerRestartInterval = ++ numMCUsPerRestartInterval / numMCUsPerRow; ++ } ++ ++ LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, ++ numLJpegRowsPerRestartInterval, ++ implicit_cast(predictorMode), ++ input.peekRemainingBuffer().getAsArray1DRef()); ++ return d.decode(); + } +- +- LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, +- numLJpegRowsPerRestartInterval, +- implicit_cast(predictorMode), +- input.peekRemainingBuffer().getAsArray1DRef()); +- return d.decode(); + } + +- // Inverted reshape case (DJI/Blackmagic CinemaDNG): +- // JPEG frame is wider than tile, e.g. JPEG=8000x1500 1-comp, tile=4000x3000. +- // Each JPEG row contains 'widthPack' tile rows concatenated. +- if (N_COMP != 1) +- ThrowRDE("Inverted reshape only supported for single-component LJpeg"); +- +- if (jpegFrameDim.x % maxRes.x != 0) +- ThrowRDE("LJpeg frame width is not a multiple of tile width"); ++ // Inverted reshape case: ++ // The effective decoded width per JPEG row exceeds the tile width. ++ // This occurs when: ++ // (a) The JPEG frame is wider than the tile (DJI/Blackmagic CinemaDNG): ++ // e.g., JPEG=8000x1500 1-comp, tile=4000x3000. ++ // (b) Multi-component JPEG with vertical MCU ratio: ++ // e.g., JPEG=592x79 2-comp, tile=592x158 (effectiveWidth=1184). ++ // In both cases, components are interleaved horizontally, and each JPEG row ++ // contains 'widthPack' tile rows of pixel data concatenated. ++ const int effectiveJpegWidth = jpegFrameDim.x * N_COMP; ++ ++ if (effectiveJpegWidth % maxRes.x != 0) ++ ThrowRDE("Effective JPEG width is not a multiple of tile width"); + if (maxRes.y % jpegFrameDim.y != 0) + ThrowRDE("Tile height is not a multiple of LJpeg frame height"); + +- const int widthPack = jpegFrameDim.x / maxRes.x; ++ const int widthPack = effectiveJpegWidth / maxRes.x; + if (widthPack * jpegFrameDim.y != maxRes.y) + ThrowRDE("Inverted reshape dimensions mismatch"); + + if (widthPack < 1 || widthPack > 4) + ThrowRDE("Unexpected row packing factor: %d", widthPack); + +- // Decode into a temporary buffer at JPEG frame dimensions. +- // MCU is {1,1} since we have a single component. +- const auto MCUSize = iPoint2D{1, 1}; ++ // Decode into a temporary buffer at effective decoded dimensions. ++ // Components are always interleaved horizontally: MCU{N_COMP, 1}. ++ const auto MCUSize = iPoint2D{N_COMP, 1}; + + // Create a temporary raw image to decode the JPEG into. + // RawImage::create with dimensions already calls createData() internally. +- RawImage tmpRaw = RawImage::create(iPoint2D(jpegFrameDim.x, jpegFrameDim.y), +- RawImageType::UINT16, 1); ++ RawImage tmpRaw = RawImage::create( ++ iPoint2D(effectiveJpegWidth, jpegFrameDim.y), RawImageType::UINT16, 1); + +- const iRectangle2D tmpFrame = {{0, 0}, {jpegFrameDim.x, jpegFrameDim.y}}; ++ const iRectangle2D tmpFrame = {{0, 0}, {effectiveJpegWidth, jpegFrameDim.y}}; + const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; + + int numLJpegRowsPerRestartInterval; +@@ -226,8 +237,8 @@ Buffer::size_type LJpegDecoder::decodeScan() { + const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); + const auto outData = mRaw->getU16DataAsUncroppedArray2DRef(); + +- const int tileW = implicit_cast(w); +- const int cpp = implicit_cast(mRaw->getCpp()); ++ const auto tileW = implicit_cast(w); ++ const auto cpp = implicit_cast(mRaw->getCpp()); + const int outRowPixels = cpp * tileW; + + for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { +@@ -238,7 +249,7 @@ Buffer::size_type LJpegDecoder::decodeScan() { + const int srcCol = pack * outRowPixels; + const int dstCol = cpp * implicit_cast(offX); + // Contiguous row-segment copy. Bounds guaranteed by validation: +- // srcCol + outRowPixels <= widthPack * outRowPixels <= jpegFrameDim.x ++ // srcCol + outRowPixels <= widthPack * outRowPixels <= effectiveJpegWidth + std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), + sizeof(uint16_t) * outRowPixels); + } +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index e31184e4..0c0cef1d 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -102,8 +102,8 @@ LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + ThrowRDE("Frame has zero size"); + + if (iPoint2D{1, 1} != frame.mcu && iPoint2D{2, 1} != frame.mcu && +- iPoint2D{3, 1} != frame.mcu && iPoint2D{4, 1} != frame.mcu && +- iPoint2D{2, 2} != frame.mcu) ++ iPoint2D{1, 2} != frame.mcu && iPoint2D{3, 1} != frame.mcu && ++ iPoint2D{4, 1} != frame.mcu && iPoint2D{2, 2} != frame.mcu) + ThrowRDE("Unexpected MCU size: {%i, %i}", frame.mcu.x, frame.mcu.y); + + if (rec.size() != static_cast(frame.mcu.area())) +@@ -423,6 +423,9 @@ ByteStream::size_type LJpegDecompressor::decode() const { + if (frame.mcu == MCU<2, 1>) { + return decodeN>(); + } ++ if (frame.mcu == MCU<1, 2>) { ++ return decodeN>(); ++ } + break; + case 3: + if (frame.mcu == MCU<3, 1>) { diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0005-Refactor-clean-up-to-satisfy-clang-tidy.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0005-Refactor-clean-up-to-satisfy-clang-tidy.patch new file mode 100644 index 000000000000..9f3219a4a7d3 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0005-Refactor-clean-up-to-satisfy-clang-tidy.patch @@ -0,0 +1,399 @@ +From d30dec4472fd0fe005b3c8e8ebc970d52a9df579 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 13:13:56 +0200 +Subject: [PATCH 5/7] Refactor & clean-up to satisfy clang-tidy + +(cherry picked from commit aeb171913a3d980cd32cd35fbecc395dae7360ed) +(cherry picked from commit 9dde76f09480d3f49cffd4c473b46c5c573be39a) +--- + .../decompressors/LJpegDecompressor.cpp | 4 +- + .../decompressors/LJpegDecoder.cpp | 289 +++++++++--------- + .../decompressors/LJpegDecompressor.cpp | 7 +- + .../decompressors/LJpegDecompressor.h | 7 +- + 4 files changed, 160 insertions(+), 147 deletions(-) + +diff --git a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +index 83e406ab..2d1b1a63 100644 +--- a/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/fuzz/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -90,9 +90,11 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) { + const int numLJpegRowsPerRestartInterval = bs.getI32(); + const int predictorMode = bs.getByte(); + ++ const rawspeed::LJpegDecompressor::DecodeSettings settings{ ++ numLJpegRowsPerRestartInterval, predictorMode}; + rawspeed::LJpegDecompressor d( + mRaw, rawspeed::iRectangle2D(mRaw->dim.x, mRaw->dim.y), frame, rec, +- numLJpegRowsPerRestartInterval, predictorMode, ++ settings, + bs.getSubStream(/*offset=*/0).peekRemainingBuffer().getAsArray1DRef()); + mRaw->createData(); + (void)d.decode(); +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index 92d25a52..de827235 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -30,18 +30,139 @@ + #include "io/Buffer.h" + #include "io/ByteStream.h" + #include +-#include + #include + #include + #include + #include +-#include + #include + +-using std::copy_n; +- + namespace rawspeed { + ++namespace { ++ ++using PerCompRecipeVec = std::vector; ++ ++struct ScanSettings final { ++ RawImage raw; ++ iRectangle2D imgFrame; ++ iPoint2D jpegFrameDim; ++ iPoint2D maxRes; ++ LJpegDecompressor::DecodeSettings decode; ++ Array1DRef input; ++}; ++ ++[[nodiscard]] int ++getNumLJpegRowsPerRestartInterval(uint32_t numMCUsPerRestartInterval, ++ iPoint2D jpegFrameDim) { ++ if (numMCUsPerRestartInterval == 0) ++ return jpegFrameDim.y; ++ ++ const int numMCUsPerRow = jpegFrameDim.x; ++ if (numMCUsPerRestartInterval % numMCUsPerRow != 0) ++ ThrowRDE("Restart interval is not a multiple of frame row size"); ++ return implicit_cast(numMCUsPerRestartInterval) / numMCUsPerRow; ++} ++ ++[[nodiscard]] iPoint2D getMaxResolution(const RawImage& raw, iPoint2D maxDim, ++ int numComponents, ++ iPoint2D jpegFrameDim) { ++ if (implicit_cast(maxDim.x) * implicit_cast(raw->getCpp()) > ++ std::numeric_limits::max()) ++ ThrowRDE("Maximal output tile is too large"); ++ ++ const auto maxRes = ++ iPoint2D(implicit_cast(raw->getCpp()) * maxDim.x, maxDim.y); ++ if (maxRes.area() != numComponents * jpegFrameDim.area()) ++ ThrowRDE("LJpeg frame area does not match maximal tile area"); ++ ++ return maxRes; ++} ++ ++[[nodiscard]] iPoint2D ++getStandardMCUSize(int numComponents, iPoint2D jpegFrameDim, iPoint2D maxRes) { ++ if (jpegFrameDim.x > maxRes.x) ++ return {}; ++ ++ if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) ++ ThrowRDE("Maximal output tile size is not a multiple of LJpeg frame size"); ++ ++ const auto mcuSize = ++ iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; ++ if (mcuSize.area() != implicit_cast(numComponents)) ++ ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); ++ ++ if (mcuSize.x < mcuSize.y) ++ return {}; ++ ++ return mcuSize; ++} ++ ++[[nodiscard]] ByteStream::size_type ++decodeStandardScan(const ScanSettings& settings, iPoint2D mcuSize, ++ const PerCompRecipeVec& rec) { ++ const LJpegDecompressor::Frame jpegFrame = {mcuSize, settings.jpegFrameDim}; ++ LJpegDecompressor d(settings.raw, settings.imgFrame, jpegFrame, rec, ++ settings.decode, settings.input); ++ return d.decode(); ++} ++ ++void copyDeinterleavedRows(const RawImage& raw, RawImage tmpRaw, uint32_t offX, ++ uint32_t offY, uint32_t tileWidth, int widthPack) { ++ const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); ++ const auto outData = raw->getU16DataAsUncroppedArray2DRef(); ++ ++ const auto cpp = implicit_cast(raw->getCpp()); ++ const int outRowPixels = cpp * implicit_cast(tileWidth); ++ ++ for (int jpegRow = 0; jpegRow < tmpRaw->dim.y; ++jpegRow) { ++ for (int pack = 0; pack < widthPack; ++pack) { ++ const int tileRow = implicit_cast(offY) + jpegRow * widthPack + pack; ++ if (tileRow >= raw->dim.y) ++ continue; ++ ++ const int srcCol = pack * outRowPixels; ++ const int dstCol = cpp * implicit_cast(offX); ++ std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), ++ sizeof(uint16_t) * outRowPixels); ++ } ++ } ++} ++ ++[[nodiscard]] ByteStream::size_type ++decodeInvertedScan(const ScanSettings& settings, int numComponents, ++ uint32_t offX, uint32_t offY, uint32_t tileWidth, ++ const PerCompRecipeVec& rec) { ++ const int effectiveJpegWidth = settings.jpegFrameDim.x * numComponents; ++ ++ if (effectiveJpegWidth % settings.maxRes.x != 0) ++ ThrowRDE("Effective JPEG width is not a multiple of tile width"); ++ if (settings.maxRes.y % settings.jpegFrameDim.y != 0) ++ ThrowRDE("Tile height is not a multiple of LJpeg frame height"); ++ ++ const int widthPack = effectiveJpegWidth / settings.maxRes.x; ++ if (widthPack * settings.jpegFrameDim.y != settings.maxRes.y) ++ ThrowRDE("Inverted reshape dimensions mismatch"); ++ if (widthPack < 1 || widthPack > 4) ++ ThrowRDE("Unexpected row packing factor: %d", widthPack); ++ ++ const auto mcuSize = iPoint2D{numComponents, 1}; ++ RawImage tmpRaw = ++ RawImage::create(iPoint2D(effectiveJpegWidth, settings.jpegFrameDim.y), ++ RawImageType::UINT16, 1); ++ const iRectangle2D tmpFrame = {{0, 0}, ++ {effectiveJpegWidth, settings.jpegFrameDim.y}}; ++ const LJpegDecompressor::Frame jpegFrame = {mcuSize, settings.jpegFrameDim}; ++ ++ LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, settings.decode, ++ settings.input); ++ const auto consumed = d.decode(); ++ ++ copyDeinterleavedRows(settings.raw, tmpRaw, offX, offY, tileWidth, widthPack); ++ return consumed; ++} ++ ++} // namespace ++ + LJpegDecoder::LJpegDecoder(ByteStream bs, const RawImage& img) + : AbstractLJpegDecoder(bs, img) { + if (mRaw->getDataType() != RawImageType::UINT16) +@@ -113,149 +234,37 @@ Buffer::size_type LJpegDecoder::decodeScan() { + if (frame.compInfo[i].superH != 1 || frame.compInfo[i].superV != 1) + ThrowRDE("Unsupported subsampling"); + +- const int N_COMP = frame.cps; ++ const int numComponents = frame.cps; + +- std::vector rec; +- rec.reserve(N_COMP); +- std::generate_n(std::back_inserter(rec), N_COMP, +- [&rec, hts = getPrefixCodeDecoders(N_COMP), +- initPred = getInitialPredictors( +- N_COMP)]() -> LJpegDecompressor::PerComponentRecipe { ++ PerCompRecipeVec rec; ++ rec.reserve(numComponents); ++ std::generate_n(std::back_inserter(rec), numComponents, ++ [&rec, hts = getPrefixCodeDecoders(numComponents), ++ initPred = getInitialPredictors(numComponents)]() ++ -> LJpegDecompressor::PerComponentRecipe { + const auto i = implicit_cast(rec.size()); + return {*hts[i], initPred[i]}; + }); + + const auto jpegFrameDim = iPoint2D(frame.w, frame.h); +- +- if (implicit_cast(maxDim.x) * implicit_cast(mRaw->getCpp()) > +- std::numeric_limits::max()) +- ThrowRDE("Maximal output tile is too large"); +- + const auto maxRes = +- iPoint2D(implicit_cast(mRaw->getCpp()) * maxDim.x, maxDim.y); +- if (maxRes.area() != N_COMP * jpegFrameDim.area()) +- ThrowRDE("LJpeg frame area does not match maximal tile area"); +- +- // Detect whether the JPEG frame uses an inverted reshape (e.g. DJI/Blackmagic +- // CinemaDNG): JPEG frame is wider than tile and shorter, with packed rows. +- // Standard (Adobe): maxRes.x >= jpegFrameDim.x (tile is wider/equal) +- // Inverted (DJI): jpegFrameDim.x > maxRes.x (JPEG frame is wider) +- const bool invertedReshape = (jpegFrameDim.x > maxRes.x); +- if (!invertedReshape) { +- // Standard case: tile width is a multiple of JPEG frame width. +- if (maxRes.x % jpegFrameDim.x != 0 || maxRes.y % jpegFrameDim.y != 0) +- ThrowRDE( +- "Maximal output tile size is not a multiple of LJpeg frame size"); +- +- const auto MCUSize = +- iPoint2D{maxRes.x / jpegFrameDim.x, maxRes.y / jpegFrameDim.y}; +- if (MCUSize.area() != implicit_cast(N_COMP)) +- ThrowRDE("Unexpected MCU size, does not match LJpeg component count"); +- +- // Standard MCU layouts have MCU.x >= MCU.y: {1,1}, {2,1}, {3,1}, +- // {4,1}, {2,2}. If the MCU is purely vertical (e.g., {1,2}), the +- // encoder uses horizontal component interleaving with wider effective +- // JPEG rows that must be reshaped. Fall through to inverted reshape. +- if (MCUSize.x >= MCUSize.y) { +- const iRectangle2D imgFrame = { +- {static_cast(offX), static_cast(offY)}, +- {static_cast(w), static_cast(h)}}; +- const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; +- +- int numLJpegRowsPerRestartInterval; +- if (numMCUsPerRestartInterval == 0) { +- numLJpegRowsPerRestartInterval = jpegFrameDim.y; +- } else { +- const int numMCUsPerRow = jpegFrameDim.x; +- if (numMCUsPerRestartInterval % numMCUsPerRow != 0) +- ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = +- numMCUsPerRestartInterval / numMCUsPerRow; +- } +- +- LJpegDecompressor d(mRaw, imgFrame, jpegFrame, rec, +- numLJpegRowsPerRestartInterval, +- implicit_cast(predictorMode), +- input.peekRemainingBuffer().getAsArray1DRef()); +- return d.decode(); +- } +- } +- +- // Inverted reshape case: +- // The effective decoded width per JPEG row exceeds the tile width. +- // This occurs when: +- // (a) The JPEG frame is wider than the tile (DJI/Blackmagic CinemaDNG): +- // e.g., JPEG=8000x1500 1-comp, tile=4000x3000. +- // (b) Multi-component JPEG with vertical MCU ratio: +- // e.g., JPEG=592x79 2-comp, tile=592x158 (effectiveWidth=1184). +- // In both cases, components are interleaved horizontally, and each JPEG row +- // contains 'widthPack' tile rows of pixel data concatenated. +- const int effectiveJpegWidth = jpegFrameDim.x * N_COMP; +- +- if (effectiveJpegWidth % maxRes.x != 0) +- ThrowRDE("Effective JPEG width is not a multiple of tile width"); +- if (maxRes.y % jpegFrameDim.y != 0) +- ThrowRDE("Tile height is not a multiple of LJpeg frame height"); +- +- const int widthPack = effectiveJpegWidth / maxRes.x; +- if (widthPack * jpegFrameDim.y != maxRes.y) +- ThrowRDE("Inverted reshape dimensions mismatch"); +- +- if (widthPack < 1 || widthPack > 4) +- ThrowRDE("Unexpected row packing factor: %d", widthPack); +- +- // Decode into a temporary buffer at effective decoded dimensions. +- // Components are always interleaved horizontally: MCU{N_COMP, 1}. +- const auto MCUSize = iPoint2D{N_COMP, 1}; +- +- // Create a temporary raw image to decode the JPEG into. +- // RawImage::create with dimensions already calls createData() internally. +- RawImage tmpRaw = RawImage::create( +- iPoint2D(effectiveJpegWidth, jpegFrameDim.y), RawImageType::UINT16, 1); +- +- const iRectangle2D tmpFrame = {{0, 0}, {effectiveJpegWidth, jpegFrameDim.y}}; +- const LJpegDecompressor::Frame jpegFrame = {MCUSize, jpegFrameDim}; +- +- int numLJpegRowsPerRestartInterval; +- if (numMCUsPerRestartInterval == 0) { +- numLJpegRowsPerRestartInterval = jpegFrameDim.y; +- } else { +- const int numMCUsPerRow = jpegFrameDim.x; +- if (numMCUsPerRestartInterval % numMCUsPerRow != 0) +- ThrowRDE("Restart interval is not a multiple of frame row size"); +- numLJpegRowsPerRestartInterval = numMCUsPerRestartInterval / numMCUsPerRow; +- } +- +- LJpegDecompressor d(tmpRaw, tmpFrame, jpegFrame, rec, +- numLJpegRowsPerRestartInterval, +- implicit_cast(predictorMode), +- input.peekRemainingBuffer().getAsArray1DRef()); +- const auto consumed = d.decode(); +- +- // Deinterleave: each JPEG row of width (widthPack * tileW) maps to +- // widthPack consecutive tile rows of width tileW. +- const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); +- const auto outData = mRaw->getU16DataAsUncroppedArray2DRef(); +- +- const auto tileW = implicit_cast(w); +- const auto cpp = implicit_cast(mRaw->getCpp()); +- const int outRowPixels = cpp * tileW; +- +- for (int jpegRow = 0; jpegRow < jpegFrameDim.y; ++jpegRow) { +- for (int pack = 0; pack < widthPack; ++pack) { +- const int tileRow = implicit_cast(offY) + jpegRow * widthPack + pack; +- if (tileRow >= mRaw->dim.y) +- continue; +- const int srcCol = pack * outRowPixels; +- const int dstCol = cpp * implicit_cast(offX); +- // Contiguous row-segment copy. Bounds guaranteed by validation: +- // srcCol + outRowPixels <= widthPack * outRowPixels <= effectiveJpegWidth +- std::memcpy(&outData(tileRow, dstCol), &tmpData(jpegRow, srcCol), +- sizeof(uint16_t) * outRowPixels); +- } +- } +- +- return consumed; ++ getMaxResolution(mRaw, maxDim, numComponents, jpegFrameDim); ++ const ScanSettings settings{mRaw, ++ {{static_cast(offX), static_cast(offY)}, ++ {static_cast(w), static_cast(h)}}, ++ jpegFrameDim, ++ maxRes, ++ {getNumLJpegRowsPerRestartInterval( ++ numMCUsPerRestartInterval, jpegFrameDim), ++ implicit_cast(predictorMode)}, ++ input.peekRemainingBuffer().getAsArray1DRef()}; ++ ++ if (const auto mcuSize = ++ getStandardMCUSize(numComponents, jpegFrameDim, maxRes); ++ mcuSize.hasPositiveArea()) ++ return decodeStandardScan(settings, mcuSize, rec); ++ ++ return decodeInvertedScan(settings, numComponents, offX, offY, w, rec); + } + + } // namespace rawspeed +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index 0c0cef1d..e0a6d769 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -52,13 +52,12 @@ namespace rawspeed { + LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + Frame frame_, + std::vector rec_, +- int numLJpegRowsPerRestartInterval_, +- int predictorMode_, ++ DecodeSettings settings, + Array1DRef input_) + : mRaw(std::move(img)), input(input_), imgFrame(imgFrame_), + frame(std::move(frame_)), rec(std::move(rec_)), +- numLJpegRowsPerRestartInterval(numLJpegRowsPerRestartInterval_), +- predictorMode(predictorMode_) { ++ numLJpegRowsPerRestartInterval(settings.numLJpegRowsPerRestartInterval), ++ predictorMode(settings.predictorMode) { + + if (mRaw->getDataType() != RawImageType::UINT16) + ThrowRDE("Unexpected data type (%u)", +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.h b/src/librawspeed/decompressors/LJpegDecompressor.h +index 672249c2..11caad31 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.h ++++ b/src/librawspeed/decompressors/LJpegDecompressor.h +@@ -45,6 +45,10 @@ public: + const iPoint2D mcu; + const iPoint2D dim; + }; ++ struct DecodeSettings final { ++ const int numLJpegRowsPerRestartInterval; ++ const int predictorMode; ++ }; + struct PerComponentRecipe final { + const PrefixCodeDecoder<>& ht; + const uint16_t initPred; +@@ -90,8 +94,7 @@ private: + public: + LJpegDecompressor(RawImage img, iRectangle2D imgFrame, Frame frame, + std::vector rec, +- int numLJpegRowsPerRestartInterval_, int predictorMode_, +- Array1DRef input); ++ DecodeSettings settings, Array1DRef input); + + [[nodiscard]] ByteStream::size_type decode() const; + }; diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0006-Address-codeChecker-findings.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0006-Address-codeChecker-findings.patch new file mode 100644 index 000000000000..85e463f3bb04 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0006-Address-codeChecker-findings.patch @@ -0,0 +1,35 @@ +From 34c3514e1c81d6e166cee078ec28a289cf4f1e73 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 14:34:47 +0200 +Subject: [PATCH 6/7] Address codeChecker findings + +(cherry picked from commit a167a44f3a5034725be51b941bff7e00e7f751aa) +(cherry picked from commit ed6e667346b1bf41ca1e76bac9b0ee8230ee09b9) +--- + src/librawspeed/decompressors/LJpegDecoder.cpp | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) + +diff --git a/src/librawspeed/decompressors/LJpegDecoder.cpp b/src/librawspeed/decompressors/LJpegDecoder.cpp +index de827235..b35dc085 100644 +--- a/src/librawspeed/decompressors/LJpegDecoder.cpp ++++ b/src/librawspeed/decompressors/LJpegDecoder.cpp +@@ -20,6 +20,7 @@ + */ + + #include "decompressors/LJpegDecoder.h" ++#include "adt/Array1DRef.h" + #include "adt/Casts.h" + #include "adt/Invariant.h" + #include "adt/Point.h" +@@ -106,8 +107,9 @@ decodeStandardScan(const ScanSettings& settings, iPoint2D mcuSize, + return d.decode(); + } + +-void copyDeinterleavedRows(const RawImage& raw, RawImage tmpRaw, uint32_t offX, +- uint32_t offY, uint32_t tileWidth, int widthPack) { ++void copyDeinterleavedRows(const RawImage& raw, const RawImage& tmpRaw, ++ uint32_t offX, uint32_t offY, uint32_t tileWidth, ++ int widthPack) { + const auto tmpData = tmpRaw->getU16DataAsUncroppedArray2DRef(); + const auto outData = raw->getU16DataAsUncroppedArray2DRef(); + diff --git a/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0007-LJpeg-validate-predictor-mode-range.patch b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0007-LJpeg-validate-predictor-mode-range.patch new file mode 100644 index 000000000000..5304e94dc832 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/01_ljpeg_predictor_modes/0007-LJpeg-validate-predictor-mode-range.patch @@ -0,0 +1,31 @@ +From 09dceddbfe629e03f76e397943ef4526f586408c Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Thu, 30 Jul 2026 18:59:55 +0000 +Subject: [PATCH 7/7] LJpeg: validate predictor mode range + +Reject predictor modes outside 1..7 in the LJpegDecompressor constructor +instead of relying on callers to pre-validate. + +Origin: darktable-org/rawspeed#963, commit cf87137 ("Address clang-tidy, +sanitizer and fuzzer findings"). Only the LJpegDecompressor hunk is taken +here, because it depends on the predictorMode member introduced earlier in +this series; the remaining hunks of that commit are static-analyzer fixes +unrelated to LJpeg and are proposed separately. +--- + src/librawspeed/decompressors/LJpegDecompressor.cpp | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/src/librawspeed/decompressors/LJpegDecompressor.cpp b/src/librawspeed/decompressors/LJpegDecompressor.cpp +index e0a6d769..f62b5ff2 100644 +--- a/src/librawspeed/decompressors/LJpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/LJpegDecompressor.cpp +@@ -116,6 +116,9 @@ LJpegDecompressor::LJpegDecompressor(RawImage img, iRectangle2D imgFrame_, + if (numLJpegRowsPerRestartInterval < 1) + ThrowRDE("Number of rows per restart interval must be positives"); + ++ if (predictorMode < 1 || predictorMode > 7) ++ ThrowRDE("Unsupported predictor mode: %i", predictorMode); ++ + if (static_cast(frame.mcu.x) * frame.dim.x > + std::numeric_limits::max() || + static_cast(frame.mcu.y) * frame.dim.y > diff --git a/tools/rawspeed_proraw/upstream_split/02_dng_unique_camera_model/0001-Support-UniqueCameraModel-Exif-tag-for-DNG.patch b/tools/rawspeed_proraw/upstream_split/02_dng_unique_camera_model/0001-Support-UniqueCameraModel-Exif-tag-for-DNG.patch new file mode 100644 index 000000000000..668e2ec83ab9 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/02_dng_unique_camera_model/0001-Support-UniqueCameraModel-Exif-tag-for-DNG.patch @@ -0,0 +1,48 @@ +From 6ffbcd935f21bfc33bd8271f369d56ca079a4f8b Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 14:19:22 +0200 +Subject: [PATCH] Support UniqueCameraModel Exif tag for DNG +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Blackmagic CinemaDNG files don't include TIFF Make/Model tags — they only have UniqueCameraModel (tag 50708), which is valid per the DNG spec. In decodeMetaDataInternal(), the code called getID() unconditionally. + +Instead of calling getID() in a try/catch (which logged a spurious error), check for MAKE+MODEL existence first. If present, call getID() as before. If absent, fall back to UNIQUECAMERAMODEL for both make and model fields — mirroring the approach already used in checkSupportInternal(). + +(cherry picked from commit a9dbfad873923c802a7c156e83899d5a5df19019) +(cherry picked from commit 11921a6519258d9fedd3313fc968b2247faf6368) +--- + src/librawspeed/decoders/DngDecoder.cpp | 18 +++++++++++++----- + 1 file changed, 13 insertions(+), 5 deletions(-) + +diff --git a/src/librawspeed/decoders/DngDecoder.cpp b/src/librawspeed/decoders/DngDecoder.cpp +index ecd11989..ae972d2a 100644 +--- a/src/librawspeed/decoders/DngDecoder.cpp ++++ b/src/librawspeed/decoders/DngDecoder.cpp +@@ -699,12 +699,20 @@ void DngDecoder::decodeMetaDataInternal(const CameraMetaData* meta) { + + TiffID id; + +- try { ++ if (mRootIFD->hasEntryRecursive(TiffTag::MAKE) && ++ mRootIFD->hasEntryRecursive(TiffTag::MODEL)) { + id = mRootIFD->getID(); +- } catch (const RawspeedException& e) { +- mRaw->setError(e.what()); +- // not all dngs have MAKE/MODEL entries, +- // will be dealt with by using UNIQUECAMERAMODEL below ++ } else if (mRootIFD->hasEntryRecursive(TiffTag::UNIQUECAMERAMODEL)) { ++ // Not all DNGs have MAKE/MODEL entries (e.g. Blackmagic CinemaDNG). ++ // Fall back to UNIQUECAMERAMODEL for identification. ++ std::string unique = ++ mRootIFD->getEntryRecursive(TiffTag::UNIQUECAMERAMODEL)->getString(); ++ if (unique.empty()) ++ ThrowRDE("UNIQUECAMERAMODEL is empty"); ++ id.make = unique; ++ id.model = unique; ++ } else { ++ ThrowRDE("DNG has neither MAKE/MODEL nor UNIQUECAMERAMODEL"); + } + + // Set the make and model diff --git a/tools/rawspeed_proraw/upstream_split/03_dng_12bit_jpeg_errmsg/0001-Add-more-meaningful-error-message-for-12-bit-JPG-fil.patch b/tools/rawspeed_proraw/upstream_split/03_dng_12bit_jpeg_errmsg/0001-Add-more-meaningful-error-message-for-12-bit-JPG-fil.patch new file mode 100644 index 000000000000..22fba061406b --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/03_dng_12bit_jpeg_errmsg/0001-Add-more-meaningful-error-message-for-12-bit-JPG-fil.patch @@ -0,0 +1,118 @@ +From ae734e86a1a584f7b39155fd9b9ca7dec7f38d21 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Sun, 19 Apr 2026 16:25:44 +0200 +Subject: [PATCH] Add more meaningful error message for 12-bit JPG files + +Some Blackmagic CinemaDNG files (Pocket Cinema Camera 4K files (3)/(4), Micro Cinema Camera (1)) use SOF1 (Extended Sequential DCT) at 12-bit precision but label tiles as TIFF compression=7 (lossless JPEG). The lossless JPEG decoder hit a DQT marker and threw "Not a valid RAW file." + +On libjpeg-turbo 2.1.5, the error is now "Unsupported JPEG data precision 12" (much clearer). + +Future work: on systems with libjpeg-turbo 3.0+, we can enable full 12-bit lossy JPEG decoding. + +(cherry picked from commit 8e10cee704b021a6a092da142cf0215ea7aab38c) +(cherry picked from commit 2840416e1715a295ba11757e87d997031c4591be) +--- + .../decompressors/AbstractDngDecompressor.cpp | 67 +++++++++++++++++++ + .../decompressors/JpegDecompressor.cpp | 4 ++ + 2 files changed, 71 insertions(+) + +diff --git a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +index b828a4fe..c68044d9 100644 +--- a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp ++++ b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +@@ -51,6 +51,64 @@ + + namespace rawspeed { + ++namespace { ++ ++// Some DNG files (e.g. Blackmagic CinemaDNG) use TIFF compression=7 ++// (lossless JPEG) but the actual tile data contains lossy DCT JPEG ++// (SOF0/SOF1/SOF2 with DQT). Detect this by scanning the first few ++// JPEG markers in the tile stream. ++[[nodiscard]] bool tileContainsLossyJpeg(const ByteStream& bs) { ++ const auto remaining = bs.getRemainSize(); ++ if (remaining < 4) ++ return false; ++ ++ // Must start with JPEG SOI marker ++ if (bs.peekByte(0) != 0xFF || bs.peekByte(1) != 0xD8) ++ return false; ++ ++ // Scan markers after SOI. Stop after a reasonable number of bytes. ++ const auto limit = ++ std::min(remaining, static_cast(1024)); ++ ByteStream::size_type pos = 2; ++ ++ while (pos + 3 < limit) { ++ if (bs.peekByte(pos) != 0xFF) ++ return false; // Invalid marker - stop scanning ++ ++ const uint8_t marker = bs.peekByte(pos + 1); ++ ++ // DQT (quantization table) is definitive proof of lossy JPEG ++ if (marker == 0xDB) // DQT ++ return true; ++ ++ // SOF0/SOF1/SOF2 = lossy DCT-based JPEG ++ if (marker == 0xC0 || marker == 0xC1 || marker == 0xC2) ++ return true; ++ ++ // SOF3 = lossless - this is what compression=7 should be ++ if (marker == 0xC3) ++ return false; ++ ++ // SOS = start of scan data - stop scanning ++ if (marker == 0xDA) ++ return false; ++ ++ // Skip this marker segment ++ if (pos + 4 > remaining) ++ return false; ++ const auto segLen = static_cast( ++ (static_cast(bs.peekByte(pos + 2)) << 8) | ++ static_cast(bs.peekByte(pos + 3))); ++ if (segLen < 2) ++ return false; ++ pos += 2 + segLen; ++ } ++ ++ return false; ++} ++ ++} // namespace ++ + template <> void AbstractDngDecompressor::decompressThread<1>() const noexcept { + #ifdef HAVE_OPENMP + #pragma omp for schedule(static) +@@ -116,6 +174,15 @@ template <> void AbstractDngDecompressor::decompressThread<7>() const noexcept { + for (const auto& e : + Array1DRef(slices.data(), implicit_cast(slices.size()))) { + try { ++#ifdef HAVE_JPEG ++ // Some cameras (e.g. Blackmagic CinemaDNG) mislabel lossy DCT JPEG ++ // tiles as compression=7 (lossless JPEG). Detect and redirect. ++ if (tileContainsLossyJpeg(e.bs)) { ++ JpegDecompressor j(e.bs.peekBuffer(e.bs.getRemainSize()), mRaw); ++ j.decode(e.offX, e.offY); ++ continue; ++ } ++#endif + LJpegDecoder d(e.bs, mRaw); + d.decode(e.offX, e.offY, e.width, e.height, + iPoint2D(e.dsc.tileW, e.dsc.tileH), mFixLjpeg); +diff --git a/src/librawspeed/decompressors/JpegDecompressor.cpp b/src/librawspeed/decompressors/JpegDecompressor.cpp +index 569bb037..a0a4da39 100644 +--- a/src/librawspeed/decompressors/JpegDecompressor.cpp ++++ b/src/librawspeed/decompressors/JpegDecompressor.cpp +@@ -139,6 +139,10 @@ void JpegDecompressor::decode(uint32_t offX, + if (JPEG_HEADER_OK != jpeg_read_header(&dinfo, static_cast(true))) + ThrowRDE("Unable to read JPEG header"); + ++ if (dinfo.data_precision != 8) ++ ThrowRDE("Lossy JPEG tiles with %d-bit precision are not yet supported.", ++ dinfo.data_precision); ++ + jpeg_start_decompress(&dinfo); + if (dinfo.output_components != static_cast(mRaw->getCpp())) + ThrowRDE("Component count doesn't match"); diff --git a/tools/rawspeed_proraw/upstream_split/04_analyzer_hardening/0001-Address-static-analyzer-sanitizer-and-fuzzer-finding.patch b/tools/rawspeed_proraw/upstream_split/04_analyzer_hardening/0001-Address-static-analyzer-sanitizer-and-fuzzer-finding.patch new file mode 100644 index 000000000000..c8bb8b03f23e --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/04_analyzer_hardening/0001-Address-static-analyzer-sanitizer-and-fuzzer-finding.patch @@ -0,0 +1,74 @@ +From 36d624c28103be782864f1a0894509d683458f51 Mon Sep 17 00:00:00 2001 +From: Philipp Lutz +Date: Thu, 30 Jul 2026 19:07:48 +0000 +Subject: [PATCH] Address static-analyzer, sanitizer and fuzzer findings + +Three independent fixes surfaced by clang-tidy / the static analyzer and by +sanitizer + fuzzer runs: + +- SimpleTiffDecoder: value-initialize raw/width/height/off/c2 in the + constructor instead of leaving them indeterminate until + prepareForRawDecoding() runs. +- AbstractDngDecompressor: include explicitly rather than + relying on it arriving transitively. +- FileReader: give the unique_ptr a plain function-pointer deleter and take + the address of std::fclose, and annotate the unix.Stream report as a + false positive -- the stream is closed by the unique_ptr deleter on every + exit path, which the analyzer does not model. + +Origin: darktable-org/rawspeed#963, commit cf87137 ("Address clang-tidy, +sanitizer and fuzzer findings"). None of these hunks touch LJpeg, so they +stand on their own; that commit's LJpegDecompressor hunk depends on the +predictor-mode work and is proposed with it instead. +--- + src/librawspeed/decoders/SimpleTiffDecoder.h | 3 ++- + .../decompressors/AbstractDngDecompressor.cpp | 1 + + src/librawspeed/io/FileReader.cpp | 9 ++++++--- + 3 files changed, 9 insertions(+), 4 deletions(-) + +diff --git a/src/librawspeed/decoders/SimpleTiffDecoder.h b/src/librawspeed/decoders/SimpleTiffDecoder.h +index 1fa79fdc..e08b5e31 100644 +--- a/src/librawspeed/decoders/SimpleTiffDecoder.h ++++ b/src/librawspeed/decoders/SimpleTiffDecoder.h +@@ -39,7 +39,8 @@ class SimpleTiffDecoder : public AbstractTiffDecoder { + + public: + SimpleTiffDecoder(TiffRootIFDOwner&& root, Buffer file) +- : AbstractTiffDecoder(std::move(root), file) {} ++ : AbstractTiffDecoder(std::move(root), file), raw(nullptr), width(0), ++ height(0), off(0), c2(0) {} + + void prepareForRawDecoding(); + +diff --git a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +index b828a4fe..81152d0f 100644 +--- a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp ++++ b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +@@ -35,6 +35,7 @@ + #include "io/ByteStream.h" + #include "io/Endianness.h" + #include "io/IOException.h" ++#include + #include + #include + #include +diff --git a/src/librawspeed/io/FileReader.cpp b/src/librawspeed/io/FileReader.cpp +index 2378ba4c..79d0f18f 100644 +--- a/src/librawspeed/io/FileReader.cpp ++++ b/src/librawspeed/io/FileReader.cpp +@@ -53,9 +53,12 @@ FileReader::readFile() const { + size_t fileSize = 0; + + #if defined(__unix__) || defined(__APPLE__) +- auto fclose = [](std::FILE* fp) { std::fclose(fp); }; +- using file_ptr = std::unique_ptr; +- file_ptr file(fopen(fileName, "rb"), fclose); ++ using file_ptr = std::unique_ptr; ++ // The opened stream is owned by `file`, whose deleter (std::fclose) closes it ++ // on every exit path. The static analyzer does not model the unique_ptr ++ // destructor, so it wrongly reports a leak here. ++ // codechecker_false_positive [unix.Stream] close done by unique_ptr deleter ++ file_ptr file(std::fopen(fileName, "rb"), &std::fclose); + + if (file == nullptr) + ThrowFIE("Could not open file \"%s\".", fileName); diff --git a/tools/rawspeed_proraw/upstream_split/05_jpegxl_dng17/0001-Add-JPEG-XL-DNG-1.7-Compression-52546-decompressor.patch b/tools/rawspeed_proraw/upstream_split/05_jpegxl_dng17/0001-Add-JPEG-XL-DNG-1.7-Compression-52546-decompressor.patch new file mode 100644 index 000000000000..c588b3578ed3 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/05_jpegxl_dng17/0001-Add-JPEG-XL-DNG-1.7-Compression-52546-decompressor.patch @@ -0,0 +1,412 @@ +From 7c37eab0e4eb6e157b720c4cac7189c334ea8a9e Mon Sep 17 00:00:00 2001 +From: Mayk Thewessen +Date: Sun, 21 Jun 2026 22:19:05 +0200 +Subject: [PATCH] Add JPEG XL (DNG 1.7 / Compression 52546) decompressor + +Adds libjxl-backed decoding of DNG tiles compressed as JPEG XL +(TIFF Compression 52546, DNG 1.7), as used by Apple ProRAW on the +iPhone 16. Structured like the existing lossy-JPEG path: + +- cmake: WITH_JPEGXL option + pkg-config libjxl detection -> HAVE_JPEGXL +- config.h.in: HAVE_JPEGXL define +- JpegXlDecompressor: RAII-guarded libjxl decode loop, validates channel + count and dimensions, clipped copy of the decoded tile into the raw buffer +- DngDecoder: accept compression 52546 in dropUnsupportedChunks +- AbstractDngDecompressor: dispatch compression 52546 to the JPEG XL decoder + +Gated behind HAVE_JPEGXL; when disabled a #pragma message warns and the +compression is reported unsupported. + +(cherry picked from commit 2c3dfc5779b604b647956ef2f4c292e943ab1d79) +--- + CMakeLists.txt | 1 + + cmake/src-dependencies.cmake | 23 +++ + src/config.h.in | 2 + + src/librawspeed/decoders/DngDecoder.cpp | 12 ++ + .../decompressors/AbstractDngDecompressor.cpp | 35 +++++ + src/librawspeed/decompressors/CMakeLists.txt | 6 + + .../decompressors/JpegXlDecompressor.cpp | 140 ++++++++++++++++++ + .../decompressors/JpegXlDecompressor.h | 56 +++++++ + 8 files changed, 275 insertions(+) + create mode 100644 src/librawspeed/decompressors/JpegXlDecompressor.cpp + create mode 100644 src/librawspeed/decompressors/JpegXlDecompressor.h + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index 2346db8d..af15a818 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -80,6 +80,7 @@ else() + set(ALLOW_DOWNLOADING_PUGIXML OFF CACHE BOOL "If pugixml src tree is not found in location specified by PUGIXML_PATH, do fetch the archive from internet" FORCE) + endif() + option(WITH_JPEG "Enable JPEG support for DNG Lossy JPEG support" ON) ++option(WITH_JPEGXL "Enable JPEG XL support for DNG 1.7 JPEG XL compression" ON) + option(WITH_ZLIB "Enable ZLIB support for DNG deflate support" ON) + if(WITH_ZLIB) + option(USE_BUNDLED_ZLIB "Build and use zlib in-tree" OFF) +diff --git a/cmake/src-dependencies.cmake b/cmake/src-dependencies.cmake +index a9a887e6..59619ea8 100644 +--- a/cmake/src-dependencies.cmake ++++ b/cmake/src-dependencies.cmake +@@ -184,6 +184,29 @@ else() + endif() + add_feature_info("Lossy JPEG decoding" HAVE_JPEG "used for DNG Lossy JPEG compression decoding") + ++unset(HAVE_JPEGXL) ++if(WITH_JPEGXL) ++ message(STATUS "Looking for JPEG XL (libjxl)") ++ find_package(PkgConfig QUIET) ++ if(PkgConfig_FOUND) ++ pkg_check_modules(libjxl IMPORTED_TARGET libjxl) ++ endif() ++ if(NOT libjxl_FOUND) ++ message(SEND_ERROR "Did not find libjxl! Either install jpeg-xl, or pass -DWITH_JPEGXL=OFF to disable JPEG XL.") ++ else() ++ message(STATUS "Looking for JPEG XL - found (${libjxl_VERSION})") ++ set(HAVE_JPEGXL 1) ++ target_link_libraries(rawspeed PRIVATE PkgConfig::libjxl) ++ set_package_properties(libjxl PROPERTIES ++ TYPE RECOMMENDED ++ DESCRIPTION "JPEG XL reference codec library" ++ PURPOSE "Used for decoding DNG JPEG XL (DNG 1.7) compression") ++ endif() ++else() ++ message(STATUS "JPEG XL is disabled, DNG JPEG XL (DNG 1.7) support won't be available.") ++endif() ++add_feature_info("JPEG XL decoding" HAVE_JPEGXL "used for DNG JPEG XL (DNG 1.7) compression decoding") ++ + unset(HAVE_ZLIB) + if (WITH_ZLIB) + message(STATUS "Looking for ZLIB") +diff --git a/src/config.h.in b/src/config.h.in +index 623d1417..0e42746e 100644 +--- a/src/config.h.in ++++ b/src/config.h.in +@@ -65,6 +65,8 @@ static_assert(RAWSPEED_LARGEPAGESIZE >= RAWSPEED_PAGESIZE, + #cmakedefine HAVE_JPEG + #cmakedefine HAVE_JPEG_MEM_SRC + ++#cmakedefine HAVE_JPEGXL ++ + #cmakedefine HAVE_CXX_THREAD_LOCAL + #cmakedefine HAVE_GCC_THREAD_LOCAL + +diff --git a/src/librawspeed/decoders/DngDecoder.cpp b/src/librawspeed/decoders/DngDecoder.cpp +index ecd11989..4d5cace6 100644 +--- a/src/librawspeed/decoders/DngDecoder.cpp ++++ b/src/librawspeed/decoders/DngDecoder.cpp +@@ -119,6 +119,9 @@ void DngDecoder::dropUnsuportedChunks(std::vector* data) { + case 9: // VC-5 as used by GoPro + #ifdef HAVE_JPEG + case 0x884c: // lossy JPEG ++#endif ++#ifdef HAVE_JPEGXL ++ case 52546: // JPEG XL (DNG 1.7) + #endif + // no change, if supported, then is still supported. + break; +@@ -140,6 +143,15 @@ void DngDecoder::dropUnsuportedChunks(std::vector* data) { + "chunk, but the jpeg support was " + "disabled at build!"); + [[clang::fallthrough]]; ++#endif ++#ifndef HAVE_JPEGXL ++ case 52546: // JPEG XL (DNG 1.7) ++#pragma message \ ++ "JPEG XL is not present! DNG JPEG XL compression will not be supported!" ++ writeLog(DEBUG_PRIO::WARNING, "DNG Decoder: found JPEG XL-encoded " ++ "chunk, but JPEG XL support was " ++ "disabled at build!"); ++ [[clang::fallthrough]]; + #endif + default: + supported = false; +diff --git a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +index b828a4fe..83b27bea 100644 +--- a/src/librawspeed/decompressors/AbstractDngDecompressor.cpp ++++ b/src/librawspeed/decompressors/AbstractDngDecompressor.cpp +@@ -49,6 +49,10 @@ + #include "decompressors/JpegDecompressor.h" + #endif + ++#ifdef HAVE_JPEGXL ++#include "decompressors/JpegXlDecompressor.h" ++#endif ++ + namespace rawspeed { + + template <> void AbstractDngDecompressor::decompressThread<1>() const noexcept { +@@ -201,6 +205,29 @@ void AbstractDngDecompressor::decompressThread<0x884c>() const noexcept { + } + #endif + ++#ifdef HAVE_JPEGXL ++template <> ++void AbstractDngDecompressor::decompressThread<52546>() const noexcept { ++#ifdef HAVE_OPENMP ++#pragma omp for schedule(static) ++#endif ++ for (const auto& e : ++ Array1DRef(slices.data(), implicit_cast(slices.size()))) { ++ try { ++ JpegXlDecompressor j(e.bs.peekBuffer(e.bs.getRemainSize()), mRaw); ++ j.decode(e.offX, e.offY); ++ } catch (const RawDecoderException& err) { ++ mRaw->setError(err.what()); ++ } catch (const IOException& err) { ++ mRaw->setError(err.what()); ++ } catch (...) { ++ // We should not get any other exception type here. ++ __builtin_unreachable(); ++ } ++ } ++} ++#endif ++ + void AbstractDngDecompressor::decompressThread() const noexcept { + invariant(mRaw->dim.x > 0); + invariant(mRaw->dim.y > 0); +@@ -232,6 +259,14 @@ void AbstractDngDecompressor::decompressThread() const noexcept { + #else + #pragma message "JPEG is not present! Lossy JPEG DNG will not be supported!" + mRaw->setError("jpeg support is disabled."); ++#endif ++ } else if (compression == 52546) { ++ /* JPEG XL (DNG 1.7) */ ++#ifdef HAVE_JPEGXL ++ decompressThread<52546>(); ++#else ++#pragma message "JPEG XL is not present! DNG JPEG XL will not be supported!" ++ mRaw->setError("JPEG XL support is disabled."); + #endif + } else { + mRaw->setError("AbstractDngDecompressor: Unknown compression"); +diff --git a/src/librawspeed/decompressors/CMakeLists.txt b/src/librawspeed/decompressors/CMakeLists.txt +index 8933a3ad..20627c34 100644 +--- a/src/librawspeed/decompressors/CMakeLists.txt ++++ b/src/librawspeed/decompressors/CMakeLists.txt +@@ -26,6 +26,8 @@ FILE(GLOB SOURCES + "JpegDecompressor.cpp" + "JpegDecompressor.h" + "JpegMarkers.h" ++ "JpegXlDecompressor.cpp" ++ "JpegXlDecompressor.h" + "KodakDecompressor.cpp" + "KodakDecompressor.h" + "LJpegDecoder.cpp" +@@ -82,4 +84,8 @@ if(WITH_JPEG AND TARGET JPEG::JPEG) + target_link_libraries(rawspeed_decompressors PUBLIC JPEG::JPEG) + endif() + ++if(WITH_JPEGXL AND TARGET PkgConfig::libjxl) ++ target_link_libraries(rawspeed_decompressors PUBLIC PkgConfig::libjxl) ++endif() ++ + target_link_libraries(rawspeed PRIVATE rawspeed_decompressors) +diff --git a/src/librawspeed/decompressors/JpegXlDecompressor.cpp b/src/librawspeed/decompressors/JpegXlDecompressor.cpp +new file mode 100644 +index 00000000..7b7b4702 +--- /dev/null ++++ b/src/librawspeed/decompressors/JpegXlDecompressor.cpp +@@ -0,0 +1,140 @@ ++/* ++ RawSpeed - RAW file decoder. ++ ++ Copyright (C) 2026 darktable developers ++ ++ This library is free software; you can redistribute it and/or ++ modify it under the terms of the GNU Lesser General Public ++ License as published by the Free Software Foundation; either ++ version 2 of the License, or (at your option) any later version. ++ ++ This library is distributed in the hope that it will be useful, ++ but WITHOUT ANY WARRANTY; without even the implied warranty of ++ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ++ Lesser General Public License for more details. ++ ++ You should have received a copy of the GNU Lesser General Public ++ License along with this library; if not, write to the Free Software ++ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ++*/ ++ ++#include "rawspeedconfig.h" // IWYU pragma: keep ++ ++#ifdef HAVE_JPEGXL ++ ++#include "adt/Array2DRef.h" ++#include "adt/Point.h" ++#include "decoders/RawDecoderException.h" ++#include "decompressors/JpegXlDecompressor.h" ++#include ++#include ++#include ++#include ++#include ++ ++using std::min; ++ ++namespace rawspeed { ++ ++namespace { ++// RAII wrapper so JxlDecoderDestroy runs on every exit path (incl. throws). ++struct JxlDecoderGuard final { ++ JxlDecoder* dec; ++ explicit JxlDecoderGuard(JxlDecoder* d) : dec(d) {} ++ JxlDecoderGuard(const JxlDecoderGuard&) = delete; ++ JxlDecoderGuard(JxlDecoderGuard&&) = delete; ++ JxlDecoderGuard& operator=(const JxlDecoderGuard&) = delete; ++ JxlDecoderGuard& operator=(JxlDecoderGuard&&) = delete; ++ ~JxlDecoderGuard() { JxlDecoderDestroy(dec); } ++}; ++} // namespace ++ ++void JpegXlDecompressor::decode(uint32_t offX, uint32_t offY) { ++ JxlDecoder* dec = JxlDecoderCreate(nullptr); ++ if (dec == nullptr) ++ ThrowRDE("JXL: JxlDecoderCreate failed"); ++ JxlDecoderGuard guard(dec); ++ ++ if (JXL_DEC_SUCCESS != ++ JxlDecoderSubscribeEvents(dec, JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE)) ++ ThrowRDE("JXL: JxlDecoderSubscribeEvents failed"); ++ ++ // rawspeed/darktable handle orientation themselves; do not auto-rotate. ++ if (JXL_DEC_SUCCESS != JxlDecoderSetKeepOrientation(dec, JXL_TRUE)) ++ ThrowRDE("JXL: JxlDecoderSetKeepOrientation failed"); ++ ++ if (JXL_DEC_SUCCESS != ++ JxlDecoderSetInput(dec, input.begin(), input.getSize())) ++ ThrowRDE("JXL: JxlDecoderSetInput failed"); ++ JxlDecoderCloseInput(dec); ++ ++ const uint32_t cpp = mRaw->getCpp(); ++ const JxlPixelFormat fmt = {/*num_channels=*/cpp, ++ /*data_type=*/JXL_TYPE_UINT16, ++ /*endianness=*/JXL_LITTLE_ENDIAN, ++ /*align=*/0}; ++ ++ JxlBasicInfo info = {}; ++ uint32_t jxl_w = 0; ++ uint32_t jxl_h = 0; ++ std::vector pixels; ++ ++ for (;;) { ++ const JxlDecoderStatus status = JxlDecoderProcessInput(dec); ++ if (status == JXL_DEC_ERROR) ++ ThrowRDE("JXL: decoding error"); ++ if (status == JXL_DEC_NEED_MORE_INPUT) ++ ThrowRDE("JXL: needs more input (truncated tile?)"); ++ if (status == JXL_DEC_BASIC_INFO) { ++ if (JXL_DEC_SUCCESS != JxlDecoderGetBasicInfo(dec, &info)) ++ ThrowRDE("JXL: JxlDecoderGetBasicInfo failed"); ++ jxl_w = info.xsize; ++ jxl_h = info.ysize; ++ if (info.num_color_channels != cpp) ++ ThrowRDE("JXL: color channel count %u does not match cpp %u", ++ info.num_color_channels, cpp); ++ continue; ++ } ++ if (status == JXL_DEC_NEED_IMAGE_OUT_BUFFER) { ++ size_t buf_size = 0; ++ if (JXL_DEC_SUCCESS != JxlDecoderImageOutBufferSize(dec, &fmt, &buf_size)) ++ ThrowRDE("JXL: JxlDecoderImageOutBufferSize failed"); ++ pixels.resize(buf_size / sizeof(uint16_t)); ++ if (JXL_DEC_SUCCESS != ++ JxlDecoderSetImageOutBuffer(dec, &fmt, pixels.data(), buf_size)) ++ ThrowRDE("JXL: JxlDecoderSetImageOutBuffer failed"); ++ continue; ++ } ++ if (status == JXL_DEC_FULL_IMAGE) ++ continue; // image now in `pixels` ++ if (status == JXL_DEC_SUCCESS) ++ break; ++ ThrowRDE("JXL: unexpected decoder status %d", static_cast(status)); ++ } ++ ++ if (pixels.empty() || jxl_w == 0 || jxl_h == 0) ++ ThrowRDE("JXL: no pixel data decoded"); ++ ++ // Copy the decoded interleaved uint16 tile into the raw buffer at ++ // (offX,offY). ++ const uint32_t copy_w = min(static_cast(mRaw->dim.x) - offX, jxl_w); ++ const uint32_t copy_h = min(static_cast(mRaw->dim.y) - offY, jxl_h); ++ ++ const Array2DRef out(mRaw->getU16DataAsUncroppedArray2DRef()); ++ for (uint32_t row = 0; row < copy_h; ++row) { ++ for (uint32_t col = 0; col < cpp * copy_w; ++col) { ++ out(static_cast(row + offY), static_cast(cpp * offX + col)) = ++ pixels[(static_cast(row) * jxl_w * cpp) + col]; ++ } ++ } ++} ++ ++} // namespace rawspeed ++ ++#else ++ ++#pragma message \ ++ "JPEG XL is not present! DNG JPEG XL (DNG 1.7) compression will not be " \ ++ "supported!" ++ ++#endif +diff --git a/src/librawspeed/decompressors/JpegXlDecompressor.h b/src/librawspeed/decompressors/JpegXlDecompressor.h +new file mode 100644 +index 00000000..7cbcef08 +--- /dev/null ++++ b/src/librawspeed/decompressors/JpegXlDecompressor.h +@@ -0,0 +1,56 @@ ++/* ++ RawSpeed - RAW file decoder. ++ ++ Copyright (C) 2026 darktable developers ++ ++ This library is free software; you can redistribute it and/or ++ modify it under the terms of the GNU Lesser General Public ++ License as published by the Free Software Foundation; either ++ version 2 of the License, or (at your option) any later version. ++ ++ This library is distributed in the hope that it will be useful, ++ but WITHOUT ANY WARRANTY; without even the implied warranty of ++ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ++ Lesser General Public License for more details. ++ ++ You should have received a copy of the GNU Lesser General Public ++ License along with this library; if not, write to the Free Software ++ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ++*/ ++ ++#pragma once ++ ++#include "rawspeedconfig.h" ++ ++#ifdef HAVE_JPEGXL ++ ++#include "common/RawImage.h" ++#include "decompressors/AbstractDecompressor.h" ++#include "io/Buffer.h" ++#include ++#include ++ ++namespace rawspeed { ++ ++// Decodes a single DNG tile whose data is a self-contained JPEG XL codestream ++// (TIFF Compression tag 52546, as used by Apple ProRAW on iPhone 16 / DNG 1.7). ++class JpegXlDecompressor final : public AbstractDecompressor { ++ Buffer input; ++ RawImage mRaw; ++ ++public: ++ JpegXlDecompressor(Buffer bs, RawImage img) ++ : input(bs), mRaw(std::move(img)) {} ++ ++ void decode(uint32_t offsetX, uint32_t offsetY); ++}; ++ ++} // namespace rawspeed ++ ++#else ++ ++#pragma message \ ++ "JPEG XL is not present! DNG JPEG XL (DNG 1.7) compression will not be " \ ++ "supported!" ++ ++#endif diff --git a/tools/rawspeed_proraw/upstream_split/README.md b/tools/rawspeed_proraw/upstream_split/README.md new file mode 100644 index 000000000000..98dafe37e857 --- /dev/null +++ b/tools/rawspeed_proraw/upstream_split/README.md @@ -0,0 +1,88 @@ +# Splitting rawspeed#963 into reviewable pull requests + +darktable-org/rawspeed#963 bundles four unrelated concerns into one 9-file +diff, which turns one review into four. This directory splits it into five +series that apply independently, so each can be reviewed on its own merits. + +All five were rebased onto upstream `develop` +(`c835b05aecfacb7343f7c424abd620aa12116c3f`, "Merge pull request #979"), +cherry-picked cleanly with zero conflicts, and **each compiles and links on +its own**. Original authorship (Philipp Lutz) is preserved in every patch. + +## The five series + +| Dir | Content | Files | Commits | Depends on | +|---|---|---|---|---| +| `01_ljpeg_predictor_modes` | Predictor modes 2 to 7, the inverted tile reshape, and their follow-up cleanups | `LJpegDecoder.cpp`, `LJpegDecompressor.{cpp,h}`, fuzz harness | 7 | none | +| `02_dng_unique_camera_model` | Honour the `UniqueCameraModel` Exif tag for DNG | `DngDecoder.cpp` | 1 | none | +| `03_dng_12bit_jpeg_errmsg` | Clearer error for 12-bit lossy-JPEG DNGs | `AbstractDngDecompressor.cpp`, `JpegDecompressor.cpp` | 1 | none | +| `04_analyzer_hardening` | Static-analyzer / sanitizer / fuzzer fixes with no LJpeg content | `SimpleTiffDecoder.h`, `AbstractDngDecompressor.cpp`, `FileReader.cpp` | 1 | none | +| `05_jpegxl_dng17` | JPEG XL decompressor (DNG 1.7, compression 52546) | 8 files, mostly new | 1 | none | + +Series 02, 03 and 04 are each a single small commit and should be quick +reviews. Landing them first shrinks #963 to the part that actually needs +decoder expertise. + +Series 05 is not part of #963 at all. It is separate local work, listed here +because it sits on the same submodule pin. + +## The one non-obvious dependency + +Upstream commit `cf87137` ("Address clang-tidy, sanitizer and fuzzer +findings") looks like a self-contained cleanup, but it is not: one of its +hunks adds + +```c +if (predictorMode < 1 || predictorMode > 7) + ThrowRDE("Unsupported predictor mode: %i", predictorMode); +``` + +to the `LJpegDecompressor` constructor. `LJpegDecompressor` is `final` and +does not inherit `AbstractLJpegDecoder`, so it has no `predictorMode` member +on `develop`. That member is introduced by the predictor-mode work itself. + +The trap: `cf87137` cherry-picks onto `develop` **without conflict**, because +the surrounding context lines all exist. It then fails to compile: + +``` +LJpegDecompressor.cpp:118:28: error: 'predictorMode' was not declared in this scope +``` + +So a naive commit-by-commit split of #963 produces a series that looks clean +and does not build. This split therefore folds that single hunk into +`01_ljpeg_predictor_modes` (as `0007-LJpeg-validate-predictor-mode-range`) +and keeps the other three hunks, which touch no LJpeg code, in +`04_analyzer_hardening`. + +## Applying a series + +``` +git clone https://github.com/darktable-org/rawspeed.git +cd rawspeed +git checkout -b ljpeg_predictor_modes c835b05a +git am /path/to/01_ljpeg_predictor_modes/*.patch +``` + +Patch filenames are left as `git format-patch` generated them: `git am` +relies on the numeric prefix for ordering. + +## How this was verified + +Configured with CMake + Ninja, `Release`, testing/benchmarking/tools/fuzzers +off, and built to completion: + +| Series | Result | +|---|---| +| 01 | builds, links, 0 errors | +| 02 | builds, 0 errors | +| 03 | builds, 0 errors | +| 04 | builds, 0 errors | +| 05 | builds, 0 errors, with `-DWITH_JPEGXL=ON` | + +Series 05 was built against libjxl **0.7.0**, so it does not require the +0.11.2 that the submodule bump notes mention. + +Not covered: no raw sample corpus was decoded here, so this verifies +compilation and separability, not decode correctness. Decode correctness for +predictor modes 1 to 7 and for ProRAW was checked separately on real files +(iPhone 12 and 15 Pro Max) and reported in the #963 thread.