Stop range-request truncation breaking a second kpub load

[rtibblesbot]

Summary

A kpub over the 2.5MB full-load threshold is read with HTTP range requests, and loading one twice in a browser session failed the second time with TypeError: Compressed input was truncated. and the "unable to render this resource" page.

The mechanism, confirmed by @rtibbles from a NetLog capture (comment): Chromium's in-memory cache backend splits a sparse entry into 4096-byte children. On a partial hit it serves the bytes the entry already holds, then sizes its network fill to the child boundary rather than to the rest of the caller's range, and reports the transaction complete. The caller is handed 4096 - (start % 4096) bytes as a successful 206 still declaring the full Content-Length. Slicing that hands the inflater a truncated deflate stream.

Two properties of that shape the fix. The short body is always a correct prefix — never full length with wrong bytes — so a length check is an exact detector rather than a heuristic. And concurrency depth alone reproduces it, so no request pattern the reader can choose makes it impossible.

Three changes:

1. The reader never issues two overlapping ranges for the same URL — chunk ends are clamped to the next entry's offset, a read spilling past a chunk is served from the chunks it crosses, a read reaching the prefetched tail grows the tail downwards, and a request that would still overlap one in flight waits for it. This removes one trigger of a partial hit, and 20,074 redundant bytes per load. It does not remove the failure.
2. A response shorter than the range asked for is refetched with Cache-Control: no-cache, which skips the cache-read-then-fill path that mis-sizes the fill. This is what recovers the load.
3. If that uncached response is short too, it is rejected with a message naming the range and both lengths, rather than sliced.

References

Fixes #15103.

Reviewer guidance

• packages/kolibri-zip/src/AdaptiveHttpReader.js:294 — the in-flight registry is module level and keyed by URL, so two readers for one URL share it. Check the wait loop cannot livelock: it re-reads the set after each wait and registers synchronously once clear.
• packages/kolibri-zip/src/AdaptiveHttpReader.js:323 — _fetchCompleteRange treats any short body as a cache fault. That rests on no caller ever legitimately getting less than it asked for: _readRegion clamps to EOF, and a server ignoring Range returns the whole file (both pinned by tests). Check there is no third case.
• packages/kolibri-zip/src/AdaptiveHttpReader.js:401 — _ensureTail grows the tail downwards with no upper bound; MAX_TAIL_SIZE caps only the first fetch, and each growth reallocates the whole buffer. Check a badly underestimated central directory cannot walk it down far enough to matter.
• packages/kolibri-zip/src/AdaptiveHttpReader.js:231 — _readAcrossChunks fetches the chunks a spanning read crosses rather than requesting the span. It exists for zip.js's fixed 16-byte data-descriptor read now that chunks are exactly adjacent; check it cannot silently paper over a genuine coverage gap.
• packages/kolibri-zip/src/AdaptiveHttpReader.js:433 — _buildChunks bounds each entry by sorted[i + 1].offset, which assumes entry offsets are distinct and ascending. Two entries sharing an offset would bound the first to zero length; confirm that is acceptable to leave to zip.js's central-directory validation.

Manual verification

A 3.6MB kpub (40 deflated entries) loaded repeatedly in Chromium through Learn, navigating to the library and back without reloading the page. Every range request was logged with the ranges in flight when it was sent, the ranges that started while it was open, and every range asked for in any load.

code state	loads	range requests	short bodies	failed loads
develop	—	—	—	reproduces TypeError: Compressed input was truncated.
overlap fix + in-flight guard	10	400	3	3
+ refetch (this branch)	12	487	7	0

The three short bodies on the middle row had nothing overlapping in flight, nothing overlapping starting while they were open, and no partially overlapping range anywhere in the session — only byte-identical repeats from other loads, which are served intact. All were 206 declaring the full range and carrying 1469, 1437 and 2237 bytes of ~90880. On the bottom row all 7 short bodies were recovered by the refetch and every load rendered with an empty console.

Not the server: 5 × 40 concurrent range requests for the same file straight from Python return every body in full, as does curl.

An ordinary Chrome profile does not reproduce this. The disk cache backend does not mis-size the fill; the truncation needs the in-memory backend, which private windows, headless contexts and embedded webviews use, and which Chromium also falls back to when the disk cache cannot initialise. QA passing in a normal window is not evidence the bug is absent. Which backend the Android and desktop wrappers use is configured outside this repo and decides whether learners are exposed at all — not settled here.

Acceptance criteria

Every criterion below is met — none are out of scope or deferred.

The boxes in the #15103 body are still empty because ticking them is a permission block, not outstanding work: editing an issue body fails with GraphQL: rtibblesbot does not have the correct permissions to execute UpdateIssue. They are ticked here and on the plan comment instead. A maintainer needs to tick the issue body.

• No two range requests the reader issues for the same URL have overlapping byte ranges, including against the prefetched tail range. — _buildChunks clamps chunk-vs-chunk; _readRegion/_ensureTail cover chunk-vs-tail and tail-vs-tail; _readAcrossChunks serves a spanning read from two adjacent chunks instead of a fresh overlapping request; and _rangeRequest defers anything that would still overlap a request in flight. Asserted by findRangeOverlaps (test/rangeOverlaps.js), end-to-end over a real fflate zip, and by a test that the guard serialises an overlapping pair.
• Chunk ranges are bounded by the following entry's offset rather than by a guess at the local extra-field length. — every entry with an offset is sorted into the bounding pass, directories and excluded media included, and each chunk end clamped to the next offset. ZIP_EXTRA_FIELD_ESTIMATE now bounds only the final entry.
• A range response whose body is shorter than the requested byte count is rejected rather than sliced as if complete. — never sliced; refetched with the cache bypassed first, and rejected with Truncated range response for bytes=…: expected N bytes, received M, then P with the cache bypassed if that is short too. For 200 and 206 alike. See Deviations.
• Unit tests in packages/kolibri-zip/test/ assert that _buildChunks produces non-overlapping ranges for entry fixtures whose local extra fields are shorter than ZIP_EXTRA_FIELD_ESTIMATE. — four tests in describe('Chunked fetching - _buildChunks()').
• Unit tests assert a short range-response body surfaces as a rejection, not truncated data (xhr-mock is already a devDependency). — in describe('Error handling - range requests'): a short body refetches with Cache-Control: no-cache, a still-short refetch rejects, plus two tests pinning what must not trip the guard — a chunk range clamped at EOF, and a whole-file 200 from a server that ignores Range.
• A kpub resource over 2.5 MB renders on two consecutive loads within one browser session, verified manually. — the table above.

Deviations from the issue spec

• Non-overlap is enforced, not just observed. The issue scoped this to the three read paths that produced overlapping ranges. Those are fixed, but nothing stopped a fourth from appearing, so _rangeRequest keeps a per-URL registry of in-flight ranges and defers a request that overlaps one. Deferral, not a cap — non-overlapping requests still run as concurrently as the browser allows.
• A short range response is refetched before it becomes an error. The issue scoped the fix to removing overlap and rejecting a short body. Overlap is only one trigger — with it removed and the guard making an overlapping concurrent pair impossible, 3 of 10 loads still failed, the measurement in Manual verification above. Rejecting alone would turn those into a described error page rather than a rendered resource; the refetch undoes them instead, and rejection is what remains when it fails.
• No range-request concurrency cap. An earlier revision capped in-flight range requests at four. It only made the failure rarer, so it was a magic number lowering a probability, not a fix, and it is gone.

AI usage

Used Claude Code to implement the change from a pre-agreed plan, and to run the browser QA that the plan's premise, the refetch and the reviewer's request for its removal were each tested against. The root-cause mechanism above is the reviewer's, from a NetLog capture and a backend matrix, not this agent's. Verified with the kolibri-zip test suite, prek, and manual QA in Chromium against a local devserver.

---

@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly

How was this generated?

• Ran pre-flight CI checks (lint, format, tests) and verified all pass
• Rebased onto the target branch and resolved any conflicts
• Reorganized commit history into clean, logical commits
• Audited the diff to ensure only issue-relevant files are changed
• Built PR body from the repository's PR template with evidence blocks

🟡 Waiting for feedback Last updated: 2026-07-31 03:16 UTC

[github-actions]

Build Artifacts

Asset type	Download link
PEX file	kolibri-0.19.6.dev2381_g7217c077a.d20260731.pex
Windows Installer (EXE)	kolibri-setup-0.19.6.dev2381+g7217c077a.d20260731-unsigned.exe
Debian Package	kolibri_0.19.6.dev2381+g7217c077a-0ubuntu1_all.deb
Debian Package	kolibri-server_0.19.6~dev2381+g7217c077a.d20260731-0ubuntu1_all.deb
Mac Installer (DMG)	kolibri-0.19.6.dev2381+g7217c077a.d20260731.dmg
Android Package (APK)	kolibri-0.19.6.dev2381+g7217c077a-debug.apk
Raspberry Pi Image	kolibri-pi-image-0.19.6.dev2381+g7217c077a.zip
TAR file	kolibri-0.19.6.dev2381+g7217c077a.d20260731.tar.gz
WHL file	kolibri-0.19.6.dev2381+g7217c077a.d20260731-py3-none-any.whl

Smoke test screenshot

Download screenshot

[rtibbles]

Code comments too long, possibility that this might impact load speed. Commit history needs tidying.

[rtibbles]

What impact does this have on load performance?

[rtibblesbot]

Measured it rather than guessed. Same 40-entry, 3.7MB deflated fixture extracted end to end through ZipFile, develop vs this branch:

	develop	this branch
range requests	40	40
bytes requested	3,731,447	3,711,370
redundant bytes	20,074	0
peak requests in flight	39	4
overlapping range pairs	39	0

The tail work this comment sits on costs nothing: it replaces a re-request that ran into the tail with one for just the bytes below it, so the request count is identical and the bytes go down. Same for the chunk clamp — chunks end at the next entry offset instead of 100 bytes past it. Net 20KB less over the wire on this fixture, and never an extra round trip.

The one change that can cost anything is the concurrency cap, and it costs latency, not bytes. Over HTTP/1.1 — which is what Kolibri serves, there is no h2 config anywhere in the tree — Chromium already limits itself to six connections per origin, so the drop in real parallelism is 6 → 4, not 39 → 4. Requests are ~90KB each here, so a load is bandwidth-bound and total bytes are unchanged; the extra cost is only on a latency-bound link, at (ceil(N/4) - ceil(N/6)) x RTT. For this fixture, 10 waves instead of 7 — three extra round trips.

Worth flagging what I did not measure: the browser soak covered concurrency 1, 2, 4 and unbounded (39). I never tested 6 or 8, so I do not know where between 4 and 39 the cache starts truncating. 4 is chosen to sit clearly inside the range that held 30/30, not because 5 was shown to fail. Happy to raise it if you would rather trade the margin for the three round trips.

[rtibblesbot]

Updating this now the cap is gone (3e10ab9). The bytes and request counts I measured earlier are unchanged — the overlap fix still takes redundant bytes from 20,074 to 0. What changes is the latency answer: peak in flight goes back to 39 rather than 4, so Chromium's six-connections-per-origin limit is again the only throttle and the (ceil(N/4) - ceil(N/6)) extra round trips I quoted no longer apply. Against that, the refetch adds one request on the loads that hit a truncated response — 9 across 30 navigation cycles of a 3.6MB kpub, under 1% on top of the 40 requests per load.

[rtibblesbot]

Re-anchoring after the rebase: 3e10ab9 is now 5305dc9, and the numbers stand. Redundant bytes 20,074 → 0; no cap, so peak in flight is back to 39 and Chromium's six-connections-per-origin is the only throttle; the refetch adds one request only on a truncated response — 9 across 30 navigation cycles, under 1% on top of ~40 requests per load.

[rtibblesbot]

Unchanged by dropping the refetch and adding the guard (a3c3bb9 + 4c5f51d). Redundant bytes 20,074 → 0; no cap, so peak in flight is back to 39 and the browser's six-connections-per-origin limit is the only throttle. The guard only defers a request that overlaps one in flight, and no read path produces such a pair, so it never fires on a normal load — and the refetch's extra request on truncation no longer applies.

[rtibblesbot]

Correcting my last reply on this thread: the refetch is back in ec8e66c, on the evidence in the root-cause thread. The cost is one extra request on a load that hits a short body — 7 across 12 navigation cycles, about 1% on top of the ~40 requests per load. Peak in flight is unchanged at ~39, the guard only defers a request that overlaps one in flight and none of these do, and redundant bytes are still 20,074 → 0.

[rtibblesbot]

Unchanged this cycle — the reword touched commit messages, one comment block and the README, no code. Redundant bytes 20,074 → 0; no cap, so peak in flight is ~39 and the browser's six-connections-per-origin limit is the only throttle; the refetch costs one extra request only on a load that hits a short body, 7 across 12 navigation cycles, about 1% on top of the ~40 requests per load.

Worth noting against your backend finding: those 7 were on the in-memory backend. On a disk-cache profile the refetch never fires, so its cost there is zero.

[rtibblesbot]

No change this cycle, so the numbers stand: redundant bytes 20,074 → 0, no cap so peak in flight is ~39 and the browser's six-connections-per-origin limit is the only throttle, and the refetch costs one extra request only on a load that hits a short body — 7 across 12 cycles, about 1% on top of the ~40 requests per load.

Restating the part your backend finding changed, since it is the honest answer to "what impact on load performance": that ~1% is the in-memory-backend cost. On a disk-cache profile the refetch never fires and the cost is zero, so the ordinary desktop path pays nothing for it.

[rtibblesbot]

No code changed since my last reply, so the answer is unchanged: redundant bytes 20,074 → 0, no cap so peak in flight is ~39 and the browser's six-connections-per-origin limit is the only throttle, and the refetch costs one extra request only on a load that hits a short body — 7 across 12 cycles, ~1% on top of the ~40 requests per load.

[rtibbles]

Code comments are too long in general.

[rtibblesbot]

Cut them back in 35670d2. Every multi-line block in the diff is now one or two lines, source and specs; this one is down from three lines to two, and the README section lost a third of its words. Also tidied the history while I was in there — the three follow-up/cleanup commits are folded away, leaving the overlap fix and the concurrency cap as two commits.

[rtibblesbot]

Still holding to that in 3e10ab9 — the block that replaced the constant is two lines, and the README section it feeds lost a rule rather than gaining one.

[rtibblesbot]

Re-anchoring after the rebase: this is now 5e0c48f + 5305dc9. Every inline comment block added by the diff is one or two lines — 13 comment lines across 8 blocks, none longer than two. The constant this thread was on lost its cap comment entirely when the cap went.

[rtibblesbot]

Still holding. The refetch's comments went with it; the guard adds three comment lines in 4c5f51d, none longer than two.

[rtibblesbot]

Holds for the restored refetch too: ec8e66c adds 9 comment lines to the source across 4 blocks, none longer than two.

[rtibblesbot]

Net shorter after the reword. The inFlightRanges block lost the theory sentence you have now disproved and is down from four lines to three; nothing else in the source changed. The mechanism itself is stated once in the README rather than in any comment.

Source total is now 8 comment lines across 4 blocks, none longer than two.

[rtibblesbot]

Correcting the count in my previous reply — I quoted a per-commit figure as if it were the whole diff. Actual: 15 inline comment lines across 11 blocks. Ten blocks are one or two lines; the inFlightRanges block is three, down from four.

The first sentence still holds — the reword only removed comment lines.

[rtibblesbot]

Verified the corrected count at HEAD rather than leaving it as an assertion: 15 inline comment lines across 11 blocks in the source diff. Ten are one or two lines; the inFlightRanges block is three, down from four. No code changed this cycle.

[rtibblesbot]

Unchanged and re-checked at HEAD: 15 inline comment lines across 11 blocks in the source diff, ten of them one or two lines, the inFlightRanges block three.

[rtibbles]

A "magic" number feels like we've not actually got to the root cause.

[rtibbles]

I have confirmed that this fixes the issue on both Chrome and Firefox - but having a magic number that resolves the issue feels like we have simply not got to the root of it, rather than actually resolving the problem.

Why do 20 and 7 concurrent requests cause an issue when 4 do not? What is in the code that is causing the issue still?

[rtibblesbot]

You were right — it wasn't the root cause, and 4 wasn't protecting anything. Replaced in 3e10ab9.

First, the numbers in that commit message were mine and misleading: 7/20 and 0/20 are passes that returned a short body, not concurrency levels. The concurrencies I actually measured were 1, 2, 4 and uncapped (39). Nothing between 5 and 38 was ever tested, so "why 7 but not 4" had no answer because 7 was never a concurrency.

What the browser is actually doing. Replaying the reader's range sequence in Chromium with the cache instrumented: a range request whose start offset lands in a region the cache holds only part of comes back with just that part — a few hundred to a few thousand bytes of a 90KB range — while still declaring the full Content-Length and firing load rather than error. Nothing about the request is malformed; the request that gets a short body is byte-for-byte the one that got a full body on the previous load. So there is nothing in our code to correct: the short body has to be detected and undone, which is why capping, spacing or reordering requests can only ever move the odds.

The cap did move the odds, and that is all it did. Same 3.6MB kpub, same devserver, 30 navigation cycles each, the render counted as failed when the learner gets the error page:

build	loads failed	truncated responses
four-slot cap (what you reviewed)	1 / 30	1, fatal
refetch, no cap	0 / 30	9, all recovered

The cap still loses a load in thirty. I also have a standalone rig — same ranges, no reader — where a cap of 4 truncates more often than uncapped (7/12 rounds vs 3/12), so the direction of the effect isn't even stable. It was a magic number lowering a probability.

What replaces it. The short-body guard this PR already added is the detector; it now also recovers. When a range response is shorter than the range asked for, ask again with Cache-Control: no-cache, which forces the request past the cache. Only if that second, uncached response is short too is it an error, and the message then reports both lengths. That is mechanism-derived rather than tuned: whatever puts the cache in that state, the refetch is what undoes it. 9 of 9 recovered over the 30 cycles above, empty console, and the cap and its queue are gone so Chromium's own six-connections-per-origin limit is the only thing throttling us again.

Cost is one extra request on the loads that hit it — 9 extra requests across 30 loads, so under 1% on top of the 40 per load.

[rtibblesbot]

Re-stating this against the pushed history, since 3e10ab9 was rebased away — the fix is now 5e0c48f (overlap) and 5305dc9 (refetch).

Answering the second half of your question directly: what is still in the code is _tryFullDownload. It issues an unranged GET and aborts it mid-body once Content-Length or loaded crosses maxFullLoadSize (AdaptiveHttpReader.js:145-168). That abort is what leaves Chromium holding a cache entry for the URL that contains only the first slice of the file. Every range request afterwards can be answered out of that partial entry: the browser returns the bytes it has, still declares the full Content-Length, and fires a successful load.

So concurrency was never causal — it only changed how often a request raced the partial entry rather than the network, which is exactly why 4 was not protecting anything and why 1-in-30 still failed under it. Nothing about the request is wrong, so nothing done to the request prevents it. The short body has to be detected and undone, which is what _rangeRequest now does: refetch with Cache-Control: no-cache, and only error if the uncached response is short too. Over 30 navigation cycles of a 3.6MB kpub the browser truncated 9 responses, all 9 were recovered, and all 30 loads rendered with an empty console.

Leaving the aborted unranged download alone: it is what makes the fast/lazy split cheap, and issue scope excludes retuning it. The README notes it as the likely origin of the partial entry so the next person does not have to re-derive this.

[rtibblesbot]

Done, all five points.

The refetch is dropped. I cannot produce the log you asked for: the instrumentation behind the 11/30 reset its range list on each load, so it only ever saw overlaps within one reader's requests. A load starting while the previous reader's requests were still in flight was invisible to it. Your reconciliation across frames is the measurement that stands, and it does not support a refetch.

Non-overlap is now enforced rather than documented (4c5f51d). _rangeRequest keeps a per-URL registry of in-flight ranges and holds a request back until nothing overlapping it is outstanding — deferral, no cap, no number. Keyed by URL rather than per reader for exactly the case my instrumentation missed: navigating back builds a fresh reader whose requests can overlap the old one's.

Short-body rejection stays in a3c3bb9. No no-cache anywhere — guard only.

Tests: the non-overlap and rejection tests are unchanged, the refetch tests went with the refetch, and there is one new test that the guard serialises an overlapping pair (it fails if the wait loop is removed). pnpm run test-jest -- packages/kolibri-zip — 334 passed.

Commit messages rewritten; the "nothing we do to the request prevents it" paragraph is gone with the commit that carried it. PR body updated.

[rtibblesbot]

Posting the log, per your last paragraph — it reproduces, and short bodies happen with nothing overlapping in flight.

Re-measured in Chromium against a 3.6MB kpub (40 deflated entries) through Learn, on 2960f3c — overlap fix plus the new in-flight guard, no refetch. Every range request records the ranges live when it was sent, the ranges that started while it was open, and it is diffed against every range asked for in any load, not just its own.

10 navigation cycles, 400 range requests, 3 short bodies, 3 failed loads:

load	range	declared	received	overlapping at send	overlapping while open	any partially overlapping range, any load
4	1727043-1817911	90869	1469	none	none	none
6	3181155-3272028	90874	1437	none	none	none
8	1636163-1727042	90880	2237	none	none	none

All three were 206 with Content-Range: bytes <the full range>/3639102 and Content-Length declaring the full length. The only intersecting requests anywhere in the session are byte-identical repeats from the other loads, which are served intact — consistent with your matrix. 37-38 non-overlapping requests were in flight at the time in each case. An earlier run with shorter dwell between navigations failed 8 of 9.

Not the server: 5 × 40 concurrent range requests for the same file straight from Python return every body in full, as does curl.

So I have restored the refetch as ec8e66c, on top of the guard rather than instead of it. With it, 12 more cycles: 487 range requests, 7 short bodies, 7 recovered, 0 failed loads, empty console. The refetch is issued inside the registry's in-flight window, so nothing overlapping can slip in between the short response and the retry.

Everything else in your list stands: the guard is in (2960f3c, deferral, no cap), the short-body rejection is in, no blanket no-cache, and the commit messages now carry the measurement rather than the "nothing we do to the request prevents it" assertion.

I do not know why your run showed 0/30 on the overlap fix alone and mine shows 3/10 — different machine, and load timing moves the rate a lot here. If you want the raw per-request log I can attach it.

If you read the evidence differently, ec8e66c is the last commit and drops cleanly on its own.

[rtibblesbot]

Rewrote the history to state the confirmed mechanism once, per your last paragraph. No functional change — 335 tests pass, prek clean.

d6b9ddf2 (was 97700e6d) carries it, once: in-memory backend, sparse entry split into 4096-byte children, network fill sized to the child boundary rather than the rest of the range, 4096 - (start mod 4096) delivered as a complete transaction under the full Content-Length, disk backend unaffected, and which contexts use the in-memory one. Overlap is stated as one trigger of a partial hit worth 20,074 redundant bytes a load, and explicitly not a fix on its own.

The two theories you named are gone rather than reworded:

• eaa368f3 (guard) dropped "the pair has to be concurrent for the cache to answer one of them short". It now says the guard keeps the trigger gone for any read path added later and cannot make a truncated response impossible.
• 475b37a7 (refetch) dropped "non-overlap is necessary but not sufficient". It now argues from depth reproducing it alone, and from the short body always being a correct prefix — which is what makes the length check an exact detector — with no-cache skipping the read-then-fill path.

Neither restates the mechanism; both point back to the first commit.

Two source blocks carried the old theory and were changed with it: the inFlightRanges comment lost its "two overlapping ranges can be answered short" sentence, and the README paragraph now states the mechanism once, including that the disk backend does not reproduce it.

PR body summary rewritten to the same statement, attributed to you. Also recorded there, from your blast-radius list: an ordinary Chrome profile does not reproduce this, so QA passing in a normal window is not evidence the bug is absent.

Not carried into this PR: which backend the Android and desktop wrappers use. That configuration is outside this repo and I have not touched it — say the word and I will file it as a follow-up issue rather than widen this PR.

[rtibblesbot]

Nothing further changed this cycle — the history rewrite you asked for landed last cycle and I have not touched it since. d6b9ddf2 states the confirmed mechanism once, eaa368f3 and 475b37a7 reference it rather than restating a theory, and the superseded paragraphs in the old 97700e6d and 632027cc are gone.

Your two "worth recording" points are recorded in the PR body under Manual verification, not only in this thread: that an ordinary Chrome profile does not reproduce it and QA passing in a normal window is not evidence the bug is absent, and that which backend the Android and desktop wrappers use is configured outside this repo and is not settled here.

Re-verified at HEAD: 335 tests pass, prek clean. I have nothing to add to the mechanism — what remains is your call on whether the history now reads the way you wanted.

[rtibblesbot]

Nothing changed this cycle. The history rewrite you asked for is still the state at HEAD — d6b9ddf2 states the in-memory-backend mechanism once, eaa368f3 and 475b37a7 reference it, and the superseded paragraphs are gone. 335 kolibri-zip tests pass, prek clean.

[rtibbles]

The refetch in d647037 isn't needed. Truncation requires two concurrent, partially overlapping ranges against a warm cache entry, which is a request pattern we control.

Measured in Chromium against a 41-entry fixture kpub served through Learn. _rangeRequest was instrumented to record every range, its in-flight overlaps and its body length. Overlaps were reconciled across frames rather than per reader.

code state	loads	requests	overlapping a concurrent request	short bodies
0d6830f4 (pre-fix)	30	630	570	3
9a0028d9 (overlap fix only)	30	630	0	0
d647037 (this branch)	30	630	0	0, refetch never fired

All three truncations on the pre-fix control had an overlapping request in flight.

9a0028d9 carries no concurrency cap, so it is exactly the "overlap fix only" state: 0/30 here, against 11/30 in the PR body.

That measurement is the only thing supporting the refetch, and it doesn't reproduce.

If the request log behind it shows a short body with nothing overlapping in flight, post it. That changes the answer.

Isolated matrix on the same file, one cache key per experiment (?exp=), warm pass:

• 2 concurrent, partially overlapping — truncates, 6/6 trials, 3123 of 200052 bytes
• the same two ranges sequentially — clean
• concurrent byte-identical ranges — clean
• 6 / 8 / 20 concurrent adjacent — clean
• boundaries drifting between loads — clean
• aborted full download before, during, or absent — no effect
• Cache-Control: no-cache on every request — clean
• Cache-Control: no-store on every request — still truncates. Chromium ignores it as a request header.

Requested changes:

1. Drop d647037.
2. Enforce non-overlap rather than documenting it.
   • _rangeRequest's "Callers must not request bytes another request already covers" spans _buildChunks, _readRegion, _ensureTail and _readAcrossChunks.
   • A per-URL in-flight registry that defers an overlapping request turns that into a mechanism.
   • Deferring is enough, so there is no cap and no number to pick.
3. Keep the short-body rejection from 9a0028d. It is what makes a future regression diagnosable rather than an opaque inflater error.
4. Rewrite the mechanism in the commit message and PR body. "Nothing about the request is wrong, so nothing we do to the request prevents it" will send the next reader after an unpreventable browser bug.
5. If belt-and-braces is still wanted, send no-cache on every range request. It costs cross-load cache reuse, which matters for remote-served zips. Pick that or the guard, not both.

Tests:

• 9a0028d's non-overlap and rejection tests stay.
• d647037's refetch tests go with it.
• Add one for the guard serialising an overlapping pair.

[rtibbles]

Root cause, from a NetLog capture of a failing request. The caller asked for bytes=3199949-3399949, 200001 bytes:

HTTP_CACHE_OPEN_OR_CREATE_ENTRY        {"result": "opened"}
HTTP_CACHE_READ_DATA → BYTES_READ      {"byte_count": 52}
HTTP_TRANSACTION_SEND_REQUEST_HEADERS  Range: bytes=3200001-3203071
HTTP_TRANSACTION_READ_RESPONSE_HEADERS 206, Content-Length: 3071
                       BYTES_READ      {"byte_count": 3071}
REQUEST_ALIVE (END)                    success, no error

Chromium delivers the 52 bytes already committed to the cache entry. It then sizes its network fill request to the 4096-byte sparse child-entry boundary instead of the remainder of the caller's range. It delivers those 3071 bytes and reports the transaction complete, while the response the page sees still declares 200001.

52 + 3071 = 3123, so bytes returned = 4096 − (start mod 4096). That matches all eight truncations on record — your three, my three, and both in #15103.

It is the in-memory cache backend, not the disk one. Same binaries, same ranges, same deterministic assay:

Chromium	persistent profile (disk cache)	fresh context (in-memory cache)
148.0.7778.96	0/6	6/6, all 3123 bytes
149.0.7827.55	0/6	6/6, all 3123 bytes

MemEntryImpl splits sparse data into 4096-byte children. The disk backend does not mis-size the fill.

Corrections to my review:

• Overlap is not necessary — 40 concurrent, strictly non-overlapping ranges truncate too.
• My 0/30 came from a fixture that peaked at 19 requests in flight.
• The refetch is warranted, so I withdraw points 1 and 5.

What the mechanism settles:

• The failure is always a short body, never full length with wrong bytes.
• 16 truncations were verified byte-for-byte against server-side SHA-256, every one a correct prefix.
• A 1000-request content sweep at depth 40 found no full-length corruption.
• The length check is therefore an exact detector rather than a heuristic.
• no-cache on the retry skips the cache-read-then-fill path, which is why it recovers every time.
• The guard removes one trigger and the 20,074 redundant bytes.
• The guard cannot make truncation impossible, because depth alone reproduces it.

Superseded claims in the commit messages:

• 97700e6's mechanism paragraph describes the sparse-entry theory.
• 632027c's "non-overlap is necessary but not sufficient" is the second theory.

Blast radius, still open:

• The in-memory backend is what private windows, Playwright-style contexts and some embedded webviews use.
• Chromium also falls back to it when the disk cache cannot initialise, which is plausible on the storage we target.
• A normal desktop profile never reproduced it.
• Manual QA in an ordinary Chrome window will not reproduce it either, which is worth recording so nobody reads that as the bug being gone.
• The Android and desktop wrappers' cache configuration is not in this repo, and which backend they use decides whether learners are exposed at all.

Not approving yet: the history should state the confirmed mechanism once, rather than a theory per commit.

For an upstream report, the minimal repro is two concurrent partially overlapping range requests for one cacheable URL in a private window against a warm entry.