Stream all available blobs in DownloadBlobs and skip missing#6201
Merged
Conversation
b49cd8b to
19252f9
Compare
afck
approved these changes
May 4, 2026
| .flat_map(|certificate| certificate.value().block().created_blob_ids()) | ||
| .collect(); | ||
| .collect::<BTreeSet<BlobId>>(); | ||
| let required_blob_ids: Vec<_> = certificates |
Contributor
There was a problem hiding this comment.
(Here's another type annotation.)
19252f9 to
039a360
Compare
039a360 to
8648b60
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The streaming
DownloadBlobsRPC added in #5976 terminates the responsestream on the first missing blob, dropping any blobs that would have
been yielded after it:
linera-service/src/proxy/grpc.rs): the handler yieldsErr(Status::not_found(...))for a missing blob, which terminatesthe gRPC response stream. Subsequent blobs queued for that batch are
silently lost.
linera-service/src/proxy/main.rs): the handlerinterleaves
RpcMessage::Error(BlobsNotFound([single_id]))itemsamong
DownloadBlobResponseitems. The current consumer inRemoteNode::download_blobspropagates the first error via?,which drops the rest of the batch on the floor.
A batch of N blobs where blob K is missing returns at most K successful
blobs to the caller, even if the validator has all of blobs K+1..N
available. The follow-up #6157 wires this RPC into the per-peer
fallback path and relies on partial progress being reported correctly,
which the current broken server semantics prevent.
Proposal
Both server handlers now yield only the blobs that are present, in
request order, and silently skip missing blobs. The client compares
the IDs it received against the IDs it asked for and re-requests any
missing IDs from another peer.
This preserves the existing wire format on both transports.
Storage-level errors that occur before any blob has been yielded
continue to surface as a top-level error on the response (gRPC) or as
an early
RpcMessage::Error(simple transport).Also addresses the review comment from afck on #6105 / #6155: the
type annotation on
created_blob_idsinClient::process_certificatesmoves from theletbinding to the.collect()call (turbofish style).Test Plan
CI.
Release Plan
testnetbranchalongside Add streaming DownloadBlobs RPC endpoint #5976. [backport] Add streaming DownloadBlobs RPC endpoint (#5976) #6156 is the WIP backport of Add streaming DownloadBlobs RPC endpoint #5976 and should be
updated to include this fix before being merged.