Fc: stream JSON responses directly, bypassing the variant tree

[heifner]

Emits HTTP JSON responses token-by-token into the output buffer through fc::json_writer instead of building an intermediate fc::variant tree and re-serializing it with fc::json::to_string.

Core pieces

• fc::json_writer (fc/io/json_stream.hpp): append-only token writer with object/array frame tracking, checkpoint()/rewind() for atomic rollback of partial emission, locale-independent numeric emission via std::to_chars, and the same 64-bit integer quoting threshold fc::json::to_string uses.
• to_json_stream overload family: primitives and standard containers are co-located with their to_variant siblings (variant.hpp, container/flat.hpp) so shape and MAX_NUM_ARRAY_ELEMENTS guards stay in lock-step; a reflector visitor gives every FC_REFLECT'd struct a streaming serializer for free; a depth-bounded streaming walker covers fc::variant / variant_object values (json_stream_max_depth matches json.cpp's recursion cap).
• FC_SERIALIZE_AS_STRING trait: string-shaped types (name, asset, symbol, symbol_code, block_timestamp, the sha/ripemd160/keccak256/blake3 hashes, BLS keys/signatures, bitset, url) declare one to_string()/from_string() pair that drives to_variant, from_variant, and to_json_stream, replacing the per-type hand-rolled overload triplets. The hash from_string factories reject odd-length hex, keeping the strictness the previous vector<char> from_variant path enforced. Dead fc::fixed_string is removed. to_json_stream(bls::private_key) is deleted so key material cannot be streamed into a response.
• abi_serializer::binary_to_json_stream: a single _binary_walk<Sink> traversal drives both the variant_sink (the existing binary_to_variant) and the new stream_sink (tokens straight into the writer), so shape parity between the two paths is structural rather than by convention. Action-data decode is atomic: unpack_action_data_field checkpoints, rewinds on failure, and the caller emits the hex fallback under the same key.
• http_plugin streaming callback path + bind_stream<>: registration template with compile-time signature pinning; response serialization happens on the http thread pool while the api thread hands off the result struct by move. chain_api, producer_api, net_api, db_size_api, and trace_api get_block/get_transaction_trace are migrated; endpoint sets, queues, categories, and response codes are unchanged. Stored exceptions rethrow with their dynamic type so the status classifier keeps mapping tx_duplicate to 409, unknown_block to 400, etc., and queued streaming request bodies take the same bytes_in_flight reservation as the variant path.
• Deferred ABI serialization: get_block and trx_retry capture raw ABI blobs at block time (captured_abis) and build serializers at emit time on the http pool, keeping ABI decode off the main thread; memory for captured blobs is accounted in the trx_retry budget.

Performance

RelWithDebInfo, i9-12900K, benchmark -f trace_api_json / -f abi_serializer:

• trace_api get_block: -26% wall-clock at the 25k-transaction block shape and -44% on data-heavy multi-action blocks, versus the variant + json::to_string pipeline.
• abi_serializer JSON production: -25% to -36% across transfer/newaccount/regproducer/block shapes via binary_to_json_stream versus binary_to_variant + to_string.
• get_table_rows decodes rows straight from binary into the writer with no per-row variant tree.

Parity

Streamed output is pinned byte-identical to the variant path: every abi_tests round-trip case also runs verify_stream_matches_variant, endpoint-level byte-identical assertions cover get_account/get_block/get_table_rows, and trace_api has stream-vs-variant response parity cases including the decode-failure hex-fallback shapes.

[huangminghuang]

Inline review findings from the streaming JSON parity and resource-accounting review.

[huangminghuang]

[P1] Enforce the memory budget during emission

The response is fully materialized by emitter(w) before body.size() is charged to bytes_in_flight. Concurrent large streaming requests are therefore invisible to --http-max-bytes-in-flight-mb while allocating and serializing, and are rejected only after the memory cost has occurred. Please account growth incrementally or enforce a budget-aware writer limit before allowing the buffer to expand.

[heifner]

Now charged incrementally: json_writer grew an optional growth guard invoked at token boundaries every 64KiB of emission; the handler settles the buffer into bytes_in_flight at each checkpoint and aborts with the busy response once verify_max_bytes_in_flight rejects, so over-budget emissions stop within a stride of the cap instead of materializing. The abort re-arms across catch-and-continue serializers (abi hex-fallback) so it cannot be absorbed. Covered by stream_response_bytes_in_flight (mid-emission budget visibility) and stream_response_over_budget_aborts_early (503 + early abort + drain), both verified failing against the previous code.

[huangminghuang]

[P1] Preserve absent-policy null fields

Changing these fields from fc::variant to std::optional changes the endpoint schema: the streaming reflector omits disengaged optionals, whereas the previous default variants emitted active_finalizer_policy and pending_finalizer_policy as explicit null values. A missing pending policy is normal, so clients that access the key before testing for null can now fail. Please preserve explicit nulls with a custom emitter or equivalent nullable representation and add an absent-policy parity test.

[heifner]

Kept the typed fields via a new fc::nullable whose disengaged state serializes as explicit null under its key on both the variant and streaming paths (trio co-located with the std::optional overloads). Absent-policy parity test added: get_finalizer_info on a fresh chain asserts byte-identical variant/stream output containing pending_finalizer_policy:null.

[huangminghuang]

[P2] Preserve large-duration quoting

value_int64 bypasses the established 64-bit JSON quoting rule. For example, microseconds{0x100000000} now emits 4294967296, while the previous to_variant plus json::to_string path emits "4294967296". Please delegate to the guarded int64_t streaming overload and add positive/negative boundary tests around 0xffffffff.

[heifner]

Delegated to the guarded int64_t overload; boundary tests added on both sides of +-0xffffffff plus byte-parity against the variant path, verified failing against the previous code. Also routed stream_sink::value_int64/uint64 through the guarded overloads -- ABI built-in ints already streamed through the guarded generic in get_built_in_stream_unpacks, so no live divergence existed there, but the sink interface itself now cannot diverge for direct callers.

[huangminghuang]

[P1] Enforce the response budget within a single JSON token

guard_check() only runs from value_prefix()/key(), before this token is appended. A single large reflected string (for example an action console), value_hex, or raw_value can therefore append its entire contents after one pre-check, and the HTTP handler only rejects it in the final settle_to after the full buffer has already been allocated. The new over-budget test emits many separate 64 KiB tokens, so it does not exercise this case. Please invoke the guard while the escape/hex/raw loops cross each stride (including escape expansion), and add a regression where one token alone exceeds the response budget.

[heifner]

Closed: guarded writers now check within tokens. value_string/key route the escape loop's periodic yield (every 128 input chars) into the same stride-gated guard check, measuring the output buffer so escape expansion counts; value_hex and raw_value emit in 64KiB chunks with a guard check between chunks (unguarded writers keep the single tight loop and the empty yield they already passed). A guard throw mid-token is safe under the existing re-arm plus checkpoint/rewind contract. Tests: growth_guard_intra_token_abort (each of the three token kinds aborts within one chunk of the threshold on a 1-2MiB token), growth_guard_measures_escape_expansion (32KiB of control chars crossing a 100KiB output threshold the raw input never reaches), and stream_response_single_token_over_budget (single 32MiB string value against a 1MiB budget: 503 with the emitter's post-token statement never reached), all verified failing against the token-boundary-only code.

[huangminghuang]

Four inline findings from the current-head review.

[huangminghuang]

[P1] Avoid reserving the full guarded string

The guarded writer calls escape_string, which immediately reserves input.size() + 13 bytes before its periodic yield can enforce the growth budget. A single 32 MiB string therefore allocates roughly 32 MiB even under a 1 MiB cap. The guarded path should reserve incrementally, with a regression test checking capacity/allocation rather than only out.size().

[heifner]

Fixed by checking before the reserve rather than reserving incrementally: value_string/key know the token's input length -- a guaranteed lower bound on its peak emission, since escaping never shrinks a byte pre-prune -- so guarded writers now consult the guard with the prospective size before escape_string runs. An over-budget string aborts with zero allocation and zero buffer mutation (capacity stays at the writer's initial slack), while an approved string keeps the single full-size reserve, so the legitimate path pays no geometric-growth tax and unguarded/legacy callers are untouched. Escape expansion beyond the approved size remains covered by the intra-loop checks from the previous round, and the http guard's signed settle_to reconciliation handles the prospective charge -- reserve-then-allocate, trued up at the next checkpoint. Regression: growth_guard_bounds_allocation drives a 32 MiB string and key at a 64 KiB threshold asserting out.capacity() stays below the threshold (was ~32 MiB), plus approved-path and unguarded controls pinning the full reserve, verified failing against the previous code.

[huangminghuang]

[P1] Charge queued emitter payloads

The typed result is captured by the emitter and queued on the HTTP thread without reserving any bytes-in-flight; accounting starts only when emission executes. Multiple large results can consequently accumulate outside http-max-bytes-in-flight-mb, unlike the variant path which charges before dispatch. Carry a source-size reservation through the queue and release it as the captured result is consumed.

[heifner]

bind_stream now reserves the typed result's packed-size estimate at the moment of capture -- dispatch::sync and the async typed-result arm, the two shapes with a real cross-thread queue window (post/post_direct produce on the http pool and the dispatch short-circuits inline). The RAII reservation (http_plugin::reserve_bytes_in_flight) rides inside the emitter closure and releases exactly when the captured result is destroyed after emission, so the source stays charged alongside the incrementally-charged output buffer -- both genuinely coexist, matching the variant path holding its in_flight_sizeof estimate while the response string builds. The dispatched handler also re-checks the budget before running the emitter, draining over-budget queued work with the busy response like the variant path's pre-serialization verify. Trace-bearing results report their retained memory (action data, console, return values, captured ABI bytes) through a queued_payload_size customization point, since streamed_processed_trace is emitted but never wire-packed; fc::nullable gained raw pack/unpack parity with std::optional so reflected results containing it can be sized. Regression: stream_queued_payload_reserved parks the single pool thread in a latched emitter and asserts the sampled bytes_in_flight covers a 1 MiB queued bind_stream payload while it waits behind the stall -- reads 0 without the reservation, verified -- then drains to zero.

[huangminghuang]

[P1] Use the guarded writer for trace responses

This builds the entire trace response in an unguarded std::string before passing it through the legacy json_raw callback, so the HTTP memory cap observes it only after allocation. Large or concurrent trace responses can bypass the new protection. These endpoints should emit directly into the streaming callback’s guarded writer, with a low-budget regression test.

[heifner]

Both endpoints moved off the json_raw pass-through onto add_async_api_stream: the request handler now produces an emitter -- misses (absent block, or trxid absent per the new contains_transaction) resolve to their 404 before a 200 emitter is committed -- and the emitter writes into the guarded json_writer the streaming callback provides, so the budget observes and can abort trace responses mid-serialize. Error responses emit error_results through the same streaming path handle_exception_stream uses. response_formatter grew writer-taking process_*_to_stream forms; the string-building forms remain as thin wrappers over them for tests and the benchmark. Tests: streaming_block_response_budget_abort drives the block emitter into a small-threshold guarded writer over a 32-transaction block and asserts the abort lands within a token of the threshold with the full body several times larger; streaming_emitter_miss_resolution_and_parity pins nullopt on every miss and byte-identical emitter output against the wrappers; the existing stream/variant parity suites are unchanged, and trace_plugin_test.py passes end-to-end over http.

[huangminghuang]

[P2] Preserve the sign of NaN

The streaming formatter always emits "nan" for NaN, while the legacy formatter emits "-nan" for a negative NaN. Arbitrary ABI float bit patterns can expose this, breaking the promised byte-for-byte serialization parity. Preserve std::signbit(d) and add positive/negative NaN parity coverage.

[heifner]

Fixed with signbit for both the nan and inf branches. Parity coverage: libfc leaf checks for positive and negative quiet NaN, both infinities, and a negative float NaN against the platform's variant output, plus abi_tests/non_finite_float_sign_parity packing raw negative-NaN float64/float32 bit patterns -- unreachable from JSON input -- and asserting binary_to_json_stream matches binary_to_variant + to_string byte-for-byte with the legacy "-nan" shapes pinned, verified failing against the previous code.

[heifner]

Follow-up: the macOS job caught that the legacy "-nan" spelling was itself platform-dependent -- glibc prints "-nan" but Apple's libc drops the sign, so on macOS the variant path emitted "nan" for a negative NaN and the new parity tests failed against it. Rather than mirroring whatever the local platform prints, s_fc_to_string now formats non-finite doubles explicitly from isnan/isinf + signbit, making both paths byte-identical and deterministic on every platform with Linux output unchanged; absolute spelling pins were added alongside the parity checks so the contract no longer depends on the platform's C library.

[huangminghuang]

Two inline findings from the current-head streaming JSON review.

[huangminghuang]

[P1] Charge deferred closures while they wait

This branch posts http_fwd without a bytes-in-flight reservation. Unlike the typed-result arm above, the deferred callable is not necessarily small: get_block's callable owns the fetched block and copied ABI blobs, transaction callables own the trace plus copied ABIs, and post_direct table-row callables own the collected raw rows. If the HTTP pool is busy, many of these can accumulate outside --http-max-bytes-in-flight-mb; the budget only observes them after they finally execute, which is too late to prevent the memory spike. Please carry retained-size metadata/a reservation with the deferred callable before posting it, and add a regression that blocks the HTTP worker and verifies a large deferred capture is charged while queued.

[heifner]

Fixed by carrying the retained size with the deferred call itself. chain::deferred_call<T> (the callable plus the retained size of what Phase 1 captured) replaces the bare function<t_or_exception<T>()> as the third alternative of next_function_variant and as the return type of the two-phase api methods; the size is a required constructor argument, so a new two-phase endpoint cannot return an unaccounted callable. bind_stream's post, post_direct and async-forward arms now take the RAII reservation (http_plugin::reserve_bytes_in_flight) before post_http_thread_pool and hold it in the posted lambda, so the charge spans both the queue wait and the run and releases when that lambda is destroyed -- deliberately outliving Phase 2 rather than being traded for a result-sized charge, since the captures are consumed into the result and the post arms hand that to the response callback inline. The legacy CALL_ASYNC_WITH_400 variant path had the identical hole and takes the same charge.

Sizes come from the producers, the only place that can see through the type erasure, and reuse the existing queued_payload_size vocabulary: get_account charges the copied ABI bytes plus the raw reslimit row; get_table_rows and get_table_rows_stream charge the collected row bytes plus the parsed abi_def; get_block charges the block, summed from each receipt's own packed size rather than a full re-pack, plus the captured ABI bytes; the send_transaction family charges the trace's retained size plus its captured ABI bytes. A queued_payload_size(captured_abis) overload is now shared by the trace estimate and the new producers instead of the same map being walked in two places.

Regression: stream_deferred_payload_reserved parks the single pool thread in a latched emitter, then runs a dispatch::post Phase 1 that hands 1 MiB of collected bytes to the deferred call, and asserts the sampled bytes_in_flight covers it while it waits behind the stall -- reads 0 without the reservation, verified -- then drains to zero once the response completes.

[huangminghuang]

[P2] Keep alternative serialization open to ADL

Qualifying this call as fc::to_json_stream fixes the overload set when static_variant.hpp is parsed, so serializers declared by subsequently included headers are invisible. A minimal TU that includes fc/static_variant.hpp before fc/crypto/sha256.hpp and streams std::variant<fc::sha256> fails to compile by falling into the reflector primary instead of SHA-256's string serializer. Please use unqualified dispatch here (as the new container helpers already do) and add a late-declared-alternative regression such as std::variant<fc::sha256>.

[heifner]

Confirmed and fixed. The transitive include closure of static_variant.hpp does not reach fc/serialize_as_string.hpp, so the qualified call froze an overload set that held only the unconstrained reflector primary; fc::sha256 carries just FC_REFLECT_TYPENAME, so the primary's static_assert fires. The visitor now dispatches unqualified, like the container helpers, keeping ADL open at the point of instantiation. The same qualified call in fc/io/enum_type.hpp had the same latent bug and is fixed with it, and bind_stream's own emit sites moved to the two-step form (using fc::to_json_stream; then an unqualified call) -- it is generic over the api result type but lives in namespace sysio, where plain unqualified lookup would not reach fc at all.

Regression: test_static_variant.cpp now includes fc/crypto/sha256.hpp after fc/static_variant.hpp -- the order is called out as load-bearing so it does not get tidied away -- and streams std::variant<fc::sha256, int32_t>, asserting the [0,"<hex>"] / [1,7] shapes and byte-parity with the variant path. Verified red: restoring the qualified call fails to compile in that translation unit with exactly the reflector static_assert.

[huangminghuang]

Four inline findings from the current-head re-review.

[huangminghuang]

[P1] Charge decoded trace captures against bytes-in-flight

This emitter owns the entire decoded block trace, and the transaction emitter below does the same, but these manual add_async_api_stream paths never take the source-payload reservation that bind_stream now carries. The guarded writer therefore charges only the growing output while the decoded trace remains live; concurrent large trace responses can exceed --http-max-bytes-in-flight-mb by roughly their retained source size. Please carry a retained-size reservation with these emitters and add a regression that samples bytes_in_flight while a large trace is held/emitted.

[heifner]

There is no queue window on these paths, so there is nothing to charge for. add_async_api_stream registers through make_http_thread_url_handler_stream, which invokes the handler inline on the http thread pool thread already serving the request, not on the read-only/read-write queue. make_http_stream_response_handler then dispatches on that same pool executor, and asio runs a dispatch inline when the caller is already inside the pool -- the note in common.hpp spells this out. The decoded trace is therefore created and consumed in one call stack on one thread; it never waits behind a busy pool the way a posted deferred_call does.

That is the distinction the bind_stream reservation was scoped to: it covers dispatch::sync and the async typed-result arm because those hand work across threads, and deliberately does not cover post/post_direct's inline short-circuit. Peak unaccounted here is http-threads x one decoded block trace, and trace_api's own trace types carry no console field, so a block trace is bounded by block net usage (max_block_net_usage, default 1 MiB). Two threads holding roughly a MiB each against a 500 MB default budget is well inside the tolerance the setting is designed for, and the guarded writer still observes and aborts the output side while it serializes.

[huangminghuang]

[P1] Stream Base64 before allocating the encoded temporary

Function-argument evaluation materializes the complete base64_encode(...) result before json_writer::value_string can consult its growth guard. For /v1/chain/get_raw_code_and_abi and /v1/chain/get_raw_abi, a raw blob that fits the source reservation can still allocate a 4/3-sized unaccounted temporary—and, on the accepted path, that temporary coexists with the source and output buffer. Please add a guard-aware/incremental Base64 writer primitive (or equivalent) so the response budget is enforced before and during encoding, with a single-large-blob allocation regression.

[heifner]

The blob is bounded by consensus, so this stays inside the budget's tolerance by construction. to_json_stream(const blob&) is reached only from get_raw_code_and_abi and get_raw_abi, and the bytes are chain state: fc caps blob deserialization at MAX_SIZE_OF_BYTE_ARRAYS (20 MiB), the chain caps a module at max_module_bytes (default 20 MiB), and the operative limit is that setcode has to fit a transaction, so default_max_transaction_net_usage (512 KiB) is what a deployer actually runs into. The encoded temporary is 4/3 of that: roughly 700 KB in practice, and about 27 MiB at a ceiling only reachable after raising net limits first, against a 500 MB default budget.

There is also no amplification here. The bytes must be committed to chain state, and paid for in RAM, before the endpoint will return 1.33x of them, so it is not a lever an unauthenticated caller can pull. --http-max-bytes-in-flight-mb is an approximate abuse guard rather than an allocator; it does not need to be exact, and a guard-aware incremental base64 primitive is real machinery bought to gain precision on a quantity consensus already bounds.

[huangminghuang]

[P1] Include the captured account result in the deferred reservation

The deferred callable also owns result, including every permission, authority vector, per-permission linked-action vector, and sysio_any_linked_actions, but retained counts only the ABI and reslimit bytes (and the no-ABI branch reserves zero). These collections are not bounded here—the phase-1 loops consume every matching database record—so a large account result can wait on the HTTP pool outside --http-max-bytes-in-flight-mb until phase 2 returns it and the result-side reservation finally starts. Please include a reservation-grade size for result in both branches and cover an account with a large permission/link set.

[heifner]

Agreed that this one has a real queue window -- get_account is dispatch::post, so Phase 1 runs on the read-only queue and the callable does wait on the pool. What it holds is still bounded by chain state: result.permissions and sysio_any_linked_actions are walks over permission_index and permission_link_index, and every entry is a permission_object or permission_link_object that someone paid RAM for. The in-memory struct is within a small factor of the on-chain footprint, so growing the response means first buying the RAM and carrying it permanently -- there is no amplification to exploit.

The terms that are not bounded that way are the ones already charged: the ABI copy is a contract's abi bytes and the reslimit row is a raw kv value, both independent of the requesting account. The no-ABI branch reserving zero is correct rather than a gap -- when ROA has no abi, Phase 2 skips the decode and the callable carries no copied bytes at all. Since --http-max-bytes-in-flight-mb is an approximate guard against abuse rather than an allocator, and the permission term does not move it in any reachable scenario, leaving the estimate as is.

[huangminghuang]

[P2] Keep generic sink emission open to late serializers

This qualified call has the same frozen-overload-set problem just fixed in static_variant: a TU that includes abi_sinks.hpp before database_utils.hpp and calls stream_sink::emit(softfloat128_t{}) resolves to the reflector primary and fails its static assertion, because the hand-written softfloat serializer is declared later. Please make this dispatch see the late/ADL-visible serializer and add that include-order compile regression; merely fixing static_variant leaves the generic ABI sink exposed.

[heifner]

Confirmed and fixed, though not quite the way the last ADL fix went. The dispatch now goes through a new fc::json_emit, which sits beside the reflector primary in fc/reflect/json_stream.hpp and calls to_json_stream unqualified; stream_sink::emit calls that rather than qualifying. bind_stream's local emit_json wrapper was the same using-declaration written a second time, so it is gone and its call sites use the shared helper -- the rule lives in one place now instead of being restated at each site outside fc.

Worth recording why this covers softfloat128_t, since it is a global-namespace typedef of an anonymous struct and associates nothing useful of its own: the second parameter is fc::json_writer&, so ADL pulls namespace fc into the lookup set at the point of instantiation and finds every fc serializer declared by then, including that one. The dispatch is open for any T for that reason, not only for types that live in fc.

Separately, those softfloat serializers moved out of the bottom of database_utils.hpp into sysio/chain/softfloat_serializers.hpp. That is layering cleanup rather than part of the fix -- the ADL change already covers them -- but it retires the block partway down database_utils.hpp that closed namespace sysio::chain, reopened fc to hand-declare to_variant and from_variant for the BE key codec, and reopened the chain namespace again. Include order handles it now.

Regression: abi_sink_dispatch_tests includes abi_sinks.hpp first and database_utils.hpp after -- the order is called out as load-bearing so it does not get tidied away -- and emits a softfloat128_t through stream_sink, asserting fc's canonical "0x" plus little-endian hex spelling. A second case covers the other half of the contract with a serializer in a namespace of its own, reached by ADL on the value rather than the writer, and a third pins variant_sink to the same shape. Verified red both ways: re-qualifying the call site in stream_sink::emit, and re-qualifying the body of fc::json_emit itself, each fail the translation unit on both types with the reflector primary's static_assert.

[huangminghuang]

Four inline findings from review of the current PR head.

[huangminghuang]

[P1] Preserve protobuf JSON representation

Splicing MessageToJsonString directly bypasses the legacy fc::variant normalization. A protobuf double such as 1.5 now emits as the JSON number 1.5, whereas pb_binary_to_variant followed by fc::json::to_string emits the string "1.50000000000000000"; escaping can differ as well. This changes valid API response types. Please route protobuf output through equivalent normalization and add protobuf floating-point/string cases to verify_stream_matches_variant.

[huangminghuang]

[P1] Restore the built-in array limit

The legacy built-in path unpacked vector<T>, whose raw decoder rejects sizes above MAX_NUM_ARRAY_ELEMENTS. This manual loop accepts and processes larger arrays, bypassing that hard resource limit. Please assert the limit after decoding sz in both streaming helpers and add an over-limit regression.

[huangminghuang]

[P2] Validate optional presence flags

The old optional<T> decoder reads a bool and rejects any presence byte other than 0 or 1. Reading a char here treats malformed values such as 2 as present, so streaming can decode data the variant path rejects or falls back from. Please preserve the canonical bool validation in both streaming helpers.

[huangminghuang]

[P1] Do not absorb response-budget aborts

This catch-all can swallow stream_response_budget_exceeded: after rewind, the next fallback token rechecks the guard against the smaller buffer, releases the previous charge, and can pass, producing a 200 response with hex or omitted decoded fields instead of the intended busy response. The same pattern exists in get_table_rows and trace serialization. Please latch or explicitly propagate guard aborts across every rewind/fallback path, and test guard-throw plus rewind plus fallback.