Skip to content

Limit and paginate get_accounts_by_authorizers results - #1894

Open
pur3miish wants to merge 4 commits into
AntelopeIO:mainfrom
pur3miish:agent/get-accounts-authorizers-limit
Open

Limit and paginate get_accounts_by_authorizers results#1894
pur3miish wants to merge 4 commits into
AntelopeIO:mainfrom
pur3miish:agent/get-accounts-authorizers-limit

Conversation

@pur3miish

@pur3miish pur3miish commented Jul 21, 2026

Copy link
Copy Markdown

Summary

  • add an optional caller-supplied limit to get_accounts_by_authorizers
  • preserve legacy complete-result behavior when limit is omitted, bounded by the existing server-side HTTP deadline
  • cap explicitly paginated requests at 1,000 rows per page
  • add opaque cursor pagination so callers can retrieve results beyond the page-size cap
  • make tied authority rows deterministic by ordering them by owner and permission name
  • document the request and response fields in the Chain API schema
  • add focused coverage for pagination, deterministic ordering, deadline expiry, and zero-limit existence checks

Why

A public key can be assigned to an unusually large number of account permissions. The endpoint previously traversed and serialized every match, so a pathological or attacker-influenced authority set could consume significant CPU, memory, HTTP worker time, and bandwidth.

The read-only API already receives the deadline derived from http-max-response-time-ms, but read_only::get_accounts_by_authorizers previously discarded it. This change threads that deadline through input deduplication and authority-index traversal. If an unpaginated legacy request cannot finish its query work before the deadline, it fails with a timeout rather than returning a partial response that looks complete.

For backward compatibility, callers that omit both limit and cursor retain complete-result behavior. Callers opt into pagination by supplying limit; the effective page size is capped at 1,000, and all matches remain obtainable through next_cursor.

Addresses the node-side result-limit portion of #1893 and incorporates the pagination, deterministic ordering, and deadline recommendations from review.

API behavior

Request Behavior
No limit, no cursor Legacy mode: return the complete result or fail when the server-side deadline expires.
limit supplied Paginated mode: return at most min(limit, 1000) rows.
cursor supplied Continue after the previous page; an explicit limit and the same accounts/keys inputs are required.
limit: 0 Existence probe: return no rows, set more if a match exists, and leave next_cursor empty because the query did not advance.

Successful unpaginated and final-page responses return more: false and an empty next_cursor.

Pagination and ordering

The input accounts and keys are deduplicated into ordered sets. Accounts are processed first, followed by keys. Authority-index rows are ordered by:

(authorizer, weight, owner, permission_name)

Adding (owner, permission_name) makes rows tied on authorizer and weight deterministic across node histories.

next_cursor is an opaque, versioned Base64URL encoding of:

  • the requested account or key currently being processed
  • the last authorizing account when traversing an account range
  • the last emitted (weight, owner, permission_name) coordinate

On the next request, the server decodes the cursor and resumes with upper_bound immediately after that coordinate, preventing the last row of the previous page from being repeated.

As with get_table_rows, changes to chain state between pages can affect a multi-page result. The cursor is continuation state, not a snapshot identifier.

Curl examples

Legacy complete-or-timeout request:

curl -sS \
  -X POST \
  http://127.0.0.1:8888/v1/chain/get_accounts_by_authorizers \
  -H 'Content-Type: application/json' \
  -d '{
    "accounts": [],
    "keys": ["PUB_K1_REPLACE_WITH_PUBLIC_KEY"]
  }'

First paginated request:

curl -sS \
  -X POST \
  http://127.0.0.1:8888/v1/chain/get_accounts_by_authorizers \
  -H 'Content-Type: application/json' \
  -d '{
    "accounts": [],
    "keys": ["PUB_K1_REPLACE_WITH_PUBLIC_KEY"],
    "limit": 1000
  }'

An incomplete page returns:

{
  "accounts": [
    {
      "account_name": "exampleacct",
      "permission_name": "active",
      "authorizing_account": null,
      "authorizing_key": "PUB_K1_REPLACE_WITH_PUBLIC_KEY",
      "weight": 1,
      "threshold": 1
    }
  ],
  "more": true,
  "next_cursor": "AQEBA..."
}

Pass that value back with the same inputs and an explicit limit:

curl -sS \
  -X POST \
  http://127.0.0.1:8888/v1/chain/get_accounts_by_authorizers \
  -H 'Content-Type: application/json' \
  -d '{
    "accounts": [],
    "keys": ["PUB_K1_REPLACE_WITH_PUBLIC_KEY"],
    "limit": 1000,
    "cursor": "AQEBA..."
  }'

Continue until more is false and next_cursor is empty.

Deadline scope

The deadline is checked while deduplicating supplied accounts/keys, before processing each input range, and while emitting matching rows. A breach throws fc::timeout_exception; it does not produce a partial success response.

This change specifically bounds account-query preparation and index traversal. It does not add a separate response-byte limit or extend the deadline into JSON serialization and network transmission after the query returns.

Validation

  • git diff --check
  • Swagger YAML parsed successfully
  • C++ coverage added for legacy behavior, pagination, ordering, and deadline expiry

Tests not run locally; CI is awaiting maintainer approval.

@heifner

heifner commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Bounding this endpoint is worth doing — the WAX key in #1893 makes the amplification concrete, and a caller-supplied limit is useful on its own. Two concerns about the unconditional max_results cap as implemented, then a suggestion.

A hard cap with no continuation makes the complete answer unobtainable. The natural consumers of this endpoint are asking "which account permissions does this key satisfy" — key-compromise auditing, or exactly the typo-squat investigation described in #1893. Those are the queries where the tail matters most. With an unconditional 1000-row cap and no way to resume, any authority set past the cap becomes permanently unqueryable through the API (no other RPC enumerates permissions by key). more: true tells a new client its answer is incomplete but gives it no way to finish, and existing clients that predate the field will silently treat the first 1000 rows as the complete authorizer set. For this particular query, a quietly incomplete answer is worse than a slow one.

The truncated set is not deterministic. The bimap left keys order rows by (value, weight) only. Rows that tie on both — the common shape for the pathological case, thousands of permissions holding the same key at weight 1 — iterate in the underlying container's insertion order, which depends on the node's local history: the startup rebuild inserts in permission-index iteration order, while a later updateauth re-inserts the row at the back of its tie run. So when the cap bites inside a tie run, which rows make the cut can differ between a long-running node and a freshly restarted one. Two nodes answering the same query differently is a rough property for an RPC.

Suggestion: cursor pagination in the existing more/next_key style. get_table_rows already returns more plus next_key for value-based resumption, and that shape fits here:

  • Make the collation deterministic first: break (value, weight) ties by (owner, permission_name) — either fold them into the left key or compare permission_info::cref by owner/name. That fixes the nondeterminism above independently of pagination.
  • Resume state is then two coordinates: the requested account/key currently being processed (inputs are deduplicated into ordered sets, so the input value itself identifies the position), plus the last emitted (weight, owner, permission_name) within its range, used as a lower bound for the next page. Encode both in an opaque next_cursor echoed back by the caller.
  • Consistency between pages carries the same caveats as get_table_rows — no worse than the rest of the API.

With a cursor in place, limit/max_results stops being a correctness question and is purely a page size.

Alternative (or complement): honor the deadline instead of clamping silently. read_only::get_accounts_by_authorizers already receives the http-max-response-time-ms deadline and drops it (unnamed parameter). Threading it into account_query_db::get_accounts_by_authorizers and failing with a timeout on breach bounds per-request work without ever producing a response that looks complete but isn't. It also covers a cost the row cap doesn't: per-request work scales with the number of supplied accounts/keys (one indexed seek each), so a large keys array does unbounded seek work even with limit: 0.

Two smaller notes if a server-side max stays: consider capping only the default while honoring a larger explicit caller limit (or making the max operator-configurable) — this endpoint is opt-in via enable-account-queries, and operators who enable it typically do so because their tooling needs complete answers. And limit: 0 currently returns zero rows with more: true, which is a reasonable existence probe if intentional, but worth documenting either way.

@pur3miish pur3miish changed the title Limit get_accounts_by_authorizers results Limit and paginate get_accounts_by_authorizers results Jul 22, 2026
@pur3miish

pur3miish commented Jul 23, 2026

Copy link
Copy Markdown
Author

A hard cap with no continuation makes the complete answer unobtainable.

The truncated set is not deterministic.

Suggestion: cursor pagination in the existing more/next_key style.

Alternative (or complement): honor the deadline instead of clamping silently.

@heifner thanks for the detailed feedback. I’ve updated the PR to address these points while preserving backward compatibility:

  • When limit is omitted, the endpoint keeps its legacy behavior and attempts to return the complete result.
  • The deadline already supplied to read_only::get_accounts_by_authorizers was unnamed and discarded. It is now passed through to account_query_db::get_accounts_by_authorizers and checked while deduplicating inputs and iterating results, so an unpaginated request fails on deadline instead of running without that bound.
  • Supplying limit opts into pagination. The explicit page size is capped at 1,000, and more plus the opaque next_cursor can be used to retrieve the remaining pages. A cursor requires an explicit limit and the original accounts/keys inputs.
  • Ordering is now deterministic by breaking (value, weight) ties with (owner, permission_name).
  • The Swagger description and focused tests document and cover continuation, legacy behavior, limit: 0, deterministic ordering, and expired deadlines.

Follow-up commits:

  • 1c6f3190b — add cursor pagination, deterministic ordering, deadline handling, documentation, and tests
  • cea34a18b — correct the account-query test signal hookup
  • b07c495a1 — preserve legacy unpaginated behavior and make pagination explicit

@pur3miish
pur3miish marked this pull request as ready for review July 24, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants