Skip to content

fix(ethrex): realign DB config with ethrex v23.0.0 - #127

Merged
CPerezz merged 3 commits into
ethereum:mainfrom
edg-l:ethrex-schema-v23
Aug 4, 2026
Merged

fix(ethrex): realign DB config with ethrex v23.0.0#127
CPerezz merged 3 commits into
ethereum:mainfrom
edg-l:ethrex-schema-v23

Conversation

@edg-l

@edg-l edg-l commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

What

The ethrex writer tracked v13–v16. Realigns it to v23.0.0:

  • bad_blocks added to Tables — 20 CFs → 21. ethrex v23 has 21; it was creating the missing one itself on first boot.
  • metadata.jsonschema_version: 3 — ethrex has been at 3 since v16. Writing 2 sent every first boot through the migration branch, which runs pending migrations over the whole DB first.
  • Boot image pin → 23.0.0, golden dump regenerated against it. Only chain_data[0x80] (hegotaTime) and the new empty bad_blocks CF change; state root unaffected.
  • Block-cache comment corrected: ethrex defaults to 12 GiB via --rocksdb.block-cache-size, not the 4 GiB the comment claimed to mirror. Value kept, now documented as deliberate.
  • transaction_locations merge operator: documented why it isn't mirrored (no rows written; a CF created without one reopens fine with one registered).
  • RUNBOOK: note that ethrex's 12 GiB default block cache OOM-kills a boot in a smaller container.

Relation to #117

Independent, but both touch dbs_cgo.go. #117 bounds peak RSS; this only adds cfIdxBadBlocks and rewrites two comments in the same options block. cache_index_and_filter_blocks is deliberately left to #117 — it already adds it, and it happens to match what ethrex v23 does. Whichever lands second needs a small rebase there.

Verification

Untagged suite, go build, go vet, gofmt all clean.

cgo suite run locally in a Dockerfile.ethrex builder image:

--- PASS: TestGenesisDumpGolden
--- PASS: TestEthrexGoldenStateRoot
--- PASS: TestGenesisHeaderGolden
--- PASS: TestE2ESuite (204.64s)

TestE2ESuite ran the full CI path (REQUIRE_SPAMOOR=1, spamoor built from master) against ghcr.io/lambdaclass/ethrex:23.0.0: db-gen → boot → genesis-root capture → oracle re-query → ~100 blocks of erc20_bloater → post-spamoor re-query. Result artifact: post_spamoor_entity_check: ok, post_spamoor_chain_advanced: true, post_spamoor_beacon_roots_ok: true, tip block 149.

@CPerezz

CPerezz commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Hey @edg-l thanks for the PR!

I was reviewing and fixing #117. Let me push the changes and merge. Such that I can go for this one! And thanks for updating htis!

@edg-l

edg-l commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up for a later PR, noting it here so it isn't rediscovered: ethrex#7095 changes both the account_codes value encoding and the account-code CF table options, so the writer needs one more realignment once that lands in a release.

Jumpdest encoding. account_codes values stay RLP(bytecode) || RLP(jumpdests), but the second half becomes a 1-bit-per-byte bitmap instead of an RLP list of u32 offsets. ethrex reads both forms from that PR onward (it discriminates on the RLP item header and rebuilds the bitmap from the bytecode for the list form), but v23.0.0 cannot read a bitmap, so this can't ride along here without breaking the boot image and golden dump this PR pins.

When it does land, in internal/ethrex/code.go:

  • Bitmap is ceil(len(code)/8) bytes, bit i set as 1 << (i % 8), so LSB-first within each byte. Same convention as bitvec<u8, Lsb0> in client/reth/bytecode_writer_cgo.go, so the bit-setting logic is already in the repo.
  • Jumpless bytecode encodes as an empty byte string (0x80), not a run of zero bytes: ethrex shares a single zero-length bitmap in that case and reads a missing byte as "not a jump destination". Runtime-invisible, but it would break golden-dump byte-identity.
  • PUSH-immediate skipping is unchanged. Only the output representation moves.

Table options. The same PR gives account_codes a 4KB data-block size (down from 32KB) and a 10-bits-per-key bloom filter, and moves account_code_metadata off the default arm to the same 4KB + bloom. In dbs_cgo.go that is one new case covering both CFs.

The table options are safe to write early, since RocksDB doesn't validate write-time table options on open, but they buy nothing on their own: neither change does anything until a snapshot is regenerated, and both want the same repin. Worth doing together.

For context on why it matters: because the current snapshots carry the list form, ethrex rebuilds the bitmap from the bytecode on every code-cache miss against them, 17.4 us per 24KB jumpdest-dense contract instead of 0.48 us. On the benchmarkoor bloatnet fixtures that is a large enough share of a cold contract-code access to move ethrex's COLD_ACCOUNT_CODE_ACCESS number in the EIP-8038 repricing fit, so a snapshot written by a post-#7095 ethrex would represent the client more accurately than the one we generate today.

@edg-l
edg-l force-pushed the ethrex-schema-v23 branch from 6cb8ab1 to 9d91d11 Compare August 4, 2026 13:26
edg-l and others added 2 commits August 4, 2026 17:27
The writer tracked ethrex v13-v16. Three things had drifted:

- bad_blocks was missing from Tables (20 CFs, ethrex v23 has 21), so
  ethrex created it itself on first boot.
- metadata.json said schema_version 2; ethrex has been at 3 since v16,
  so every first boot ran the migration branch over the whole DB.
- Boot image and golden dump were pinned to v16.0.0.

Repins all three to v23.0.0 and regenerates genesis_dump.json against it.
Every state-bearing CF is byte-identical to the v16 dump; only
chain_data[0x80] (gains hegotaTime) and the new empty bad_blocks CF
differ, so the state root and the Go codecs are unaffected.

Also corrects the block-cache comment (ethrex defaults to 12 GiB via
--rocksdb.block-cache-size, not the 4 GiB the comment claimed; the
smaller cache this writer runs stays as sized) and documents why the
transaction_locations merge operator is not mirrored.
- bad_blocks landed in ethrex v22.0.0 (#6948), hegotaTime in v21.0.0
  (#6326) — CHANGELOG and gen/README attributed both to v23
- internal/ethrex/doc.go still claimed the v15.0.0 pin (stale since
  v16); now v23.0.0
- RUNBOOK: the 12 GiB block cache is a lazily-filled ceiling — soften
  'OOM-killed before it serves a request' accordingly
- StoreSchemaVersion comment: note a value above ethrex's is a hard
  MigrationFailed boot error (the loud counterpart to the silent
  drop_obsolete_cfs CF cleanup)
- comment trim per repo bar; CHANGELOG entry 22 -> 9 lines; reflow two
  broken comment paragraphs
@CPerezz
CPerezz force-pushed the ethrex-schema-v23 branch from 9d91d11 to 40bdc3d Compare August 4, 2026 15:29
@CPerezz

CPerezz commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the PR @edg-l
Pushed two commits on top (your commit untouched):

  • rebase onto main (CHANGELOG conflict with the client/geth: mirror geth's per-level Pebble options + parity guard #124 entry — only conflict).
  • 40bdc3d: three corrections found while verifying — bad_blocks landed in v22.0.0 (ethrex#6948) and hegotaTime in v21.0.0 (ethrex#6326), both were attributed to v23; internal/ethrex/doc.go still claimed the v15 pin (stale since v16, missed file); and the RUNBOOK OOM sentence softened (the cache is a lazily-filled LRU ceiling — it doesn't kill a boot on its own, it can as it fills on a large DB). Plus the repo's comment-minimalism pass: CHANGELOG entry 22→9 lines, tx-locations comment 7→4, and the StoreSchemaVersion comment now records the loud-vs-silent asymmetry (schema overrun fails the boot; CF overrun is silently dropped).

@CPerezz
CPerezz merged commit 4d8d593 into ethereum:main Aug 4, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants