Add backup/restore tests for Memory and ScyllaDB backends (Phase 0b)#6392
Draft
ndr-ds wants to merge 4 commits into
Draft
Add backup/restore tests for Memory and ScyllaDB backends (Phase 0b)#6392ndr-ds wants to merge 4 commits into
ndr-ds wants to merge 4 commits into
Conversation
Instruction Count Benchmark Results
Deterministic metrics — reproducible across runs (34 benchmarks)
Cache-dependent metrics — expect fluctuations between runs (34 benchmarks)
|
Extends the DatabaseBackup trait and cross-key boot tests introduced in #6332 to cover MemoryDatabase and ScyllaDbDatabase in addition to RocksDB. linera-views: - MemoryDatabase: backup serializes all root-key partitions to memory_backup.bcs (BCS-encoded BTreeMap<root_key, BTreeMap<k, v>>). - ScyllaDbDatabaseInternal: full-table SELECT scan written to scylladb_backup.bcs in the same format; uses a fresh thread + current_thread Tokio runtime to bridge the sync trait into async CQL. - JournalingKeyValueDatabase<D>: added DatabaseBackup forwarding impl to complete the ScyllaDB wrapper chain. - DatabaseBackup doc comment updated to be backend-agnostic. linera-core: - Backup test module now compiled under #[cfg(test)] (was rocksdb-only), with per-backend #[cfg(feature)] guards inside the file. - 9 tests total: 3 scenarios x 3 backends (memory, rocksdb, scylladb). - Memory restore: BCS deserialize + open_shared + write_batch. - ScyllaDB restore: same BCS format, strips get_big_root_key prefix byte, re-inserts via ScyllaDbDatabaseInternal::open_shared + write_batch.
…le-threaded test runtime
ndr-ds
force-pushed
the
ndr-ds/phase-0b-memory-scylla-backup-tests
branch
from
f129f15 to
103df17
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Phase 0a (#6332) verified that a
WorkerStatecan boot from a RocksDB backup produced by a different validator. This follow-up extends theDatabaseBackuptrait and cross-key boot tests to theMemoryDatabaseandScyllaDbDatabasebackends.Proposal
linera-viewschanges:DatabaseBackupforMemoryDatabase: BCS-serializes all root-key partitions tomemory_backup.bcs(BTreeMap<root_key, BTreeMap<k, v>>).DatabaseBackupforScyllaDbDatabaseInternal: full-tableSELECT root_key, k, vscan written toscylladb_backup.bcsin the same format. Uses a fresh thread +current_threadTokio runtime to bridge the sync trait into async CQL without nesting runtimes.DatabaseBackupforwarding impl forJournalingKeyValueDatabase<D>— fills the missing link in the ScyllaDB wrapper chain (LruCachingDatabase → ValueSplittingDatabase → JournalingKeyValueDatabase → ScyllaDbDatabaseInternal).DatabaseBackupdoc comment updated to be backend-agnostic.linera-corechanges:#[cfg(test)](wasrocksdb-only), with per-backend#[cfg(feature = "...")]guards inside the file.open_shared+write_batch.get_big_root_keyleading byte to recover the logical root key, re-inserts viaScyllaDbDatabaseInternal::open_shared+write_batch.Test Plan
cargo test --package linera-core --features test worker_backup_tests— pass.cargo test --package linera-core --features test,rocksdb worker_backup_tests— pass.scylladb.ymlCI workflow (Dockerscylladb/scylla:6.1).cargo clippyclean on all three feature combinations (test,test,rocksdb,test,rocksdb,scylladb).Release Plan
Links