Skip to content

Use exponential backoff for outbound peer reconnection retries - #4107

Open
ayushsingh82 wants to merge 1 commit into
near:mainfrom
ayushsingh82:exponential-backoff-peer-reconnect
Open

Use exponential backoff for outbound peer reconnection retries#4107
ayushsingh82 wants to merge 1 commit into
near:mainfrom
ayushsingh82:exponential-backoff-peer-reconnect

Conversation

@ayushsingh82

Copy link
Copy Markdown

Summary

  • PersistentConnection's dial loop in crates/node/src/p2p.rs retried on a fixed 1s delay, so a peer that's persistently offline gets dialed — and logged at info level — once a second forever, spamming logs (and the network) while unrelated to any other issue.
  • Replaced the fixed delay with backon::ExponentialBuilder (1s min, 60s max, jittered, unbounded retries), matching the pattern already used in crates/node/src/indexer/tee.rs.
  • Backoff resets back to the fast 1s retry once a connection succeeds, so a peer that flaps doesn't inherit a long delay from its previous outage.
  • Added the resolved retry_delay to the existing "could not connect, retrying" log line for visibility into the current backoff state.

Closes #4063

Verification

  • cargo check -p mpc-node --all-features: clean
  • cargo clippy -p mpc-node --all-targets --all-features -- -D warnings: clean (type-checks test code too)
  • cargo fmt -- --check: clean
  • Didn't complete a full cargo test link+run locally (same disk constraint as Remove account_id parameter from IndexerViewClient methods #4102 — linking mpc-node's test binary needs more disk than available here). The two existing PersistentConnection tests (persistent_connection__should_keep_retrying_when_dial_attempt_hangs, persistent_connection__should_dial_new_address_after_resolver_update) use #[tokio::test(start_paused = true)], so tokio's virtual clock auto-advances through sleeps regardless of duration — they only assert the loop keeps retrying/dialing, not on the fixed 1s timing, so the growing backoff delays shouldn't affect them. Happy to fix up anything CI's test run surfaces.

Test plan

  • CI clippy job passes
  • CI test suite passes (including the two PersistentConnection retry tests)

PersistentConnection's dial loop retried on a fixed 1s delay, so a
persistently offline peer got dialed (and logged at info level) once a
second forever. It now backs off exponentially between 1s and 60s
(matching the pattern already used in indexer/tee.rs), resetting back
to the fast retry once a connection succeeds.

Closes near#4063
@ayushsingh82

Copy link
Copy Markdown
Author

@netrome tagging you as the issue author — this is ready for review whenever you have time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exponential backoff when trying to reconnect to peers

1 participant