Skip to content

fix(sdk): reconnect on dropped socket via ConnectionManager#419

Merged
TinDang97 merged 1 commit into
mainfrom
fix/sdk-connection-manager-reconnect
Jul 20, 2026
Merged

fix(sdk): reconnect on dropped socket via ConnectionManager#419
TinDang97 merged 1 commit into
mainfrom
fix/sdk-connection-manager-reconnect

Conversation

@TinDang97

@TinDang97 TinDang97 commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Problem

The Rust SDK's MoonClient wrapped a redis::aio::MultiplexedConnection, which opens one socket at connect time and never rebuilds it. When the backend drops the connection — a server restart, an idle-timeout reap, a transient network blip — every subsequent command fails with broken pipe forever; the only recovery is to reconstruct the client (restart the whole consuming process).

This wedged Lunaris in production: after a live Moon daemon flip, the lunaris-mcp server's pooled connection went permanently dead and memory.recall returned broken pipe on every call until the MCP process was restarted.

Fix

Switch MoonClient (and all 9 sub-clients, which clone the shared conn) to redis::aio::ConnectionManager — the reconnecting variant. It transparently re-establishes the socket with backoff, is Clone, and shares reconnect state across clones via Arc, so a heal on any handle repairs every sub-client. connect_with_timeout preserves the existing per-command response-timeout contract via ConnectionManagerConfig.

Test (red/green)

sdk/rust/tests/reconnect.rs spawns a real redis-server, drives a PING through a long-lived client, kills the server, restarts it on the same port, and asserts the same client recovers within a bounded window:

  • RED (MultiplexedConnection): never recovers → times out.
  • GREEN (ConnectionManager): reconnects → PONG.

Self-contained (spawns its own server; skips cleanly if redis-server is absent).

Scope

moondb is a standalone crate (not a member of the moon server workspace), so this does not touch the server build. Incidental: fixed pre-existing moon:: rename leftovers (crate is now moondb) in integration.rs + three doctests + one malformed FT.AGGREGATE doctest, so cargo test -p moondb (tests + doctests) is green.

skip-changelog: SDK-internal reliability fix; the commit touches only sdk/rust/*.

Summary by CodeRabbit

  • New Features

    • Rust SDK connections now automatically reconnect after the backend restarts.
    • Long-lived clients can resume operations without requiring a new connection.
  • Bug Fixes

    • Improved connection resilience across cache, graph, messaging, session, temporal, text, vector, and workspace operations.
  • Documentation

    • Updated Rust examples and integration tests to use the current SDK naming and return types.

The Rust SDK's MoonClient wrapped a redis::aio::MultiplexedConnection,
which opens one socket at connect time and NEVER rebuilds it. When the
backend drops the connection — a server restart, an idle-timeout reap, a
transient network blip — every subsequent command fails with `broken
pipe` forever, and the only recovery is to reconstruct the client (i.e.
restart the whole consuming process).

This wedged Lunaris in production: after a live Moon daemon flip, the
lunaris-mcp server's pooled connection went permanently dead and
memory.recall returned `broken pipe` on every call until the MCP process
was restarted.

Switch MoonClient (and all 9 sub-clients, which clone the shared conn) to
redis::aio::ConnectionManager, the reconnecting variant. It transparently
re-establishes the socket with backoff, is Clone, and shares reconnect
state across clones via Arc so a heal on any handle repairs every
sub-client. connect_with_timeout preserves the existing per-command
response-timeout contract via ConnectionManagerConfig (io-failsafe).

Regression test tests/reconnect.rs spawns a real redis-server, drives a
PING through a long-lived client, kills the server, restarts it on the
same port, and asserts the SAME client recovers within a bounded window:
RED (MultiplexedConnection) never recovers; GREEN (ConnectionManager)
reconnects. Self-contained (spawns its own server; skips if redis-server
is absent).

Incidental: fixed pre-existing `moon::` rename leftovers (the crate is now
`moondb`) in integration.rs and three doctests, plus a malformed
FT.AGGREGATE doctest, so `cargo test -p moondb` (tests + doctests) is
green alongside the new regression test.

author: Tin Dang
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@TinDang97 TinDang97 added the skip-changelog Skip the CHANGELOG.md update gate for this PR label Jul 20, 2026
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5a2cdd41-28df-46b6-877e-3ffdfbf18de0

📥 Commits

Reviewing files that changed from the base of the PR and between 5db9629 and 0134323.

⛔ Files ignored due to path filters (1)
  • sdk/rust/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (12)
  • sdk/rust/Cargo.toml
  • sdk/rust/src/cache.rs
  • sdk/rust/src/client.rs
  • sdk/rust/src/graph.rs
  • sdk/rust/src/mq.rs
  • sdk/rust/src/session.rs
  • sdk/rust/src/temporal.rs
  • sdk/rust/src/text.rs
  • sdk/rust/src/vector.rs
  • sdk/rust/src/workspace.rs
  • sdk/rust/tests/integration.rs
  • sdk/rust/tests/reconnect.rs

📝 Walkthrough

Walkthrough

The Rust SDK now uses Redis ConnectionManager connections, including timeout-aware construction and reconnect behavior. All Redis-backed clients adopt the new connection type, and a restart regression test verifies recovery of a long-lived client.

Changes

Redis connection manager migration

Layer / File(s) Summary
Connection manager setup and timeout handling
sdk/rust/Cargo.toml, sdk/rust/src/client.rs
Enables the Redis connection-manager feature, updates MoonClient construction and timeout configuration, and exposes the updated connection type through inner_mut.
Client connection field migration
sdk/rust/src/cache.rs, sdk/rust/src/graph.rs, sdk/rust/src/mq.rs, sdk/rust/src/session.rs, sdk/rust/src/temporal.rs, sdk/rust/src/text.rs, sdk/rust/src/vector.rs, sdk/rust/src/workspace.rs
Changes Redis connection fields to ConnectionManager while preserving existing command methods.
Reconnect validation and SDK examples
sdk/rust/tests/reconnect.rs, sdk/rust/tests/integration.rs, sdk/rust/src/graph.rs, sdk/rust/src/vector.rs, sdk/rust/src/text.rs
Adds backend restart recovery coverage and updates examples, doctests, validation formatting, and vector decode paths.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Test
  participant MoonClient
  participant Redis
  Test->>Redis: Start backend and verify PING
  Test->>MoonClient: Create long-lived client
  Test->>Redis: Kill and restart backend
  MoonClient->>Redis: Retry ping through ConnectionManager
  Redis-->>MoonClient: Return PONG
  MoonClient-->>Test: Confirm recovery
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: switching the Rust SDK to reconnect via ConnectionManager.
Description check ✅ Passed The description is detailed and covers the problem, fix, test plan, and scope, though it does not follow the repository template headings exactly.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/sdk-connection-manager-reconnect

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@TinDang97
TinDang97 merged commit ea7bd12 into main Jul 20, 2026
13 checks passed
TinDang97 added a commit to pilotspace/lunaris that referenced this pull request Jul 20, 2026
…y guard

Root cause of the production `memory.recall` -> `broken pipe` wedge: the
Moon SDK's MoonClient wrapped a redis::aio::MultiplexedConnection, which
opens one socket at connect time and never rebuilds it. After a live Moon
daemon flip (or any socket drop / idle reap), every later command failed
with `broken pipe` forever until the consuming process (lunaris-mcp) was
restarted.

Two layers, per the "both" decision:

1. SDK fix (vendor/moon bump 3f99f50 -> 0134323, moon PR pilotspace/moon#419):
   MoonClient now wraps redis::aio::ConnectionManager, the reconnecting
   variant — transparently re-establishes the socket with backoff, shares
   reconnect state across sub-client clones via Arc. connect_with_timeout
   keeps the per-command response-timeout contract via
   ConnectionManagerConfig. The pin moves ONLY the SDK (byte-identical to
   v0.8.1 otherwise) — zero server/embedded-moon delta.

2. Belt-and-suspenders guard (crate::retry): even with a reconnecting
   manager, the single command that races the reconnect can surface a
   transient error before the heal completes. `with_conn_retry` retries a
   read AT MOST ONCE on a classified-transient connection fault (broken
   pipe / connection reset|refused|aborted|closed / not connected), wired
   on the two recall reads (vector_search, keyword_search). Non-transient
   errors (missing index, NotSupported) are never retried; the cap is one
   retry — never an unbounded loop.

Tests:
- SDK: vendor/moon/sdk/rust/tests/reconnect.rs spawns a real redis-server,
  kills+restarts it, asserts a long-lived client recovers (RED on
  MultiplexedConnection, GREEN on ConnectionManager).
- Guard: crate::retry unit tests — classification + retry-once-then-succeed
  + no-retry-on-non-transient + one-retry-cap + success-path. Red/green
  proven by neutering the retry branch (discriminating tests flip RED).

author: Tin Dang
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Skip the CHANGELOG.md update gate for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant