docs(nats-traffic): code-verified NATS traffic & storage sizing model#461
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughDocumentation-only update to ChangesNATS Traffic Estimation Doc
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/nats-traffic-estimation.md (1)
235-240: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRename
PUSHtoPUSH_NOTIFICATIONhere.This is the only remaining row that uses the old/generic label, so it now reads like a different stream than the inventory and sizing tables. Keeping the name consistent avoids confusion with a non-existent
PUSHstream.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/nats-traffic-estimation.md` around lines 235 - 240, The table in this section still uses the old generic stream label, so update the `PUSH` entry in the sizing comparison to `PUSH_NOTIFICATION` to match the inventory and other sizing tables. Keep the wording aligned with the existing stream names used throughout the doc so the row clearly refers to the same server-side stream and not a separate `PUSH` stream.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@docs/nats-traffic-estimation.md`:
- Around line 235-240: The table in this section still uses the old generic
stream label, so update the `PUSH` entry in the sizing comparison to
`PUSH_NOTIFICATION` to match the inventory and other sizing tables. Keep the
wording aligned with the existing stream names used throughout the doc so the
row clearly refers to the same server-side stream and not a separate `PUSH`
stream.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 53c5642a-eb86-4c19-96e5-f2a5f3d7aa1c
📒 Files selected for processing (1)
docs/nats-traffic-estimation.md
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/nats-traffic-estimation.md`:
- Around line 457-462: The fenced bot pipeline diagram in the documentation is
missing a language tag, causing the MD040 lint failure. Update the diagram fence
in the NATS traffic estimation doc to use the existing diagram content with a
text language label, and keep the block otherwise unchanged so the Markdown
linter recognizes it correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 780aa051-b357-42f8-aff5-defdbd0fde2f
📒 Files selected for processing (1)
docs/nats-traffic-estimation.md
73a2abe to
f4912b6
Compare
…on.md) Rework docs/nats-traffic-estimation.md into a current, code-verified sizing model for a single site's NATS/JetStream traffic and storage. Inventory (§2): - §2.1 JetStream streams corrected to pkg/stream/stream.go + pkg/subject: ROOMS, PUSH_NOTIFICATION, HR, MIGRATION_OPLOG subjects/names; OUTBOX + flat INBOX federation (outbox-worker forwards to remote INBOX); parallel BOT_* pipeline - §2.2 server->client delivery subjects surveyed (RoomEvent/UserRoomEvent, SubscriptionUpdate, RoomKeyUpdate, PresenceState); stale stream.msg / metadata / notification subjects removed - §2.3 request/reply inventory surveyed across all services (client-facing by service + server-to-server), incl. presence R/R and lifecycle publishes Model (§4-§10): - Params: M=2.1M (incl user->botDM), M_room=2.0M, F=100, R_sub=10, R_hist=150, R_room=100, R_member=20, C_pres=10, R_presq=5, N=3, f_fed=20% (federated share of member changes -> OUTBOX), plus bot params; 4M split = 2.1M human + 1.9M bot - §6 single-connection baseline: JetStream (incl bot + OUTBOX) ~35M ops/ ~32 GB, core delivery ~448M/~278 GB, client + server-to-server R/R; TOTAL ~0.49B/day, ~0.46 TB/day - OUTBOX carries only the cross-site subset of member changes (R_member x U x f_fed); INBOX = local search feed + federated inflow - §7 multi-connection (D scaling), §8 migration phase (standalone), §9 storage (~8 GB; CANONICAL streams at 1-day TTL), §10 per-fab summary, §11 caveats All volumes are capacity-planning estimates, parameterized for re-runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7c7aaLBkrM3zDiyT67GL8
f4912b6 to
52eeba1
Compare
Design-review write-up (problem/impact/design/benefits/feasibility) for the bot/admin auth + admin-portal + unified /api/v1 gateway work. Includes planned bot-traffic capacity context from #461. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e only Volume adjustments (total messages 4.0M → 4.5M): - BOT_PLATFORM (user→bot) 0.1M → 0.5M, so M (MESSAGES/MESSAGES_CANONICAL) 2.1M → 2.5M; M_push set to M_room (2.0M) since user→bot forwards to BOT_PLATFORM and doesn't push a human - BOT_MESSAGES_CANONICAL / BOT_PUSH_NOTIFICATION 1.9M → 2.0M - room.key: rotation only on member-remove (add just hands the current key to the new member), so R_member × U × F ÷ 2 (~42M → ~21M deliveries) Recomputed §4, §6.1 (JS subtotal ~39M/~36 GB), §6.2 (message row (M_room+M_bot)×F = 400M/520 GB; core ~470M/~543 GB), §6.4 (~0.52B/day, ~0.73 TB), §6.6, §7.2/§7.3, §9 (~9 GB), §10 (all 14 fabs), §11. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7c7aaLBkrM3zDiyT67GL8
Summary
Reworks
docs/nats-traffic-estimation.mdinto a current, code-verified capacity model for a single site's NATS/JetStream traffic and storage. Every stream, subject, and driver is checked againstpkg/stream,pkg/subject, and the service handlers; all volumes are parameterized so the model can be re-run. Documentation only — no code changes.What the doc now contains
Inventory (§2), surveyed from code
ROOMS,PUSH_NOTIFICATION,HR,MIGRATION_OPLOG); OUTBOX + flat INBOX federation (outbox-worker forwards to remote INBOX); parallelBOT_*pipeline;SYSTEMstream.RoomEvent/UserRoomEvent,SubscriptionUpdate,RoomKeyUpdate,PresenceState; stalestream.msg/metadata.update/notificationsubjects removed.Traffic & storage model (§3–§11)
R_sub,R_hist,R_room,R_member,C_pres, …), plusf_fed,N.(M_room+M_bot)×Fdominates); client + server-to-server R/R. TOTAL ≈ 0.52B ops/day, ≈ 0.73 TB/day.Notes
BOT_*pipeline, andSYSTEMare new/target design not yet inpkg/stream— the doc flags them as such.f_fed≈20%,N≈3, bot fan-out, room-key rotation on member-remove,M_push=M_room) are called out inline for easy tuning.🤖 Generated with Claude Code