Skip to content

API-318: Add nansen mcp verify — client-independent authenticated MCP setup verification - #508

Open
gulshngill wants to merge 3 commits into
mainfrom
feat/api-318-mcp-verify
Open

API-318: Add nansen mcp verify — client-independent authenticated MCP setup verification#508
gulshngill wants to merge 3 commits into
mainfrom
feat/api-318-mcp-verify

Conversation

@gulshngill

@gulshngill gulshngill commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Problem

MCP clients can list the hosted Nansen MCP server's tools without a valid API key — the server serves tools/list (and free tools) unauthenticated, and tool-argument validation also runs before auth. So a broken credential setup looks successful until the first paid data call fails. There was no user-facing way to verify an MCP setup end to end.

Solution

New diagnostic command — client-independent, runnable with zero checkout:

npx -y nansen-cli mcp verify --api-key <key>

It performs a real authenticated data-path check, not just discovery:

  1. mcp-api-key — resolves the key (--api-keyNANSEN_API_KEY~/.nansen/config.json) and reports the masked key + source.
  2. mcp-server — JSON-RPC tools/list for reachability (explicitly labelled "unauthenticated; reachability only").
  3. mcp-authtools/call on the cheapest paid canary tool (nansen_score_top_tokens, ~1 credit). Server error text is mapped to actionable next steps: key rejected → check the client's NANSEN-API-KEY header / rotate at app.nansen.ai; insufficient credits → top up; rate limited → retry (and the setup is not reported verified).

Verified ⟺ the paid canary call succeeds with a well-formed result. On failure the command exits non-zero through the CLI's unified error envelope (code: MCP_VERIFY_FAILED); --json returns the structured checks (failure carries them in details). Success message is honest about scope: the CLI proves the key works against the server's paid data path and reminds the user to put that same key in their client's header; README documents an in-client canary prompt for that final hop.

No new dependencies — the stateless streamable-HTTP server needs exactly two fetch POSTs (JSON or SSE-framed responses both handled).

Changes

  • src/mcp-verify.js (new) — MCP JSON-RPC probe + checks + report formatting
  • src/cli.jsmcp verify wiring, HELP entry; bare nansen mcp prints usage only (never runs the paid probe implicitly)
  • src/doctor.js — export resolveAuthConfig, extract shared formatChecks (doctor output unchanged, proven by existing tests)
  • src/schema.jsonmcp verify command schema
  • README.md — "Verify your MCP setup" section + troubleshooting row
  • src/__tests__/mcp-verify.test.js (new, 22 tests) — mocked-fetch coverage of SSE/JSON parsing, every auth-failure mapping, timeout/network/unparseable paths, key precedence, report format, and the non-zero --json exit contract
  • .changeset/mcp-verify.md — minor

Testing

  • npm test: 2310 passed, 2 skipped (57 files) — all fetch mocked. Re-run after merging main (2026-08-28).
  • npm run lint: clean
  • Live smoke against production MCP: bogus key → key-rejected mapping + exit 1; valid key → verified + exit 0; --json failure → single machine-readable envelope; bare nansen mcp → usage, no network call

Note for docs: the verify one-liner is worth adding to the MCP "Connecting" page on docs.nansen.ai (external repo, out of scope here).

🤖 Generated with Claude Code

The hosted Nansen MCP server answers tools/list (and free tools) without
a valid API key, so a broken credential setup looks successful until the
first paid data call. `nansen mcp verify` runs a real authenticated
data-path check: key resolution, unauthenticated tools/list reachability,
then a paid tools/call canary (~1 credit) with server error text mapped
to actionable next steps. Non-zero exit through the unified error
envelope on failure; --json returns the structured checks. Runnable with
zero checkout via npx.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@nansen-pr-reviewer

nansen-pr-reviewer Bot commented Aug 19, 2026

Copy link
Copy Markdown

pr-reviewer Summary for #1b52831

No issues found

The code review completed successfully with no findings.

Review effort: 3/5 (Moderate)

Summary

This PR adds nansen mcp verify — a well-designed diagnostic command that closes a real gap: MCP's unauthenticated tools/list gave false confidence that a broken key setup was working. The implementation is clean and the approach is sound.

Overall assessment: The production code is correct, the error-handling paths are thorough, and the test suite covers every important branch (SSE vs JSON responses, all auth-failure mappings, timeout, network failure, malformed results, key-precedence, and the non-zero --json exit contract). No findings.

Specific things reviewed and confirmed correct:

  • parseArgs / flags['api-key'] guard — The boolean-flag whitelist in parseArgs does not include api-key, so a valueless --api-key correctly lands in flags (not options). The if (flags['api-key']) guard at line 938 catches this precisely, with a second guard for null/array values from JSON-parsed repeated flags.
  • tools/list sent unauthenticated — The mcpRequest call for tools/list omits apiKey, confirmed both in code and in the test that asserts listCall[1].headers does not have NANSEN-API-KEY.
  • Auth-failure classification ordering — Rate-limit matched before credits (the comment explains why: "credit rate limit" text would match both). The mixed-text test at line 170 pins this.
  • HTTP non-2xx + unparseable body pathMcpRequestError carries status but no rpcMessage; the catch block in runMcpVerifyChecks synthesizes { error: { message: text } } and passes error.status as httpStatus to authFailureCheck, so the HTTP status code still participates in the regex match. ✓
  • isRpcError coverage — Handles message.error present, message.result.isError === true, and the top-level message.isError === true variant. All tested.
  • doctor.js refactorformatChecks extracted cleanly; formatDoctorReport output is byte-identical. resolveAuthConfig and check are exported without behavioural change.
  • Changesetminor bump is correct for a new user-facing command. Package name nansen-cli is correct.
  • schema.jsonmcp.verify entry is present with api-key, url, and json options. Schema test at line 338 pins it.
  • Telemetry/offline classificationisMcpUsage correctly identifies bare nansen mcp (usage-only, no network) vs nansen mcp verify (live call), suppressing background telemetry for the former.

Token usage: 18,732 input, 5,867 output, 867,800 cache read, 53,422 cache write | Usage Guide

New pushes are reviewed automatically with a 10-minute cooldown between reviews. To request a review at any time, comment @nansen-pr-reviewer re-review.

Both remediation strings sent users to https://app.nansen.ai/account?tab=api --
the URL API-390 removed today from the MCP server (nansen-ra#3452) and from the
Kong 401 (nansen-api#1832). Merging this as-is would reintroduce it.

Use https://app.nansen.ai/api?tab=api, the key management view, matching both.

Deliberately NOT /auth/agent-setup: that page auto-fires POST /api-keys on load
and mints a key. Right for onboarding, wrong in an error path -- the user here
already has a key and needs to inspect or rotate it, and Free/Pioneer cap at 1
key so they would get a 403 on arrival.

Adds a test pinning the URL and excluding both rejected alternatives; verified
it fails if the URL regresses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant