-
Notifications
You must be signed in to change notification settings - Fork 48
feat: add evlog/eve integration
#399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
7868bc1
feat(eve): add evlog/eve integration with defineEvlogHook
HugoRCD 7fb6b82
feat(eve): add Clearbill support example on Eve and Next.js
HugoRCD 0a2538f
docs(eve): add use-case docs and semantic PR scope
HugoRCD 8c36fd5
chore(eve): gitignore build artifacts in example
HugoRCD ca5ac59
docs(eve): flatten nav, svg logo, lowercase branding, and fix geist o…
HugoRCD afd8759
Merge remote-tracking branch 'origin/main' into feat/eve-integration
HugoRCD 10a174f
adress code rabbit feedbacks
HugoRCD ca0208b
fix(eve): guard ensureInit when logger is already configured
HugoRCD 57273b3
feat(eve): simplify wide events and harden session memory
HugoRCD 4150973
feat(eve): add split demo UI with fslog panel and fix useLogger in tools
HugoRCD b0dcea5
fix(eve): address CodeRabbit review
HugoRCD File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "evlog": minor | ||
| --- | ||
|
|
||
| Add `evlog/eve` with `defineEvlogHook()` for one wide event per agent turn and `useLogger()` in tools (AsyncLocalStorage on `turn.started`; pass `ctx` only when ALS is unavailable) — full drain, enrich, and tail-sampling pipeline. Tracks tool durations (including post-approval resumes), session context carry-over with LRU eviction (`maxSessions`), slim `eve.phase` / `eve.sessionTurns` fields, and compact HITL `approval`. The turn logger is bound via AsyncLocalStorage on `turn.started`; pass `ctx` when ALS is unavailable. Turn state is shared via `globalThis` when eve bundles hooks and tools separately. `finalizeAudit()` no longer crashes on partial `audit` objects missing `actor` fields. Fixes `_auditForceKeep` leaking on force-kept events and skips Nitro runtime probes on Next.js hosts. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "evlog": patch | ||
| --- | ||
|
|
||
| Fix stream server mis-detection when co-located with eve dev: return 404 (not SSE 200) for non-root GET paths, and re-bind the turn logger on `actions.requested` so tool handlers can resolve `useLogger()`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,7 @@ jobs: | |
| docs | ||
| dx | ||
| elysia | ||
| eve | ||
| express | ||
| fastify | ||
| fs | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,263 @@ | ||
| --- | ||
| title: eve | ||
| description: Export one evlog wide event per eve agent turn — token usage, tool executions, business context, drains, enrichers, and tail sampling alongside Agent Runs and OpenTelemetry. | ||
| navigation: | ||
| title: eve | ||
| icon: i-custom-eve | ||
| links: | ||
| - label: Source Code | ||
| icon: i-simple-icons-github | ||
| to: https://github.com/HugoRCD/evlog/tree/main/examples/eve | ||
| color: neutral | ||
| variant: subtle | ||
| --- | ||
|
|
||
| [eve](https://eve.dev/docs/introduction) ships built-in observability: **Agent Runs** on Vercel and optional **OpenTelemetry** spans via `agent/instrumentation.ts`. `evlog/eve` adds a third layer — **exportable wide events** per turn with your full evlog pipeline (drains, enrichers, tail sampling, audit). | ||
|
|
||
| ::callout{icon="i-lucide-info" color="info"} | ||
| eve is currently in beta; hook and stream event shapes may change before GA. Pin `eve` and `evlog` versions in production agents. | ||
| :: | ||
|
|
||
| ::prompt | ||
| --- | ||
| description: Add evlog wide events to my eve agent | ||
| icon: i-custom-eve | ||
| actions: | ||
| - copy | ||
| - cursor | ||
| - windsurf | ||
| --- | ||
|
|
||
| Add evlog wide events to my eve agent. | ||
|
|
||
| - Install evlog: pnpm add evlog | ||
| - Create agent/hooks/evlog.ts with defineEvlogHook from 'evlog/eve' | ||
| - Pass drain, enrich, and keep options (same as HTTP middleware integrations) | ||
| - In tools, import useLogger from 'evlog/eve' and call useLogger() inside execute() — the turn logger is bound via AsyncLocalStorage when defineEvlogHook() is registered; pass ctx only if ALS is unavailable in your runtime | ||
| - User message content is redacted by default (redactMessage: true); set false only after reviewing PII policy | ||
| - Keep eve Agent Runs / OTel instrumentation — evlog/eve is additive | ||
|
|
||
| Docs: https://www.evlog.dev/use-cases/eve | ||
| Adapters: https://www.evlog.dev/integrate/adapters/overview | ||
|
|
||
| :: | ||
|
|
||
| ## When to use what | ||
|
|
||
| | Need | Use | | ||
| | --- | --- | | ||
| | Debug a session in Vercel | eve **Agent Runs** (automatic) | | ||
| | Span-level traces in Datadog / Honeycomb | eve **`agent/instrumentation.ts`** + OTel exporter | | ||
| | Wide events to Axiom / Better Stack / FS, billing, audit, tail sampling | **`evlog/eve`** | | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ### 1. Install | ||
|
|
||
| ::code-group | ||
| ```bash [pnpm] | ||
| pnpm add evlog eve | ||
| ``` | ||
| ```bash [bun] | ||
| bun add evlog eve | ||
| ``` | ||
| ```bash [yarn] | ||
| yarn add evlog eve | ||
| ``` | ||
| ```bash [npm] | ||
| npm install evlog eve | ||
| ``` | ||
| :: | ||
|
|
||
| ### 2. Add the hook | ||
|
|
||
| Create `agent/hooks/evlog.ts`: | ||
|
|
||
| ```typescript [agent/hooks/evlog.ts] | ||
| import { defineEvlogHook } from 'evlog/eve' | ||
| import { createAxiomDrain } from 'evlog/axiom' | ||
|
|
||
| export default defineEvlogHook({ | ||
| init: { env: { service: 'my-agent' } }, | ||
| drain: createAxiomDrain(), | ||
| enrich: (ctx) => { | ||
| ctx.event.region = process.env.VERCEL_REGION | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| eve auto-discovers hook files under `agent/hooks/`. No HTTP middleware — the unit of work is an agent **turn**, not a request. | ||
|
|
||
| ### 3. Log business context from tools | ||
|
|
||
| `defineEvlogHook()` binds the turn logger via **AsyncLocalStorage** on `turn.started`. Inside tool `execute()` handlers, call `useLogger()` with no arguments — same ergonomics as `useLogger(event)` in Nuxt or Hono. | ||
|
|
||
| In the [example agent](https://github.com/HugoRCD/evlog/tree/main/examples/eve), support tools attach customer and order context as the agent works a refund: | ||
|
|
||
| ```typescript [agent/tools/lookup_order.ts] | ||
| import { defineTool } from 'eve/tools' | ||
| import { useLogger } from 'evlog/eve' | ||
| import { z } from 'zod' | ||
|
|
||
| export default defineTool({ | ||
| description: 'Look up an order by id.', | ||
| inputSchema: z.object({ orderId: z.string() }), | ||
| async execute({ orderId }) { | ||
| const order = await fetchOrder(orderId) | ||
| const log = useLogger() | ||
| log.set({ order: { id: order.id, amount: order.amount, status: order.status } }) | ||
| return order | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| ::callout{icon="i-lucide-plug" color="neutral"} | ||
| **Fallback:** if `useLogger()` throws outside a turn, pass eve tool `ctx`: `useLogger(ctx)`. This covers separate hook/tool bundles or runtimes where AsyncLocalStorage does not propagate. With a standard eve agent layout and `agent/hooks/evlog.ts` registered, you should not need it. | ||
| :: | ||
|
|
||
| ### 4. Next.js web chat (optional) | ||
|
|
||
| Wrap `next.config.ts` with `withEve()` from `eve/next` and use `useEveAgent()` from `eve/react` in your app. eve starts alongside `next dev` and proxies `/eve/v1/*` on the same origin — tool calls, approvals, and `ask_question` work out of the box. See the [example agent](https://github.com/HugoRCD/evlog/tree/main/examples/eve). | ||
|
|
||
| ## Wide event shape | ||
|
|
||
| Each completed turn emits one event: | ||
|
|
||
| ```json [Wide Event — turn awaiting approval] | ||
| { | ||
| "method": "EVE", | ||
| "path": "/sessions/sess_abc/turns/turn_0", | ||
| "status": 200, | ||
| "duration": "7.9s", | ||
| "service": "clearbill-support-agent", | ||
| "eve": { | ||
| "sessionId": "sess_abc", | ||
| "turnId": "turn_0", | ||
| "turnSequence": 0, | ||
| "phase": "awaiting-approval", | ||
| "sessionTurns": 1 | ||
| }, | ||
| "customer": { "slug": "acme-corp", "plan": "enterprise" }, | ||
| "order": { "id": "4821", "amount": 890, "currency": "USD" }, | ||
| "approval": { "status": "pending", "tool": "issue_refund" }, | ||
| "ai": { | ||
| "calls": 2, | ||
| "steps": 2, | ||
| "inputTokens": 11724, | ||
| "outputTokens": 282, | ||
| "finishReason": "tool-calls", | ||
| "tools": [ | ||
| { "name": "lookup_customer", "durationMs": 26, "success": true }, | ||
| { "name": "lookup_order", "durationMs": 13, "success": true } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| After approval, the next turn carries the outcome: | ||
|
|
||
| ```json [Wide Event — refund completed] | ||
| { | ||
| "path": "/sessions/sess_abc/turns/turn_1", | ||
| "eve": { | ||
| "sessionId": "sess_abc", | ||
| "turnId": "turn_1", | ||
| "turnSequence": 1, | ||
| "sessionTurns": 2 | ||
| }, | ||
| "refund": { "orderId": "4821", "amount": 890, "status": "refunded" }, | ||
| "audit": { "action": "refund.issued", "target": { "type": "order", "id": "4821" } }, | ||
| "approval": { "status": "approved", "tool": "issue_refund" }, | ||
| "ai": { | ||
| "tools": [{ "name": "issue_refund", "durationMs": 993, "success": true }], | ||
| "finishReason": "stop" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Token usage and tool executions are accumulated from eve stream events (`step.completed`, `actions.requested`, `action.result`). Business fields set via `useLogger()` carry across turns in the same session. Link turns in analytics with `eve.sessionId` + `eve.turnSequence` — each event stays self-contained. `eve.phase` is only set for non-routine endings (`awaiting-approval`, `rejected`, `failed`). | ||
|
|
||
| ## Production | ||
|
|
||
| Long-running eve agents should disable terminal pretty-printing and use a non-blocking drain: | ||
|
|
||
| ```typescript [agent/hooks/evlog.ts] | ||
| import { defineEvlogHook } from 'evlog/eve' | ||
| import { createAxiomDrain } from 'evlog/axiom' | ||
| import { createDrainPipeline } from 'evlog/pipeline' | ||
|
|
||
| const drain = createDrainPipeline({ batch: { size: 50, intervalMs: 5000 } })( | ||
| createAxiomDrain(), | ||
| ) | ||
|
|
||
| export default defineEvlogHook({ | ||
| init: { | ||
| env: { service: 'my-agent', environment: 'production' }, | ||
| pretty: false, | ||
| sampling: { rates: { info: 10 } }, | ||
| }, | ||
| drain, | ||
| maxSessions: 256, | ||
| }) | ||
| ``` | ||
|
|
||
| | Concern | Recommendation | | ||
| | --- | --- | | ||
| | Terminal output | `init.pretty: false` — pretty-print is for local dev only | | ||
| | Drain latency | Batch or async HTTP drains; never block the turn on I/O | | ||
| | Head sampling | `init.sampling.rates` — eve emits one event per turn, not per token | | ||
| | Memory | `maxSessions` (default `256`) evicts oldest idle session state | | ||
| | Load | Hook handlers are O(1) per stream event; cost is dominated by your drain | | ||
|
|
||
| ## Options | ||
|
|
||
| | Option | Description | | ||
| | --- | --- | | ||
| | `init` | Passed to `initLogger()` on first hook invocation | | ||
| | `drain` / `enrich` / `keep` / `plugins` | Same as HTTP integrations ([plugins](/extend/plugins)) | | ||
| | `redactMessage` | Default `true` — omits user message text from the wide event | | ||
| | `cost` / `model` | Optional token pricing (`ModelCost` from `evlog/ai`) → `ai.estimatedCost` | | ||
| | `maxSessions` | In-memory session cap for context carry-over (default `256`) | | ||
| | `include` / `exclude` | Route-style filters on turn paths (`/sessions/*/turns/*`) | | ||
|
|
||
| ## Tail sampling | ||
|
|
||
| Use `keep` to force-keep turns with failed tools or high token usage: | ||
|
|
||
| ```typescript [agent/hooks/evlog.ts] | ||
| export default defineEvlogHook({ | ||
| drain: createAxiomDrain(), | ||
| keep: (ctx) => { | ||
| const ai = ctx.context.ai as { | ||
| inputTokens?: number | ||
| outputTokens?: number | ||
| tools?: Array<{ success: boolean }> | ||
| } | undefined | ||
| const totalTokens = (ai?.inputTokens ?? 0) + (ai?.outputTokens ?? 0) | ||
| if (totalTokens > 10_000) ctx.shouldKeep = true | ||
| if (ai?.tools?.some(t => !t.success)) ctx.shouldKeep = true | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| ## Audit logs | ||
|
|
||
| Combine with [Audit Logs](/use-cases/audit/overview): register `auditEnricher()` via `init.plugins` or a global plugin, and call `log.audit()` inside tools when a human approval gate fires. Tool rejections surface on `action.result` with `status: "rejected"`. | ||
|
|
||
| ## Run locally | ||
|
|
||
| ```bash | ||
| git clone https://github.com/HugoRCD/evlog | ||
| cd evlog | ||
| pnpm install | ||
| pnpm run example:eve | ||
| ``` | ||
|
|
||
| The `examples/eve` project is a **support refund copilot** (Clearbill SaaS): lookup customer → lookup order → issue refund, with eve approvals when amount > $100. Run `pnpm run example:eve`, open **http://localhost:3000**, and click a starter prompt. | ||
|
|
||
| ## What to read next | ||
|
|
||
| - [eve hooks guide](https://eve.dev/docs/guides/hooks) — stream event vocabulary | ||
| - [eve instrumentation](https://eve.dev/docs/guides/instrumentation) — OTel spans (complementary) | ||
| - [AI SDK use case](/use-cases/ai-sdk/overview) — when you own the model loop directly | ||
| - [Adapters overview](/integrate/adapters/overview) — drain destinations | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| .eve | ||
| .evlog | ||
| .next | ||
| .output | ||
| *.tsbuildinfo |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.