-
Notifications
You must be signed in to change notification settings - Fork 47
feat: add evlog/eve integration
#399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
7868bc1
7fb6b82
0a2538f
8c36fd5
ca5ac59
afd8759
10a174f
ca0208b
57273b3
4150973
b0dcea5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "evlog": minor | ||
| --- | ||
|
|
||
| Add `evlog/eve` with `defineEvlogHook()` for one wide event per agent turn and `useTurnLogger(ctx)` in tools — full drain, enrich, and tail-sampling pipeline. Turn state is shared via `globalThis` when eve bundles hooks and tools separately. `finalizeAudit()` no longer crashes on partial `audit` objects missing `actor` fields. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,7 @@ jobs: | |
| docs | ||
| dx | ||
| elysia | ||
| eve | ||
| express | ||
| fastify | ||
| fs | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,201 @@ | ||
| --- | ||
| title: eve | ||
| description: Export one evlog wide event per eve agent turn — token usage, tool executions, business context, drains, enrichers, and tail sampling alongside Agent Runs and OpenTelemetry. | ||
| navigation: | ||
| title: eve | ||
| icon: i-custom-eve | ||
| links: | ||
| - label: Source Code | ||
| icon: i-simple-icons-github | ||
| to: https://github.com/HugoRCD/evlog/tree/main/examples/eve | ||
| color: neutral | ||
| variant: subtle | ||
| --- | ||
|
|
||
| [eve](https://eve.dev/docs/introduction) ships built-in observability: **Agent Runs** on Vercel and optional **OpenTelemetry** spans via `agent/instrumentation.ts`. `evlog/eve` adds a third layer — **exportable wide events** per turn with your full evlog pipeline (drains, enrichers, tail sampling, audit). | ||
|
|
||
| ::callout{icon="i-lucide-info" color="info"} | ||
| eve is currently in beta; hook and stream event shapes may change before GA. Pin `eve` and `evlog` versions in production agents. | ||
| :: | ||
|
|
||
| ::prompt | ||
| --- | ||
| description: Add evlog wide events to my eve agent | ||
| icon: i-custom-eve | ||
| actions: | ||
| - copy | ||
| - cursor | ||
| - windsurf | ||
| --- | ||
|
|
||
| Add evlog wide events to my eve agent. | ||
|
|
||
| - Install evlog: pnpm add evlog | ||
| - Create agent/hooks/evlog.ts with defineEvlogHook from 'evlog/eve' | ||
| - Pass drain, enrich, and keep options (same as HTTP middleware integrations) | ||
| - In tools, import useTurnLogger from 'evlog/eve' and call useTurnLogger(ctx) inside execute() | ||
| - User message content is redacted by default (redactMessage: true); set false only after reviewing PII policy | ||
| - Keep eve Agent Runs / OTel instrumentation — evlog/eve is additive | ||
|
|
||
| Docs: https://www.evlog.dev/use-cases/eve | ||
| Adapters: https://www.evlog.dev/integrate/adapters/overview | ||
|
|
||
| :: | ||
|
|
||
| ## When to use what | ||
|
|
||
| | Need | Use | | ||
| | --- | --- | | ||
| | Debug a session in Vercel | eve **Agent Runs** (automatic) | | ||
| | Span-level traces in Datadog / Honeycomb | eve **`agent/instrumentation.ts`** + OTel exporter | | ||
| | Wide events to Axiom / Better Stack / FS, billing, audit, tail sampling | **`evlog/eve`** | | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ### 1. Install | ||
|
|
||
| ::code-group | ||
| ```bash [pnpm] | ||
| pnpm add evlog eve | ||
| ``` | ||
| ```bash [bun] | ||
| bun add evlog eve | ||
| ``` | ||
| ```bash [yarn] | ||
| yarn add evlog eve | ||
| ``` | ||
| ```bash [npm] | ||
| npm install evlog eve | ||
| ``` | ||
| :: | ||
|
|
||
| ### 2. Add the hook | ||
|
|
||
| Create `agent/hooks/evlog.ts`: | ||
|
|
||
| ```typescript [agent/hooks/evlog.ts] | ||
| import { defineEvlogHook } from 'evlog/eve' | ||
| import { createAxiomDrain } from 'evlog/axiom' | ||
|
|
||
| export default defineEvlogHook({ | ||
| init: { env: { service: 'my-agent' } }, | ||
| drain: createAxiomDrain(), | ||
| enrich: (ctx) => { | ||
| ctx.event.region = process.env.VERCEL_REGION | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| eve auto-discovers hook files under `agent/hooks/`. No HTTP middleware — the unit of work is an agent **turn**, not a request. | ||
|
|
||
| ### 3. Log business context from tools | ||
|
|
||
| In the [example agent](https://github.com/HugoRCD/evlog/tree/main/examples/eve), support tools attach customer and order context as the agent works a refund: | ||
|
|
||
| ```typescript [agent/tools/lookup_order.ts] | ||
| import { defineTool } from 'eve/tools' | ||
| import { useTurnLogger } from 'evlog/eve' | ||
| import { z } from 'zod' | ||
|
|
||
| export default defineTool({ | ||
| description: 'Look up an order by id.', | ||
| inputSchema: z.object({ orderId: z.string() }), | ||
| async execute({ orderId }, ctx) { | ||
| const order = await fetchOrder(orderId) | ||
| const log = useTurnLogger(ctx) | ||
| log.set({ order: { id: order.id, amount: order.amount, status: order.status } }) | ||
| return order | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| ### 4. Next.js web chat (optional) | ||
|
|
||
| Wrap `next.config.ts` with `withEve()` from `eve/next` and use `useEveAgent()` from `eve/react` in your app. eve starts alongside `next dev` and proxies `/eve/v1/*` on the same origin — tool calls, approvals, and `ask_question` work out of the box. See the [example agent](https://github.com/HugoRCD/evlog/tree/main/examples/eve). | ||
|
|
||
| ## Wide event shape | ||
|
|
||
| Each completed turn emits one event: | ||
|
|
||
| ```json [Wide Event — support refund turn] | ||
| { | ||
| "method": "EVE", | ||
| "path": "/sessions/sess_abc/turns/turn_0", | ||
| "status": 200, | ||
| "duration": "4.2s", | ||
| "service": "clearbill-support-agent", | ||
| "eve": { | ||
| "sessionId": "sess_abc", | ||
| "turnId": "turn_0", | ||
| "turnSequence": 0 | ||
| }, | ||
| "customer": { "slug": "acme-corp", "plan": "enterprise", "mrr": 2400 }, | ||
| "order": { "id": "4821", "amount": 890, "currency": "USD" }, | ||
| "refund": { "orderId": "4821", "amount": 890, "requiresApproval": true }, | ||
| "audit": { "action": "refund.issued", "actor": { "type": "agent", "id": "…" }, "target": { "type": "order", "id": "4821" } }, | ||
| "ai": { | ||
| "calls": 3, | ||
| "steps": 3, | ||
| "inputTokens": 3312, | ||
| "outputTokens": 814, | ||
| "tools": [ | ||
| { "name": "lookup_customer", "success": true }, | ||
| { "name": "lookup_order", "success": true }, | ||
| { "name": "issue_refund", "success": true } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Token usage and tool executions are accumulated from eve stream events (`step.completed`, `action.result`). eve owns the model loop — you do not wrap the model with `evlog/ai`. | ||
|
|
||
| ## Options | ||
|
|
||
| | Option | Description | | ||
| | --- | --- | | ||
| | `init` | Passed to `initLogger()` on first hook invocation | | ||
| | `drain` / `enrich` / `keep` / `plugins` | Same as HTTP integrations ([plugins](/extend/plugins)) | | ||
| | `redactMessage` | Default `true` — omits user message text from the wide event | | ||
| | `include` / `exclude` | Route-style filters on turn paths (`/sessions/*/turns/*`) | | ||
|
|
||
| ## Tail sampling | ||
|
|
||
| Use `keep` to force-keep turns with failed tools or high token usage: | ||
|
|
||
| ```typescript [agent/hooks/evlog.ts] | ||
| export default defineEvlogHook({ | ||
| drain: createAxiomDrain(), | ||
| keep: (ctx) => { | ||
| const ai = ctx.context.ai as { | ||
| inputTokens?: number | ||
| outputTokens?: number | ||
| tools?: Array<{ success: boolean }> | ||
| } | undefined | ||
| const totalTokens = (ai?.inputTokens ?? 0) + (ai?.outputTokens ?? 0) | ||
| if (totalTokens > 10_000) ctx.shouldKeep = true | ||
| if (ai?.tools?.some(t => !t.success)) ctx.shouldKeep = true | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| ## Audit logs | ||
|
|
||
| Combine with [Audit Logs](/use-cases/audit/overview): register `auditEnricher()` via `init.plugins` or a global plugin, and call `log.audit()` inside tools when a human approval gate fires. Tool rejections surface on `action.result` with `status: "rejected"`. | ||
|
|
||
| ## Run locally | ||
|
|
||
| ```bash | ||
| git clone https://github.com/HugoRCD/evlog | ||
| cd evlog | ||
| pnpm install | ||
| pnpm run example:eve | ||
| ``` | ||
|
|
||
| The `examples/eve` project is a **support refund copilot** (Clearbill SaaS): lookup customer → lookup order → issue refund, with eve approvals when amount > $100. Run `pnpm run example:eve`, open **http://localhost:3000**, and click a starter prompt. | ||
|
|
||
| ## What to read next | ||
|
|
||
| - [eve hooks guide](https://eve.dev/docs/guides/hooks) — stream event vocabulary | ||
| - [eve instrumentation](https://eve.dev/docs/guides/instrumentation) — OTel spans (complementary) | ||
| - [AI SDK use case](/use-cases/ai-sdk/overview) — when you own the model loop directly | ||
| - [Adapters overview](/integrate/adapters/overview) — drain destinations | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| .eve | ||
| .next | ||
| .output | ||
| *.tsbuildinfo |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # evlog + eve — Support refund demo | ||
|
|
||
| A **support copilot** for a fake SaaS ("Clearbill"). A rep asks for a refund → the agent looks up the customer and order → issues the refund (with **human approval** when amount > $100). | ||
|
|
||
| Each completed turn emits **one evlog wide event** with the full story: customer, order, refund, audit trail, token usage, and tool outcomes. That's the point of the demo — not abstract `tenantId` fields. | ||
|
|
||
| ## The story | ||
|
|
||
| > "Acme Corp was double-charged on order #4821 ($890). Issue a refund." | ||
|
|
||
| 1. `lookup_customer` → attaches `customer.{id, plan, mrr}` to the wide event | ||
| 2. `lookup_order` → attaches `order.{id, amount, product}` | ||
| 3. `issue_refund` → **approval UI** (amount > $100) → attaches `refund` + `audit.refund.issued` | ||
|
|
||
| Finance opens PostHog (or your drain) and gets **one JSON object** per turn — no log scavenger hunt. | ||
|
|
||
| ### Example wide event (after approval) | ||
|
|
||
| ```json | ||
| { | ||
| "service": "clearbill-support-agent", | ||
| "method": "EVE", | ||
| "path": "/sessions/sess_…/turns/turn_…", | ||
| "status": 200, | ||
| "customer": { | ||
| "id": "cust_8f2a", | ||
| "slug": "acme-corp", | ||
| "name": "Acme Corp", | ||
| "plan": "enterprise", | ||
| "mrr": 2400 | ||
| }, | ||
| "order": { | ||
| "id": "4821", | ||
| "amount": 890, | ||
| "currency": "USD", | ||
| "product": "Enterprise — annual add-on (5 seats)" | ||
| }, | ||
| "refund": { | ||
| "orderId": "4821", | ||
| "amount": 890, | ||
| "reason": "Double charge", | ||
| "requiresApproval": true | ||
| }, | ||
| "audit": { | ||
| "action": "refund.issued", | ||
| "actor": { "type": "agent", "id": "clearbill-support-copilot" }, | ||
| "target": { "type": "order", "id": "4821" }, | ||
| "outcome": "success" | ||
| }, | ||
| "ai": { "calls": 3, "tools": […] } | ||
| } | ||
| ``` | ||
|
|
||
| Compare with order **#1102** ($49) — same flow, **no approval** (under the $100 threshold). | ||
|
|
||
| ## Run | ||
|
|
||
| From the monorepo root (Turbo runs `evlog` dev:prepare so `evlog/eve` resolves): | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| pnpm run example:eve | ||
| ``` | ||
|
|
||
| Open **http://localhost:3000** and click a starter prompt, or paste your own. | ||
|
|
||
| Requires `POSTHOG_API_KEY` in the repo root `.env` for the drain (events still log to stdout without it). | ||
|
|
||
| ## Files | ||
|
|
||
| | Path | Role | | ||
| | --- | --- | | ||
| | `agent/instructions.md` | Support copilot persona + fake CRM table | | ||
| | `agent/lib/support-data.ts` | Acme Corp & Startup Inc fake data | | ||
| | `agent/tools/lookup_*.ts` | CRM lookups → `useTurnLogger(ctx)` | | ||
| | `agent/tools/issue_refund.ts` | Refund + approval gate + audit fields | | ||
| | `agent/hooks/evlog.ts` | Wide event per turn, tail-keep on refunds/audit | | ||
|
|
||
| Docs: https://evlog.dev/use-cases/eve |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| import { defineAgent } from 'eve' | ||
|
|
||
| export default defineAgent({ | ||
| model: 'anthropic/claude-sonnet-4.6', | ||
| }) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| import { eveChannel } from 'eve/channels/eve' | ||
| import { localDev, placeholderAuth, vercelOidc } from 'eve/channels/auth' | ||
|
|
||
| export default eveChannel({ | ||
| auth: [ | ||
| // Open on localhost for `eve dev` and the REPL; ignored in production. | ||
| localDev(), | ||
| // Lets the eve TUI and your Vercel deployments reach the deployed agent. | ||
| vercelOidc(), | ||
| // Placeholder for production browser auth — swap for Auth.js, Clerk, or none() before deploy. | ||
| placeholderAuth(), | ||
|
Comment on lines
+5
to
+11
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== File outline ==\n'
ast-grep outline examples/eve/agent/channels/eve.ts --view expanded || true
printf '\n== File contents ==\n'
cat -n examples/eve/agent/channels/eve.ts
printf '\n== Search for placeholderAuth references ==\n'
rg -n "placeholderAuth\(|vercelOidc\(|localDev\(" examples/eve -S || true
printf '\n== Search for related docs/comments ==\n'
rg -n "placeholderAuth|public demo|browser requests in production|vercelOidc|localDev|eve dev|Next.js copilot" -S . || trueRepository: HugoRCD/evlog Length of output: 1815 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== Auth helper definitions ==\n'
rg -n "placeholderAuth|localDev|vercelOidc|none\(" . -S --glob '!**/node_modules/**' || true
printf '\n== Lint / formatting config ==\n'
rg -n '"semi"|semi:|`@typescript-eslint/semi`|prettier|biome|eslint' . -S --glob '!**/node_modules/**' --glob '!**/dist/**' --glob '!**/build/**' || true
printf '\n== Example docs mentioning deploy/public demo ==\n'
rg -n "public demo|deploy|production|browser requests|auth provider|none\(\)" examples README.md . -S --glob '!**/node_modules/**' || trueRepository: HugoRCD/evlog Length of output: 50369 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== Auth helper definition file(s) ==\n'
fd -a "auth.*" examples/eve packages . | sed -n '1,40p'
printf '\n== Relevant source slices ==\n'
for f in \
examples/eve/agent/channels/eve.ts \
eslint.config.mjs \
examples/eve/package.json
do
if [ -f "$f" ]; then
printf '\n--- %s ---\n' "$f"
cat -n "$f" | sed -n '1,220p'
fi
done
printf '\n== Semicolon-related lint config ==\n'
rg -n "semi|no-extra-semi|semi-style|`@typescript-eslint/semi`" eslint.config.mjs package.json examples/eve/package.json packages/evlog/package.json . -S --glob '!**/node_modules/**' || trueRepository: HugoRCD/evlog Length of output: 21559 🌐 Web query:
💡 Result: In the Eve Agent Framework, authentication for channels is managed via the Citations:
Replace 🤖 Prompt for AI Agents |
||
| ], | ||
| }) | ||
Uh oh!
There was an error while loading. Please reload this page.