Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/billed-usage-unit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@tanstack/ai-event-client': minor
'@tanstack/ai': minor
'@tanstack/ai-fal': minor
'@tanstack/ai-grok': minor
'@tanstack/ai-openai': minor
---

Add a self-describing `billed` field to `TokenUsage` so non-token billed quantities carry the unit they are counted in (#816). `usage.billed` is `{ quantity, unit }` with a `BillingUnit` union (`'seconds'`, `'units'`, `'images'`, ... open-ended), replacing the guesswork previously needed to interpret the bare `unitsBilled` / `durationSeconds` counts — those two fields are now deprecated but still populated for backward compatibility. The fal adapters report `{ quantity, unit: 'units' }`, Grok video `{ quantity, unit: 'seconds' }`, and the OpenAI/Grok duration-billed transcription paths `{ quantity, unit: 'seconds' }`. `otelMiddleware` emits the pair as `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` span attributes.
2 changes: 1 addition & 1 deletion docs/adapters/grok.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ const { jobId } = await generateVideo({

Like the Grok Imagine image models, sizing is aspect-ratio based: the `size` option takes an `aspectRatio_resolution` template. Supported aspect ratios are `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `3:2`, and `2:3`; supported resolutions are `480p`, `720p`, and `1080p` (e.g. `"9:16_1080p"`). The resolution suffix is optional.

When the job completes, the adapter reports usage on the result: `usage.unitsBilled` carries the billed seconds of video and `usage.cost` the exact cost in USD, both as returned by the xAI API.
When the job completes, the adapter reports usage on the result: `usage.billed` carries the billed seconds of video (`{ quantity, unit: 'seconds' }`) and `usage.cost` the exact cost in USD, both as returned by the xAI API.

See [Video Generation](../media/video-generation) for the full jobs/polling flow, streaming mode, and the `useGenerateVideo` hook.

Expand Down
11 changes: 8 additions & 3 deletions docs/advanced/otel.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,10 @@ Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the sam
| root / iteration | `gen_ai.usage.cache_read.input_tokens` | cached prompt tokens, when reported |
| root / iteration | `gen_ai.usage.cache_creation.input_tokens` | cache-write prompt tokens, when reported |
| root / iteration | `gen_ai.usage.reasoning.output_tokens` | reasoning/thinking tokens, when reported |
| root / iteration | `tanstack.ai.usage.duration_seconds` | duration-based billing (e.g. transcription), when reported |
| root / iteration | `tanstack.ai.usage.billed_quantity` | non-token billed quantity, when reported |
| root / iteration | `tanstack.ai.usage.billed_unit` | unit of the billed quantity (`seconds`, `units`, ...) |
| root / iteration | `tanstack.ai.usage.duration_seconds` | deprecated duration count; read `billed_quantity`/`billed_unit` instead |
| root / iteration | `tanstack.ai.usage.units_billed` | deprecated bare unit count; read `billed_quantity`/`billed_unit` instead |
| root / iteration | `tanstack.ai.usage.upstream_cost` | gateway upstream cost (e.g. OpenRouter), when reported |
| root / iteration | `tanstack.ai.usage.upstream_input_cost` | upstream input cost split, when reported |
| root / iteration | `tanstack.ai.usage.upstream_output_cost` | upstream output cost split, when reported |
Expand All @@ -90,7 +93,9 @@ Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the sam
| tool | `gen_ai.tool.type` | `function` |
| tool | `tanstack.ai.tool.outcome` | `success` / `error` |

Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (duration-based billing, the upstream cost split) are TanStack-namespaced.
Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (the billed quantity/unit pair, the upstream cost split, and the deprecated bare counts) are TanStack-namespaced.

For non-token billing (seconds of video or transcription, fal's endpoint units, ...), `tanstack.ai.usage.billed_quantity` and `tanstack.ai.usage.billed_unit` are emitted as a pair from `usage.billed`, so backends can label and aggregate media usage without knowing the provider. The deprecated `duration_seconds` / `units_billed` attributes carry the same quantities without the unit and remain emitted for backward compatibility.

### Metrics

Expand Down Expand Up @@ -231,7 +236,7 @@ Each media call produces one `CLIENT` span tagged with the activity's `gen_ai.op
| `generateSpeech` | `text_to_speech` |
| `generateTranscription` | `transcription` |

The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including `tanstack.ai.usage.units_billed` for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked.
The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including the `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` pair for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked.

`otelMiddleware` applies the same `spanNameFormatter`, `attributeEnricher`, `onBeforeSpanStart`, and `onSpanEnd` extension points to media spans — the span info is discriminated by `kind`, where media spans report `kind: 'generation'`. For a custom backend, implement the base `GenerationMiddleware` contract directly; its hooks (`onStart` / `onUsage` / `onFinish` / `onAbort` / `onError`) receive the `GenerationMiddlewareContext` and fire for every activity, chat included. The `GenerationMiddleware` types are exported from the package root, while the `otelMiddleware` value lives on the `@tanstack/ai/middlewares/otel` subpath so importing `@tanstack/ai` never requires the optional `@opentelemetry/api` peer.

Expand Down
10 changes: 5 additions & 5 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -271,19 +271,19 @@
"label": "Audio Generation",
"to": "media/audio-generation",
"addedAt": "2026-04-23",
"updatedAt": "2026-06-08"
"updatedAt": "2026-07-04"
},
{
"label": "Image Generation",
"to": "media/image-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-01"
"updatedAt": "2026-07-04"
},
{
"label": "Video Generation",
"to": "media/video-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-01"
"updatedAt": "2026-07-04"
},
{
"label": "Generation Hooks",
Expand All @@ -310,7 +310,7 @@
"label": "OpenTelemetry",
"to": "advanced/otel",
"addedAt": "2026-05-08",
"updatedAt": "2026-06-17"
"updatedAt": "2026-07-04"
}
]
},
Expand Down Expand Up @@ -527,7 +527,7 @@
"label": "Grok (xAI)",
"to": "adapters/grok",
"addedAt": "2026-04-15",
"updatedAt": "2026-06-24"
"updatedAt": "2026-07-04"
},
{
"label": "Groq",
Expand Down
7 changes: 4 additions & 3 deletions docs/media/audio-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,10 @@ interface AudioGenerationResult {
}
// Canonical TokenUsage (same shape as chat), present when the provider
// reports it (e.g. Gemini Lyria via generateContent). Usage-billed providers
// (fal) instead surface `usage.unitsBilled` — the real billed quantity read
// from fal's `x-fal-billable-units` result header. Multiply by the endpoint's
// unit price (fal pricing API) for the exact cost.
// (fal) instead surface `usage.billed` ({ quantity, unit: 'units' }) — the
// real billed quantity read from fal's `x-fal-billable-units` result header.
// Multiply the quantity by the endpoint's unit price (fal pricing API) for
// the exact cost.
usage?: TokenUsage
}
```
Expand Down
15 changes: 8 additions & 7 deletions docs/media/image-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,7 +396,7 @@ interface ImageGenerationResult {
// Canonical TokenUsage (same shape as chat). Token-billed models also surface
// a per-modality breakdown on `promptTokensDetails` (e.g. text vs image input
// tokens for gpt-image-1). Usage-billed providers (fal) instead surface
// `usage.unitsBilled` — see the note below.
// `usage.billed` ({ quantity, unit }) — see the note below.
usage?: TokenUsage
}

Expand All @@ -408,9 +408,9 @@ interface GeneratedImage {
```

> **Cost tracking (fal):** fal bills by usage-based units rather than tokens. The
> fal image adapter surfaces the real billed quantity as `usage.unitsBilled`
> (read from fal's `x-fal-billable-units` result header). Multiply it by the
> endpoint's unit price from
> fal image adapter surfaces the real billed quantity as `usage.billed` —
> `{ quantity, unit: 'units' }`, read from fal's `x-fal-billable-units` result
> header. Multiply the quantity by the endpoint's unit price from
> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` for the exact cost —
> no `fetch` interceptor needed.

Expand All @@ -424,9 +424,10 @@ const result = await generateImage({
prompt: 'a serene mountain lake',
})

if (result.usage?.unitsBilled != null) {
const cost = result.usage.unitsBilled * unitPrice // unitPrice from fal pricing API
console.log(`Billed ${result.usage.unitsBilled} units (~$${cost})`)
if (result.usage?.billed) {
const { quantity, unit } = result.usage.billed
const cost = quantity * unitPrice // unitPrice from fal pricing API
console.log(`Billed ${quantity} ${unit} (~$${cost})`)
}
```

Expand Down
21 changes: 13 additions & 8 deletions docs/media/video-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,7 @@ adapter.snapDuration(2.5) // 3 — clamped/rounded into range
adapter.snapDuration(99) // 15
```

Generated clips include an audio track. When the job completes, the adapter reports `usage.unitsBilled` (billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result.
Generated clips include an audio track. When the job completes, the adapter reports `usage.billed` (`{ quantity, unit: 'seconds' }` — billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result.

## Response Types

Expand Down Expand Up @@ -660,19 +660,24 @@ interface VideoUrlResult {
jobId: string
url: string // URL to download/stream the video
expiresAt?: Date // When the URL expires
// Usage for the completed generation, when the adapter reports it. fal
// populates `usage.unitsBilled` from its `x-fal-billable-units` header.
// Usage for the completed generation, when the adapter reports it. The
// billed quantity is self-describing: fal reports
// `usage.billed = { quantity, unit: 'units' }` (from its
// `x-fal-billable-units` header), Grok Imagine reports
// `{ quantity, unit: 'seconds' }`.
usage?: TokenUsage
}
```

> **Cost tracking (fal):** fal bills media generation by usage-based units
> rather than tokens. The fal adapters surface the real billed quantity as
> `usage.unitsBilled` (denominated in the endpoint's priced unit). Combine it
> with the endpoint's unit price from
> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to compute the exact
> cost (`unitsBilled * unitPrice`). The same `usage.unitsBilled` is surfaced
> on image, audio, speech, and transcription results.
> `usage.billed` — `{ quantity, unit: 'units' }`, where `'units'` marks fal's
> endpoint-defined priced unit. Combine the quantity with the endpoint's unit
> price from `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to
> compute the exact cost (`billed.quantity * unitPrice`). The same
> `usage.billed` is surfaced on image, audio, speech, and transcription
> results. (The deprecated bare count `usage.unitsBilled` is still populated
> for backward compatibility.)

## Model Variants

Expand Down
10 changes: 5 additions & 5 deletions examples/ts-react-media/src/components/ImageGenerator.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -304,12 +304,12 @@ export default function ImageGenerator({
className="w-full h-auto"
/>
</div>
{modelResult.result.usage?.unitsBilled != null && (
{modelResult.result.usage?.billed && (
<p className="text-xs text-gray-500">
Billed {modelResult.result.usage.unitsBilled} fal unit
{modelResult.result.usage.unitsBilled === 1
? ''
: 's'}{' '}
Billed {modelResult.result.usage.billed.quantity}{' '}
{modelResult.result.usage.billed.unit === 'units'
? `fal unit${modelResult.result.usage.billed.quantity === 1 ? '' : 's'}`
: modelResult.result.usage.billed.unit}{' '}
— multiply by the endpoint unit price for USD cost
</p>
)}
Expand Down
34 changes: 26 additions & 8 deletions examples/ts-react-media/src/components/VideoGenerator.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { useEffect, useRef, useState } from 'react'
import { Film, Loader2, Shuffle, Upload, X } from 'lucide-react'
import type { BilledUsage } from '@tanstack/ai'
import type { VideoMode } from '@/lib/models'

import {
Expand All @@ -21,9 +22,25 @@ type JobState =
model: string
progress?: number | undefined
}
| { status: 'completed'; url: string; unitsBilled?: number; cost?: number }
| { status: 'completed'; url: string; billed?: BilledUsage; cost?: number }
| { status: 'error'; message: string }

/**
* Human label for a billed quantity, driven by the unit the adapter reported —
* no guessing from provider identity or cost presence.
*/
function describeBilled({ quantity, unit }: BilledUsage): string {
const plural = quantity === 1 ? '' : 's'
switch (unit) {
case 'seconds':
return `${quantity} second${plural} of video`
case 'units':
return `${quantity} fal unit${plural}`
default:
return `${quantity} ${unit}`
}
}

interface VideoGeneratorProps {
initialImageUrl?: string | null
}
Expand Down Expand Up @@ -98,7 +115,7 @@ export default function VideoGenerator({
[model]: {
status: 'completed',
url: url,
unitsBilled: urlResult.usage?.unitsBilled,
billed: urlResult.usage?.billed,
cost: urlResult.usage?.cost,
},
}))
Expand Down Expand Up @@ -424,16 +441,17 @@ export default function VideoGenerator({
{state.cost != null ? (
<p className="text-xs text-gray-500">
Billed ${state.cost.toFixed(3)}
{state.unitsBilled != null
? ` for ${state.unitsBilled} second${state.unitsBilled === 1 ? '' : 's'} of video`
{state.billed
? ` for ${describeBilled(state.billed)}`
: ''}
</p>
) : (
state.unitsBilled != null && (
state.billed && (
<p className="text-xs text-gray-500">
Billed {state.unitsBilled} fal unit
{state.unitsBilled === 1 ? '' : 's'} — multiply by the
endpoint unit price for USD cost
Billed {describeBilled(state.billed)}
{state.billed.unit === 'units'
? ' — multiply by the endpoint unit price for USD cost'
: ''}
</p>
)
)}
Expand Down
4 changes: 2 additions & 2 deletions examples/ts-react-media/src/lib/server-functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -252,8 +252,8 @@ export const createVideoJobFn = createServerFn({ method: 'POST' })
case 'grok-imagine-video': {
// Direct xAI Imagine API (XAI_API_KEY) — no fal in between. The base
// grok-imagine-video (v1.0) supports text-to-video; durations are
// 1-15 integer seconds. Completed jobs report usage.unitsBilled
// (billed seconds) and usage.cost (exact USD).
// 1-15 integer seconds. Completed jobs report usage.billed
// ({ quantity, unit: 'seconds' }) and usage.cost (exact USD).
return generateVideo({
adapter: grokVideo('grok-imagine-video'),
prompt: asTextPrompt(data.prompt),
Expand Down
Loading