Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions sdk/guides/agent-server/openai-gateway.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@
- `X-Session-API-Key: <key>`
- `Authorization: Bearer <key>`

OpenAI-compatible requests can also pass observability overrides for a new OpenHands conversation created behind the request:

<Note>
Observability override headers are available in OpenHands agent-server `v1.30.0` and later.
</Note>

- `X-OpenHands-Observability-Span-Name`: stable child span name to emit under the `conversation` root span
- `X-OpenHands-Observability-Tags`: comma-separated span tags
- `X-OpenHands-Observability-Metadata`: JSON object attached as trace metadata

Use `X-OpenHands-Observability-Span-Name` when an observability backend, such as Laminar signals, should route or classify traces by a specific span name. The root span remains named `conversation`.

## Prepare a Profile

OpenAI-compatible traffic is backed by an agent-server LLM profile. Create one with the native profile API first:
Expand Down Expand Up @@ -64,6 +76,9 @@
curl -i "$AGENT_SERVER_URL/v1/chat/completions" \
-H "Authorization: Bearer $SESSION_API_KEY" \
-H "Content-Type: application/json" \
-H "X-OpenHands-Observability-Span-Name: pr_review_evaluation" \
-H "X-OpenHands-Observability-Tags: pr-review,evaluation" \
-H 'X-OpenHands-Observability-Metadata: {"repo":"OpenHands/software-agent-sdk"}' \
-d "{
\"model\": \"$OPENHANDS_MODEL\",
\"messages\": [
Expand Down Expand Up @@ -99,6 +114,11 @@
messages=[
{"role": "user", "content": "Summarize this repository."},
],
extra_headers={
"X-OpenHands-Observability-Span-Name": "repo_summary",
"X-OpenHands-Observability-Tags": "repo-summary,docs",
"X-OpenHands-Observability-Metadata": '{"source":"docs-example"}',
},
)
completion = response.parse()
conversation_id = response.headers["X-OpenHands-ServerConversation-ID"]
Expand Down Expand Up @@ -152,7 +172,7 @@
</Tab>
<Tab title="Chat UIs">

For Open WebUI, LibreChat, Chatbot UI, and similar OpenAI-compatible frontends, configure a custom OpenAI provider with:

Check warning on line 175 in sdk/guides/agent-server/openai-gateway.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/agent-server/openai-gateway.mdx#L175

Did you really mean 'Chatbot'?

- **Base URL**: `https://YOUR_AGENT_SERVER/v1`
- **API key**: your agent-server session API key
Expand Down
25 changes: 25 additions & 0 deletions sdk/guides/observability.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Observability & Tracing
description: Enable OpenTelemetry tracing to monitor and debug your agent's execution with tools like Laminar, MLflow, Honeycomb, or any OTLP-compatible backend.

Check warning on line 3 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L3

Did you really mean 'MLflow'?
---

> A full setup example is available [below](#example-full-setup).
Expand All @@ -12,7 +12,7 @@
- **[Laminar](https://laminar.sh/)** - AI-focused observability with trace inspection, signals, and browser session replay
- **[MLflow](https://mlflow.org/)** - Open-source AI platform with tracing, evaluation, and LLM governance
- **[Honeycomb](https://www.honeycomb.io/)** - High-performance distributed tracing
- **Any OTLP-compatible backend** - Including Jaeger, Datadog, New Relic, and more

Check warning on line 15 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L15

Did you really mean 'Datadog'?

The SDK automatically traces:
- Agent execution steps
Expand All @@ -37,7 +37,7 @@
That's it. Run your agent code normally and traces will be sent to Laminar automatically.

<Note>
For Laminar-specific walkthroughs, see the official docs for [OpenHands SDK tracing](https://laminar.sh/docs/tracing/integrations/openhands-sdk), [session replay for browser agents](https://laminar.sh/docs/tracing/browser-agent-observability), [viewing traces](https://laminar.sh/docs/platform/viewing-traces), and [signals](https://laminar.sh/docs/signals/introduction).

Check warning on line 40 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L40

Did you really mean 'walkthroughs'?
</Note>

For **self-hosted Laminar** deployments, you can also configure custom ports:
Expand All @@ -48,13 +48,13 @@
export LMNR_GRPC_PORT=8001
```

If you need help deciding between Laminar Cloud and self-hosted Laminar, see Laminar's official [hosting options](https://laminar.sh/docs/hosting-options).

Check warning on line 51 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L51

Did you really mean 'Laminar's'?

### Why use Laminar with OpenHands?

Laminar is especially useful when you want to understand how an agent behaved across one run or across many runs:

- Inspect a single run in transcript, tree, or timeline views to see prompts, tool calls, outputs, and nested agent activity. See Laminar's guide to [viewing traces](https://laminar.sh/docs/platform/viewing-traces).

Check warning on line 57 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L57

Did you really mean 'Laminar's'?
- Watch browser automation alongside trace spans with [session replay for browser agents](https://laminar.sh/docs/tracing/browser-agent-observability).
- Define [signals](https://laminar.sh/docs/signals/introduction) to classify failures, user friction, or success patterns across many traces.
- Keep each OpenHands conversation grouped under a single session ID so multi-turn debugging is easier.
Expand Down Expand Up @@ -140,6 +140,31 @@

In `tool.execute`, the tool calls are traced individually, such as `bash`, `file_editor`, or `task_tracker`.

### Named Child Spans for Signals

<Note>
`observability_span_name` is available in OpenHands SDK `v1.30.0` and later.
</Note>

By default, each conversation trace starts with a root span named `conversation`. Keep that root span name stable for trace grouping. When you need an observability backend to route or classify a run by span name, set `observability_span_name` on the conversation. The SDK emits that value as an immediate child span under the `conversation` root.

This is useful for Laminar signals or other backend rules that match a specific span name:

```python icon="python" wrap
conversation = Conversation(
agent=agent,
workspace=".",
observability_span_name="pr_review_evaluation",
observability_tags=["pr-review", "evaluation"],
observability_metadata={
"repo": "OpenHands/software-agent-sdk",
"workflow": "nightly-eval",
},
)
```

Use stable, low-cardinality names such as `pr_review_evaluation`, `support_triage`, or `release_note_check`. Span names must be non-empty strings up to 128 characters and may contain letters, numbers, dots, underscores, colons, slashes, and hyphens.

## Configuration Reference

### Environment Variables
Expand Down Expand Up @@ -175,8 +200,8 @@

The SDK supports both HTTP and gRPC protocols:

- **`http/protobuf`** or **`otlp_http`** - HTTP with protobuf encoding (recommended for most backends)

Check warning on line 203 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L203

Did you really mean 'protobuf'?
- **`grpc`** or **`otlp_grpc`** - gRPC with protobuf encoding (use only if your backend supports gRPC)

Check warning on line 204 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L204

Did you really mean 'protobuf'?

## Platform-Specific Configuration

Expand Down Expand Up @@ -212,24 +237,24 @@
6. Save that key as the **Laminar Project API Key** in the Admin Console.
7. Redeploy, then start a conversation in OpenHands.

In OHE, environment variables with `LMNR_` and `LLM_` prefixes are automatically forwarded to the SDK runtime. That makes it possible to configure Laminar endpoint settings such as `LMNR_BASE_URL`, `LMNR_PROJECT_API_KEY`, and `LMNR_FORCE_HTTP`, as well as the LLM that powers Laminar's own AI features (chat-with-trace, SQL-with-AI, and [signals](https://laminar.sh/docs/signals/introduction)) via `LLM_PROVIDER`, `LLM_BASE_URL`, and `LLM_MODEL_SMALL|MEDIUM|LARGE`.

Check warning on line 240 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L240

Did you really mean 'Laminar's'?

`LLM_PROVIDER` accepts `gemini` (Laminar's default), `openai`, or `bedrock`. Set it to `openai` whenever you point `LLM_BASE_URL` at an OpenAI-compatible gateway (for example LiteLLM, OpenRouter, or vLLM), not just the public OpenAI API. For the full list of supported values, see Laminar's official [self-hosting configuration reference](https://laminar.sh/docs/self-hosting/configuration).

Check warning on line 242 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L242

Did you really mean 'Laminar's'?

Check warning on line 242 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L242

Did you really mean 'Laminar's'?

For the full OHE flow with screenshots and configuration examples, see [Analytics in OpenHands Enterprise](/enterprise/analytics).

### MLflow Setup

Check warning on line 246 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L246

Did you really mean 'MLflow'?

[MLflow](https://mlflow.org/) is an open-source AI platform that accepts OpenTelemetry traces out of the box, alongside evaluation and LLM governance capabilities.

1. Start your MLflow tracking server:

Check warning on line 250 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L250

Did you really mean 'MLflow'?

```bash icon="terminal" wrap
uvx mlflow server
```

<Note>
For other deployment options (pip, Docker Compose, etc.), see [Set Up MLflow Server](https://mlflow.org/docs/latest/genai/getting-started/connect-environment/).

Check warning on line 257 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L257

Did you really mean 'MLflow'?
</Note>

2. Configure the environment variables:
Expand All @@ -240,7 +265,7 @@
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/protobuf"
```

Navigate to the MLflow UI (for example, `http://localhost:5000`), select the experiment, and open the **Traces** tab to view the recorded traces.

Check warning on line 268 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L268

Did you really mean 'MLflow'?

### Honeycomb Setup

Expand Down Expand Up @@ -335,7 +360,7 @@

4. **Validate headers**: Check that authentication headers are properly URL-encoded.

For Laminar-specific troubleshooting, see Laminar's official [tracing troubleshooting guide](https://laminar.sh/docs/tracing/troubleshooting).

Check warning on line 363 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L363

Did you really mean 'Laminar's'?

## Troubleshooting

Expand Down Expand Up @@ -364,7 +389,7 @@

**Solutions**:
- Tracing has minimal overhead when properly configured
- Disable tracing in development by unsetting environment variables

Check warning on line 392 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L392

Did you really mean 'unsetting'?
- Use asynchronous exporters (default in most OTLP configurations)

## Example: Full Setup
Expand Down Expand Up @@ -429,5 +454,5 @@

- **[Analytics in OpenHands Enterprise](/enterprise/analytics)** - Deploy Laminar inside OHE and send conversation traces automatically
- **[Metrics Tracking](/sdk/guides/metrics)** - Monitor token usage and costs alongside traces
- **[LLM Registry](/sdk/guides/llm-registry)** - Track multiple LLMs used in your application

Check warning on line 457 in sdk/guides/observability.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

sdk/guides/observability.mdx#L457

Did you really mean 'LLMs'?
- **[Security](/sdk/guides/security)** - Add security validation to your traced agent executions
Loading