Added standalone doc tests

.claude/skills/doc-tests/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: doc-test-guides
 description: Add executable doc tests to agentgateway documentation guides using the doc test framework. Use when the user asks to "add doc tests", "add tests to a guide", "add tests to a topic", mentions "YAMLTest", or is working on quickstart guides, standalone binary guides, or Kubernetes doc pages that should generate runnable scripts from code blocks.
-version: 1.0.0
+version: 1.2.0
 ---
 
 # Doc test guides skill
@@ -25,6 +25,7 @@ Use this skill when adding tests to documentation guides in the `agentgateway/we
    > **Critical**: Most Kubernetes topic pages (e.g. `content/docs/kubernetes/latest/resiliency/timeouts/request.md`) are thin wrappers that only contain `{{< reuse "agw-docs/pages/..." >}}`. **Always place doc-test blocks in the reuse file** (`assets/agw-docs/pages/...`), never in the content wrapper. This way both `latest` and `main` versions automatically inherit the tests — you only need to add them once.
 3. **Extractor** resolves `{{< reuse "..." >}}` from `assets/`, so the script is built from the expanded content. Reference the **content file** in `test:` sources; the extractor will follow reuse.
 4. **Block order**: Selected blocks are emitted in document order (by file and `start_line`). Hidden blocks (e.g. "start server in background") must appear *before* any visible block that depends on them (e.g. curl). The extractor sorts selected blocks by `(file_path, start_line)` so hidden blocks are not deferred to the end.
+5. **Byte-identical blocks are silently dropped**: `build_script()` in `scripts/doc_test_extract.py` keeps a `seen` set of block contents and skips any block whose content (after stripping leading and trailing newlines) exactly matches an earlier selected block. Only the **first** copy reaches the generated script — there is no warning. See "Repeated commands across sections" under step 3 for what this breaks and how to avoid it.
 
 ---
 
@@ -51,6 +52,38 @@ Use this skill when adding tests to documentation guides in the `agentgateway/we
 - **Display-only YAML blocks**: Some pages show YAML configs as plain display blocks (no `cat <<'EOF'` shell wrapper), unlike LLM guides that wrap configs in shell commands. You can't tag a display-only YAML block with `paths=` because it isn't a runnable shell command. Instead, add a **hidden** `{{< doc-test >}}` block that writes the config with `cat <<'EOF' > config.yaml`. See `content/docs/standalone/main/mcp/mcp-authz.md` for an example.
 - **External service dependencies**: When a config example depends on an external service that can't be trivially stood up in the test (e.g. Keycloak on port 9000, a custom OIDC provider), skip that example and only test self-contained ones. It's better to test one example well than to skip the entire page.
 
+#### Repeated commands across sections
+
+The extractor drops any block whose content is byte-identical to a block it already selected, keeping only the first. Nothing is logged, so the remaining copies just quietly do not run.
+
+This bites on multi-section pages where each section repeats the same command. For example, a page with four sections that each write a config and then validate it:
+
+```
+{{< doc-test paths="my-test" >}}
+agentgateway -f config.yaml --validate-only
+{{< /doc-test >}}
+```
+
+Four blocks like that collapse to **one**, so three of the four configs are never validated even though the test passes. The visible config-writing blocks survive because each one contains a different config.
+
+Give each repeated block distinguishing content — a comment naming the section is enough, and it makes the generated script easier to read:
+
+```
+{{< doc-test paths="my-test" >}}
+# Multi-level delegation: validate the config written by step 1
+agentgateway -f config.yaml --validate-only
+{{< /doc-test >}}
+```
+
+To confirm nothing was dropped, count the commands in the generated script and compare against the source:
+
+```sh
+grep -c "validate-only" content/docs/standalone/main/<page>.md
+grep -c "validate-only" out/tests/generated/<script-name>.sh
+```
+
+The same trap applies to any repeated command, such as an identical `start_gateway`, `sleep 3`, or curl warmup loop used in more than one section.
+
 ### 4. Long-running processes (standalone binary)
 
 - The guide may show "run agentgateway" in the foreground. For the generated script, the process must run in the background so the script can continue (e.g. curl, YAMLTest).
@@ -60,6 +93,71 @@ Use this skill when adding tests to documentation guides in the `agentgateway/we
   - `trap 'kill $AGW_PID 2>/dev/null' EXIT`
   - `sleep 3`
 - Do **not** add a path to the visible "run agentgateway" block so it is not included in the script; only the hidden block is.
+- **Helper backends: pick a port no documented config uses.** When a test needs a local backend to forward to, remember that doc tests run sequentially within a CI shard, so a helper process that outlives its script can collide with the next test. Port `8080` is the one documented configs use most (for example, the body-buffering page's `host: localhost:8080`), so choose something else (`8081`, `8082`) for a backend you invented, and keep the documented port for the test whose page actually specifies it.
+- **Make readiness checks identity-aware.** A loop like `curl -sf http://127.0.0.1:8080/ && break` passes as soon as *anything* answers on that port, so a stale process from an earlier test satisfies it and the real assertion then fails for a misleading reason. Probe for a response only your backend produces, and fail with a clear message if it never appears:
+  ```sh
+  for i in $(seq 1 30); do
+    [ "$(curl -sf --max-time 5 -X POST -d probe http://127.0.0.1:8080/ 2>/dev/null)" = "probe" ] && break
+    sleep 1
+  done
+  if [ "$(curl -sf --max-time 5 -X POST -d probe http://127.0.0.1:8080/ 2>/dev/null)" != "probe" ]; then
+    echo "FAIL: the echo backend did not come up on 127.0.0.1:8080 (is the port already in use?)"
+    exit 1
+  fi
+  ```
+- **Cleanup runs under `set -e`.** Generated scripts start with `set -euo pipefail`, so a failing command inside an `EXIT` trap aborts the handler and the script exits non-zero even when every assertion passed. Guard cleanup with `|| true`, and guard unset variables for `set -u`:
+  ```sh
+  stop_gateway() {
+    [ -n "${AGW_PID:-}" ] || return 0
+    kill "$AGW_PID" 2>/dev/null || true
+    wait "$AGW_PID" 2>/dev/null || true
+    AGW_PID=""
+  }
+  ```
+
+#### Never start a background process inside `$( )`
+
+A helper that both starts the gateway and returns a value looks convenient, but it breaks in two ways at once when it is called in a command substitution:
+
+```sh
+# BROKEN
+tool_names_for() {
+  agentgateway -f "$1" &
+  AGW_PID=$!                 # set in the SUBSHELL, invisible to the caller
+  curl ... | jq -r '...'
+}
+NAMES=$(tool_names_for config.yaml)
+```
+
+1. **The PID is lost.** `$( )` runs in a subshell, so `AGW_PID` never reaches the parent. A later `stop_gateway` sees it empty and returns without killing anything, so the gateway keeps holding its port. The next config then starts a gateway that cannot bind, and every assertion after that silently runs against the *previous* config.
+2. **The gateway's output is captured.** The background process inherits the substituted stdout, so its startup log ends up concatenated into the returned value.
+
+Symptom: the first assertion passes, later ones fail or hang for no obvious reason, and the test eventually times out. This is easy to misread as a product bug.
+
+Split the two jobs, so the process starts in the parent shell and only pure-curl code runs inside `$( )`:
+
+```sh
+start_gateway() {                       # call from the parent, never inside $( )
+  agentgateway -f "$1" > "agw-$1.log" 2>&1 &
+  AGW_PID=$!
+}
+
+wait_for_tools() {                      # pure curl, safe inside $( )
+  local out=""
+  for i in $(seq 1 15); do
+    out=$(query_something 2>/dev/null || true)
+    [ -n "$out" ] && break
+    sleep 2
+  done
+  echo "$out"
+}
+
+start_gateway config.yaml
+NAMES=$(wait_for_tools)
+stop_gateway
+```
+
+Redirect the process's output to a file as well, so nothing can leak into a captured value. When a page needs several configs in sequence, `stop_gateway` between them and confirm the port is actually released before the next start.
 
 ### 5. Env vars and placeholders
 
@@ -305,6 +403,9 @@ When in doubt, flag the failure to the user rather than silently adjusting the t
 
 - [ ] Path tags and `{{< doc-test >}}` blocks added in the **asset** file(s) (`assets/agw-docs/...`), **not** in the content wrapper files — even if multiple content files (e.g. `latest/` and `main/`) reuse the same asset.
 - [ ] Multiple paths in `paths="..."` are **comma-separated**, not space-separated — `paths="a,b"` ✓, `paths="a b"` ✗ (spaces make the whole string a single path, silently excluding the block).
+- [ ] No two selected blocks are **byte-identical** — the extractor keeps only the first and silently drops the rest, so a repeated `--validate-only` (or `start_gateway`, or warmup loop) across sections leaves later sections untested. Add a comment naming the section, then verify with `grep -c` on the source vs. the generated script. See "Repeated commands across sections" under step 3.
+- [ ] A helper backend the test invents uses a port **no documented config uses** (avoid `8080`), its readiness check probes for a response only that backend produces, and cleanup is guarded with `|| true` so a failing `kill` in an `EXIT` trap does not fail the script under `set -e`.
+- [ ] No helper **starts a background process inside `$( )`** — the PID is set in the subshell and lost, so the process is never killed and later configs run against the stale one; and its output gets captured into the returned value. Start with `start_gateway` in the parent shell, keep only pure-curl code inside the substitution. See "Never start a background process inside `$( )`" under step 4.
 - [ ] If the guide has a long-running server, a **hidden** doc-test block starts it in the background (and optional trap/sleep); visible "start server" block has **no** path.
 - [ ] Placeholders in shell blocks are quoted or use `${VAR:-default}` so the script has no syntax errors.
 - [ ] `test:` front matter on the **content** page lists the right `file` and `path`; file path is the content path (extractor follows reuse). For pages with no testable content (index pages, no code blocks), use `test: skip` instead — counts toward coverage without generating test cases.

content/docs/kubernetes/latest/llm/inference/_index.md

@@ -3,4 +3,6 @@ title: Inference workloads
 weight: 20
 description: Route to your own self-hosted generative AI models with inference workloads.
 url: /docs/kubernetes/latest/inference/
+aliases:
+  - /docs/kubernetes/latest/llm/inference/
 ---

content/docs/kubernetes/main/llm/inference/_index.md

@@ -3,4 +3,6 @@ title: Inference workloads
 weight: 20
 description: Route to your own self-hosted generative AI models with inference workloads.
 url: /docs/kubernetes/main/inference/
+aliases:
+  - /docs/kubernetes/main/llm/inference/
 ---

content/docs/standalone/latest/configuration/routes.md

@@ -3,8 +3,40 @@ title: Routes
 weight: 30
 description: Match HTTP and TCP traffic on a gateway and forward it to backends.
 next: /configuration/traffic-management
+test:
+  routes:
+  - file: ${versionRoot}/configuration/routes.md
+    path: routes
 ---
 
+{{< doc-test paths="routes" >}}
+# ============================================================================
+# Doc test coverage for this guide (these comments are not rendered on the page)
+# ============================================================================
+# WHAT THIS TEST VALIDATES:
+#   * "HTTP routes": the example config is accepted by agentgateway
+#     (--validate-only), covering the `gateways`, `protocol: HTTP`, `name`,
+#     `gateways: [...]`, `hostnames`, `matches.path.pathPrefix`, and
+#     `backends[].host` / `weight` fields the route table documents.
+#   * "TCP routes": the `tcpRoutes` example is accepted, covering `protocol: TCP`
+#     and the simpler TCP route structure.
+#   * "Example configuration with policies": the route-with-CORS example is
+#     accepted, covering `policies.cors` on a route and an inline `backends[].mcp`
+#     backend with a `stdio` target.
+#
+# WHAT THIS TEST DOES NOT VALIDATE (and why):
+#   * That traffic is actually matched and forwarded - requires config/traffic the
+#     page omits; every example points at a placeholder backend
+#     (`http.example.com:8080`, `postgres.example.com:5432`) that the test cannot
+#     stand up, so only config acceptance is asserted.
+#   * The `matches` header, method, and query options, and the "attaches to the
+#     gateway named `default`" fallback - display-only table rows with no example
+#     config on this page. Matching is covered by the Request matching guide.
+#   * The CORS policy's runtime behavior - covered by the CORS guide's own test;
+#     here the block only proves the policy is accepted on a route.
+{{< reuse "agw-docs/snippets/install-agentgateway-binary.md" >}}
+{{< /doc-test >}}
+
 {{< gloss "Route" >}}Routes{{< /gloss >}} are the entry points for traffic to your agentgateway. They attach to [gateways]({{< link-hextra path="/configuration/gateways/" >}}) and are used to route traffic to {{< gloss "Backend" >}}backends{{< /gloss >}}.
 
 ## Types of routes
@@ -36,6 +68,28 @@ routes:
     weight: 1
 ```
 
+{{< doc-test paths="routes" >}}
+cat <<'EOF' > config-http.yaml
+# yaml-language-server: $schema=https://agentgateway.dev/schema/config
+gateways:
+  http-proxy:
+    port: 8080
+    protocol: HTTP
+routes:
+- name: http-backend
+  gateways: [http-proxy]
+  hostnames:
+  - "example.com"
+  matches:
+  - path:
+      pathPrefix: /
+  backends:
+  - host: http.example.com:8080
+    weight: 1
+EOF
+agentgateway -f config-http.yaml --validate-only
+{{< /doc-test >}}
+
 HTTP routes support various matching options for incoming requests. For more information, see the [Request matching]({{< link-hextra path="/configuration/traffic-management/matching/" >}}) guide.
 
 ### TCP routes
@@ -60,6 +114,23 @@ tcpRoutes:
     weight: 1
 ```
 
+{{< doc-test paths="routes" >}}
+cat <<'EOF' > config-tcp.yaml
+# yaml-language-server: $schema=https://agentgateway.dev/schema/config
+gateways:
+  postgres-proxy:
+    port: 5432
+    protocol: TCP
+tcpRoutes:
+- name: postgres-backend
+  gateways: [postgres-proxy]
+  backends:
+  - host: postgres.example.com:5432
+    weight: 1
+EOF
+agentgateway -f config-tcp.yaml --validate-only
+{{< /doc-test >}}
+
 For more information, see [TCP route matching]({{< link-hextra path="/configuration/traffic-management/matching#tcp-routes" >}}).
 
 ## Route configuration
@@ -115,6 +186,34 @@ routes:
           args: ["@modelcontextprotocol/server-everything"]
 ```
 
+{{< doc-test paths="routes" >}}
+cat <<'EOF' > config-policies.yaml
+# yaml-language-server: $schema=https://agentgateway.dev/schema/config
+gateways:
+  default:
+    port: 3000
+routes:
+- policies:
+    cors:
+      allowOrigins:
+      - "*"
+      allowHeaders:
+      - mcp-protocol-version
+      - content-type
+      - cache-control
+      exposeHeaders:
+      - "Mcp-Session-Id"
+  backends:
+  - mcp:
+      targets:
+      - name: everything
+        stdio:
+          cmd: npx
+          args: ["@modelcontextprotocol/server-everything"]
+EOF
+agentgateway -f config-policies.yaml --validate-only
+{{< /doc-test >}}
+
 ## Next steps
 
 After you configure routes, you might want to apply policies to them or learn more about traffic management options.