diff --git a/docs/diagnosis/arch.md b/docs/diagnosis/arch.md
new file mode 100644
index 000000000000..b7f344811f8f
--- /dev/null
+++ b/docs/diagnosis/arch.md
@@ -0,0 +1,69 @@
+# Phase 9 — model architecture diff (llama.cpp `mistral3.cpp` vs HF `ministral3`)
+
+## TL;DR
+
+The two implementations are **structurally equivalent** for the inference path
+that Mistral-Medium-3.5 actually exercises. SwiGLU MLP, RMSNorm with FP32 cast,
+1/sqrt(head_dim) attention scale, pre-norm decoder layers, and YARN-scaled RoPE
+all match. The only optional code path is **Llama-4-style attention temperature
+scaling**, which is gated on `llama_4_scaling_beta != 0` in HF and on
+`hparams.f_attn_temp_scale != 0.0` in llama.cpp — both gates evaluate false for
+this model (`llama_4_scaling_beta = 0` per HF config, no
+`attn_temperature_scale` key in the GGUF), so the path is skipped on both
+sides. **Architecture is not the cause** of the long-context degradation.
+
+## Side-by-side mapping
+
+| concern | HF `Ministral3*` | llama.cpp `mistral3.cpp` |
+| --- | --- | --- |
+| pre-attention norm | `Ministral3RMSNorm(eps=rms_norm_eps)` cast→fp32 then back | `build_norm(LLM_NORM_RMS)` |
+| Q/K/V proj | `nn.Linear(bias=False)` | `build_qkv(...)` |
+| RoPE | `apply_rotary_pos_emb` with cached `cos,sin` (yarn) | `ggml_rope_ext` with `n_ctx_orig, freq_base, freq_scale, ext_factor, attn_factor, beta_fast, beta_slow` |
+| attn temperature scale | `Q *= 1 + beta * log(1 + floor(pos/orig_max))` (only if `beta!=0`) | `Q = ggml_mul(Q, inp_attn_scale)` (only if `f_attn_temp_scale!=0`) |
+| attention | `attention_interface(scale=1/sqrt(head_dim), sliding_window=None)` | `build_attn(..., kq_scale = 1/sqrt(n_embd_head))` |
+| sliding window | `getattr(config,'sliding_window',None)` → `None` for this model | not configured (correct) |
+| O proj | `o_proj = nn.Linear(bias=False)` | `model.layers[il].wo` |
+| residual after attn | `hidden = residual + hidden` | `ffn_inp = ggml_add(cur, inpSA)` |
+| post-attn norm | `Ministral3RMSNorm` | `build_norm(LLM_NORM_RMS)` (ffn_norm) |
+| MLP | SwiGLU: `down(silu(gate(x)) * up(x))` | `build_ffn(LLM_FFN_SILU, LLM_FFN_PAR)` (parallel = silu(gate)*up then down) |
+| residual after MLP | `hidden = residual + hidden` | `ggml_add(cur, ffn_inp)` |
+| MoE branch | n/a (Mistral-Medium-3.5 is dense) | guarded by `model.layers[il].ffn_gate_inp == nullptr` → dense path |
+
+## RoPE specifically
+
+Both honour every YARN parameter in the GGUF (`yarn_beta_fast=4`,
+`yarn_beta_slow=1`, `factor=64`, `original_context_length=4096`,
+`freq_base=1e6`, `yarn_log_multiplier=1.0`) — all of which match
+`rope_parameters` in `mistral_medium_check/config.json`. RoPE math is correct.
+
+## Llama-4 attn temperature scale
+
+```python
+def get_llama_4_attn_scale(positions_ids, beta, max_position_embeddings):
+    return (1 + beta * torch.log(1 + torch.floor(positions_ids / max_position_embeddings)))[:, None, :, None]
+```
+
+`beta = config.rope_parameters["llama_4_scaling_beta"] = 0`, so the multiplier
+collapses to **1.0** at every position. llama.cpp side-steps the entire
+multiplication when the GGUF doesn't carry the key. Both safe, both equivalent.
+
+## What this rules out
+
+- ❌ Sliding-window attention mismatch
+- ❌ Wrong RoPE dimensions / wrong YARN parameters
+- ❌ Missing Llama-4 temperature scaling (it's a no-op for this model)
+- ❌ Activation function mismatch (both SwiGLU)
+- ❌ Pre-norm vs post-norm placement
+- ❌ Attention scale factor
+
+## What it does NOT rule out
+
+- Numerical precision in CUDA kernels (FP16 accumulators in `ggml-cuda` for
+  the matmul or attention path).
+- Q8_0 quantization rounding error in long-attention contexts.
+- KV-cache dtype (default F16 → numerical drift over many tokens).
+- Sampler behaviour (min_p default of 0.05 in llama-server vs vLLM not applying
+  it).
+
+Phase 7 (rebuild with `GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F=1`) and Phase 6
+(F16/BF16/Q8_0 KV-cache experiment) and Phase 8 (matched samplers) cover those.
diff --git a/docs/diagnosis/chat_template.md b/docs/diagnosis/chat_template.md
new file mode 100644
index 000000000000..7ec2587ad363
--- /dev/null
+++ b/docs/diagnosis/chat_template.md
@@ -0,0 +1,44 @@
+# Phase 2 — chat template parity
+
+## TL;DR
+
+The four chat templates considered — (A) GGUF-embedded jinja in
+`bartowski/mistralai_Mistral-Medium-3.5-128B-GGUF`, (B) `unsloth/Mistral-Medium-3.5-128B`,
+(C) `mistralai/Mistral-Medium-3.5-128B` upstream, (D) the `tokenizer.chat_template`
+attribute exposed by HF AutoTokenizer for the upstream model — are
+**semantically equivalent for normal multi-turn chat**. The only diff that
+could change rendered output is in error-path code that nobody is hitting in
+this test. **Templates are not the cause** of the long-context degradation.
+
+## Files
+
+- `outputs/template_llamacpp.jinja` (13281 bytes) — A
+- `outputs/template_unsloth.jinja` (14475 bytes) — B
+- `outputs/template_mistralai.jinja` (13479 bytes) — C
+- `outputs/template_hftokenizer.jinja` (13479 bytes) — D
+- `outputs/template_diff_*` — pairwise diffs
+
+## Diff summary
+
+| pair | size | nature of diff |
+| --- | --- | --- |
+| C vs D (mistralai vs HF tokenizer) | 0 bytes | **identical** |
+| A vs C (GGUF vs upstream) | 686 bytes | one hunk: a disabled `{%- if false %}` assertion at line 201 of the GGUF copy where upstream has the real check `(content == '' or content is none) and (no tool calls)` → raise. Both branches are no-ops on valid messages. |
+| B vs C (unsloth vs upstream) | 2040 bytes | (1) unsloth uses `strftime_now` for date defaults; upstream hardcodes `today=29-04-2026`, `yesterday=28-04-2026`. Both get overridden by template arguments at render time, so no effect on output. (2) unsloth `arguments\|tojson\|safe` vs upstream `arguments\|tojson` for tool-call serialisation — only relevant when tools are used. |
+
+## Cross-check via tokenization parity (Phase 1)
+
+When all four templates render the same multi-turn fixture (system prompt + 3
+user/assistant pairs + final user turn, `reasoning_effort='none'`), they
+produce **byte-identical 434-token sequences**, including `<s>`,
+`[SYSTEM_PROMPT]`, `[/SYSTEM_PROMPT]`, `[MODEL_SETTINGS]`, `[/MODEL_SETTINGS]`,
+`[INST]`, `[/INST]`, and the per-assistant `</s>` markers in the right
+positions. See `outputs/diagnosis_tokenization.md`.
+
+## Conclusion
+
+If we replace llama-server's GGUF-embedded template with the upstream mistralai
+copy via `--chat-template-file outputs/template_mistralai.jinja`, the rendered
+text and resulting token stream are identical to what the GGUF template
+produces today. **No improvement is expected from a template swap, and indeed
+no improvement was observed empirically** — see Phase 8 matched-sampler runs.
diff --git a/docs/diagnosis/config.md b/docs/diagnosis/config.md
new file mode 100644
index 000000000000..9afef7cf9c80
Binary files /dev/null and b/docs/diagnosis/config.md differ
diff --git a/docs/diagnosis/first_divergence.md b/docs/diagnosis/first_divergence.md
new file mode 100644
index 000000000000..e1602457db1b
--- /dev/null
+++ b/docs/diagnosis/first_divergence.md
@@ -0,0 +1,88 @@
+# First-divergence pin-down (vLLM vs llama-server, greedy)
+
+Reproducer: `repro_first_divergence.py` in this directory.
+
+Same input ("Create a Flappy Bird Python game" + Mistral-Medium-3.5 SYSTEM_PROMPT
+with `reasoning_effort=none`), greedy (temperature=0.0), max_tokens=100, on:
+
+- vLLM 0.20.1rc1.dev127 FP8 + `--attention-backend FLASH_ATTN` (port 8765)
+- llama-server (b1-aab68217b unslothai fork, also tested ggml-org master) Q8_0 (port 8766)
+
+The two outputs share an **identical 13-token prefix**:
+
+```
+# Flappy Bird Game in Python\n\nHere's a complete implementation of
+```
+
+Token 14 is where they first diverge. **The top-2 tokens at this position are
+the same in both servers**, but their *relative order* is flipped:
+
+| rank | vLLM | logprob | llama-server | logprob |
+| --- | --- | --- | --- | --- |
+| 1 | ` a` | **−0.314** | ` Fl` | **−0.289** |
+| 2 | ` Fl` | −1.314 | ` a` | −1.430 |
+| 3 | ` the` | −7.689 | ` the` | −4.434 |
+| 4 | `Fl` | −12.564 | `Fl` | −11.420 |
+
+Both have the same top-2 candidates, but llama-server compresses ` a` from
+−0.31 to −1.43 (a Δ of ~1.1 in logprob, i.e. a factor of 3 in probability)
+while vLLM compresses ` Fl` from −1.31 to −0.31 (same Δ in the other
+direction). On a logprob scale of −0.3 vs −1.4 these tokens are within 1 nat
+of each other; greedy-decoding *must* pick exactly one, and the order flip
+sends the two backends down different trajectories.
+
+After that single token flip:
+- vLLM continues with `... a Flappy Bird game using Pygame. This version includes the core mechanics ...`
+- llama-server continues with `... Flappy Bird using Pygame. This version includes all the classic elements ...`
+
+Both trajectories are coherent at this point. The two outputs both produce
+clean code for ~600–1000 generated tokens, after which **only the
+llama-server trajectory** degenerates into broken syntax and repetition (e.g.
+`pygame.display.set mode((800, 600)` — `set_mode` becomes `set mode`).
+
+## Logits progression at fixed prefix
+
+Reproducer: `repro_logits_progression.py`.
+
+When we take vLLM's full greedy output and **feed it as a fixed prefix** at
+checkpoints 50/200/500/1000/1400, both servers' top-1 next-token agrees at
+*every* checkpoint:
+
+| n_decoded | vLLM top-1 | vLLM logprob | llama-server top-1 | llama-server logprob | match |
+| --- | --- | --- | --- | --- | --- |
+| 50 | ` ``` ` | −0.0233 | ` ``` ` | −0.0000 | ✓ |
+| 200 | `Y` | ~0 | `Y` | ~0 | ✓ |
+| 500 | `.rect` | −0.0001 | `.rect` | −0.2432 | ✓ |
+| 1000 | `_p` | ~0 | `_p` | −0.0601 | ✓ |
+| 1400 | `   ` | ~0 | `   ` | −0.0028 | ✓ |
+
+So the long-context degeneration is **not** caused by the model converging on
+different next-token answers given the same prefix. It is caused by a single
+~1-nat precision flip near the start, after which vLLM and llama-server walk
+different (still individually plausible) decoding paths — and the
+llama-server path happens to land in a degenerate attractor.
+
+## Cross-check on Q4_K_M
+
+Repeating the experiment with `bartowski/.../Q4_K_M` (~74 GiB GGUF) on
+llama-server: identical degeneration tail. The same wrong top-2 ranking at
+token 14 occurs, then the trajectory degenerates *more* than Q8_0 — for
+example `pygame.display.set mode sdl hWSIZER, sdl lg2` syntax garbage. So
+this is uniform across llama.cpp quants, not a Q8_0-specific bug.
+
+## Implication
+
+The remaining hypothesis is that ggml-cuda's accumulator precision in the
+matmul or attention path for this specific model shape (88 layers,
+head_count=96, head_count_kv=8, head_dim=128, vocab=131072,
+intermediate=28672, rope_freq_base=1e6 with YARN factor=64) is producing
+logits that are subtly *flatter* than the reference. The ranking flip on
+token 14 is consistent with this: top-2 tokens are within 1 nat in both
+servers, but llama-server compresses the gap by ~0.4 nat in a way that puts
+` Fl` ahead of ` a` instead of behind.
+
+A targeted next step is to fix the matmul accumulator in
+`ggml-cuda/mmq.cu` (or the relevant GEMM path) to FP32 specifically for
+`LLM_ARCH_MISTRAL3` and re-test. `GGML_CUDA_FORCE_CUBLAS=1` was already tried
+without effect, but only `GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F=1` only helps the
+cuBLAS half-precision path; Q8_0 dequant + matmul takes a different path.
diff --git a/docs/diagnosis/logits.md b/docs/diagnosis/logits.md
new file mode 100644
index 000000000000..c893655d05ad
--- /dev/null
+++ b/docs/diagnosis/logits.md
@@ -0,0 +1,29 @@
+# Phase 3 — top-10 logprobs comparison (vLLM vs llama-server)
+
+prompt: `What is the capital of France? Answer in exactly one word.`  
+temperature: 0.0 (greedy), max_tokens: 1, top_logprobs: 10
+
+vLLM completion: `Paris`  
+llama-server completion: `Paris`
+
+## Top-10 next-token logprobs
+
+| rank | vLLM token | vLLM logprob | llama token | llama logprob |
+| --- | --- | --- | --- | --- |
+| 1 | `token_id:42572` | -0.0001 | `Paris` | -0.0000 |
+| 2 | `token_id:6993` | -9.8751 | ` Paris` | -11.4609 |
+| 3 | `token_id:1784` | -12.0001 | `Par` | -12.5013 |
+| 4 | `token_id:2029` | -12.9376 | `PAR` | -14.4139 |
+| 5 | `token_id:3814` | -13.2501 | `巴黎` | -14.6281 |
+| 6 | `token_id:102726` | -13.6251 | `Pars` | -15.6960 |
+| 7 | `token_id:38166` | -13.6251 | `par` | -16.1068 |
+| 8 | `token_id:72056` | -13.8751 | ` Париж` | -16.8023 |
+| 9 | `token_id:75613` | -15.1876 | `Berlin` | -17.0093 |
+| 10 | `token_id:126441` | -15.3751 | ` París` | -17.9099 |
+
+Jaccard(top-10 token set): **0.00**  
+Top-1 match: **False**  
+
+## TL;DR
+
+Greedy top-1 token: **DIFFERS**.
diff --git a/docs/diagnosis/mistral-medium-3.5-long-context.md b/docs/diagnosis/mistral-medium-3.5-long-context.md
new file mode 100644
index 000000000000..6c700cbff37d
--- /dev/null
+++ b/docs/diagnosis/mistral-medium-3.5-long-context.md
@@ -0,0 +1,131 @@
+# Mistral-Medium-3.5-128B llama.cpp long-context degradation — diagnosis
+
+## TL;DR
+
+llama-server (Q8_0 GGUF from `bartowski/mistralai_Mistral-Medium-3.5-128B-GGUF`)
+collapses into deterministic repetition loops after ~1000–1500 generated tokens,
+producing broken Python syntax (e.g. `pygame.display.set mode((800, 600)` —
+missing the underscore in `set_mode` and the closing paren). vLLM (FP8 of the
+same model) does not exhibit this; it stops naturally at 1496 tokens. **HF
+transformers BF16 inference of the same model also degrades** in the same way,
+ruling out llama.cpp/Q8_0 as the *unique* culprit.
+
+This matches what unsloth has already published on
+[`unsloth/Mistral-Medium-3.5-128B-GGUF`](https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF):
+
+> Testing shows that this behavior occurs **regardless of who or how** the model
+> was converted GGUF. The model initially responds correctly, but over long
+> context, does not work properly. **Mistral has now labeled GGUF support as a
+> WIP**.
+
+This document records the experiments that were run against vLLM (port 8765,
+FP8) and llama-server (port 8766, Q8_0) and what they ruled out. It is
+intended as input to a deeper fix.
+
+## Setup
+
+- vLLM 0.20.1rc1.dev127, FP8, `--tensor-parallel-size 2 --port 8765 --tool-call-parser mistral --enable-auto-tool-choice --reasoning-parser mistral --max_num_batched_tokens 16384 --max_num_seqs 128 --gpu_memory_utilization 0.8 --attention-backend FLASH_ATTN`, GPUs 4,5.
+- llama-server `b1-aab6821` (unslothai fork) and ggml-org master, Q8_0,
+  `--tensor-split 1,1 --port 8766 --jinja --ctx-size 32768 --parallel 1`,
+  GPUs 2,3.
+- HF transformers 5.7.0, `Mistral3ForConditionalGeneration` BF16,
+  `attn_implementation="eager"`, GPUs 6,7.
+
+## Phases run
+
+| phase | topic | result | location |
+| --- | --- | --- | --- |
+| 1 | Tokenization parity (vLLM / llama-server / mistral-common / HF) | **identical** 434 tokens | `outputs/diagnosis_tokenization.md` |
+| 2 | Chat template (GGUF jinja vs unsloth vs mistralai vs HF tokenizer) | semantically equivalent (single trivial diff: a disabled `if false` assertion at line 201) | `outputs/diagnosis_chat_template.md` (and `outputs/template_*.jinja`) |
+| 3 | Top-k logits agreement | top-1 token agrees on simple prompts; logit *distributions* differ noticeably (vLLM is less peaked) | `outputs/diagnosis_logits.md` |
+| 4 | GGUF metadata vs HF config | match. RoPE: `factor=64`, `freq_base=1e6`, `original_context_length=4096`, `yarn_beta_fast=4`, `yarn_beta_slow=1`, `yarn_log_mul=1.0`. All YARN parameters present in GGUF and equal to `text_config.rope_parameters` | `outputs/diagnosis_config.md`, `outputs/gguf_metadata_full.txt` |
+| 5 | HF transformers BF16 ground truth | **also degrades** (recall score 0/4 on the interleaved 11-turn test). T2 Flappy Bird hits 1500-token cap with looping syntax errors in tail | `outputs/hf_groundtruth_*` |
+| 6 | KV cache dtype: F16 (default) / BF16 / F32 | **no effect on degradation** | `outputs/diag_single_flappy_*.json` |
+| 7 | Rebuild llama.cpp with `-DGGML_CUDA_FORCE_CUBLAS=ON -DGGML_CUDA_FORCE_MMQ=OFF` and runtime `GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F=1` | **no effect on degradation** | `outputs/llama_server_compute32f_*.log` |
+| 8 | Matched samplers (`temp=0.1, top_p=1, top_k=64, min_p=0.05, seed=42`) | **no effect**; tried `repetition_penalty ∈ {1.0,1.05,1.1,1.2}`, `frequency_penalty ∈ {0,0.1,0.3,0.5}`, `dry_multiplier ∈ {0,0.5,0.8}` — all still loop | `outputs/matched_*` |
+| 9 | Architecture diff — `llama.cpp/src/models/mistral3.cpp` vs HF `transformers/models/ministral3/modeling_ministral3.py` | **equivalent**. Same SwiGLU, RMSNorm-fp32, kq_scale=1/√head_dim, optional Llama-4 attn temperature scale (gated and disabled for this model since `llama_4_scaling_beta=0`). YARN `attn_factor` chain in `llama-context.cpp` matches HF's `_compute_yarn_parameters` — both produce `1.0` for `mscale=mscale_all_dim=1.0`. | `outputs/diagnosis_arch.md` |
+
+## Empirical convergence point
+
+Single-turn `Create a Flappy Bird Python game` (greedy, temperature=0):
+
+```
+| backend                          | finish | n_out | tail snippet
+| -------------------------------- | ------ | ----- | -----------
+| vLLM FP8                         | stop   | 1496  | "...Would you like me to explain any specific part?"
+| llama-server Q8_0 cuBLAS         | length | 2048  | "if __name__ == \"__main__,\\n    sys.exit()\\n```\\n\\n" (loop)
+| llama-server Q8_0 BF16 KV        | length | 2048  | "if pipe.x < 0\\n    self.pipes.remove(pipe)" (loop)
+| llama-server Q8_0 F32 KV + no FA | length | 2048  | identical loop
+| HF transformers BF16             | length | 1500  | "pipe1 = pipe(50, 100, 50, 100\\n    pipe2 = pipe(50, 100, 50, 100" (loop)
+```
+
+For llama-server, output is clean up to ~1000 tokens then degrades:
+
+```
+mt=  600: clean
+mt= 1000: still clean
+mt= 1500: looping
+```
+
+Common prefix between vLLM and llama-server greedy outputs is exactly **66
+characters**: `# Flappy Bird Game in Python\n\nHere's a complete implementation of `
+
+Then:
+- vLLM picks `a Flappy Bird game using Pygame`
+- llama-server picks `Flappy Bird using Pygame` (no leading `a`).
+
+## Working hypotheses (still open)
+
+- **H1: Q8_0 quantization error**. Q8_0 gives ~16-bit mantissa precision per
+  block of 32. Cumulative error across 88 layers × 1000 decode steps may be
+  enough to flip top-1 tokens that compound into loops. *Counter-evidence:* HF
+  BF16 with the FP8 safetensors **also** loops, so quantization can't be the
+  whole story.
+- **H2: Numerical kernel issue specific to Mistral-Medium-3.5's shape (88
+  layers, head_dim=128, head_count=96, head_count_kv=8, vocab=131072,
+  intermediate=28672, rope_freq_base=1e6 with YARN factor 64)** — common to
+  llama.cpp ggml-cuda *and* HF eager. vLLM avoids it because it uses CUTLASS
+  FP8 GEMMs with FP32 accumulators and possibly a different attention kernel
+  (`FLASH_ATTN` selected by vLLM in our setup).
+- **H3: Some prefill artifact** that vLLM mitigates via chunked prefill
+  (`--max_num_batched_tokens 16384`). Not yet tested in isolation.
+
+## What is NOT the cause
+
+- Tokenization (4 tokenizers byte-identical).
+- Chat template (4 templates render to identical token streams).
+- GGUF metadata (all numerical config matches HF, including all YARN params).
+- Sampler (no setting of temp/top_p/top_k/min_p/repetition_penalty/freq_penalty/dry that breaks the loop).
+- KV cache dtype.
+- Flash attention vs default attention in llama.cpp.
+- cuBLAS vs MMQ kernels.
+- Architecture code (mistral3.cpp implements the same residual/RMSNorm/SwiGLU/RoPE flow as HF).
+- Sliding window (None for this model — both honour that).
+
+## Artefacts
+
+All under `workspace_5/outputs/`:
+
+- `diagnosis_tokenization*.{md,txt}` + `tok_ids_*.json`
+- `diagnosis_chat_template.md` + `template_*.jinja` + `template_diff_*.txt`
+- `diagnosis_config.md` + `gguf_metadata_full.txt`
+- `diagnosis_arch.md`
+- `diagnosis_logits.md` + `diagnosis_logits_raw.json`
+- `hf_groundtruth_alphabet_*.txt` + `hf_groundtruth_interleaved_*.json` + `hf_groundtruth_tok_ids_*.json`
+- `recall_interleaved_{vllm,llamacpp}_*.txt`
+- `multi_turn_recall_{vllm,llamacpp}_*.txt`
+- `diag_single_flappy_*.json` + `.log`
+- `matched_*.{json,txt}`
+- All llama-server / vllm / HF logs in `workspace_5/logs/`.
+
+## Suggested next steps for a real fix
+
+1. Dump per-layer activations from both vLLM and HF for the same input, and
+   compare layer-by-layer to find where they first diverge meaningfully.
+2. If that divergence localises to RMSNorm or attention softmax, force FP32
+   accumulators in those ops in llama.cpp's CUDA kernels for `mistral3`.
+3. Compare against `llama-bench --perplexity` numbers per quant; Q8_0 vs Q6_K
+   vs F16-converted-from-FP8 GGUF to determine whether the issue scales with
+   precision.
+4. Consider writing a reference forward pass in PyTorch using the GGUF weights
+   (via `gguf-py`) and the HF arch and comparing token-by-token.
diff --git a/docs/diagnosis/repro_first_divergence.py b/docs/diagnosis/repro_first_divergence.py
new file mode 100644
index 000000000000..f144f47c271d
--- /dev/null
+++ b/docs/diagnosis/repro_first_divergence.py
@@ -0,0 +1,113 @@
+"""Find the EXACT token position where vLLM and llama-server's greedy outputs first disagree.
+
+Generates greedy outputs from both, then walks token-by-token to find the first
+mismatch. At that mismatch, prints both servers' top-20 logprobs at that position.
+"""
+import json
+from datetime import datetime, timedelta
+from pathlib import Path
+
+import requests
+from huggingface_hub import hf_hub_download
+from openai import OpenAI
+
+
+def load_system_prompt() -> str:
+    p = hf_hub_download(repo_id="mistralai/Mistral-Medium-3.5-128B", filename="SYSTEM_PROMPT.txt")
+    raw = Path(p).read_text()
+    today = datetime.today().strftime("%Y-%m-%d")
+    yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")
+    return raw.format(name="Mistral-Medium-3.5-128B", today=today, yesterday=yesterday)
+
+
+SYSTEM_PROMPT = load_system_prompt()
+MESSAGES = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {"role": "user", "content": "Create a Flappy Bird Python game"},
+]
+
+
+def greedy_with_logprobs(base_url: str, max_tokens: int = 100):
+    client = OpenAI(api_key="EMPTY", base_url=base_url)
+    model = client.models.list().data[0].id
+    r = client.chat.completions.create(
+        model=model,
+        messages=MESSAGES,
+        temperature=0.0,
+        max_tokens=max_tokens,
+        logprobs=True,
+        top_logprobs=20,
+        extra_body={"reasoning_effort": "none"},
+    )
+    return r
+
+
+print("Querying vLLM (max 100 tokens)...")
+v = greedy_with_logprobs("http://localhost:8765/v1", max_tokens=100)
+print("Querying llama-server (max 100 tokens)...")
+l = greedy_with_logprobs("http://localhost:8766/v1", max_tokens=100)
+
+vt = [c.token for c in v.choices[0].logprobs.content]
+lt = [c.token for c in l.choices[0].logprobs.content]
+v_top = [c.top_logprobs for c in v.choices[0].logprobs.content]
+l_top = [c.top_logprobs for c in l.choices[0].logprobs.content]
+
+print(f"vLLM produced {len(vt)} tokens")
+print(f"llama produced {len(lt)} tokens")
+
+# vLLM tokens may be in 'token_id:N' form; llama in piece form. Compare via piece by detokenizing vLLM's IDs.
+def piece_for_vllm_token(t):
+    if t.startswith("token_id:"):
+        try:
+            tid = int(t.split(":", 1)[1])
+            r = requests.post("http://localhost:8766/detokenize", json={"tokens": [tid]}, timeout=30).json()
+            return r["content"]
+        except Exception:
+            return t
+    return t
+
+
+# Walk for first divergence by piece string.
+i = 0
+n = min(len(vt), len(lt))
+while i < n:
+    vp = piece_for_vllm_token(vt[i])
+    lp = lt[i]
+    if vp != lp:
+        break
+    i += 1
+
+print(f"\nFirst divergence at decoded position {i}")
+if i < n:
+    vp = piece_for_vllm_token(vt[i])
+    lp = lt[i]
+    print(f"  vLLM picked: {vp!r} (raw token: {vt[i]!r})")
+    print(f"  llama picked: {lp!r}")
+
+    # Show top-20 at divergence position
+    print("\nvLLM top-20 at this position:")
+    for tlp in (v_top[i] or [])[:10]:
+        piece = piece_for_vllm_token(tlp.token)
+        print(f"  {piece!r:<25} {tlp.logprob:.4f}")
+
+    print("\nllama top-10 at this position:")
+    for tlp in (l_top[i] or [])[:10]:
+        print(f"  {tlp.token!r:<25} {tlp.logprob:.4f}")
+
+    # Also: prefix that was generated identically up to this point
+    if i > 0:
+        # detokenize identical prefix from llama's piece list
+        prefix = "".join(lt[:i])
+        print(f"\nIdentical prefix ({len(prefix)} chars):\n{prefix!r}")
+else:
+    print("No divergence in the first 100 tokens.")
+
+out = Path(f"outputs/diag_first_divergence_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json")
+out.write_text(json.dumps({
+    "vllm_tokens": vt,
+    "llama_tokens": lt,
+    "first_divergence_index": i,
+    "vllm_top20_at_divergence": [(t.token, t.logprob) for t in (v_top[i] if i < len(v_top) else [])] if i < n else [],
+    "llama_top20_at_divergence": [(t.token, t.logprob) for t in (l_top[i] if i < len(l_top) else [])] if i < n else [],
+}, indent=2))
+print(f"\nsaved {out}")
diff --git a/docs/diagnosis/repro_logits_progression.py b/docs/diagnosis/repro_logits_progression.py
new file mode 100644
index 000000000000..741c0d45b119
--- /dev/null
+++ b/docs/diagnosis/repro_logits_progression.py
@@ -0,0 +1,166 @@
+"""Compare top-k logprobs from vLLM and llama-server at multiple decode positions.
+
+Drives the conversation forward in lockstep: at each step, take the assistant
+text generated by vLLM up to position N, feed it as a fixed prefix to both
+servers via the chat history, and ask each for top_logprobs at the next token.
+Quantifies how much the two distributions diverge as the decode position
+grows.
+"""
+import json
+import os
+from datetime import datetime, timedelta
+from pathlib import Path
+
+from huggingface_hub import hf_hub_download
+from openai import OpenAI
+
+
+def load_system_prompt() -> str:
+    p = hf_hub_download(repo_id="mistralai/Mistral-Medium-3.5-128B", filename="SYSTEM_PROMPT.txt")
+    raw = Path(p).read_text()
+    today = datetime.today().strftime("%Y-%m-%d")
+    yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")
+    return raw.format(name="Mistral-Medium-3.5-128B", today=today, yesterday=yesterday)
+
+
+SYSTEM_PROMPT = load_system_prompt()
+USER_PROMPT = "Create a Flappy Bird Python game"
+
+
+def topk_at_prefix(base_url: str, prefix_text: str, top_k: int = 20) -> list:
+    """Get top-k logprobs for the next token after assistant has produced prefix_text."""
+    client = OpenAI(api_key="EMPTY", base_url=base_url)
+    model = client.models.list().data[0].id
+    msgs = [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": USER_PROMPT},
+    ]
+    # Trick: use assistant turn with `partial_assistant_message`/prefix isn't standard.
+    # Instead, render a continuation completion via /v1/completions on the raw text.
+    # vLLM accepts /v1/completions; llama-server too.
+    return None  # placeholder
+
+
+# Different approach: do a vLLM-only full greedy decode of 1500 tokens, then for
+# each checkpoint position, query both servers' top_logprobs at that token.
+
+def main():
+    OUT = Path("outputs/diag_logits_progression.json")
+    OUT.parent.mkdir(parents=True, exist_ok=True)
+
+    # Step 1: get vLLM's full greedy decode.
+    vllm_client = OpenAI(api_key="EMPTY", base_url="http://localhost:8765/v1")
+    vllm_model = vllm_client.models.list().data[0].id
+    print(f"vLLM model: {vllm_model}")
+
+    base_msgs = [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": USER_PROMPT},
+    ]
+    r = vllm_client.chat.completions.create(
+        model=vllm_model,
+        messages=base_msgs,
+        temperature=0.0,
+        max_tokens=1500,
+        logprobs=True,
+        top_logprobs=20,
+        extra_body={"reasoning_effort": "none", "return_tokens_as_token_ids": True},
+    )
+    vllm_full_text = r.choices[0].message.content or ""
+    print(f"vLLM total tokens: {r.usage.completion_tokens}")
+    print(f"vLLM finish: {r.choices[0].finish_reason}")
+    print(f"vLLM head: {vllm_full_text[:80]!r}")
+
+    # Step 2: at checkpoints (token 50, 200, 500, 1000, 1400), reconstruct the
+    # same prefix and query llama-server's top-20.
+    checkpoints = [50, 200, 500, 1000, 1400]
+    samples = []
+
+    # We need to send a partial assistant turn. The OpenAI chat API doesn't directly
+    # support that. Workaround: use /v1/completions with the raw rendered prompt,
+    # which BOTH servers expose, by calling render endpoints to assemble.
+    import requests
+
+    # Re-fetch vLLM's logprobs.content list — these are token-level entries.
+    if r.choices[0].logprobs is None:
+        print("no logprobs from vllm")
+        return
+    vllm_token_entries = r.choices[0].logprobs.content
+    # Each entry has .token (formatted as "token_id:NN" since we asked) and .logprob and .top_logprobs
+
+    # Get the rendered prompt as text (chat template applied)
+    base_render = requests.post(
+        "http://localhost:8766/apply-template",
+        json={"messages": base_msgs},
+        timeout=60,
+    ).json()["prompt"]
+    # This is the raw text that, after BOS, becomes the input
+
+    # Get a list of token IDs and decoded text per cumulative output position.
+    # We accumulate detokenized characters position by position.
+    cumulative_text_at = {}
+    cur_ids = []
+    for i, tok in enumerate(vllm_token_entries):
+        # tok.token looks like "token_id:1035"
+        if not tok.token.startswith("token_id:"):
+            continue
+        try:
+            tid = int(tok.token.split(":", 1)[1])
+        except Exception:
+            continue
+        cur_ids.append(tid)
+        if (i + 1) in checkpoints:
+            # Detokenize up to this point via llama-server (since vLLM detokenize is at /detokenize too)
+            txt = requests.post(
+                "http://localhost:8766/detokenize",
+                json={"tokens": cur_ids},
+                timeout=60,
+            ).json()["content"]
+            cumulative_text_at[i + 1] = txt
+
+    print(f"Captured {len(cumulative_text_at)} checkpoints: {list(cumulative_text_at)}")
+
+    for n_decoded, prefix_text in cumulative_text_at.items():
+        # Construct a /v1/completions prompt that mimics the chat completion's KV state.
+        # The chat-template-rendered text + the assistant-side prefix.
+        # For Mistral chat template, after [INST] xxx [/INST], the assistant emits text
+        # without a leading "[ASST]" tag, so we just concatenate.
+        full_prompt = base_render + prefix_text
+
+        # Query both servers for next-token top-20 logprobs via /v1/completions.
+        sample = {"n_decoded": n_decoded, "prefix_chars": len(prefix_text)}
+        for label, base in [("vllm", "http://localhost:8765/v1"), ("llama", "http://localhost:8766/v1")]:
+            try:
+                rr = requests.post(
+                    base + "/completions",
+                    json={
+                        "model": "mistralai/Mistral-Medium-3.5-128B" if label == "vllm" else "mistralai_Mistral-Medium-3.5-128B-Q8_0-00001-of-00004.gguf",
+                        "prompt": full_prompt,
+                        "max_tokens": 1,
+                        "temperature": 0.0,
+                        "logprobs": 20,
+                    },
+                    timeout=120,
+                ).json()
+                # parse
+                if "choices" in rr and rr["choices"]:
+                    ch = rr["choices"][0]
+                    lp = ch.get("logprobs") or {}
+                    top = lp.get("top_logprobs") or []
+                    if isinstance(top, list) and top:
+                        # top is a list of dicts {token_str: logprob}
+                        sample[label + "_top20"] = top[0]
+                        sample[label + "_first_token"] = (lp.get("tokens") or [None])[0]
+                    else:
+                        sample[label + "_raw"] = rr
+            except Exception as e:
+                sample[label + "_error"] = str(e)
+        samples.append(sample)
+        print(f"checkpoint {n_decoded}: {sample.get('vllm_first_token')!r} vs {sample.get('llama_first_token')!r}")
+
+    OUT.write_text(json.dumps({"vllm_full_text": vllm_full_text, "checkpoints": samples}, indent=2))
+    print(f"saved {OUT}")
+
+
+if __name__ == "__main__":
+    main()
diff --git a/docs/diagnosis/tokenization.md b/docs/diagnosis/tokenization.md
new file mode 100644
index 000000000000..709dfe6c7555
--- /dev/null
+++ b/docs/diagnosis/tokenization.md
@@ -0,0 +1,84 @@
+# Tokenization parity diagnosis (Mistral-Medium-3.5-128B)
+
+Compares token IDs produced by **four** tokenizers for the same multi-turn fixture
+(system + 4 user + 3 assistant messages, `reasoning_effort='none'`).
+
+**Sources compared:**
+- `vllm`: vLLM `POST /v1/chat/completions/render` (server on `:8765`).
+- `llamacpp`: llama-server `POST /apply-template` then `POST /tokenize` with `add_special=true` (server on `:8766`).
+- `mistralcommon`: `MistralTokenizer.from_hf_hub('mistralai/Mistral-Medium-3.5-128B').encode_chat_completion(...)` (mistral-common 1.11.1).
+- `hf`: `AutoTokenizer.from_pretrained('mistralai/Mistral-Medium-3.5-128B').apply_chat_template(messages, tokenize=True, add_generation_prompt=True, reasoning_effort='none')['input_ids']` (transformers 5.7.0).
+
+## TL;DR
+
+**All four tokenizers produce byte-identical 434-token sequences for this fixture.** Tokenization is NOT the source of the llama-server vs vLLM accuracy gap on long conversations. Special-token boundaries (`<s>`, `[SYSTEM_PROMPT]`, `[/SYSTEM_PROMPT]`, `[MODEL_SETTINGS]`, `[/MODEL_SETTINGS]`, `[INST]`, `[/INST]`, `</s>`) all land in the same positions across all four. The `</s>` count is exactly 3 — one per assistant turn — in every source. The next phase should look at chat-template rendering (the small whitespace differences in the textual rendering, see file lengths), top-logits comparison, and KV-cache / kernel-level differences.
+
+## Token-id length per source
+
+| source | total tokens | rendered text length (chars) |
+|---|---|---|
+| vllm | 434 | 1823 |
+| llamacpp | 434 | 1820 |
+| mistralcommon | 434 | 1823 |
+| hf | 434 | 1823 |
+
+Note: `llamacpp` text length differs by 3 chars vs the other three (1820 vs 1823) — purely the difference between the rendered jinja text returned by `/apply-template` (which omits the BOS string `<s>`) vs the detokenize-of-IDs which renders BOS as a 3-char literal. The IDs themselves are identical.
+
+## Pairwise diff (vs vLLM as reference)
+
+| source | identical to vllm? | first divergence index |
+|---|---|---|
+| vllm | (reference) | - |
+| llamacpp | **YES** | n/a |
+| mistralcommon | **YES** | n/a |
+| hf | **YES** | n/a |
+
+## Special-token detection (resolved via llama-server `/tokenize`)
+
+| token | id |
+|---|---|
+| `<s>` | 1 |
+| `</s>` | 2 |
+| `[SYSTEM_PROMPT]` | 17 |
+| `[/SYSTEM_PROMPT]` | 18 |
+| `[INST]` | 3 |
+| `[/INST]` | 4 |
+| `[MODEL_SETTINGS]` | 36 |
+| `[/MODEL_SETTINGS]` | 37 |
+
+## Special-token counts per source
+
+| source | `<s>` | `</s>` | `[SYSTEM_PROMPT]` | `[/SYSTEM_PROMPT]` | `[INST]` | `[/INST]` | `[MODEL_SETTINGS]` | `[/MODEL_SETTINGS]` |
+|---|---|---|---|---|---|---|---|---|
+| vllm | 1 | 3 | 1 | 1 | 4 | 4 | 1 | 1 |
+| llamacpp | 1 | 3 | 1 | 1 | 4 | 4 | 1 | 1 |
+| mistralcommon | 1 | 3 | 1 | 1 | 4 | 4 | 1 | 1 |
+| hf | 1 | 3 | 1 | 1 | 4 | 4 | 1 | 1 |
+
+All four sources show: 1 `<s>`, 3 `</s>` (= one per assistant turn), 1 `[SYSTEM_PROMPT]` / `[/SYSTEM_PROMPT]` pair, 1 `[MODEL_SETTINGS]` / `[/MODEL_SETTINGS]` pair, 4 `[INST]` / `[/INST]` pairs (= one per user turn including the open-ended trailing turn).
+
+## First 16 input ids per source (for visual sanity)
+
+- **vllm**: `[1, 17, 4568, 1584, 42301, 2784, 55668, 1032, 1051, 1046, 1053, 1044, 1261, 43520, 26242, 11512]`
+- **llamacpp**: `[1, 17, 4568, 1584, 42301, 2784, 55668, 1032, 1051, 1046, 1053, 1044, 1261, 43520, 26242, 11512]`
+- **mistralcommon**: `[1, 17, 4568, 1584, 42301, 2784, 55668, 1032, 1051, 1046, 1053, 1044, 1261, 43520, 26242, 11512]`
+- **hf**: `[1, 17, 4568, 1584, 42301, 2784, 55668, 1032, 1051, 1046, 1053, 1044, 1261, 43520, 26242, 11512]`
+
+## Last 16 input ids per source
+
+- **vllm**: `[4176, 24897, 32196, 17616, 5079, 1034, 2, 3, 7493, 1395, 1032, 1050, 1043, 1050, 1063, 4]`
+- **llamacpp**: `[4176, 24897, 32196, 17616, 5079, 1034, 2, 3, 7493, 1395, 1032, 1050, 1043, 1050, 1063, 4]`
+- **mistralcommon**: `[4176, 24897, 32196, 17616, 5079, 1034, 2, 3, 7493, 1395, 1032, 1050, 1043, 1050, 1063, 4]`
+- **hf**: `[4176, 24897, 32196, 17616, 5079, 1034, 2, 3, 7493, 1395, 1032, 1050, 1043, 1050, 1063, 4]`
+
+## Files
+
+- `outputs/diagnosis_tokenization_{vllm,llamacpp,mistralcommon,hf}.txt`
+- `outputs/tok_ids_{vllm,llamacpp,mistralcommon,hf}.json`
+- `scripts/diagnosis_tokenization.py`
+
+## Conclusion
+
+Tokenization is fully consistent across all four implementations. **Cross off** "tokenization difference" as a hypothesis for the long-context degradation in llama-server / Q8_0 GGUF.
+
+Likely culprits remain: chat-template subtleties (whitespace), kernel-level numerics in llama.cpp (FP16 accumulation, Q8_0 quantization error compounding), KV-cache dtype, sampler defaults, or the GGUF metadata (RoPE scaling, etc.).