From 63f4a4bf569b696a7d3b25cd3d32f2de1450c3a3 Mon Sep 17 00:00:00 2001 From: Petr Onderka Date: Fri, 24 Jul 2026 14:22:11 +0200 Subject: [PATCH] Improve CI scan duplicate detection Replace the nonexistent Build Analysis attachment with the GitHub check and require verbatim searches for complete test identifiers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d555b634-18ce-458d-a831-096e0b0c87b3 --- .../references/ci-scan.instructions.md | 22 +++++++++++++++++-- .github/skills/ci-scan/references/playbook.md | 9 +++++++- 2 files changed, 28 insertions(+), 3 deletions(-) diff --git a/.github/skills/ci-scan/references/ci-scan.instructions.md b/.github/skills/ci-scan/references/ci-scan.instructions.md index 4badd16283..736808e70c 100644 --- a/.github/skills/ci-scan/references/ci-scan.instructions.md +++ b/.github/skills/ci-scan/references/ci-scan.instructions.md @@ -44,7 +44,7 @@ other section is identical so the two scanners behave the same way. | Pipeline | `MachineLearning-CI`, definition id **167** | | Branch scanned | `refs/heads/main` | | Helix present | **yes** (`Send to Helix` legs route xunit work items through `helix.dot.net`) | -| Build Analysis present | **yes** (Arcade attaches `Build_Analysis_KnownIssues_v1`; `Known Build Error` issues feed it) | +| Build Analysis present | **yes** (the GitHub `Build Analysis` check reports matches from `Known Build Error` issues) | | Issue model | `Known Build Error` KBE issues consumed by Build Analysis | | Issue labels | `Known Build Error`, `blocking-clean-ci`, and `Build` for compile-time breaks | | Title prefix | `[ci-scan] ` | @@ -80,7 +80,19 @@ Use these patterns consistently during local execution. - Timeline: `/builds/{id}/timeline?api-version=7.1` returns a flat `records[]`; reconstruct the `Stage -> Phase -> Job -> Task` tree via `parentId`. A failed record with a non-null `log.id` is a leaf worth reading. - Task log: each leaf record exposes `log.url`; `curl -s "$log_url"` returns plain text. - **Helix REST** **(profile: Helix)** - `https://helix.dot.net/api/jobs/{jobId}/workitems?api-version=2019-06-17`. Each work item has `Name`, `State`, `ExitCode`, `ConsoleOutputUri`. A work item failed when `ExitCode != 0` or `State == "Failed"`. Extract `{jobId}` from a `Send to Helix` task log line `Sent Helix Job: `. -- **Build Analysis attachment** **(profile: Build Analysis)**, best-effort dedupe - `https://dev.azure.com/dnceng-public/public/_apis/build/builds/{id}/attachments/Build_Analysis_KnownIssues_v1?api-version=7.1`. A `404` means none is attached; treat that as "no known-issue match" and continue, never as an error. +- **Build Analysis GitHub check** **(profile: Build Analysis)**, best-effort dedupe. + Read the source SHA from the AzDO build, then query the commit's check runs: + + ```bash + source_sha='' + gh api "repos/dotnet/machinelearning/commits/${source_sha}/check-runs" \ + --jq '.check_runs[] | select(.name == "Build Analysis" and .status == "completed") | {id, text: .output.text}' + ``` + + If the report links the source build's failure to an existing issue, record + `existing-issue #`. Reports omit known errors when they exceed GitHub's + output limits, so absence is not proof that Build Analysis did not match. + Always continue with the exact issue searches below after a miss. ## Source build selection and follow-up gate @@ -183,6 +195,12 @@ one query: tracker; a freshly closed "fixed" issue means do not re-file unless the failure clearly recurs after the fix. +When a failure includes a complete test method identifier, search that +identifier verbatim before deriving a shorter stem. Do not truncate +underscore-delimited identifiers (for example, keep +`AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async`); +GitHub search does not reliably prefix-match them. + On a confirmed match record `existing-issue #` and draft nothing. Verify the candidate actually matches the same test/family AND the same failing line before trusting it; a coincidental substring hit is not a match. When two candidates look equally plausible and you cannot disambiguate, record diff --git a/.github/skills/ci-scan/references/playbook.md b/.github/skills/ci-scan/references/playbook.md index ac4a0c40b3..fee0eebcf4 100644 --- a/.github/skills/ci-scan/references/playbook.md +++ b/.github/skills/ci-scan/references/playbook.md @@ -65,7 +65,14 @@ For each stable signature, run the per-signature follow-up gate from [Source bui ## Step 5 - Dedup -Run [Search existing issues](ci-scan.instructions.md#search-existing) (cross-run, the Build Analysis dedup path) and then the [Same-run dedup cache](ci-scan.instructions.md#dedup-cache) (signature-only key). On a match draft nothing and record `existing-issue #` or `dup of drafted-issue draft- earlier in this run`. +Inspect the source commit's `Build Analysis` GitHub check as described in +[Data sources](ci-scan.instructions.md#data-sources), then run +[Search existing issues](ci-scan.instructions.md#search-existing) and the +[Same-run dedup cache](ci-scan.instructions.md#dedup-cache) (signature-only +key). A positive Build Analysis link or matching issue suppresses the draft; +after a Build Analysis miss, always continue the exact issue searches because +the rendered check can be truncated. Record `existing-issue #` or +`dup of drafted-issue draft- earlier in this run`. ## Step 6 - Draft the KBE