fix(hcpctl): replace has_any with join in kusto must-gather query to avoid 10,000-argument limit

[Giulio Frasca (gmfrasca)]

What

Replace has_any(maestro_resource_ids) with a join kind=inner in the detailedInfraOrchestrationLogs KQL query template. Also add isnotempty(cluster_id) guards as defense-in-depth.

Why

The DEV Kusto environment produces >13,000 distinct maestro resource IDs per cluster over a 24-hour query window, exceeding Kusto's has_any() 10,000-element limit. This causes every e2e run to fail on the kusto_logs_present test with SEM0026: The arguments array exceeded the allowed limit.

The join kind=inner approach has no element limit and scales with environment growth. The isnotempty(cluster_id) guards prevent a secondary failure mode where toscalar() returns an empty string and has "" matches all rows.

Testing

• Unit tests: existing TestBuildMerged_MultipleChildren and TestBuildAllCustomQueries fixture tests updated and passing.
• KQL validation (make verify-kql) requires Kusto emulator; will be verified in CI.
• No new E2E tests; this fix unblocks the existing kusto_logs_present E2E test which is currently failing across all PRs.

Special notes for your reviewer

Investigated via ADX: confirmed ingestion latency is ~3-6 min (not the issue). The root cause is volume — a single cluster's has cluster_id filter against 24 hours of maestro logs in the shared DEV environment now yields 13,260 distinct resource IDs, exceeding has_any()'s hard limit.

PR Checklist

• PR is scoped to a single task (no mixed concerns)
• Title follows Conventional Commits format
• Summary explains the "Why" behind the change
• Linked to relevant ticket/issue
• Screenshots included (if graph/UI/metrics changes)
• Self-reviewed the diff
• CI/CD checks are passing (ignore Tide)
• Draft PR used for WIP (if applicable)
• Commit history is clean (rebased/squashed)
• Tricky code blocks are commented
• Specific reviewers tagged
• All comment threads resolved before merge

[Copilot]

Pull request overview

This PR updates an hcpctl must-gather Kusto query template to avoid ADX/Kusto’s has_any() 10,000-argument limit by switching the Maestro “associated resource IDs” filter to a join kind=inner, and adds isnotempty(cluster_id) guards to prevent empty-string matching from over-selecting logs.

Changes:

• Replace has_any(maestro_resource_ids) with a join kind=inner (maestro_resource_ids) on resourceId for Maestro associated-resource log selection.
• Add where isnotempty(cluster_id) guards to prevent broad matches when cluster_id is empty.
• Update golden fixtures for hcpctl query-building tests to reflect the new KQL.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
tooling/hcpctl/pkg/kusto/templates/custom/detailed_infra_orchestration_logs.kql.gotmpl	Switch Maestro associated-resource filtering from has_any to join, and add isnotempty(cluster_id) guards.
tooling/hcpctl/testdata/zz_fixture_TestBuildMerged_MultipleChildren.yaml	Update expected rendered KQL fixture for merged queries.
tooling/hcpctl/testdata/zz_fixture_TestBuildAllCustomQueries.yaml	Update expected rendered KQL fixture for “all custom queries” output.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gmfrasca
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• test/OWNERS
• tooling/hcpctl/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

[Giulio Frasca (gmfrasca)]

superceded by #6666

Looks like that PR also addresses the same issue and is a cleaner fix. Will close this pending CI/merge on that one.

[openshift-ci]

Giulio Frasca (@gmfrasca): The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-parallel	e8abf5d	link	true	/test e2e-parallel

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.