[lts_03_2025] CI: migrate to Microsoft-hosted agents (retired ACR/self-hosted pools break all PRs)

[Ewerton Scaboro da Silva (ewertons)]

Problem

The C-canary pipeline cannot run at all on lts_03_2025. Every Linux job pulls its container from csdkcontainerregistry.azurecr.io through the csdk-containers endpoint and runs on the sdk-c--* self-hosted pools. Both the registry and those pools were retired; main moved off them in #2701.

The first job of the Setup stage fails in Initialize containers:

Could not fetch access token for ACR. Please configure Managed Service Identity (MSI)
for Azure Container Registry with the appropriate permissions.

Setup gates everything, so all downstream stages are skipped and any PR targeting this branch is red before a line is compiled. Observed on build 162517 (PR #2746), which failed 14 seconds in. This is not transient and will not fix itself.

Change

Bring the CI configuration to parity with main: take main's version of build/.vsts-ci.yml and the build/test scripts it drives. This corresponds to the eight pipeline commits made on main since this branch diverged at 5a48d3fd8:

Commit	PR
c59df07db	Update CODEOWNERS and ubuntu-20.04 runner pool (deprecated) (#2677)
5f11fab75	Run Cleanup stage even if cancelled (#2682)
c5d43cf83	Replace pipeline with Microsoft-hosted agents (#2701)
891fa3074	Self-sufficient hosted-agent optimizations (#2718)
1317b5724	Make AZURE_LOCATION a runtime parameter (#2717)
64736930a	Replace Node.js check_submodules with PowerShell (#2721)
fa6ad5d5a	Disable NTLM in linux-setup-raspberry.sh (#2733)
c86b2d6a5	Make CodeQL opt-in per job (#2743)

Applied as an end state rather than eight cherry-picks: the intermediate commits conflict with each other in build_all/linux/run_tests.sh, and the only state that matters is the one main runs green today.

Every file changed is CI or test infrastructure. No product source, no submodule pointer changes, no version change.

Audit performed against this branch's tree

• All build_all/, jenkins/, testtools/ and samples/ paths the pipeline invokes exist here.
• The cmake options it passes resolve the same way as on main (enable_ipv6 is a c-utility option on both branches; -DpsX509IndividualEnrollments is a PowerShell parameter, not a cmake one).
• The YAML parses: 4 stages, 30 jobs, no remaining azurecr.io reference.
• The non-YAML script changes are the hosted-agent adaptations: ccache launcher, ccache added to setup.sh deps, and the gcc-12+ -Wmaybe-uninitialized false-positive entry dropped from jenkins/linux_c_option_test.sh. The .supp changes only add valgrind suppressions, which cannot mask a new failure mode into existence.

Caveats for the reviewer

• This PR validates itself. Azure Pipelines builds the PR merge commit, so the run on this PR exercises the new YAML. That run is the real test of this change — please read it rather than trusting the audit above.
• I could not build or run anything locally (no toolchain available in my environment), so I am relying entirely on this PR's CI.
• This branch's source is ~14 months older than main. main's pipeline was tuned against main's source, so it is possible some job surfaces a genuine, pre-existing LTS-only failure (older gcc-13 interactions, E2E drift). Those would be real signal, not fallout from this change, but they may need follow-up commits here.
• The E2E jobs provision live Azure resources via the iot hub sdk service connection. That connection is pipeline-scoped, not branch-scoped, so it should work here, but this is the first time it will be exercised from this branch.
• lts_03_2025 reaches its maintenance end date on 2026-10-07, so this restores a working gate for roughly six remaining weeks.

Why now

#2746 backports the DPS safe_add_size_t hardening (#2738, MSRC 128877) to this branch and cannot go green until this lands. Recommend merging this first, then rerunning #2746.

Relates to AB#39392647.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Ewerton Scaboro da Silva (ewertons)]

Status note: build 162518 has been sitting in notStarted for ~22 minutes, queued behind roughly 15 concurrent builds across the azure-iot-sdks org (horton-*, Java, C#, Python gates). That is hosted-agent capacity, not a problem with this change.

One thing already confirmed by this run, though: the stage graph now resolves to Requirements / Setup create_azure_resources / Tests / Cleanup and the jobs are queued for real agents. On build 162517 the old YAML died 14s in at Initialize containers before any job was scheduled. So the ACR blocker is gone; what remains is whether the jobs themselves pass against this branch's older source.

Please check the completed run before merging — as noted above, I could not build or test locally, and a genuine LTS-only failure is plausible given the ~14-month source gap.