Skip to content

Multiple Helix work items fail on maccatalyst/tvos CoreCLR Release #126460

Description

@kotlarmilos

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1362636
Build error leg or test failing: Microsoft.Extensions.DependencyInjection.ExternalContainers.Tests.WorkItemExecution, Microsoft.Extensions.Hosting.Unit.Tests.WorkItemExecution, System.Diagnostics.TraceSource.Config.Tests.WorkItemExecution, System.Reflection.Emit.ILGeneration.Tests.WorkItemExecution, System.Reflection.Emit.Tests.WorkItemExecution, System.Reflection.Metadata.Tests.WorkItemExecution, System.Security.Cryptography.Tests.WorkItemExecution
Pull request: #125441

Affected CI jobs: build_MacCatalyst_x64_Release_AllSubsets_CoreCLR, build_maccatalyst_arm64_Release_AllSubsets_CoreCLR, build_tvos_arm64_Release_AllSubsets_CoreCLR, build_tvos_arm64_Release_AllSubsets_CoreCLR_RuntimeTests, build_tvos_arm64_Release_AllSubsets_Mono

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item (Microsoft\\.Extensions\\.DependencyInjection\\.ExternalContainers\\.Tests|Microsoft\\.Extensions\\.Hosting\\.Unit\\.Tests|System\\.Diagnostics\\.TraceSource\\.Config\\.Tests|System\\.Reflection\\.Emit\\.ILGeneration\\.Tests|System\\.Reflection\\.Emit\\.Tests|System\\.Reflection\\.Metadata\\.Tests|System\\.Security\\.Cryptography\\.Tests|Microsoft\\.Bcl\\.Cryptography\\.Tests|Microsoft\\.Bcl\\.Memory\\.Tests) in job [0-9a-f-]+ has failed\\. \\[/Users/runner/[^\\]]*sendtohelixhelp\\.proj\\]",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Summary

Multiple Helix work items fail on maccatalyst CoreCLR Release (both x64 and arm64) and tvos-arm64 CoreCLR/Mono Release, including the tvos-arm64 device queue OSX.15.Amd64.AppleTV.Open. Work items crash without producing individual test results (Build Analysis reports DeadLetter or BadExit).

Affected work items:

  • Microsoft.Extensions.DependencyInjection.ExternalContainers.Tests
  • Microsoft.Extensions.Hosting.Unit.Tests
  • System.Diagnostics.TraceSource.Config.Tests
  • System.Reflection.Emit.ILGeneration.Tests
  • System.Reflection.Emit.Tests
  • System.Reflection.Metadata.Tests
  • System.Security.Cryptography.Tests
  • Microsoft.Bcl.Cryptography.Tests (tvos-arm64 CoreCLR + Mono, DeadLetter, exit -3 / 21)
  • Microsoft.Bcl.Memory.Tests (tvos-arm64 CoreCLR, BadExit, exit 21)

Note: This issue uses timeline ErrorPattern matching because Helix console logs for this build expired. Once a fresh build reproduces this failure, update with specific error text from the Helix console log and set ExcludeConsoleLog to false.

Update (2026-06-17, build 1468533): added baseservices-exceptions, Exceptions, Microsoft.Bcl.Cryptography.Tests, and Microsoft.Bcl.Memory.Tests to the ErrorPattern; these recur on the tvos-arm64 OSX.15.Amd64.AppleTV.Open device queue (same crash family) but were not previously attributed by Build Analysis.

Update (2026-07-07): narrowed the ErrorPattern to Apple mobile CoreCLR library legs only. The previous pattern matched Work item <name> in job .* has failed on any leg, so it attributed unrelated desktop failures. The generic CoreCLR runtime-test suite names Exceptions and baseservices-exceptions run on every platform through src/tests/Common/helixpublishwitharcade.proj and matched desktop Linux and Windows runtime-test legs (for example build 1494680 on Windows and build 1495630 on Linux). The remaining library work items also run on all platforms. The pattern now requires the macOS runner path /Users/runner/... and the library helix project src/libraries/sendtohelixhelp.proj, which excludes Linux (/__w/) and Windows (D:\a\) hosts and the runtime-test project. Exceptions and baseservices-exceptions are dropped because they only appear on helixpublishwitharcade.proj, which cannot be distinguished from osx-x64 desktop runtime tests on a single log line. If tvos-arm64 runtime-test crashes need attribution, match a mobile-specific crash signature from the Helix console log instead.

Note

This issue was filed with the assistance of GitHub Copilot.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1496248
Error message validated: [Work item (Microsoft\.Extensions\.DependencyInjection\.ExternalContainers\.Tests|Microsoft\.Extensions\.Hosting\.Unit\.Tests|System\.Diagnostics\.TraceSource\.Config\.Tests|System\.Reflection\.Emit\.ILGeneration\.Tests|System\.Reflection\.Emit\.Tests|System\.Reflection\.Metadata\.Tests|System\.Security\.Cryptography\.Tests|Microsoft\.Bcl\.Cryptography\.Tests|Microsoft\.Bcl\.Memory\.Tests) in job [0-9a-f-]+ has failed\. \[/Users/runner/[^\]]*sendtohelixhelp\.proj\]]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/7/2026 12:34:07 PM UTC

Report

Build Repository Step Name Console log Pull Request
1526936 dotnet/runtime Send to Helix Log
1526929 dotnet/runtime Send to Helix Log #131375
1526817 dotnet/runtime Send to Helix Log #131375
1526281 dotnet/runtime Send to Helix Log
1526034 dotnet/runtime Send to Helix Log #131355
1523812 dotnet/runtime Send to Helix Log #131115
1523323 dotnet/runtime Send to Helix Log
1521521 dotnet/runtime Send to Helix Log #124721
1522836 dotnet/runtime Send to Helix Log
1521470 dotnet/runtime Send to Helix Log
1521339 dotnet/runtime Send to Helix Log #130622
1521102 dotnet/runtime Send to Helix Log
1521355 dotnet/runtime Send to Helix Log
1520933 dotnet/runtime Send to Helix Log
1520451 dotnet/runtime Send to Helix Log
1519700 dotnet/runtime Send to Helix Log
1519953 dotnet/runtime Send to Helix Log
1519822 dotnet/runtime Send to Helix Log
1519657 dotnet/runtime Send to Helix Log #130622
1519390 dotnet/runtime Send to Helix Log #131041
1519288 dotnet/runtime Send to Helix Log
1519262 dotnet/runtime Send to Helix Log #131080
1519186 dotnet/runtime Send to Helix Log
1519091 dotnet/runtime Send to Helix Log
1519179 dotnet/runtime Send to Helix Log #131132
1519130 dotnet/runtime Send to Helix Log #131049
1519105 dotnet/runtime Send to Helix Log #130512
1519068 dotnet/runtime Send to Helix Log #123920
1518978 dotnet/runtime Send to Helix Log #124721
1518979 dotnet/runtime Send to Helix Log
1518497 dotnet/runtime Send to Helix Log #126625
1518559 dotnet/runtime Send to Helix Log
1516926 dotnet/runtime Send to Helix Log
1516430 dotnet/runtime Send to Helix Log
1515988 dotnet/runtime Send to Helix Log
1515623 dotnet/runtime Send to Helix Log
1515046 dotnet/runtime Send to Helix Log #130622
1515072 dotnet/runtime Send to Helix Log
1515081 dotnet/runtime Send to Helix Log
1514458 dotnet/runtime Send to Helix Log
1512245 dotnet/runtime Send to Helix Log
3023960 dotnet-runtime Send to Helix Log #62940
1511237 dotnet/runtime Send to Helix Log
1510254 dotnet/runtime Send to Helix Log
3023144 dotnet-runtime Send to Helix Log #62940
1507089 dotnet/runtime Send to Helix Log
3022864 dotnet-runtime Send to Helix Log #62896
1509133 dotnet/runtime Send to Helix Log #128799
1508982 dotnet/runtime Send to Helix Log
3022498 dotnet-runtime Send to Helix Log #62896
1508374 dotnet/runtime Send to Helix Log
1506900 dotnet/runtime Send to Helix Log #128799
1507181 dotnet/runtime Send to Helix Log
1499665 dotnet/runtime Send to Helix Log
1505016 dotnet/runtime Send to Helix Log
1503537 dotnet/runtime Send to Helix Log #130469
1503536 dotnet/runtime Send to Helix Log #130468
3019331 dotnet-runtime Send to Helix Log #62802
1502636 dotnet/runtime Send to Helix Log
1502631 dotnet/runtime Send to Helix Log #130438
1502635 dotnet/runtime Send to Helix Log
1501158 dotnet/runtime Send to Helix Log #130402
1502303 dotnet/runtime Send to Helix Log
1501202 dotnet/runtime Send to Helix Log
1500901 dotnet/runtime Send to Helix Log #130328
1500996 dotnet/runtime Send to Helix Log #130168
1500869 dotnet/runtime Send to Helix Log #130340
1500658 dotnet/runtime Send to Helix Log
1499654 dotnet/runtime Send to Helix Log #130328
1500276 dotnet/runtime Send to Helix Log #130269
1500206 dotnet/runtime Send to Helix Log
1499832 dotnet/runtime Send to Helix Log #130341
1500024 dotnet/runtime Send to Helix Log
1495257 dotnet/runtime Send to Helix Log
1499581 dotnet/runtime Send to Helix Log #130360
1499030 dotnet/runtime Send to Helix Log
1498648 dotnet/runtime Send to Helix Log #130168
1498414 dotnet/runtime Send to Helix Log
1498243 dotnet/runtime Send to Helix Log #130269
1497883 dotnet/runtime Send to Helix Log #130168
1496784 dotnet/runtime Send to Helix Log #130292
1496783 dotnet/runtime Send to Helix Log #130168
1496347 dotnet/runtime Send to Helix Log
1496248 dotnet/runtime Send to Helix Log #124721
1496266 dotnet/runtime Send to Helix Log
1495952 dotnet/runtime Send to Helix Log
1495630 dotnet/runtime Send tests to Helix (Unix) Log #130142
1493534 dotnet/runtime Send tests to Helix (Unix) Log #130141
1495180 dotnet/runtime Send to Helix Log
1495192 dotnet/runtime Send to Helix Log
1495016 dotnet/runtime Send to Helix Log
1494859 dotnet/runtime Send to Helix Log #130232
1494829 dotnet/runtime Send to Helix Log
1494680 dotnet/runtime Send tests to Helix (Windows) Log #129992
1494534 dotnet/runtime Send to Helix Log
1494467 dotnet/runtime Send to Helix Log #130221
1494308 dotnet/runtime Send to Helix Log
1494206 dotnet/runtime Send to Helix Log #130156
1494157 dotnet/runtime Send tests to Helix (Unix) Log #130203
1494160 dotnet/runtime Send to Helix Log
Displaying 100 of 157 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
3 33 157

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions