Align the data of frozen arrays

[EgorBo]

Just opus-5 experiment:

Align the data of arrays on the frozen object heap so that vectorized code doesn't have its loads straddle cache lines.

FOH objects are laid out back to back, so where an array's data lands is arbitrary. This leaves a gap in front of arrays of at least 128 bytes so that their first element starts on a FOH_DATA_ALIGNMENT boundary - 64 bytes, or 16 on arm64 where Vector128 is the widest load. The gap is formatted as a free object so the segment stays walkable, and the object walk used by EnumModuleFrozenObjects skips those the same way gc_heap::walk_heap does.

Strings are left alone: their characters sit at an offset that is never 8 byte aligned, so padding can't put them on a cache line anyway.

The change is duplicated in the NativeAOT copy of FrozenObjectHeapManager, which needed a way to get at the free object MethodTable (RhGetFreeObjectMethodTable).

Cost is up to one alignment plus a free object header per array, and only for arrays that are frozen (static readonly), so it is bounded and paid once at startup.

Benchmark

See #131347 (comment)

the static readonly array is allocated on FOH, intel and amd targets reported 20% perf boost.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @anicka-net, @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

This PR changes CoreCLR GC LOH allocation / compaction to optionally insert additional per-object padding so that the array data start lands on a configurable (default 64-byte) alignment boundary, and adds a GC test to validate alignment for LOH arrays / strings across allocations, free-list reuse, and LOH compaction.

Changes:

• Add a new GC config (GCLOHDataAlignment) and plumb it through GC initialization to drive LOH “data alignment” padding behavior.
• Update LOH planning, relocation, and UOH allocation paths to compute and preserve variable padding (including encoding extra alignment padding into LOH relocation metadata during compaction).
• Add a new GC test project validating data alignment for LOH arrays and the expected skew for strings.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/tests/GC/Features/LOHCompaction/lohalignment.csproj	New test project for validating LOH data alignment behavior.
src/tests/GC/Features/LOHCompaction/lohalignment.cs	Adds alignment validation for arrays/strings across new alloc, free-list alloc, and LOH compaction.
src/coreclr/gc/relocate_compact.cpp	Applies variable LOH padding during compaction and asserts post-move alignment.
src/coreclr/gc/plan_phase.cpp	Computes alignment-aware LOH padding during planning and records extra pad for relocation.
src/coreclr/gc/interface.cpp	Initializes loh_data_alignment from the new GC config and validates accepted values.
src/coreclr/gc/gcpriv.h	Updates method signatures to pass alignment padding info where needed.
src/coreclr/gc/gcinternal.h	Adds alignment helpers/state and encodes extra alignment padding in LOH relocation metadata.
src/coreclr/gc/gcconfig.h	Introduces the GCLOHDataAlignment configuration key.
src/coreclr/gc/gc.cpp	Defines the new global loh_data_alignment.
src/coreclr/gc/allocation.cpp	Adjusts LOH/UOH allocation to reserve/track alignment padding and validate aligned results.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (3)

src/coreclr/vm/frozenobjectheap.cpp:276

• HasComponentSize() is true for strings and arrays (incl. multidimensional arrays). Using it here together with ARRAYBASE_SIZE means large strings / MD arrays can get padded using the SZARRAY data offset, which won’t actually align their payload as intended. Restrict to arrays and compute the data offset from the MethodTable base size (works for MD arrays too).

    if ((objectSize < FOH_MIN_SIZE_TO_ALIGN) || !type->HasComponentSize())
    {
        return 0;
    }

    const size_t dataAddr = (size_t)m_pCurrent + ARRAYBASE_SIZE;
    size_t pad = ALIGN_UP(dataAddr, (size_t)FOH_DATA_ALIGNMENT) - dataAddr;

src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Runtime/FrozenObjectHeapManager.cs:234

• Like CoreCLR, HasComponentSize also includes strings. Padding based on ArrayBaseSize will not align string payloads (and it also won’t match MD-array element offsets). If the intent is to align array element data, restrict to IsArray and compute the data offset from BaseSize (covers MD arrays).

                if ((objectSize < FOH_MIN_SIZE_TO_ALIGN) || !type->HasComponentSize)
                {
                    return 0;
                }

                nuint dataAddr = (nuint)m_pCurrent + ArrayBaseSize;
                nuint pad = (FOH_DATA_ALIGNMENT - (dataAddr & (FOH_DATA_ALIGNMENT - 1))) & (FOH_DATA_ALIGNMENT - 1);

src/coreclr/gc/gcinternal.h:1219

• The comment says this aligns “the elements of an array”, but poh_data_offset() is hardcoded to sizeof(ArrayBase), which is only the element-data offset for SZ arrays. Strings and MD arrays have different data offsets, and POH allocations aren’t type-specific at this layer. Please tighten the wording to reflect what is actually being aligned.

// Objects on the POH are padded so that their data - sizeof(ArrayBase) into the object,
// where the elements of an array start - is aligned on this boundary. 0 disables it.
extern size_t poh_data_alignment;

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

src/coreclr/vm/frozenobjectheap.cpp:273

• HasComponentSize() is true for both arrays and strings (see MethodTable::IsStringOrArray), but the padding math uses ARRAYBASE_SIZE, which is array-specific. This will also insert padding in front of large frozen strings without actually guaranteeing string buffer alignment, increasing memory usage with no clear benefit. Consider restricting this to arrays (or handle strings separately with the correct string data offset).

    if ((objectSize < FOH_MIN_SIZE_TO_ALIGN) || !type->HasComponentSize())
    {
        return 0;
    }

src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Runtime/FrozenObjectHeapManager.cs:230

• HasComponentSize includes strings as well as arrays, but ArrayBaseSize is an array layout constant. This would apply padding to large strings too (and won’t ensure string buffer alignment). If the intent is array-only alignment, restrict this check to type->IsArray.

                if ((objectSize < FOH_MIN_SIZE_TO_ALIGN) || !type->HasComponentSize)
                {
                    return 0;
                }

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/coreclr/vm/frozenobjectheap.cpp:18

• The PR title/description talk about aligning objects on the LOH to 64 bytes, but this change is in FrozenObjectHeap (FOH) and also sets FOH_DATA_ALIGNMENT to 16 on Arm64. Please reconcile scope and alignment guarantees (update PR title/description, or adjust the implementation/constant naming to match the stated goal).

#ifdef TARGET_ARM64
#define FOH_DATA_ALIGNMENT 16
#else
#define FOH_DATA_ALIGNMENT 64
#endif

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

src/coreclr/vm/frozenobjectheap.cpp:14

• FOH_DATA_ALIGNMENT is hardcoded to 64, but the PR description claims arm64 should use 16 (since Vector128 is the widest load there). Either update the description or make the alignment platform-conditional so the implementation matches the stated intent.

// The data of arrays of at least FOH_MIN_SIZE_TO_ALIGN bytes is put on a cache line. The
// threshold keeps the gap we leave in front of an array small relative to the array.
#define FOH_DATA_ALIGNMENT 64
#define FOH_MIN_SIZE_TO_ALIGN 512

src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Runtime/FrozenObjectHeapManager.cs:36

• FOH_DATA_ALIGNMENT is hardcoded to 64 here as well, which contradicts the PR description’s claim that arm64 should use 16-byte alignment. Either update the description or make this constant platform-conditional to keep CoreCLR and NativeAOT behavior aligned with the stated intent.

        // The data of arrays of at least FOH_MIN_SIZE_TO_ALIGN bytes is put on a cache
        // line. The threshold keeps the gap we leave in front of an array small relative
        // to the array.
        private const nuint FOH_DATA_ALIGNMENT = 64;
        private const nuint FOH_MIN_SIZE_TO_ALIGN = 512;

[EgorBo]

@EgorBot -arm -amd -intel

using BenchmarkDotNet.Attributes;
using System.Runtime.CompilerServices;
using System.Runtime.Intrinsics;

public class Benchs
{
    static readonly int[] data = new int[1024];

    [Benchmark]
    public int Bench() => SumSafe(data);

    [MethodImpl(MethodImplOptions.NoInlining)]
    public static int SumSafe(Span<int> span)
    {
        int sum = 0;
        if (Vector512.IsHardwareAccelerated)
        {
            var acc512 = Vector512<int>.Zero;
            while (span.Length > Vector512<int>.Count)
            {
                acc512 += Vector512.Create(span);
                span = span.Slice(Vector512<int>.Count);
            }
            sum = Vector512.Sum(acc512);
        }
        foreach (int value in span)
            sum += value;
        return sum;
    }
}

[MichalPetryka]

Would it be possible to align smaller objects to 16 instead?

[EgorBo]

Probably? didn't want to overcomplicate

[EgorBo]

I think it's likely not worth as is

1. FOH: CoreCLR almost never allocate arrays on it, only in one extremely fragile case. NAOT rarely uses the dynamic FOH manager. it mostly relies on the AOT one.
2. POH makes sense, a bit verbose change
3. LOH makes sense but a lot of hard-to-review changes.