re-visitng color accurate HEVC VA-API

[alatteri]

Hello UG people,

Recently, with the help of CODEX, I prototyped a stand alone method to get 10bit RGB 4:4:4 HEVC using hardware acceleration on Intel, NUC12+ specifically. The method works, and the color accuracy is greats since there is no RGB<->YUV conversion. It maintains Full Range and Rec709 colorspace. This is cheap, small and power efficient hardware that allows realtime 4K color accurate compression/decompression, no longer needing expensive and big (compared to NUC) machines with NVDIA GPU. Even this tiny cube would be able to compress/decomress 4K 10bit 444 in realtime since it has N150 proc which is supported.

I have validated the color accuracy on scopes, and the hardware compression is highly performant.

I would love to see this implemented in UG. The workflow is below, and if you want I'm happy to post the source code. It was built around Gstreamer extensions, with just a minimal amount of code required for DeckLink R12L and doing the R12L->Y410 conversion, so hopefully the ideas can apply to UG.

The video pipeline uses Intel VA-API hardware encoding and decoding—not Intel QSV.

DeckLink SDI input
R12L: packed, full-range, 12-bit RGB 4:4:4
↓
Scale RGB from 12 to 10 bits
round(value × 1023 / 4095)
↓
Pack RGB into a Y410 surface
No RGB-to-YUV matrix conversion
↓
Intel VA-API HEVC encoder
vah265lpenc
Main 4:4:4 10 profile
↓
Intel VA-API HEVC decoder
vah265dec
Main 4:4:4 10 output as Y410
↓
Recover the three 10-bit RGB components
↓
Scale RGB from 10 to 12 bits
round(value × 4095 / 1023)
↓
DeckLink SDI output
R12L: packed, full-range, 12-bit RGB 4:4:4

Y410 is a packed 32-bit-per-pixel format with three 10-bit component fields. We use those fields as RGB containers:

Y410 U/Cb field ← Blue
Y410 Y field ← Green
Y410 V/Cr field ← Red

There is no RGB-to-YUV color matrix. Green is not calculated as luminance, and red and blue are not converted into color-difference signals. The Intel hardware sees a Y410 surface, so the HEVC stream identifies these as Y, Cb, and Cr, but the values actually represent G, B, and R.

The Intel VA-API encoder compresses them as three full-resolution HEVC 4:4:4 components:

HEVC Y component carries Green
HEVC Cb component carries Blue
HEVC Cr component carries Red

All three components are 3840×2160 at 10-bit precision. There is no 4:2:2 or 4:2:0 chroma subsampling. They are three coded components within one synchronized HEVC stream—not three independent streams or necessarily three separate memory buffers.

The R12L ↔ Y410 conversion normally runs on the Intel GPU using a custom OpenCL kernel.

The encoder is vah265lpenc, meaning the Intel VA-API low-power hardware HEVC encoder. The receiver uses the VA-API vah265dec hardware decoder. QSV elements are not used because the required Main 4:4:4 10-bit path was not available through the tested QSV implementation.

• The pipeline explicitly sets this GStreamer raw-video colorimetry:

colorimetry=1:0:5:1

In GStreamer’s order, that means:

Range: 1 = full range
Matrix: 0 = unknown/unspecified
Transfer: 5 = BT.709
Primaries: 1 = BT.709

So we explicitly declare:

• Full-range component values
• BT.709 transfer characteristic
• BT.709 color primaries
• No declared YCbCr matrix

The “matrix unknown” value is intentional in the current identity-mapped RGB path: we are not applying a BT.709 RGB-to-YCbCr matrix. However, it is important to be precise: we are not currently signalling an explicit HEVC identity matrix coefficient either—we mark the matrix as unspecified.

For the encoded stream, we explicitly constrain only:

Codec profile: HEVC Main 4:4:4 10
Stream format: byte-stream
Alignment: access unit

We do not directly set HEVC VUI properties such as:

• video_full_range_flag
• colour_primaries
• transfer_characteristics
• matrix_coefficients

Instead, vah265lpenc receives the raw Y410 caps and may derive the HEVC VUI from them. The current code does not independently inspect the resulting HEVC bitstream to prove exactly which VUI fields Intel writes.

At the DeckLink boundaries:

• R12L is defined by the DeckLink SDK as full-range 12-bit RGB.
• The private DeckLink caps explicitly carry color-range=full.
• Receiver output is configured as R12L, 4:4:4, single-link SDI.
• We do not add HDR mastering metadata, MaxCLL/MaxFALL, PQ, or HLG metadata for this SDR Rec.709 path.

Therefore, the exact current situation is:

Metadata Current setting
━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Numeric range Full
───────────────── ────────────────────────────
Primaries BT.709
───────────────── ────────────────────────────
Transfer BT.709
───────────────── ────────────────────────────
Matrix Unspecified
───────────────── ────────────────────────────
HEVC profile Main 4:4:4 10
───────────────── ────────────────────────────
HDR metadata None
───────────────── ────────────────────────────
DeckLink format Full-range 12-bit RGB R12L

One correction to my earlier answer: 1:0:5:1 does not mean an explicitly signalled identity matrix. It means the matrix is unspecified. The pixel mapping itself is identity-style—B→Cb slot, G→Y slot, R→Cr slot—but the HEVC matrix metadata is not currently explicitly marked as identity.

[alatteri]

This is CODEX's analysis of using this technique within UG:

## Integrating our identity-RGB HEVC technique

The cleanest integration would extend UltraGrid’s existing libavcodec compressor and decompressor rather than introduce GStreamer into UltraGrid.

### Encoder changes

Add a selectable conversion mode, for example:

-c libavcodec:codec=H265:encoder=hevc_vaapi:rgb-identity

In that mode, add a new conversion:

R12L → YUV444P10LE identity mapping

For each pixel:

Y10  = round(G12 × 1023 / 4095)
Cb10 = round(B12 × 1023 / 4095)
Cr10 = round(R12 × 1023 / 4095)

This should be a separate converter, not a modification to UltraGrid’s existing r12l_to_yuv444p10le(), because the existing conversion is correct for ordinary interoperable YCbCr workflows.

Then change the VA-API encoder setup so it does not always create an NV12 hardware-frame pool. When identity RGB is selected, it should request a VA-API surface compatible with 10-bit 4:4:4—typically represented through FFmpeg as YUV444P10LE software frames uploaded into a corresponding VA-API 4:4:4 10-bit surface.

The implementation must query the VA-API hardware-frame constraints instead of assuming the driver supports it. The Intel driver on our machines has already demonstrated this capability through GStreamer’s Y410 VA-API path, but FFmpeg’s VA-API mapping must still be tested independently.

The encoder should then be configured for:

Codec:      HEVC
Encoder:    hevc_vaapi
Profile:    Range Extensions / Main 4:4:4 10
Bitrate:    60 Mb/s
GOP:        24 frames
B-frames:   0
Async depth: minimum practical value

### Decoder changes

The receiver should recognize the identity-mapped stream and request:

HEVC VA-API decode
      ↓
10-bit 4:4:4 decoded components
      ↓
identity Y/Cb/Cr → R/G/B extraction
      ↓
R12L

Add a matching conversion:

YUV444P10LE identity mapping → R12L

For each pixel:

R12 = round(Cr10 × 4095 / 1023)
G12 = round(Y10  × 4095 / 1023)
B12 = round(Cb10 × 4095 / 1023)

That R12L data can be written directly into the DeckLink display buffer returned by display_decklink_getf().

### Identifying the special stream

An identity-mapped stream will display with dramatically incorrect colors in an ordinary HEVC decoder. The stream therefore needs an explicit marker.

UltraGrid already inserts a private HEVC SEI message carrying information about the original UltraGrid pixel format. That mechanism should be extended with an identity-RGB identifier, for example:

Original format: R12L
Component mapping: RGB_IDENTITY_GBR
Source depth: 12
Coded depth: 10
Range: full
Primaries: BT.709
Transfer: BT.709

A patched UltraGrid decoder would detect this marker and select the identity conversion. An unpatched decoder would continue using its normal YCbCr conversion, avoiding silent changes to existing behavior.

### Color metadata

For this mode, the HEVC VUI should explicitly signal:

video_full_range_flag          = 1
colour_primaries               = 1   (BT.709)
transfer_characteristics       = 1   (BT.709)
matrix_coefficients            = 0   (GBR/identity)

This would actually improve on our current GStreamer implementation, which marks the matrix as unspecified rather than explicitly signalling matrix coefficient zero.

UltraGrid’s current helper marks ordinary YUV formats as BT.709 limited range. Identity mode must override that behavior and explicitly set:

codec_ctx->color_range     = AVCOL_RANGE_JPEG;
codec_ctx->colorspace      = AVCOL_SPC_RGB;
codec_ctx->color_primaries = AVCOL_PRI_BT709;
codec_ctx->color_trc       = AVCOL_TRC_BT709;

The resulting VPS/SPS/VUI should be parsed in a test to confirm that the Intel encoder actually emits those values.

## GPU conversion options

There are two practical stages.

First, implement the identity conversions on the CPU inside UltraGrid’s existing conversion tables. This is the simplest way to validate correctness, interoperability and bit-exact channel mapping.

Then port our OpenCL kernels as optional optimized converters:

DeckLink R12L in CPU memory
      ↓ OpenCL upload
GPU unpack + 12→10 scaling + component separation
      ↓
VA-API-compatible 4:4:4 10-bit input

and:

VA-API decoded components
      ↓
GPU component extraction + 10→12 scaling + R12L packing
      ↓ OpenCL download
DeckLink display buffer

[alatteri]

In further discussions with CODEX, we've decided converting back to R12L is un-neccesary, and the best path is Y410->R10K (remap RGB not YUV conversion), then use the param r10k-full-range. Since th compressed stream is 10bit anyway, we gain nothing by going to R12L and that just increases load.