Lifetime keepalive silently reverts channel output encoding to the preset default (F32 grants lost fleet-wide)

[mijahauan]

Summary

set_channel_lifetime() — the keepalive every client must run to stop channels self-destructing — makes radiod silently revert a channel's output encoding to the preset default. A channel created with encoding=Encoding.F32 runs as F32 for one keepalive interval and is S16BE from then on. Nothing notices, because verify_channel() checks frequency but never encoding.

Net effect: every long-lived channel in the fleet is S16BE regardless of what was requested.

Reproduction (AC0G-B4, radiod on the HamSCI fork, ka9q-python current)

Scratch channel, preset="iq", sample_rate=12000, encoding=Encoding.F32 (=4), lifetime=200:

scratch ssrc=1605393904, requested F32(4)
  after creation                    : 4  (F32)      <- request IS honored
  -> sending set_channel_lifetime (the keepalive the recorder runs)
  after 4 keepalives                : 2  (S16BE)    <- silently reverted

The request is not being denied at creation — it is honored and then lost. Confirmed against live production channels on the same host, all long-running, all reverted:

freq=  2500000.0  sr=24000  encoding=2
freq= 10000000.0  sr=24000  encoding=2
freq= 45375000.0  sr=96000  encoding=2   (hf-timestd T6, requests F32)

Why the request can't simply be re-asserted at creation

RadiodControl.create_channel() already can't send encoding in the creation packet — that path is disabled in-tree:

# Encoding setting - NOT sending in main buffer anymore
# Radiod requires OUTPUT_ENCODING to be sent in a separate command after creation
# if encoding > 0:
#    encode_int(cmdbuffer, StatusType.OUTPUT_ENCODING, encoding)

so encoding is fired as a separate, unacknowledged follow-up command. That works — until the first keepalive undoes it.

Why this matters beyond precision

This is not only a resolution nit. F32 was chosen deliberately for the metrology chain: signal and noise vary substantially over a 24-hour period, and the requirement is to capture that without AGC, whose unpredictable scaling is unacceptable in scientific measurements. Silently dropping to fixed-point 16-bit with agc_enable=False, gain=0.0 reintroduces exactly the dynamic-range problem F32 was chosen to avoid — with no headroom strategy at either end.

Measured on B4's T6 channel: the signal sits at ~34 counts RMS out of ±32767, i.e. about 5 of 16 bits, for a quantization SNR of ~38–41 dB against a channel SNR of 51 dB. Quantization, not the RF, is currently the noise floor on that channel.

(For the T6 timing work in HamSCI/hf-timestd#7 this does not explain the displaced-peak bug — the 0.5 s integration averages quantization away entirely — but it does bound a proposed short-integration fine stage, which integrates ~5 samples rather than 48 000.)

Suggested fixes

1. Re-assert encoding on every keepalive — have set_channel_lifetime() carry OUTPUT_ENCODING (or re-send it immediately after), so the keepalive stops being lossy. Smallest change, fixes the fleet.
2. Verify encoding in verify_channel() / ensure_channel() so a lost grant is loud rather than silent. The current contract says "verifies the channel meets the requested specifications" while checking frequency only.
3. Consider whether radiod should reset output encoding on a LIFETIME-only command at all — arguably the upstream bug, but (1) makes clients correct regardless of radiod version.

A station-side workaround for anyone who needs F32 now: set encoding in radiod's preset/config so the default is F32, rather than relying on the per-channel grant surviving.

[mijahauan]

Measured impact on the science channels — this is not a rounding nit

Levels on AC0G-B4's live metrology channels (12 s capture, per-component RMS of the int16 I/Q actually on the wire, ~23:30 UTC):

channel              RMS     peak  bits used  quant SNR  headroom
-----------------------------------------------------------------
T6 BPSK_PPS         24.4       34        5.6      38.5dB    59.7dB
SHARED_15000         4.3       26        3.1      23.5dB    62.0dB
SHARED_10000         4.2       22        3.1      23.2dB    63.5dB
SHARED_5000          2.5       40        2.3      18.9dB    58.3dB
SHARED_2500          1.6       23        1.7      15.1dB    63.1dB
WWV_20000            0.7       13        0.4       7.1dB    68.0dB
WWV_25000            0.6        3        0.2       6.0dB    80.8dB

The broadcast channels are sitting in the bottom 1–4 LSBs of a 16-bit word, with 6–23 dB of quantisation SNR and 58–81 dB of headroom entirely unused. WWV_20000 and WWV_25000 are at sub-LSB RMS — those channels are quantisation-dominated, not noise-dominated.

Part of the low absolute level on 20/25 MHz is genuine (evening propagation), but the scaling is systemically wrong independent of that, and there is a clean control for it in the table: T6 is a locally injected signal at a known, constant, strong level, and it still only reaches 5.6 bits. If a local injection cannot fill the word, nothing will.

Consequence for this issue: with the F32 grant lost, agc_enable=False and gain=0.0, there is no mechanism left to place these signals anywhere sensible in the word. F32 was chosen precisely so that 24-hour signal/noise variation could be captured without AGC; the silent revert to fixed-point 16-bit removes both the range and the placement, and every day of GRAPE/dTEC data collected since is quantisation-limited at the low bands.

Station-side mitigation that does not wait on a library release

Adding a dedicated preset — [iq-f32], a copy of [iq] plus encoding = f32le — and pointing the metrology channel creation at it makes the preset default F32, so the keepalive's revert becomes harmless. It only affects channels using that preset, so it does not disturb S16-expecting clients (wspr/psk recorders), which matters: an encoding change under a client that assumes a sample width has caused real breakage here before (wspr-recorder's slot clock ran 2× fast on a stale grant).

That is a config change, testable on one station. The fix in this issue is still the right one — it makes the grant survive by construction and makes a lost grant loud instead of silent — but the preset gets the science channels out of the bottom two bits immediately.

[mijahauan]

Client-side fix on fix/keepalive-preserves-encoding — plus: this does not reproduce on the current pin

The revert is build-dependent

Re-ran the repro on AC0G-B4 against ka9q-radio 14d780af (the current ka9q_radio_compat pin; radiod binary built 2026-08-05, unrestarted since):

scratch ssrc, preset=iq, sr=12000, encoding=F32(4), lifetime=3000 frames (~60s)
  after creation        : 4 (F32)
  after  1 keepalive(s) : 4 (F32)
  after  4 keepalive(s) : 4 (F32)
  after 10 keepalive(s) : 4 (F32)
VERDICT: does NOT reproduce

Corroborated in production on the same host: seven long-lived channels (six metrology + the T6 HF-PPS channel) were set to F32 and held it for 2.5 hours with hf-timestd's lifetime keepalive thread running throughout (~420 keepalive log entries per 30 min).

The original report was against a HamSCI-fork radiod build, so the fault appears fork-specific or since fixed upstream. Worth noting for anyone chasing this: a first attempt to reproduce looked like a revert but was actually the channel expiring — lifetime=200 frames is only ~4 s at 50 Hz, so a keepalive interval of 4 s races the timer and the channel simply vanishes. Use a lifetime comfortably longer than the poll interval, and distinguish "channel gone" from "encoding changed" when reading the result.

Fix kept anyway

Suggestions (1) and (2) are implemented, deliberately unconditional rather than version-gated — the fleet does not run a single radiod build, and a silent downgrade costs ~11 bits of dynamic range while surfacing only as unexplained quantisation in the science data.

• set_channel_lifetime() re-asserts the encoding last requested for that SSRC (tracked from create_channel/set_output_encoding, overridable via a new encoding= argument, suppressed with encoding=0). Clients that never requested an encoding still send no tag, so pre-0f8b622 behaviour is unchanged.
• verify_channel() now compares granted against requested and fails loudly, naming both.

Live-validated end to end on 14d780af:

1. after creation                 : F32LE(4)
2. after 3 keepalives             : F32LE(4)      verify_channel() -> True
3. encoding forced to S16BE by another client
                                  : S16BE(2)      verify_channel() -> False
   "Channel N encoding mismatch: requested F32LE(4), radiod is serving S16BE(2)"
4. after next keepalive           : F32LE(4)      verify_channel() -> True

So the keepalive is self-healing and a lost grant is now loud.

Note on the B4 F32 symptom in the original report

The production channels listed as encoding=2 in this issue turned out to have a second, independent cause that was not the keepalive: hf-timestd's channel-create path tested if encoding == 'float' while its config spells the value "F32", so _set_float_encoding() never ran and the channel silently kept radiod's preset default. Fixed in HamSCI/hf-timestd@8d2656f. Suggestion (3) — whether radiod should reset encoding on a LIFETIME-only command — is still worth raising upstream if it can be reproduced on a current build.

[mijahauan]

Closing — fixed on main, with the mechanism corrected.

What shipped

• 5c67e66 fix(multi_stream): re-sends agc/gain/filter-edges/requested-encoding (audit F1, P0)
• 8e8c0ab fix(control): tune() records _requested_encoding so keepalives re-assert it (audit F4)

set_channel_lifetime() now carries OUTPUT_ENCODING alongside LIFETIME whenever a
requested encoding is on record for the SSRC (control.py:2114), which is fix option (1)
from the report. RadiodControl keeps _requested_encoding: Dict[int, int]
(control.py:908), populated by create_channel, tune() and the explicit setter.

Fix option (2) also landed: verify_channel() now checks encoding, not just frequency
(control.py:1620) and logs a named mismatch referencing this issue. Regression coverage
is in tests/test_keepalive_encoding.py.

Mechanism correction

The report's model — "radiod resets encoding on a LIFETIME-only command" — did not hold up.
Re-tested against ka9q-radio 14d780af on B4 (2026-08-07) it did not reproduce; what
was actually observed was ensure_channel reading back a stale encoding while racing its
own unacknowledged OUTPUT_ENCODING follow-up. The flip self-heals within ~90 s.

The fix was kept anyway and deliberately made unconditional rather than version-gated: one
extra TLV per keepalive makes the client correct regardless of which radiod it is talking
to, and that is not a version we control. So the outcome is right even though the diagnosis
in the report was not.

Suggestion (3) — whether radiod should reset encoding on a LIFETIME-only command — is moot
given the above and is not being carried upstream.