labgrid-client: Add an internal console

[sjg20]

The microcom-based console used by labgrid-client has a few limitations:

• console output is lost between when the board is reset and microcom connects;
• microcom has no txdelay handling, so boards can miss characters when the client is sending input quickly (e.g. pasted test commands);
• between labgrid-client console starting and microcom attaching, a few characters can echo back to the caller, which can break test systems which rely on output.

For many use cases, microcom is more than is needed. This PR adds a small internal terminal that solves the above. Enable it with:

• labgrid-client console -i (or --internal), or
• LG_CONSOLE=internal in the environment, so existing scripts can opt in without being rewritten

Press Ctrl-] twice (quickly) to exit.

Refactoring done along the way:

• Move all terminal handling into a new file labgrid/util/term.py, keeping client.py thinner
• Split _check_allowed() into a side-effect-free is_allowed() plus the existing raise-on-failure helper, so the new terminal loop can poll without try/except
• Flush the console device only when needed, so console data is preserved across strategy-driver → interactive transitions when the internal terminal is in use

What is the feature used for? Console access in environments where microcom is unavailable or its limitations bite: U-Boot tests, post-reset capture, and any flow that needs txdelay.

How does labgrid benefit? No external dependency for a basic console, plus clean handoff between strategy-driver use and interactive use without losing data.

How was it verified? Used the internal console against a QEMU target end-to-end. The unit tests under tests/test_term.py and tests/test_client_unit.py cover both the external (microcom/telnet) and internal paths: keystroke handling, txdelay, exit-char deadline, logfile output, listen-only mode and check-allowed termination.

No hardware-specific changes.

Checklist

• Documentation for the feature
• Tests for the feature
• Add a section on how to use the feature to doc/usage.rst
• PR has been tested
• Man pages have been regenerated

[codecov]

Codecov Report

❌ Patch coverage is 82.63473% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.5%. Comparing base (db1cba3) to head (7571145).
⚠️ Report is 78 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
labgrid/remote/client.py	27.0%	27 Missing ⚠️
labgrid/util/term.py	98.4%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #1870      +/-   ##
=========================================
+ Coverage    46.0%   61.5%   +15.4%     
=========================================
  Files         180     183       +3     
  Lines       14439   14989     +550     
=========================================
+ Hits         6654    9224    +2570     
+ Misses       7785    5765    -2020     

Flag	Coverage Δ
3.10	61.4% <82.6%> (+15.4%)	⬆️
3.11	61.4% <82.6%> (+15.4%)	⬆️
3.12	61.4% <82.6%> (+15.3%)	⬆️
3.13	61.4% <82.6%> (+15.4%)	⬆️
3.14	61.4% <82.6%> (+15.3%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[Emantor]

While I do agree with the concept of an internal console for labgrid-client, my expectation is more in line to use existing modules like Python Prompt Toolkit instead of writing our own terminal handling from scratch.

[sjg20]

While I do agree with the concept of an internal console for labgrid-client, my expectation is more in line to use existing modules like Python Prompt Toolkit instead of writing our own terminal handling from scratch.

OK I will take a look

[sjg20]

I've switched the terminal handling over to prompt_toolkit.

It now uses create_input() for the terminal, inp.raw_mode() to put it into raw mode and inp.attach() to watch the keyboard from the event loop, so labgrid no longer touches termios itself and no longer polls stdin.

But...two things worth mentioning.

1. The keystrokes are still read with os.read(), not read_keys(). A console has to be a transparent byte stream: whatever the user types must reach the board exactly as typed. read_keys() runs stdin through PosixStdinReader (which decodes it to str) and then Vt100Parser, which turns escape sequences into KeyPress objects and buffers incomplete ones. That is exactly right for a REPL, but it would mean arrow keys, function keys, Ctrl-C and any 8-bit data no longer arrive at the board unchanged. So I use prompt_toolkit for the terminal (raw mode, event-loop hookup) and read the raw bytes myself.

If you'd rather I went via read_keys() and reconstructed the bytes from KeyPress.data, I can, but it seemed like a lot of work to undo something we don't want in the first place.

2. Doing it this way fixed two bugs - prompt_toolkit's raw_mode() clears more than my hand-coded version did: as well as ECHO/ICANON/ISIG, it clears IXON/IXOFF and ICRNL/INLCR/IGNCR. So previously Ctrl-S and Ctrl-Q were being eaten by the tty as flow control instead of reaching the board, and CR was being translated to NL on the way in. Both are now correct.

Watching stdin from the event loop also removes the busy-poll: the old loop called time.sleep(0.001) from inside an async function, which blocked the event loop.

prompt_toolkit is added to the dependencies in pyproject.toml.

[Emantor]

Looking at this PR I expected the console abstraction to also use windowing from prompt-toolkit, with the future goal of adding an inline command prompt to run client commands directly from the console as well. I don't know if you are willing to look into that, thanks for reworking this so far.

[sjg20]

Thanks for taking a look.

I did consider prompt-toolkit's full-screen/windowing support, but went the other way deliberately: the internal console aims to be a transparent byte pipe, so that escape sequences, control characters and interactive programs on the board (editors, menuconfig, etc.) work exactly as they do with microcom. A prompt-toolkit Application takes over the terminal and re-renders it, which makes that transparency hard to keep - every byte from the board would need to be interpreted or routed around the renderer.

That said, I like the idea of an inline command prompt and I think it can be layered on top of this PR rather than replacing it. The run() loop already owns both streams, so a follow-up could watch for a hotkey (say Ctrl-] once, since exit is Ctrl-] twice) and switch into a prompt-toolkit session to run a client command, then drop back to the raw console. That keeps the transparent path as the default and only brings in windowing when the user asks for it.

So yes, I am happy to look into that as a follow-up, if it is OK to land the basic console first? I can file an issue to track the inline-prompt idea so it does not get lost.