Add Quickdial plugin for STT and TTS#6490
Conversation
β¦n header on WS (addresses review)
| try: | ||
| async with self._tts._ensure_session().ws_connect(url, headers=ws_headers) as ws: | ||
| await asyncio.gather(_send(ws), _recv(ws)) | ||
| if segment_open: | ||
| output_emitter.end_segment() |
There was a problem hiding this comment.
π‘ Streaming voice synthesis can hang after the reply finishes
The realtime text-to-speech stream never tells the server that its input is complete and its listening loop (_recv at livekit-plugins/livekit-plugins-quickdial/livekit/plugins/quickdial/tts.py:236-258) only stops when the network socket closes, so after all text is sent and audio is received the stream can wait forever.
Impact: On a persistent connection the streaming synthesis task never completes, so a turn using the streaming path can stall indefinitely.
Why asyncio.gather never returns on a reused socket
In SynthesizeStream._run, await asyncio.gather(_send(ws), _recv(ws)) (livekit-plugins/livekit-plugins-quickdial/livekit/plugins/quickdial/tts.py:262) waits for BOTH coroutines. _send (tts.py:210-232) drains self._input_ch, sends one request per flush, and returns once the input channel closes β but it never signals end-of-input to the server. _recv (tts.py:236-258) loops on ws.receive() and only breaks on WSMsgType.CLOSED/CLOSE/CLOSING; a server "end" event merely closes the segment (segment_open = False) and keeps looping. For a persistent streaming socket (the whole reason to aggregate per-flush and reuse one connection) the server does not close after "end", so _recv blocks forever and gather never resolves, leaving _run pending.
The repo's analogous plugin (livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/stt.py:349-365) avoids this by using asyncio.wait(..., return_when=FIRST_COMPLETED) followed by utils.aio.gracefully_cancel(...) rather than gather. Note this path is not the default (capabilities declare streaming=False), so it only affects callers using stream() directly.
Prompt for agents
In SynthesizeStream._run in livekit-plugins/livekit-plugins-quickdial/livekit/plugins/quickdial/tts.py, the WebSocket receive loop (_recv) only terminates when the socket emits CLOSED/CLOSE/CLOSING, and asyncio.gather(_send, _recv) waits for both coroutines. On a persistent streaming connection the server does not necessarily close the socket after emitting the final 'end' event, so _recv blocks on ws.receive() forever and _run never returns. Additionally, _send never notifies the server that no more text will arrive. Consider (1) sending an explicit end-of-input/flush message to the server after the input channel closes so it knows the request is complete, and (2) coordinating the two coroutines so the run finishes once synthesis for the sent text is complete β e.g. follow the pattern used in livekit-plugins-gnani/.../stt.py which uses asyncio.wait(..., return_when=FIRST_COMPLETED) plus utils.aio.gracefully_cancel(...) instead of asyncio.gather. The same concern applies to SpeechStream._run in the sibling stt.py.
Was this helpful? React with π or π to provide feedback.
Adds a
livekit-plugins-quickdialplugin providing STT and TTS via Quickdial β a CPU-optimized, real-time voice API (no GPU, priced per character). It follows the existing plugin structure (mirrorslivekit-plugins-gnani) and registers viaPlugin.register_plugin.Files
livekit-plugins/livekit-plugins-quickdial/β package (__init__,stt,tts,models,log,version,py.typed,pyproject.toml,README.md)livekit-agents/pyproject.tomlβ added thequickdialoptional-dependency extra.github/next-release/changeset-quickdial-plugin.mdβ changesetexamples/voice_agents/quickdial_agent.pyβ runnable exampleTwo coordination items π
livekit-plugins-quickdialon PyPI (from our standalone build). Happy to transfer the PyPI project to LiveKitβs org so your release bot can publish it β just let me know the account/org to add as owner (or your preferred process).Notes: I was unable to regenerate
uv.lockin my environment β happy for a maintainer to regenerate or point me at the right command. Likewise glad to add unit tests to match your harness, and to adjust anything (copyright headers, versioning) to your conventions. The standalone source also lives at https://github.com/samay-ai/livekit-plugins-quickdial. Thanks!