fix(voice): expose speech handle generation errors#1965
fix(voice): expose speech handle generation errors#1965rosetta-livekit-bot[bot] wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: 8459a80 The changes in this PR will be included in the next version bump. This PR includes changesets to release 35 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| const speechError = this.lastSpeechHandle.exception(); | ||
| if (speechError) { | ||
| this.doneFut.reject(speechError); | ||
| return; | ||
| } |
There was a problem hiding this comment.
🔍 Error propagation only checks the last-completing speech handle
In RunResult._markDone() (agents/src/voice/testing/run_result.ts:246), the new exception() check only inspects this.lastSpeechHandle, which is set to whichever handle last triggered _markDoneIfNeeded (agents/src/voice/testing/run_result.ts:226-227). If multiple speech handles are watched and the errored handle completes before a non-errored one, the error is silently lost because lastSpeechHandle will point to the non-errored handle. In practice this is unlikely since typical runs involve a single speech handle, but multi-handle scenarios (e.g. tool-triggered child handles) could mask errors.
Was this helpful? React with 👍 or 👎 to provide feedback.
| return; | ||
| } | ||
|
|
||
| const generationEvent = await generationPromise; | ||
| let generationEvent: GenerationCreatedEvent; | ||
| try { | ||
| generationEvent = await generationPromise; | ||
| } catch (error) { | ||
| const cause = asError(error); | ||
| this.logger.error(cause, 'failed to generate a reply'); | ||
| speechHandle._markDone(cause); | ||
| this.agentSession._updateAgentState('listening'); | ||
| return; | ||
| } |
There was a problem hiding this comment.
🔍 Reachability of try/catch depends on ThrowsPromise.race semantics
In realtimeReplyTask, waitIfNotInterrupted at agents/src/voice/agent_activity.ts:3900 races generationPromise against the interrupt future using ThrowsPromise.race. If generationPromise rejects and ThrowsPromise.race propagates that rejection (like standard Promise.race), the error would throw from line 3900 and never reach the try/catch at lines 3907-3915. The try/catch is only reachable if ThrowsPromise.race treats rejections as settled values rather than propagating them. The existing pre-PR code also did await generationPromise after waitIfNotInterrupted without a try/catch, suggesting ThrowsPromise does NOT propagate rejections from race — otherwise the old code would have had unhandled errors in production. This is worth confirming against the @livekit/throws-transformer implementation.
(Refers to lines 3900-3915)
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
SpeechHandle.exception()to expose stored speech generation errors without makingawait handlerejectgenerateReplyfailures as completed speech handles carrying the errorRunResultTesting
pnpm test agents/src/voice/speech_handle.test.tspnpm test agents/src/voice/testing/run_result.test.tspnpm build:agentscue-clitext-mode runtime validation with a temporary local FakeLLM worker, assertingconversation_item_added(.item.message.role="ASSISTANT")Ported from livekit/agents#6304
Original PR description
Fixes #6224. Alternative approach to #6226.
Problem: when a realtime
generate_replyorsayfails (e.g.RealtimeError("generate_reply timed out.")), the error is only logged inside_realtime_reply_task, so application code has no way to detect the failure and react.Why not raise from
await speech_handle(the approach in #6226): most speech handles are never awaited (e.g. replies generated from user turns), so setting the exception on the done future triggers "Future exception was never retrieved" warnings for every failure. It also changes behavior for all existing code awaiting handles (including framework-internal awaits likeawait speech_handle.interrupt()in_user_turn_completed_task), and diverges from the pipeline path, which intentionally keeps LLM failures off the done future (see_on_llm_task_done) and only raises throughsession.run().Changes:
SpeechHandle._mark_done(error=...)records the error in an_errorattribute; the done future always resolves normally, soawait handle/wait_for_playout()never raise.SpeechHandle.exception(), mirroringasyncio.Future.exception(): returns the error once the handle is done (Noneif it succeeded), raisesInvalidStateErrorbefore that.sayandgenerate_reply) record the error; pipeline LLM failures are recorded into the same attribute, so both backends report errors consistently.RunResultreads the same attribute, soawait session.run(...)still raises on generation failures.