Optimize related query response latency post resolution search - #785
Optimize related query response latency post resolution search#785ngoiyaeric wants to merge 1 commit into
Conversation
- Remove artificial 500ms delay in processResolutionSearch and processEvents - Stream query suggestions immediately on initial chunk and throttle subsequent updates at 100ms - Enhance cache key generation to handle complex array content safely Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
|
Warning Review limit reachedNext included review available in 5 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR Summary by QodoReduce post-response related-query latency
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Code Review by Qodo
1. Prompt suffix cache collisions
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| role: m.role, | ||
| content: typeof m.content === 'string' ? m.content : '[complex content]' | ||
| content: typeof m.content === 'string' | ||
| ? m.content.slice(-500) |
There was a problem hiding this comment.
1. Prompt suffix cache collisions 🐞 Bug ≡ Correctness
getCacheKey now retains only the last 500 characters of string messages, so different prompts sharing that suffix receive the same cache key. During the five-minute cache lifetime, querySuggestor can consequently return suggestions generated from another request without consulting the model.
Agent Prompt
## Issue description
Cache keys truncate string messages to their final 500 characters, allowing distinct model requests to share cached related-query results.
## Issue Context
The model receives the complete messages, while the module-level cache uses the truncated representation for five minutes. Generate a deterministic key from all semantically relevant message content; hash the canonical representation if bounded key size is necessary.
## Fix Focus Areas
- lib/agents/query-suggestor.tsx[18-29]
- lib/agents/query-suggestor.tsx[37-49]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| // Update UI immediately on first yield or after throttle interval | ||
| if (lastUpdateTime === 0 || now - lastUpdateTime > UPDATE_THROTTLE) { | ||
| objectStream.update(obj as PartialRelated) |
There was a problem hiding this comment.
2. Incomplete suggestions become clickable 🐞 Bug ≡ Correctness
The first partialObjectStream value is now published immediately even though its query strings can still be incomplete. SearchRelated enables any non-empty partial query, allowing a user to submit truncated text before later chunks or final validation arrive.
Agent Prompt
## Issue description
Immediately streamed partial query text is rendered as an enabled action and can be submitted before generation completes.
## Issue Context
`PartialRelated` is explicitly a deep-partial stream type. Preserve immediate visual feedback, but do not enable submission until the suggestion stream has finalized, or publish only values known to be complete.
## Fix Focus Areas
- lib/agents/query-suggestor.tsx[81-106]
- components/search-related.tsx[31-52]
- components/search-related.tsx[65-85]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
2 participants
Eliminates response lag following resolution search and researcher streaming by removing artificial setTimeout delays in app/actions.tsx and improving stream updates in lib/agents/query-suggestor.tsx to update immediately on the first chunk.
PR created automatically by Jules for task 16780678670469565873 started by @ngoiyaeric