Skip to content

fix(aio): stop the wizard sample from dropping feedback answers #89603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
phillram wants to merge 3 commits into
base: master
Choose a base branch
from
+103 −11
Open

fix(aio): stop the wizard sample from dropping feedback answers #89603

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9e54771
fix(aio): stop the wizard sample from dropping feedback answers
phillram Aug 26, 2026
557a9e7
fix(aio): keep the wizard sample's rating when the follow-up is dismi…
phillram Aug 27, 2026
f13d84d
chore(aio): explain why the sample run is trimmed to one path
phillram Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions products/ai_observability/frontend/feedback-view/wizard/codeExamples.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { getManualCaptureExample } from './codeExamples'

describe('getManualCaptureExample', () => {
it('does not mark the rating event incomplete for a rating that ends the survey', () => {
const thumbsEvent = getManualCaptureExample({ followUpEnabled: true })
.split("posthog.capture('survey sent', {")[1]
.split('})')[0]

// The wizard branches a thumbs up straight to End, so it never sends a follow-up event.
// A literal `false` here leaves every thumbs up submission without a completed event,
// which hides it from surveys that do not enable partial responses.
expect(thumbsEvent).not.toMatch(/\$survey_completed: false\b/)

@greptile-apps greptile-apps Bot Aug 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Test misses required behavior

This assertion only rejects the literal $survey_completed: false in the first event, so it still passes if completion is removed or made unconditional, the rating condition is reversed, or the completed follow-up event stops repeating the rating.

Context Used: Do not attempt to comment on incorrect alphabetica... (source)

Prompt To Fix With AI 
This is a comment left during a code review.
Path: products/ai_observability/frontend/feedback-view/wizard/codeExamples.test.ts
Line: 12

Comment:
**Test misses required behavior**

This assertion only rejects the literal `$survey_completed: false` in the first event, so it still passes if completion is removed or made unconditional, the rating condition is reversed, or the completed follow-up event stops repeating the rating.

**Context Used:** Do not attempt to comment on incorrect alphabetica... ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@carlos-marchal-ph carlos-marchal-ph Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a real issue!

@phillram phillram Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 557a9e7.

The test now runs the generated sample against a stub posthog and asserts the events a pasted integration would send, over three paths: thumbs up, thumbs down with text, thumbs down dismissed.

Each weakness you listed was checked by mutating the source and re-running:

Mutation Result
rating event hard-codes $survey_completed: false 1 failure
follow-up event drops the rating re-send 2 failures
expectsFollowUp = rating === 1 (reversed) 3 failures
no-follow-up variant hard-codes $survey_response: 1 3 failures
follow-up event only sent with text 1 failure

})
})
22 changes: 14 additions & 8 deletions products/ai_observability/frontend/feedback-view/wizard/codeExamples.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,9 @@ const generateProps = (props: Prop[], indent = 2): string => {
export function getManualCaptureExample({ surveyId = 'your-survey-id', followUpEnabled }: CodeExampleParams): string {
const thumbsProps: Prop[] = [
{ key: '$survey_id', value: `'${surveyId}'`, comment: 'ID for the survey you just created' },
{ key: '$survey_response', value: '1', comment: '1 = thumbs up, 2 = thumbs down' },
followUpEnabled
? { key: '$survey_response', value: 'rating' }
: { key: '$survey_response', value: '1', comment: '1 = thumbs up, 2 = thumbs down' },
{ key: '$ai_trace_id', value: 'traceId', comment: 'your generated trace ID' },
...(followUpEnabled
? [
Expand All @@ -56,11 +58,7 @@ export function getManualCaptureExample({ surveyId = 'your-survey-id', followUpE
value: 'submissionId',
comment: 'unique ID to link thumbs + follow-up',
},
{
key: '$survey_completed',
value: 'true',
comment: 'or false if there is negative feedback followup',
},
{ key: '$survey_completed', value: '!expectsFollowUp' },

@greptile-apps greptile-apps Bot Aug 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Optional follow-up drops ratings

When a user selects thumbs down but does not submit the optional follow-up, $survey_completed: !expectsFollowUp leaves the only rating event incomplete, causing the rating to be filtered out of survey results.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: products/ai_observability/frontend/feedback-view/wizard/codeExamples.ts
Line: 61

Comment:
**Optional follow-up drops ratings**

When a user selects thumbs down but does not submit the optional follow-up, `$survey_completed: !expectsFollowUp` leaves the only rating event incomplete, causing the rating to be filtered out of survey results.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@carlos-marchal-ph carlos-marchal-ph Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good one too. Either the snippet sends the completed event on dismissal too, or the wizard creates the survey with enable_partial_responses: true, which makes the filter take the latest event per $survey_submission_id and makes false safe. The cumulative re-send is still needed either way.

@greptile-apps greptile-apps Bot Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — this is a valid gap in the follow-up-enabled snippet. If a user dismisses the follow-up after thumbs down, the only event remains $survey_completed: false, so the submission is omitted from results when partial responses are disabled.

I’d recommend adding a dismissal path that sends a final cumulative survey sent event with $survey_response: rating and $survey_completed: true. That keeps the wizard’s existing API-survey behavior unchanged. Enabling enable_partial_responses would also address the filtering, but changes survey-level semantics and should only be used if retaining abandoned submissions is intentional. The cumulative re-send is still required in either approach.

@phillram phillram Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, and fixed in 557a9e7.

The second event now fires when the follow-up closes, not only when text is submitted, and carries $survey_response_1: followUpText ('' on dismissal). Every path now ends on exactly one completed event.

On the enable_partial_responses alternative: leaving it off. The wizard also renders this snippet for an existing survey the user picks from the list, and those surveys carry whatever setting they were created with. A snippet that always ends on a completed event is correct either way.

Worth noting for anyone reading later: with partial responses off, exactly one completed event per submission is required. Marking both events complete would count the submission twice. So the !expectsFollowUp shape is load-bearing, not decoration.

]
: []),
]
Expand All @@ -70,10 +68,17 @@ export function getManualCaptureExample({ surveyId = 'your-survey-id', followUpE
{ key: '$ai_trace_id', value: 'traceId' },
]

const submissionIdLine = followUpEnabled
// The survey branches to the follow-up on a thumbs down and ends on a thumbs up, so the
// sample derives both the rating and the completion flag from one variable. Hard-coding
// `$survey_completed: false` would leave every thumbs up without a completed event.

@carlos-marchal-ph carlos-marchal-ph Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rationale is also in the generated snippet on line 79 and again in the test. Which of the three is the durable home? This one also sits ten lines below the $survey_completed line it explains rather than next to it.

@phillram phillram Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated snippet is the durable home, since that is the artifact a customer reads. The TypeScript comment and the test comment are both gone.

What is left sits next to what it explains: the branching note directly above const expectsFollowUp, and the reason the closing event is mandatory directly above that event.

const followUpPreamble = followUpEnabled
? `// Generate a unique ID to link \`survey sent\` events into a single user feedback event
const submissionId = crypto.randomUUID()

const rating = 2 // 1 = thumbs up, 2 = thumbs down

@carlos-marchal-ph carlos-marchal-ph Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rating is load-bearing now, expectsFollowUp and $survey_completed both derive from it. Copied verbatim, every submission reports a thumbs down and marks itself incomplete, which is the failure this PR is fixing. It also disagrees with the no-follow-up variant, which still renders $survey_response: 1.

Could we make the placeholder look like one?

Suggested change
const rating = 2 // 1 = thumbs up, 2 = thumbs down
const rating = clickedThumbsUp ? 1 : 2 // 1 = thumbs up, 2 = thumbs down

@phillram phillram Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken, and applied to both variants, since the no-follow-up sample had the same problem with $survey_response: 1. Both now open with your line:

const rating = clickedThumbsUp ? 1 : 2 // 1 = thumbs up, 2 = thumbs down

// Only a thumbs down opens the follow-up, so a thumbs up completes the submission here
const expectsFollowUp = rating === 2

`
: ''

Expand All @@ -82,14 +87,15 @@ posthog.capture('survey shown', {
${generateProps(surveyShownProps)}
})

${submissionIdLine}// When user clicks thumbs up/down, send a survey event
${followUpPreamble}// When user clicks thumbs up/down, send a survey event
posthog.capture('survey sent', {
${generateProps(thumbsProps)}
})`

if (followUpEnabled) {
const followUpProps: Prop[] = [
{ key: '$survey_id', value: `'${surveyId}'` },
{ key: '$survey_response', value: 'rating', comment: 're-send the thumbs response so it still shows' },
{ key: '$survey_response_1', value: "'the AI hallucinated hedgehogs everywhere'" },
{ key: '$ai_trace_id', value: 'traceId' },
{
Expand Down
Loading