Skip to content

fix(max): tell users chat charts have no export button - #90024

Open
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/fixposthog-ai-stop-max-inventing-a-32608b
Open

fix(max): tell users chat charts have no export button#90024
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/fixposthog-ai-stop-max-inventing-a-32608b

Conversation

@posthog

@posthog posthog Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Problem

  • Users who ask Max how to export a chat table get pointed at a download icon and a "..." menu that do not exist. The instruction is unfollowable, and the only real path (open the chart as an insight, then export there) goes unmentioned.
  • Max renders every chat visualization as an embedded, read-only query. The toolbar below the chart has exactly two controls: open as a new insight, and show/hide. There is no export affordance on this surface.
  • The three prompt templates that describe what sits below a chart named only the open-insight icon, so the model filled the gap with generic dashboard-UI priors.
  • This is not side-panel specific: the same shared code renders in the full /ai scene too, so every export question is answered wrongly on both surfaces.

Changes

  • Max now tells users a chat chart cannot be exported directly, and routes them to open the chart as a new insight and export from the insight page.
  • Mechanical: one export sentence added to each of the three prompt sites — ee/hogai/context/insight/prompts.py, ee/hogai/tools/create_insight.py, ee/hogai/tools/task.py. The copy is identical across all three.

The report also lists an optional frontend export action on VisualizationWidget as a larger follow-up. This PR takes the cheapest fix that stops the wrong answer on both surfaces at once.

How did you test this code?

  • The change is additive prompt text inside existing strings. ast.parse confirms the three modules still parse.
  • No prompt-snapshot tests cover these strings, so none needed regeneration.
  • The agent did not run the live model or manually reproduce a Max conversation.

Automatic notifications

  • Publish to changelog?

Docs update

None.

🤖 Agent context

Autonomy: Fully autonomous

  • Root cause: the model's below-chart guidance omitted export, so it hallucinated dashboard-style export controls. Confirmed the frontend toolbar (products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx) exposes only open-in-new and show/hide, and that DataVisualization returns early on embedded, skipping the export button.
  • Chose the prompt fix over adding a frontend export button because it addresses the observed symptom (the wrong answer) on both the side panel and the full /ai scene with the smallest change.
  • Skills invoked: /writing-user-facing-copy, /writing-simplified-technical-english, /simplify.

Created with PostHog Desktop from this inbox report.

Max renders every chat visualization as an embedded, read-only query, so the toolbar below the chart has only the open-as-insight and show/hide controls. It has no download button and no export menu. The three prompt templates that tell the model what sits below a chart named only the open-insight icon, so the model filled the gap with generic dashboard-UI priors and pointed users at controls that do not exist.

Add one export sentence to each of the three prompt sites. Each now states that a chat chart cannot be exported directly and tells the model to route the user to open the chart as a new insight, then export from the insight page.

Generated-By: PostHog Desktop
Task-Id: f41a8c3a-6781-439a-ad4a-c7311806d88a
@posthog posthog Bot added the skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs label Aug 27, 2026
@trunk-io

trunk-io Bot commented Aug 27, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@posthog

posthog Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

🦔 PostHog Review reviewed this pull request

Found 0 must fix, 2 should fix, 0 consider.

Published 2 findings (view the review).

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

Playwright — all passed

All tests passed.

View test results →

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trivial, additive prompt-text change with no code logic, dependency, security, or API impact; not risky territory so no independent assurance is required.

Gate mechanics and policy version
Gate Result
prerequisites all clear
deny-list no deny categories matched
size 6L, 3F substantive — within ceiling
tier T1-agent / T1a-trivial (6L, 3F, single-area, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 9f271d2 · reviewed head 9f271d2

@posthog

posthog Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review

Found 2 should fix.

{{/insight_url}}
{{^insight_url}}
This insight is not saved in the project, so it cannot be accessed via a URL. Any `Artifact ID` above is scoped to this conversation, not an insight short ID, so an `/insights/...` link built from it would 404. If the user wants a saved insight, tell them to open the chart as a new insight from the icon below it and save it from there.
This insight is not saved in the project, so it cannot be accessed via a URL. Any `Artifact ID` above is scoped to this conversation, not an insight short ID, so an `/insights/...` link built from it would 404. If the user wants a saved insight, tell them to open the chart as a new insight from the icon below it and save it from there. You cannot export a chart directly from the conversation, because the chart has no download button and no export menu. To export the data, tell the user to open the chart as a new insight from the icon below it, then export it from the insight page.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saved insight charts skip the export correction

should_fix

Why we think it's a valid issue
  • Checked: the branch condition in ee/hogai/context/insight/prompts.py:13, the insight_url property, every call site that builds an InsightContext, the has_transient_artifacts flag in ee/hogai/tools/task.py, and the controls that VisualizationWidget renders.
  • Found: InsightContext.insight_url returns a URL whenever insight_short_id is set (ee/hogai/context/insight/context.py:62-67). Three call sites set it for saved insights: ee/hogai/tools/read_data/tool.py:449 (read one insight), ee/hogai/context/dashboard/context.py:176 (each insight on a read dashboard), and ee/hogai/context/context.py:537 (an insight the user attaches). All three take the {{#insight_url}} branch, so the new export sentence at prompts.py:13 never reaches the model for a saved insight.
  • Found: ee/hogai/tools/task.py:190-213 sets has_transient_artifacts = True only for artifacts whose source is not ArtifactSource.INSIGHT. A task that returns saved insights alone keeps the flag false, so the whole block at task.py:100-103 — including the export sentence — is omitted.
  • Found: the widget renders the same two controls for both sources. products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx:126-146 shows only the open-target button and the show/hide button, and getArtifactOpenTarget (same file, lines 56-64) gives a saved insight an "Open insight" link, not an export control. So a saved insight in chat has no download button either.
  • Impact: the PR removes the wrong export answer for charts Max generates, but leaves it reachable on the saved-insight paths, which cover read-insight, read-dashboard, and user-attached-insight renders. A user who asks how to export one of those charts can still get directions to a control that does not exist.
  • Impact: the miss is partly softened, not closed. The saved branches print a real Insight URL, and task.py:102 tells the model that URL is a real link. That gives the model a correct route to offer, so the wrong answer is less likely there than on the transient path. Nothing in either branch states that the chat chart itself has no export control, so the invented download icon stays available to the model.
Issue description

The new export instruction only appears inside {{^insight_url}}. ReadDataTool._read_insight() sets insight_short_id for saved insights, so the formatter omits this instruction. The UI still renders saved insights in VisualizationWidget, which has no download control. Export questions about these charts can still produce the false directions that this PR intends to remove.

Suggested fix

Add guidance for the {{#insight_url}} branch. Tell Max to open the existing insight and export from that page. Move the warning outside has_transient_artifacts in TASK_ARTIFACTS_PROMPT, then give separate routes for Artifact ID and Insight ID entries.

Prompt to fix with AI (copy-paste)
## Context
@ee/hogai/context/insight/prompts.py#L13

<issue_description>
The new export instruction only appears inside `{{^insight_url}}`. `ReadDataTool._read_insight()` sets `insight_short_id` for saved insights, so the formatter omits this instruction. The UI still renders saved insights in `VisualizationWidget`, which has no download control. Export questions about these charts can still produce the false directions that this PR intends to remove.
</issue_description>

<issue_validation>
- **Checked:** the branch condition in `ee/hogai/context/insight/prompts.py:13`, the `insight_url` property, every call site that builds an `InsightContext`, the `has_transient_artifacts` flag in `ee/hogai/tools/task.py`, and the controls that `VisualizationWidget` renders.
- **Found:** `InsightContext.insight_url` returns a URL whenever `insight_short_id` is set (`ee/hogai/context/insight/context.py:62-67`). Three call sites set it for saved insights: `ee/hogai/tools/read_data/tool.py:449` (read one insight), `ee/hogai/context/dashboard/context.py:176` (each insight on a read dashboard), and `ee/hogai/context/context.py:537` (an insight the user attaches). All three take the `{{#insight_url}}` branch, so the new export sentence at `prompts.py:13` never reaches the model for a saved insight.
- **Found:** `ee/hogai/tools/task.py:190-213` sets `has_transient_artifacts = True` only for artifacts whose source is not `ArtifactSource.INSIGHT`. A task that returns saved insights alone keeps the flag false, so the whole block at `task.py:100-103` — including the export sentence — is omitted.
- **Found:** the widget renders the same two controls for both sources. `products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx:126-146` shows only the open-target button and the show/hide button, and `getArtifactOpenTarget` (same file, lines 56-64) gives a saved insight an "Open insight" link, not an export control. So a saved insight in chat has no download button either.
- **Impact:** the PR removes the wrong export answer for charts Max generates, but leaves it reachable on the saved-insight paths, which cover read-insight, read-dashboard, and user-attached-insight renders. A user who asks how to export one of those charts can still get directions to a control that does not exist.
- **Impact:** the miss is partly softened, not closed. The saved branches print a real `Insight URL`, and `task.py:102` tells the model that URL is a real link. That gives the model a correct route to offer, so the wrong answer is less likely there than on the transient path. Nothing in either branch states that the chat chart itself has no export control, so the invented download icon stays available to the model.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Add guidance for the `{{#insight_url}}` branch. Tell Max to open the existing insight and export from that page. Move the warning outside `has_transient_artifacts` in `TASK_ARTIFACTS_PROMPT`, then give separate routes for `Artifact ID` and `Insight ID` entries.
</potential_solution>


INSIGHT_TOOL_PROMPT = """
Use this tool to generate an insight from a structured plan. It will return a visualization that the user can analyze and a textual representation for your analysis. These visualizations are transient and only exist within the current conversation—they are not saved to the project. To save an insight permanently, users should click the open insight icon below the chart in the conversation.
Use this tool to generate an insight from a structured plan. It will return a visualization that the user can analyze and a textual representation for your analysis. These visualizations are transient and only exist within the current conversation—they are not saved to the project. To save an insight permanently, users should click the open insight icon below the chart in the conversation. You cannot export a chart directly from the conversation, because the chart has no download button and no export menu. To export the data, tell the user to open the chart as a new insight from the icon below it, then export it from the insight page.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transient export route omits the save step

should_fix

Why we think it's a valid issue
  • Checked: the CTA target the chat widget builds for a transient chart, the short_id state of a new insight, the gate on the insight page's Export menu, and every in-page export control that does not need a saved insight.
  • Found: the CTA for a transient chart is urls.insightNew({ query }) (products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx:47-54), and createEmptyInsight('new') leaves short_id: undefined (frontend/src/scenes/insights/insightLogic.tsx:93-102). So the page the user reaches holds an unsaved insight.
  • Found: the insight page hides its whole Export menu until the insight is saved. frontend/src/scenes/insights/SidePanel/InsightSceneMenuBar.tsx:124 computes canExport = exportContext != null && insight.short_id != null, and line 269 renders the Export submenu — PNG, CSV, and XLSX — only when canExport is true. PNG additionally passes insight: insight.id, which an unsaved insight does not have.
  • Found: one in-page export survives without saving, but only for some charts. frontend/src/queries/nodes/InsightViz/InsightVizDisplay.tsx:436-458 shows a CSV/XLSX button above the detailed results table, and hasDetailedResultsTable is true only for trends and stickiness, minus the display types in DISPLAY_TYPES_WITHOUT_DETAILED_RESULTS (frontend/src/scenes/insights/insightVizDataLogic.ts:2341-2345). SQL and table insights keep their own DataTableExport menu.
  • Impact: for a funnel, retention, paths, or lifecycle chart, and for any PNG request, the user follows the new instruction and lands on a page with no export control at all. That repeats the same unfollowable-direction failure this PR removes, on the path the PR itself creates.
  • Impact: the miss is partial, not total. Each of the three prompt sites mentions saving in a nearby sentence, the unsaved page shows a Save button, and trends charts keep the detailed-results CSV button. So the user can recover, and the model may chain save before export on its own. The export sentence still names a route that does not work as written.
Issue description

getQueryOpenTarget() opens an unsaved /insights/new query. The main insight export menu requires insight.short_id, so it does not appear until the user saves the insight. Unsaved editors expose only limited table exports and no PNG export. A PNG request or funnel-data request therefore reaches another page with no usable export action.

Suggested fix

Tell the model to open the chart, save the new insight, and then use the saved insight's export menu. Update the identical text in all three prompt sites.

Prompt to fix with AI (copy-paste)
## Context
@ee/hogai/tools/create_insight.py#L19

<issue_description>
`getQueryOpenTarget()` opens an unsaved `/insights/new` query. The main insight export menu requires `insight.short_id`, so it does not appear until the user saves the insight. Unsaved editors expose only limited table exports and no PNG export. A PNG request or funnel-data request therefore reaches another page with no usable export action.
</issue_description>

<issue_validation>
- **Checked:** the CTA target the chat widget builds for a transient chart, the short_id state of a new insight, the gate on the insight page's Export menu, and every in-page export control that does not need a saved insight.
- **Found:** the CTA for a transient chart is `urls.insightNew({ query })` (`products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx:47-54`), and `createEmptyInsight('new')` leaves `short_id: undefined` (`frontend/src/scenes/insights/insightLogic.tsx:93-102`). So the page the user reaches holds an unsaved insight.
- **Found:** the insight page hides its whole Export menu until the insight is saved. `frontend/src/scenes/insights/SidePanel/InsightSceneMenuBar.tsx:124` computes `canExport = exportContext != null && insight.short_id != null`, and line 269 renders the Export submenu — PNG, CSV, and XLSX — only when `canExport` is true. PNG additionally passes `insight: insight.id`, which an unsaved insight does not have.
- **Found:** one in-page export survives without saving, but only for some charts. `frontend/src/queries/nodes/InsightViz/InsightVizDisplay.tsx:436-458` shows a CSV/XLSX button above the detailed results table, and `hasDetailedResultsTable` is true only for trends and stickiness, minus the display types in `DISPLAY_TYPES_WITHOUT_DETAILED_RESULTS` (`frontend/src/scenes/insights/insightVizDataLogic.ts:2341-2345`). SQL and table insights keep their own `DataTableExport` menu.
- **Impact:** for a funnel, retention, paths, or lifecycle chart, and for any PNG request, the user follows the new instruction and lands on a page with no export control at all. That repeats the same unfollowable-direction failure this PR removes, on the path the PR itself creates.
- **Impact:** the miss is partial, not total. Each of the three prompt sites mentions saving in a nearby sentence, the unsaved page shows a Save button, and trends charts keep the detailed-results CSV button. So the user can recover, and the model may chain save before export on its own. The export sentence still names a route that does not work as written.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Tell the model to open the chart, save the new insight, and then use the saved insight's export menu. Update the identical text in all three prompt sites.
</potential_solution>

@posthog
posthog Bot marked this pull request as ready for review August 27, 2026 11:39
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team August 27, 2026 11:40
@trunk-io

trunk-io Bot commented Aug 27, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants