fix(read_file): return image bytes in MCP content so the model can see images#488
Conversation
…e images
read_file on an image previously put the image bytes only in
structuredContent (for the preview widget) and sent the host model a
text-only summary ("Image file: ... (image/png)"). The model received
no image block, so reading an image was effectively a no-op for the
model — it could never actually see the image.
Add the image content block back into the `content` array while keeping
structuredContent unchanged, so:
- the host model receives the image and can analyze it
- the preview widget still renders from structuredContent as before
The previous "text-only for broad host compatibility" approach blinded
every host that can handle images, which is the common case.
|
Caution Review failedPull request was closed or merged during review No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe PR enhances the image file reading response in ChangesImage Content in MCP Response
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
#488 added the image bytes back into read_file's MCP `content` array so the host model can see images, but test-file-handlers.js was never updated and still asserted the pre-#488 contract ("avoid image content item for host compatibility"). That assertion is now inverted from the shipped behaviour and was the only failing test in the suite. Update Test 9 to assert the current contract for both raster (PNG) and SVG reads: the `content` array includes an `image` item carrying base64 `data` and the correct `mimeType` (image/png, image/svg+xml), alongside the existing structuredContent checks. No production code change — handler behaviour is unchanged.
Problem
read_fileon an image returned the image bytes only instructuredContent(consumed by the preview widget) and sent the host model a text-only summary likeImage file: ... (image/png).The model received no image content block, so reading an image was effectively a no-op for the model — it could never actually see the image. The previous code comment described this as "text-only for broad host compatibility," but it blinded every host that can handle images, which is the common case.
Fix
Add the image content block back into the MCP
contentarray (text summary +type: "image"block), while leavingstructuredContentunchanged so the preview widget keeps rendering as before.Result:
structuredContentTesting
npm run buildcleanread_fileon a PNG now returns the image to the model (previously only the text summary came back).Notes
This is scoped to the image branch of
handleReadFile. The separateedit_blockfreeze (DOCX) is being investigated independently and is not part of this PR.Summary by CodeRabbit