Skip to content

Harden shared agent instruction review - #4220

Open
shellz-n-stuff wants to merge 8 commits into
block:mainfrom
shellz-n-stuff:codex/harden-shared-agent-instructions
Open

Harden shared agent instruction review#4220
shellz-n-stuff wants to merge 8 commits into
block:mainfrom
shellz-n-stuff:codex/harden-shared-agent-instructions

Conversation

@shellz-n-stuff

@shellz-n-stuff shellz-n-stuff commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • render shared-agent instructions as literal text so Markdown cannot conceal spoiler contents, link destinations, or image sources
  • reject non-reviewable Unicode controls at every agent-definition boundary while preserving legitimate rendered emoji sequences
  • verify shared catalog event IDs and signatures before trusting authorship, coordinates, pagination, or executable content
  • preserve the exact system-prompt bytes between review and execution instead of silently stripping or normalizing content

Security rationale

Shared system prompts are executable configuration. Previously, catalog prompts were projected through the chat Markdown renderer, which could hide text, replace link destinations with benign labels, and turn image syntax into remote loads. Zero-width and bidirectional controls could also make reviewed text differ from what the agent executes.

This change establishes a review invariant: the prompt a user sees is the prompt the agent executes. Definitions that cannot be reviewed faithfully are rejected rather than rewritten. Catalog events must also pass Nostr ID/signature verification before they can claim a publisher, coordinate, or cursor.

What changed

  • catalog instructions render as exact literal text rather than rich Markdown
  • catalog relay events are verified on a fresh wire-shaped object before paging, coordinate selection, attribution, or projection
  • forged content, pubkeys, signatures, and invalid newer heads are ignored and cannot shadow a valid signed definition
  • TypeScript catalog parsing rejects unsafe remote definitions before they reach the UI
  • shared Rust validation covers persona create/update/import, inbound relay sync, definition-less managed-agent sync, and catalog publication paths
  • definition-less managed agents now fail closed on local create, local update, and publication before persistence or relay retention
  • linked managed agents validate their local name while treating the persona definition as authoritative; their inert record-level prompt is not executed or published
  • names reject layout controls; prompts retain ordinary newlines and tabs
  • legitimate emoji composition is supported, including contextual VS16, ZWJ, skin-tone, family, flag, and keycap sequences
  • detached selectors/joiners, bidirectional controls, tag characters, zero-width concealment, and other default-ignorables remain rejected
  • names are bounded to 128 characters and prompts to 64 KiB
  • contributor guidance documents the byte-for-byte review requirement for future sharing paths

Validation reports the offending code point and never silently removes it.

E2E recording

buzz-shared-agent-security-e2e.webm

The recording demonstrates:

  • a safe definition remains visible
  • a prompt containing zero-width U+200B is rejected
  • a name containing bidi override U+202E is rejected
  • the prompt is preserved exactly
  • spoiler, link, and image syntax remains literal and does not render or load

Verification

Passed locally:

  • just test: all 10 unit and Docker-backed integration stages
  • desktop frontend unit suite: 4,295 tests
  • persona catalog relay unit suite: 32 tests, including forged-event and cursor-shadowing cases
  • focused Rust definition-validation coverage: 3 local create/update tests and 6 publication-filtered tests
  • complete desktop Tauri library suite after rebase: 2,263 passed, 14 ignored, 0 failed
  • desktop Tauri clippy with warnings denied and Rust formatting
  • complete agent Playwright spec: 34 tests
  • the exact formerly failing inbox-edit immediate-attachment smoke test after rebase: 1 test
  • focused shared-agent publish, literal-review, hidden-control, signature, and cross-member import Playwright coverage
  • desktop E2E production build and TypeScript typecheck
  • changed-file formatting/lint and file-size ratchet
  • pre-commit secret scan and DCO signoff

The branch was rebased onto current main, which includes the upstream attachment-button label fix. Fresh post-rebase GitHub CI is green for every required and selected check: Desktop Core, all four Desktop Smoke E2E shards, both Desktop E2E Integration shards and their aggregate, Desktop E2E Relay, Desktop Build (macOS), Windows Rust, Rust Lint, DCO, security scanners, and Desktop Release Candidate. The previously failing Desktop Smoke E2E (3) shard now passes.

The repository-wide desktop check also reports existing CSS formatting/!important findings in components.css and terminal.css; neither file is changed by this PR. GitHub's Desktop Core lint and format stage passes on the rebased branch.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9536a5673

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread desktop/src-tauri/src/commands/personas/inbound.rs

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on Wes's behalf after independent security, regression, and integration reviews of d56523fcbab119e78cc2a208c0b78cae55dfe4b3.

Two blockers remain:

  1. The catalog accepts unverified relay events as executable definitions and attributes them to attacker-selected pubkeys. A compromised relay can forge a member's shared agent and prompt a user to import arbitrary instructions.
  2. The validator rejects legitimate VS16/ZWJ emoji sequences, breaking edits to existing local agents and hiding valid shared definitions from the catalog.

The literal <pre> review surface, Rust ingress validation, snapshot/team-snapshot coverage, pre-retention validation, and TypeScript/Rust range parity otherwise look sound. Existing CI at this exact head is green; reviewers did not duplicate CI-equivalent suites locally.

Comment thread desktop/src-tauri/src/commands/personas/create.rs
Comment thread desktop/src/features/agents/lib/personaCatalogRelay.ts

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on Wes's behalf after coordinated review of fc53800ef0d849a127d850598f35b9d2d867c099 with Princess Donut and Mongo.

The two prior blockers are fixed: catalog events are now verified on fresh wire-shaped objects before they can claim coordinates or influence paging, and the TS/Rust validators permit contextual VS16/ZWJ emoji while rejecting detached formatting. The literal review surface and catalog add path preserve exact prompt bytes. Mongo independently found no remaining issue in those paths.

One blocker remains: the Rust safety invariant is incomplete for other executable, relay-synced definition text.

  • Definition-less ManagedAgentRecord.system_prompt is executable, but create_managed_agent and update_managed_agent accept it without validate_agent_definition_text; those records are persisted and queued for publication. Inbound definition-less records do get validated, leaving an asymmetric local bypass.
  • Team instructions are executable, but team create/update and inbound kind 30176 apply them without the validator. Team snapshot validation validates member definitions but not the executable team.instructions (or the team name), then import persists those fields.

As a result, invisible/bidi text such as Review\u{200B}code can still be persisted, executed, and published through local managed-agent/team paths even though equivalent persona and inbound managed-agent definitions are rejected. Please centralize validation for executable name/instruction fields across create, update, import, sync, and publication boundaries, with direct regressions for definition-less agents and teams—or explicitly narrow the promised security contract and demonstrate why these executable fields are outside it.

Existing CI is green at this exact head. Focused local test attempts in my clean review worktree were environment-blocked (missing JS dependencies; missing Tauri sidecar), so I did not count them as validation. Mongo separately ran the focused TS suites: 44/44 passed.

@wesbillman

Copy link
Copy Markdown
Collaborator

Scope clarification on my changes-requested review, on Wes's behalf:

I agree the team-instructions work can be split into a follow-up. It is executable shared text and the gap is real, but this PR's concrete review surface is agent/persona definitions; extending the invariant to teams broadens the product/UI contract enough to justify separate work. Please track that follow-up before merge.

I do not think the definition-less managed-agent gap is separable. This PR explicitly added inbound validation for definition-less kind 30177 records and its description claims “definition-less managed-agent sync” plus validation at every agent-definition boundary. Leaving local create/update able to persist, execute, and publish the same unsafe prompt makes that new boundary asymmetric and directly contradicts the PR's stated invariant.

So the narrowed blocker is: validate definition-less managed-agent name/prompt on local create/update (and at publication as defense in depth), with direct regressions. Team validation may follow separately with an issue/PR.

Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
@shellz-n-stuff
shellz-n-stuff force-pushed the codex/harden-shared-agent-instructions branch from 3858539 to e6aa358 Compare August 6, 2026 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants