Skip to content

Resolve tool parameter schemas from the input side #163

Description

@shibayan

What happens

resolveJsonSchema (packages/core/src/tools/json-schema.ts:55-92) converts a schema-library object by calling its toJSONSchema() with no arguments. In Zod 4 that describes the schema's output: the value after defaults are filled in and transforms have run. What a model needs is the input contract, the JSON it is allowed to send.

Measured with zod 4.4.3 (the version the examples pin):

const s = z.object({ a: z.string(), d: z.enum(['x', 'y']).default('x') });

s.toJSONSchema()
→ {"type":"object","properties":{"a":{"type":"string"},"d":{"default":"x","type":"string","enum":["x","y"]}},"required":["a","d"],"additionalProperties":false}

s['~standard'].jsonSchema.input({ target: 'draft-2020-12' })
→ {"type":"object","properties":{"a":{"type":"string"},"d":{"default":"x","type":"string","enum":["x","y"]}},"required":["a"]}

const t = z.object({ n: z.string() }).transform((v) => v.n.length);

t.toJSONSchema()
→ throws: Transforms cannot be represented in JSON Schema

t['~standard'].jsonSchema.input({ target: 'draft-2020-12' })
→ {"type":"object","properties":{"n":{"type":"string"}},"required":["n"]}

Consequences:

  • A parameter with .default() is advertised as required. The model is told it must send a value the schema exists to supply, and a provider that enforces the schema rejects an omission the tool would have accepted.
  • A schema with .transform() cannot be declared at all. tool() resolves the schema eagerly, so the declaration throws. Yet the runtime is built for transforms: arguments are validated through ~standard.validate (packages/core/src/client/function-execution.ts:442) and execute is typed with the schema's output (ToolInputOf, packages/core/src/tools/tool.ts:102-107). Only the schema resolution refuses them.
  • A Standard Schema without toJSONSchema() is rejected with "Standard Schema does not define JSON Schema output" (json-schema.ts:78-84). That was true when written. The Standard JSON Schema companion spec now exists, and zod 4.4.3 exposes it as ~standard.jsonSchema with input(...) and output(...) (measured above). The vendor-neutral path the error message denies is there.

The same function feeds skill scripts (packages/core/src/skills/skill.ts:171) and structured output (packages/core/src/client/structured-output.ts:54,66), so the input/output confusion applies wherever a schema is shown to a model.

Why it matters

A tool's schema is the contract the model writes JSON against. Advertising the post-parse shape describes a value the model never produces. The reference implementations do not have this seam (.NET derives the schema from parameter reflection, Python from the Pydantic input model), so they do not exhibit the problem; here the framework's own promise is that the model's arguments go through the schema and the handler receives the parsed result, and that only holds if the model is shown the input side.

Required implementation

  • Resolve the input side. Prefer Standard JSON Schema (~standard.jsonSchema.input({ target })) when the schema exposes it; fall back to a library-specific conversion only when it does not, and when that fallback is Zod's toJSONSchema() pass { io: 'input' }.
  • Drop the "Standard Schema does not define JSON Schema output" rejection. A Standard Schema that exposes neither Standard JSON Schema nor a conversion method still fails at declaration, with an accurate message.
  • Keep the root $schema stripping.
  • Decide and pin how additionalProperties behaves after the change. The output side carried additionalProperties: false, the input side does not; any provider that sends strict function schemas must be re-measured rather than assumed.
  • Apply the same resolution to structured output. There too the model produces the schema's input.

Acceptance criteria

  • A reproduction-first test declares a tool with a defaulted field and asserts the advertised schema lists only the non-defaulted field under required, failing before the change.
  • A tool declared with .transform() can be declared, is advertised with the pre-transform schema, and its execute receives the transformed value.
  • A Standard Schema exposing ~standard.jsonSchema but no toJSONSchema() resolves.
  • additionalProperties behaviour is asserted for each provider that sends strict schemas, before and after.
  • The comment block on resolveJsonSchema describes the resolution order that actually runs.
  • CHANGELOG.md records the change in advertised schemas; pnpm check passes.

Explicitly out of scope

Validating arguments against the JSON Schema for tools declared with a raw JSON Schema object (no Standard Schema); that path is unchanged.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    coreUsage: [Issues, PRs], Target: packages/core

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions