Usage-policy gate matches directories, not path spellings: canonicalize both sides

llp/0049-hypignore-usage-policy.spec.md

@@ -62,6 +62,12 @@ governs. Because V1 has only `ignore` and no "un-ignore" directive
 mechanism is not repo-bound: a `.hypignore` anywhere in the ancestor chain
 (including outside any git repo) governs its subtree.
 
+**Extended-by: [LLP 0050 §canonicalization](./0050-ignore-enforced-in-adapters.decision.md#canonicalization)**:
+"the ancestor chain" is over **real** paths, not lexical ones. A `cwd` is
+matched over the set of spellings that denote it (as-given plus
+symlink-resolved), and the most restrictive verdict any spelling produces wins,
+so a `.hypignore` cannot be escaped by reaching its subtree through a symlink.
+
 ## Classes {#classes}
 
 | Class | V1 | Meaning |

llp/0050-ignore-enforced-in-adapters.decision.md

@@ -79,6 +79,121 @@ imported by both adapters exactly as they already import
 concept and does not inspect rows (only the adapter knows which field is the
 `cwd`).
 
+## What "the same directory" means to the shared matcher {#canonicalization}
+
+The matcher compares directory *paths*, and a directory has more than one path.
+Until this section existed, the matcher used `path.resolve` on both sides, which
+is purely **lexical**: it normalizes `.`/`..` and makes a path absolute but
+follows no symlinks. So the ancestor walk from a symlinked `cwd` climbed the
+*link's* parents and never met the `.hypignore` governing the real directory: a
+user's `.hypignore` was silently not in force for a project reached through a
+symlink, which on macOS is ordinary (`/tmp` to `/private/tmp`, Homebrew
+prefixes, iCloud-backed `~/Documents`). Because this is the *single shared*
+matcher, every consumer inherited the hole: all four adapter capture seams,
+`hyp purge`, the query-seam visibility filter (LLP 0105), `hyp ignore --check` /
+`policy show`, and the machine-local list membership test.
+
+**Decision: a directory is matched over the *set* of path spellings that denote
+it, and the most restrictive verdict any spelling produces wins.** The set is
+the as-given (lexical) spelling plus the canonical, symlink-resolved spelling
+when it differs, computed by `src/core/usage-policy/canonical.js`. Both sides of
+every comparison get this treatment: the incoming `cwd`, and every stored
+machine-local entry's `dir`.
+
+Resolving over the *set* rather than switching to the canonical form alone is
+the load-bearing part, and it is a privacy argument rather than a tidiness one.
+Canonicalizing only the incoming `cwd` does close the capture leak, but the
+machine-local store keeps whatever path the user supplied (LLP 0071, LLP 0103),
+so an entry the user marked by its symlink spelling stops governing: the class
+drops from `local-only` to `full` and that directory **starts forwarding**. That
+trades a capture leak for a forwarding leak. Taking the most restrictive verdict
+across both spellings closes the first without opening the second, and makes the
+fail-safe structural rather than a special case: a `realpath` that fails removes
+a *candidate* spelling, never a verdict some other spelling already produced.
+
+**The invariant is that canonicalization only ever moves the gate toward more
+restrictive, never toward `full`**, and one step needs explicit care to hold it.
+Merging verdicts across spellings is monotone, but the machine-local list's
+*nearest-governs* step is an argmax over match depth, and an argmax discards
+verdicts instead of merging them. A less restrictive entry (an explicit `sync`
+carve-out, say) that gains reach through its canonical spelling can therefore
+become the deepest match and displace a broader restrictive entry that already
+governed - which would punch a hole in a private tree declared under the other
+spelling and start that directory recording and forwarding. So the
+nearest-governs rule is evaluated **twice**, once over the declared spellings
+alone (what the lexical matcher decided) and once over the widened set, and the
+more restrictive of the two answers wins, the declared one breaking a class tie
+because it is the spelling the user typed. Widening an entry's reach can then
+only add restriction.
+
+The visible cost is that a nested loosening does not cross spellings *when the
+broader restrictive entry is one the declared pass already found*: an explicit
+`sync`/`full` carve-out declared under one spelling does not loosen a broader
+restrictive entry that matches the `cwd` by its own declared spelling, and a
+user who wants that carve-out has to declare it under the same spelling as the
+entry it carves out of (`hyp policy show` on the path reports the class actually
+in force, so the situation is diagnosable). That is the privacy-safe direction
+of the trade, and it is the same direction the `cwd` side already takes.
+
+That condition is load-bearing and the rule should not be read without it,
+because the two-pass guard can only preserve a verdict the declared pass
+actually produced. When the broader restrictive entry reaches the `cwd` *only*
+through canonicalization, the declared pass matches nothing, there is no lexical
+verdict to preserve, and ordinary nearest-governs decides among entries that are
+all in the canonical namespace: a deeper carve-out does then win. That is not a
+hole in the invariant above. The lexical matcher matched neither entry in that
+shape, so `full` is exactly what it returned as well, and the outcome is the one
+the user would have got by declaring both entries canonically in the first
+place. Pinned by `resolve: between two entries that both reach only by
+canonicalization, the deeper carve-out governs`.
+
+Three consequences worth stating outright, because they are what a reader of the
+privacy gate will ask:
+
+- **A declaration stays authoritative as written.** Stored entries keep their
+  as-declared `dir` on disk; nothing is rewritten. An entry governs both its
+  declared spelling and, when resolvable, the canonical spelling of what it
+  points at. If the declared path's target changes or disappears, the
+  declaration still governs the declared spelling, so **no stored entry ever
+  silently loses its class**; it merely stops governing the canonical form of a
+  target it no longer names. A filesystem change never revokes a user's
+  declaration.
+- **Migration is canonicalize-on-read, additively.** There is no on-disk
+  migration and no version bump: entries written before this decision gain
+  canonical reach the moment it ships. Writes still store the caller-resolved
+  path, so `policy show` and `--check` echo the spelling the user typed. The one
+  write-side change is *upsert identity*: re-marking a directory through a
+  different spelling replaces the existing entry rather than appending a second
+  governor for the same directory (which would let the nearest-governs
+  tie-break, not the user, decide the class).
+- **`realpath` is not free, so it rides the existing cache.** The per-`cwd`
+  memoization is keyed on the lexical path and consulted *before*
+  canonicalization, so the cost is one `realpath` per distinct `cwd` per TTL
+  window - the identical bound [LLP 0049](./0049-hypignore-usage-policy.spec.md#requirements)
+  R6 already sets for the ancestor walk - and **zero** syscalls on a cache hit,
+  which is the per-exchange hot path. Entry spellings are computed once per list
+  parse, inside the same TTL. `hyp purge`'s subtree predicate runs per row, so it
+  memoizes the verdict per distinct row `cwd` for the life of one purge run.
+
+A canonicalization that does not fully resolve is reported as a structured
+`usage_policy.canonicalize_failed` event carrying `error_kind:
+path_canonicalize_failed`, the `errno`, how far it got, and a **hashed** path -
+never a raw local path, the same discipline the `usage_policy.export_drop`
+aggregate uses. `ENOENT` is routine (a deleted `cwd`, a not-yet-created
+directory), so it logs at `debug`; only a wholly unresolvable path escalates to
+`warn`. `realpath` is all-or-nothing, so rather than discard a failure entirely
+the canonicalizer resolves the deepest existing ancestor and rejoins the
+unresolved tail: with `/tmp` a symlink, `/tmp/proj/not-created-yet` still
+canonicalizes usefully, which matters because the symlink is almost always an
+*ancestor*, not the leaf.
+
+`isEqualOrDescendant` stays lexical and pure, for callers comparing two strings
+that are already canonical and must not touch the filesystem. The
+spelling-agnostic predicate a CLI verb wants when it asks "which stored entry
+governs this directory?" is `scopeGoverns`, which has to be the same predicate
+`resolve` used, or `policy show` names a governor the gate did not use and
+`policy unset` refuses to remove an entry the gate is enforcing.
+
 ## Why not the gateway
 
 - The gateway is the **provider-agnostic** proxy ([LLP 0016](./0016-ai-gateway.decision.md)).
@@ -101,6 +216,10 @@ worse coupling than both importing core.
 
 - Code that lands this carries `@ref LLP 0050 [implements]` on the adapter
   projector/backfill drop sites and on the `src/core/usage-policy/` matcher.
+- Every consumer of the shared matcher inherits
+  [§canonicalization](#canonicalization) for free; no adapter, `hyp purge`, or
+  query-seam change is needed to gain it, which is the same argument that put the
+  matcher in core in the first place.
 - The gateway source and recorder are not modified.
 - A future caller-supplied `cwd` for raw-proxy traffic would add a *new* call
   site that reuses the same core matcher — no change to this decision.

llp/0071-machine-local-exclusion-list.decision.md

@@ -47,7 +47,12 @@ absolute directory paths:
   already defines: a `cwd` is `local-only` when it equals, or is a path-segment
   descendant of, any listed directory. This is a second source into the one
   shared resolver ([LLP 0070 §resolver](./0070-local-only-export-seam.decision.md#resolver)),
-  not a second matcher.
+  not a second matcher. **Extended-by:
+  [LLP 0050 §canonicalization](./0050-ignore-enforced-in-adapters.decision.md#canonicalization)**:
+  membership is over **real** paths: an entry governs through any spelling of its
+  declared directory, so an entry the user marked by a symlink spelling still
+  covers the real directory (and vice versa) and never silently drops to `full`.
+  The stored `dir` is left exactly as declared; canonicalization happens on read.
 - The file is **read locally by the export driver and the CLI**, written by the
   login picker and the durable authoring command
   ([LLP 0072 §cli](./0072-enrollment-dir-picker.decision.md#cli)), under the same

src/core/cache/purge.js

@@ -2,7 +2,7 @@
 
 import path from 'node:path'
 
-import { isEqualOrDescendant } from '../usage-policy/matcher.js'
+import { scopeGoverns } from '../usage-policy/matcher.js'
 import { discoverCachePartitions, readCursorSync, writeCursor } from './partition.js'
 import { deleteMatchingRows, scanRowsFromTable, tableExists } from './iceberg/store.js'
 import { resolveIcebergDir } from './storage.js'
@@ -22,9 +22,10 @@ import { resolveIcebergDir } from './storage.js'
  * Four target shapes (LLP 0104 decision):
  *
  *  - `{ kind: 'subtree', path }` — rows whose `cwd` equals or descends from
- *    `path` (the LLP 0049 §scope ancestor rule via {@link isEqualOrDescendant}),
- *    regardless of the path's usage class: an explicit purge may remove any
- *    data, `local-only` and synced included.
+ *    `path` (the LLP 0049 §scope ancestor rule via {@link scopeGoverns}, so a
+ *    row recorded under one spelling of a directory is still purged when the
+ *    target names the other), regardless of the path's usage class: an explicit
+ *    purge may remove any data, `local-only` and synced included.
  *  - `{ kind: 'session', id }` — one session's rows. `session_id` is the
  *    partition key (LLP 0030); the predicate still scans every partition
  *    because the on-disk cache is partitioned by source, not session.
@@ -87,11 +88,24 @@ function buildPredicate(target, purgedCwds) {
   switch (target.kind) {
     case 'subtree': {
       const base = path.resolve(target.path)
+      // `scopeGoverns` canonicalizes both sides, which costs a `realpath`; the
+      // predicate runs per row, and a cache holds many rows per distinct `cwd`,
+      // so memoize the verdict per `cwd` for the lifetime of this one purge run
+      // (short-lived by construction, so staleness is not a concern).
+      // @ref LLP 0050#canonicalization [implements]: canonical-aware subtree purge, one `realpath` per distinct row `cwd`
+      /** @type {Map<string, boolean>} */
+      const inScope = new Map()
       return {
         columns: ['cwd'],
         predicate: (row) => {
           if (typeof row.cwd !== 'string' || row.cwd === '') return false
-          if (!isEqualOrDescendant(path.resolve(row.cwd), base)) return false
+          const cwd = path.resolve(row.cwd)
+          let governed = inScope.get(cwd)
+          if (governed === undefined) {
+            governed = scopeGoverns(cwd, base, { component: 'cache-purge' })
+            inScope.set(cwd, governed)
+          }
+          if (!governed) return false
           noteCwd(row)
           return true
         },

src/core/commands/clients.js

@@ -26,8 +26,10 @@ import {
   CLASS_RANK,
   createUsagePolicyResolver,
   findRepoRoot,
-  isEqualOrDescendant,
+  governingListEntry,
   localOnlyListPath,
+  sameDirectory,
+  scopeGoverns,
   readLocalOnlyEntries,
   writeLocalOnlyEntries,
 } from '../usage-policy/index.js'
@@ -910,7 +912,13 @@ export async function runMarkMachineLocal({ targetDir, ctx, targetClass, compone
   }
 
   const entries = await readLocalOnlyEntries({ stateDir })
-  const withoutTarget = entries.filter((entry) => entry.dir !== resolvedTarget)
+  // Upsert identity is "denotes the same directory", not "is the same string":
+  // re-marking a directory through a different spelling must update its class,
+  // not append a second entry that governs the same directory at a different
+  // class (which would make the resolver's nearest-governs tie-break decide the
+  // user's privacy for them).
+  // @ref LLP 0050#canonicalization [implements]: one stored entry per directory, whichever spelling declared it
+  const withoutTarget = entries.filter((entry) => !sameDirectory(entry.dir, resolvedTarget))
   await writeLocalOnlyEntries({ stateDir, entries: [...withoutTarget, { dir: resolvedTarget, class: targetClass }] })
   getLogger('usage-policy').info('usage_policy.mark', {
     [Attr.COMPONENT]: component,
@@ -990,7 +998,7 @@ export async function runUnignore(argv, ctx) {
  *
  * Removes every machine-local entry that governs `targetDir`, equal to it,
  * or an ancestor of it (the same segment-aware rule the shared resolver
- * applies, reused here via {@link isEqualOrDescendant} rather than
+ * applies, reused here via {@link scopeGoverns} rather than
  * re-derived, R8), mirroring dotfile `unignore`'s "remove the governing
  * thing" semantics. When `targetClass` is given, removal is scoped to that
  * one class and entries of a different class are left alone (LLP 0104
@@ -1014,7 +1022,9 @@ export async function runUnmarkMachineLocal({ targetDir, ctx, targetClass, compo
   const stateDir = readObservabilityEnv(ctx.env).stateDir
   const entries = await readLocalOnlyEntries({ stateDir })
   const governing = entries.filter(
-    (entry) => (targetClass === undefined || entry.class === targetClass) && isEqualOrDescendant(targetDir, entry.dir)
+    (entry) =>
+      (targetClass === undefined || entry.class === targetClass) &&
+      scopeGoverns(targetDir, entry.dir, { component })
   )
   if (governing.length === 0) {
     if (targetClass === undefined) {
@@ -1128,12 +1138,16 @@ export async function runIgnoreCheck({ targetDir, ctx, json, vocabulary = INTERN
  * should count: the directory containing the governing `.hypignore` when
  * governed by a dotfile (unchanged from before the machine-local list
  * existed), or — when governed by the machine-local store
- * (`result.governedBy === listPath`) — the most specific (longest) entry
- * that actually matches `base`, found via the shared
- * {@link isEqualOrDescendant} predicate rather than a second copy of path
- * logic (R8). The `resolve()` call already decided *whether* something
- * governs; this only identifies *which* listed directory did, for display
- * and scoping the residual count.
+ * (`result.governedBy === listPath`), the entry the gate itself used, from
+ * the shared {@link governingListEntry} selector rather than a second copy of
+ * the selection rule (R8). The `resolve()` call already decided *whether*
+ * something governs; this only identifies *which* listed directory did, for
+ * display and scoping the residual count - so it has to make the same choice
+ * the resolver made. Re-deriving it from `scopeGoverns` plus "longest declared
+ * string" does not: once an entry can match through its canonical spelling,
+ * the longest declared string and the deepest matching spelling are different
+ * entries, and `--check` would scope its residual count to one while
+ * reporting the other's class.
  *
  * @param {{ result: ResolveResult, base: string, stateDir: string, listPath: string }} args
  * @returns {Promise<string>}
@@ -1142,9 +1156,8 @@ async function resolveCheckScopeDir({ result, base, stateDir, listPath }) {
   if (!result.governedBy) return base
   if (result.governedBy !== listPath) return path.dirname(result.governedBy)
   const entries = await readLocalOnlyEntries({ stateDir })
-  const matches = entries.filter((entry) => isEqualOrDescendant(base, entry.dir))
-  if (matches.length === 0) return base
-  return matches.reduce((best, entry) => (entry.dir.length > best.dir.length ? entry : best)).dir
+  const governing = governingListEntry(base, entries, { component: 'cmd-ignore-check' })
+  return governing === null ? base : governing.dir
 }
 
 /**