Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions md/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@
- [Template](./rfds/TEMPLATE/README.md)
- [Accepted](./rfds/accepted.md) <!-- put accepted rfds in this section; the file goes in the rfds directory -->
- [Registry-centric plugin distribution](./rfds/registry-centric-plugins/README.md)
- [Predicate caching](./rfds/predicate-caching/README.md)
- [Completed](./rfds/completed.md) <!-- move completed rfds to this section -->
- [Configuration parsing and normalization](./rfds/config-normalization/README.md)
- [RFD Process](./rfds/rfd-process/README.md)
-
90 changes: 90 additions & 0 deletions md/rfds/predicate-caching/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Predicate caching

## TL;DR

Predicates (especially custom predicates) spawn processes on every sync. This RFD lets them emit watch declarations via JSONL events. Symposium caches results and skips re-evaluation while watched files/env vars are unchanged. No watch declaration = never cached.

## Problem

Auto-sync means predicates re-evaluate on every agent session start. A workspace with 10 plugins, each with a custom predicate, forks 10+ processes every time.

## Design

Custom predicates already emit JSONL events to stdout. We add a `Watch` event variant:

```rust
#[derive(Serialize, Deserialize)]
enum CustomPredicateEvent {
// ... existing variants (e.g. SelectedCrates) ...
Watch {
files: Vec<PathBuf>,
env: HashMap<String, String>,
}
}
```

Emitted as a JSONL line:

```jsonl
{"watch": {"files": ["CargoBrazil.toml"], "env": {"LAMBDA_ENV": "prod"}}}
```

The predicate result still comes from exit status — the watch event is just a cache hint. Predicates that don't emit a `Watch` event are never cached (current behavior preserved).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be the opposite. If you don't emit watch hint(s), you are ALWAYS cached, because we assume we don't have to cache.

Also, let's make it clear that predicates can emit multiple watch hints that are all unioned together.


Both `files` and `env` are optional (either alone is sufficient to enable caching). Files are relative to workspace root.

## SDK helper

The `symposium-sdk` crate provides a helper that reads env vars and automatically emits the cache line:

```rust
// In a custom predicate binary:
let val = symposium_sdk::env::var("LAMBDA_ENV")?;
// ^ reads the var AND emits {"watch": {"env": {"LAMBDA_ENV": "prod"}}}
```

This way predicate authors get caching for free when they use the SDK.

## How it works

Fingerprint for files is `mtime + size` (same model as Cargo). A missing file is a valid fingerprint state — invalidates when the file appears. Env fingerprint is the literal string value (or absent).

Cache lives at `~/.symposium/cache/predicates.json`, keyed by the normalized predicate string. No workspace-root keying needed — watch paths are resolved relative to the current workspace root at stat time, so different workspaces naturally produce different fingerprints.

Algorithm:

1. Look up predicate in cache
2. If found and all watched files + env vars match fingerprints → use cached result
3. Otherwise → evaluate, store result + watch if declared
4. On write, prune entries for predicates no longer in any manifest

Cache is discarded on Symposium version upgrade.

## Built-in predicates

- `workspace-member()` → no cache needed (already cheap, in-memory)
- `path_exists(path)` → cached, watches the path itself
- `depends-on(name)` → cached, watches PM manifest (e.g. `Cargo.lock`)
- `env(FOO=BAR)` → cached, watches env value of `FOO`
- `shell(cmd)` / custom predicates → cached only when they emit a `Watch` event

## PM integration

`list_deps` return type expands to include optional watch paths:

```rust
struct DepsResult {
ids: Vec<PackageId>,
watch: Option<Vec<PathBuf>>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these should be optional. Also, do we need environments here?

I'm not exactly clear on the role of DepsResult, I guess I have to go digging.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally thought to extend list_deps(What packages does this workspace depend on?) to also return watch paths (DepsResult { ids, watch }) so that Symposium could cache the entire dependency list and skip re-calling the PM when manifest files haven't changed.

We can defer this because the PackageManager trait is still being defined in Jack's latest PR and the shape isn't settled yet. Also, the cargo PM's list_deps is already cheap (reads in-memory workspace data). Once PM interface stabilizes, we can add this.

}
```

`CargoPm` returns `[Cargo.lock]`. This is a return-type change, not a new method.

## Implementation steps

1. Add `Watch` variant to `CustomPredicateEvent`. Parse it from predicate stdout. No caching yet — just capture the data.
2. Cache storage in `~/.symposium/cache/predicates.json`: read/write, fingerprint comparison, dead-entry pruning.
3. Wire built-in predicates: `path_exists` emits file watch, `env` emits env watch, `depends-on` uses PM watch paths.
4. Add `symposium_sdk::env::var()` helper that auto-emits the cache event.
5. PM `list_deps` watch support (coordinate with pm-split).
Loading