hpcc is a distributed compilation and caching tool. A drop-in compiler wrapper that caches build artifacts locally and remotely, and optionally distributes compilation across a cluster of workers — with a strong tenant-isolation story suitable for environments (banks, regulated enterprises) where running user-supplied compilation on shared hardware is politically untenable.
This document is the index. Each phase, plus the cross-cutting structure
and design-decisions sections, lives in its own file under plan/. What's
done vs. what's open per phase lives in each phase doc — git log is the
authoritative chronology.
| Phase | Description | Status |
|---|---|---|
| Phase 1 | Core Compiler Wrapping | Done |
| Phase 2 | Daemon Architecture | Done |
| Phase 3 | Remote Cache | Done |
| Phase 4 | Distributed Compilation in Per-Tenant VMs | Done |
| Phase 5 | Observability & Polish | In progress |
- Phase 1: Core Compiler Wrapping — flag parsing, input hashing, local disk cache, drop-in wrapper.
- Phase 2: Daemon Architecture — loopback-TCP daemon, client-server protocol, dedup, fallback.
- Phase 3: Remote Cache — S3 store, lookup order, failure handling, eviction, bucket provisioning.
- Phase 4: Distributed Compilation in Per-Tenant VMs — raw Firecracker, runtime abstraction, VM lifecycle, image-as-environment, agent wire schema, scheduler, worker, wire protocol, audit, paranoid mode, rootfs hardening.
- Phase 5: Observability & Polish — stats, cache inspection, miss reasons, configuration, security event log, eviction.
- Project Structure — repo layout.
- Design Decisions — hashing, storage, sandbox model, wire protocols, security.
Related sub-plans:
- CAS-mode source dispatch — the §4.5 content-addressed
source-staging path. Shipped as the default
source_mode = "cas"; PREPROCESSED stays selectable as a fallback. Probe-then-upload,.hpccproject marker for cross-developer hits, paranoid-mode- friendly trust boundary,.S/.incbinsupport. - Multi-tenant isolation — promoting
tenant_idfrom a JWT label to a namespace boundary: per-tenant IdP at the scheduler, per-tenant storage prefixes on every store, per-tenant upload quota onUploadBlobs. Closes the §4.5 cross-tenant probe disclosure by construction.