Skip to content

Integrate capability-aware Apple Silicon runtime#138

Open
Paul (nuclearspike) wants to merge 9 commits into
Lightricks:mainfrom
nuclearspike:codex/desktop-pro-apple-perf
Open

Integrate capability-aware Apple Silicon runtime#138
Paul (nuclearspike) wants to merge 9 commits into
Lightricks:mainfrom
nuclearspike:codex/desktop-pro-apple-perf

Conversation

@nuclearspike

@nuclearspike Paul (nuclearspike) commented Jul 26, 2026

Copy link
Copy Markdown

Summary

  • Consolidates draft PRs Fix silent loss of completed Extend results #132, Clarify Pro model availability in local mode #133, Fix timeline gap frame conditioning #134, and Release reused diffusion cache before VAE decode #136 without modifying their source branches.
  • Adds capability-aware Fast T2V/I2V routing: auto uses MLX only for local text encoding, cached BF16 weights, and the exact clean 0.14.20.dev1 runtime at revision 3171bac4ba901c0237faea2678c34034b37abc2a. Prepared embeddings, A2V, Retake, Extend, IC-LoRA, and image generation stay on Torch.
  • Runs MLX in a fresh process group for every job. The backend never imports MLX; cancellation kills the whole group and process exit is the allocator teardown boundary.
  • Resolves local Fast dimensions before conditioning or inference. Distilled 540p uses the budget-bounded 896x512 two-stage grid (512x896 portrait), removes hidden per-axis rounding, reports the resolved pair in the completion API, rejects off-grid MLX commands, and makes strict QA require API dimensions to equal ffprobe output.
  • Persists a prompt-safe JSONL profile for every MLX job and exposes live/last-job allocator active, cache, peak, phase, terminal status, path, timestamp, and exact runtime identity through the existing telemetry API and Settings UI. Incomplete tails, invalid timestamps, non-finite values, and killed-child cancellation are handled without breaking telemetry.
  • Adds cross-product fcntl Metal leasing, process/system/MPS/MLX memory telemetry, capability routing and exact provenance in Settings, and a strict HD performance evidence harness that captures the post-child terminal profile.
  • Keeps q8 expert-only with a visible quality warning; auto never selects it. Low-RAM MLX enables runtime auto-tiling.

Evidence

  • Backend: 598 passed.
  • TypeScript and Pyright: clean.
  • OpenAPI generated bindings and drift check: clean.
  • LSSD semantic contract check: clean.
  • Production frontend and Electron bundles: passed.
  • Full macOS production build from the integration base: passed; signed app, DMG, and ZIP produced.
  • Live Apple M5 Max policy smoke: exact MLX runtime verified, BF16 cache detected (139,683,517,037 bytes), auto selected MLX for eligible Fast local-text work, backend idle RSS 497 MiB.
  • Pre-fix production smoke proved the former mismatch: resolver 960x544 versus actual sidecar/output 960x512. The final post-fix profiled smoke is delegated to QA.
  • PR Release reused diffusion cache before VAE decode #136 MPS 1280x704x121 benchmark: median 220.570s to 209.465s (-5.04%); peak 44,504 MiB to 41,713.5 MiB (-6.27%); identical output.
  • Persistent MLX worker was rejected: retained about 50.8 GiB and request 2 was slower.

Draft blocker

AI Studio does not yet implement the same shared local-metal.lock contract. Keep cross-product real-Metal benchmarks and merge blocked until that side is present and interop is proven.

@nuclearspike

Copy link
Copy Markdown
Author

Final production qualification (Apple M5 Max, 128 GB):

  • Exact clean MLX runtime 0.14.20.dev1@3171bac4ba901c0237faea2678c34034b37abc2a, MLX 0.31.1.
  • Final production POST /api/generate AUTO→MLX BF16 smoke at head e76b1e9: strict PASS, API/command/ffprobe all 896x512x121, 66.61 s wall, 38,585 MiB MLX peak, 34.136 GiB process-tree RSS, 41.133 GiB physical footprint, 100% GPU utilization, terminal active 0 MiB/cache 2 MiB, worker exited and shared flock released.
  • Aspect-safe 540p mapping improves ratio error from 5.47% to 1.56% and removes hidden dimension rounding; portrait parity is covered.
  • Cross-product Electron/AI Studio lease test PASS, including wait ordering, cancellation, crash release, and responsive CPU-only control-plane endpoints.
  • Capability policy AUTO-selects MLX only for parity-proven Fast T2V/I2V with local text encoding; prepared embeddings, A2V, Retake, Extend, IC-LoRA, and image generation retain Torch feature coverage.
  • Product validation: 598 backend tests, TypeScript/Pyright, OpenAPI drift, production frontend/Electron bundles, and LSSD all clean. Final QA: 21/21 focused tests and LSSD clean. Signed app/DMG/ZIP build passed earlier on the combined branch.

The branch is ready for maintainer review.

@nuclearspike
Paul (nuclearspike) marked this pull request as ready for review July 27, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant