Perplexity has open sourced Lily, a local inference engine built for Qwen3.6-35B-A3B on Apple silicon. The project is written in Rust and Metal, and it powers Hybrid Compute, Perplexity’s system for splitting AI tasks between cloud models and local models.
A standalone Lily demo is public in the pplx-garden repository. It offers greedy text generation through a minimal OpenAI-compatible HTTP API, giving developers a direct way to test local inference without using the full Perplexity Mac app.
A local engine built around Apple silicon
Lily targets Apple silicon Macs, but its memory needs are substantial. Its 4-bit checkpoint weighs 19.4 GB, compressed from about 70 GB of bfloat16 weights. An Apple silicon Mac with 32 GB or more of unified memory is the realistic minimum for running Lily.
The engine reconstructs weights inside grouped GEMM while holding results in threadgroup memory. Its performance work also includes a prefill change that raised end-to-end prefill by 77.4% at a 512-token prompt. Moving from 16-row to 32-row tiles with four simdgroups added 13.2% at 2K tokens, while a register-resident Gated DeltaNet scan added another 5.6%.
On a 40-core, 128 GB M5 Max, Lily averaged 4,156 prefill tokens per second and 170.0 decode tokens per second. MLX-LM averaged 3,388 prefill tokens per second and 126.4 decode tokens per second in the same comparison.
The figures also change with the prompt and context size. At a 4K prompt and 4K context, Lily reached 5,749.9 tokens per second for prefill and 186.6 tokens per second for decode. Its perplexity was 0.04% higher than a teacher-forced check across 192 positions.
Hybrid Compute connects local and cloud models
Perplexity’s shipping Hybrid Compute product takes the same local approach and connects it to cloud-based models such as Opus 5 or GPT-5.6 Sol. The feature splits tasks between those cloud models and local models, with the entire system integrated into the Mac app.
Local models available for Hybrid Compute include Gemma E4B and two variants of Qwen’s 35-billion parameter 3.6 model. Installing a local model does not require opening the Mac’s terminal, and Hybrid Compute displays CPU, GPU, and memory usage while it operates.
The product also includes a check for sensitive content before a task moves to the cloud. Perplexity’s Jon Staff described the feature this way:
“This is integrated directly into the Mac app. Any time you try to upload files or send information, we’re going to automatically check for sensitive content and make sure that you want to share that data to the cloud.”
That setup gives users a choice between local processing and cloud-based capability. Tokens generated by local models are not charged, while cloud models remain available for tasks that need them.
Hardware limits shape the experience
Hybrid Compute is available only on Apple Silicon Macs running macOS 15. The shipping product lists 24 GB as its minimum, but Perplexity recommends at least 32 GB of unified memory for Hybrid Compute and describes 32 GB as the level for best results.
Lily’s 19.4 GB checkpoint helps explain why memory matters. The model file alone takes up a large share of a 24 GB system, before the Mac app and other work are counted. The realistic minimum for Lily is therefore higher than the product’s listed 24 GB minimum for Hybrid Compute.
Hybrid Compute is available to Pro and Max subscribers and enterprise customers. Its local model support gives those users a way to run selected tasks on their own Mac, while its cloud options handle models that need more capability.
Staff also drew a clear line between local and frontier systems: “The short answer is that a fully frontier output is going to almost always be better in terms of raw artifact creation. It’s more expensive, it’s more capable.” Lily does not replace those cloud models. Instead, it adds a local path for supported tasks and exposes the engine behind that path as open-source software.
Based on
