Researchers from UC Berkeley and UT Austin have developed FreeToken, an edge-native MoE serving engine built to run large language models on personal hardware. Its headline result is striking: FreeToken can run the 753B GLM-5.2 model on a single workstation GPU.
The project also targets smaller systems. FreeToken can deliver interactive speed for a 35B model on an 8 GB laptop GPU, while a 284B model can run on a gaming desktop. That puts model serving across a range of hardware, from a laptop GPU to a workstation setup, without limiting the project to one machine class.
FreeToken Brings Large Models to Local Hardware
FreeToken is designed as a serving engine rather than a single-purpose model package. The project exposes OpenAI- and Anthropic-compatible endpoints on port 1919, giving developers a familiar way to connect applications to models running through the system.
The software is published on GitHub under the Apache-2.0 license, available on PyPI as freetoken v0.1.2, and shipped as a desktop app for Windows and Linux. Its stated target platform is Linux x86_64 with an NVIDIA GPU using driver r580+ and CUDA 13.
Those details make the project feel practical for developers who want to test local serving without building the entire setup from scratch. The endpoint support, package release, and desktop app cover different ways to access FreeToken, while the stated Linux requirements define the main hardware path.
Measured Speeds Show the Range
On an RTX 5090, FreeToken sustains 77–83 tokens per second on Qwen3.6-35B-A3B in BF16. It reaches 22–25 tokens per second on DeepSeek-V4-Flash on the same GPU.
Those measurements add useful detail to the headline claims. FreeToken is not only aimed at fitting large models into local memory; it also reports sustained token rates for two named models on an RTX 5090. The results provide a clearer picture of how the engine behaves with different workloads.
The project also reports measured BP:BH figures for two systems. The ratio is 52.7:77.3 on an RTX 5090 server and 11.8:47.5 on a 4060 laptop. Together, those numbers show that the reported behavior differs between workstation-class and laptop hardware.
FreeToken’s model range is the main story here. A 35B model at interactive speed on an 8 GB laptop GPU sits at one end, while the 753B GLM-5.2 model on a single workstation GPU sits at the other, with a 284B model on a gaming desktop between them.
Local Serving Versus API Pricing
The same model family also has an API pricing point for comparison. GLM-5.3 is available through an API at $1.40 per million input tokens and $4.40 per million output tokens. Its API price is unchanged from GLM-5.2.
GLM-5.3 also has a cached input price of $0.26 per million tokens. Based on the sum of its input and output costs, the listed GLM-5.3 API cost is $5.80 per million tokens.
That contrast gives FreeToken a clear place in the larger model-serving picture. An API offers usage priced by tokens, while FreeToken focuses on running models on personal hardware, including a 753B model on one workstation GPU. The project does not erase the need for capable hardware, but its reported results push the boundary of what that hardware can handle.
As of August 23, 2026, FreeToken combines the large-model claim with a public software release, compatible endpoints, and measured speeds across two GPUs. Its core promise is simple: bring models that sound too large for personal systems into reach through an edge-native serving engine.
Based on
