Large language models (LLMs) have changed how we interact with technology. But they face a big challenge: processing long texts is slow and costly. A startup called Subquadratic claims to have cracked this problem with a new model named SubQ.

SubQ uses a fresh approach to attention mechanisms, the core process that helps LLMs understand language. Traditional models rely on “dense attention,” a method that compares every word with every other word. This method grows slower and more expensive as the text gets longer, because the work increases quadratically.

SubQ replaces dense attention with “sub-quadratic sparse attention.” Instead of checking every word against all others, it picks only important connections. This cuts the number of computations drastically. The result? SubQ can handle context windows of up to 12 million tokens — that’s like reading hundreds of books at once.

How SubQ Changes the Game

This new attention method means SubQ runs faster and uses far less energy. The creators say it runs up to 56 times faster than popular methods like FlashAttention-2. It also uses less than 5% of the compute power that traditional models need for similar tasks.

SubQ can process huge documents or codebases in a single pass. It even matches the performance of leading models from big AI companies on tasks like coding and reasoning. Yet, it does so at a fraction of the usual cost.

Subquadratic started by sharing limited details, which made some experts skeptical. But independent tests by a third-party firm, Appen, confirmed many of SubQ’s claims. The tests showed SubQ’s ability to find relevant information in massive texts while reducing compute needs nearly 1,000 times compared to dense attention.

Why This Matters

LLMs today struggle with scaling because of how attention works. When a model reads a text twice as long, the number of computations roughly quadruples. This quickly makes very long text processing impractical.

SubQ’s sparse attention focuses only on meaningful word relationships. This saves compute resources without losing accuracy. Unlike earlier approaches like Longformer or BigBird, SubQ can link important words no matter their position in the text.

This breakthrough could reshape industries that rely on analyzing large datasets quickly. Think legal firms scanning thousands of contracts, financial analysts reviewing massive reports, or developers searching through millions of lines of code.

SubQ also tackles a technical bottleneck called the KV cache. This cache stores key and value vectors for tokens during processing, and it grows linearly with text length. At very large contexts, this cache uses more memory than the model itself. New compression methods like TurboQuant and OSCAR aim to shrink this cache, making long-context models more efficient. These advancements complement what SubQ is doing on the attention side.

Subquadratic plans to roll out SubQ to partners soon. The company believes this is just the start of moving beyond transformers as the core architecture. The hope is that SubQ will kick off a new era where LLMs become faster, cheaper, and more energy-friendly.

Still, some remain cautious. The startup has yet to release SubQ widely for public testing. And the full technical details are not fully disclosed. But the independent verification and strong benchmark results have shifted many opinions from doubt to curiosity.

For now, SubQ offers a glimpse of what future language models might look like. They will break past the limits of current tech and allow us to handle vast amounts of information in real time. This could speed up AI development and bring smarter, more capable tools to everyone.