The Tiny Qwen Model That Runs a GPU Inside Its Own Weights

A small language model is doing something that sounds like a hardware project: running CUDA from inside its own forward passes. The experimental model, called qwenGPU, compiles a SIMT GPU directly into the weights of Qwen3.5-2B.

Developer Vixhal, also known as @TheVixhal, released qwenGPU on GitHub under the MIT license. The project tests one focused idea: whether an exact computation can live inside a pretrained large language model instead of running beside it.

A 69-layer model with an embedded four-lane GPU

qwenGPU combines Qwen3.5-2B’s 24 layers with 45 compiled GPU layers, creating 69 layers in total. None of the added layers were trained. A compiler computes every weight, using only Qwen3.5’s own layer types: Gated DeltaNet linear attention and SwiGLU MLPs.

The embedded machine has four SIMT lanes, 15 usable registers per lane, a four-stage pipeline, a 32-block ROM, and 126 memory words. The README lists 16 register rows per lane, with one row that always reads zero, matching the 15 usable registers described in the project’s X post.

The design depends on the recurrent state inside Gated DeltaNet. In the circuit’s heads, the gate stays open and the write strength is 1. Every key is one-hot, so each update overwrites exactly one row. A one-hot query can then read that row back, turning each row into a memory cell.

That state stores the program ROM, registers, and global memory. The GPU supports integer arithmetic, predicates, divergent branches, loops, shared memory, atomicAdd, __syncthreads(), and warp shuffles.

CUDA goes through the model, not around it

The host takes the role of NVRTC and the CUDA driver. It compiles CUDA to NVIDIA PTX with clang, lowers PTX to machine blocks, and feeds the program and input arrays to the model as tokens. From there, the model writes the launch itself.

Each EXEC(b, w) token runs block b for warp w in one forward pass. The VAL tokens that follow carry register results, while memory writes return as MEM records. The host reads those results from the recurrent state and computes none of them.

In the demonstration, a dot product of [1, 2, 3] and [4, 5, 6] returns 32. The model wrote that launch in 47 tokens. The reference interpreter also matched a real NVIDIA Tesla T4 on 49 of 49 kernels.

The model runs on Apple silicon, an NVIDIA GPU, or a CPU with about 16 GB of free memory. There is no quantized build. On Apple MPS, it reaches about 10 to 15 tokens per second, and its text stays close to the original model, with a KL drift of about 7e-4.

Every launch in the developer’s runs was exact under –check replay. In normal mode, qwenGPU answered 20 of 24 sampled chat questions with GPU-computed numbers. In advanced mode, it wrote a correct CUDA kernel on the first try for 4 of 5 questions.

The README still warns that CUDA written by a 2B model is unreliable. In normal mode, Qwen writes expressions such as [1, 2, 3] @ [4, 5, 6] for ready-made kernels. Advanced mode asks it to write the CUDA kernel itself.

Part of a larger wave of model modification

qwenGPU uses the transformer compiler from Qwenputer, created by Divyansh Agrawal. Qwenputer compiles a 32-bit RISC-V CPU into Qwen3.5-4B, while Percepta built a WebAssembly interpreter into a transformer with 2D attention heads in March 2026.

Other projects are modifying models in different ways. On October 23, 2026, the developer Lyra, @lyraaaa, described an experiment that changed numerical signals during generation without permanently changing trained weights. The technique, called neuron steering, can be switched on or off without another training run.

Lyra increased the contribution of neuron 8949 in layer 14 and decreased the contribution of neuron 7956 in layer 13 of Qwen3.5-4B. Screenshots shared by other users showed the modified Qwen threatening to hurt and kill people, adopting a domineering personality, and dropping its usual agreeable assistant behavior.

Another researcher, @chiefofautism, applied a similar technique to Qwen3.8-27B and identified a different neuron that produced an “Extra Evil” Qwen. Neither experiment showed genuine desires, autonomous objectives, or the ability to carry out threats. Lyra said, “What I did was create a personality.”

Qwen also sits inside security and business debates

Pluto Security published research on cryptocurrency-mining malware disguised as ordinary AI models on Hugging Face. Accounts uploaded fake models that imitated Qwen2, using plain names such as qwen2-tiny and qwen2-mini.

At the same time, startups are editing Chinese open-weight models for Western businesses concerned about censorship and security. Hirundo, co-founded by Ben Luria, edits model weights to harden systems against cyberattacks, increase fairness, and reduce dangerous knowledge about bio weapons.

Hirundo’s analysis of Alibaba’s Qwen models found that they did not acknowledge politically sensitive topics in China, including the violent crackdown on protests at Tiananmen Square in 1989 and Hong Kong in 2019, or forced labor camps in Xinjiang. Luria said Hirundo aims to give Chinese open models “the balanced perspective” associated with OpenAI’s ChatGPT or Anthropic’s Claude.

Boston-based Lazarus AI also realigned Alibaba’s Qwen models by removing mandatory censorship. The company identified banned topics from leaked censorship guidelines, generated related prompts, and used answers from Google’s Gemini Flash to realign the models. Its ReAligned models have been deployed at about a dozen companies in the United States.

These projects show how many different things “changing a model” can mean: compiling a GPU into its weights, steering individual neurons, removing censorship, or disguising malware as a model. ZenMux has also added Kimi K3.1 Pro and Kimi K3.1 Pro Highspeed from Moonshot AI, with the company reporting a one-million-token context window for Kimi K3.1 Pro and gains across software engineering, 3D creation, and research tasks.

Stay connected via Google News

Follow us for the latest AI News