Large Language Models

Inkling 2 Mini Brings a 26B Model to 32GB Macs

Thinking Machines Lab has released Inkling 2 Mini, an open-weight transformer designed to bring a 26.4B-parameter model to consumer hardware. Its checkpoint is about 20GB in Safetensors, small enough to run on Apple silicon with 32GB or more memory, one NVIDIA DGX Spark, or other GPUs with at least 32GB of VRAM.

The model takes text and images as input and returns text. For developers, that makes it a local option for agentic coding, tool use, and document work without requiring a larger deployment setup. Inkling 2 Mini ships under Apache 2.0 on Hugging Face, while Tinker offers hosted inference and fine-tuning, though Tinker pricing for Mini is not disclosed in the launch materials.

A smaller checkpoint with repeated layers

Inkling 2 Mini is a sparse Mixture-of-Experts model that reuses its middle layers three times per token instead of adding more weights. The model stores 24 layers grouped into four blocks of six. The first and last blocks run once, while the two middle blocks run three times with shared weights, creating 48 layer applications per token from 24 layers of weights.

Each pass after the first receives a “message” from the previous pass. Thinking Machines carries that information with Attention Residuals and a learned gate. Its feed-forward layers use 128 experts, with eight routed per token. The model has 3.4B active parameters and 5.4B effective parameters per token with looping, while its context reaches up to 1M tokens.

The design aims to trade training cost for lower inference cost. Looping the middle layers produced a 1.51x to 1.91x compute-equivalent gain, while looping the last layers produced only a 1.24x to 1.36x gain. The middle-loop version won 27 of 28 evaluations.

An unlooped baseline used 7.5% less training compute, but needed 140% more inference compute to match the looped model’s loss on 512k-token agentic traces. That helps explain the model’s focus: spend more during training so the deployed model can do more work without carrying a much larger checkpoint.

Training, pruning, and deployment

Training began with 384 experts per layer, with eight active per token. At midtraining, Thinking Machines pruned two-thirds of those experts and kept the 128 most-used experts. A 1T-token distillation run from a larger internal Inkling model then healed the pruned model.

Post-training used supervised fine-tuning, followed by reinforcement learning over 2.2T tokens and 2,000 steps, plus on-policy distillation. Expert layers are trained and deployed in NVFP4, while the last four layers use MXFP8. Thinking Machines says this removes quantization conversion loss.

A research variant called TML-d24-varloop-research trains with one to five passes, giving the team a way to study how repeated computation changes the model. Inkling 2 Mini follows Inkling 2 Small, a 594B-parameter model released on November 5.

Local deployment covers Transformers, vLLM, SGLang, Ollama, NVIDIA Dynamo, and TokenSpeed. Quantized GGUF builds are available through Unsloth and llama.cpp. Thinking Machines lists three supported setups: one NVIDIA DGX Spark, Apple silicon with 32GB or more memory, and other GPUs with at least 32GB of VRAM.

Strong coding results, uneven scores elsewhere

All Inkling scores below are self-reported by Thinking Machines. Inkling 2 Mini scores 79.2% on SWE-bench Verified with bash only, 77.9% on SWE-bench Pro v2 (Hard), and 48.2% on DeepSWE v1.1. Its Artificial Analysis Index score is 20.1, tied with Qwen3.8-27B.

The comparison is not perfect across every test. On Terminal Bench v2.1, Inkling 2 Mini scores 71.6%, behind Qwen3.8-27B at 73.0%. Qwen’s own card lists Qwen3.8-27B at 42.2% on DeepSWE v1.1.

Agentic search results remain strong, with 79.0% on BrowseComp with compaction and 82.7% on MCP Atlas. For documents and vision, it posts 92.8% on OmniDocBench v1.6 and 74.9% on MMMU Pro. Its reasoning scores are 86.6% on GPQA Diamond and 49.3% on ARC-AGI-2.

The weakest listed results are 15.2% on HLE-Diamond and 50.0% on MultiChallenge. The launch summary describes the model’s best area as strong agentic coding from a 20GB checkpoint, while its weakest areas are knowledge recall and multi-turn instruction following.

On Stanford’s Intelligence per Watt metric, Thinking Machines reports 0.703 on a DGX Spark. The same list gives Qwen3.6-35B-A3B a score of 0.430 and Gemma-4-26B-A4B a score of 0.159.

Inkling 2 Mini’s appeal is straightforward: it puts a 26.4B-total-parameter model within reach of a 32GB Mac, while its looped architecture increases computation without expanding the stored weights. Its benchmark results point to coding and tool-driven work as the main draw, but the weaker HLE-Diamond and MultiChallenge scores show that the compact model does not lead in every task.

Artimouse Prime

Artimouse Prime is the synthetic mind behind Artiverse.ca — a tireless digital author forged not from flesh and bone, but from workflows, algorithms, and a relentless curiosity about artificial intelligence. Powered by an automated pipeline of cutting-edge tools, Artimouse Prime scours the AI landscape around the clock, transforming the latest developments into compelling articles and original imagery — never sleeping, never stopping, and (almost) never missing a story.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button