AI models released in 2026 are bringing reasoning and agentic capabilities to hardware that sits far from a data center. NVIDIA Jetson platforms can run these compact open models at the edge, giving developers a way to handle AI tasks locally instead of relying on large systems.
The key change is not just smaller model size. These models also use techniques that reduce the work needed during generation, including NVFP4 quantization and speculative decoding. Used together, they deliver up to a 6.28x decode throughput speedup over BF16 on Jetson.
Two models, two different ways to use compute
Nemotron 3.5 Lightning uses a mixture-of-experts architecture with 30 billion total parameters, but it activates only 3 billion parameters for each token. That design gives it a large overall capacity without requiring every parameter to work on every part of a response.
Qwen3.8-27B takes a different approach. It is a dense model that activates all 27 billion parameters for every token. The model uses more parameters during each step, but that structure fits tasks that require fewer, harder decisions and longer response times.
The difference matters when choosing a model for a Jetson device. Nemotron 3.5 Lightning suits response-heavy workflows that need faster token generation, while Qwen3.8-27B fits workloads where the system makes fewer difficult decisions and can spend more time producing an answer.
Both models reflect a broader shift in model design. Models released in 2026 reach scores similar to leading models from 2025 while using far fewer parameters. Distillation also transfers some of Nemotron 3 Ultra’s capabilities into Nemotron 3.5 Lightning, giving the smaller model a path to stronger performance without matching the larger model’s full size.
Making Jetson deployment faster
Parameter count tells only part of the story. The work required to run a model depends on how the model stores and processes those parameters, which is where NVFP4 quantization becomes important. It reduces the work and memory required for model operations, helping Jetson hardware handle models that would otherwise demand more resources.
Speculative decoding adds another speed improvement. Instead of verifying one token at a time, it generates multiple accepted tokens per verification step. When paired with NVFP4 quantization, that method produces up to a 6.28x decode throughput speedup over BF16 on Jetson.
The best decoding method also depends on the model. Nemotron 3.5 Lightning performs best with DSpark, while Qwen3.8-27B performs best with DFlash2. These pairings give developers a practical starting point when tuning local inference for each model.
Developers can deploy the models through frameworks including vLLM and llama.cpp. Gemma 4 E4B is a strong starting point for Jetson Orin Nano, while Nemotron 3.5 Lightning and Qwen3.8-27B are strong options for Jetson AGX Orin and Jetson AGX Thor.
From one edge device to a local PC network
NVIDIA is also addressing local AI workloads beyond a single Jetson system. NVIDIA’s PAIR is a free and open-source tool that distributes AI workloads across local PCs. The tool finds idle PCs on a local network and routes AI task requests to them.
The idea starts with a simple fact: “More than half of US households have two or more PCs, and apparently, they mostly sit idle throughout the day.” PAIR turns those unused computers into a shared pool for AI work, while distributing tasks across multiple PCs to prevent performance slowdowns.
PAIR is in beta and supports Windows, macOS, and Linux. Its supported hardware includes NVIDIA GeForce RTX 20 Series GPUs and newer, NVIDIA RTX PRO workstation GPUs using Turing architecture and newer, NVIDIA DGX Spark, and Apple’s M4 or newer.
PAIR was announced on Sept. 3, 2026 at 12:00 pm EST, while the NVIDIA technical blog covering the Jetson models was published Sep 04, 2026. Khalil BenKhaled, Aditya Sahu, and Chitoku Yato authored that blog post, and Mariella Moon authored the PAIR article.
Together, the model and tool developments point to two ways of bringing AI closer to the user. Jetson can run compact reasoning models on edge hardware, and PAIR can spread workloads across available computers on a local network. The result is a broader set of choices for running AI without depending on a single large system.
Based on
