VAST Data has launched a new AI inference architecture that redefines how large language models and multi-agent AI systems handle data. By running their AI Operating System directly on NVIDIA BlueField-4 DPUs, they are creating a faster, more efficient way to manage context at scale. This new approach aims to support the growing needs of persistent, multi-turn AI conversations and agentic AI environments.

Revolutionizing AI Storage and Context Sharing

The core of VAST’s new platform is built on NVIDIA BlueField-4 DPUs and Spectrum-X Ethernet networking. This setup allows VAST to eliminate traditional storage tiers and create a shared, pod-scale key-value cache. Instead of moving data back and forth, the system provides deterministic, high-speed access to inference context across multiple nodes. This means AI models can reuse and extend conversation history more efficiently, even under heavy workloads.

By embedding critical data services directly into the GPU servers, VAST reduces delays caused by data transfers and minimizes unnecessary copying. This architecture removes many bottlenecks typical in legacy systems, making context access faster and more predictable. The result is a streamlined data path from GPU memory to persistent storage, enabling high-performance, persistent inference that can keep up as AI conversations grow more complex.

Scaling AI with Shared, Policy-Driven Data Services

The platform leverages VAST’s Disaggregated Shared-Everything (DASE) architecture, which allows each host to access a common, globally coherent context namespace. This design reduces coordination overhead, making it easier to scale AI inference across large clusters. It also supports policy-driven management of context, offering features like isolation, auditability, and lifecycle controls. These are essential for deploying AI in regulated or revenue-critical environments.

Performance isn’t just about raw compute anymore. It depends heavily on how well the system can move, govern, and share context at line rate. VAST’s approach turns context management into a shared infrastructure that stays fast and predictable, even as AI models become more agentic and require persistent reasoning over long conversations. This breakthrough means AI systems can operate more efficiently and securely at scale.

Beyond raw speed, VAST aims to help organizations transition AI inference from experimental setups into production. It provides the tools for managing context with policies, ensuring data security, and maintaining high availability. This makes it easier for enterprises to deploy AI services that are reliable, compliant, and capable of handling complex multi-turn interactions without sacrificing performance.

In summary, VAST’s new inference architecture powered by NVIDIA BlueField-4 DPUs marks a significant step forward. It combines innovative hardware and software design to deliver persistent, shared, and policy-driven context management. This approach prepares AI infrastructure for the agentic, long-lived AI systems of the future, making them faster, more efficient, and easier to scale.

Inspired by

• https://ai-techpark.com/vast-data-redesigns-ai-inference-architecture-for-the-agentic-era-with-nvidia/