The Agent Stack Is Becoming the Real AI Advantage

The biggest AI decision may not be the model at all. A better harness, a smarter run mode, and a clear control plane can decide whether an agent becomes useful software or an expensive operational risk.

That shift is moving enterprise AI into a new phase. Agentic systems now trigger workflows, access sensitive data, and make operational decisions, pushing teams past pilot projects and into questions about reliability, security, governance, cost control, and vendor sustainability.

The Harness Can Matter More Than the Model

Most teams treat “which model” as the important decision. Yet the way an agent receives tasks, uses tools, manages context, and handles feedback can reshape its results without changing the underlying model.

“The harness engineering literature keeps pointing somewhere else,” Michal Sutter wrote. In LangChain’s Terminal-Bench experiment, changing only the harness moved a coding agent from roughly 30th place into the top 5, with the same model throughout.

That result turns the spotlight toward architecture. A capable model still needs an execution system that controls its steps, connects the right tools, and prevents wasted work. The model remains important, but the surrounding system determines how much of that ability reaches production.

Vineet Vijay captured the cost and safety lesson: “The instinct to reach for the most capable model for every decision is understandable, but in domains where wrong answers have real consequences, the more valuable engineering work is deciding what should never touch the model at all.”

One decision process uses three stages: deterministic rules, retrieval, and an LLM call. Routing only the genuinely ambiguous 10 to 15% of cases to the LLM cut inference cost by roughly 6X, while a cascade architecture can reduce costs by approximately 6X compared with an all-LLM baseline.

Three Run Modes Reveal Three Cost Profiles

The Python agent Decode makes the operational choice visible by separating its work into three run modes, each with a different latency profile and inference provider:

• Mode 1: Interactive, online
• Mode 2: Remote, offline
• Mode 3: Async, online

That split matters because an agent waiting for a user behaves differently from a batch process handling a large document set. Decode’s default test model, Qwen3.6 35B, runs on a single H200, and the hardware bill changes with the way the agent stays active.

At frontier API rates of $3 per million input tokens and $15 per million output tokens, processing 1,000 documents with 30,000 input tokens each and 500 output tokens per document costs near $97. “The lesson’s arithmetic lands near $97,” Sutter wrote.

Batched processing on a serverless GPU at around 3,000 tokens per second reduces the work to under three hours of GPU time, at roughly $13. Modal’s published pricing lists an H200 SXM at $0.001261 per second, or about $4.54 per hour.

The danger becomes clear in interactive systems. “Leave an interactive agent idle overnight waiting on a y confirmation, and ten idle hours add roughly $45 to the bill.” A single paused workflow can erase the savings expected from a carefully selected model or provider.

These modes also show why latency and price belong in the same design conversation. Interactive work needs responsiveness, remote offline work favors batch economics, and async online work creates another balance between availability and compute use. The best option depends on the work, not just the model benchmark.

Enterprise Agents Need A Stack, Not A Collection Of Pilots

Piecemeal, pilot-era stacks expose organizations to integration brittleness, runaway costs, stale or untrusted data, and governance gaps. Few agent stacks built during the pilot era were designed for enterprise-grade standards, yet agentic AI has moved into core workflows.

The Discover the Enterprise Agentic AI Technology Stack blueprint maps six layers:

• Application
• Data & AI Lifecycle Management Tools
• Foundational Models
• Agentic Execution & Orchestration Engine
• Data Platform
• Infrastructure

The marketplace now spans this full stack, from applications to infrastructure. Teams without a clear architecture can end up with overlapping vendors, control gaps, and little insight into agent activity, while budgets flow toward visible capabilities and leave observability, governance, and integration underfunded.

Vendor selection therefore reaches beyond model quality. The six criteria are functional use cases; operability and production reliability; deployment compatibility and flexibility; governance, security, and compliance; integration and ecosystem fit; and implementation and operational costs.

Adoption patterns show why this decision remains unsettled. Eighty-five percent of enterprises use two or more orchestration tools, 64% use three, and 15% run a single platform. Microsoft AI Foundry/Copilot Studio appears in 70% of stacks, OpenAI’s Agents SDK in 68%, and Anthropic’s Claude Platform in 47%.

More than half of respondents, 53%, expect the primary control plane to be hybrid by the end of 2026. Platform movement is also active: 15% plan to change platforms within three months, 24% within three to six months, and 28% within six to 12 months.

Future platform interest points in several directions. Forty-three percent of builders are exploring the Anthropic-built model, roughly one-third are looking at Google’s Enterprise Agent Platform, 31% are focusing on custom in-house orchestration, and 25% are investigating OpenAI’s options.

Control, Cost, And Visibility Define The Next Phase

Platform satisfaction reaches 4.17 out of 5, while ease of implementation scores 3.91 and value for money scores 3.63. Buyers rank flexibility at 29%, security and permissions at 17%, production reliability at 15%, and control over agent execution at 15% among their top considerations.

Builders invest most in agent monitoring and debugging, at 31%, and security and permissions enforcement, at 30%. Enterprises prioritize task completion reliability at 30%, multi-step workflow management at 27%, developer productivity at 23%, and operational stability at 13%. End-user experience ranks as a top factor for 7% of respondents.

The control gaps are hard to ignore. Security and permissioning limitations lead concerns at 37%, followed by vendor lock-in at 23%, limited visibility and observability at 22%, and inflexibility around models and tools at 16%.

One in five enterprises cannot stop a runaway AI agent’s spending in real time. Control methods include native platform controls at 30%, custom gateway plumbing at 25%, dynamic routing at 25%, and reactive monitoring at 21%. Among enterprises with 10,000-plus employees, 18% exercise only reactive control, compared with 23% of smaller enterprises.

Autonomy remains limited across most deployments. Only 2% of respondents say 76 to 100% of their systems are advanced and autonomous, while 14% report 51 to 75%, 47% report 26 to 50%, and 35% report 1 to 25%; 3% are still deploying only chatbots. Seventy-one percent say a quarter or fewer of their deployed agents can complete multi-step work with autonomy, and about 10% have deployed agents at scale.

“The most critical architectural decision an IT leader can make is building a technology stack designed not for today’s answers, but for tomorrow’s unknowns,” said Andrew Kum-Seun. Bill Wong put the standard plainly: “The vendors worth betting on are the ones that make agents easy to observe, explain, debug, govern, and remove safely.”

The next AI advantage will come from systems that know when to reason, when to retrieve, when to follow rules, and when to stop. The organizations that build that control into the stack now will be ready as agents take on more work, more data, and more operational power.

Stay connected via Google News

Follow us for the latest AI News