Tech execs pushing to get agentic AI projects into production will have to surmount complicated challenges to prevent their efforts from failing, according to the CEO of a San Francisco-based AI startup.

Companies need to establish a roadmap, outline deliverables, and experiment to achieve successful project execution, Curtis Northcutt, Co-founder and CEO of Cleanlab, said in an interview last week with Computerworld. “The moment that these enterprises and these CIOs take a break or the moment that you think, ‘Oh, we finally got it figured out’ — that’s the moment you fall behind,” he said.

Cleanlab

Agentic AI projects are challenging because the technology is changing so rapidly that it’s difficult to stay on top of developments. Companies have to switch AI goals and technologies rapidly because of unprecedented advances.

While AI has been around for a while using technologies such as voice recognition and natural language processing, the arrival of generative AI and large language models (LLMs) three years ago changed the whole market.

After OpenAI’s ChatGPT came a string of new technologies, including RAG and agentic RAG, larger training models for improved responses from LLMs — and now, AI agents and multimodal models, which are changing agentic responses.

“And then someone else comes in and they just have a much smarter and better LLM,” Northcutt said. “And this is every week for enterprises.”

Cleanlab, which offers products that make orchestration safer and less complicated by reducing hallucination and failure rates, conducted a survey of 1,837 tech executives and engineers, in which only 5% had AI agents in production.

“Between 60% and 70% of everyone that we chatted with, both in survey and also in sales calls, they were changing their entire stack — their LLM, the AI stack that they build an agent on — every three months at least,” Northcutt said.

A good agentic AI plan involves clearly defining the use case, creating a product requirement document with the inputs and outputs, and assigning a product manager to champion the project.

Once an early prototype is in place, “then the interesting stuff starts happening,” he said. “So, we’ve got something that works very, very simply.”

At that point, organizations should add guardrails and monitoring to the system, “so we know what’s going on if anything goes wrong. And we’re still not ready for production. We’re just putting those in place,” Northcutt said.

After that, organizations can add orchestration, which adds the “ability for you to be able to change something or add a different route or feedback or fallback responses if it makes a mistake.”

Humans also need to be in the loop to give advice or guidance and directly improve any AI system. “…Then after that, you build all the knowledge curation and training on that advice. And this is all happening when you’re already in production. This is a post-production world.”

That said, most agentic AI projects are still either not ready for prime time or aspirational. “The world thinks that agents are a lot further on than they really are in reality. It’s going to take…probably a lot longer for this stuff to actually hit the mainstream,” Northcutt said.

In many cases, the market is confused about which agentic AI technology to implement and where to put in orchestration, guardrails and other steps. “Companies are also getting caught up in ‘what my prompt should even be’ and ‘how do I even get something built before they can even start improving the system,’” Northcutt said.

IT decision makers should innovate while also looking for connect with partners already specializing in AI to get projects past the finish line, Northcutt said. He noted that companies continue to be concerned about control of models and data, which can stall progress. 

CIOs don’t have to invest a lot to experiment with AI, especially when done through partnerships, and they can still keep control of data and AI projects, Northcutt said.

But he offered a cautionary note about how quickly agentic AI would be a reliable technology in day-to-day operations. “The reality is real AI agents that are agentic and have tool calling … is probably early 2027,” Northcutt said.