Devops used to be a simple process. Take one component of the stack, run some unit tests, check a microservice in isolation, confirm it passed integration tests, and ship it. The problem is, that doesn’t test what actually matters—whether the system as a whole can handle production workloads.

This simple approach breaks down fast when AI workloads start generating massive volumes of data that need to be captured, processed, and fed back into models in real time. If data pipelines can’t keep up, AI systems can’t perform. Traditional observability approaches can’t handle the volume and velocity of data that these systems now generate.

From component testing to platform thinking

Devops must evolve beyond simple CI/CD automation. That means teams need to build comprehensive internal platforms—what I think of as “paved roads”—that replicate whole production environments. For data-intensive applications, developers should be able to create dynamic data pipelines and immediately verify that what comes out the other end meets their expectations.

Testing for resilience needs to happen at every layer of the stack, not just in staging or production. Can your system handle failure scenarios? Is it actually highly available? We used to wait until upper environments to add redundancy, but that doesn’t work when downtime immediately impacts AI inference quality or business decisions.

The challenge is that many teams bolt on observability as an afterthought. They’ll instrument production but leave lower environments relatively blind. This creates a painful dynamic where issues don’t surface until staging or production, when they cost significantly more to fix.

The solution is instrumenting at the lowest levels of the stack, even in developers’ local environments. This adds tooling overhead up front, but it allows you to catch data schema mismatches, throughput bottlenecks, and potential failures before they become production issues.

Connecting technical metrics to business goals

It’s no longer enough to worry about whether something is “up and running.” We need to understand whether it’s running with sufficient performance to meet business requirements. Traditional observability tools that track latency and throughput are table stakes. They don’t tell you if your data is current, or whether streaming data is arriving in time to feed an AI model that’s making real-time decisions. True visibility requires tracking the flow of data through the system, ensuring that events are processed in order, that consumers keep up with producers, and that data quality is consistently maintained throughout the pipeline.

Streaming platforms should play a central role in observability architectures. When you’re processing millions of events per second, you need deep instrumentation at the stream processing layer itself. The lag between when data is produced and when it is consumed should be treated as a critical business metric, not just an operational one. If your consumers fall behind, your AI models will make decisions based on old data.

The schema management problem

Another common mistake is treating schema management as an afterthought. Teams hard-code data schemas in producers and consumers, which works fine initially but breaks down as soon as you add a new field. If producers emit events with a new schema and consumers aren’t ready, everything grinds to a halt. 

By adding a schema registry between producers and consumers, schema evolution happens automatically. The producer updates its schema version, the consumer detects the change, pulls down the new schema, and keeps processing, with no downtime required.

This kind of governance belongs at the foundation of data pipelines, not something added later. Without it, every schema change becomes a high-risk event.

The devops role is evolving

Implementing all these changes requires a different skill set. Rather than just coding infrastructure, you need to understand your organization’s business objectives and trace them back to operational decisions.

As AI handles more coding tasks, developers will have more bandwidth to apply this more holistic systems thinking. Instead of spending 30 minutes writing a function, they can spend one minute prompting an AI to do the same thing, and 29 minutes understanding why the function is needed in the first place. Junior developers who once owned a narrow slice of functionality will have time to understand the entire module they’re building.

As developers spend less time coding and more time orchestrating systems, everyone can start thinking more like an architect. That means AI is not eliminating jobs; it’s giving people more time to think about the “why” instead of just the “what.”

Making AI a copilot, not a black box

Developers will trust AI tools when they can see the reasoning behind the code being generated. That means showing the AI’s actual thought process, not just providing a citation or source link. Why did the AI choose a particular library? Which frameworks did it consider and reject?

Tools like Claude and Gemini are getting much better at exposing their reasoning, allowing developers to understand where a prompt might have led the AI astray and adjust accordingly. This transparency turns AI from a black box into more of a copilot. For critical operations, like production deployments and hotfixes, human approval is still essential. But explainability makes the collaboration between developers and AI tools actually work.

The path forward

Devops teams that cling to component-level testing and basic monitoring will struggle to keep pace with the data demands of AI. The teams that do well will be the ones that invest in comprehensive observability early on, instrument their entire stack from local development to production, and make it easy for engineers to see the connection between technical decisions and business outcomes.

This shift won’t be trivial. It will require cultural change, new tooling, and a willingness to slow down up front to move faster later on. But we’re past the point where we can hope our production applications behave like they did in staging. End-to-end observability will be the foundation for building resilient systems as AI continues to progress.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.