Developers have spent the past decade trying to forget databases exist. Not literally, of course. We still store petabytes. But for the average developer, the database became an implementation detail; an essential but staid utility layer we worked hard not to think about.

We abstracted it behind object-relational mappers (ORM). We wrapped it in APIs. We stuffed semi-structured objects into columns and told ourselves it was flexible. We told ourselves that persistence was a solved problem and began to decouple everything. If you needed search, you bolted on a search system. Ditto for caching (grab a cache), documents (use a document store), relationships (add a graph database), etc. We thought we were being clever but really we were shifting complexity from the database engine into glue code, pipelines, and operational overhead.

The architectural frailty of this approach has been laid bare by AI.

In an AI-infused application, the database stops being a passive store of record and becomes the active boundary between a probabilistic model and your system of record. The difference between a cool demo and a mission-critical system is not usually the large language model (LLM). It is the context you can retrieve, the consistency of that context, and the speed at which you can assemble it.

AI has made the database visible again. We are not suddenly loving SQL again, but we are realizing that AI memory is just another database problem. Your database is no longer just where data lives. It is where context gets assembled, and in AI, context is everything.

Consistency and hallucinations

To understand why we are struggling, we have to look at how we got here. We didn’t eliminate the database during the past 10 years. We actually multiplied it.

Modern application design taught us to route around database limits. We built caches, search clusters, stream processors, and a cacophony of purpose-built stores. We convinced ourselves that “polyglot persistence” (wiring together five different “best of breed” systems) was architectural enlightenment. In reality, it was mostly just résumé-driven development that shifted complexity from the database engine to the application code.

That worked when “eventual consistency” was acceptable. In AI, it doesn’t work. At all.

Consider a practical retrieval-augmented generation (RAG) pipeline. It is rarely as simple as “fetch vectors.” A real enterprise AI workflow needs vector similarity search to find semantic matches, document retrieval to fetch the content, graph traversal to understand relationships like permissions or hierarchy, and time-series analysis to ensure the data is not stale. In the “bolt-on” architecture of the last decade, you implement that by composing specialized services: a vector database, a document store, a graph database, and a time-series system.

Such a bolt-on architecture wasn’t ideal but neither was it the deal-killer that AI makes it. Every arrow in the flow is a network hop, and every hop adds serialization overhead. Making matters worse, every separate system introduces a new consistency model. You are paying a tax in complexity, latency, and inconsistency when you split what should be one logical context into six different physical systems. AI is uniquely sensitive to this tax.

When a normal web app shows stale data, a user might see an old inventory count for a few seconds. When an AI agent retrieves inconsistent context (perhaps fetching a vector that points to a document that has already been updated in the relational store), it constructs a plausible narrative based on false premises. We call these hallucinations, but often the model is not making things up. It is being fed stale data by a fragmented database architecture. If your search index is “eventually consistent” with your system of record, your AI is “eventually hallucinating.”

How about if your transactional system is the source of truth, but your vector index updates asynchronously? Well, you’ve built a time lag into your agent’s memory. If your relationship data is synced through a pipeline that can drift, your agent can “know” relationships that are no longer true. If permissions are checked in one system while content is fetched from another, you are one bug away from data leakage.

This problem gets worse when we move from passive chatbots to active agents. We expect the next generation of AI to perform tasks, not just summarize text. But doing things requires transactions. If your agent needs to update a customer record (relational), re-index their preference profile (vector), and log the interaction (document), and your database architecture requires a saga pattern to coordinate those writes across three different systems, you have built a fragility engine. In a fragmented stack, a failure halfway through that workflow leaves your agent’s world in a corrupted state. You cannot build a reliable agent if it cannot rely on atomicity, consistency, isolation, durability (ACID) guarantees across its entire memory space.

That is not an AI problem. It’s a basic architecture problem. As I have noted, you cannot build a reliable agent on unreliable data infrastructure. Reliability work for agents is inseparable from database work.

Architectural constraints

For years, developers have accepted good-enough performance because we could simply scale horizontally. If a query was slow, we added nodes. But AI workloads are computationally heavy, and the underlying physics of our data structures matters again.

Take the simple act of reading a JSON document. In some popular document stores, the underlying binary format requires sequential field scanning. That is an O(n) operation. To find a field at the end of a large document, the engine must scan everything preceding it. That may work for a simple CMS, but it completely breaks down when you’re operating at enterprise scale (say, 100,000 requests per second during a viral event), because nanoseconds compound. An O(n) scan at that volume can waste dozens of CPU cores just parsing data. In contrast, newer binary formats use hash-indexed navigation to allow O(1) jumps to specific fields, delivering performance gains of 500 times or more for deep document traversal.

This is not a micro-optimization. It is a fundamental architectural constraint. “Just nanoseconds” is what you say when you have not operated at the scale where latency kills the user experience. In an AI world where inference is already slow, your database cannot afford to add latency due to inefficient algorithmic foundations.

Stop building infrastructure

I am not saying there is no place for specialized tools. But the mistake we made in the last decade was assuming we could assemble five specialized systems and get something simpler than one general system. For AI, the question to ask is not “Which database has vector search?” The question is “Where does my context live, and how many consistency boundaries do I cross to assemble it?”

If the answer is “a lot,” you are signing up to build infrastructure. You are building pipelines, retries, reconciliation logic, and monitoring for lag you will eventually treat as normal.

The alternative is to stop treating data models as physical mandates. In the past, if we needed a graph, we copied data to a graph database. If we needed vectors, we copied data to a vector store. That copying is the root of the synchronization evil.

The scientific approach is to treat data as having one canonical form that can be projected into whatever shape the application needs. Do you need to traverse relationships? The database should project a graph view. Do you need to search semantics? It should project a vector view. These should not be copies that require pipelines. They should be different lenses on the same single source of truth. When the record updates, every projection updates instantly.

If you are building an AI-infused application and your plan involves maintaining multiple pipelines to keep multiple databases in sync, you are not building AI. You are building a context delivery system, and you are taking on all the operational risk that comes with it.

Here is the decision rule I would use. Count how many consistency boundaries your agent crosses to answer a question. Count how many physical copies of the same truth exist across your stack. If either number keeps growing, your reliability work is already in trouble.

This is why the database matters again. AI is forcing us to confront what we spent a decade abstracting away: The hard part of software is turning messy reality into a coherent, queryable representation of the world. In the age of AI, context rules. Databases, for all their supposed boringness, are still the best tool we have for delivering that context securely at scale. They allow you to delete the infrastructure that provides no business value: the ETL jobs, the synchronization pipelines, the object-relational mapping layers, and the distributed transaction coordinators. The best code is the code you do not write.