Generative AI is rapidly transforming how databases work and how they are integrated with large language models (LLMs). Experts are exploring ways to make these technologies work together seamlessly. One key figure in this space is Sailesh Krishnamurthy, VP of engineering for databases at Google Cloud. He leads teams that develop database solutions for Google Cloud and Google’s major services like Search and YouTube. Recently, he shared insights on the challenges and opportunities at the intersection of databases and generative AI.

Bridging the Gap Between LLMs and Operational Data

Krishnamurthy explains that LLMs possess vast amounts of world knowledge, making them valuable for enterprise use. The idea is to combine this knowledge with company data to improve decision-making and automation. However, integrating LLMs with sensitive enterprise data isn’t straightforward. The biggest challenge is ensuring security and privacy because much of the data is heavily permissioned and confidential.

He notes that one approach is information retrieval—using LLMs to search through a collection of documents to find relevant info. But problems arise when data needs to be extracted from live databases and turned into documents for search engines. This process can lead to data becoming outdated or “stale,” especially if the information changes frequently. Krishnamurthy emphasizes the importance of keeping data secure and fresh when connecting LLMs to operational systems.

Choosing Between Federation and Replication

Krishnamurthy discusses two main ways to connect databases with AI systems: federation and replication. Federation involves querying data sources directly in real-time, while replication copies data out of the database and keeps it updated elsewhere. Both methods have pros and cons. Federation reduces data duplication and helps keep information current, but can be slower or more complex to manage. Replication makes data more accessible but risks lagging behind real-time changes.

He observes that many organizations lean towards federation because it allows LLMs and other tools to access data dynamically. This is especially useful when companies want to avoid copying sensitive data or when data changes frequently. Krishnamurthy also points out that many systems use APIs—like microservices—to give controlled access to parts of a database. This way, LLMs can ask for specific information without seeing everything, which helps with security and data governance.

He stresses that the industry has traditionally built small, specialized services for database access. While this approach improves security and control, it also limits what the LLM can see. The challenge is to find ways for these microservices to work together efficiently and securely, providing enough data for AI systems without exposing sensitive information.

Overall, Krishnamurthy believes that the future of databases will involve smarter ways to connect and share data with generative AI. As these technologies evolve, companies will need to balance accessibility, security, and freshness of data to unlock the full potential of AI-driven applications.

Inspired by

• https://www.infoworld.com/article/4112974/generative-ai-and-the-future-of-databases.html

Sources

• github.com
• google.com
• google.com