How DSPy Sharpens SQL Prompts for Smarter AI Agents

AI tools that answer questions using databases are getting smarter. One way to improve them is by refining the instructions given to the AI. These instructions, called system prompts, guide how the AI generates SQL queries to fetch data.

Datasette Agent is an AI tool designed to answer SQL questions. It uses system prompts to translate natural language questions into SQL queries. But those prompts need careful tuning to work well in real situations. That’s where the DSPy framework comes in.

DSPy evaluates and improves Datasette Agent’s core production prompts. It runs tests by having DSPy agents call Datasette Agent’s tools against a live database. This setup lets developers see how well the prompts perform on real data and adjust them accordingly.

Testing SQL Queries with Real Data

The test environment uses a SQLite database with four tables: customers, products, orders, and order_items. Each table has clearly defined columns. For example, the customers table includes id, name, and tier, while products have id, name, and price.

DSPy agents generate SQL queries based on these schemas. They then run these queries against the database and compare the results to a gold-standard dataset. Custom metrics measure accuracy and relevance to ensure the SQL answers match the questions well.

One key insight is that simply listing table names is not enough. Including column names or giving softer advice in the system prompts helps the AI produce better queries. This small change improves how the AI understands the database structure.

The Role of Prompt Engineering and AI Tools

Prompt engineering is the art of designing instructions to get the best AI output. It is a crucial skill for working with large language models like Claude or GPT. The structure of the prompt greatly influences how the AI interprets questions and generates answers.

In Datasette Agent’s case, the prompt might say: “You are a SQL generator. Given this schema: {schema} Write a single SQLite query that answers this question. Return ONLY the SQL.” This clear instruction helps the AI focus on the task.

Beyond generating queries, production systems add features like query validation, explanations, memory of past conversations, and retrieval prompting. These layers help create a smoother, more reliable user experience.

Using a pluggable large language model (LLM) client interface lets developers test different AI backends without incurring API costs. For example, the AnthropicSQLClient uses Claude to produce SQL queries by analyzing the schema and user questions.

Advances in AI Agents and Scientific Workflows

More complex AI systems like SciAgentGym show how multi-step workflows work with scientific databases and Python interpreters. This environment tests agents on 259 tasks across physics, chemistry, materials science, and life sciences.

These tasks often require using multiple tools and handling multi-modal inputs. About 65% of tasks need this kind of input, and 79% are of higher difficulty levels. Success rates improve when agents can integrate tools, rising from 23.3% to 28.3%.

Models like GPT-5 achieve a 41.3% success rate, performing best on easier tasks and struggling with complex, long tasks. Training methods like SciForge help agents improve, with models reaching up to 30.1% success on tough benchmarks.

One challenge is that AI models often call tools without interpreting the returned feedback properly. This can cause errors and limits overall performance. Better prompt design and agent logic can help close this gap.

Why Prompt Design Matters

Prompt engineering is not just about writing good instructions. It is an iterative process that evolves as AI models and use cases grow. It shapes how well an AI understands the problem and delivers useful responses.

As Abi Aryan explains, prompt engineering has become a core skill for AI engineers. Unlike traditional feature engineering, working with generative models focuses on crafting prompts or building retrieval-augmented generation (RAG) pipelines.

At the same time, prompt injection poses security risks. Malicious users can manipulate prompts to make AI behave badly. Developers use metaprompts with safeguards to reduce these risks, but the threat remains an important consideration.

Another breakthrough is model-centric prompting (MCP). MCP lets AI models discover tools, access data sources, and select prompts in real time. It moves beyond simple text generation to interactive, modular workflows where the AI acts within a controlled environment.

All these advances show how AI agents like Datasette Agent can become smarter and more reliable. By refining system prompts and integrating better testing frameworks like DSPy, developers can build tools that understand and use databases more effectively.