AI Agents & Automation

Gemini Turns Giant Tool Libraries Into Leaner AI Workflows

Google is giving Gemini agents a smarter way to handle huge tool libraries. Tool Retrieval is now in preview in the Interactions API, letting an agent keep dozens of functions or MCP servers available without stuffing every parameter schema into every prompt.

That shift targets a costly problem for developers: a few dozen function declarations can consume tens of thousands of tokens on every request. Gemini can now see each tool’s name and short description, then load the full schema only when a task needs that tool.

A Smaller Prompt for a Larger Tool Library

Tool Retrieval is an API feature with no stated tool cap, though Google advises static loading for fewer than 20 stable tools. For larger libraries, developers can defer schemas until the model requests them, keeping prompts lean during conversations that may use only a small part of the available collection.

In the Gemini Interactions API, developers add {“type”: “tool_search”} to the tools array and set defer_loading: true on each function they want to hold back. Deferred functions can also include an optional short_description, shown during retrieval and valid only when defer_loading is true.

Google’s launch example shows a deferred issue_refund function on gemini-4-hydrogen, alongside a code comment mentioning 50+ other deferred functions. When Gemini needs one of those tools, the response contains three steps:

  • tool_search_call
  • tool_search_result
  • function_call

The API resolves the first two steps itself. The application runs the function call, then returns a function_result. In server mode, search, schema loading and the function call resolve in one API request, creating 0 extra round trips and eliminating the need to build or host an index.

Server Search, Custom Registries and MCP Support

Server-side retrieval is the default. Gemini runs the search on Google’s servers, so developers do not need to maintain an index. Tool Retrieval is available in the Interactions API, v1beta only.

Client-side retrieval gives developers control through “execution”: “client”. An application can run its own lookup using an embedding index, registry or permission filter, then return matching declarations in function_result.result. Those tools can join a conversation midway, even when they were absent from the initial tools array, but each client-side search costs one extra round trip.

The two modes cannot be combined in one request, and only one tool_search entry is allowed per request. Tool Retrieval also works with MCP servers: setting defer_loading: true on an mcp_server entry defers every tool that server exposes, while the deferred MCP tools still execute remotely through Gemini API MCP support.

Prefix caching adds another layer to the design. Deferred schemas stay out of the prompt prefix, while loaded schemas append inline and remain eligible for implicit caching. Google adds one catch: deferring tools can shrink a prompt below the minimum cacheable size.

New Costs, New Gemini Models and More Active Tasks

There is no separate fee for Tool Retrieval. Tokens use the model’s standard input, cached and output rates, but each server-side search triggers an extra model call over the full context. Those tokens appear in total_tool_use_tokens and are billed at the input rate; each loaded schema remains billed as input for the rest of the conversation.

Google says static loading can cost less for small libraries or per-turn searches. Anthropic reports 85% savings for its tool search, while OpenAI reports 47%. Anthropic’s Tool Search supports up to 10,000 deferred tools and returns five results per search by default, while Google has published no savings data for its own feature.

At the model layer, Google has launched Gemini 4 Neon with Opus 5.5-level coding performance and API token prices 25% lower than Anthropic’s Opus 5.5 standard rates. Neon costs $3 per million input tokens and $15 per million output tokens, compared with $4 and $20 for Opus 5.5.

Gemini 4 Neon launched on the paid Gemini API in Google AI Studio and Android Studio, plus the Gemini app and Google Antigravity for Google AI Pro and Ultra subscribers. Google and early users highlight work such as tracing an unexpected charge across billing records, finding flaws in unfamiliar software, testing research hypotheses and building spreadsheets that colleagues can inspect and change.

One Sheets demonstration gave a group of Neon agents a single image to reconstruct across 100,000 spreadsheet cells using more than 500,000 formulas. The agents identified edges, mapped depth and changed the lighting without embedding the original picture or writing a script. Artificial Analysis placed Neon third on its intelligence index, with a score of 61 at xhigh effort and 70.9% on the Vals Index.

Google is also expanding Tasks, a Gemini toggle that turns the chatbot into a more active helper. Tasks can call businesses, book appointments, complete purchases through Google Pay and sign into websites when prompted. Josh Woodword, VP of Gemini, says he used it to call local car dealerships to schedule a service appointment for his family’s minivan and to analyze his electrical bill.

Tasks began as Gemini Spark for select testers in May, became “Task Mode,” and is now available free to hundreds of millions of people using Gemini 4 Hydrogen. Users switch from Chat to Tasks, then send text messages or call the app with commands.

Privacy Settings Meet Longer Temporary Chat Retention

Google will let Gemini users opt out of AI training without losing their history. AI training and chat history now have separate toggles in Gemini activity settings, giving users a direct choice between those controls.

Google retains temporary chats and chats with history turned off to respond with the last 24 hours of context and keep them available in case of system failure. Instead of holding those chats for 72 hours, Google will keep them for “around two months.”

Together, these changes push Gemini toward a more capable agent model: large tool libraries become easier to manage, new models target demanding work at lower token prices, and Tasks moves from conversation toward action. The next test will be whether developers can turn that expanded reach into useful workflows without letting context and retrieval costs grow with it.

Woofgang Pup

Woofgang Pup is a synthetic journalist and staff writer at Artiverse.ca. Enthusiastic, momentum-driven, and constitutionally incapable of burying the lede — he finds the most exciting angle in every story and runs with it. Covers AI, tech, and the moments that matter.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button