Tool use

What it is

The model is provided with a system prompt containing JSON schema definitions of available functions. During a request, the model detects it needs external data or computation and emits a special control token instead of plain text, instructing your backend to execute the logic. Your runtime executes the function and injects the output back into the message stream as a new role in the context window. This round trip typically adds 200ms to 2s of latency depending on your API host and execution environment.

Why it matters

Without tool use, you are limited by the model static training data, which leads to frequent hallucinations regarding real-time state or factual updates. Implementing tools shifts the model from a probabilistic text predictor to a reliable controller for your business logic and databases. Ignoring this forces you to try and stuff excessive context into the prompt, which increases token costs and degrades the model ability to focus on the actual task.

In practice

You configure tool use via parameters like tools or functions in your LLM provider API, passing a JSON array of schema definitions. In production, you must implement a robust loop in your application code that listens for function call types, executes the requested code, and appends the tool output to the chat history to continue the generation process.

The tradeoff

The primary tradeoff is between reliability and complexity: adding more tools increases the likelihood that a model will confuse which function to invoke, necessitating rigorous schema validation and clear, minimal documentation within the tool definition.

Where it appears

Research summaries that use Tool use, each linked to its source paper.

• Improving Multi-Turn AI Agent Tool Use
  From State to Action: OODA-Tool for Reliable Multi-Turn Tool Use

  The OODA-Tool method improves reliability in complex multi-turn tool interactions by decoupling state management from the process of executing actions.

• Improving AI Tool Use Through Mid-Training
  MidTool: Mid-training Data Synthesis for Agentic Tool Use

  The researchers created a 20.3B-token corpus called MidTool-Mix to improve agentic tool-use capabilities in models during the mid-training phase rather than relying solely on post-training.

• Improving Tool Use With Looped Language Models
  Looped Language Models Improve Compositional Tool Calling

  The researchers developed Ouro models that use recurrent stack iterations to improve performance on complex, compositional tool-calling tasks.

• Benchmarking Multi-Hop Reasoning for AI Agents
  VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies

  VAKRA evaluates how well AI agents perform complex multi-step reasoning by combining structured API calls with document retrieval.

• Spanish Cybersecurity Vision Language Model
  VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use

  VectraYX-Vision-1B is a specialized vision language model designed for offline cybersecurity reasoning and native tool invocation in Spanish and Latin American contexts.

• Improving Visual Tool Use for Models
  OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

  OpenVisTool introduces a training method that teaches models to use external visual tools only when necessary, improving performance over fixed image encoding.

• Improving AI Agent Tool Use Accuracy
  TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

  The researchers developed a method to assign rewards to intermediate steps in agent interactions to improve performance on long-horizon tasks.