Tool use
Tool use is the mechanism where a model pauses inference to output structured arguments for a function, allowing it to interact with external APIs or local software environments.
What it is
The model is provided with a system prompt containing JSON schema definitions of available functions. During a request, the model detects it needs external data or computation and emits a special control token instead of plain text, instructing your backend to execute the logic. Your runtime executes the function and injects the output back into the message stream as a new role in the context window. This round trip typically adds 200ms to 2s of latency depending on your API host and execution environment.
Why it matters
Without tool use, you are limited by the model static training data, which leads to frequent hallucinations regarding real-time state or factual updates. Implementing tools shifts the model from a probabilistic text predictor to a reliable controller for your business logic and databases. Ignoring this forces you to try and stuff excessive context into the prompt, which increases token costs and degrades the model ability to focus on the actual task.
In practice
You configure tool use via parameters like tools or functions in your LLM provider API, passing a JSON array of schema definitions. In production, you must implement a robust loop in your application code that listens for function call types, executes the requested code, and appends the tool output to the chat history to continue the generation process.
The tradeoff
The primary tradeoff is between reliability and complexity: adding more tools increases the likelihood that a model will confuse which function to invoke, necessitating rigorous schema validation and clear, minimal documentation within the tool definition.