Tool Use and Function Calling

RAG lets a language model read external information. It can retrieve policy pages, meeting notes, documentation, and other text, then place that evidence into the context.

But what if the answer does not exist in a document?

Questions like What is my bank balance?, What is the weather right now?, Calculate the mortgage payment, Send this email, or Book me the cheapest flight from Vancouver to Tokyo next Friday require computation, live state, or action.

The model does not need more static knowledge. It needs a way to interact with the outside world. Tool use is how an LLM becomes the controller of a larger software system.

A human employee does not memorize everything. They know when to use a calculator, open a calendar, search a database, ask accounting, or send an email. Intelligence is partly reasoning and partly choosing the correct tool.

LLM tool use works similarly. The model reads the conversation and the available tool descriptions. It decides whether a tool is needed. If so, it emits a structured tool call. The application validates and executes that call. The result returns to the model, which writes the final user-facing answer.

The model itself does not check the weather, book the flight, or multiply large numbers. It produces structured instructions that software can execute.

Function Calling Is Still Token Prediction

Tool use is not a separate mental organ inside the model. The model still predicts tokens:

The output $y$ may be natural language, or it may be a structured string that represents a function call. The conversation, system prompt, tool descriptions, and schemas all become context that changes the next-token distribution.

If the context includes a tool named search_flights with fields origin, destination, and date, the model can learn to emit JSON matching that schema:

{
  "function": "search_flights",
  "arguments": {
    "origin": "Vancouver",
    "destination": "Tokyo",
    "date": "next Friday",
    "sort_by": "price"
  }
}

Why Schemas Work

Without a schema, the model can output arbitrary text. With a schema, the application can require a specific structure: required fields, types, enums, ranges, and nested objects. Some systems also use constrained decoding, where invalid next tokens are removed from the decoder's valid set.

If the valid token set at a step is $V_{\text{valid}}$ rather than the full vocabulary $V$, decoding searches a smaller space:

This does not make the tool call semantically correct, but it improves syntactic reliability.

A tool is an external capability exposed to the model through a description and schema, then executed by the application.

Function Signatures

A tool schema resembles a normal function signature:

The model must choose the function and generate arguments. Argument generation is information extraction: from weather tomorrow in Vancouver, extract city=Vancouver, date=tomorrow, and a default units=metric.

For What is the weather tomorrow in Vancouver?, the system might proceed like this:

1. User asks the question.
2. The model reads available tool descriptions.
3. The model selects get_weather.
4. The model emits JSON arguments.
5. The application parses and validates the JSON.
6. The application calls the weather API.
7. The API returns temperature, condition, humidity, and wind.
8. The model turns raw JSON into a useful natural-language answer.

For Email Sarah tomorrow's forecast, the system needs sequential tools: first weather, then email. Sending the email should require confirmation because it changes the outside world.

Applications should never blindly trust model output. Validate before execution:

• Required fields are present.
• Values have the correct type.
• Enums use allowed values.
• Numbers are in safe ranges.
• The user has permission.
• The tool actually exists.
• The action is safe or confirmed.

{
  "function": "search_flights",
  "arguments": {
    "origin": "Vancouver",
    "destination": 42,
    "date": null
  }
}

This is valid-looking JSON, but invalid for the schema. destination should be a string, and date is required.

Tool calls also fail at runtime: API timeout, invalid city, authentication failure, network error, unavailable provider, or partial failure. A good assistant should recover gracefully, retry when appropriate, ask for missing information, or explain what failed.

A single function call solves one narrow step. Many requests require a loop:

The loop should have guardrails: maximum tool calls, timeouts, tool allowlists, confirmation for side effects, and clear stopping criteria. Without these, a model can enter an infinite tool loop or take actions the user did not approve.

Tool use raises the stakes because model output can become real action. Security must be enforced by application code.

• Prompt injection can trick the model into calling dangerous tools.
• Tool injection can hide malicious instructions in tool output.
• SQL injection can appear inside generated database arguments.
• Command execution tools require strict sandboxing and allowlists.
• Emails, purchases, bookings, deletes, and transfers require confirmation.
• Tool results should be treated as data, not higher-priority instructions.

The rule is simple: never trust model outputs merely because they are structured. Structure makes execution possible; validation and permission checks make it safe.

From Scratch Dispatcher

from dataclasses import dataclass
from typing import Callable, Any

@dataclass
class Tool:
    name: str
    description: str
    required: dict[str, type]
    execute: Callable[[dict[str, Any]], dict[str, Any]]

def validate(tool: Tool, arguments: dict[str, Any]) -> None:
    for field, expected_type in tool.required.items():
        if field not in arguments:
            raise ValueError(f"Missing required field: {field}")
        if not isinstance(arguments[field], expected_type):
            raise TypeError(f"{field} must be {expected_type.__name__}")

def dispatch(call: dict[str, Any], tools: dict[str, Tool]) -> dict[str, Any]:
    tool_name = call.get("function")
    if tool_name not in tools:
        raise ValueError(f"Unknown tool: {tool_name}")

    tool = tools[tool_name]
    arguments = call.get("arguments", {})
    validate(tool, arguments)
    return tool.execute(arguments)

Execution Loop

messages = [{"role": "user", "content": user_request}]

for step in range(max_tool_calls):
    model_output = llm(messages, tools=tool_schemas)

    if model_output.type == "final":
        return model_output.content

    if model_output.type == "tool_call":
        result = dispatch(model_output.tool_call, tools)
        messages.append({
            "role": "tool",
            "name": model_output.tool_call["function"],
            "content": result,
        })

raise RuntimeError("Tool loop exceeded the allowed step limit.")

Real systems add authentication, retries, logging, rate limits, user confirmation, audit trails, and typed schemas.