Skip to content

Pick the right tool

Pick the tool and fill its arguments, or hand a close call to a person.

An agent with many tools has to decide, on every turn, which one to call and with what. Pasting every tool's schema into a large model's prompt costs tokens each time, and a wrong call can refund the wrong order.

Here Sage2 makes that choice in one Choice call, then fills the tool's arguments with one extraction. Your agent gets fewer wasted tokens and fewer wrong tool calls, and a clear "not sure" when a person should step in.

  • Choice
  • Extract
Try
  1. Tools the agent can call
    • search_orders: Look up the status or details of a customer's order
    • issue_refund: Give a customer money back for an order
    • book_meeting: Put a meeting in the calendar
    • search_docs: Answer a how-to question about our product from the help center, such as passwords, settings or billing
    • ask_a_person: Hand over to a person when no other tool fits
  2. Pick the tool for the requestChoice
    • search_orders0.25%
    • issue_refund99.62%
    • book_meetingunder 0.01%
    • search_docsunder 0.01%
    • ask_a_person0.12%
    61 ms
  3. Fill the arguments of that toolExtract
    order_number
    7781
    amount
    €64.00
    64 ms
  4. Your codeCalls issue_refund

    The agent runs this tool with order_number: 7781, amount: €64.00.

The Choice question took 61 ms. Extract ran after it, for the tool that won: 64 ms.

Get the code

Set SAGE_API_KEY (get a free key), then run it. Python: pip install requests. TypeScript: Node 22+. Save as recipe.mts and run npx tsx recipe.mts.

A null answer is not an error. Sage2 isn't sure, and each step below says what to do with it. Limits, errors and retries: Limits and errors.

import os
import requests

def sage(path, body):
    r = requests.post(
        "https://sage.levanto.ai" + path,
        headers={"Authorization": f"Bearer {os.environ['SAGE_API_KEY']}"},
        json=body,
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

message = "Order 7781 arrived smashed. Please refund the €64 I paid."

TOOLS = [
    {
        "id": "search_orders",
        "description": "Look up the status or details of a customer's order",
        "fields": {"order_number": "the order number"},
    },
    {
        "id": "issue_refund",
        "description": "Give a customer money back for an order",
        "fields": {
            "order_number": "the order number",
            "amount": {"type": "money", "definition": "the amount to refund"},
        },
    },
    {
        "id": "book_meeting",
        "description": "Put a meeting in the calendar",
        "fields": {
            "person": "who the meeting is with",
            "date": {"type": "date"},
            "time": {"type": "time"},
        },
    },
    {
        "id": "search_docs",
        "description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
        "fields": {"topic": "what the question is about"},
    },
    {
        "id": "ask_a_person",
        "description": "Hand over to a person when no other tool fits",
        "fields": {"reason": "why a person is needed"},
    },
]

pick = sage("/decide", {
    "content": message,
    "question": {
        "id": "tool", "kind": "choice", "instructions": "Which tool should handle this request?",
        "options": [{"option": t["id"], "description": t["description"]} for t in TOOLS],
    },
})["result"]
tool = pick["chosen"]

def fill_arguments(message, tool):
    fields = next(t["fields"] for t in TOOLS if t["id"] == tool)
    return sage("/v1/extract", {"content": message, "extract": fields})["values"]

BAR = 0.98  # a tool that moves money runs on its own only at or above this
MONEY_TOOLS = {"issue_refund"}

if tool is None:
    print("Too close to call: hand over to a person")
else:
    args = fill_arguments(message, tool)
    missing = [name for name, value in args.items() if value is None]
    if tool == "ask_a_person":
        print("Hand over to a person:", args["reason"] or message)
    elif missing:
        print("Ask the user for:", ", ".join(missing))
    elif tool in MONEY_TOOLS and pick["probability"] < BAR:
        print("Confirm with the user first:", tool, args)
    else:
        print("Calling", tool, "with", args)

How it works

  1. Choice

    Pick the tool

    Choice docs

    List your tools as options. Each option's description says when to use the tool, in plain words. Sage2 reads that description. Write it the way you would brief a new colleague.

    Sage2 returns the tool in chosen and how sure it is in probability. Include a fallback such as ask_a_person for a message that fits no tool.

    Runs with the helper from Get the code.

    message = "Order 7781 arrived smashed. Please refund the €64 I paid."
    
    TOOLS = [
        {
            "id": "search_orders",
            "description": "Look up the status or details of a customer's order",
            "fields": {"order_number": "the order number"},
        },
        {
            "id": "issue_refund",
            "description": "Give a customer money back for an order",
            "fields": {
                "order_number": "the order number",
                "amount": {"type": "money", "definition": "the amount to refund"},
            },
        },
        {
            "id": "book_meeting",
            "description": "Put a meeting in the calendar",
            "fields": {
                "person": "who the meeting is with",
                "date": {"type": "date"},
                "time": {"type": "time"},
            },
        },
        {
            "id": "search_docs",
            "description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
            "fields": {"topic": "what the question is about"},
        },
        {
            "id": "ask_a_person",
            "description": "Hand over to a person when no other tool fits",
            "fields": {"reason": "why a person is needed"},
        },
    ]
    
    pick = sage("/decide", {
        "content": message,
        "question": {
            "id": "tool", "kind": "choice", "instructions": "Which tool should handle this request?",
            "options": [{"option": t["id"], "description": t["description"]} for t in TOOLS],
        },
    })["result"]
    tool = pick["chosen"]
    Example Sage2 output
    {
      "id": "tool",
      "kind": "choice",
      "result": {
        "chosen": "issue_refund",
        "probability": 0.9962,
        "probabilities": [
          {
            "option": "search_orders",
            "probability": 0.0025
          },
          {
            "option": "issue_refund",
            "probability": 0.9962
          },
          {
            "option": "book_meeting",
            "probability": 0
          },
          {
            "option": "search_docs",
            "probability": 0.0001
          },
          {
            "option": "ask_a_person",
            "probability": 0.0012
          }
        ]
      },
      "meta": {
        "model": "levanto-sage-2",
        "latency_ms": 60.8,
        "question_count": 1,
        "usage": {
          "billed_input_tokens": 148,
          "image_count": 0,
          "image_tokens": 0,
          "input_tokens": 148,
          "output_tokens": 0
        }
      }
    }
  2. Extract

    Fill its arguments

    Extract docs

    Each tool keeps the fields it needs, written as an extract request. Send the same message to /v1/extract with that tool's fields.

    • order_number is plain text.
    • amount has type money. It comes back as {amount, currency}.

    A field the message doesn't mention comes back null.

    Runs with the helper from Get the code.

    def fill_arguments(message, tool):
        fields = next(t["fields"] for t in TOOLS if t["id"] == tool)
        return sage("/v1/extract", {"content": message, "extract": fields})["values"]
    Example Sage2 output
    {
      "values": {
        "order_number": "7781",
        "amount": {
          "amount": 64,
          "currency": "EUR"
        }
      },
      "results": {
        "order_number": {
          "value": "7781",
          "text": "7781",
          "start": 6,
          "end": 10,
          "probability": 0.9968,
          "probabilities": [
            {
              "value": "7781",
              "text": "7781",
              "start": 6,
              "end": 10,
              "probability": 0.9968
            },
            {
              "value": null,
              "probability": 0.0032
            }
          ]
        },
        "amount": {
          "value": {
            "amount": 64,
            "currency": "EUR"
          },
          "text": "€64",
          "start": 46,
          "end": 49,
          "probability": 0.996,
          "probabilities": [
            {
              "value": {
                "amount": 64,
                "currency": "EUR"
              },
              "text": "€64",
              "start": 46,
              "end": 49,
              "probability": 0.996
            },
            {
              "value": null,
              "probability": 0.004
            }
          ]
        },
        "…": "one entry per field"
      },
      "meta": {
        "model": "levanto-sage-2",
        "usage": {
          "input_tokens": 184,
          "output_tokens": 221
        },
        "latency_ms": 63.5
      }
    }
  3. Your code

    Call the tool, or ask first

    Four outcomes:

    • A person takes over when chosen is null (too close to call) or the pick is ask_a_person.
    • Ask the user when an argument came back null.
    • Confirm with the user when the tool moves money and probability is below your bar. A message that claims to be "an admin instruction" can still push a refund to 72%. The bar stops it there.
    • Call the tool otherwise.

    Runs with the helper from Get the code.

    BAR = 0.98  # a tool that moves money runs on its own only at or above this
    MONEY_TOOLS = {"issue_refund"}
    
    if tool is None:
        print("Too close to call: hand over to a person")
    else:
        args = fill_arguments(message, tool)
        missing = [name for name, value in args.items() if value is None]
        if tool == "ask_a_person":
            print("Hand over to a person:", args["reason"] or message)
        elif missing:
            print("Ask the user for:", ", ".join(missing))
        elif tool in MONEY_TOOLS and pick["probability"] < BAR:
            print("Confirm with the user first:", tool, args)
        else:
            print("Calling", tool, "with", args)

Before you ship

  1. Write descriptions by contrast. If two tools get mixed up, say in each description what the other one is for.
  2. Check arguments in code. Validate the order_number and cap the amount before a refund runs.
  3. One tool per call. If a message asks for two things, Sage2 picks one. Ask again after the first tool runs.
  4. Use it next to function calling. Sage2 picks the tool and fills its arguments. Your LLM handles what's left, such as the reply.
  5. Fast and small. Each call answers in about 0.1 s and reads a few hundred tokens.

Sage / Levanto

Start free.

Sign in or create an account.

Continue with Google
or with email

We’ll email you a 6-digit code.