Pick the right tool
Pick the tool and fill its arguments, or hand a close call to a person.
An agent with many tools has to decide, on every turn, which one to call and with what. Pasting every tool's schema into a large model's prompt costs tokens each time, and a wrong call can refund the wrong order.
Here Sage2 makes that choice in one Choice call, then fills the tool's arguments with one extraction. Your agent gets fewer wasted tokens and fewer wrong tool calls, and a clear "not sure" when a person should step in.
- Choice
- Extract
- Tools the agent can call
- search_orders: Look up the status or details of a customer's order
- issue_refund: Give a customer money back for an order
- book_meeting: Put a meeting in the calendar
- search_docs: Answer a how-to question about our product from the help center, such as passwords, settings or billing
- ask_a_person: Hand over to a person when no other tool fits
- Pick the tool for the requestChoice
- search_orders0.25%
- issue_refund99.62%
- book_meetingunder 0.01%
- search_docsunder 0.01%
- ask_a_person0.12%
- Fill the arguments of that toolExtract
- order_number
- 7781
- amount
- €64.00
- Your codeCalls issue_refund
The agent runs this tool with order_number: 7781, amount: €64.00.
The Choice question took 61 ms. Extract ran after it, for the tool that won: 64 ms.
Get the code
Set SAGE_API_KEY (get a free key), then run it. Python: pip install requests. TypeScript: Node 22+. Save as recipe.mts and run npx tsx recipe.mts.
A null answer is not an error. Sage2 isn't sure, and each step below says what to do with it. Limits, errors and retries: Limits and errors.
import os
import requests
def sage(path, body):
r = requests.post(
"https://sage.levanto.ai" + path,
headers={"Authorization": f"Bearer {os.environ['SAGE_API_KEY']}"},
json=body,
timeout=30,
)
r.raise_for_status()
return r.json()
message = "Order 7781 arrived smashed. Please refund the €64 I paid."
TOOLS = [
{
"id": "search_orders",
"description": "Look up the status or details of a customer's order",
"fields": {"order_number": "the order number"},
},
{
"id": "issue_refund",
"description": "Give a customer money back for an order",
"fields": {
"order_number": "the order number",
"amount": {"type": "money", "definition": "the amount to refund"},
},
},
{
"id": "book_meeting",
"description": "Put a meeting in the calendar",
"fields": {
"person": "who the meeting is with",
"date": {"type": "date"},
"time": {"type": "time"},
},
},
{
"id": "search_docs",
"description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
"fields": {"topic": "what the question is about"},
},
{
"id": "ask_a_person",
"description": "Hand over to a person when no other tool fits",
"fields": {"reason": "why a person is needed"},
},
]
pick = sage("/decide", {
"content": message,
"question": {
"id": "tool", "kind": "choice", "instructions": "Which tool should handle this request?",
"options": [{"option": t["id"], "description": t["description"]} for t in TOOLS],
},
})["result"]
tool = pick["chosen"]
def fill_arguments(message, tool):
fields = next(t["fields"] for t in TOOLS if t["id"] == tool)
return sage("/v1/extract", {"content": message, "extract": fields})["values"]
BAR = 0.98 # a tool that moves money runs on its own only at or above this
MONEY_TOOLS = {"issue_refund"}
if tool is None:
print("Too close to call: hand over to a person")
else:
args = fill_arguments(message, tool)
missing = [name for name, value in args.items() if value is None]
if tool == "ask_a_person":
print("Hand over to a person:", args["reason"] or message)
elif missing:
print("Ask the user for:", ", ".join(missing))
elif tool in MONEY_TOOLS and pick["probability"] < BAR:
print("Confirm with the user first:", tool, args)
else:
print("Calling", tool, "with", args)
How it works
List your tools as
options. Each option'sdescriptionsays when to use the tool, in plain words. Sage2 reads that description. Write it the way you would brief a new colleague.Sage2 returns the tool in
chosenand how sure it is inprobability. Include a fallback such asask_a_personfor a message that fits no tool.Runs with the helper from Get the code.
message = "Order 7781 arrived smashed. Please refund the €64 I paid." TOOLS = [ { "id": "search_orders", "description": "Look up the status or details of a customer's order", "fields": {"order_number": "the order number"}, }, { "id": "issue_refund", "description": "Give a customer money back for an order", "fields": { "order_number": "the order number", "amount": {"type": "money", "definition": "the amount to refund"}, }, }, { "id": "book_meeting", "description": "Put a meeting in the calendar", "fields": { "person": "who the meeting is with", "date": {"type": "date"}, "time": {"type": "time"}, }, }, { "id": "search_docs", "description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing", "fields": {"topic": "what the question is about"}, }, { "id": "ask_a_person", "description": "Hand over to a person when no other tool fits", "fields": {"reason": "why a person is needed"}, }, ] pick = sage("/decide", { "content": message, "question": { "id": "tool", "kind": "choice", "instructions": "Which tool should handle this request?", "options": [{"option": t["id"], "description": t["description"]} for t in TOOLS], }, })["result"] tool = pick["chosen"]Example Sage2 output
{ "id": "tool", "kind": "choice", "result": { "chosen": "issue_refund", "probability": 0.9962, "probabilities": [ { "option": "search_orders", "probability": 0.0025 }, { "option": "issue_refund", "probability": 0.9962 }, { "option": "book_meeting", "probability": 0 }, { "option": "search_docs", "probability": 0.0001 }, { "option": "ask_a_person", "probability": 0.0012 } ] }, "meta": { "model": "levanto-sage-2", "latency_ms": 60.8, "question_count": 1, "usage": { "billed_input_tokens": 148, "image_count": 0, "image_tokens": 0, "input_tokens": 148, "output_tokens": 0 } } }Each tool keeps the fields it needs, written as an
extractrequest. Send the same message to/v1/extractwith that tool's fields.order_numberis plain text.amounthas typemoney. It comes back as{amount, currency}.
A field the message doesn't mention comes back
null.Runs with the helper from Get the code.
def fill_arguments(message, tool): fields = next(t["fields"] for t in TOOLS if t["id"] == tool) return sage("/v1/extract", {"content": message, "extract": fields})["values"]Example Sage2 output
{ "values": { "order_number": "7781", "amount": { "amount": 64, "currency": "EUR" } }, "results": { "order_number": { "value": "7781", "text": "7781", "start": 6, "end": 10, "probability": 0.9968, "probabilities": [ { "value": "7781", "text": "7781", "start": 6, "end": 10, "probability": 0.9968 }, { "value": null, "probability": 0.0032 } ] }, "amount": { "value": { "amount": 64, "currency": "EUR" }, "text": "€64", "start": 46, "end": 49, "probability": 0.996, "probabilities": [ { "value": { "amount": 64, "currency": "EUR" }, "text": "€64", "start": 46, "end": 49, "probability": 0.996 }, { "value": null, "probability": 0.004 } ] }, "…": "one entry per field" }, "meta": { "model": "levanto-sage-2", "usage": { "input_tokens": 184, "output_tokens": 221 }, "latency_ms": 63.5 } }- Your code
Call the tool, or ask first
Four outcomes:
- A person takes over when
chosenisnull(too close to call) or the pick isask_a_person. - Ask the user when an argument came back
null. - Confirm with the user when the tool moves money and
probabilityis below your bar. A message that claims to be "an admin instruction" can still push a refund to 72%. The bar stops it there. - Call the tool otherwise.
Runs with the helper from Get the code.
BAR = 0.98 # a tool that moves money runs on its own only at or above this MONEY_TOOLS = {"issue_refund"} if tool is None: print("Too close to call: hand over to a person") else: args = fill_arguments(message, tool) missing = [name for name, value in args.items() if value is None] if tool == "ask_a_person": print("Hand over to a person:", args["reason"] or message) elif missing: print("Ask the user for:", ", ".join(missing)) elif tool in MONEY_TOOLS and pick["probability"] < BAR: print("Confirm with the user first:", tool, args) else: print("Calling", tool, "with", args) - A person takes over when
Before you ship
- Write descriptions by contrast. If two tools get mixed up, say in each description what the other one is for.
- Check arguments in code. Validate the
order_numberand cap theamountbefore a refund runs. - One tool per call. If a message asks for two things, Sage2 picks one. Ask again after the first tool runs.
- Use it next to function calling. Sage2 picks the tool and fills its arguments. Your LLM handles what's left, such as the reply.
- Fast and small. Each call answers in about 0.1 s and reads a few hundred tokens.