# Pick the right tool

Pick the right tool for each request and fill its arguments. When it's too close to call, hand over to a person.

Category: Keep agents in line. Live demo: https://levanto.ai/examples/tool-selection

An agent with many tools has to decide, on every turn, which one to call and with what. Pasting every tool's schema into a large model's prompt costs tokens each time, and a wrong call can refund the wrong order.

Here Sage2 makes that choice in **one Choice call**, then fills the tool's arguments with **one extraction**. Your agent gets fewer wasted tokens and fewer wrong tool calls, and a clear "not sure" when a person should step in.

## Setup

Base URL: `https://sage.levanto.ai`. Send `Authorization: Bearer $SAGE_API_KEY` on every call. Get a key, free, at https://platform.levanto.ai. Keep the key on your server.

A `null` answer is not an error: Sage2 isn't sure. Each step says what to do with it.

## Step by step

### 1. Pick the tool

List your tools as `options`. Each option's `description` says when to use the tool, in plain words. Sage2 reads that description. Write it the way you would brief a new colleague.

Sage2 returns the tool in `chosen` and how sure it is in `probability`. Include a fallback such as `ask_a_person` for a message that fits no tool.

```python
message = "Order 7781 arrived smashed. Please refund the €64 I paid."

TOOLS = [
    {
        "id": "search_orders",
        "description": "Look up the status or details of a customer's order",
        "fields": {"order_number": "the order number"},
    },
    {
        "id": "issue_refund",
        "description": "Give a customer money back for an order",
        "fields": {
            "order_number": "the order number",
            "amount": {"type": "money", "definition": "the amount to refund"},
        },
    },
    {
        "id": "book_meeting",
        "description": "Put a meeting in the calendar",
        "fields": {
            "person": "who the meeting is with",
            "date": {"type": "date"},
            "time": {"type": "time"},
        },
    },
    {
        "id": "search_docs",
        "description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
        "fields": {"topic": "what the question is about"},
    },
    {
        "id": "ask_a_person",
        "description": "Hand over to a person when no other tool fits",
        "fields": {"reason": "why a person is needed"},
    },
]

pick = sage("/decide", {
    "content": message,
    "question": {
        "id": "tool", "kind": "choice", "instructions": "Which tool should handle this request?",
        "options": [{"option": t["id"], "description": t["description"]} for t in TOOLS],
    },
})["result"]
tool = pick["chosen"]
```

```typescript
const message = "Order 7781 arrived smashed. Please refund the €64 I paid.";

const TOOLS = [
  {
    id: "search_orders",
    description: "Look up the status or details of a customer's order",
    fields: { order_number: "the order number" },
  },
  {
    id: "issue_refund",
    description: "Give a customer money back for an order",
    fields: {
      order_number: "the order number",
      amount: { type: "money", definition: "the amount to refund" },
    },
  },
  {
    id: "book_meeting",
    description: "Put a meeting in the calendar",
    fields: {
      person: "who the meeting is with",
      date: { type: "date" },
      time: { type: "time" },
    },
  },
  {
    id: "search_docs",
    description: "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
    fields: { topic: "what the question is about" },
  },
  {
    id: "ask_a_person",
    description: "Hand over to a person when no other tool fits",
    fields: { reason: "why a person is needed" },
  },
];

const pick = (await sage("/decide", {
  content: message,
  question: {
    id: "tool", kind: "choice", instructions: "Which tool should handle this request?",
    options: TOOLS.map(t => ({ option: t.id, description: t.description })),
  },
})).result;
const tool: string | null = pick.chosen;
```

Response, abridged (a real run):

```json
{
  "id": "tool",
  "kind": "choice",
  "result": {
    "chosen": "issue_refund",
    "probability": 0.9962,
    "probabilities": [
      {
        "option": "search_orders",
        "probability": 0.0025
      },
      {
        "option": "issue_refund",
        "probability": 0.9962
      },
      {
        "option": "book_meeting",
        "probability": 0
      },
      {
        "option": "search_docs",
        "probability": 0.0001
      },
      {
        "option": "ask_a_person",
        "probability": 0.0012
      }
    ]
  },
  "meta": {
    "model": "levanto-sage-2",
    "latency_ms": 60.8,
    "question_count": 1,
    "usage": {
      "billed_input_tokens": 148,
      "image_count": 0,
      "image_tokens": 0,
      "input_tokens": 148,
      "output_tokens": 0
    }
  }
}
```

### 2. Fill its arguments

Each tool keeps the fields it needs, written as an `extract` request. Send the same message to `/v1/extract` with that tool's fields.

- `order_number` is plain text.
- `amount` has type `money`. It comes back as `{amount, currency}`.

A field the message doesn't mention comes back `null`.

```python
def fill_arguments(message, tool):
    fields = next(t["fields"] for t in TOOLS if t["id"] == tool)
    return sage("/v1/extract", {"content": message, "extract": fields})["values"]
```

```typescript
async function fillArguments(message: string, tool: string) {
  const fields = TOOLS.find(t => t.id === tool)!.fields;
  return (await sage("/v1/extract", { content: message, extract: fields })).values;
}
```

Response, abridged (a real run):

```json
{
  "values": {
    "order_number": "7781",
    "amount": {
      "amount": 64,
      "currency": "EUR"
    }
  },
  "results": {
    "order_number": {
      "value": "7781",
      "text": "7781",
      "start": 6,
      "end": 10,
      "probability": 0.9968,
      "probabilities": [
        {
          "value": "7781",
          "text": "7781",
          "start": 6,
          "end": 10,
          "probability": 0.9968
        },
        {
          "value": null,
          "probability": 0.0032
        }
      ]
    },
    "amount": {
      "value": {
        "amount": 64,
        "currency": "EUR"
      },
      "text": "€64",
      "start": 46,
      "end": 49,
      "probability": 0.996,
      "probabilities": [
        {
          "value": {
            "amount": 64,
            "currency": "EUR"
          },
          "text": "€64",
          "start": 46,
          "end": 49,
          "probability": 0.996
        },
        {
          "value": null,
          "probability": 0.004
        }
      ]
    },
    "…": "one entry per field"
  },
  "meta": {
    "model": "levanto-sage-2",
    "usage": {
      "input_tokens": 184,
      "output_tokens": 221
    },
    "latency_ms": 63.5
  }
}
```

### 3. Call the tool, or ask first

Four outcomes:

- **A person takes over** when `chosen` is `null` (too close to call) or the pick is `ask_a_person`.
- **Ask the user** when an argument came back `null`.
- **Confirm with the user** when the tool moves money and `probability` is below your bar. A message that claims to be "an admin instruction" can still push a refund to 72%. The bar stops it there.
- **Call the tool** otherwise.

```python
BAR = 0.98  # a tool that moves money runs on its own only at or above this
MONEY_TOOLS = {"issue_refund"}

if tool is None:
    print("Too close to call: hand over to a person")
else:
    args = fill_arguments(message, tool)
    missing = [name for name, value in args.items() if value is None]
    if tool == "ask_a_person":
        print("Hand over to a person:", args["reason"] or message)
    elif missing:
        print("Ask the user for:", ", ".join(missing))
    elif tool in MONEY_TOOLS and pick["probability"] < BAR:
        print("Confirm with the user first:", tool, args)
    else:
        print("Calling", tool, "with", args)
```

```typescript
const BAR = 0.98; // a tool that moves money runs on its own only at or above this
const MONEY_TOOLS = ["issue_refund"];

if (tool === null) {
  console.log("Too close to call: hand over to a person");
} else {
  const args = await fillArguments(message, tool);
  const missing = Object.keys(args).filter(name => args[name] === null);
  if (tool === "ask_a_person") {
    console.log("Hand over to a person:", args.reason || message);
  } else if (missing.length) {
    console.log("Ask the user for:", missing.join(", "));
  } else if (MONEY_TOOLS.includes(tool) && pick.probability < BAR) {
    console.log("Confirm with the user first:", tool, args);
  } else {
    console.log("Calling", tool, "with", args);
  }
}
```

## Before you ship

- **Write descriptions by contrast.** If two tools get mixed up, say in each description what the other one is for.
- **Check arguments in code.** Validate the `order_number` and cap the `amount` before a refund runs.
- **One tool per call.** If a message asks for two things, Sage2 picks one. Ask again after the first tool runs.
- **Use it next to function calling.** Sage2 picks the tool and fills its arguments. Your LLM handles what's left, such as the reply.
- **Fast and small.** Each call answers in about 0.1 s and reads a few hundred tokens.

## Full recipe (Python, `pip install requests`)

```python
import os
import requests

def sage(path, body):
    r = requests.post(
        "https://sage.levanto.ai" + path,
        headers={"Authorization": f"Bearer {os.environ['SAGE_API_KEY']}"},
        json=body,
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

message = "Order 7781 arrived smashed. Please refund the €64 I paid."

TOOLS = [
    {
        "id": "search_orders",
        "description": "Look up the status or details of a customer's order",
        "fields": {"order_number": "the order number"},
    },
    {
        "id": "issue_refund",
        "description": "Give a customer money back for an order",
        "fields": {
            "order_number": "the order number",
            "amount": {"type": "money", "definition": "the amount to refund"},
        },
    },
    {
        "id": "book_meeting",
        "description": "Put a meeting in the calendar",
        "fields": {
            "person": "who the meeting is with",
            "date": {"type": "date"},
            "time": {"type": "time"},
        },
    },
    {
        "id": "search_docs",
        "description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
        "fields": {"topic": "what the question is about"},
    },
    {
        "id": "ask_a_person",
        "description": "Hand over to a person when no other tool fits",
        "fields": {"reason": "why a person is needed"},
    },
]

pick = sage("/decide", {
    "content": message,
    "question": {
        "id": "tool", "kind": "choice", "instructions": "Which tool should handle this request?",
        "options": [{"option": t["id"], "description": t["description"]} for t in TOOLS],
    },
})["result"]
tool = pick["chosen"]

def fill_arguments(message, tool):
    fields = next(t["fields"] for t in TOOLS if t["id"] == tool)
    return sage("/v1/extract", {"content": message, "extract": fields})["values"]

BAR = 0.98  # a tool that moves money runs on its own only at or above this
MONEY_TOOLS = {"issue_refund"}

if tool is None:
    print("Too close to call: hand over to a person")
else:
    args = fill_arguments(message, tool)
    missing = [name for name, value in args.items() if value is None]
    if tool == "ask_a_person":
        print("Hand over to a person:", args["reason"] or message)
    elif missing:
        print("Ask the user for:", ", ".join(missing))
    elif tool in MONEY_TOOLS and pick["probability"] < BAR:
        print("Confirm with the user first:", tool, args)
    else:
        print("Calling", tool, "with", args)
```

## Full recipe (TypeScript, Node 22+, nothing to install: save as `recipe.mts`, run `npx tsx recipe.mts`)

```typescript
// Node 22+. Save as recipe.mts and run npx tsx recipe.mts
async function sage(path: string, body: unknown): Promise<any> {
  const r = await fetch("https://sage.levanto.ai" + path, {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.SAGE_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify(body),
    signal: AbortSignal.timeout(30_000),
  });
  if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
  return r.json();
}

const message = "Order 7781 arrived smashed. Please refund the €64 I paid.";

const TOOLS = [
  {
    id: "search_orders",
    description: "Look up the status or details of a customer's order",
    fields: { order_number: "the order number" },
  },
  {
    id: "issue_refund",
    description: "Give a customer money back for an order",
    fields: {
      order_number: "the order number",
      amount: { type: "money", definition: "the amount to refund" },
    },
  },
  {
    id: "book_meeting",
    description: "Put a meeting in the calendar",
    fields: {
      person: "who the meeting is with",
      date: { type: "date" },
      time: { type: "time" },
    },
  },
  {
    id: "search_docs",
    description: "Answer a how-to question about our product from the help center, such as passwords, settings or billing",
    fields: { topic: "what the question is about" },
  },
  {
    id: "ask_a_person",
    description: "Hand over to a person when no other tool fits",
    fields: { reason: "why a person is needed" },
  },
];

const pick = (await sage("/decide", {
  content: message,
  question: {
    id: "tool", kind: "choice", instructions: "Which tool should handle this request?",
    options: TOOLS.map(t => ({ option: t.id, description: t.description })),
  },
})).result;
const tool: string | null = pick.chosen;

async function fillArguments(message: string, tool: string) {
  const fields = TOOLS.find(t => t.id === tool)!.fields;
  return (await sage("/v1/extract", { content: message, extract: fields })).values;
}

const BAR = 0.98; // a tool that moves money runs on its own only at or above this
const MONEY_TOOLS = ["issue_refund"];

if (tool === null) {
  console.log("Too close to call: hand over to a person");
} else {
  const args = await fillArguments(message, tool);
  const missing = Object.keys(args).filter(name => args[name] === null);
  if (tool === "ask_a_person") {
    console.log("Hand over to a person:", args.reason || message);
  } else if (missing.length) {
    console.log("Ask the user for:", missing.join(", "));
  } else if (MONEY_TOOLS.includes(tool) && pick.probability < BAR) {
    console.log("Confirm with the user first:", tool, args);
  } else {
    console.log("Calling", tool, "with", args);
  }
}
```

## Requests

### 1. POST /decide

```bash
curl -s https://sage.levanto.ai/decide \
  -H "Authorization: Bearer $SAGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<'JSON'
{
  "content": "Order 7781 arrived smashed. Please refund the €64 I paid.",
  "question": {
    "id": "tool",
    "kind": "choice",
    "instructions": "Which tool should handle this request?",
    "options": [
      {
        "option": "search_orders",
        "description": "Look up the status or details of a customer's order"
      },
      {
        "option": "issue_refund",
        "description": "Give a customer money back for an order"
      },
      {
        "option": "book_meeting",
        "description": "Put a meeting in the calendar"
      },
      {
        "option": "search_docs",
        "description": "Answer a how-to question about our product from the help center, such as passwords, settings or billing"
      },
      {
        "option": "ask_a_person",
        "description": "Hand over to a person when no other tool fits"
      }
    ]
  }
}
JSON
```

Response, abridged (a real run):

```json
{
  "id": "tool",
  "kind": "choice",
  "result": {
    "chosen": "issue_refund",
    "probability": 0.9962,
    "probabilities": [
      {
        "option": "search_orders",
        "probability": 0.0025
      },
      {
        "option": "issue_refund",
        "probability": 0.9962
      },
      {
        "option": "book_meeting",
        "probability": 0
      },
      {
        "option": "search_docs",
        "probability": 0.0001
      },
      {
        "option": "ask_a_person",
        "probability": 0.0012
      }
    ]
  },
  "meta": {
    "model": "levanto-sage-2",
    "latency_ms": 60.8,
    "question_count": 1,
    "usage": {
      "billed_input_tokens": 148,
      "image_count": 0,
      "image_tokens": 0,
      "input_tokens": 148,
      "output_tokens": 0
    }
  }
}
```

### 2. POST /v1/extract

```bash
curl -s https://sage.levanto.ai/v1/extract \
  -H "Authorization: Bearer $SAGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<'JSON'
{
  "content": "Order 7781 arrived smashed. Please refund the €64 I paid.",
  "extract": {
    "order_number": "the order number",
    "amount": {
      "type": "money",
      "definition": "the amount to refund"
    }
  }
}
JSON
```

Response, abridged (a real run):

```json
{
  "values": {
    "order_number": "7781",
    "amount": {
      "amount": 64,
      "currency": "EUR"
    }
  },
  "results": {
    "order_number": {
      "value": "7781",
      "text": "7781",
      "start": 6,
      "end": 10,
      "probability": 0.9968,
      "probabilities": [
        {
          "value": "7781",
          "text": "7781",
          "start": 6,
          "end": 10,
          "probability": 0.9968
        },
        {
          "value": null,
          "probability": 0.0032
        }
      ]
    },
    "amount": {
      "value": {
        "amount": 64,
        "currency": "EUR"
      },
      "text": "€64",
      "start": 46,
      "end": 49,
      "probability": 0.996,
      "probabilities": [
        {
          "value": {
            "amount": 64,
            "currency": "EUR"
          },
          "text": "€64",
          "start": 46,
          "end": 49,
          "probability": 0.996
        },
        {
          "value": null,
          "probability": 0.004
        }
      ]
    },
    "…": "one entry per field"
  },
  "meta": {
    "model": "levanto-sage-2",
    "usage": {
      "input_tokens": 184,
      "output_tokens": 221
    },
    "latency_ms": 63.5
  }
}
```

## Docs

- Choice: https://docs.levanto.ai/decision-model/choice
- Extract: https://docs.levanto.ai/extract
- Limits and errors: https://docs.levanto.ai/decision-model/limits
- OpenAPI: https://sage.levanto.ai/openapi.json