# Optimize context

Decide which documents an LLM needs for each question, so it reads less and answers from the right source.

Category: Keep agents in line. Live demo: https://levanto.ai/examples/context-selection

Stuffing every document into the prompt is the easy way to give an LLM context. It's also how a question about a broken grinder ends up reading the API reference and the notes from a 2019 offsite.

Ask Sage2 **which documents the question needs** first, with one Tags call, and send only those. The LLM reads fewer tokens, and fewer irrelevant pages pull its answer off course.

## Setup

Base URL: `https://sage.levanto.ai`. Send `Authorization: Bearer $SAGE_API_KEY` on every call. Get a key, free, at https://platform.levanto.ai. Keep the key on your server.

A `null` answer is not an error: Sage2 isn't sure. Each step says what to do with it.

## Step by step

### 1. Describe your documents

Give each document an `id`, a title and a one-line summary. Sage2 decides from the summary alone, never the full document.

```python
question = "My grinder broke after 8 months. Can I get my money back or a free repair?"

DOCS = [
    {
        "id": "refund_policy",
        "title": "Refund policy",
        "summary": "How refunds work, time limits, and exceptions",
    },
    {
        "id": "shipping_rates",
        "title": "Shipping rates",
        "summary": "Delivery prices and times by country",
    },
    {
        "id": "api_reference",
        "title": "API reference",
        "summary": "Endpoints and authentication for developers",
    },
    {
        "id": "warranty",
        "title": "Warranty terms",
        "summary": "What is covered when a product breaks, and for how long",
    },
    {
        "id": "brand_voice",
        "title": "Brand voice guide",
        "summary": "How we write marketing copy",
    },
    {
        "id": "offsite_2019",
        "title": "2019 offsite notes",
        "summary": "Team retreat agenda and who brought the guitar",
    },
]
```

```typescript
const question = "My grinder broke after 8 months. Can I get my money back or a free repair?";

const DOCS = [
  {
    id: "refund_policy",
    title: "Refund policy",
    summary: "How refunds work, time limits, and exceptions",
  },
  {
    id: "shipping_rates",
    title: "Shipping rates",
    summary: "Delivery prices and times by country",
  },
  {
    id: "api_reference",
    title: "API reference",
    summary: "Endpoints and authentication for developers",
  },
  {
    id: "warranty",
    title: "Warranty terms",
    summary: "What is covered when a product breaks, and for how long",
  },
  {
    id: "brand_voice",
    title: "Brand voice guide",
    summary: "How we write marketing copy",
  },
  {
    id: "offsite_2019",
    title: "2019 offsite notes",
    summary: "Team retreat agenda and who brought the guitar",
  },
];
```

### 2. Ask which ones the question needs

Send the user's question as `content` and one tag per document. Each tag's `name` is what Sage2 reads. Its `id` is the key you get back.

For every tag, Sage2 returns `applies` (`true`, `false`, or `null` when it's too close to call) and a `probability`.

```python
tags = sage("/decide", {
    "content": question,
    "question": {
        "id": "context", "kind": "tags",
        "instructions": "An assistant has to answer this question. Tag each document it needs to read. A document it doesn't need does not apply.",
        "tags": [{"id": doc["id"], "name": f"{doc['title']}: {doc['summary']}"} for doc in DOCS],
    },
})["result"]["tags"]
```

```typescript
const tags = (await sage("/decide", {
  content: question,
  question: {
    id: "context", kind: "tags",
    instructions: "An assistant has to answer this question. Tag each document it needs to read. A document it doesn't need does not apply.",
    tags: DOCS.map(doc => ({ id: doc.id, name: `${doc.title}: ${doc.summary}` })),
  },
})).result.tags;
```

Response, abridged (a real run):

```json
{
  "id": "context",
  "kind": "tags",
  "result": {
    "tags": [
      {
        "id": "refund_policy",
        "probability": 0.9984,
        "applies": true
      },
      {
        "id": "shipping_rates",
        "probability": 0,
        "applies": false
      },
      {
        "id": "api_reference",
        "probability": 0,
        "applies": false
      },
      {
        "id": "warranty",
        "probability": 1,
        "applies": true
      },
      {
        "id": "brand_voice",
        "probability": 0,
        "applies": false
      },
      {
        "id": "offsite_2019",
        "probability": 0,
        "applies": false
      }
    ]
  },
  "meta": {
    "model": "levanto-sage-2",
    "latency_ms": 160.6,
    "question_count": 1,
    "usage": {
      "billed_input_tokens": 1078,
      "image_count": 0,
      "image_tokens": 0,
      "input_tokens": 1078,
      "output_tokens": 0
    }
  }
}
```

### 3. Build the prompt

Send your LLM the documents that apply. Keep the unsure ones too (`applies` is `null`). One extra page costs less than a wrong answer.

```python
# Keep what applies, and what Sage isn't sure about (applies is None).
needed = [t["id"] for t in tags if t["applies"] is not False]
context = [doc for doc in DOCS if doc["id"] in needed]
print("Give the LLM:", [doc["title"] for doc in context])
```

```typescript
// Keep what applies, and what Sage isn't sure about (applies is null).
const needed: string[] = tags.filter((t: any) => t.applies !== false).map((t: any) => t.id);
const context = DOCS.filter(doc => needed.includes(doc.id));
console.log("Give the LLM:", context.map(doc => doc.title));
```

## Before you ship

- **Summaries do the work.** If Sage2 picks a document it should not, sharpen the summary of that document.
- **Check permissions in code.** Sage2 picks what's relevant, not what this user may see. List only the documents they may read.
- **Use Tags here.** A question can need several documents. Each document gets its own verdict.
- **Up to 120 tags** per question. For a larger library, narrow it down first, for example by product or section.
- **Cost grows with the number of documents, not their length.** Sage2 reads only the summaries. Six documents take about 0.1 s and 1,100 tokens.

## Full recipe (Python, `pip install requests`)

```python
import os
import requests

def sage(path, body):
    r = requests.post(
        "https://sage.levanto.ai" + path,
        headers={"Authorization": f"Bearer {os.environ['SAGE_API_KEY']}"},
        json=body,
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

question = "My grinder broke after 8 months. Can I get my money back or a free repair?"

DOCS = [
    {
        "id": "refund_policy",
        "title": "Refund policy",
        "summary": "How refunds work, time limits, and exceptions",
    },
    {
        "id": "shipping_rates",
        "title": "Shipping rates",
        "summary": "Delivery prices and times by country",
    },
    {
        "id": "api_reference",
        "title": "API reference",
        "summary": "Endpoints and authentication for developers",
    },
    {
        "id": "warranty",
        "title": "Warranty terms",
        "summary": "What is covered when a product breaks, and for how long",
    },
    {
        "id": "brand_voice",
        "title": "Brand voice guide",
        "summary": "How we write marketing copy",
    },
    {
        "id": "offsite_2019",
        "title": "2019 offsite notes",
        "summary": "Team retreat agenda and who brought the guitar",
    },
]

tags = sage("/decide", {
    "content": question,
    "question": {
        "id": "context", "kind": "tags",
        "instructions": "An assistant has to answer this question. Tag each document it needs to read. A document it doesn't need does not apply.",
        "tags": [{"id": doc["id"], "name": f"{doc['title']}: {doc['summary']}"} for doc in DOCS],
    },
})["result"]["tags"]

# Keep what applies, and what Sage isn't sure about (applies is None).
needed = [t["id"] for t in tags if t["applies"] is not False]
context = [doc for doc in DOCS if doc["id"] in needed]
print("Give the LLM:", [doc["title"] for doc in context])
```

## Full recipe (TypeScript, Node 22+, nothing to install: save as `recipe.mts`, run `npx tsx recipe.mts`)

```typescript
// Node 22+. Save as recipe.mts and run npx tsx recipe.mts
async function sage(path: string, body: unknown): Promise<any> {
  const r = await fetch("https://sage.levanto.ai" + path, {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.SAGE_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify(body),
    signal: AbortSignal.timeout(30_000),
  });
  if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
  return r.json();
}

const question = "My grinder broke after 8 months. Can I get my money back or a free repair?";

const DOCS = [
  {
    id: "refund_policy",
    title: "Refund policy",
    summary: "How refunds work, time limits, and exceptions",
  },
  {
    id: "shipping_rates",
    title: "Shipping rates",
    summary: "Delivery prices and times by country",
  },
  {
    id: "api_reference",
    title: "API reference",
    summary: "Endpoints and authentication for developers",
  },
  {
    id: "warranty",
    title: "Warranty terms",
    summary: "What is covered when a product breaks, and for how long",
  },
  {
    id: "brand_voice",
    title: "Brand voice guide",
    summary: "How we write marketing copy",
  },
  {
    id: "offsite_2019",
    title: "2019 offsite notes",
    summary: "Team retreat agenda and who brought the guitar",
  },
];

const tags = (await sage("/decide", {
  content: question,
  question: {
    id: "context", kind: "tags",
    instructions: "An assistant has to answer this question. Tag each document it needs to read. A document it doesn't need does not apply.",
    tags: DOCS.map(doc => ({ id: doc.id, name: `${doc.title}: ${doc.summary}` })),
  },
})).result.tags;

// Keep what applies, and what Sage isn't sure about (applies is null).
const needed: string[] = tags.filter((t: any) => t.applies !== false).map((t: any) => t.id);
const context = DOCS.filter(doc => needed.includes(doc.id));
console.log("Give the LLM:", context.map(doc => doc.title));
```

## Requests

### 1. POST /decide

```bash
curl -s https://sage.levanto.ai/decide \
  -H "Authorization: Bearer $SAGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<'JSON'
{
  "content": "My grinder broke after 8 months. Can I get my money back or a free repair?",
  "question": {
    "id": "context",
    "kind": "tags",
    "instructions": "An assistant has to answer this question. Tag each document it needs to read. A document it doesn't need does not apply.",
    "tags": [
      {
        "id": "refund_policy",
        "name": "Refund policy: How refunds work, time limits, and exceptions"
      },
      {
        "id": "shipping_rates",
        "name": "Shipping rates: Delivery prices and times by country"
      },
      {
        "id": "api_reference",
        "name": "API reference: Endpoints and authentication for developers"
      },
      {
        "id": "warranty",
        "name": "Warranty terms: What is covered when a product breaks, and for how long"
      },
      {
        "id": "brand_voice",
        "name": "Brand voice guide: How we write marketing copy"
      },
      {
        "id": "offsite_2019",
        "name": "2019 offsite notes: Team retreat agenda and who brought the guitar"
      }
    ]
  }
}
JSON
```

Response, abridged (a real run):

```json
{
  "id": "context",
  "kind": "tags",
  "result": {
    "tags": [
      {
        "id": "refund_policy",
        "probability": 0.9984,
        "applies": true
      },
      {
        "id": "shipping_rates",
        "probability": 0,
        "applies": false
      },
      {
        "id": "api_reference",
        "probability": 0,
        "applies": false
      },
      {
        "id": "warranty",
        "probability": 1,
        "applies": true
      },
      {
        "id": "brand_voice",
        "probability": 0,
        "applies": false
      },
      {
        "id": "offsite_2019",
        "probability": 0,
        "applies": false
      }
    ]
  },
  "meta": {
    "model": "levanto-sage-2",
    "latency_ms": 160.6,
    "question_count": 1,
    "usage": {
      "billed_input_tokens": 1078,
      "image_count": 0,
      "image_tokens": 0,
      "input_tokens": 1078,
      "output_tokens": 0
    }
  }
}
```

## Docs

- Tags: https://docs.levanto.ai/decision-model/tags
- Limits and errors: https://docs.levanto.ai/decision-model/limits
- OpenAPI: https://sage.levanto.ai/openapi.json