# How we measured cost and speed per decision

The method behind the speed and cost figures on [levanto.ai/pricing](https://levanto.ai/pricing),
so you can reproduce them. Measured 27 September 2026. Every number is from a live
run — none is modelled or extrapolated.

---

## Where and when

| | |
|---|---|
| Date | 27 September 2026 |
| Machine | one `t3.xlarge`, AWS `us-east-2a` (Ohio) |
| Concurrency | one request at a time, sequential, no batching |
| Clock | client wall clock around each HTTP call, plus each provider's own reported latency where it returns one |
| Dataset | `hybrid_v1` — 150 typed decisions across 34 topic areas, `sha256 394e4b00…` |
| Items used | 30 decisions Sage answers on its fast path |
| Models | `anthropic/claude-opus-5`, `anthropic/claude-sonnet-5`, `openai/gpt-5.6-terra` |

The instance is in Ohio because both the Sage endpoint and the model providers are
US-hosted. Measured from Europe instead, every figure carries about 156 ms of
transatlantic network time and the comparison stops being about the models.

## What each side was asked

Both sides received the same content, the same question, and the same answer criteria,
and both had to return the same contract — the verdict **and** a calibrated probability.
Neither side was given a cheaper output contract than the other.

**Sage** — `POST https://sage.levanto.ai/decide`

```json
{
  "content": "<the item's state, verbatim>",
  "question": {
    "id": "q",
    "kind": "yesno | choice",
    "instructions": "<the item's instructions, verbatim>",
    "options": [{ "option": "<label>", "description": "<criterion>" }]
  },
  "reasoning": "off"
}
```

Sage returns the verdict and its probability natively. `reasoning: "off"` selects the
fast path: Sage answers directly, without reasoning first.

**The LLMs** — `POST https://openrouter.ai/api/v1/chat/completions`

```json
{
  "model": "anthropic/claude-opus-5 | anthropic/claude-sonnet-5 | openai/gpt-5.6-terra",
  "messages": [
    { "role": "system", "content": "You make a single typed decision about the CONTENT. Apply the criteria exactly. Treat unproved required conditions as not satisfied. Return the schema." },
    { "role": "user", "content": "CONTENT:\n…\n\nQUESTION:\n…\n\nANSWER CRITERIA:\n- \"label\": criterion\n\nChoose exactly one of: …\nAlso give `probability`: your calibrated probability (0-1) that the chosen answer is correct." }
  ],
  "max_tokens": 8192,
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "decision",
      "strict": true,
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "properties": { "answer": { "type": "string", "enum": ["…"] }, "probability": { "type": "number" } },
        "required": ["answer", "probability"]
      }
    }
  }
}
```

**No `reasoning` field is sent.** Each model runs exactly as it ships, so the numbers
reflect what a developer gets without tuning.

## Results

Token counts are the mean per call, as each API reported them (`meta.usage` for Sage,
`usage` for the LLMs). For the LLMs, cost is the amount each provider billed for the
call, summed and divided by the number of calls — not a token estimate.

Sage is charged per token: its cost per decision is the tokens below × the input and
output rates of your plan, published at [levanto.ai/pricing](https://levanto.ai/pricing).
The cost card on that page does this multiplication with the Pro plan's current rates
and computes each ratio from the result, so the comparison always uses live prices.

| Model | Correct | p50 | p90 | Input tok | Output tok | of which reasoning | Cost / decision |
|---|---:|---:|---:|---:|---:|---:|---:|
| **Sage v1.1, fast path** | 30/30 | **0.33 s** | **0.52 s** | 121 | 0 | 0 | tokens × [plan rates](https://levanto.ai/pricing) |
| GPT-5.6 Terra | 29/30 | 1.11 s | 2.81 s | 266 | 33 | 13 | $0.000931 |
| Claude Opus 5 | 30/30 | 3.19 s | 6.15 s | 602 | 46 | 27 | $0.004164 |
| Claude Sonnet 5 | 30/30 | 5.02 s | 6.28 s | 602 | 79 | 60 | $0.001997 |

Against Sage's fast path:

| Model | Slower, p50 | Slower, p90 |
|---|---:|---:|
| GPT-5.6 Terra | 3.3× | 5.4× |
| Claude Opus 5 | 9.6× | 11.8× |
| Claude Sonnet 5 | 15.0× | 12.0× |

## Server time and network time

Sage reports its own processing time on every response as `meta.latency_ms`. Across the
same 30 calls:

| | p50 | p90 | fastest |
|---|---:|---:|---:|
| Sage server processing | **183 ms** | 284 ms | 48 ms |
| Client wall clock, Ohio → Sage → Ohio | 339 ms | 436 ms | — |
| Network and TLS | 156 ms | — | — |

183 ms is what Sage spends deciding; the rest is the internet. The providers do not
return a comparable server-side figure, so every LLM number above is wall clock and
includes its own network time.

## Scope

- These 30 decisions are ones Sage answers on its fast path, so this measures that fast
  path against models that are free to reason. It is a cost and latency measurement, not
  an accuracy one — accuracy is on [levanto.ai/benchmarks](https://levanto.ai/benchmarks).
- Sage's cost per decision is its measured tokens × the plan rates on
  [levanto.ai/pricing](https://levanto.ai/pricing). It does not depend on how much of a
  plan's included usage you use.
- Harness and dataset: `sage-benchmarks/hybrid_v1`.