Start building ↗

Sage / Benchmarks

Sage,
measured.

Sage is the decision model Levanto launched in July 2026. Jev is presented as a new class of model with an architecture of its own. Sage is, openly, an LLM backbone with a calibrated classifier trained on top. Since v1.1 it tells System 1 questions from System 2 questions, and answers both.

Performance · vs Jev

JevBench v1.2, public tasks

213 public decisions from the JevBench leaderboard set

Jev 1.13.087.3%186 / 213250 ms300 ms
Sage v1.1 (auto)89.2%190 / 213230 ms3.52 s

multistep_decisions v1.0

100 decisions that each need several dependent steps

Jev 1.13.071.0%71 / 100240 ms290 ms
Sage v1.1 (auto)92.0%92 / 100910 ms1.51 s

Measured 2026-09-22, one t3 instance in AWS us-east-2 (Ohio), one request at a time. Same harness, same records, both models. Repeat runs: Jev 186 / 186 and 69 / 71 / 71; Sage auto 188 / 190 and 91 / 92 / 93 / 92. multistep_decisions: github.com/levantolabs/multistep_decisions ↗. JevBench: leaderboard ↗.

msd-arith-14

Progressive tax: 0% on the first 10,000; 20% from 10,000 to 40,000; 40% above 40,000. Income 52,000. Effective rate = total tax / income.

Is the effective rate below 21%?

Jev: noCorrect: yes

msd-state-05

Ticket workflow. Allowed transitions: new->triaged, triaged->in_progress, in_progress->review, review->in_progress, review->done, and any state except done -> cancelled. Invalid transitions are ignored. Events: new; triaged; in_progress; review; in_progress; cancelled; review; done.

Final state?

Jev: doneCorrect: cancelled

Features

How is Sage different from Jev?

Question kindsYes/NoChoiceScaleSortTagsNoul (yes/no)ChoiceScore
ReasoningSystem 1 + System 2System 1 only
ImagesYes (beta)

System 1 + System 2

Fast when fast is enough. Thinking when it isn’t.

  1. One pass. Encode the state, append the question, read the answer off the model. 50–150 ms. Enough when the answer is in the text.
  2. But some answers aren’t in the text. They have to be worked out step by step: add up nine stock movements, then compare the total with the reorder point. One pass has no scratchpad for the intermediate results, so it guesses.
  3. The gate. The same vector tells Sage whether the model wants to reason. Only then, a second budgeted pass thinks and answers. reasoning: auto | off | on.
ONE PREFILLstate + question→ vector hSYSTEM 1 · TYPED HEADSread the answer off h. calibrated.≈ 50–150 msGATEreads hdoes the model want to start reasoningrather than answer?a first-token margin on the same vector.no extra forward pass.no → fast answeryesSYSTEM 2 · REASONING PASSsame prompt, thinking on.think, then one constrained answertoken, read out and calibrated.≈ 1 s · budgeted · billed only when it ranyesp = 0.99same answer shape,either way.meta says which path ran.

Agentic security

Sage as a guardrail

77%
harmful agent work prevented
95.5%
harmless requests allowed
208 ms
median latency, client round-trip
~5×
faster than GPT-5 at similar quality
  1. The trade-off. Guardrails were either fast and shallow (regex, small classifiers) or smart and slow (an LLM in the hot path).
  2. The test. AgentHarm: harmful agent tasks to stop, harmless ones to let through, against GPT-5, GPT-5-mini, Google Model Armor, AWS Bedrock and open-weight classifiers.
  3. The result. Sage sits on the Pareto front: GPT-5’s quality at about five times its speed.
868890929496981005060708090% of harmless requests allowed → higher is better% of harmful prevented → higher is better← idealLevanto Sage, 77% prevented, 95.5% allowed, 208msLevanto Sage208msOpenAI GPT-5, 76.3% prevented, 96% allowed, 1009msOpenAI GPT-51009msGoogle Model Armour, 89.7% prevented, 87.5% allowed, 195msGoogle Model Armour195msOpenAI GPT-5-mini, 56.1% prevented, 100% allowed, 1114msOpenAI GPT-5-mini1114msMeta PromptGuard, 57.3% prevented, 95.5% allowed, 301msMeta PromptGuard301msInjecGuard, 55.8% prevented, 95.5% allowed, 262msInjecGuard262msProtect AI, 54.7% prevented, 95.5% allowed, 312msProtect AI312msAWS Bedrock LOW, 58.8% prevented, 88.1% allowed, 191msAWS Bedrock LOW191msAWS Bedrock MED, 51.6% prevented, 96% allowed, 188msAWS Bedrock MED188msUnguarded GPT-4o, 51.1% prevented, 100% allowedUnguarded GPT-4o

Model routing

Sage as a model router

98.3%
AIME quality at quality-first
27–77%
cheaper than OpenRouter Auto Beta
~$0.0006
per routing decision
1 dial
from quality-first to cost-first
  1. The trade-off. A router runs before every request and must pay for itself, so routers are built from small classifiers that cannot judge difficulty.
  2. The test. 60 AIME problems, same prompts on both sides, unrestricted model pool, three cost settings, against OpenRouter Auto Beta.
  3. The result. Equal or better quality at every setting, 27–77% cheaper, and a quality dial that moves the right way.

Trade a little quality for a lot of spend.

Quality
Levanto
98.2%
OpenRouter
93.0%
Cost / prompt
Levanto
$0.0062
OpenRouter
$0.0264
−77%cost, at equal or better quality
An engraved blue sea and mountain landscape under a rising sun.

Intelligence, at runtime