Stop adding “uncertain” to your decision model’s answer options

When a decision model returns a probability for each option, use the options to describe business outcomes. Handle low confidence in application code.

Sage, the decision model from Levanto Labs, also returns null when it cannot select a clear winner. Keep that review path when you remove a generic “uncertain” option.

I’ve seen the same pattern across integrations: a developer asks for a workflow decision, and the coding agent adds “uncertain,” “not sure,” or “low confidence” alongside the actual outcomes.

A refund workflow ends up with this menu:

[
  "approve_refund",
  "decline_refund",
  "escalate_to_human",
  "uncertain"
]

The last option gives the model another way to express doubt through the answer itself. In integrations I’ve worked on, this can send too many cases to a fallback path.

The fix is to define meaningful outcomes, test the confidence signal against real examples, and use an explicit threshold to control automation.

Should “uncertain” be an answer option?

A decision-model response gives your application two distinct pieces of information:

• The selection: Which outcome best fits the supplied context?
• The confidence signal: How strongly does the model support that selection?

Keep those meanings separate.

For the refund workflow, define these outcomes:

A high-confidence selection of escalate_to_human is valid. The model may be confident that a policy exception requires a manager’s approval.

A low-confidence selection of approve_refund is also possible. Your application can send it for review because the selection does not meet your automation threshold.

These are different reasons for review. Keeping them separate makes the workflow easier to measure and debug.

An additional option belongs in the menu when it describes a meaningful state or action. For example, request_missing_information can identify a missing receipt and trigger a request to the customer.

An explicit “insufficient evidence” outcome can also be valid when assessing evidence sufficiency is the task. Define what it means and what the application does next.

The rule I use is simple: every option should have a defined meaning and a defined consequence in application code.

How does Sage report uncertainty?

For Sage Choice decisions, the relevant fields are:

Sage describes its selection probabilities as calibrated. You should still check how they perform on your own workload.

In this article, confidence threshold means the cutoff your application applies to result.probability.

Removing an “uncertain” option does not mean forcing every response into an action. Handle a null selection explicitly. Also keep any valid business outcome that requires human review.

Handle low confidence in TypeScript

Put the automation decision in application code.

The example below uses the chosen and probability fields from Sage’s Choice response. Validate the response shape and permitted action names at the API boundary, then pass response.result to the function. TypeScript types alone do not validate incoming JSON. Ticket, the review queues, and executeValidatedAction are application-specific.

type Action =
  | "approve_refund"
  | "decline_refund"
  | "escalate_to_human";

type ChoiceResult = {
  chosen: Action | null;
  probability: number | null;
};

function routeDecision(
  ticket: Ticket,
  result: ChoiceResult,
  threshold: number
) {
  if (
    !Number.isFinite(threshold) ||
    threshold < 0 ||
    threshold > 1
  ) {
    throw new RangeError("Threshold must be between 0 and 1.");
  }

  if (result.chosen === null) {
    return queueReview(ticket, {
      reason: "no_selection",
      result
    });
  }

  if (result.chosen === "escalate_to_human") {
    return queueHumanReview(ticket, {
      reason: "policy_requires_review",
      result
    });
  }

  const score = result.probability;

  if (
    score === null ||
    !Number.isFinite(score) ||
    score < 0 ||
    score > 1
  ) {
    return queueReview(ticket, {
      reason: "invalid_confidence",
      result
    });
  }

  if (score <= threshold) {
    return queueReview(ticket, {
      reason: "below_automation_threshold",
      result
    });
  }

  // Existing authorization and business-rule checks still apply.
  return executeValidatedAction(ticket, result.chosen);
}

This structure records why a case was deferred. The team can distinguish policy-required review from low confidence, missing selections, and invalid responses. An invalid threshold raises a configuration error before any action executes; the caller must handle that error without executing the action.

The example applies to Choice decisions. Other answer types need their own evaluation. For tags, evaluate each tag and any action that depends on several tags together.

How do you choose a confidence threshold?

Start with representative examples and reference decisions. For customer support, use tickets labeled by experienced managers under the same policy supplied to Sage.

Keep the evaluation context consistent. If the manager saw account history that the model did not receive, a disagreement may reveal missing context.

For each example, record:

example_id
reference_action
selected_action
selection_probability
model_version
prompt_version
policy_version

Record null responses, failed requests, and review outcomes separately. They affect how much work the integration can complete automatically.

If reviewers disagree, adjudicate those examples or track the disagreement.

Consider the following example.

Hypothetical threshold calculation. These figures are not measured Sage results.

Suppose the selected action matches the adjudicated reference label on 9,400 of 10,000 tickets. That gives 94% agreement and 600 errors.

For this calculation, assume every selection is otherwise eligible for automatic execution. This isolates the effect of the confidence threshold.

In this hypothetical example, sending the lower-confidence group for review produces:

• 98.5% accuracy among automatically accepted decisions: 7,880 correct decisions out of 8,000.
• 80% automation coverage: 8,000 of 10,000 decisions pass the threshold.
• 80% error capture: the review path receives 480 of the original 600 errors.

The review path also receives 1,520 correct decisions. That is the cost of the threshold: more review work in exchange for fewer errors passing through automatically.

The remaining 120 errors still matter. The threshold does not identify every incorrect answer.

This calculation explains threshold selection. It does not measure the effect of removing an “uncertain” option.

Test several thresholds

The 0.75 threshold is illustrative. It is not a recommended Sage default.

Compare accuracy among accepted decisions with automation coverage and review volume. Choose a threshold based on the consequences of errors and the capacity of the review process.

Check results by action. Aggregate accuracy can hide poor performance on a rare but important category, such as refund approvals.

Different actions may need different thresholds. Use enough labeled examples to evaluate each threshold meaningfully.

This confidence-based acceptance step follows the selective prediction approach: accept some predictions and abstain on others. Evaluate the trade-off between coverage and error among accepted predictions on your own data.

Validate the complete workflow

Use one labeled dataset to develop the prompt and choose the threshold. Then freeze the configuration and test it on a separate held-out dataset.

If you compare answer options with and without “uncertain,” use the same model version, policy, context, and test examples. Tune each version’s threshold on development data before comparing accepted accuracy and coverage on the held-out data.

Give each review path a specific job. It might:

• Retrieve missing account information and make another Sage call.
• Supply a more relevant policy excerpt.
• Ask another model or an LLM judge to assess the case.
• Escalate to a human.

Repeating an unchanged request may add little information. Another model can make the same mistake. Human review can also be inconsistent.

Measure the complete workflow:

• Accuracy among automatically executed decisions.
• Automation coverage and review volume.
• Errors that pass the threshold.
• Performance by action and ticket category.
• Final accuracy, cost, and elapsed time after review.

The hypothetical 98.5% figure describes only the automatically accepted group. Overall workflow accuracy also depends on what happens to the other 2,000 cases.

After deployment, sample automatically executed decisions as well as deferred cases. Reviewing only escalations will miss high-confidence errors.

Re-evaluate when policies, traffic, prompts, or model versions change.

Start with the answer options your coding agent generated. Keep outcomes with a defined business meaning. Handle null selections and low confidence in code. Then test which decisions your application can execute automatically and which need another step.

Use the Sage Choice API documentation to define your options and handle the response fields. Follow the Sage Quickstart to make your first call, or use the Sage agent skill when building with a coding agent.