Two signals, two questions
A prediction tells you which result the model favors. Uncertainty helps you decide whether to use that prediction automatically. Keep these separate in both the code and the interface.
For Choice and Score, TypeSafe returns a probability distribution and a derived confidence statistic describing its concentration. Noul has no separate confidence field. The official confidence guide explains the distinction.
A confidence value is not a certificate that an individual answer is correct. Read it as one input into a policy you must evaluate.
Design a review lane
Consider a document classifier. One lane accepts sufficiently certain labels. Another sends ambiguous documents to a person or a more capable model. A third can reject documents that are outside the supported categories.
The official confidence-routing pattern describes this architectural approach. Its practical benefit is that the system has a place to put uncertainty instead of disguising it as a definite result.
Pick thresholds with evidence
Start with a held-out set that reflects the inputs you expect. Try several thresholds and record both automatic coverage and the errors inside the automatic lane.
| Measure | The question it answers |
|---|---|
| Automatic coverage | How much work bypasses review? |
| Automatic-lane error rate | How often does that lane act incorrectly? |
| Review volume | Can the fallback actually handle the remaining work? |
| Total cost and latency | Does the complete workflow still meet its goal? |
There is no universal threshold that makes every task safe or useful. A suggested folder label and an irreversible account change have very different consequences.
Preserve the uncertain result
Do not overwrite an ambiguous answer with a confident-looking default just to satisfy a downstream interface. Preserve the original score and the reason the item entered review. That lets you distinguish a model error from a routing-policy error later.
For Noul, evaluate both sides of the probability scale. Depending on the application, a middle band can lead to review while values near either end support opposite decisions.
Recheck after a model change
A moving model alias may eventually resolve to a different version. Keep the response's model identifier with your evaluation results. If thresholds are part of an important workflow, compare the new model against the same held-out set before changing the deployed version. The model reference describes versioned IDs and aliases.
See the community email-screening case for an example of a two-stage workflow. Its reported numbers are specific to that creator's experiment.