What is calibrated confidence?

Calibrated confidence is a model score that matches reality: of the answers it rates 0.8, about 80 percent turn out right, so the number can be acted on.

Calibration is a property of the score, not of the answer. A model can be sure and wrong. Calibration is the thing that makes its certainty worth reading.

The plain test for calibration

Take every answer the model scored around 0.8. Count how many turned out right.

If it is near 80 in 100, the score is calibrated and you can gate on it. If it is 55 in 100, the model is overconfident and 0.8 means nothing. If it is 98 in 100, it is underconfident, and you are paying people to review work that did not need them.

That is the entire idea, and almost nobody states it this simply. Calibration is a frequency claim about a bucket of answers, not a feeling about any single one. You cannot tell from a single answer. You need a set whose real outcome you already know.

The check has a shape you can run on a spreadsheet:

Those numbers are illustrative, not measured. A model whose 0.9 bucket comes back at 62 percent is not one you can automate against at 0.9, however good the answers read.

Two practical notes on running it. Bucket edges come from where your scores actually land rather than from round numbers, which is why the ladder above stops at 0.3 instead of at zero. And a bucket needs enough answers to say anything: fifty outcomes pin a bucket sitting near 80 percent to roughly plus or minus 11 points, which is enough to tell 80 percent from 55 percent and never enough to tell it from 85.

Why asking a model how confident it is does not give you this

Ask a chat model "how confident are you?" and you get a sentence, produced the same way the answer was: token by token, shaped by how you phrased the prompt and by what confident writing looks like in the training data. It is a claim about certainty rather than a measurement of one.

This is not a reason to sneer at the technique. Real signals exist: token log probabilities are numbers the model actually produced, measuring how often repeated answers agree is a genuine method, and so is training a small verifier on your own labelled cases. A self-reported score can even turn out to be calibrated, which is an empirical question the bucket test answers.

The narrow point is that the number inside the sentence has not been measured against anything, and adding "be honest about your uncertainty" to the instruction does not change that.

Three different things get called confidence

Only the third column is calibration. The first two are inputs to it.

The derived kind, and where it comes from

Some APIs hand you the middle column directly, computed rather than composed. TypeSafe's documentation for its Jev model states the mechanism in one line: choice and score answers "carry a confidence between 0 to 1, derived from the answer's probability distribution". Their confidence page adds that the shape of that distribution is what carries the meaning, concentrated on one outcome being a confident answer and spread out an uncertain one. All of that is vendor-published and early access, and it is the number a decision worker on WorkerKit gates on. Their yes/no question type carries no confidence field at all.

A derived score is a better starting point than a generated one, because it is a statistic over something the model produced rather than prose about it. See system one model for the shape of model that returns one, structured output for why a declared schema makes the distribution possible, and Jev, TypeSafe's System One model for what that vendor does and does not publish about it.

It is still not calibration. Derived confidence says the mass was concentrated. Calibration says that when it concentrates, the answer is right that often.

A spread answer is not always doubt

Spread means the model did not put its mass on one option. It does not say why, and there are three reasons:

• The input is genuinely ambiguous, and hedging is correct.
• Two options are both acceptable. A support email fairly described as billing or as account splits the mass between them, and either route is fine.
• Your options overlap, so the model is being asked to separate things that are not separate. That is a defect in the question rather than doubt in the model, and it is fixed by writing options that exclude each other.

So a low score is a reason to stop and hand over, not evidence that the answer was wrong.

Calibration on their data is not calibration on yours

A vendor's calibration result is a claim about the vendor's distribution of inputs. Your inbox is not their benchmark set. The same score can be honest on generic English and badly optimistic on your invoices, your dialect of shorthand, and the three edge cases your customers produce weekly.

TypeSafe's own documentation makes the same argument from the other end: threshold values, it says, depend on your domain and on how the model performs for your use case, and different actions in one system should be gated at different levels depending on the consequences of being wrong. Its own bands are prose rather than numbers, and the 0.9 and 0.5 in its worked code example are illustrative rather than a published ladder: even at 0.9, that example confirms with the user before acting on a high stakes decision.

Any number you gate on has to be re-derived against your own labels, which is what a confidence threshold is and how you set one.

When there is nothing to calibrate

Calibration is a measurement, and some setups cannot produce one. Four of them, before you budget an afternoon for labelling:

• The model returns no score. A general chat model handed a classification task returns an answer and nothing to bucket, and a yes-or-no probability is the answer itself rather than a confidence in it. With no score there is nothing to calibrate, and the controls left are structural ones.
• Nobody ever learns the outcome. Calibration needs the truth to arrive later. If the worker files an item into a queue no one audits, its answers are never scored right or wrong at all, and no amount of volume produces a label. Fix the feedback path first; the measurement is downstream of it.
• The interesting cases are rare, even at high volume. Ten thousand items a month with eleven genuine escalations in them will fill the top bucket fast and never fill the bottom one, which is the bucket the whole decision turns on. Route the rare class to a person and measure the common one.
• The setup changes weekly. A calibration result belongs to one model, one instruction and one option list. Change any of the three and it expires. If you are still rewriting the question every few days, you are measuring something that will not exist next week.

Why this matters for unattended work in particular

When a person is watching, a bad score costs a second of attention. An AI worker runs on a schedule at 3am and nobody reads the borderline case. Where a score is what decides between acting and escalating, that score is the entire review, which is why it has to mean something.

A decision worker returns a confidence with every judgment, and that is not calibration: it is the model's read of one answer, where calibration is a hit rate across a bucket of answers whose outcomes you already know. No platform can compute calibration for you, because the labelled set it needs is yours. Every run of a decision worker leaves a run receipt, so you can read back a month of decisions and mark which were right. That marked set is the data the bucket test needs.

Until you have it, gate on structure rather than on a number: hold each app one step below the action you would regret, and write the uncertain tail into the instruction as a handover. Support Inbox Triager is built that way. It classifies against a fixed rubric it fetches at run time, drafts replies and lists anything abusive or legal for a human, and its email permissions cover reading, labelling and creating a reply draft, with no send operation granted at all. Its escalation does not depend on a threshold being right. More of that pattern is in failing safely on an unattended fleet.

One limit no score removes: a calibrated number does not make an unsupervised decision about a person acceptable. Hiring, credit, benefits and discipline carry legal obligations whatever the confidence value. Sorting and surfacing for a reviewer is fine. Deciding is not.

FAQ

What does calibrated confidence mean?

It means the score matches the observed hit rate. Of all the answers a model scores 0.8, about 80 percent should be correct; of those scored 0.6, about 60 percent. A number that does not hold to that is one you cannot act on, however precise it looks.

Can I trust a model that says it is 95 percent sure?

Not on the strength of the sentence. A model asserting 95 percent in generated text is writing, not measuring, and a confident wrong answer often scores itself highest. Trust it after you have checked a bucket of such answers against known outcomes on your own data.

How do I check whether a confidence score is calibrated?

Collect a few hundred answers with their scores, label which were actually right, group them into score buckets, and compare each bucket's hit rate with its score. Two hundred labelled cases is enough to see whether a threshold is plausible. It takes a spreadsheet and honest labels, not a statistics team.

Does low confidence mean the answer is wrong?

No. It means the model's answer was spread across options. Sometimes the input is genuinely ambiguous, sometimes two options are both acceptable, and sometimes your option list overlaps and should be rewritten. Low confidence is a reason to route to a person, not a verdict on correctness.

Is the confidence a worker returns already calibrated?

Not by itself. A decision worker returns a confidence with each judgment and will not act below its confidence floor, but that score is the model's read of one answer, not a hit rate measured on your outcomes. Only your own labelled runs produce that. Until they do, what holds is structural: per-app permissions, contact rules, redaction, and a receipt for every run you can label afterwards.