Cost per acceptable AI result

Measure AI worker value using accepted outcomes, including execution, retries, human review, correction and maintenance in the total cost.

By Adam Shamos. Updated 2026-10-29.

Cost per acceptable AI result is the total operating cost of a job over an evaluation period, divided by the number of results that meet its agreed acceptance criteria. It measures what your company spends to get usable work, including the work people still do around the AI worker.

A low token bill can accompany expensive checking and correction. A more expensive model can be economical if it produces enough additional usable work or reduces that effort. Measure both against the same job and quality standard.

Define an acceptable result before counting it

For the Inbound Lead Qualifier, one completed model response is not the business result. The job reads an inquiry and CRM context, grades the lead with reasons, creates or updates the contact, logs a note and prepares a reply draft. It can start on demand or on its supported inbound event. The customer reply is left as a draft.

An acceptance rule for that job could require the correct contact, a grade supported by the inquiry and CRM information, and a draft that answers the inquiry without unsupported claims. Agree on what makes each condition pass before the evaluation starts.

Keep four counts separate:

The denominator is the total number of acceptable results, including corrected ones if your definition permits them. A failed attempt followed by a successful retry is one result and two attempts. A reply with three drafts is still one result. An unresolved item is not an acceptable result merely because a run ended successfully.

Include the costs outside the run receipt

Run receipts supply execution evidence and costs. They cannot supply your labour rate, the time spent checking work, or your definition of a successful business outcome. Your evaluation combines those sources.

Avoid counting a charge twice. If model spend appears in both a run record and a provider invoice, use one as the cost and the other as reconciliation. If a prepaid wallet funded that spend, the wallet load itself is not another copy of the consumption cost. Track any applicable fees separately.

For a shared platform plan, state how you allocate the plan charge across jobs. For setup, choose whether the trial includes the whole initial cost or spreads it across a stated operating period. Show the choice. Report trial cost and expected ongoing cost separately when the allocation materially changes the conclusion.

The pricing page gives current charges, and the AI worker cost breakdown explains the usage meters. This evaluation adds the labour and quality information those meters cannot answer.

Work through the arithmetic

The figures below are hypothetical evaluation inputs, not WorkerKit prices or measured customer results. Suppose two configurations process the same 100 eligible inquiries. The team values review and correction time at $60 an hour and applies the same acceptance rules to both.

Configuration B spends more on execution but less overall in these assumed conditions. The calculation alone does not establish which model or workflow will behave that way on your work. It shows why execution spend is an insufficient buying measure.

Also retain the results that the average conceals: 20 inquiries remain unacceptable under A and 10 under B. Record their failure reasons and which ones needed urgent handling. Neither configuration is acceptable for production if those failures violate a requirement the business cannot waive.

Use the same work for a fair comparison

Choose representative cases, including incomplete inputs, ambiguous records and the exceptions your team routinely handles. Preserve the input conditions so the second configuration does not receive an easier job or benefit from records the first one already changed. Use a controlled evaluation setup for cases that would otherwise cause duplicate app writes.

Hold the acceptance rule constant. If one configuration produces a short summary and another produces a checked CRM record plus a draft, their output units are different. Either evaluate the same deliverable or cost the remaining work needed to reach it.

Keep the same review policy during the comparison too. Reducing review because you expect a model to be better can make a configuration appear cheaper while allowing more errors through. Report first-review acceptance separately from eventual acceptance so correction is visible.

Change one substantial choice at a time where practical: the model, the instructions, the input scope or the division between workers. This makes the result useful for the next decision. The model selection article covers comparing supported models on your own work.

Keep a worksheet for each job

Your worksheet can be a simple table maintained by the person who owns the job. Record the evaluation dates, configuration, input population, acceptance criteria and cost allocation assumptions above it.

For each eligible item, record its identifier, associated run receipts, first-review outcome, final outcome, review minutes, correction minutes and failure reason. Count the item once in the outcome totals and include every attempt in cost. Track maintenance effort for the evaluation period separately from the individual items it supports.

At the end, calculate total cost, acceptable results and the ratio. Then check the distribution: did most of the correction come from a specific source or kind of inquiry? A narrow failure category can suggest a better routing rule or a smaller job. A broad failure pattern may mean the method is not ready.

WorkerKit does not automatically compute this full business metric. It records execution and outcomes; your company supplies acceptance decisions and labour costs. Use the inbound sales guide to define the job, then retain the evaluation alongside the method you keep improving.

Improve the result without hiding work

Several changes can improve economics, but each needs another comparison.

A decision worker can route selected items to a language worker, so expensive work is reserved for cases that need it. Include the decision step's cost and check the items it excludes. A cheaper system that silently misses good leads has changed the result rather than improved the same job. The screening and routing article develops that tradeoff.

An Action worker can perform a specified app operation without a model. This can remove an unnecessary model step, but incorrect inputs from its caller still lead to incorrect work. Include the effort of detecting and repairing those mistakes.

Better instructions can reduce ambiguity. Where learning is enabled for language workers, instruction changes can be checked against reviewed runs before applying under the configured approval or proven-changes mode. Keep the evaluation effort in the cost calculation, and compare outcomes after the change rather than assuming improvement.

When this metric is not enough

Cost per acceptable result is one decision measure. Response time, data handling, capacity during a busy period and the consequences of a rare error can matter more than the average cost. Set those requirements separately and reject configurations that fail them.

Compare the worker with your current process, including the tools you already operate. If your existing setup delivers the same acceptable work with less cost and upkeep, adding WorkerKit has not earned its place. A useful AI workforce grows from jobs that justify being kept.

Time released from review or execution is additional capacity. It becomes a cash saving only when spending actually falls. State whether your business case depends on handling more work, responding sooner, reducing expenditure or some combination. Do not put a labour value into the worksheet and present it as a guaranteed reduction in payroll.

For a broader buying decision, the enterprise AI automation platform evaluation combines this cost measure with mandatory controls, operational checks and the decisions each owner needs to make.

FAQ

Is cost per acceptable result the same as cost per run?

No. Cost per run measures an execution unit, while cost per acceptable result includes all attempts and the human work needed to produce an accepted outcome. Several runs can contribute to one result, and some runs produce no acceptable result.

What if no result meets the acceptance criteria?

If there are zero acceptable results, the ratio is undefined. Report the total cost and zero acceptable results, then investigate the failures before using a per-result figure. The absence of usable output is itself the finding.

Should corrected results count?

Corrected results can count if they meet the agreed final acceptance criteria and all correction effort is included. Also report acceptance on first review so a high final pass rate does not conceal a large repair burden.

Does WorkerKit calculate our ROI?

WorkerKit provides execution records and editable kit estimates. Your team supplies the acceptance criteria, labour costs and other business assumptions needed to evaluate the result. Browse kits to identify a job to measure; accounts are available by invitation through the early-access waitlist.