How to roll out an AI worker safely
Four stages from read-only to autonomous, what to check before each promotion, and the signals that mean go back a step rather than push on.
The tempting way to deploy an AI worker is to grant it what the job needs and turn it on. It usually works, which is the problem: it teaches you nothing about the cases where it will not.
A staged rollout costs about two weeks of patience and converts an unknown into something you have evidence about.
The four stages
| Stage | Access | What runs | What you learn |
|---|---|---|---|
| 1. Read-only | Read on every app | Full schedule | Whether it understands your data |
| 2. Drafts | Write, but instruction says draft only | Full schedule | Whether you agree with its output |
| 3. Narrow send | Write, sending to a limited set | Full schedule | Whether it behaves with real consequences |
| 4. Autonomous | Write, full scope | Full schedule | Nothing new, if the first three went well |
The stages differ by blast radius, not by how much the worker does. It runs its real schedule on your real data from day one, which is the point: a worker tested on a sample is a worker you have tested on a sample.
Stage 1: read-only, one week
Set every app to Read in the app firewall and let the worker run its intended schedule. It cannot change anything, so the worst case is a wrong opinion in a receipt.
What you are checking is comprehension rather than behaviour:
- Does it identify the right items? A triage worker that reads the correct threads and categorises them oddly is fixable. One reading the wrong threads is misconfigured.
- Does its reasoning match yours on the easy cases? If it disagrees with you on the obvious ones, the instruction is wrong, not the model.
- What surprised you? Those are lines for the instruction, and they are the whole reason this stage exists.
Executive Daily Briefing never leaves this stage: it holds Read on calendar, email and tasks and writes nothing anywhere, which is why it is the best first worker to deploy at all.
Stage 2: drafts, one to two weeks
Grant Write, and have the instruction produce drafts rather than sends. This is the stage most workers should stay in longest, and some should stay in forever.
The check changes from "does it understand" to "would I send this". Read every draft for the first few days, then sample. You are looking for a specific thing: the confidently wrong one. Drafts that are obviously bad are harmless because you catch them. The one that reads perfectly and contains a wrong commitment is the actual risk, and you will only find it by reading carefully while the stakes are still zero.
Support Inbox Triager works this way by design, reaching email and tasks to draft replies and file tickets. Drafting is the safe default, and promoting past it is a decision to make deliberately rather than by forgetting to.
Stage 3: narrow send
If the job genuinely requires sending, narrow who before widening what. Contact rules decide which people a worker may reach, so a first send stage can be limited to a set where a mistake is recoverable: internal recipients, a pilot segment, your own address.
This stage exists because stages 1 and 2 share a blind spot. Both are judged by you reading output at your leisure. Sending introduces timing, volume and a recipient who did not ask to be part of your evaluation, and those produce failures the earlier stages cannot surface.
Stage 4: autonomous
If the first three stages went well, this stage teaches you nothing new, which is exactly the condition for entering it.
What changes is monitoring cadence rather than mechanism. Receipt retention becomes the constraint: 1 day on Free, 30 days on Pro, 365 on Team. A worker acting autonomously on a monthly review cycle needs more than one day of history to review.
What to check before each promotion
| Before promoting to | Require |
|---|---|
| Drafts | A week of read-only runs, no comprehension errors, edge cases written into the instruction |
| Narrow send | Every draft read for three days, then a week sampled, no confidently-wrong output |
| Autonomous | Two weeks of narrow sending, no recipient-affecting mistake, a review cadence agreed |
The rule that makes this work: one promotion at a time. Widening access and increasing frequency in the same week means a change in behaviour has two candidate causes and you will guess wrong.
The signals to go back a step
Rollouts are usually described as forward-only, which is why people push on through evidence they should have acted on.
- The same category of mistake twice. Once is an edge case for the instruction. Twice is the instruction being wrong about something structural.
- You stop reading the receipts because they are always fine. That is when a drift starts and nobody is looking.
- You cannot explain a decision it made. Not disagree, explain. If the reasoning is opaque on a case you understand well, it will be opaque on ones you do not.
- The job changed. A worker rolled out against last quarter's process is now running a process nobody uses.
Going back a stage is cheap: narrow the permission and the worker keeps running with whatever is left, recording on the receipt what it could not do. Nothing is deleted and no configuration is lost.
What this costs
Nothing extra in capacity terms, which is the argument for doing it.
| Stage | Runs | Tool calls a day | Notes |
|---|---|---|---|
| Read-only week | Full schedule | Same as production | Reads cost the same as writes |
| Drafts | Full schedule | Roughly the same | A draft is one call |
| Narrow send | Full schedule | Same | Fewer recipients, same work |
A worker doing about 120 tool calls a day costs that in every stage, comfortably inside Free's 500. The rollout costs **time and attention**, not money, which is worth knowing before treating it as an expense to skip.
The reasonable objection
"Four stages and three weeks to deploy one worker is not viable when I want a fleet of ten."
Correct as stated, and the resolution is that stages are per job shape, not per worker. Once you have taken one email-drafting worker through the sequence, the second one starts at drafts, because you now have evidence about how workers behave on your mail and what your instructions tend to miss.
The full sequence is for your first worker of a kind and for anything whose mistakes reach a customer. A read-only digest can go live this afternoon.
When to skip stages
- Read-only workers. There is nothing to promote to. Deploy it.
- A kit you have already run in another form. Start where your evidence starts.
- Reversible, internal-only output. A draft in your own drafts folder has approximately no blast radius.
- You are evaluating the kit, not the job. If you are deciding whether to use a worker at all, run it read-only and decide from that.
FAQ
How should I roll out an AI agent for the first time?
In stages of increasing blast radius: read-only for a week, then producing drafts for one to two weeks, then sending to a narrow set of recipients, then autonomous. The worker runs its real schedule on real data throughout; only the consequences of a mistake change.
How long should a worker run read-only before I grant write access?
About a week, and long enough that something has surprised you. The surprises are the edge cases your instruction is still missing, and read-only is the stage where finding them costs nothing.
What should I look for when reviewing drafts?
The confidently wrong one. Obviously bad drafts are harmless because you catch them; the risk is output that reads perfectly and contains a wrong commitment or a policy you do not have. That only shows up by reading carefully while stakes are still zero.
Can I reduce a worker's permissions after granting them?
Yes, at any time. The worker keeps running with whatever access is left and records on its receipt what it could not do, so stepping back a stage costs nothing and loses no configuration.
Do I have to do this for every worker?
No. The full sequence is for your first worker of a given shape and for anything whose mistakes reach a customer. Later workers of a familiar shape can start at drafts, and read-only workers can go live immediately.