Here is a runbook, as it exists in most organisations today. "If the error rate is high, check whether a deploy went out recently. Look at the pod status. If pods are restarting it's probably OOM — check the limits. Otherwise check the payment service, it's usually that." Accurate, useful, and completely unexecutable.
Turning that into something an agent can run is not a translation exercise. It forces you to answer questions the prose let you leave vague, and those questions are the interesting part.
What the prose leaves out
Read that paragraph again as a machine would. High — above what, measured how, over what window? Recently — within ten minutes or two hours? Probably OOM — what would confirm it, and what should happen if it is not? Usually that — how would we know this time?
A human reader fills those gaps from context. That is exactly why the page decays: the gaps are where the knowledge actually lives, and they are never written down.
The four parts of a runbook held as data
| Part | What it is | What it forces you to decide |
|---|---|---|
| Match rules | Which alerts this runbook handles — name, service, severity, source, required labels. | Exactly when this applies, rather than "for error-rate-ish problems". |
| Variables | Named values derived from the incident, with defaults — service_name, namespace, threshold. | What the runbook needs from the alert, and what to do when the alert does not carry it. |
| Steps | An ordered list of typed actions with parameters, each saving its output under a name. | The actual query. Not "check the error rate" but the PromQL. |
| Conditions | run_if / skip_if expressions over anything an earlier step saved. | The branches the prose hid behind "if … otherwise". |
The same runbook, converted
That paragraph of prose becomes this. Note what had to be decided to write it — and that every one of those decisions was previously in somebody's head.
Tell the source somebody has it, so the page stops escalating while the agent works.
error_rate
The PromQL the prose meant by “high”, with the threshold as a variable off the alert label.
pod_status
Restart counts and readiness — the evidence behind “probably OOM”.
recent_deploys
“Recently” becomes an explicit window, so the answer is the same every time.
dep_health
The “it’s usually the payment service” hunch, made checkable.
error_logs
Conditional: runs only when the error rate is actually elevated. run_if {{error_rate}} > 0.05
classification
Weighs the five pieces of evidence above and names a cause with a confidence.
review
States what the conclusion rests on, and lowers the confidence where a link is assumed.
Skipped when the classification is a false positive. skip_if {{classification.category}} == 'false_positive'
Auto-resolves only when the reviewed confidence is above the floor and the cause is benign.
The conversion is the value, not the output
Most teams expect the benefit to be automation. The first benefit is actually that somebody had to write down what “recently” means, and now everyone uses the same definition. We have watched this argument happen in a room — two engineers who had worked the same alert for a year discovering they meant different windows.
Why this can be tested and prose cannot
Because the runbook is structured, it can be walked without being run. The dry run builds a sample alert from the runbook's own match rule, interpolates every variable, and reports what each step would do — without executing anything, so a rehearsal can never page anyone.
from the match rule"] --> B["Resolve variables"] B --> C{"For each step"} C --> D["Action type known?"] C --> E["Every {{reference}}
resolvable?"] C --> F["System configured
in this deployment?"] C --> G["Conditions reference
something that exists?"] D & E & F & G --> H["Per-step verdict:
ok · warning · blocked"] H --> I(["Plus: which steps
change something"]) style I fill:#fff4e2,stroke:#f4a03a,stroke-width:2px
That last output matters more than it looks. Before you activate a runbook, you are told which of its steps would change something outside the platform — acknowledge a page, create a ticket, silence an alert, restart something. It is a blast-radius statement generated from the runbook itself rather than from a promise.
Getting from the wiki to here
The migration people fear is converting hundreds of pages. Do not do that. Three practical points from watching teams start:
- Start with the three most-fired alerts, not the most-documented. The wiki is longest where the incidents were most interesting; the value is highest where they are most frequent. These are rarely the same pages.
- Import the prose and keep it attached. The original page stays on the runbook as its source, so a reader can always see what the human version said. The parse is a starting point, not a replacement.
- Add steps at incident closure, not in a documentation sprint. The moment to write a step down is the moment someone just ran it by hand and is annoyed about it.
The honest limitation
Some runbook knowledge does not convert. “Ask Priya, she knows how this cluster behaves” is real, useful, and not a step. Encode what is mechanical; leave the rest as prose attached to the runbook, where an engineer will still find it.
Next in this series
The agent that says “I don't know” — what happens between step 7 and step 8 above, and why the verification step is the one that decides whether anyone trusts the output.