AIOps & LLMs in Operations

The Incident Response & Postmortems page described the worst minutes in operations: the site is down, alerts are flooding in, five engineers are reading the same logs, and the clock on MTTR is running. Two waves of machine assistance promise to shorten exactly that moment. The older one — AIOps — has been around since the late 2010s. The newer one — LLMs in the loop — arrived with force in 2024–2025 and is still finding its shape. This page covers both, and is honest about the new failure mode they introduce: a confident machine that is sometimes confidently wrong. The book’s thread holds, but with a sharp asterisk.

AIOps: machine learning on telemetry (the older wave)

AIOps — a term Gartner coined around 2016–2017 for Artificial Intelligence for IT Operations — predates the LLM boom entirely. It is classical machine learning and statistics applied to the flood of telemetry your systems emit, aimed at three jobs:

• Anomaly detection — learn a metric’s normal shape (including daily/weekly seasonality) and flag deviations, instead of relying on a human to pick a static threshold that’s wrong half the day.
• Event correlation / noise reduction — when one root failure triggers fifty downstream alerts, cluster them into a single incident so on-call sees one problem, not a wall of pages. A direct attack on the alert fatigue the incident page warned about.
• Forecasting — predict capacity exhaustion or a cost overrun before it happens.

None of this is new or speculative; it is mature, useful, and unglamorous. The honest caveat is that AIOps is only as good as its data and tends to need real tuning to avoid trading one kind of noise (too many alerts) for another (false anomalies). It augments the metrics-and-alerting discipline; it does not replace it.

The 2024–2025 wave: LLMs in the loop

What changed recently is the arrival of large language models at tasks that used to require a human reading and writing prose under pressure. As of 2024–2025 the well-supported, genuinely useful patterns are:

Task	What the LLM does	The toil it removes
Log / alert summarization	Compress thousands of log lines or a noisy alert storm into a few sentences of “here’s what seems to be happening”	The human slog of reading raw logs at 3 a.m.
Incident triage assist	Suggest likely affected services, related past incidents, and a first hypothesis	The cold-start of “where do I even look?”
Runbook copilot	Answer “how do we fail over the payments DB?” from your runbooks and docs in natural language	Hunting through wikis mid-incident
Postmortem drafting	Assemble the incident timeline from the chat and logs into a first-draft postmortem	The blank-page tax after a long night
Natural-language querying	”Show me error rate for checkout in the last hour” without writing the query language	The barrier of every tool’s bespoke query syntax

Notice every one of these is assistive: it drafts, summarizes, suggests, and surfaces. The human still decides and acts. That distinction is not a limitation to be engineered away — as the failure modes below show, it is the design.

Under the hood — why a runbook copilot needs RAG, not just a model

A naive runbook copilot — “ask the model how to fail over our database” — fails badly, because the model doesn’t know your systems and will cheerfully invent a plausible-sounding procedure. The pattern that makes it work is RAG — retrieval-augmented generation:

   question  ──►  RETRIEVE the relevant chunks from YOUR
                  runbooks / docs / past incidents (a search step)
                       │
                       ▼
                  GENERATE an answer grounded ONLY in those
                  retrieved chunks, with citations back to the source
                       │
                       ▼
   answer + links the engineer can verify before acting

Retrieval grounds the model in your actual, current documentation instead of its training data, which both makes the answer correct for your environment and gives the engineer a source to check. Grounding plus citation is what turns “a chatbot that might be making this up” into “a faster path to the runbook you’d have read anyway.” It does not eliminate the risk that the model misreads the retrieved text — which is why the human still verifies before running the command.

LLMOps: operating the thing that operates you

The moment an LLM is in your production incident path, it becomes a production system with its own operational concerns — and the discipline that has grown up around this since roughly 2023 is LLMOps (LLM operations). It is what MLOps was for classical models, specialized for the peculiar properties of generative models. The core concerns:

• Evals. You cannot unit-test an LLM with assertEqual — its output is open-ended. Instead you build an eval: a curated set of inputs with graded or reference outputs, run on every prompt or model change, so you can tell whether a “tweak” actually improved triage quality or quietly regressed it. Evals are to LLM features what CI tests are to code — the regression net.
• Prompt and version management. The prompt is configuration. A reworded system prompt can change behavior as much as a code change, so prompts get versioned, reviewed, and rolled back like code, and the underlying model version is pinned — because the provider updating the model under you can shift outputs with no change on your side.
• Token cost and latency in production. Every call costs money per token and adds latency, and both scale with how much context you stuff in. In an active incident, a triage assistant that takes 30 seconds to answer is worse than useless. Cost and latency become first-class SLOs, monitored like any other.
• Guardrails and observability. You trace prompts, retrieved context, and responses (so a bad answer can be debugged), and you put guardrails on what the model is allowed to do — which, for ops, almost always means it suggests, a human executes.

By the numbers — cheap per call, real at fleet scale

LLM pricing is quoted per token and changes constantly, so keep this qualitative and dated to ~2025. The point is the shape, not the digits:

   one log-summary call:
     input ≈ 10,000 tokens (a chunk of logs) + output ≈ 500 tokens
     at a typical mid-2020s rate of "fractions of a cent per 1k tokens"
       → on the order of cents per call   (verify current pricing!)

   per call:        ~pennies          → feels free
   per incident:    dozens of calls   → still small
   per fleet:       thousands of alerts/day × auto-summarize each
                    → a real, recurring line item — and added latency
                      on the hot path of every alert

The trap is reasoning from the per-call price (“it’s basically free”) and being surprised by the fleet-scale bill and latency. This is precisely the unit-economics lesson: cost per call is meaningless; cost per unit of value — per incident triaged, per engineer-hour saved — is the number to judge it by. An LLM that costs cents per alert but saves an engineer ten minutes is a bargain; the same LLM auto-summarizing alerts nobody reads is pure waste with extra latency.

It actually happened — Air Canada’s chatbot, and the cost of trusting an automated answer unsupervised

A clean, documented warning about letting an automated system answer without a human check comes from outside ops but transfers exactly. In February 2024, a tribunal in British Columbia (the Civil Resolution Tribunal, in Moffatt v. Air Canada) held the airline liable after its website chatbot told a customer he could claim a bereavement fare retroactively — which contradicted the airline’s actual policy. The airline argued the chatbot was a separate entity responsible for its own statements; the tribunal disagreed and held the company accountable for what its bot said. (The tribunal never established what technology drove the chatbot; whether or not it was an LLM, the lesson is the same.) The operational lesson is direct: an automated support system stated something plausible and wrong, with confidence, and a human acted on it. In an incident, the equivalent — a runbook copilot confidently inventing a failover step, or a triage assistant fingering the wrong service — is exactly the failure mode RAG-grounding, evals, and “it suggests, a human executes” exist to contain. The model’s fluency is not the same as being right.

The thread — with an asterisk

The manual, error-prone step this wave removes is the human toil of reading and synthesizing under pressure — scrolling thousands of log lines, correlating an alert storm by hand, hunting the runbook, facing the blank-page postmortem at the end of a long night. AIOps removes the correlation toil; LLMs remove the reading and drafting toil. Done well, MTTR falls and on-call’s cognitive load drops at exactly the moment it spikes.

The asterisk is the honest part, and it is new to this chapter. Every prior tool in this book — a reconciliation loop, a canary, a scanner — when it acted, acted deterministically and correctly on its inputs. An LLM does not. It can be confidently wrong, and an automated step that is sometimes wrong is more dangerous than a manual step that is reliably slow, if you let it act unsupervised. So this wave only makes production safer under a strict condition: the LLM stays a copilot — grounded in your real data (RAG), regression-tested (evals), version-controlled (prompts), cost-and-latency-budgeted, and with a human owning the decision and the keys. The toil it removes is real; the judgment it cannot be trusted with is the blameless-postmortem reminder that human-in-the-loop is a feature, not a transitional phase. Next, a frontier where the new component is deterministic again, and the question returns to pure efficiency: WASM and the edge.

The architect’s lens

Five questions for putting machines — and LLMs — in the ops loop:

• Why does it exist? Because the worst minutes in operations — an alert storm, five engineers reading the same logs, MTTR ticking — are full of human toil that machine assistance can shorten: AIOps (classical ML on telemetry) since the late 2010s, and LLMs in the loop since 2024–2025.
• What problem does it solve? Reading and synthesizing under pressure: AIOps clusters an alert storm into one incident and flags anomalies; LLMs summarize logs, draft postmortems, and answer runbook questions — each assistive, drafting and suggesting while the human decides.
• What are the trade-offs? Unlike every other tool in this book, an LLM can be confidently wrong — so it needs RAG-grounding, evals (the CI-test analogue), versioned prompts, and token cost/latency budgeted as SLOs; judge it by unit economics (per incident triaged), not the pennies-per-call price.
• When should I avoid it? Letting the model act unsupervised — a runbook copilot inventing a failover step, à la Air Canada’s chatbot — is the failure mode; keep a human owning the decision and the keys.
• What breaks if I remove it? The reading/correlating/drafting toil returns at the moment cognitive load spikes — though the asterisk is that removing an unsupervised LLM is often the safer call.

Check your understanding

1. Distinguish AIOps (the older wave) from the 2024–2025 LLM wave. Name one classic AIOps job and the incident-response problem it attacks.
2. Why does a runbook copilot need RAG rather than just a model? What two things does grounding-plus- citation buy you?
3. What is an eval, and why can’t you test an LLM feature with ordinary assertEqual-style unit tests? What earlier discipline is the eval the analogue of?
4. Using the by-the-numbers box, explain why “it’s basically free per call” is the wrong way to judge an LLM ops feature, and what number you should judge it by instead.
5. State the book’s thread for this page with its asterisk: what toil do LLMs remove, what new failure mode do they introduce, and under what condition do they actually make production safer?

Show answers

1. AIOps is classical ML/statistics on telemetry (anomaly detection, event correlation, forecasting), around since the late 2010s; the LLM wave applies generative models to prose-shaped tasks (log summarization, triage, runbook Q&A, postmortem drafting), arriving 2024–2025. A classic AIOps job is event correlation / noise reduction — clustering many downstream alerts into one incident, attacking alert fatigue.
2. Without retrieval, the model answers from its training data and will invent a plausible-but-wrong procedure for your systems. RAG retrieves the relevant chunks from your actual runbooks/docs and grounds the answer in them. Grounding-plus-citation buys (a) correctness for your environment and (b) a source the engineer can verify before acting.
3. An eval is a curated set of inputs with graded/reference outputs, run on every prompt or model change to detect quality regressions. Ordinary unit tests don’t work because LLM output is open-ended — there’s no single exact string to assert. The eval is the analogue of CI tests: the regression net for LLM behavior.
4. Per-call cost is pennies, so it feels free — but at fleet scale (thousands of alerts/day, each summarized) it becomes a real recurring bill plus added latency on every alert’s hot path. Judge it by unit economics: cost per unit of value (per incident triaged, per engineer-hour saved), not per call — summarizing alerts nobody reads is pure waste with extra latency.
5. LLMs remove the toil of reading and synthesizing under pressure (log-scrolling, alert correlation, runbook-hunting, postmortem-drafting). The new failure mode is that an LLM can be confidently wrong — an automated step that’s sometimes wrong is more dangerous than a slow-but-reliable manual one if it acts unsupervised. It makes production safer only as a copilot: RAG-grounded, eval-tested, prompt/version- controlled, cost/latency-budgeted, with a human owning the decision and execution.