Who Carries the Cost When an Automation Runs Out of Control

Almost no automation proposal says who carries the cost when an automation runs out of control. The question gets asked reliably after the first incident. Asked before you sign, it takes twenty minutes and costs nothing.

There is a reason it comes up so late. Most people picture a failure as a standstill. Something stops, someone notices, someone repairs it. The expensive case is the opposite. The automation keeps working, faster than intended, and nobody sees it, because nobody looks as long as nothing shows up in red.

How two harmless processes add up to an expensive one

Suppose a company runs two automations. The first reacts to every change in the customer system and writes the current state into a shared table so that sales can follow along. The second watches that table and writes changes back into the customer system so that nothing gets lost when someone corrects a row by hand.

Each one on its own is unremarkable. Together they form a circle. As soon as the first one writes into the table automatically, it wakes the second. The second writes back and wakes the first again. From that moment the process no longer needs a human. It runs until something stops it, and in many environments the only thing that stops it is an exhausted quota.

This pattern is not the result of sloppy work. It is the result of time. The second automation is usually built months after the first, often by a different person, for a purpose unrelated to the first one. The circle is nowhere visible, because nobody ever drew it.

Why testing does not catch it

Each automation is tested on its own. You change a record, watch it arrive, tick the box. A circle, however, needs two active participants and an automatic write as its trigger. The two are rarely present at the same time during a test.

Then there is the clock. If something runs unusually often during the working week, somebody eventually notices. If it starts on a Friday evening, the circle works undisturbed for two days.

Where the real damage sits

The consumed operations are the visible part and usually the smaller one. What costs more is what the circle leaves behind in your systems.

Records overwritten hundreds of times, whose last valid state nobody can prove any more. A change history that is useless for the affected period. In the worse case, mails or order confirmations that actually reached customers. And a blocked account that takes every other automation down with it, including the ones that carry daily operations. Cleaning that up almost always takes longer than the incident itself.

What the proposal should contain

Four points belong in every statement of work, and all four are cheap to build when they are planned from the start.

First, an upper limit. Every process gets a number of runs per hour or per day, beyond which it stops itself and reports instead of continuing. That is the emergency brake, and it matters more than the other three.

Second, origin detection. An automation has to be able to tell that the change it is reacting to was made by another automation rather than by a person. Technically that is a marker on the record. In business terms it is the rule that breaks the circle.

Third, a stop switch your own people can operate. Someone inside your company must be able to halt everything without calling the supplier. If the off switch sits only with the provider, your operations depend on the provider being reachable.

Fourth, alerts addressed to a named person rather than a shared mailbox. An alert nobody owns is not an alert.

Who is liable when it happens anyway

This is worth reading in the contract before signing, and three questions cover it.

What counts as accepted? If acceptance only covers the normal case, everything else becomes goodwill later. Misbehaviour under load belongs in the acceptance criteria, not in a verbal assurance.

Who pays for the rework? On a time and materials basis you pay for the defect and for fixing it. That is the quiet disadvantage of the model and it is rarely said out loud. Under a fixed price with an agreed scope, the estimating risk sits with the supplier, which is where it belongs.

Who covers third party cost? Fees for the platform your automation runs on are your fees, even when a construction fault caused them. If you want a different arrangement, it has to be written into the contract.

The question to ask in the sales meeting

Ask what happens if this process starts ten thousand times tonight. The answer separates suppliers more reliably than any reference list.

Anyone who replies by explaining how the process stops itself and who it notifies has cleaned up such a mess before. Anyone who replies that this cannot happen has not thought the case through. It can always happen, because the second automation that closes the circle is often built a year later.

Sophera Consulting puts these limits into the analysis before anything is built: which processes can trigger each other, where the ceiling sits, who operates the stop switch, and who receives the alert. It is written into the statement of work and included in the fixed price, with no subscription. The entry point is the free automation check.

The recommendation

Do not treat the emergency brake as a technical detail the supplier will handle correctly on its own. Treat it as part of the order. Insist on three things in writing: a ceiling per process, a stop switch inside your own company, and a named recipient for fault reports.

If that costs extra, it is still the cheapest line in the whole proposal. And if a supplier refuses to put it in writing, you have learned something worth more than the price comparison.