AI automation: a checkbox is not a sign-off.
AI automation should get only as much leeway as you can check.
The usual setups sit at two extremes: an agent that does everything on its own, or a human in the loop who signs off on every result until it is just a checkbox. With the first, nobody catches the error, and with the second, people become rubber stamps who forward what they never read. At SUMAX a person signs off on whatever leaves the building, and whatever stays in-house and is easy to check runs without a sign-off per result.
Before I approve an automated workflow, it has to be clear where a person checks the result and who switches it off when something goes wrong. Which model it runs on, I decide last.
Run it, approve it or leave it?
Think of a workflow you want AI to handle and answer six questions about it. You get my assessment, based on the same rules I use in our company, and what each station has to do for this workflow.
- AccessWho is asking, and is this role allowed to?
- CacheIs there a valid result already?
- Model routingWhich model is enough for this task?
- ReviewDoes the result follow the rules?
- Cost ledgerWhat did it cost, and who is it booked to?
- ApprovalWho signs off before it leaves the building?
Done before launch
- Test cases with simple, ambiguous and deliberately hard examples.
- The way back to doing it by hand is ready.
- A named person who is allowed to switch it off.
Decisions about people can trigger additional obligations under the EU AI Act. The EU AI Act for mid-sized companies
The rules behind it
- Conventional software usually handles fixed data formats more reliably. AI pays off once language is ambiguous or documents keep changing.
- Nothing goes live without a named person who owns the workflow and is allowed to switch it off.
- If a result can hardly be checked and an error can get expensive, the decision stays with the team that does the work.
- Anything that leaves the company, is hard to check or can cause expensive errors gets approved by a person before it moves on.
- Anything that stays inside, is easy to check and cannot cause expensive errors runs without approval for every single result.
This is how I decide in our company. It is general guidance, not legal advice and not a review of your specific case. Your answers stay in your browser, nothing is sent.
If fixed rules can do it, you don't need AI.
TakeawayThe more predictable the task, the less leeway the automation gets.
Conventional software usually handles fixed data formats like spreadsheets or form fields more reliably, whether it's a custom program or RPA. AI pays off when language is ambiguous or documents keep changing. A language model (LLM) judges and writes, so it needs checkpoints. Anthropic separates workflows, where the process sets the steps, from agents, which pick their own path and tools. Every bit of freedom raises the bar for permissions, monitoring and stop conditions. An agent fits where the path to a solution is open and a mistake is cheap. Where the steps are known, the workflow sets the order.
Nothing leaves the building without a person's sign-off.
TakeawayBefore you build, settle which steps run automatically and where a person decides.
A content workflow shows the pattern. The brief sets the question, research supplies the sources, a model writes the draft. In the expert review a person checks the claims and the fit for the audience, and only then does a person sign off. Content for healthcare clients gets one more check before sign-off, against German health advertising law, the Heilmittelwerbegesetz. Anything that breaks a rule goes back to the editor, even if the rest reads well. With this workflow, SUMAX now produces several times as many expert articles per editorial hour as in 2024.
Before anything leaves the building, a person signs off.
Sign off on everything and you get checkboxes instead of checks.
TakeawayPut sign-offs where an error leaves the building or gets expensive, and let checkable in-house work run without one.
A sign-off on every result sounds safe. But when the same kind of draft shows up every day, the sign-off becomes a click, and the person doing it becomes a rubber stamp. So sign-offs go where they decide something. Anything that leaves the building, is hard to check or can cause expensive errors gets approved by a person. Anything that stays in-house and can be checked against a source or a rule runs without approval per result, as long as an error gets caught in the next step. If a result can hardly be checked and an error can get expensive, I don't automate it. The decision stays with the team that does the work, and AI does the groundwork, like sorting or summarizing documents.
The model gets picked last.
TakeawayPick the model only after the test cases, and rerun them with every new model version.
The first question is often which model is best. I ask it last. First come test cases with simple examples, ambiguous inputs and deliberately hard ones. Every result is measured against fixed criteria: factually correct, complete, traceable, in the right format. Then the model is chosen for the task, the smallest one that delivers the quality needed. A second AI pass catches some hallucinations, but it doesn't replace an independent fact check or a spot check by a person. When a new model version comes out, the same test cases run again.
No off switch, no launch.
TakeawayDon't launch a workflow until the fallback and the off switch are in place.
Cost comes from model calls, data access, retries, support and the time spent reviewing, so there is no flat figure. At SUMAX tens of thousands of AI calls a month run through our own layer between applications and models. It books every call and its cost to an application and keeps the model switchable. If the review eats up the benefit, the workflow isn't worth it. Before launch you also need a log of errors and changes, a ready way back to doing the work by hand, and a named person who can switch the workflow off without going through IT.
From practice.
Frequently asked questions
What is the difference between RPA and AI automation?
RPA runs defined steps by fixed rules, for example with spreadsheets or form fields. AI automation also handles free text and documents that change, which is why it needs checkpoints. Where fixed rules are enough, RPA or a custom program is usually the more reliable choice.
Which processes can you automate with AI?
Frequent, clearly bounded tasks with ambiguous inputs, accessible data and a result you can check against a source or a rule. If the result stays in-house and an error gets caught in the next step, the workflow can run without approving each result. If a result can hardly be checked and an error can get expensive, the decision stays with the team, and AI only does the groundwork.
Does every AI result need human approval?
No. Anything that leaves the company, is hard to check or can cause expensive errors gets approved by a person. Anything that stays in-house and is easy to check runs without approval per result, otherwise the approval turns into a checkbox. The owner sees errors and costs and can switch the workflow off.
Is AI automation a good fit for small and mid-sized companies?
Yes, once the foundation is in place: users, permissions and access are settled once for everyone before tools for single tasks arrive. Then no tool has to answer those questions again. A workflow that rarely runs often can't carry the effort for testing, review and support.
How much does AI automation cost?
There is no flat figure. The cost is made up of model calls, data access, retries, support and review time. If you book every call and its cost to the workflow and put the review time next to it, you can see whether it pays for itself.
Who can switch off an AI automation?
The person who owns the workflow on the business side. They shouldn't have to go through IT to do it, and the fallback to manual work has to be ready. Until that person is named, the workflow doesn't go live.