Running AI: catch failures, respond under control
You only catch AI failures if you check availability, output format and business quality separately. Responding under control means putting a limit on retries.
You only catch failures in AI operations if you watch three things separately: whether the system is reachable, whether it delivers the agreed format and whether the result is usable for the business. Responding under control means limiting retries and setting a clear way out for everything else.
A green status light only shows that a service responds. It doesn't tell you whether the specific request worked or whether the result fits. The business level is the hardest. A language model produces likely answers, not checked ones, which is why hallucinations often sound plausible. An answer in the right format can still be wrong.
Limit automatic recovery
With a temporary error, a retry or a restart often helps. If the problem keeps coming back, the automation has to stop at some point. So every call needs a timeout and a fixed number of retries. If the error persists after the allowed attempts, the process switches to a fallback that makes sense for the business or ends with a message people can understand.
Unlimited retries mask errors and run up costs. On top of that, work gets done twice and the cause stays hidden.
Unlimited retries mask errors and run up costs.
Give errors context people can act on
A good error message names the affected application and the step that failed. Four details help with the analysis: which step failed, which model was involved, which attempt it was and what it cost. Identical messages that keep repeating can be grouped, so whoever is on it sees what matters quickly.
Personal data and sensitive content should not end up in logs in plain text. Log files fall under the GDPR as soon as they contain personal data. Security has to be there from the start. A log should make errors easy to find without becoming a data protection risk itself.
Combine technical and business checks
An answer can look formally correct and still be wrong. That is why you need both, technical monitoring and a check of the quality on the business side. Whether a system may catch an error automatically or a person decides depends on the consequences. A failed internal draft can simply run again. A result that leaves the company gets checked by a person before it goes anywhere.
Responsibility for a result that gets passed on can't be handed to a model.
A retry is not a repair
An example: a data source or a model doesn't respond within the timeout. A limited retry can help here. If the disruption continues, the only option is a fallback that makes sense for the business, such as another model that handles the task just as well. If there is no such fallback, the process ends with a clear message to the person who started it.
A retry bridges a temporary disruption. It doesn't fix the cause. Timeouts, limited retries and clear escalation keep the process under control. They don't produce a system that repairs itself.
Every attempt is a new call and costs money. At SUMAX every AI call runs through our own layer between the applications and the models and is booked with its cost. Retries show up in the same records, so you can see when they pile up. A limited retry with a clear end is cheaper than any loop without one.