AI infrastructure: a pile of tools is not a system
AI infrastructure starts with users, permissions and access for everyone, then tools. Your own layer keeps models swappable and books the cost of every call.
AI infrastructure starts with the foundation for everyone, meaning users, permissions and access, and only then with tools. Most companies do it the other way around. Every department signs up for whatever tool looks good for its task, and a year later the company has a pile of tools and no system. Each of those isolated solutions comes with its own credentials, its own costs and its own security gaps, and nobody adds them up. That's why at SUMAX we're building our own layer between our applications and the AI models. It lets us switch models, and every call gets booked with its cost.
A pile of tools is not a system.
Since 2023 I've been moving SUMAX, an online marketing agency in Dortmund, Germany, onto AI together with the team. Before that, our workflows lived largely in spreadsheets and individual people's heads. When you start from there, you quickly build a small separate solution for every new task. That works for a while, but at some point the same work starts repeating all over the place.
When several applications pull from the same data source or the same model, they don't each need their own connection. Every separate connection brings its own credentials, its own error handling and its own costs. A shared layer puts permissions, model choice and cost tracking in one place. You get a system when every new application plugs into the same layer instead of rebuilding the same connection. Count how many times your company has already built the same connection. Every repeat is a signal for a shared layer.
You get a system when every new application plugs into the same layer.
The foundation applies to everyone, or it doesn't apply.
Technology brings no order as long as nobody has settled who uses which application, with which permissions and through which access. That question comes first. Security and data protection are part of it from day one, because retrofitting them into a patchwork of isolated tools is hard work. The EU AI Act entered into force on August 1. What it means for each application can only be answered by someone who knows which AI runs where and on which data.
A tool connected around the foundation punches a hole in every rule. Don't let any AI application go live without the shared users, permissions and access.
Process knowledge belongs in the infrastructure.
Whatever used to sit in heads and spreadsheets has to live where people and applications can find it. Without that, every tool guesses in its own way, and each one guesses differently. Only then do tools for individual tasks make sense. Ours get built with the people who will use them, because they know best where the routine work is. AI should take that off their hands. The people stay.
The same rule applies to every tool on the shared layer: whoever passes on an AI result is responsible for it. No infrastructure changes that. Get a workflow's knowledge into one place before you give it a tool.
Tie yourself to one model and you redo the work at every switch.
New models come out at short intervals. This year alone brought GPT-4o in May and Claude 3.5 Sonnet in June, and both times the question of which model fits which task opened up again. If every application is hard-wired to one provider, every switch means rework everywhere.
With a shared layer you can switch the model without touching every application. Whether the models run in the provider's cloud or on your own hardware makes little difference, as long as the interface to the applications stays stable. Connect every application to the layer in front of the model.
Only per-call tracking tells you what AI infrastructure costs.
There's no lump-sum answer to what AI infrastructure costs. There is an exact answer per task and per application, once every call is booked with its cost. Then, before any new tool is added, you can see which business task justifies its usage. Without that tracking, leadership sees one total per vendor and guesses where the money goes. Book every call with its cost and its application from day one.
A shared layer is also a shared point of failure.
The price of bundling is a central dependency. If the layer goes down, every connected application stops. So it has to be clear at all times which applications depend on it.
A layer like this gets built step by step. Start with the connection that repeats most often. For each connected application, write down what it's for, what input it needs, what it produces and where it can fail. You'll need that description at the latest when you first change the layer, because a change there hits everything. Every further step of bundling has to earn its place, and what only one application needs stays with that application.
Five questions before your next AI tool.
- Do other applications already use the same data source or the same model?
- Who gets access, with which permissions, and can you revoke it in one place?
- Can you switch the model without rebuilding the application?
- Is every call tracked with its cost and its application?
- Which applications stop if the shared layer goes down, and does anyone know?
If the answer to the first question is yes, the tool belongs on the shared layer.
A company needs a system for AI, and you only get one if the foundation comes before the next tool. Make a list of every AI tool in the building, note who has access to each one and who sees its costs, and start with the gaps. How a single workflow gets ready for production first is in AI pilot to production: pick the tool last.