--- title: "AI pilot to production: pick the tool last – Carl-Uwe Würfel" description: "An AI pilot is ready for production once it runs without the pilot team. Plan back from the final handoff: sources, review, access. Pick the tool last." url: https://carl-wuerfel.de/en/notes/first-ai-pipelines-in-production author: Carl-Uwe Würfel --- 1. [Home](https://carl-wuerfel.de/en) 2. [Notes](https://carl-wuerfel.de/en/notes) 3. AI infrastructure # AI pilot to production: pick the tool last An AI pilot is ready for production once it runs without the pilot team. Plan back from the final handoff: sources, review, access. Pick the tool last. [Carl-Uwe Würfel](https://carl-wuerfel.de/en/about)First published February 8, 2024 · Updated November 5, 2026 · 4 min read[Deutsche Fassung](https://carl-wuerfel.de/notizen/erste-produktive-ki-pipelines) An AI pilot is ready for production once it runs without the people who tested it. When a pilot doesn't make it into daily work, the usual move is to swap the tool or wait for the next model. I think that's the wrong fix. In a pilot, a small group covers the gaps in the workflow, usually without noticing, and in production that group is gone. At our company the knowledge they used for that sat for a long time in scattered spreadsheets and individual heads. So the planning starts at the final handoff. First you settle what gets handed over, then you pick the tool. ## In a pilot, a small group fills the gaps. The pilot team knows which source is missing, which answer needs a second look and when a draft only sounds right. But none of it is written down. In day-to-day operation, colleagues who know none of this start using the workflow. Errors show up that never surfaced in the pilot, and the workflow is declared a failure when it simply was never described. That's how a [pilot graveyard](https://carl-wuerfel.de/en/glossary/pilot-graveyard) fills up with trials that went well and still never made it into daily work. Write down what the pilot team quietly makes up for before you hand the workflow to anyone else. ## The final handoff defines the workflow. An AI workflow in production needs a clear start and an end you can check. In an editorial team that might be briefing, research, draft and approval. Nothing should get lost between those steps: which sources were used, which claims are still shaky, what's already been checked. > An example from editorial work: if the AI draft comes without sources and without flags on uncertain claims, the research has to be done again, and the time saved is gone. So a usable handoff has three parts: the text, the sources and the open points. Those three parts tell you which sources the workflow may use, where the review sits and who signs off. Start with the tool and you end up building the workflow around it, only to find out at the handoff what's missing. Decide what gets handed over at the end, then choose the tool that delivers exactly that. First you settle what gets handed over, then you pick the tool. _Figure: An editorial workflow of briefing, research, draft and approval ends in a handoff of text, sources and open points. Planning starts at that final handoff, and only then is the tool chosen; if sources are missing, the research has to be done again._ ## A draft without sources invites pass-through. Text that sounds good gets picked up and forwarded easily. Without sources it can hardly be checked, and whoever sends it on anyway becomes a pass-through. AI does the groundwork here, and the judgment stays with the editors. Checking happens before anything gets passed on, in a pilot and in production alike. Build the review step into the workflow instead of leaving it to individual goodwill. ## One unchecked change turns production back into a pilot. In production a workflow keeps changing. A prompt gets tweaked, a data source is added, an interface changes. Any of these can shift results that were right yesterday. That's why prompts are part of the workflow's documentation. The business side judges whether the workflow still does its job. The technical side keeps systems and interfaces running, and both agree on changes. Document every change to prompts, sources and interfaces so someone else can follow it. ## The shared base comes before the second workflow. Since 2023 I've been moving SUMAX onto AI as a whole company, together with the team. With the second and third workflow the question changes. Now you need a shared base: users, permissions and access for everyone, security from the start, and process knowledge in one place instead of in heads and spreadsheets. That includes new role descriptions and training on real tasks from each team's own work. The pilot group should be the people who run the workflow every day. Tools get built with them, and they're the ones who carry the workflow later. Build the base before the second workflow gets its own isolated solution. ## Six things, or it's still a pilot. Before any rollout, check that each of these is in writing: 1. the purpose, in two sentences 2. the sources the workflow may use, and the ones it may not 3. the review step: who checks what before a result moves on 4. who owns the result and who owns the technology 5. the permissions and access, set up 6. the rules for changes to prompts, sources and interfaces If one is missing, the workflow is still a pilot, no matter how good the tool is. A pilot shows that something works. Production means it works without the pilot team. Take your most successful pilot, hand it to someone who wasn't involved for one week and write down every question they ask. That gives you the description the workflow is still missing. How to set up the decision about a trial beforehand is in [AI adoption: stop treating it as an experiment](https://carl-wuerfel.de/en/notes/ai-is-no-longer-an-experiment). ## Further reading * [AI roadmap for business: owners and dates](https://carl-wuerfel.de/en/ai-roadmap) * [AI automation with a human in the loop](https://carl-wuerfel.de/en/ai-automation) * [The transformation at SUMAX](https://carl-wuerfel.de/en/transformation) ## Contents 1. In a pilot, a small group fills the gaps. 2. The final handoff defines the workflow. 3. A draft without sources invites pass-through. 4. One unchecked change turns production back into a pilot. 5. The shared base comes before the second workflow. 6. Six things, or it's still a pilot. 7. Further reading **Carl-Uwe Würfel** COO & Chief AI Officer, SUMAX [Questions about this piece](https://carl-wuerfel.de/en/contact) ## Read next. * [**AI answers by language: the market in the query decides**](https://carl-wuerfel.de/en/notes/ai-visibility-by-language-market) * [**Shutting down an AI system: keeping it is a decision too**](https://carl-wuerfel.de/en/notes/when-we-switch-off-an-ai-system) * [**Running AI: catch failures, respond under control**](https://carl-wuerfel.de/en/notes/self-healing-ai-systems)