Context engineering

On every call, a language model knows two things: what it learned in training and what is in its context window. Context engineering deals with the second. That includes the system prompt, the request, documents from a search, earlier messages, notes from previous sessions, descriptions of the available tools and their results. Anthropic describes context engineering as the set of strategies for curating and maintaining the optimal set of tokens during inference, including everything that lands in the window outside the prompt itself.

The term took off in June 2025. Shopify CEO Tobi Lütke wrote on June 18 that he really liked "context engineering" better than prompt engineering. AI researcher Andrej Karpathy added his "+1" on June 25 and called it the delicate art and science of filling the context window with just the right information for the next step. The backdrop is AI agents that work in many steps, call tools and pull in documents. There, the whole information package decides the outcome, and the wording of the instruction is only one part of it.

For executives, context engineering is where quality, cost and data protection in an AI application meet. What sits in the context determines the answer. Every token in it is billed. And every piece of information in it goes to the model, often to an outside vendor. Which data an application may see is therefore a leadership question.

What goes into the context.

In a simple chat, the context is small: an instruction from the vendor and the question. In a business application, many pieces come together. The system prompt sets role, rules and output format. Then there is the request itself, hits from a knowledge base, the conversation so far, stored notes on earlier cases, examples of good results, descriptions of the tools the model may call and the results of those calls.

All pieces share the same window, measured in tokens. Every extra piece takes space from the others and lengthens the input, which is processed and billed again on every call.

The model does not, on its own, tell reliable parts from unreliable ones. An outdated price list in the context looks as credible as the current one. And what is not in the context does not exist for the model unless it appeared in training. If needed information is missing, the model often fills the gap with a plausible guess, a so-called hallucination.

Context engineering means deciding, for each task, what goes in and what stays out.

More context is not better.

Context windows have grown a lot in recent years. That doesn't mean you should fill them. Several studies show that models handle long inputs worse than short ones.

The study "Lost in the Middle" by Nelson F. Liu and colleagues (2023) showed that models use information at the beginning and end of a long input better than in the middle. In July 2025, the research team at database vendor Chroma tested 18 models and found that performance varies as input length grows, even on simple tasks. Anthropic calls the effect context rot and describes an attention budget that every additional token draws down.

Developer Drew Breunig has described typical failure patterns, which LangChain picked up: an error enters the context and keeps getting reused, too much material distracts, superfluous information confuses, and contradictory statements sit side by side.

A large window is a ceiling, then, not a target. Good applications give the model as little as possible and as much as necessary.

The core techniques.

In July 2025, the team at framework vendor LangChain sorted the techniques into four groups. Write means storing information outside the window, such as an agent's notes that it reloads later. Select means pulling in only relevant content: search hits as in RAG, the tools that fit the task, relevant memories. Compress means summarizing long histories and trimming old tool results. Isolate means handing subtasks to separate agents with their own windows that report back only a result.

Anthropic describes the same ideas for long-running tasks: compaction of the history, structured note-taking and sub-agents. It adds just-in-time retrieval. Instead of passing everything up front, the agent gets references such as file paths or links and fetches content only when it needs it. For tools, Anthropic recommends a minimal set with clearly separated functions.

Order matters too. Prompt caching only works if the beginning of the input stays exactly the same. Stable parts like the system prompt and tool descriptions go first, changing parts last. That cuts cost and latency on repeated calls.

How it differs from prompt engineering, RAG and MCP.

These terms sit on different levels and complement each other.

Prompt engineering shapes the instruction: task, role, examples, format. It asks how something is phrased. Context engineering asks what information the model gets overall, and it includes the instruction. The system prompt belongs to both.

RAG (retrieval-augmented generation) is one technique within context engineering. It searches a knowledge base for relevant passages before the answer and places them in the context. Context engineering also decides how many hits go in, how the history gets trimmed and which tools are visible.

The Model Context Protocol (MCP), which Anthropic introduced as an open standard in November 2024, is an interface. Through MCP, an application connects tools and data sources. That increases what can reach the context. What actually goes in remains a context engineering decision.

Fine-tuning sits on another level entirely. It changes the model through additional training. Context engineering leaves the model unchanged and shapes only its input.

Access rights, data protection and security.

Whatever is in the context, the model sees, and often an outside vendor with it. Three rules follow.

First, the context may only contain what the requesting person is allowed to see. That has to be enforced at retrieval, not in the instruction to the model. An instruction like "Don't reveal salaries" is no protection, because the model can ignore it or be talked into it with clever questions.

Second, personal data in the context falls under the GDPR, with a legal basis, purpose limitation and a contract with the model vendor. Anything not needed stays out.

Third, anything that enters the context from outside is a potential attack surface. An email, a web page or a tool result can contain hidden instructions that the model follows. The OWASP project ranks this prompt injection first on its list of top risks for applications built on language models (2025 edition).

Good applications log what was in the context for each call. That is the only way to trace later why an answer came out the way it did.

What good implementation looks like.

In a good implementation, the context for each task is set on purpose instead of growing by accident. For each task it is written down which sources, tools and parts of the history belong in it.

Testing uses real cases. A fixed list of tasks with known correct results serves as the test for every change, whether a new model, a different search or a shorter history. For applications with search, two things are checked separately: whether the right information was in the context and whether the model used it correctly.

Other signs: every call is logged with its cost, so you can see which pieces bloat the input. Outdated sources are excluded before they can be found. The system is allowed to say it has no relevant information. Tools are few and clearly described.

Cost depends mainly on input length times the number of calls, plus the effort for building, testing and maintaining the sources. When hiring a service provider, ask how the context is assembled, measured and logged for each task.

Where the term stops working

Context engineering improves what a model gets to see, not the model itself. Capabilities the model lacks, and knowledge that is written down nowhere, can't be added by even the best context. It differs from prompt engineering as the design of the instruction, from RAG as one technique for retrieving knowledge and from fine-tuning, which changes the model through training. Good context makes a correct result more likely, but a person still has to check it.