System Prompt

A system prompt sets the role the model plays, what it is responsible for, the tone and format of its answers, the rules it follows and how it handles missing information. It comes from whoever runs the application, not from the user, and it applies to every message in the conversation.

Technically, the system prompt is a separate part of the request sent to a model's API. Anthropic calls the field system; OpenAI now refers to developer messages. How messages are structured exactly depends on the interface. The principle is similar across the major providers: the operator's instructions are kept separate from user input.

For executives, the system prompt is where company rules enter an AI assistant: tone, scope, how to handle uncertainty. That makes it part of the application's documentation, and it needs the same upkeep. It is not a security boundary.

How a system prompt works at its core.

Language models have no memory between two requests. Every request therefore contains everything the model should know: the system prompt, the conversation so far and the user's new message, often along with documents from a search or descriptions of available tools.

The sequence is always the same. The application puts the system prompt first, followed by the earlier messages and the new user message. The model generates its answer within the bounds of the instructions. With the next message the process starts over, and the system prompt is sent again.

Models are trained to give instructions in this position extra weight. OpenAI's Model Spec describes a chain of command: OpenAI's own rules come first, then developer instructions, then the user's. If a user contradicts the system prompt, the model should generally follow the system prompt.

The word "should" matters. A system prompt shifts probabilities; it does not fix a procedure. The model usually follows the instructions, but not with the reliability of a program executing a rule.

What belongs in a system prompt.

A useful system prompt answers four questions. First, role: who is the model in this application, whom does it work for and what is it responsible for? Second, format: what tone, length and structure should answers have, such as short sentences, cited sources or a table? Third, rules: what should the model do, what not, and where does it point people for questions outside its scope? Fourth, gaps: does the model ask a follow-up question, name the missing information or decline to answer?

Often there are also examples of good answers, background on the task and descriptions of the tools the model may call. Anthropic's documentation recommends using the system prompt to give the model a role and advises clear, direct instructions that also explain the reason behind a rule.

What does not belong: credentials, passwords, internal pricing logic and anything that would cause harm if published. Nor do entire manuals, price lists or document collections. An application is better off retrieving that knowledge through search when a question calls for it, for example with retrieval-augmented generation (RAG).

System prompt, user message and context.

The user message, often called the user prompt, holds the individual task: a question, a text to revise, a request. The system prompt holds whatever stays the same across tasks. That gives the most important rule of thumb: standing rules go in the system prompt, changing tasks go in the message.

Consumer chat products usually have several layers. The provider has its own system prompt that users do not see. Many products also offer fields for custom instructions or projects that work in a similar way. Since August 2024, Anthropic has published the system prompts of its Claude apps in its release notes. If you build your own application on the API, you write the system prompt yourself.

The overall context is a separate concept. A model's context window holds the system prompt, the conversation history, any documents and the answer. A long system prompt takes up room that documents or long conversations then lack.

Fine-tuning is the other way to shape behavior. It retrains the model itself on examples instead of sending instructions with every request. For most business applications, a well-maintained system prompt is enough, because it can be changed and tested in minutes.

The limit: an instruction is not a safeguard.

A system prompt tells the model what to do. It cannot force the model to comply. Cleverly worded input, known as prompt injection, can get a model to override its rules. That input does not have to come from the user. It can sit in a web page, an email or a document the application processes.

The Open Worldwide Application Security Project (OWASP) ranks prompt injection first in its top 10 risks for applications built on language models. The 2025 edition added system prompt leakage as a separate risk: the contents of a system prompt can be extracted. OWASP advises against treating the system prompt as either a secret or a security control.

What actually protects sits outside the model. Permissions determine which data and functions an application can reach, using the rights of the individual user. A person approves actions with consequences, such as a payment or an email to customers. Logs record what was asked and answered. The application manages credentials itself; the model never sees them. The system prompt complements these controls but replaces none of them.

Cost, length and caching.

Because the system prompt goes out with every request, every word counts in every call. Billing is per token, and a long system prompt raises the cost of each request. For an application with heavy traffic, that becomes a line item of its own.

The major providers offer prompt caching for this. A fixed opening section of the request, typically the system prompt, is stored for a while. According to Anthropic's documentation, reading from the cache costs one tenth of the normal input token price, while the first write costs 1.25 or 2 times the normal price depending on cache duration (as of November 2026). For caching to work, the system prompt has to stay word-for-word identical across many requests. Variable details such as the date or the user's name are better placed after the fixed part.

Length also matters for quality. Many rules in one prompt can contradict or crowd each other out. A system prompt should contain as much as necessary and as little as possible. There is no fixed limit. A few clear lines can work better than several pages nobody can keep track of.

What a good system prompt looks like.

A good system prompt is short enough for a colleague to read and understand in a few minutes. Each rule can be tested on its own. "Answer only from the provided sources and cite the source" is testable; "be helpful and accurate" is not.

It is treated like code. There is a version number, a change history and someone who decides on changes. Anyone who wants to trace an answer from the application needs to know which system prompt was in effect at the time.

Every change starts with a test against a fixed set of questions. These include typical requests, edge cases and questions the model is deliberately not supposed to answer, for instance because the answer is not in the sources. Only when answers after the change are at least as good as before does the new version go live. Switching models counts as a change too: the same prompt can behave differently on another model.

Finally, the system prompt matches the technical permissions. Whatever it forbids, the application also blocks technically where data and actions are concerned. A rule that exists only in the prompt is a request to the model.

Where the term stops working

A system prompt steers a model's behavior; it is not a technical access barrier. Sensitive data and actions need permissions and controls outside the model. The term is distinct from prompt engineering, the broader craft of writing good instructions; from the context window, whose space it shares; and from fine-tuning, which builds behavior into the model itself through retraining.