Every term, and where it comes from

A model run in a loop with tools and a goal, until it finishes or hits a limit.

Not a persistent being. The loop is a program somebody wrote, and it stops.

Introduced in “Agents”

Write a step, run the tool, put the result back, look again. The agency lives here, not in the model.

Introduced in “Agents”

The broad term for shaping a model to behave as intended. Mostly happens in post-training.

Introduced in “Post-training”

The HTTP endpoint that takes an assembled context and returns tokens.

Introduced in “Interfaces”

The operation where each token weights every earlier token and pulls in a blend of them.

Not focus or intent. It is a similarity score, computed and applied.

Introduced in “Attention”

A complete record of every action taken, including the ones that were blocked.

Introduced in “Guardrails and permissions”

Producing one token at a time, each conditioned on everything already written.

Introduced in “The forward pass”

Working backwards through the layers to compute how much each parameter contributed to the error.

Introduced in “Training”

A point where everything waits for everything else. Justified only when a stage needs all prior results at once.

Introduced in “Orchestration”

How much damage one wrong action can do. Guardrails shrink this; they do not make the model reliable.

Introduced in “Guardrails and permissions”

The method that builds the vocabulary: start from bytes, repeatedly merge the most frequent adjacent pair, stop at the target size.

Not a compression format, though it began as one.

Introduced in “Tokens”

Tokens a model produces working through a problem before giving its answer. Ordinary output, produced by the ordinary loop.

Not a window into the computation. It is text the model generated, and it can be wrong while the answer is right, or the reverse.

Introduced in “Reasoning”

The special tokens a given model family uses to mark role boundaries. Using the wrong one degrades output quietly.

Introduced in “The prompt and its roles”

A saved copy of the weights at some point during training.

Introduced in “The weights”

Cutting documents into passages small enough to retrieve and large enough to still mean something.

Introduced in “Retrieval”

Replacing a long history with a short summary mid-run so the window can keep going.

Introduced in “Context sharding”

The fixed token allowance for one turn, and the competing claims on it.

Introduced in “Context engineering”

The single sequence of tokens the model is given each turn, and its maximum size.

Not memory. It is rebuilt from scratch by a program on every turn.

Introduced in “The context window”

The collected text a model is trained on, after filtering and deduplication.

Not a database the model can consult later. It is gone by the time the model runs.

Introduced in “The corpus”

How alike two vectors are, measured by the angle between them rather than the distance.

Introduced in “Embeddings”

Writing the answer, one token at a time. Sequential, and the reason output costs more.

Introduced in “Cost and latency”

Removing repeated and near-repeated documents, typically discarding 20-50% of raw crawl.

Introduced in “The corpus”

One of the thousands of numbers in a vector. Individually they mean nothing readable.

Introduced in “Embeddings”

A list of numbers standing for a token or passage, positioned so similar things sit near each other.

A token’s input embedding is fixed; its hidden state at a later layer is not. Both get called “the embedding”.

Introduced in “Embeddings”

A capability that is absent at one scale and present at a larger one, without appearing gradually in between.

Not magic, and not well explained. Partly an artifact of measuring with pass-or-fail tests.

Introduced in “Scale”

A fixed set of cases with known good outcomes, run on every change rather than every release.

Introduced in “Evaluation and failure”

Removing something from the context to make room. The model is never told what went.

Introduced in “The context window”

A confident, plausible, false output. Often called hallucination.

Not a malfunction. Producing the most plausible continuation is exactly what the model was trained to do.

Introduced in “Evaluation and failure”

Running independent work side by side. Costs N times the tokens for roughly the time of the slowest.

Introduced in “Orchestration”

Training an existing model further, producing a new file or a small companion adapter.

Does not edit the original weights, and is not how a chat appears to remember you.

Introduced in “The weights”

One run of the token sequence through every layer, producing one probability distribution.

Introduced in “The forward pass”

A small block of machine-readable metadata at the top of an otherwise human-readable file.

Introduced in “Markdown and the plain-text interface”

The model writing a structured request to run a tool, which your code then decides whether to honour.

The model never runs anything itself.

Introduced in “Tools and function calling”

The fact that one model trained on one objective can do many unrelated tasks it was never specifically taught.

Not evidence of understanding. It is what predicting text well enough turns out to require.

Introduced in “Scale”

Nudging every parameter in the direction that would have reduced the loss, over and over.

Introduced in “Training”

Holds relationships and answers what connects to what.

Introduced in “The stores”

Always taking the single most likely token. Repetitive, and still not truly deterministic in production.

Introduced in “Sampling and temperature”

One parallel attention computation. Different heads specialise, and the specialisation is learned.

Introduced in “Attention”

Running the finished file over some text to produce more text. The second of the two halves.

Not learning. Nothing is stored and nothing changes during inference.

Introduced in “What we mean by "AI" here”

A plain text file of standing rules, pasted into the context on every turn.

Nothing parses it. Its contents arrive as text like anything else.

Introduced in “Markdown and the plain-text interface”

The usual format for describing a tool. Its description field is the only documentation the model gets.

Introduced in “Tools and function calling”

Generating several candidate answers and scoring them independently, to catch errors one confident run would not.

Introduced in “Orchestration”

Returns a thing by its name, very fast. Suits sessions, caches and saved summaries.

Introduced in “The stores”

The date the training data was collected. Nothing after it exists to the model.

Introduced in “The corpus”

Stored intermediate values for tokens already processed, so each new token avoids recomputing them.

Introduced in “Cost and latency”

A file of numbers that, given a sequence of tokens, produces a probability for every token that could come next.

Not a program with rules in it, and not a search over stored documents.

Introduced in “What we mean by "AI" here”

One stage of the stack: attention across positions, then a transform of each position.

Introduced in “Parameters”

How large each nudge is. Too small and it crawls; too large and it diverges.

Introduced in “Training”

Giving each tool the narrowest permission that still lets it work.

Introduced in “Guardrails and permissions”

The raw scores over the whole vocabulary, before they are turned into probabilities.

Introduced in “The forward pass”

One number measuring how wrong a prediction was. The entire training process minimises it.

Introduced in “Training”

The measured tendency to use information at the start and end of a long context more reliably than the middle.

Introduced in “The context window”

Plain text with a few visible conventions for headings, lists and code, readable by both a person and a model.

Introduced in “Markdown and the plain-text interface”

A shared convention for exposing tools and data, so any client can connect to any provider without bespoke wiring.

Introduced in “Interfaces”

In this site, always the file of weights. Not the chat product, not the company, not the API.

People say "the model" for the whole product. Almost every misconception starts there.

Introduced in “What we mean by "AI" here”

Using a second model to grade output that resists automatic checking. Good at comparisons, poor at absolute scores.

Introduced in “Evaluation and failure”

One adjustable number inside the model. Training changes these and nothing else.

Not a fact or a rule. No single parameter holds any one thing.

Introduced in “Parameters”

Anything that survives between runs. All of it lives outside the model, in a file or a store.

Introduced in “Memory”

Items flowing through stages independently, so nothing waits for the slowest item at each step.

Introduced in “Orchestration”

Adjusting the model toward responses humans ranked higher. RLHF is one method of doing it.

Introduced in “Post-training”

Reading the prompt. Happens in one parallel pass, and sets the time to the first word.

Introduced in “Cost and latency”

The long first training run, where the only objective is predicting the next token.

Introduced in “The corpus”

Reusing an unchanged prefix across requests at a discount. Why stable content belongs first.

Introduced in “Cost and latency”

Text arriving from a document, page or tool result that the model follows as if it were an instruction.

Not a bug waiting to be patched. It follows from everything being one string, and it is handled with permissions.

Introduced in “The prompt and its roles”

Attention work grows with the square of the input length, so doubling the input roughly quadruples it.

Introduced in “Attention”

Storing parameters at lower precision to shrink the file and speed it up, at some cost to quality.

Introduced in “Parameters”

The three projections of each token: what it is looking for, what it offers, and what it passes on.

Introduced in “Attention”

Retrieval-augmented generation: fetching relevant passages and pasting them in before the model answers.

Introduced in “Retrieval”

Reordering the top candidates with a slower, more accurate model before they go in the window.

Introduced in “Retrieval”

Declining a request. A trained behaviour, not a hard rule the machine enforces.

Introduced in “Post-training”

A change that fixes one class of failure and breaks another. The reason a single aggregate score hides things.

Introduced in “Evaluation and failure”

Structured records, queried exactly. Answers how many and which, not what resembles this.

Introduced in “The stores”

A label marking who a piece of the context came from: system, user, assistant, or a tool.

Introduced in “The prompt and its roles”

The observed relationship between how much data, compute and parameters go into a training run and how well the result predicts text. Smooth, and predictive of loss.

They predict the loss, not what the model will be able to do.

Introduced in “Scale”

A library wrapping that endpoint in your language.

Introduced in “Interfaces”

Splitting work so no single context has to hold all of it.

Not parallel thinking. Each shard is a separate run that cannot see the others.

Introduced in “Context sharding”

The step that turns raw scores into probabilities summing to one.

Introduced in “The forward pass”

A saved fact that is out of date being recalled with exactly the same confidence as a current one.

Introduced in “Memory”

Holding no memory between calls. The model is stateless; the program around it is not.

Introduced in “The weights”

What ends the loop: a declaration of success, an iteration cap, a spend cap, or a failure.

Introduced in “Agents”

Sending each token to the screen as it is chosen, which is why an answer appears to type itself out.

Introduced in “Interfaces”

A separate run with its own clean window and a narrow brief, returning a summary to a coordinator.

Introduced in “Context sharding”

Replacing completed work with a short account of it to free room. Lossy, and the loss is silent.

Introduced in “Context engineering”

Marking a saved fact as replaced rather than overwriting it silently, so the change is visible.

Introduced in “Memory”

Training on examples of requests and good responses, which teaches the assistant format.

Introduced in “Post-training”

Standing instructions placed at the front of the context on every turn.

Not a setting the machine enforces. It is text the model was trained to weight heavily.

Introduced in “The prompt and its roles”

How much the odds are flattened before a token is drawn. Low concentrates, high spreads.

Not a creativity dial. It reshapes fixed odds; it does not change what the model considered.

Introduced in “Sampling and temperature”

Spending more work at the moment of answering rather than during training, usually by generating more tokens before the reply.

Not a different mechanism. It is the same loop, run for longer.

Introduced in “Reasoning”

A chunk of characters with a fixed ID. It is the unit a model actually reads, and the unit you are billed in.

Not a word and not a syllable.

Introduced in “Tokens”

The program that cuts text into tokens and back again, using a vocabulary fixed when the model was built.

Introduced in “Tokens”

An ordinary function your program exposes to the model by describing it in the context.

Introduced in “Tools and function calling”

Whatever your code returned, pasted back into the context as plain text. Also where untrusted content enters.

Introduced in “Tools and function calling”

Keeping only the k most likely candidates before drawing one.

Introduced in “Sampling and temperature”

Keeping the smallest set of candidates whose probabilities sum to p. Also called nucleus sampling.

Introduced in “Sampling and temperature”

The one-off process that produced the file, by adjusting its numbers until it predicted text well.

Not something that happens while you talk to it. It finished before you arrived.

Introduced in “What we mean by "AI" here”

The architecture almost every current language model uses: stacked attention and feed-forward layers.

Introduced in “Parameters”

An ordered list of numbers treated as a coordinate in a space of many dimensions.

Introduced in “Embeddings”

Stores embeddings and answers what is most similar to this. The only one of the four specific to AI.

Introduced in “The stores”

Finding stored passages nearest to a question in embedding space.

Misses exact terms like names and error codes, which is why production systems add keyword search.

Introduced in “Retrieval”

The complete fixed set of tokens a model knows, each with an ID. It never changes after training.

Introduced in “Tokens”

The frozen parameters written to disk. This file is the model.

Introduced in “The weights”