AI model glossary

Plain-language definitions for the terms used across this directory. Each entry answers the question in its first sentence.

Token

A token is the unit a language model reads and writes in, roughly three to four characters of English text or about three-quarters of a word.

Providers price and limit models by token count rather than by word or character count, because tokens map directly to the model's internal vocabulary.

A short word like "cat" is usually one token; a rare word or a piece of code can split into several.

Context window

The context window is the maximum number of tokens a model can hold in a single request, counting the prompt, any attached files, and the model's own reply combined.

Exceed it and the request fails or gets truncated, not gracefully degraded.

A larger context window lets a model read a longer document or hold a longer conversation in one call, but it does not by itself make the model more accurate at any given task.

Output token limit

The output token limit is the maximum number of tokens a model can generate in a single response, separate from and usually much smaller than its context window.

A model with a 200,000-token context window might still cap a single reply at 8,000 or 64,000 tokens.

Hitting the limit mid-answer truncates the response; the caller has to request a continuation.

Model ID vs alias vs snapshot

A model ID is the exact string an API call sends to select a model, and it is the only identifier this directory treats as authoritative.

An alias is a rolling name, such as "latest" or a bare family name, that a provider silently repoints to a newer snapshot over time, so the same alias can mean a different model next month.

A snapshot is a dated, frozen model ID, such as one carrying a `-2025-06-20`-style suffix, that always resolves to the exact same weights.

Pin to a snapshot ID in production if behavior stability matters more than always running the newest version.

Lifecycle states

Lifecycle state is a model's position in its provider's own retirement pipeline: active, legacy, deprecated, or retired.

Active means the provider recommends it for new use. Legacy means it still works but a newer model in the family is now recommended instead.

Deprecated means the provider has published a sunset date after which the model ID will stop serving requests. Retired means that date has passed and the model ID no longer works.

Zero data retention

Zero data retention (ZDR) is a provider setting or contract term under which API request and response content is not stored after the response is returned, beyond what is technically required to serve the call.

It typically has to be requested or enabled explicitly, and eligibility can depend on account type, region, or a signed agreement.

It is distinct from "does not train on your data": a provider can retain data for abuse monitoring while still not training on it, or vice versa.

Data residency

Data residency is a guarantee about which geographic region a provider processes and stores a customer's request data in.

It matters for regulatory regimes, such as EU data protection rules, that restrict moving certain data outside a jurisdiction.

Not every provider offers a choice of region for every model or plan tier; where residency is unpublished, this directory records it as not published rather than guessing.

Sub-processor

A sub-processor is a third-party vendor a provider relies on to help deliver its service, such as a cloud infrastructure host, that may also handle customer data in the process.

Providers that handle regulated data are generally expected to publish a list of their sub-processors and notify customers of changes.

A model built on top of another company's infrastructure, such as a model hosted on a hyperscaler's cloud, inherits that hyperscaler as a sub-processor for governance purposes.

Prompt caching

Prompt caching is a provider feature that stores a previously sent portion of a prompt, such as a long system instruction or reference document, so a later request reusing that same portion is billed and processed faster.

It reduces both cost and latency for repeated large-context calls, typically at a fraction of the normal input token price for the cached portion.

Cache lifetimes are short, usually minutes, and a cache miss simply falls back to full-price processing.

Batch API

A batch API is an asynchronous request mode that accepts a large set of prompts at once and returns results within a window, commonly 24 hours, instead of a single synchronous reply.

Providers price batch requests at a discount, often half the price of the equivalent synchronous call, because the workload can be scheduled during spare capacity.

It suits large offline jobs, such as bulk classification, where an immediate response is not required.

Open weights vs open source

Open weights means a model's trained parameter files are published for anyone to download and run, regardless of what license governs their use.

Open source, applied strictly, requires not just the weights but also a license meeting an open-source definition and, ideally, the training data and code needed to reproduce the model.

Most models marketed as "open source" today are open weights under a permissive or partially restrictive license; this directory records the license terms rather than the marketing label.

Quantization

Quantization is a technique that reduces the numerical precision used to store a model's weights, for example from 16-bit to 8-bit or 4-bit numbers.

It shrinks the model's memory footprint and speeds up inference, at some cost to output quality that grows as precision drops further.

Quantized versions of an open-weights model are what let it run on a single consumer GPU instead of a data-center cluster.

Distillation

Distillation is a training technique where a smaller "student" model is trained to reproduce the outputs of a larger "teacher" model, rather than being trained from scratch on raw data alone.

It lets a provider ship a fast, cheap model that retains much of the larger model's behavior on common tasks.

A distilled model typically underperforms its teacher on tasks far outside the teacher's demonstrated outputs, since it never saw the teacher's underlying training process directly.