How to Reduce LLM API Costs Without Hurting Quality
Caching, model routing, prompt trimming and batching explained in plain English, with when to use each technique and how to cut your LLM API bill without making answers worse.
Artificial Intelligence
What is Context Window?
Quick answer
The maximum amount of text, measured in tokens, that a language model can consider at one time, including the prompt and its reply.
Context Window, explained
Every language model has a fixed limit on how many tokens it can process in a single request. That limit is the context window. It covers the system instructions, the conversation so far, any documents you attach, and the answer the model generates.
When a conversation or document exceeds the window, older or extra content has to be dropped, summarized, or retrieved selectively. For example, a support assistant reviewing a long email thread might lose track of details from the first message if the thread is larger than its window. Larger windows let models work with whole reports or codebases, but they can increase cost and response time, and models do not always use information buried in the middle of very long inputs reliably. Techniques such as retrieval-augmented generation help by sending only the most relevant passages.
Related terms
Large Language Model
An AI model trained on vast amounts of text to understand and generate human-like language, code, and other content.
Retrieval-Augmented GenerationA technique that improves AI answers by first retrieving relevant documents and then giving them to a language model as context.
Token (AI)The basic unit of text a language model reads and writes, usually a word, part of a word, or punctuation mark.
Guides that mention Context Window
Artificial Intelligence
Trending
What Is a Large Language Model and How Does It Work?
How large language models work in plain English, from tokens and training to next-word prediction, with a simple diagram and no math.
Artificial Intelligence
Trending
What Are AI Reasoning Models and When Are They Worth It?
Reasoning models think before they answer, which makes them slower and pricier. Here is how they work, what they cost and when the extra thinking pays off.