What Is a Large Language Model and How Does It Work?

You type a question into a chat assistant, and a few seconds later you get a fluent, well-organized answer. It feels like talking to someone who has read everything. Then it invents a statistic, or cites a policy your company never had.

Both experiences come from the same machine: a large language model, or LLM. Once you understand the 3 basic ideas behind it (tokens, training and prediction), its strengths and its odd failures both start to make sense. This guide walks through each one without any math.

What a large language model is in one paragraph

A large language model is a type of AI trained on enormous amounts of text to do one job: predict what text should come next. Give it “The capital of France is” and it predicts “Paris.” Give it a customer complaint and an instruction to reply politely, and it predicts, one piece at a time, the words a helpful reply would contain.

“Large” refers to 2 things: the volume of training text and the size of the model itself, which can contain billions of internal settings called parameters. “Language model” means it models patterns in language, such as which words tend to follow which, in what context and in what style.

LLMs are a form of deep learning. If those terms are new, our explainers on what artificial intelligence is and machine learning vs deep learning give you the background in a few minutes.

Step 1: Text becomes tokens

LLMs do not read letters or whole words the way you do. They break text into tokens, which are common chunks of characters. A short word might be a single token, while a long or rare word might be split into several.

OpenAI’s rule of thumb for English is that 1 token is about 4 characters, or roughly three-quarters of a word, so 100 tokens is about 75 words. Other languages and other models split text differently.

Tokens matter to you for 2 practical reasons:

• Pricing. Most LLM APIs charge per token, counting both the text you send and the text the model writes back.
• Limits. Every model has a context window, the maximum number of tokens it can consider at once. Your instructions, any documents you paste in, the conversation so far and the answer all have to fit inside it.

Step 2: The model learns from huge amounts of text

Training happens in stages. The first, called pretraining, is where the model picks up language itself.

During pretraining, the model sees a massive collection of text and plays a fill-in-the-next-token game billions of times. It reads a passage, guesses the next token, checks the real answer and adjusts its internal parameters slightly to make a better guess next time. Repeat that enough and the model absorbs grammar, facts that appear often, writing styles, common reasoning patterns and even some coding ability.

Most modern LLMs use a neural network design called the transformer, introduced in the 2017 research paper “Attention Is All You Need” by Ashish Vaswani and colleagues. Its key idea, attention, lets the model weigh how much each token in the input relates to every other token. That is how it can tell whether “it” in a sentence refers to the invoice or the customer. For how the underlying network learns, see our guide to what a neural network is.

Fine-tuning turns a text predictor into an assistant

A freshly pretrained model is good at continuing text but not at following instructions. So developers add further training:

• Instruction tuning. The model studies examples of requests paired with good responses.
• Learning from human feedback. People rank several model answers from best to worst, and the model is trained to prefer the higher-ranked style. OpenAI’s 2022 InstructGPT paper described this approach, called reinforcement learning from human feedback (RLHF), and reported that labelers preferred the outputs of a much smaller tuned model over a far larger untuned one.

This stage is why chat assistants answer questions, refuse some requests and keep a consistent tone.

Step 3: The model predicts, one token at a time

When you send a prompt, the model does not search a database for an answer. It runs your tokens through its network and produces a list of possible next tokens, each with a probability. It picks one, adds it to the text and repeats the whole process for the next token, until the answer is complete.

Here is the full loop in a simple diagram:

Your prompt | v [ 1. Tokenize ] "Summarize this invoice" -> [Sum][mar][ize][ this][ invoice] | v [ 2. Process ] The trained network weighs every token against every other | v [ 3. Predict ] Most likely next token: "The" (62%) "This" (21%) ... | v [ 4. Append ] Add the chosen token to the text | +-----> repeat steps 2 to 4 until the answer is finished | v Answer shown to you

The percentages above are illustrative only. Settings such as “temperature” control how often the model picks a less likely token. Low temperature makes answers more predictable; higher temperature makes them more varied and creative.

Key takeaway: An LLM writes by predicting the most plausible next token, over and over. It is optimized to sound right, not to be right, which is why fluent answers still need fact-checking.

What LLMs are good at and where they fall short

The most important weakness is hallucination: a confident answer that is simply false, such as a made-up citation. Our guide to AI hallucinations explains why it happens and the checks that catch it.

How businesses connect LLMs to their own information

Because an LLM only knows what was in its training data, businesses usually extend it in one of 3 ways:

Model size is also a choice. Smaller models can run on a laptop or phone, which helps with privacy and cost. Our guide to small language models and on-device AI covers when a smaller model is enough.

Where to start using an LLM at work

1. Pick a writing or reading task you repeat weekly, such as summarizing meeting notes or drafting first replies to common emails.
2. Use a business-grade account with clear data controls, and check whether your inputs are used for training.
3. Write specific prompts. Say who the audience is, what format you want, what to include and what to avoid.
4. Give it the facts. Paste in the source material instead of asking the model to remember it.
5. Review everything before it leaves the building. Check names, numbers, dates and claims against the source.
6. Track the time saved for a month, then decide whether to expand to more tasks or teams.

The bottom line

A large language model is a giant, trained text predictor. Tokens are its alphabet, pretraining gives it broad knowledge of language, fine-tuning makes it helpful and token-by-token prediction produces every answer you see. That design makes LLMs excellent at drafting, summarizing and rewriting, and unreliable as a source of facts on their own. Give them good inputs, keep a person reviewing the output and they become one of the most useful tools a business can adopt.

Sources

• arXiv: Attention Is All You Need (Vaswani et al., 2017) — the transformer architecture behind most LLMs
• arXiv: Training language models to follow instructions with human feedback (Ouyang et al., 2022) — instruction tuning and RLHF
• OpenAI Help Center: What are tokens and how to count them? — token-to-word rule of thumb for English
• NIST AI 600-1: Generative AI Profile — risks of generative AI, including confabulation