Artificial Intelligence

What is Large Language Model?

Quick answer

An AI model trained on vast amounts of text to understand and generate human-like language, code, and other content.

Large Language Model, explained

Large language models (LLMs) are typically built on the transformer architecture and trained to predict the next token, a word or word fragment, across huge text datasets. Through this simple objective at enormous scale, they learn grammar, facts, reasoning patterns, and styles of writing.

After initial training, LLMs are usually refined with techniques such as instruction tuning and reinforcement learning from human feedback so they follow requests helpfully and safely. They power chat assistants, writing tools, coding assistants, search features, and AI agents. LLMs have important limitations: they can hallucinate, reflect biases in their training data, and have a knowledge cutoff. Techniques like retrieval-augmented generation and tool use help address some of these weaknesses.

Related terms

Chain-of-Thought Prompting

A prompting technique that asks a language model to work through a problem step by step before giving its final answer.

Chatbot

A software program that holds a conversation with people through text or voice, answering questions or helping complete simple tasks.

Context Window

The maximum amount of text, measured in tokens, that a language model can consider at one time, including the prompt and its reply.

Foundation Model

A large AI model trained on broad data that can be adapted to many tasks, such as writing, coding, or image analysis.

Generative AI

AI systems that create new content, such as text, images, audio, video, or code, based on patterns learned from training data.

Hallucination

When an AI model produces information that sounds plausible but is false, fabricated, or not supported by its source material.

Mixture of Experts

A model design that splits a neural network into specialized sub-networks and activates only a few of them for each input.

Multimodal AI

AI systems that can process and combine multiple types of input or output, such as text, images, audio, and video.

Natural Language Processing

A field of AI focused on enabling computers to understand, interpret, and generate human language.

Prompt Engineering

The practice of designing and refining instructions given to AI models to get more accurate, useful, and reliable outputs.

Quantization

Reducing the numerical precision of an AI model's weights, for example from 16-bit to 4-bit numbers, to make it smaller and faster.

Reinforcement Learning from Human Feedback

A training method, known as RLHF, that improves an AI model by rewarding responses people rate as more helpful, accurate, or safe.

Retrieval-Augmented Generation

A technique that improves AI answers by first retrieving relevant documents and then giving them to a language model as context.

Small Language Model

A compact language model, typically with a few hundred million to around ten billion parameters, built to run efficiently on modest hardware.

Temperature (AI)

A setting that controls how random or predictable a language model's output is, with lower values giving more consistent answers.

Token (AI)

The basic unit of text a language model reads and writes, usually a word, part of a word, or punctuation mark.

Tokenization

Replacing sensitive data with non-sensitive substitutes called tokens, or breaking text into tokens that AI models can process.

Transformer

A neural network architecture, introduced in 2017, that uses attention mechanisms and underpins most modern large language models.

Zero-Shot Learning

The ability of an AI model to perform a task or recognize categories it was not explicitly trained on, without any examples.

Guides that mention Large Language Model