AI systems that can process and combine multiple types of input or output, such as text, images, audio, and video.
Early AI models typically handled one type of data: a language model worked with text, an image classifier with pictures. Multimodal models can take in several kinds of information at once and reason across them.
For example, you could photograph a broken appliance and ask an assistant what part is likely faulty, upload a chart and request a written summary, or describe a scene and receive a generated image. Multimodal capabilities support accessibility tools that describe images for blind users, video analysis, robotics, and medical applications combining scans with clinical notes. These models are often built by connecting encoders for different data types to a shared model, frequently based on transformers. Evaluating their accuracy across modalities remains challenging.
A field of AI that enables computers to interpret and understand images and video.
Generative AIAI systems that create new content, such as text, images, audio, video, or code, based on patterns learned from training data.
Large Language ModelAn AI model trained on vast amounts of text to understand and generate human-like language, code, and other content.
A plain-English guide to computer vision: how machines classify and detect objects in images, and where the technology already works in retail, factories and healthcare.
Multimodal AI understands text, images, audio and video together. Here is how it works and how businesses use it to read invoices, photos and calls in one workflow.
What AI tokens and context windows are, how they affect your bill, why chatbots "forget" and how to get better answers by managing what goes into the model.