MY ONE BLOGby Manishdai
Artificial Intelligence

What is Transformer?

Quick answer

A neural network architecture, introduced in 2017, that uses attention mechanisms and underpins most modern large language models.

Transformer, explained

The transformer was introduced in the 2017 research paper Attention Is All You Need by researchers at Google. Its key innovation is self-attention, which lets the model weigh how relevant every word in a sequence is to every other word, capturing context across long passages.

Earlier recurrent models processed text one word at a time, which made training slow. Transformers process sequences in parallel, making them well suited to GPUs and allowing training on vast datasets. They now power large language models, machine translation, code assistants, and many image, audio, and multimodal models. A known limitation is that standard attention becomes expensive as input length grows, an active area of research for more efficient variants.

Related terms

Large Language Model

An AI model trained on vast amounts of text to understand and generate human-like language, code, and other content.

Mixture of Experts

A model design that splits a neural network into specialized sub-networks and activates only a few of them for each input.

Natural Language Processing

A field of AI focused on enabling computers to understand, interpret, and generate human language.

Neural Network

A machine learning model made of layers of connected nodes, loosely inspired by the brain, that learns patterns by adjusting connection weights.

Guides that mention Transformer