Transformer
Also called transformer architecture, transformer model
The neural network design behind almost all modern language models, built around a mechanism called attention.
The transformer is a neural network architecture introduced by Google researchers in the 2017 paper "Attention Is All You Need". It replaced older designs that read text one word at a time with one that processes all the tokens in a sequence together, using a mechanism called attention to work out how each token relates to every other token.
Two features made transformers dominant. Processing tokens in parallel suits modern GPUs, so transformers can be trained on far more data in less time. And attention lets the model connect words that are far apart, such as a pronoun and the name it refers to several sentences earlier.
A transformer is built from repeated layers. Each layer has attention, which mixes information between tokens, and a feed-forward network, which processes each token's information further. Large language models stack dozens or more of these layers. The "T" in GPT stands for transformer, and the same architecture is used for images, audio, protein structures and more.
Example
In "The trophy did not fit in the suitcase because it was too big", attention helps the model link "it" to "the trophy". Change "big" to "small" and attention links "it" to "the suitcase".
Common mistake
Assuming "transformer" describes how a model thinks or what it knows. It is the structure of the network, like describing a building's frame. Two transformer models can behave very differently depending on their training data, size and fine-tuning.