Knowledge MaxxingTransformer
A transformer is a neural network architecture that processes an entire sequence at once using self-attention, instead of reading it one element at a time like earlier models.
Prerequisite: Deep learning
What matters
Processing the whole sequence in parallel lets transformers train efficiently on large datasets and directly relate distant parts of a sequence to each other, which is why they underlie most modern language models.
Example
A transformer reading a sentence can directly relate the word 'it' near the end to a noun near the beginning, without passing information step by step through every word in between.
Common misconception
A transformer is not itself a language model; it is the architecture that most modern language models, including many non-text ones, are built from.
Recall question
What lets a transformer relate distant words to each other more directly than earlier sequence models?