Knowledge MaxxingRepresentation learning
Representation learning is a model automatically learning useful internal features from raw data, rather than a person manually engineering those features beforehand.
Prerequisite: Attention mechanism
What matters
This shift is what let deep learning succeed on raw data like images and audio, where hand-designing effective features by hand had proven extremely difficult.
Example
A deep network trained on raw images learns its own internal representation of concepts like edges and shapes, rather than being given those features by a person.
Common misconception
Representation learning does not mean feature engineering is no longer useful; well-designed input can still improve a model, even one capable of learning its own representations.
Recall question
What shift did representation learning enable compared to hand-designed features?