Knowledge MaxxingMultimodal model
A multimodal model is a single system that can process or generate more than one type of data, such as text and images together, rather than only one.
Prerequisite: Model input
What matters
True multimodal models learn a shared representation across data types, letting them reason across a photo and a question about it together, rather than handling each modality in an isolated pipeline.
Example
A multimodal model can look at a photo of a fridge's contents and answer 'What can I cook with this?' by combining image understanding with text generation.
Common misconception
Stitching together two separate single-modality tools with a script is not the same as a genuinely multimodal model that reasons across both together.
Recall question
What distinguishes a genuinely multimodal model from two separate tools glued together?