Knowledge Maxxing← Back to topic library
ML MaxxingAdvanced75 sec
Batch inference
Batch inference runs a trained model over a large collection of examples all at once on a schedule, rather than responding to individual requests as they arrive.
Prerequisite: Model registry
What matters
Because results are not needed instantly, batch inference can be scheduled during off-peak hours and processed more efficiently in bulk than handling each request separately.
Example
A recommendation system might run batch inference overnight to precompute suggested products for every customer, ready to serve the next day.
Common misconception
Batch inference is not suited to situations needing an immediate, per-request answer; online inference is used instead when results are needed right away.
Recall question
Why is batch inference well suited to tasks that don't need an instant response?
Keywords and related concepts
batch inferencescheduled predictionMLOpsonline inference