Deployment & Deliveryintermediate11 min · 9 steps
Autoscaling
Add and remove pods, workers and GPU nodes automatically from the right saturation signal — fast up, slow down, within minimum and maximum limits.
pods3
in-flight/pod60
heavy workerrorrequestsignaldata
metrics are illustrativeProblemStep 1 of 9
Fixed capacity is wrong twice a day
Requests in flight swing from under 20 at night to ~180 at 9:00 (illustrative). Three fixed pods (120 in-flight capacity) are fine at night and overwhelmed every morning; seven fixed pods survive the morning and sit mostly idle the rest of the day.
Neither is right for long. Capacity has to follow load.
Lesson 12 of 20 in Deployment & Delivery