MLOpsintermediate13 min · 8 steps
ML Infrastructure and Orchestration
Pipelines as DAGs, the orchestrators that run them — Airflow, Kubeflow, TFX — and the CPUs, GPUs and TPUs underneath.
requestsignaldataheavy workresponse
ProblemStep 1 of 8
Cron jobs that don't know about each other
The first ML “platform” is often a few cron jobs on one GPU box: extract at 01:00, train at 03:00, evaluate at 05:00. Each job assumes the previous one finished.
One night the extract runs long. Training starts anyway on yesterday's data, evaluation passes, and nobody notices — there's no dependency tracking, no retry and no record of what ran.
Watch outTime-based scheduling is not dependency management.
Lesson 5 of 7 in MLOps