Loading field map
The central tension in machine learning systems is the gap between a model that works in a research notebook and a model that works reliably in production. A Jupyter notebook might train a classifier on a static dataset, but deploying that classifier means handling live data streams, unpredictable traffic spikes, model versioning, monitoring for performance decay, and coordinating teams that own different parts of the pipeline. ML systems emerged as a distinct engineering subfield precisely to bridge this gap: they provide the infrastructure, abstractions, and operational practices that turn a trained artifact into a dependable production service.
The first framework to crystallize was Distributed Training Systems, which took shape around 2010. Early machine learning practitioners who wanted to train large models on massive datasets quickly hit the limits of a single machine. Training a deep neural network on millions of images could take weeks on one GPU. The core commitment of Distributed Training Systems was to reframe training as a distributed computing problem: how do you split the data, the model parameters, or both across many machines while keeping the training algorithm correct and efficient?
Parameter Server architectures became an early landmark. They separated the storage of model parameters from the worker nodes that computed gradients, allowing asynchronous updates that kept all machines busy. Later frameworks like TensorFlow and PyTorch Distributed introduced data-parallel and model-parallel strategies, each with different trade-offs between communication overhead and convergence speed. The key conceptual shift was that training was no longer a single-machine batch job; it became a coordinated, fault-tolerant, and often asynchronous distributed process. This framework did not replace single-machine training—many small models still fit on one GPU—but it created a new category of infrastructure for the models that did not.
By 2016, organizations had trained large models, but they faced a second problem: how to run those models in production with low latency and high throughput. A model trained on a GPU cluster might be exported as a file, but loading that file into a web server and handling requests at scale required a different set of engineering concerns. Model Serving and Inference Systems emerged as a distinct framework, separate from training infrastructure.
The relationship between serving and training is one of co-evolution rather than replacement. Training systems produce model artifacts; serving systems consume them. But the engineering priorities diverge sharply. Serving systems care about latency, throughput, request batching, hardware acceleration for inference, and graceful handling of model version updates without downtime. TensorFlow Serving, for example, introduced a serving architecture that could load multiple model versions simultaneously and route traffic between them, enabling canary deployments and A/B testing. NVIDIA Triton Inference Server added support for multiple frameworks and optimized GPU utilization for inference workloads. The distinctive contribution of this framework was to treat inference as a first-class systems problem, not as an afterthought to training.
As organizations accumulated training and serving infrastructure, a new pain point emerged: the overall lifecycle of a machine learning project was fragmented. Data preparation, training, evaluation, deployment, and monitoring were often stitched together with ad-hoc scripts, making it hard to reproduce results, track experiments, or recover from failures. ML Pipeline and Workflow Management frameworks, which began appearing around 2017, addressed this by providing a unified orchestration layer.
Kubeflow Pipelines and Apache Airflow (adapted for ML workflows) allowed practitioners to define the entire lifecycle as a directed acyclic graph of steps. Each step—data validation, training, model evaluation, deployment—could be containerized, versioned, and rerun independently. The framework's core commitment was reproducibility: if a pipeline definition is checked into version control, anyone should be able to replay the exact sequence of operations that produced a given model. This framework did not replace Distributed Training or Model Serving; instead, it provided the coordination infrastructure that connected them. It implicitly defined the "ML lifecycle" as a new conceptual object—something that could be designed, debugged, and optimized as a whole.
Even with pipelines and serving systems in place, a persistent problem remained: training-serving skew. A model might be trained on features computed one way, but the production serving code might compute those same features slightly differently—a missing normalization, a different timestamp rounding, a silently updated lookup table. Feature Management Systems, which emerged around 2018, tackled this by treating features as managed infrastructure rather than ad-hoc preprocessing code.
Feature stores like Feast and Tecton introduced a centralized registry where feature definitions, transformations, and serving logic were stored and versioned. The same feature computation could be used during training (to produce training data) and during serving (to produce live inference inputs), eliminating the skew at its source. The distinctive contribution of this framework was to narrow the scope of what data engineers and ML engineers each owned: feature logic became a shared, versioned artifact rather than duplicated code in notebooks and serving scripts. Feature Management Systems coexist with ML Pipeline frameworks—pipelines orchestrate the jobs that compute features, while feature stores manage the definitions and serve the values—but they resolve a tension that pipelines alone could not address: the consistency of features across time and environments.
Today, these four frameworks form a layered stack. Distributed Training Systems handle the compute-intensive training phase. Model Serving Systems handle the latency-sensitive inference phase. ML Pipeline and Workflow Management orchestrates the steps that connect them, and Feature Management Systems ensure that the data flowing through those steps is consistent. No single framework has absorbed the others; each remains active and continues to evolve.
What the leading frameworks agree on is that ML systems must be treated as first-class software engineering artifacts, not as one-off research experiments. They share a commitment to versioning, reproducibility, monitoring, and modular design. Where they disagree is on the degree of integration. Some teams advocate for end-to-end platforms (like Vertex AI or SageMaker) that bundle training, serving, pipelines, and feature management into a single managed service. Others prefer modular, composable tools that can be swapped independently—a choice that trades convenience for flexibility. A related debate concerns MLOps, which extends these frameworks with practices for continuous integration, continuous delivery, and automated retraining. MLOps is not itself a foundational framework in the same sense as the four described here; it is an operational methodology that builds on top of them, much as DevOps builds on top of infrastructure-as-code and monitoring tools.
The subfield's trajectory shows a clear pattern: each framework emerged to address a specific gap left by earlier infrastructure. Distributed training made large-scale model development possible. Model serving made those models usable in products. Pipeline management made the lifecycle reproducible. Feature management made the data consistent. Together, they transformed machine learning from a model-centric craft into a systems engineering discipline.