- Atlas
- Computer Science
- ML Systems
ML Systems
Machine learning systems are the machinery that turns statistical models into working software.
10 frameworks, from 1965 to today · Data Science · Computer Science
- Statistical Computing and Batch Analysis 1965-2005
- MapReduce 2004-2016
- Graph-Based Deep Learning Frameworks 2007-Present
- GPU-Accelerated Deep Learning Systems 2012-Present
- Feature store 2015-Present
- Embedding and Sparse Recommendation Systems 2016-Present
- Model and Feature Lineage as Data Assets 2016-Present
- Model and Data Parallelism with Mixed Precision 2017-Present
- Lifecycle Observability and Continuous Training 2018-Present
- MLOps 2018-Present
Overview
Machine learning systems are the machinery that turns statistical models into working software. A model on a laptop is a research artifact; a model serving millions of predictions a second, retrained nightly on fresh data, monitored for drift, and rolled back when it misbehaves is an ML system. The subfield's practitioners are not primarily the people who invent learning algorithms. They are the engineers and scientists who ask: how do we train on data too large for one machine, serve predictions fast enough to matter, keep training and serving consistent, and know when a deployed model has gone stale? What is at stake is the gap between a promising result and a dependable product—a gap that has swallowed far more models than it has delivered.
Statistical computing and the batch model
Before there was an ML systems field, there was statistical computing. From the 1960s onward, packages such as BMDP, SPSS, and later S and R gave statisticians a way to fit models by writing a few lines rather than hand-coding matrix algebra. The workflow was batch: assemble a dataset, run a script, read the output, repeat tomorrow. This culture established two habits that persist—models as artifacts produced by scripts, and the analyst as the person who runs them.
The first real systems problem was scale. Fitting a regression on millions of rows exceeded what a single program could hold in memory. The response in the 1990s and 2000s was to push computation to where the data lived. MapReduce, described by Dean and Ghemawat in 2004, split work across a cluster of commodity machines: a map step processes chunks in parallel, a reduce step aggregates. Its open-source implementation, Hadoop, made cluster-scale batch processing available to ordinary teams. Mahout, an early library built on Hadoop, expressed clustering and recommendation algorithms as MapReduce jobs. The lesson was structural: learning at scale is a distributed-systems problem, and the algorithm must be rewritten to fit the execution model.
The rise of the feature pipeline
As models moved from one-off analyses into products, a new problem appeared: the same transformation had to happen twice, once in the training script and once in the serving code, and the two drifted apart. A feature computed one way offline and another way online produces a model that looks accurate in evaluation and fails in production—a failure mode known as training-serving skew.
The response, emerging around the mid-2010s, was to make features first-class infrastructure. A feature store is a system that computes a feature once and serves it both to training jobs (as historical values) and to live prediction (as the latest value), guaranteeing the two agree1. Uber's Michelangelo platform, described publicly in 2017, was an influential example; Feast and similar open-source projects followed. Around the same time, teams began treating models and features as data assets with lineage—records of which data and which transformations produced which artifact—so that a model could be audited, reproduced, or rolled back. This was a conceptual shift: the model was no longer the deliverable but one node in a tracked pipeline.
GPUs and the deep learning stack
The decisive break came from hardware. Graphics processors, built to apply the same operation to many pixels at once, turned out to be well suited to the matrix multiplications at the heart of neural networks. Researchers had used GPUs for neural nets since the mid-2000s, but the field changed when the tooling caught up. Theano, released in 2007, and then TensorFlow (2015) and PyTorch (2016) let practitioners define a computation as a graph and have the framework handle gradients and device placement automatically. A graph-based framework represents the model as nodes (operations) and edges (data flowing between them), which makes it possible to optimize, parallelize, and differentiate the whole computation2.
This stack made deep learning practical, and deep learning made new systems demands. Models grew from millions to billions of parameters, and a single GPU could no longer hold them. Model parallelism—splitting one model across many devices—became necessary, alongside mixed-precision training, which stores some numbers in 16-bit floats instead of 32-bit to halve memory and speed up arithmetic while keeping enough precision to converge. These techniques, developed through the late 2010s, are what allowed the largest models to be trained at all.
Recommendations, embeddings, and sparse systems
Meanwhile, the highest-volume ML systems in the world were not image classifiers but recommenders. A recommender must score millions of items for millions of users in milliseconds, and the dominant representation became the embedding: a dense vector of numbers learned for each user and item, so that similarity becomes a geometric question. The systems challenge is that the embedding tables are enormous—far larger than any single machine's memory—and access is sparse, touching only a handful of rows per request. Industrial systems such as Google's Wide & Deep (2016) and subsequent deep recommendation models drove specialized infrastructure: sharded embedding tables, sparse updates during training, and low-latency lookup at serving. This line of work is where ML systems and large-scale information retrieval meet.
MLOps and the lifecycle
By the late 2010s, practitioners had accumulated a pile of partial solutions and a name for the discipline that binds them: MLOps, machine learning operations, by analogy with DevOps. The claim was that a model has a lifecycle—data collection, training, evaluation, deployment, monitoring, retraining, retirement—and that each stage needs tooling and ownership3.
Two ideas anchor it. The first is continuous training: because the world changes, a deployed model's inputs drift away from what it learned, so the system must detect degradation and retrain automatically. The second is lifecycle observability: logging predictions, inputs, and outcomes so that a team can see, after the fact, why a model behaved as it did4. Together these turn a model from a static file into a monitored service with a feedback loop. The field's vocabulary—model registry, experiment tracking, drift detection, canary rollout—dates largely from this period.
The present
Today the main approaches descend directly from these turning points. Feature stores and lineage tracking are standard in mature organizations; graph-based frameworks (PyTorch, TensorFlow, and JAX) are the default substrate for deep learning; model and data parallelism with mixed precision are routine for large models; and MLOps platforms bundle training, deployment, and monitoring into one pipeline. The open questions are real ones. How do you keep training-serving consistency when features are computed by large language models rather than simple aggregations1? How do you monitor a system whose outputs are open-ended text? And how do you make the whole lifecycle reproducible when the data itself is continuously changing5? The field began by asking how to run a model at scale; it now asks how to keep one trustworthy over time.
References
- Feature store, Wikipedia
- PyTorch, Wikipedia
- MLOps, Wikipedia
- Observability (software), Wikipedia
- Data version control, Wikipedia
The frameworks of ML Systems
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1965-2005
Statistical Computing and Batch Analysis
How do we fit statistical models with code?
- 2004-2016
- 2007-Present
Graph-Based Deep Learning Frameworks
How do we represent a model as a computation graph?
- 2012-Present
GPU-Accelerated Deep Learning Systems
How do we train deep networks fast with GPUs?
- 2015-Present
- 2016-Present
Embedding and Sparse Recommendation Systems
How do we serve recommendations from huge sparse tables?
- 2016-Present
Model and Feature Lineage as Data Assets
How do we track which data produced which model?
- 2017-Present
Model and Data Parallelism with Mixed Precision
How do we train models too large for one device?
- 2018-Present
Lifecycle Observability and Continuous Training
How do we know when a deployed model has gone stale?
- 2018-Present
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
GPU-Accelerated Deep Learning Systems grew out of Graph-Based Deep Learning Frameworks
GPU-accelerated deep learning systems grew directly out of graph-based frameworks like Theano, which first made it practical to define a computation as a graph and have the framework handle gradients and device placement. TensorFlow and PyTorch continued this lineage.
Carried by Theano (2007); TensorFlow (2015); PyTorch (2016).
Embedding and Sparse Recommendation Systems influenced by GPU-Accelerated Deep Learning Systems
GPU-accelerated deep learning provided the computational substrate for deep recommendation models like Wide & Deep, which combined embedding layers with deep networks. The training of large embedding tables relied on GPU acceleration.
Lifecycle Observability and Continuous Training influenced by Feature store
Feature stores provided the consistent feature computation that continuous training and observability depend on to detect drift. The infrastructure for serving features online enabled monitoring of training-serving consistency.
Model and Data Parallelism with Mixed Precision influenced by Embedding and Sparse Recommendation Systems
The enormous embedding tables of sparse recommendation systems motivated techniques for sharding and parallelizing model parameters across devices, which informed model parallelism. The challenge of training sparse, large-memory models contributed to mixed-precision and parallelism methods.
Beyond the field
Where ML Systems's ideas came from in other fields, and what they led to there: the strongest 12 of 38 links.
MapReduce led to Hadoop Ecosystem in Big Data Processing
The Hadoop ecosystem's processing layer is a direct implementation of the MapReduce model, the same idea filed under ML systems.
Carried by Doug Cutting.
Graph-Based Deep Learning Frameworks influenced by Backpropagation and gradient-based training in Deep Learning
Graph-based deep learning frameworks train graph neural networks by backpropagation over message-passing computations. The gradient-based training machinery developed in the 1980s is the direct basis for how these frameworks compute and apply gradients.
GPU-Accelerated Deep Learning Systems influenced by Backpropagation and gradient-based training in Deep Learning
GPU deep-learning systems accelerate backpropagation and gradient-based training, which remain the core algorithm. The frameworks exist to run that algorithm efficiently on parallel hardware.
GPU-Accelerated Deep Learning Systems influenced Deep Learning at Scale in Deep Learning
GPU-accelerated training, notably the two-GPU convolutional network of Krizhevsky, Sutskever and Hinton in 2012, made large-scale deep learning feasible and set off the scale era. The systems work supplied the compute methods the framework depends on.
Carried by AlexNet (2012); Alex Krizhevsky; Ilya Sutskever.
Model and Data Parallelism with Mixed Precision grew out of Distributed and Scalable Machine-Learning Parallelism in Parallel Algorithms
Model and data parallelism with mixed precision is a direct refinement of distributed and scalable ML parallelism, extending its parameter-server and collective-communication schemes with mixed-precision arithmetic. The later work grew out of the earlier scaling methods as a branch specialized for deep networks.
MLOps grew out of DevOps in Systems Lifecycle
MLOps grew directly out of DevOps, applying its principles of continuous integration, continuous delivery and automated monitoring to machine learning pipelines. The later framework extended DevOps practices to cover data and model versioning and retraining.
Statistical Computing and Batch Analysis led to Statistical package culture in Statistical Computing
Statistical computing and batch analysis is the same era's practice of running statistical jobs in batches, which the package culture grew out of as interactive packages replaced hand-written batch jobs. The package culture is a refinement of that batch computing practice.
Statistical Computing and Batch Analysis led to Reproducible and literate analysis in Statistical Computing
Reproducible analysis grew out of statistical computing and batch analysis, addressing the difficulty of reproducing analyses run as ad hoc scripts. Literate programming tools were applied to statistical batch workflows to make them auditable.
Carried by Sweave (2002).
Statistical Computing and Batch Analysis led to Statistical programming languages in Statistical Computing
Statistical programming languages grew out of earlier statistical computing and batch analysis practice, where programs were written to run analyses in batches. S, developed at Bell Labs in the 1970s, was designed as an interactive language replacing that batch workflow.
Carried by John Chambers; S (1976).
Statistical Computing and Batch Analysis led to Scalable and approximate computation in Statistical Computing
Scalable and approximate computation extends statistical computing and batch analysis to datasets too large for in-memory processing. Batch analysis techniques were generalized into distributed and streaming approximations.
MapReduce influenced by Batch Processing in Computer Systems
MapReduce's execution model descends from classical batch processing: jobs are run as offline batches over stored data with no interactive session. Dean and Ghemawat explicitly framed MapReduce as a batch-oriented model for large clusters, drawing on the batch-job scheduling tradition of mainframe systems.
Carried by Jeffrey Dean; Sanjay Ghemawat.
MapReduce influenced by Data Parallelism in Parallel Algorithms
MapReduce's map and reduce phases are a data-parallel formulation: the same operation is applied independently across partitions of the input. The design drew on decades of data-parallel programming ideas in which parallelism is expressed over data elements rather than explicit threads.

ML Systems is a planet in Nousfarer
Its city has a tower for each of these 10 frameworks. Land, walk in, and climb one concept at a time.
Land on ML SystemsRelated fields
Fields its ideas are linked to
- Deep Learning 6 links
- Statistical Computing 4 links
- Computer Vision 3 links
- Parallel Algorithms 3 links
- Parallel Computing 3 links
- Machine Learning 2 links
- Nosql Systems 2 links
- Information Retrieval 2 links
- Systems Lifecycle 2 links
- Linear Algebra 1 link
- Graph Algorithms 1 link
- Big Data Processing 1 link