Loading field map
Hardware accelerators are specialized computing engines designed to execute a narrow set of tasks far more efficiently than a general-purpose processor. The central problem they address is the "generality tax": a central processing unit (CPU) is built to handle any conceivable program, but that flexibility requires fetching instructions, decoding them, managing out-of-order execution, and maintaining precise memory coherence. Each of these overheads consumes time and energy. An accelerator abandons most of that flexibility, fixing its data paths and control logic to a specific algorithm or class of algorithms, thereby achieving orders-of-magnitude improvements in speed and energy efficiency for that workload.
The subfield of hardware accelerators sits at the intersection of computer architecture, VLSI (very-large-scale integration) design, and compiler technology. It studies not only the circuits themselves but also the systems that integrate them: how accelerators communicate with CPUs, how data moves between them, how programs are partitioned between general and specialized hardware, and how multiple accelerators coexist. The stakes are high because the end of Dennard scaling and the slowing of Moore's law have made it increasingly difficult to improve general-purpose processor performance through clock frequency or transistor density alone. Accelerators have become the primary means of continuing performance gains for important application domains, from graphics and machine learning to networking and cryptography.
The idea of specialized hardware predates the modern term "accelerator." Early computers themselves were often built for specific military or scientific purposes. The first widely recognized accelerators, however, emerged in the 1980s as attached processors for personal computers and workstations. Graphics processing units (GPUs) began as fixed-function pipelines for rasterizing triangles and shading pixels, with no programmability at all. Digital signal processors (DSPs) were programmable but with specialized instruction sets and memory architectures tuned for repetitive multiply-accumulate operations common in audio and communications.
A crucial shift occurred when GPUs became programmable. By the early 2000s, graphics chips had evolved into massively parallel processors with thousands of simple cores, and researchers discovered that their floating-point throughput could be harnessed for non-graphics workloads like scientific simulation. This gave rise to the general-purpose GPU (GPGPU) movement and, eventually, to programming models like CUDA and OpenCL that treated the GPU as a general-purpose parallel computer. The GPU's success demonstrated that a hardware design originally motivated by a single application domain could become a broadly useful accelerator for any workload with massive data-level parallelism.
The second major wave came from field-programmable gate arrays (FPGAs). FPGAs are chips whose logic blocks and interconnects can be reconfigured after manufacturing. They offer a middle ground between the fixed logic of an application-specific integrated circuit (ASIC) and the instruction-driven flexibility of a CPU. For decades, FPGAs were used primarily for prototyping and for glue logic in embedded systems. But as their capacity grew and high-level synthesis tools improved, they became viable accelerators for networking, signal processing, and, more recently, data-center workloads. Microsoft's use of FPGAs in its data centers for Bing search ranking and network acceleration was a prominent demonstration that reconfigurable accelerators could be deployed at scale.
The most recent and most visible wave is the domain-specific accelerator, epitomized by the tensor processing unit (TPU) introduced by Google in 2016. The TPU is an ASIC designed specifically for neural network inference and training. Its data path is a systolic array—a regular grid of processing elements that pass data to their neighbors—optimized for the matrix multiplications that dominate deep learning. The TPU's success, and the subsequent proliferation of AI accelerators from many companies, marked a return to the oldest form of acceleration: building a chip for one algorithm. But it did so with modern manufacturing technology and a clear-eyed focus on the economics of data-center workloads.
The field is organized less by rival schools than by a set of design axes along which every accelerator makes choices. Understanding these axes is the key to mapping the subfield.
The most fundamental axis is the degree of programmability. At one extreme lies the fixed-function ASIC: a circuit that implements a specific algorithm, such as the AES encryption standard or a particular video codec. It has no instruction set, no memory management unit, and no ability to run code. Its control logic is a finite-state machine that sequences the data path through the algorithm's steps. The advantage is maximal efficiency: every transistor exists to perform a useful operation, and there is no energy spent on fetching and decoding instructions. The disadvantage is brittleness: if the algorithm changes—say, a new encryption standard is adopted—the chip is obsolete.
At the other extreme lies the programmable accelerator, such as a GPU or DSP. These chips have instruction sets, but the instructions are specialized for the target domain. A GPU's instruction set includes operations for texture sampling, blending, and vector arithmetic; a DSP's includes multiply-accumulate with saturation and circular buffer addressing. The programmability allows the same hardware to serve multiple generations of algorithms and multiple applications within a domain. The cost is the same generality tax that afflicts CPUs, though smaller because the instruction set is narrower.
Between these extremes lie reconfigurable devices like FPGAs and coarse-grained reconfigurable arrays (CGRAs). FPGAs are essentially programmable at the logic-gate level: the designer configures lookup tables, multiplexers, and routing switches to implement a circuit. This offers near-ASIC efficiency for a specific design, but the design can be changed by loading a new configuration bitstream. The cost is that FPGAs are less efficient than ASICs because the programmable routing and lookup tables consume area and power. CGRAs are a less common middle ground: they provide an array of word-level arithmetic units with programmable interconnects, offering more efficiency than FPGAs but less flexibility.
The tension between fixed-function and programmable is not resolved by choosing one; it is managed by partitioning the workload. A modern system-on-chip (SoC) contains a CPU, a GPU, a DSP, and numerous fixed-function blocks for video, audio, and security. The art of system architecture is deciding which functions deserve fixed hardware, which need programmability, and how they all share memory and power.
A second axis concerns how the accelerator organizes computation. A control-flow architecture, like a CPU or GPU, is organized around a program counter: instructions are fetched, decoded, and executed in sequence, with branches determining the next instruction. A dataflow architecture, by contrast, has no program counter. Instead, operations are triggered by the arrival of their operands. A systolic array is a classic dataflow design: each processing element receives data from its neighbors, performs an operation, and passes the result onward. The computation is a pipeline of data moving through the array, and the "program" is the physical structure of the array itself.
Dataflow designs are attractive for regular, repetitive computations like matrix multiplication, convolution, and filtering, where the same operation is applied to a stream of data. They eliminate the overhead of instruction fetch and scheduling, and they naturally exploit pipelining and parallelism. Their weakness is irregularity: if the computation has complex control flow—data-dependent branches, variable-length loops, or sparse data structures—a dataflow architecture either stalls or requires elaborate mechanisms to handle the irregularity.
Modern accelerators often combine both styles. A GPU is fundamentally a control-flow machine, but its massive thread-level parallelism and its hardware scheduler create a dataflow-like throughput for regular workloads. A TPU is a dataflow machine for the dense matrix operations of neural networks, but it has a control processor that sequences the operations and manages the data movement between the systolic array and memory.
A third axis, increasingly dominant, is the focus on data movement. For many workloads, the energy cost of moving data from memory to the compute unit far exceeds the energy cost of the computation itself. This observation has driven a family of approaches that restructure the accelerator around memory.
The simplest form is the use of on-chip scratchpad memory or a carefully managed cache hierarchy. The accelerator's performance depends on keeping data in the fast, local memory and minimizing off-chip accesses. More radical approaches include processing-in-memory (PIM), where computation is placed inside or near the memory cells, and near-memory computing, where logic is stacked on top of or beside DRAM. These approaches are motivated by the "memory wall"—the growing gap between processor speed and memory latency—and by the energy cost of moving bits across a memory bus.
The memory-centric view also shapes how accelerators are programmed. A programmer or compiler must decide how to tile the computation so that data reused across operations stays in the accelerator's local memory. This is a fundamentally different programming model from the CPU's assumption of a flat, coherent memory space. The tension between the accelerator's need for explicit data management and the programmer's desire for a simple memory model is a persistent theme.
No accelerator operates in isolation. A central question of the subfield is how accelerators are integrated into a larger computing system. The earliest accelerators were separate chips on a board, connected by a bus like PCI Express. Data was copied from CPU memory to the accelerator's memory, the accelerator ran, and the results were copied back. This model, still common for discrete GPUs, has high per-operation overhead but allows the accelerator to be a self-contained unit.
Modern integration is far tighter. Accelerators are increasingly integrated into the same package as the CPU, sharing a coherent memory space and communicating through a high-bandwidth interconnect. This reduces the data-copy overhead and allows fine-grained cooperation between the CPU and the accelerator. The extreme form is the system-on-chip, where the CPU, GPU, DSP, and fixed-function blocks all share the same memory controller and power management unit.
The integration problem has a software dimension as well. The CPU must launch accelerators, manage their memory, and handle their completion. This requires a runtime system, device drivers, and a programming model. The dominant models—CUDA for GPUs, OpenCL for heterogeneous systems, and various vendor-specific frameworks for AI accelerators—each embody a particular answer to the question of how much control the programmer has over data placement and scheduling. The trend has been toward higher-level abstractions, where the compiler and runtime make the partitioning decisions, but the tension between programmer control and automatic management remains unresolved.
The present landscape of hardware accelerators is defined by several durable features. First, the field has shifted from a focus on peak throughput to a focus on energy efficiency and total cost of ownership. An accelerator's value is measured not just in operations per second but in operations per second per watt, and increasingly in the cost of the entire system—including the servers, cooling, and power delivery—needed to deploy it.
Second, the field has become application-driven in a way it was not during the CPU-centric era. The most active area of accelerator design is machine learning, where the computational demands of training and inference have outpaced the improvements of general-purpose hardware. This has led to a proliferation of AI accelerators with widely varying architectures: systolic arrays, dataflow engines, sparse-matrix accelerators, and analog in-memory computing. The field is still converging on which architectures will dominate, and it is possible that the current diversity will persist because different workloads—training vs. inference, vision vs. language, dense vs. sparse—favor different designs.
Third, the field has become more closely coupled to compiler and programming-model research. An accelerator is only as useful as the software that can target it. The most successful accelerators, like GPUs, have robust software ecosystems. The least successful are those that required heroic manual programming. The current frontier is the development of compilers that can automatically map high-level programs—written in Python, C++, or domain-specific languages—onto heterogeneous collections of accelerators. This is a hard problem because the compiler must reason about data layout, memory bandwidth, and parallelism across devices with very different characteristics.
Fourth, the field is increasingly concerned with the "dark silicon" problem: as transistors shrink, it becomes impossible to power all of them at once. The response has been to build specialized blocks that are powered on only when needed. This has made accelerators a central tool of power management, not just performance. A modern chip is a collection of specialized engines, each with its own power domain, and the operating system or a hardware manager decides which engines to activate.
Finally, the field has a persistent tension between the desire for a universal accelerator—a single chip that can accelerate any workload—and the reality that specialization is what provides the gains. The search for a "programmable accelerator" that captures the efficiency of specialization without the brittleness has produced many designs, from VLIW (very long instruction word) processors to reconfigurable dataflow engines, but none has achieved the universality of the CPU or the efficiency of the ASIC. The field's history suggests that this tension is not a problem to be solved but a design space to be explored, with each point in the space representing a different trade-off between efficiency, flexibility, and ease of programming.
The durable landscape, then, is not a settled set of winners and losers but a continuing exploration of this design space. The CPU remains the host and orchestrator, but the real work of computation increasingly happens in specialized engines. The subfield of hardware accelerators is the study of how to design those engines, how to integrate them, and how to program them—a set of questions that will remain central to computing as long as the demand for performance outpaces the improvements of general-purpose hardware.