Hardware Accelerators

A hardware accelerator is a piece of computing machinery built to do one class of work much faster, or much more efficiently, than a general-purpose processor can.

9 frameworks, from 1965 to today · Computer Architecture · Computer Science

still a working view ended · each leads to its entry below

Overview

A hardware accelerator is a piece of computing machinery built to do one class of work much faster, or much more efficiently, than a general-purpose processor can1. The general-purpose CPU is designed to be good at everything: it fetches instructions, decodes them, checks for hazards, and executes whatever comes its way. That flexibility costs silicon area and energy1. An accelerator trades the flexibility away1. It hardwires the structure of a particular computation into circuits, so that the work flows through dedicated datapaths instead of being interpreted instruction by instruction1.

The stakes are economic and physical. For decades, making a single CPU core faster was the easy path to more performance. That path largely closed in the mid-2000s, when clock frequencies stopped rising because power density and heat became unmanageable. Since then, the gains have come from doing many things at once and from building specialized engines for the workloads that dominate2. Hardware acceleration is now the main way computers get faster at the tasks people actually care about: graphics, machine learning, signal processing, networking, and cryptography. Its history is a story of repeated swings between general-purpose and special-purpose design, each swing driven by a workload that the general machine could no longer handle well.

Coprocessors and the Floating-Point Problem (1960s–1980s)

Early computers handled arithmetic in software or with whatever integer hardware they had. Scientific computing needed floating-point numbers—values with a fractional part and an exponent—and doing that arithmetic in software was slow. The first response was the coprocessor: a separate chip that sat beside the main CPU and took over a specific class of instructions.

The classic example is the Intel 8087, released in 1980 as a companion to the 8088 and 8086 processors. When the CPU encountered a floating-point instruction, it handed the work to the 8087, which performed it in hardware. The arrangement kept the main processor simple and cheap for buyers who did not need floating point, while giving scientific users real speed. The idea was widely copied: Motorola's 68881 accompanied the 68020, and similar parts appeared across the industry. The coprocessor established a durable pattern—a specialized unit attached to a general core, invoked for a narrow task—that recurs throughout this history.

Digital Signal Processors: Arithmetic Built for Real Time (1980s)

A second lineage grew out of telecommunications and audio. Signals—sound, radar returns, radio—arrive as streams of samples that must be filtered, transformed, and compressed continuously, in real time. The dominant operation is the multiply-accumulate (MAC): multiply two numbers and add the result to a running total. A filter or a Fourier transform is thousands of these in sequence.

General-purpose CPUs of the era were poor at this. They lacked fast multipliers, and their memory addressing could not keep up with streaming data. The digital signal processor (DSP) was designed around the problem3. Texas Instruments' TMS32010, introduced in 1982, is often cited as the first commercially successful single-chip DSP. It had a hardware multiplier, special addressing modes that stepped through arrays automatically, and an architecture that could fetch an instruction and operands in one cycle. Later DSPs added barrel shifters (circuits that shift a number by many bit positions at once) and multiple MAC units. The DSP showed that committing silicon to one arithmetic pattern could beat a general CPU by large factors on the workloads that mattered, and it made possible the digital modems, speech codecs, and later mobile phones that depend on real-time signal math.

Reconfigurable Computing: Hardware That Changes Its Mind (1980s–1990s)

A field-programmable gate array (FPGA) is a chip full of logic blocks and programmable wiring; the user configures it after manufacture to implement a circuit4. In the late 1980s and 1990s, researchers asked whether an FPGA could serve as a reconfigurable accelerator—a piece of hardware that is reshaped to fit each computation, then reshaped again for the next.

The appeal was that a circuit built for one algorithm runs it with the efficiency of dedicated hardware, yet the same chip can be reprogrammed for a different algorithm5. Systems such as the Splash arrays, built at the Supercomputing Research Center in the late 1980s for systolic string matching, demonstrated the approach on real problems. The catch was programming: describing a computation as a circuit was far harder than writing software, and reconfiguring the chip took time. Reconfigurable computing found a durable niche in prototyping, networking, and embedded systems, and it remains a live approach, but it never displaced the general CPU for mainstream computing.

The Graphics Pipeline: Fixed Function to Programmable (1980s–2000s)

Graphics became the largest accelerator market of all. Rendering a three-dimensional scene—projecting geometry, shading surfaces, drawing pixels—is a flood of parallel arithmetic, and CPUs of the 1980s and 1990s could not keep up. The answer was the graphics processing unit (GPU), built as a fixed-function pipeline: a chain of dedicated stages, each doing one job, with the data flowing from one to the next6.

Early 3D accelerators such as 3dfx's Voodoo, released in 1996, handled rasterization and texture mapping in hardware and transformed PC gaming. Through the late 1990s the pipeline grew more elaborate, with hardware for transformation and lighting. Then came a decisive turn. Around 2001, with Microsoft's DirectX 8 and the GPUs that supported it, the fixed stages began to be replaced by programmable shaders—small programs the developer writes to control how vertices and pixels are processed67. NVIDIA's GeForce 3 was an early part of this shift8. The GPU became a machine with many small programmable processors rather than a rigid chain. This mattered beyond graphics: once the pipeline was programmable, it could be aimed at other problems.

GPGPU: The Graphics Chip as a General Computer (2000s)

Researchers noticed that the GPU's thousands of parallel arithmetic lanes were well suited to any problem that could be expressed as the same operation applied to many data elements. Using a graphics chip for non-graphics work became known as general-purpose computing on GPUs (GPGPU)7. Early efforts required disguising the computation as a graphics problem, which was awkward. The turning point was NVIDIA's CUDA, introduced in 2007, a programming model that let developers write general parallel code for the GPU directly9. The GPU's design—many simple cores sharing control logic, optimized for throughput rather than single-thread latency—proved a natural fit for scientific simulation, and later for the matrix arithmetic at the heart of machine learning. GPGPU moved the GPU from a graphics accessory to a mainstream compute engine.

Systolic Arrays and Matrix Engines (1970s onward, resurgent 2010s)

The systolic array is an idea from the late 1970s, associated with H. T. Kung and Charles Leiserson: arrange a grid of simple processing elements so that data pulses through them rhythmically, like a heartbeat, with each element passing results to its neighbors10. The structure is ideal for matrix operations, where the same multiply-accumulate pattern repeats across a grid. The idea lay mostly dormant in mainstream computing for decades because general processors handled matrices adequately.

Machine learning revived it. Training and running neural networks is dominated by matrix multiplication, and the arithmetic is regular enough to hardwire. Google's Tensor Processing Unit, a custom chip for neural-network inference deployed in its data centers from around 2015, used a systolic array of multiply-accumulate units as its core. The design delivered large gains in throughput per watt over general hardware for that workload. The systolic array became a standard building block of matrix engines—dedicated units for dense linear algebra—now found in AI accelerators and, increasingly, inside general CPUs and GPUs as well10.

Domain-Specific Architecture: The Current Consensus (2010s–present)

By the 2010s the pattern had a name. Domain-specific architecture means designing a machine around the structure of one family of workloads rather than for generality2. The argument was crystallized in a widely cited 2017 paper by John Hennessy and David Patterson, who observed that as general-purpose scaling slowed, the gains would come from specialization—chips tailored to a domain, paired with the software that targets them.

The clearest demonstration was the rise of dedicated AI silicon: Google's TPU, and a wave of startup and incumbent accelerators for training and inference. The same logic spread to networking (smart network interface cards and programmable switches), cryptography, video encoding, and databases. The trade-off is real: a domain-specific chip is fast and efficient within its domain and useless outside it, and it must be redesigned as the domain's algorithms change. That fragility is why the field keeps returning to reconfigurable and programmable designs as hedges.

Memory-Centric Design: Moving the Bottleneck (2010s–present)

As accelerators got faster, a new limit appeared: moving data to and from memory cost more time and energy than the arithmetic itself. This is the memory wall, and it reshaped accelerator design. Memory-centric designs put computation close to where data lives—processing in or near memory—to avoid shuttling it back and forth1. Examples include processing-in-memory architectures that perform operations inside DRAM, and the large on-chip memories and high-bandwidth memory stacks that AI accelerators rely on. The shift reflects a broader recognition: for many modern workloads, the accelerator's job is as much about managing data movement as about computing.

Where the Field Stands

Today the landscape is plural. General CPUs carry specialized units—vector extensions, matrix engines, cryptographic instructions—absorbing accelerator ideas into the general machine. GPUs have become programmable parallel computers that also accelerate AI. Dedicated domain-specific chips handle the largest AI and networking workloads. FPGAs and other reconfigurable fabrics serve applications that need hardware efficiency with the ability to change4. DSPs persist in embedded signal processing3. The open questions are the ones the history keeps raising: how specialized is too specialized, given that algorithms shift; how to program heterogeneous systems of many accelerators without drowning developers in complexity; and how to keep feeding fast arithmetic units as the memory wall tightens. Each swing of the pendulum between general and special purpose has answered one of these and exposed the next.

References

  1. Hardware acceleration (introduction, Hardware execution units, Applications, Emerging hardware architectures), Wikipedia
  2. Domain-specific architecture (History, introduction), Wikipedia
  3. Digital signal processor, Wikipedia
  4. Field-programmable gate array, Wikipedia
  5. Reconfigurable computing, Wikipedia
  6. Fixed-function (computer graphics) (introduction, History), Wikipedia
  7. General-purpose computing on graphics processing units (History, introduction), Wikipedia
  8. GeForce 3 series, Wikipedia
  9. CUDA (introduction, History), Wikipedia
  10. Systolic array (introduction, History), Wikipedia

The frameworks of Hardware Accelerators

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1965-1995

    Coprocessor

    How can a narrow arithmetic task be offloaded from the main processor?

    Fly to its tower in Nousfarer

  2. 1978-Present

    Systolic array

    How should processing elements be arranged to stream matrix arithmetic?

    Fly to its tower in Nousfarer

  3. 1982-Present

    Digital signal processor

    How can streaming signal math be done in real time?

    Fly to its tower in Nousfarer

  4. 1988-Present

    Reconfigurable Computing

    Can hardware be reshaped to fit each computation?

    Fly to its tower in Nousfarer

  5. 1996-2001

    Fixed-Function Graphics Pipeline

    How can a chain of dedicated stages render 3D scenes quickly?

    Fly to its tower in Nousfarer

  6. 2001-Present

    Programmable Shader Architecture

    How should developers control how vertices and pixels are processed?

    Fly to its tower in Nousfarer

  7. 2003-Present

    General-purpose computing on graphics processing units

    Can a graphics chip be used as a general parallel computer?

    Fly to its tower in Nousfarer

  8. 2013-Present

    Memory-Centric Architecture

    Should computation be organized around data movement rather than processing?

    Fly to its tower in Nousfarer

  9. 2014-Present

    Domain-Specific Architecture

    Should machines be designed around one family of workloads?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

CoprocessorSystolic arrayDigital signal processorReconfigurable ComputingFixed-Function Graphics PipelineProgrammable Shader ArchitectureGeneral-purpose computing on graphics…Memory-Centric ArchitectureDomain-Specific Architecture

Beyond the field

Where Hardware Accelerators's ideas came from in other fields, and what they led to there: the strongest 12 of 37 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Hardware Accelerators is a planet in Nousfarer

Its city has a tower for each of these 9 frameworks. Land, walk in, and climb one concept at a time.

Land on Hardware Accelerators

Related fields

Fields its ideas are linked to

More in Computer Architecture

See where Hardware Accelerators sits on the Map of Human Knowledge