Microarchitecture

Microarchitecture is the level of computer design that sits between an instruction set architecture (ISA) — the vocabulary of instructions a program can use — and the silicon that actually executes them.

14 frameworks, from 1950 to today · Computer Architecture · Computer Science

still a working view ended · each leads to its entry below

Overview

Microarchitecture is the level of computer design that sits between an instruction set architecture (ISA) — the vocabulary of instructions a program can use — and the silicon that actually executes them1. Where the ISA says what a processor must do, the microarchitecture decides how: which units exist, how instructions flow through them, how many can be in flight at once, and how the machine keeps itself busy while waiting for memory. It is the discipline that turned a processor from a device that executes one instruction at a time into one that juggles hundreds of instructions, several programs, and multiple cores while staying within a power budget. Its history is a story of chasing parallelism that software could not easily express, and of repeatedly discovering that the bottleneck had moved somewhere new.

Microprogrammed control and the discipline of regularity

Early computers of the 1940s and 1950s controlled their hardware directly with logic circuits wired to perform each instruction. As instruction sets grew more elaborate, designers faced a problem: complex instructions meant complex, hard-to-debug control logic2. Maurice Wilkes proposed a solution in 1951: store the control signals as a sequence of microinstructions in a control memory, so that executing one machine instruction means running a small program of microinstructions. This was microprogrammed control, and it made control design regular and changeable — a processor's behavior could be altered by rewriting its microcode rather than rewiring it134.

IBM's System/360 family, announced in 1964, made microprogramming central to a commercial strategy: different models at different price points could share one ISA, with cheaper models using microcode to emulate instructions that costlier models implemented in hardware. The alternative, hardwired control, built each instruction's control signals directly into fixed logic2. Hardwired control was faster but inflexible; microcode was slower but easier to design and extend2. For two decades the trade-off favored microcode in most general-purpose machines, and the microcode of machines like the VAX and the Intel x86 line became a substantial engineering artifact in its own right.

Pipelining: overlapping the stages of execution

A processor that fetches, decodes, executes, and writes back one instruction before starting the next wastes most of its hardware most of the time1. Pipelining fixes this by overlapping those stages, as an assembly line does: while one instruction executes, the next is being decoded and the one after fetched1. The idea appeared in early machines — IBM's Stretch (1961) and the Atlas computer at Manchester (1962) used overlapping execution — and became standard in the 1960s and 1970s.

Pipelining introduced a new class of problem: hazards5. A data hazard occurs when an instruction needs a result the previous one has not yet produced; a control hazard occurs when a branch makes it unclear which instruction to fetch next5. Early pipelines handled data hazards by stalling or by forwarding results directly between stages, and handled branches by delaying the fetch until the branch resolved. The cost of a mispredicted branch — the pipeline flush — became a central quantity in processor design1.

The RISC argument and the load-store discipline

By the late 1970s, instruction sets had grown baroque, and microcoded implementations were struggling to keep pipelines full. In 1980 David Patterson and David Ditzel published an argument for what became known as RISC, the reduced instruction set computer: simple, fixed-length instructions that could be decoded quickly and pipelined efficiently, with memory accessed only through explicit load and store instructions. The Berkeley RISC project and John Hennessy's Stanford MIPS project built working machines around these ideas in the early 1980s, and commercial RISC processors followed.

The RISC philosophy did not merely simplify instructions; it changed what microarchitects could assume. A regular instruction format made it practical to fetch and decode several instructions per cycle, and a load-store design made it easier to schedule memory operations. The debate between RISC and the incumbent complex instruction sets was fierce, but the deeper legacy was methodological: microarchitecture became an empirical discipline, with simulators and benchmark suites used to compare designs before silicon was built.

Caches and the widening memory gap

Processors were getting faster than main memory could feed them16. The cache — a small, fast memory holding recently used data — had been proposed by Wilkes in 1965 and appeared in machines such as the IBM System/360 Model 85 (1968). Through the 1980s and 1990s, cache design became a core part of microarchitecture: multiple levels of cache, separate instruction and data caches, and replacement policies tuned to observed program behavior. The memory hierarchy meant that a processor's performance depended as much on what it could keep close as on how fast it could compute, and cache misses became a dominant source of stalls that later techniques were built to hide1.

Dynamic scheduling, scoreboarding, and out-of-order execution

A pipeline that executes instructions strictly in program order stalls whenever an instruction waits for a slow operation1. Dynamic scheduling lets the hardware reorder execution: an instruction can issue as soon as its operands are ready, even if earlier instructions are still waiting17. The CDC 6600 (1964) used a technique called scoreboarding to track dependencies and allow out-of-order execution, and IBM's System/360 Model 91 (1967) introduced register renaming and a form of out-of-order issue in its floating-point unit, using a structure that anticipated the reorder buffer.

The modern form crystallized in the 1990s. Robert Tomasulo's algorithm, developed for the IBM 360/91, used reservation stations to hold instructions until operands arrived and a common data bus to broadcast results8. The reorder buffer, described in the late 1980s and implemented in processors such as the Intel Pentium Pro (1995), let the machine execute out of order but retire results in program order, preserving the illusion of sequential execution. Register renaming removed false dependencies by giving each new write to a register a fresh physical register19. Together these made it possible to keep many instructions in flight, at the cost of substantial complexity in the hardware that tracks them.

Superscalar issue and the limits of instruction-level parallelism

Pipelining overlaps stages of different instructions; superscalar issue goes further by starting several instructions in the same cycle10. Machines like the IBM Power1 (1990) and the DEC Alpha 21064 (1992) could issue multiple instructions per cycle, and by the mid-1990s four-wide superscalar designs were common. The limit was not the width of the issue logic but the available parallelism in ordinary programs: dependencies between instructions meant that wider machines often sat idle.

Branch prediction and speculative execution were the answer to control hazards1. Instead of waiting to learn whether a branch is taken, the processor predicts and fetches down the predicted path, discarding work if the prediction was wrong111. Two-level adaptive predictors, developed in the early 1990s, used history of past branch outcomes to improve accuracy, and by the late 1990s prediction rates above 95 percent were routine. Speculation let the machine keep its pipeline full across branches, but it also meant that incorrect work was performed and thrown away — and, as became clear much later, that speculative execution could leak information through side channels12.

Simultaneous multithreading and the move to many cores

Even a wide out-of-order machine leaves execution units idle when a single program lacks parallelism. Simultaneous multithreading (SMT) lets one physical core issue instructions from several threads in the same cycle, filling those gaps113. The idea was explored in research machines in the 1990s, notably the Tera computer and the MIT T project, and reached the commercial mainstream with Intel's Hyper-Threading in 2002. SMT improved throughput on workloads with multiple threads but did not raise single-thread performance1.

The larger shift came when clock frequency stopped rising. Around the mid-2000s, power and heat limits — sometimes called the "power wall" — made it impractical to keep increasing single-core speed. The response was chip multiprocessing: putting multiple processor cores on one die114. Commercial dual-core processors appeared in 2005 from both Intel and AMD, and core counts grew steadily thereafter. This was not a new microarchitectural idea so much as a change in what microarchitecture was for: instead of extracting more parallelism from one instruction stream, designers replicated cores and relied on software to use them.

Power, dark silicon, and the end of easy scaling

As transistors shrank, the power density of chips rose faster than cooling could handle, and designers could no longer switch on all the transistors at full speed at once15. The phenomenon was named dark silicon in a 2011 paper by Michael Taylor and colleagues: a growing fraction of a chip's area would have to be powered down or run slowly at any given time. Microarchitecture responded with power-aware design — dynamic voltage and frequency scaling, clock gating, power gating of idle units, and heterogeneous cores that pair high-performance and low-power designs. The ARM big.LITTLE architecture, introduced in 2011, was one commercial expression of this idea.

Domain-specific accelerators

The most recent turn is toward specialization. If general-purpose cores can no longer get much faster, the alternative is to build hardware tailored to particular workloads. Graphics processing units were the first widely deployed accelerators, and their microarchitecture — many simple lanes executing in lockstep — influenced later designs. More recently, machine-learning accelerators such as Google's Tensor Processing Unit (deployed in 2015) built systolic arrays of multiply-accumulate units for neural-network inference, and field-programmable gate arrays and fixed-function blocks for video, cryptography, and networking became standard parts of system-on-chip designs. The microarchitectural question shifted from "how do we make one core faster?" to "which computation deserves its own silicon, and how does it connect to the rest?"

Where microarchitecture stands

Today's processors combine nearly every technique above: superscalar, out-of-order cores with deep pipelines, multi-level caches, sophisticated branch predictors, SMT, and multiple cores on a die, often alongside specialized accelerators. The open questions are shaped by the history: how to keep improving performance when frequency and voltage scaling have stalled; how to design secure speculation after side-channel attacks showed that speculative execution leaks secrets; how to manage power and heat across heterogeneous chips; and how to make accelerators programmable without losing their efficiency. The discipline that began by microcoding control signals now spends much of its effort deciding what not to compute, and where to put the silicon that remains.

References

  1. Microarchitecture (introduction, Relation to instruction set architecture, Multicycle microarchitecture, Instruction pipelining, Aspects, Branch prediction, Increasing execution speed, Cache, Out-of-order execution, Register renaming, Multiprocessing and multithreading), Wikipedia
  2. Control unit (Hardwired control unit), Wikipedia
  3. Microassembler, Wikipedia
  4. Microcode, Wikipedia
  5. Hazard (computer architecture), Wikipedia
  6. Cache hierarchy (Background), Wikipedia
  7. Out-of-order execution, Wikipedia
  8. Tomasulo's algorithm, Wikipedia
  9. Register renaming, Wikipedia
  10. Superscalar processor, Wikipedia
  11. Branch predictor, Wikipedia
  12. SWAPGS (security vulnerability) (introduction, History), Wikipedia
  13. Simultaneous multithreading, Wikipedia
  14. Multi-core processor, Wikipedia
  15. Dark silicon, Wikipedia

The frameworks of Microarchitecture

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1950-1985

    Hardwired control

    How are control signals generated for each instruction?

    Fly to its tower in Nousfarer

  2. 1951-1985

    Microcode

    How can control logic be made regular and changeable?

    Fly to its tower in Nousfarer

  3. 1961-Present

    Instruction pipelining

    How can several instructions be overlapped in execution?

    Fly to its tower in Nousfarer

  4. 1964-1995

    Dynamic scheduling and scoreboarding

    How can instructions execute as soon as their operands are ready?

    Fly to its tower in Nousfarer

  5. 1964-Present

    Out-of-order execution

    How can instructions execute out of program order safely?

    Fly to its tower in Nousfarer

  6. 1965-Present

    Cache hierarchy

    How can a fast processor be fed by slow memory?

    Fly to its tower in Nousfarer

  7. 1980-Present

    Reduced instruction set computer

    What instruction set makes fast pipelines practical?

    Fly to its tower in Nousfarer

  8. 1985-Present

    Branch prediction and speculative execution

    Which instruction should be fetched after a branch?

    Fly to its tower in Nousfarer

  9. 1990-Present

    Superscalar processor

    How can several instructions start in the same cycle?

    Fly to its tower in Nousfarer

  10. 1995-Present

    Simultaneous multithreading

    How can one core keep busy on several programs at once?

    Fly to its tower in Nousfarer

  11. 2000-Present

    Power-aware microarchitecture

    How can performance be gained within a power budget?

    Fly to its tower in Nousfarer

  12. 2005-Present

    Hardware acceleration

    When should specialized hardware do a computation instead of a general processor?

    Fly to its tower in Nousfarer

  13. 2005-Present

    Multi-core processor

    How can many processors share one chip?

    Fly to its tower in Nousfarer

  14. 2007-Present

    Domain-specific accelerator microarchitecture

    How should hardware be tailored to one kind of workload?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Hardwired controlMicrocodeInstruction pipeliningDynamic scheduling and scoreboardingOut-of-order executionCache hierarchyReduced instruction set computerBranch prediction and speculative exe…Superscalar processorSimultaneous multithreadingPower-aware microarchitectureHardware accelerationMulti-core processorDomain-specific accelerator microarch…

Beyond the field

Where Microarchitecture's ideas came from in other fields, and what they led to there: the strongest 12 of 61 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Microarchitecture is a planet in Nousfarer

Its city has a tower for each of these 14 frameworks. Land, walk in, and climb one concept at a time.

Land on Microarchitecture

Related fields

Fields its ideas are linked to

More in Computer Architecture

See where Microarchitecture sits on the Map of Human Knowledge