- Atlas
- Medicine & Health
- Image Reconstruction
Image Reconstruction
Image reconstruction is the step that turns what a scanner actually measures into a picture a clinician can read.
7 frameworks, from 1970 to today · Medical Imaging · Medicine & Health
Overview
Image reconstruction is the step that turns what a scanner actually measures into a picture a clinician can read. A CT scanner never sees a slice of a body; it records how much X-ray intensity survives a path through the patient, from hundreds of angles. An MRI scanner records radio signals emitted by hydrogen nuclei in a magnetic field1. Reconstruction is the mathematics that converts these indirect measurements—projections, frequency samples, detector counts—into an image of tissue. Without it, the hardware is a collection of sensors. With it, the measurements become anatomy.
What is at stake is everything downstream: whether a tumour is visible, whether a fracture is displaced, whether a scan can be done fast enough for a child to hold still or safely enough for a patient to receive less radiation. The history of reconstruction is a history of trading off image quality, acquisition speed, radiation dose, and computation—and of repeatedly finding that better mathematics could buy what better hardware could not.
The Geometry Problem and the First CT Reconstructions
The mathematical foundation came long before the medical application. In 1917 the Austrian mathematician Johann Radon proved that a two-dimensional function can be recovered from its integrals along all lines through it—that is, from a complete set of projections. This "inverse Radon transform" is the theoretical heart of CT2. For decades it was a curiosity in pure mathematics.
The practical breakthrough came from Godfrey Hounsfield, an engineer at EMI in Britain, who built the first clinical X-ray CT scanner and published results in 1972; he shared the 1979 Nobel Prize in Physiology or Medicine with Allan Cormack, who had independently derived the mathematics of reconstruction from projections in the early 1960s. Hounsfield's early scanners used an iterative algebraic method—guessing an image, comparing its computed projections to the measured ones, and correcting—because computing power was scarce. The images were slow to produce but showed soft tissue in cross-section for the first time.
Filtered Backprojection Becomes the Standard
The method that made CT practical was filtered backprojection (FBP). The idea: take each projection, apply a mathematical filter that sharpens it and suppresses blur, then "smear" (backproject) each filtered projection back across the image plane and sum them. Naive backprojection alone produces a blurry result—the classic star artifact—because each point is smeared along every line that passed through it. The filter corrects for this, and the result is a fast, direct, non-iterative reconstruction.
FBP's appeal was speed and predictability: it could be computed directly from the data with no iteration, and its behavior was well understood. Through the 1970s and 1980s it became the default for CT, and the same projection-slice logic was adapted to other geometries—fan-beam and cone-beam CT, and later the helical scanning that let a scanner sweep continuously as the patient moved through it. In nuclear medicine, analogous backprojection methods turned gamma-camera counts into tomographic images. FBP's limitation was equally clear: it assumes ideal, noise-free, complete data. When projections are few, noisy, or truncated—as in low-dose scans or limited-angle acquisitions—it degrades badly.
Iterative Reconstruction Returns
Iterative reconstruction addresses exactly that weakness3. Instead of inverting the data in one pass, it starts with an estimate of the image, simulates what measurements that estimate would produce (a "forward model"), compares to the actual measurements, and updates the estimate—repeating until the two agree3. The approach was used in the earliest CT scanners and in early emission tomography, but it was computationally expensive and largely abandoned when FBP proved fast enough.
It returned as computing power grew and as dose reduction became a priority. Iterative methods can incorporate a model of the imaging physics and of the noise statistics, so they can produce usable images from fewer or noisier measurements than FBP requires. In CT this allowed lower-dose protocols; in nuclear medicine, iterative methods such as the expectation-maximization algorithm (developed in the 1970s and 1980s for emission tomography) became standard because they handle the Poisson statistics of radioactive decay counts properly. The cost was computation time and a change in image texture that radiologists had to learn to read.
Statistical and Model-Based Reconstruction
The next step made the model explicit. Statistical reconstruction uses a likelihood model of the measurement noise—for example, that photon counts follow a Poisson distribution—so the algorithm knows which measurements are trustworthy and weights them accordingly3. Model-based reconstruction goes further, incorporating a physical model of the scanner itself: detector blur, beam hardening (the tendency of lower-energy X-rays to be absorbed preferentially, which distorts CT numbers), scatter, and system geometry. The reconstruction then solves an optimization problem: find the image most consistent with the measurements under these models3.
This family of methods—sometimes called model-based iterative reconstruction in CT—produced markedly better low-dose images than earlier iterative schemes, at the price of heavy computation and a somewhat "smoothed" appearance that raised questions about whether fine detail was being preserved or invented. The debate over image texture and diagnostic confidence became a recurring theme: a reconstruction that looks cleaner is not automatically more diagnostically faithful.
Parallel Imaging: Using Coils as Extra Information
Magnetic resonance imaging posed a different reconstruction problem. MRI data are acquired in "k-space," the spatial-frequency domain; an image is the Fourier transform of the acquired data4. Acquiring all of k-space is slow, and scan time is the central constraint in MRI—patients move, breaths happen, and long scans are costly.
Parallel imaging, developed in the late 1990s, exploited a hardware fact: modern MRI uses arrays of receiver coils, each with a different spatial sensitivity profile. If several coils each "see" the same anatomy with different sensitivity, their signals carry redundant but differently weighted information, and that redundancy can substitute for some of the k-space samples that would otherwise be needed. Methods such as SENSE (sensitivity encoding) and GRAPPA (a coil-by-coil reconstruction) reconstruct the missing data using coil sensitivity maps or coil-to-coil correlations. The result was faster scans—often two- to four-fold acceleration—and the approach became routine in clinical MRI. Its limitation: acceleration increases noise amplification and artifacts, and the coil sensitivities must be known or estimated accurately.
Compressed Sensing: Reconstructing from Fewer Samples
Compressed sensing, introduced in the signal-processing literature around 2006 by Emmanuel Candès, Justin Romberg, and Terence Tao (with related work by David Donoho), rests on a striking claim: if an image is sparse in some transform domain—meaning most of its coefficients are near zero—it can be reconstructed from far fewer measurements than classical sampling theory requires, provided the measurements are acquired in a suitable randomized or incoherent way and the reconstruction enforces sparsity.
Applied to MRI, this meant that undersampled k-space could be reconstructed by solving an optimization that balances data consistency against sparsity in a transform such as wavelets or total variation. The first demonstrations in MRI came in the late 2000s, and the approach enabled dramatic acceleration—dynamic cardiac imaging, whole-heart imaging, and faster pediatric scans among them. Compressed sensing also entered CT and other modalities. Its practical difficulties were real: reconstruction is iterative and slow, the choice of sparsity transform matters, and aggressive acceleration can produce blocky or over-smoothed images. It nonetheless reshaped how the field thought about sampling: the question became not "how many measurements do we need?" but "what do we know about the image that lets us use fewer?"
Deep Learning Reconstruction
The most recent turn replaces hand-designed priors with learned ones35. Deep learning reconstruction trains a neural network—often a convolutional network or, more recently, an unrolled network that alternates data-consistency steps with learned denoising—on pairs of images, typically high-quality reference reconstructions or fully sampled data and their undersampled or noisy counterparts. The network learns a mapping from degraded measurements to clean images.
The approach spread rapidly across modalities in the 2010s and 2020s. In MRI, learned reconstructions achieved acceleration factors beyond what compressed sensing delivered; in CT, deep-learning image reconstruction (often marketed under vendor-specific names) allowed low-dose scans with noise levels approaching or surpassing iterative methods; in nuclear medicine, learned denoising improved low-count PET. The appeal is speed at inference and strong performance on the kinds of images seen in training. The open questions are equally sharp: whether a network trained on one population, scanner, or protocol generalizes to another; whether it can hallucinate or remove genuine pathology; and how such reconstructions should be validated and regulated. These concerns have made the field's long-standing tension—between image that looks good and image that is faithful—more urgent than ever.
Where Reconstruction Stands Now
Today's clinical practice is a layered inheritance. Filtered backprojection remains a fast baseline and is still used where speed and predictability matter. Iterative and model-based methods are standard in low-dose CT and emission tomography. Parallel imaging and compressed sensing are routine in MRI. Deep-learning reconstruction is increasingly deployed, often as a learned component inside an iterative or data-consistency framework rather than as a pure black box3.
The field's open questions are continuous with its history: how to reconstruct from fewer and noisier measurements without losing or inventing diagnostic information; how to make learned methods robust across sites and populations; how to quantify uncertainty in a reconstructed image; and how to combine physical models with data-driven priors rather than choosing between them. The through-line is unchanged since Radon: the picture is never directly observed, only inferred—and the quality of the inference is what the reconstructionist is responsible for.
References
- Magnetic resonance imaging, Wikipedia
- Tomographic reconstruction, Wikipedia
- Iterative reconstruction (introduction, Basic concepts, Statistical reconstruction, Learned Iterative Reconstruction), Wikipedia
- K-space in magnetic resonance imaging, Wikipedia
- Deep tomographic reconstruction, Wikipedia
The frameworks of Image Reconstruction
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1970-Present
Iterative reconstruction
How can an image be estimated by repeated correction?
- 1972-Present
Filtered Backprojection
How can projections be inverted directly into an image?
- 1980-Present
Statistical Reconstruction
How should measurement noise be modeled in reconstruction?
- 1997-Present
- 2000-Present
Model-Based Reconstruction
How can scanner physics be built into reconstruction?
- 2006-Present
- 2016-Present
Deep tomographic reconstruction
Can networks learn the reconstruction mapping from data?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Filtered Backprojection replaced Iterative reconstruction
Hounsfield's first clinical CT scanner used iterative algebraic reconstruction, but filtered backprojection proved far faster and became the standard method through the 1970s, displacing iteration for routine CT.
Carried by Godfrey Hounsfield.
Statistical Reconstruction grew out of Iterative reconstruction
Statistical reconstruction grew out of iterative reconstruction by adding explicit likelihood models of measurement noise, notably the expectation-maximization algorithm for emission tomography developed in the late 1970s and 1980s.
Carried by Lawrence Shepp; Yoram Vardi.
Model-Based Reconstruction grew out of Statistical Reconstruction
Model-based reconstruction extended statistical reconstruction by adding a physical model of the scanner—detector blur, beam hardening, scatter, geometry—to the noise likelihood, solving a fuller optimization problem.
Deep tomographic reconstruction replaced Compressed Sensing
Deep learning reconstruction replaced hand-designed sparsity priors with learned priors, often in unrolled networks that alternate data-consistency steps with learned denoising, achieving acceleration beyond compressed sensing.
Model-Based Reconstruction grew out of Iterative reconstruction
Model-based iterative reconstruction in CT descends from the forward-model-and-update loop of early iterative reconstruction, now enriched with detailed system physics models.
Statistical Reconstruction reacted against Filtered Backprojection
Statistical reconstruction was motivated by the failure of filtered backprojection on noisy, count-limited emission data, where FBP's assumption of ideal complete data breaks down and Poisson statistics must be modeled instead.
Compressed Sensing influenced by Statistical Reconstruction
Compressed sensing reconstruction in MRI was framed as an optimization balancing data consistency against a sparsity prior, building on the statistical reconstruction tradition of solving inverse problems with explicit regularization.
Compressed Sensing influenced by Parallel Imaging
Compressed sensing MRI demonstrations in the late 2000s built on the undersampled k-space acceleration agenda opened by parallel imaging, and the two were often combined to push acceleration further.
Deep tomographic reconstruction influenced by Model-Based Reconstruction
Unrolled deep reconstruction networks inherit the model-based iterative structure of alternating forward-model data-consistency steps with a regularization step, with the regularizer now learned from data.
Beyond the field
Where Image Reconstruction's ideas came from in other fields, and what they led to there: the strongest 12 of 17 links.
Iterative reconstruction grew out of Inverse problem in Computational Imaging
Iterative reconstruction is a computational strategy for solving inverse problems, and the inverse-problem formulation (ill-posedness, regularization, data fitting) supplied its mathematical framing. The algebraic reconstruction and expectation-maximization algorithms of the early 1970s were presented explicitly as inverse-problem solutions.
Carried by Richard Gordon; Gabor Herman.
Model-Based Reconstruction grew out of Tomographic reconstruction in Imaging Physics
Model-based reconstruction is a branch of tomographic reconstruction that replaces analytic inversion with an explicit system model and iterative statistical estimation. It grew out of the tomography literature and its forward models.
Carried by Jeffrey Fessler.
Model-Based Reconstruction grew out of Tomographic Reconstruction in Inverse Problems
Model-based reconstruction descends from tomographic reconstruction theory, extending the forward-operator formulation with accurate system models and priors. The same projection geometry underlies both.
Carried by Jeffrey Fessler.
Iterative reconstruction grew out of Tomographic reconstruction in Imaging Physics
Iterative reconstruction is a branch of tomographic reconstruction that grew out of it, using algebraic and statistical methods to solve the same projection equations when analytic inversion is ill-posed. The two are related as general framework and specific method.
Iterative reconstruction influenced by Regularization Theory in Inverse Problems
Regularization theory, especially Tikhonov's work, gave iterative reconstruction its criteria for stopping and for stabilizing ill-posed updates, and penalty terms in iterative algorithms descend from it. The connection runs through the inverse-problems literature of the late 1960s and 1970s.
Carried by Andrey Tikhonov.
Statistical Reconstruction led to Computational and Model-Based Imaging in Imaging Physics
Statistical reconstruction methods, developed from the 1980s for emission tomography, provided the likelihood and prior models that model-based imaging generalised.
Statistical Reconstruction influenced Deep-learning reconstruction and analysis in Quantitative Imaging
Statistical reconstruction supplied the probabilistic measurement models and regularization ideas that deep-learning reconstruction extends with learned priors. Learned reconstruction methods were developed as data-driven successors to statistical iterative reconstruction.
Statistical Reconstruction influenced by Inverse problem in Computational Imaging
Statistical reconstruction framed tomography as a statistical inverse problem, borrowing the inverse-problem notions of ill-posedness and regularization and adding likelihood and prior models. The Bayesian and penalized-likelihood formulations of the 1980s came from that inverse-problems background.
Carried by Ken Sauer; Charles Bouman.
Parallel Imaging influenced Deep-learning reconstruction and analysis in Quantitative Imaging
Parallel imaging's undersampling-and-reconstruction framework, including learned or calibrated coil-sensitivity methods, provided a key problem setting that deep-learning reconstruction took up for accelerated MRI. Deep networks were applied to the parallel-imaging reconstruction task.
Parallel Imaging influenced Compressed sensing in Signals And Systems
Parallel imaging in MRI, especially SENSE and SMASH undersampling with coil sensitivity encoding, was a direct precursor that compressed sensing generalised to arbitrary sparse recovery.
Parallel Imaging influenced Hybrid and Multimodality Imaging in Medical Imaging
Parallel imaging accelerated MRI acquisition by using multiple receiver coils, a technique that was incorporated into hybrid and multimodality systems to keep scan times practical. The reconstruction methods of parallel imaging passed into combined modality workflows.
Parallel Imaging influenced Compressed Sensing in Computational Imaging
Parallel imaging in MRI, especially SENSE and GRAPPA, demonstrated that undersampled data could be reconstructed using coil-sensitivity priors, an early practical instance of the sparsity-and-priors recovery that compressed sensing formalized. The compressed-sensing MRI work built directly on this undersampling reconstruction tradition.

Image Reconstruction is a planet in Nousfarer
Its city has a tower for each of these 7 frameworks. Land, walk in, and climb one concept at a time.
Land on Image ReconstructionRelated fields
Fields its ideas are linked to
- Computational Imaging 5 links
- Imaging Physics 4 links
- Inverse Problems 3 links
- Quantitative Imaging 3 links
- Signals And Systems 1 link
- Medical Imaging 1 link
More in Medical Imaging
- Diagnostic Radiology
- Imaging Informatics
- Imaging Physics
- Interventional Radiology
- Medical Image Analysis
- Medical Imaging
- Molecular Imaging
- Nuclear Medicine
- Quantitative Imaging
See where Image Reconstruction sits on the Map of Human Knowledge