Loading field map
Image reconstruction is the branch of medical imaging concerned with converting raw measurements acquired by a scanner into a usable visual representation of the body's interior. Every tomographic imaging modality—computed tomography (CT), positron emission tomography (PET), single-photon emission computed tomography (SPECT), and magnetic resonance imaging (MRI)—collects data not as an image but as a set of indirect measurements, often called projections or k-space samples. The reconstruction problem is to estimate the spatial distribution of some physical property—X-ray attenuation, radiotracer concentration, or nuclear spin density—from these measurements, which are related to the desired image by a known mathematical model.
The stakes are practical and clinical. Reconstruction determines image quality: spatial resolution, contrast, noise, and the presence of artifacts. It also determines radiation dose in CT and nuclear medicine, because lower-dose scans produce noisier measurements that require more sophisticated reconstruction to remain diagnostically useful. In MRI, reconstruction speed affects whether a scan can be completed in a breath-hold or in real time for cardiac or interventional applications. The field therefore sits at the intersection of applied mathematics, signal processing, and clinical radiology, and its central questions concern how to invert an ill-posed measurement process reliably, quickly, and with quantifiable confidence.
All tomographic modalities share a common structure: the scanner measures line integrals or spatial frequency samples of the object, and the reconstruction algorithm must invert this transformation. In CT, an X-ray source and detector array rotate around the patient, measuring the attenuation of X-rays along thousands of straight lines through the body. The set of these line integrals is the Radon transform of the attenuation coefficient distribution. In PET and SPECT, detectors record gamma rays emitted by a radiotracer; each detected pair of coincident photons in PET defines a line along which an annihilation occurred, and the data are again line integrals, though degraded by attenuation, scatter, and random coincidences. In MRI, the situation differs: the scanner measures samples of the spatial Fourier transform of the proton density, modulated by relaxation times. The measurement space, called k-space, is filled by applying magnetic field gradients that encode spatial position into the phase and frequency of the received signal.
The common mathematical difficulty is that the measurement operator is not directly invertible in practice. The data are finite, noisy, and incomplete. In CT, only a finite number of projections are acquired, and each projection is sampled at finite detector resolution. In MRI, k-space is always undersampled relative to the Nyquist criterion when scan time is limited. In PET, the number of detected coincidence events is limited by the injected dose and acquisition time, so the data are Poisson-distributed counts with low signal-to-noise ratio in many regions. The reconstruction problem is therefore an inverse problem: given a forward model that predicts measurements from an unknown image, estimate the image that best explains the observed data, subject to constraints that encode prior knowledge about what medical images look like.
The earliest and conceptually simplest approach to reconstruction is analytical. The central result is the filtered backprojection (FBP) algorithm for CT, developed in the 1960s and 1970s. The mathematics rests on the Fourier slice theorem, which states that the one-dimensional Fourier transform of a projection at a given angle equals a line through the two-dimensional Fourier transform of the object at that same angle. If projections are available at all angles, the object's Fourier transform is fully known, and the image can be recovered by inverse Fourier transformation. In practice, the data are discrete and finite, so the algorithm is implemented by filtering each projection with a ramp filter—which compensates for the nonuniform sampling density in Fourier space—and then backprojecting, or smearing, each filtered projection back along its original ray paths and summing over all angles.
Filtered backprojection is fast, linear, and well understood. Its limitations are equally clear. The ramp filter amplifies high-frequency noise, so low-dose CT scans reconstructed with FBP appear noisy. The algorithm assumes ideal line integrals and does not naturally accommodate physical effects such as beam hardening, scatter, or the finite size of the X-ray source. It also requires a complete set of projections; if the angular sampling is sparse, streak artifacts appear. For these reasons, FBP served as the workhorse of CT for decades and remains in clinical use, but it is no longer the state of the art for challenging acquisitions.
In MRI, the analogous analytical approach is the inverse Fourier transform of the k-space data. When k-space is fully sampled on a Cartesian grid, this is exact and fast. The practical problem is that full sampling is time-consuming, and the Fourier approach cannot directly handle non-Cartesian trajectories such as spirals or radial lines without interpolation, which introduces artifacts. The analytical framework also provides no natural way to incorporate prior knowledge or to reconstruct from undersampled data.
The recognition that reconstruction is an inverse problem led to a different family of methods: iterative reconstruction. Instead of attempting a direct inversion, these algorithms start with an initial estimate of the image, compute what measurements that estimate would produce using the forward model, compare them with the actual measured data, and update the image to reduce the discrepancy. The process repeats until convergence.
The earliest iterative methods, developed in the 1970s for emission tomography, were algebraic reconstruction techniques (ART) and their variants, which treated the problem as solving a large system of linear equations. These were computationally expensive and initially slower than FBP, but they had a crucial advantage: they could incorporate a more accurate forward model. In PET and SPECT, the forward model could include attenuation correction, scatter estimation, and the Poisson statistics of photon counting. The maximum likelihood expectation maximization (MLEM) algorithm, introduced in the 1980s, explicitly modeled the data as Poisson random variables and sought the image that maximized the likelihood of the observed counts. Its accelerated successor, ordered subset expectation maximization (OSEM), became the clinical standard in nuclear medicine.
In CT, iterative reconstruction gained clinical traction much later, in the late 2000s, when computing power made it practical. The key development was the addition of a regularization term to the objective function. Because the inverse problem is ill-posed—many images can explain the noisy data nearly equally well—the algorithm must favor certain solutions. Regularization encodes prior knowledge, typically that the image is piecewise smooth: it penalizes large differences between neighboring pixels while allowing sharp edges at tissue boundaries. This approach, sometimes called model-based iterative reconstruction (MBIR), produces images with lower noise than FBP at the same radiation dose, or equivalent noise at lower dose. The cost is computation time and a dependence on the choice of regularization strength, which affects the texture and apparent sharpness of the image.
The relationship between analytical and iterative methods is not one of simple replacement. FBP remains in use for many routine CT scans because it is fast, deterministic, and well calibrated. Iterative methods are preferred for low-dose protocols, for obese patients where photon starvation is an issue, and in PET where statistical noise dominates. Modern clinical scanners often offer both, and radiologists must learn to interpret images with different noise textures produced by different reconstruction methods.
A major conceptual advance came from the application of compressed sensing theory to MRI, beginning around 2006. The insight is that if the image is sparse—that is, representable with few nonzero coefficients in some transform domain, such as wavelets or finite differences—then it can be reconstructed accurately from far fewer measurements than the Nyquist criterion requires. The reconstruction is formulated as an optimization problem that minimizes a sparsity-promoting penalty, typically the L1 norm of the transform coefficients, subject to consistency with the measured k-space data.
This approach transformed the practice of MRI by enabling significant undersampling. Instead of acquiring every line of k-space, the scanner can acquire a random subset, and the reconstruction algorithm fills in the missing data by exploiting sparsity. The clinical benefit is faster scans, which improve patient comfort, reduce motion artifacts, and enable dynamic imaging of the heart or contrast uptake. The same principles have been applied to CT, where sparse-view acquisitions reduce radiation dose, and to PET, where they improve image quality at low count statistics.
Compressed sensing is not a rival to iterative reconstruction but a specific instance of it: the forward model is the undersampled Fourier transform, and the regularization is the sparsity penalty. Its importance lies in making explicit the trade-off between measurement strategy and reconstruction. The choice of which k-space lines to acquire, or which projection angles to use, is now understood to be part of the reconstruction problem, not a separate data-acquisition concern. This has led to joint design of acquisition and reconstruction, sometimes called integrated or end-to-end optimization.
The most recent major development is the application of deep learning to image reconstruction, which began in earnest around 2016 and has since become a dominant research area. Deep neural networks are trained on large datasets of paired low-quality and high-quality images—for example, low-dose CT reconstructed with FBP and the corresponding normal-dose images, or undersampled MRI and the fully sampled reference. The network learns a mapping from the degraded input to the desired output, effectively learning a prior from data rather than from hand-crafted regularization.
Several architectural strategies exist. Some methods use a neural network as a post-processing step, taking an FBP or iterative reconstruction and refining it. Others unroll an iterative algorithm: each iteration of a model-based reconstruction is replaced by a network layer, so the network learns the update steps and the regularization implicitly. Still others use generative models that learn the distribution of plausible images and reconstruct by finding an image that both matches the measurements and lies on the learned manifold. A common theme is that the forward model is retained—the network is constrained to produce images consistent with the measured data—while the prior is learned.
The appeal of deep learning is dramatic: these methods often produce images with lower noise and higher apparent resolution than iterative reconstruction, at much lower computational cost at inference time. The concerns are equally significant. The behavior of a trained network on inputs outside its training distribution is not guaranteed, and subtle artifacts can be introduced that mimic real pathology or obscure it. The field is actively debating how to validate these methods, how to characterize their uncertainty, and whether the learned prior can be trusted for clinical decisions. Regulatory approval has been granted for some deep-learning reconstruction products, but the standard of evidence remains an open question. The relationship to earlier approaches is one of layering: deep learning does not replace the forward model or the inverse-problem formulation but provides a new way to learn the regularization and the inversion.
The field today is characterized by coexistence rather than succession. Analytical methods remain in use where speed and simplicity matter. Iterative and model-based methods are standard for low-dose and photon-limited acquisitions. Compressed sensing is routine in MRI protocols that exploit undersampling. Deep learning is rapidly being integrated into commercial scanners, often combined with iterative or compressed-sensing frameworks rather than replacing them entirely.
Several cross-cutting concerns shape current practice. One is the need for quantitative accuracy: reconstruction is not only about producing a pleasing image but also about preserving the numerical values that clinicians use for diagnosis, such as CT attenuation numbers (Hounsfield units) or PET standardized uptake values. Another is robustness: algorithms must perform consistently across patients, body regions, and scanner hardware. A third is computational efficiency, since clinical workflows demand reconstruction times of seconds to minutes. The field also increasingly emphasizes uncertainty quantification—providing not just a single image but an indication of which features are reliable and which are artifacts of the reconstruction process.
The central intellectual tension remains the same as it has been since the 1970s: how to balance fidelity to the measured data against the assumptions used to fill in missing information. Every reconstruction method embodies this trade-off differently. Analytical methods make minimal assumptions but require complete data. Iterative methods make explicit statistical and smoothness assumptions. Compressed sensing assumes sparsity. Deep learning assumes that the training data are representative. The choice among them is a clinical and scientific judgment, informed by the specific imaging task, the available data quality, and the consequences of error.