Loading field map
Every CT scan, MRI, or PET image begins as raw sensor data—a set of measurements that bear little resemblance to the anatomy they represent. The task of turning those measurements into a coherent picture is image reconstruction, and it has always been a contest between competing demands: speed, because patients cannot hold still indefinitely; fidelity, because subtle lesions must not be lost; and, in modalities that use ionizing radiation, dose, because every exposure carries risk. The history of reconstruction is the story of three broad frameworks that each made different trade-offs among these demands, and that today coexist, compete, and increasingly hybridize.
For the first three decades of clinical computed tomography (CT) and for much of early MRI, reconstruction meant filtered back projection (FBP). FBP is an analytical method: it treats the reconstruction problem as a direct mathematical inversion of the measurement process. In CT, for example, the scanner records a set of line integrals (projections) through the patient. FBP applies a high-pass filter to each projection and then back-projects the filtered data into the image space, summing contributions from all angles to produce a cross-sectional slice.
The appeal of FBP was speed. Because the algorithm is deterministic and can be implemented as a single pass through the data, it ran in seconds even on the modest computers of the 1970s and 1980s. It became the clinical workhorse for CT, angiography, and nuclear medicine. Yet FBP had a fundamental limitation: it assumed that the measurement data were complete and noiseless. In practice, photon noise, scatter, and limited angular sampling introduced artifacts—streaks, blurring, and graininess—that degraded image quality, especially when clinicians tried to reduce radiation dose by lowering the tube current. FBP offered no principled way to incorporate prior knowledge about the scanner's physics or the patient's anatomy. By the late 1990s, as dose awareness grew, the pressure to find an alternative became acute.
Iterative reconstruction (IR) emerged alongside FBP in the 1970s but remained a research tool for decades because of its computational cost. Instead of a direct inversion, IR formulates reconstruction as an optimization problem: find the image that best explains the measured data according to a forward model of the imaging system. The algorithm starts with an initial guess, simulates what measurements that guess would produce, compares them to the actual data, and updates the image to reduce the discrepancy. This cycle repeats until convergence.
What distinguished IR from FBP was its ability to incorporate a physical model of the scanner—including noise statistics, detector response, and even the geometry of the X-ray beam. By modeling the measurement process more accurately, IR could suppress noise and artifacts without sacrificing spatial resolution. The critical clinical consequence was dose reduction: IR allowed CT scans to be performed at 30–60% lower radiation dose while maintaining diagnostic image quality. By the mid-2000s, commercial CT vendors began offering IR as an option, and it gradually displaced FBP for most body CT applications.
Yet IR did not fully replace FBP. For applications where speed was paramount—such as real-time fluoroscopy or cardiac imaging—FBP remained in use because IR's iterative loop required seconds to minutes per slice. The two frameworks coexisted for years, with FBP handling high-throughput, low-dose-sensitive tasks and IR reserved for dose-critical studies. This division of labor reflected a deeper philosophical divide: FBP trusted a fast, closed-form solution; IR trusted a slower, model-based optimization that could incorporate more of the scanner's physics.
The 2010s brought a third framework: deep learning reconstruction (DLR). Rather than handcrafting a forward model or a filter, DLR learns a mapping from raw data (or from an initial FBP/IR image) to a high-quality reconstruction using large datasets of paired low-quality and high-quality examples. Early DLR methods acted as post-processing denoisers, cleaning up FBP images to reduce noise while preserving edges. More recent architectures, such as unrolled networks, embed the iterative structure of IR inside a neural network, learning the update steps themselves. These hybrid models absorb the logic of IR—the forward model, the data consistency term—while replacing the handcrafted regularizer with a learned one.
The advantage of DLR is dramatic speed and quality. A well-trained network can reconstruct a CT slice in milliseconds, matching FBP's speed while exceeding IR's noise reduction. In MRI, DLR has enabled accelerated acquisition by reconstructing undersampled k-space data that would produce aliasing artifacts under conventional methods. The cost, however, is a new set of uncertainties. DLR models are data-driven: their performance depends on the training distribution, and they can fail unpredictably on out-of-distribution anatomy or pathology. Unlike FBP and IR, whose behavior is governed by explicit mathematical models, DLR introduces a black-box element that challenges clinical validation and regulatory approval.
Today, all three frameworks remain active, but their roles have shifted. FBP persists in legacy systems and in applications where speed and simplicity are non-negotiable, such as real-time interventional guidance. IR is the standard of care for routine diagnostic CT, especially in pediatric and screening protocols where dose minimization is paramount. DLR is rapidly entering clinical use, often as an enhancement layer on top of IR or as a full replacement in MRI acceleration.
The most interesting developments lie at the boundaries. Unrolled networks, for instance, are a direct hybridization: they preserve the iterative structure of IR but learn the regularizer from data. This approach narrows the gap between model-based and data-driven reconstruction, combining the interpretability of IR with the performance of DLR. Similarly, some commercial CT systems now offer a continuum of reconstruction options, letting radiologists choose between FBP, IR, and DLR for each clinical task.
Despite this convergence, a fundamental disagreement persists. Proponents of model-based methods (IR) argue that explicit physics guarantees reliability: if the forward model is correct, the reconstruction is trustworthy. Proponents of DLR counter that learned models can capture complex noise and artifact patterns that no handcrafted model can, and that rigorous testing on diverse datasets can provide equivalent assurance. The debate is not merely academic—it shapes regulatory standards, clinical deployment strategies, and the direction of research funding.
What the leading frameworks agree on is that no single approach is sufficient for all clinical scenarios. Speed, dose, and fidelity remain in tension, and each framework makes a different cut through that trade-off space. The future of image reconstruction is likely to be pluralistic: hybrid algorithms that combine analytical, iterative, and learned components, selected adaptively based on the imaging task, the patient, and the clinical context.