- Atlas
- Computer Science
- Virtual Reality
Virtual Reality
Virtual reality (VR) is the attempt to put a person inside a computer-generated world: to replace what the eyes, ears, hands, and body would otherwise perceive with synthetic signals, and to make the person feel present in a place that does not physically exist.
13 frameworks, from 1957 to today · Computer Graphics · Computer Science
- Multisensory immersion 1957-1970
- Flight simulation and military training systems 1960-Present
- Head-mounted display 1968-Present
- Wired glove 1982-Present
- Augmented reality 1990-Present
- Consumer VR product wave 1991-1999
- Cave automatic virtual environment 1992-Present
- Presence 1992-Present
- Haptic technology 1993-Present
- Mobile and standalone VR 2014-Present
- Room-scale and inside-out tracking 2015-Present
- Foveated rendering 2016-Present
- Neural rendering 2020-Present
Overview
Virtual reality (VR) is the attempt to put a person inside a computer-generated world: to replace what the eyes, ears, hands, and body would otherwise perceive with synthetic signals, and to make the person feel present in a place that does not physically exist. In computer graphics terms, VR is the hardest version of the rendering problem, because the image must be produced not for a passive viewer but for an active participant, fast enough that the world appears to hold still as the head turns. The history of the subfield is therefore a history of two intertwined struggles: building displays and trackers that can fool the senses, and building graphics systems that can keep up with a human body in motion.
Sensorama, Sutherland, and the first head-mounted displays
The earliest working prototypes came from outside academic computing. In the late 1950s and early 1960s Morton Heilig, a cinematographer, built the Sensorama, a coin-operated cabinet that played stereoscopic film with stereo sound, wind, vibration, and smells to a single seated viewer. It was immersive cinema rather than interactive simulation, but it established the multisensory ambition that would recur throughout the field.
The decisive step toward interactive VR was Ivan Sutherland's 1968 "Sword of Damocles" head-mounted display at Harvard and the University of Utah. The display was suspended from the ceiling because it was too heavy for a wearer, and it showed simple wireframe rooms seen from a viewpoint that updated with head position12. Its importance was conceptual: it demonstrated that a computer could render a world that responded to where the user looked, the founding claim of the head-mounted display paradigm. Around the same time, flight simulation was already using wide-field visual systems and motion platforms to train pilots, and the military services became the field's most reliable early patron, funding head-tracked displays and helmet-mounted sights through the 1970s.
The 1980s: gloves, goggles, and the naming of the field
The 1980s turned prototypes into a recognizable research programme. At Bell Laboratories in the early 1980s, Gary Grimes and colleagues built a data glove that measured finger bending, giving the hand a way to act inside a virtual world. At NASA's Ames Research Center, Scott Fisher led the Virtual Interface Environment Workstation project, which combined a wide-angle stereoscopic head-mounted display, head and hand tracking, and 3D audio into a single system. Jaron Lanier's company VPL Research, founded in the mid-1980s, commercialized the DataGlove and the EyePhone headset and popularized the term "virtual reality."13 The era's characteristic claim was that presence—the felt sense of being somewhere else—could be engineered by saturating the senses and coupling them to the body's movements.
The graphics bottleneck was severe. Rendering two eyes at interactive rates with 1980s hardware meant crude geometry and low resolution, and latency between head motion and image update caused nausea. This is where the field's debt to computer graphics became explicit: VR's demands drove work on real-time rendering, stereo display, and low-latency tracking that fed back into graphics generally.
Projection, CAVEs, and the multisensory turn
A competing answer to the head-mounted display emerged in the early 1990s. At the University of Illinois at Chicago, Carolina Cruz-Neira, Daniel Sandin, and Thomas DeFanti built the CAVE, a room whose walls were rear-projection screens showing a stereo image computed for the viewpoint of one tracked viewer wearing shutter glasses. The CAVE traded the headset's intimacy for a wide field of view, low weight, and the ability to share the space with colleagues—an early form of what would later be called social presence. Projection-based immersion became a standard tool for engineering visualization, architectural walkthroughs, and scientific data exploration.
In parallel, researchers pursued multisensory simulation beyond sight and sound: haptic and force-feedback devices such as the PHANToM arm (developed at MIT in the early 1990s) let users feel virtual surfaces, and work on spatial audio, smell, and vestibular cues explored how far the illusion could be pushed. The recurring finding was that presence is not simply a function of pixel count; coherence between what the user does and what the senses report matters more than raw fidelity.
Presence research and the science of "being there"
By the late 1990s a research community had formed around measuring presence itself. The journal Presence: Teleoperators and Virtual Environments, founded in 1992, became its forum, and researchers developed questionnaires and physiological measures to test when users felt transported. The presence research programme reframed VR from a display technology into a psychological phenomenon, and it produced the field's most durable lesson: the goal is not photorealism but perceptual and behavioral plausibility. This insight later shaped everything from rendering priorities to the design of social VR.
The consumer boom and bust
The 1990s saw a wave of consumer VR products—Sega's and Nintendo's headset plans, the Virtuality arcade systems, and a flood of low-cost goggles—backed by a media narrative of imminent mass adoption. Most failed commercially. The displays were heavy, resolution was low, tracking was imprecise, and content was thin. The bust taught the field that the head-mounted display paradigm required components that did not yet exist: cheap, high-resolution microdisplays, low-latency inertial sensors, and graphics chips fast enough to render stereo at frame rates that would not make users sick. The research continued in universities and military labs through the 2000s, but the consumer market went quiet.
Mobile, room-scale, and the second wave
The turnaround came from smartphones. The 2010s put high-resolution displays, tiny inertial measurement units, and powerful mobile graphics into millions of pockets. The Oculus Rift, whose 2012 Kickstarter campaign re-energized the field, and later the HTC Vive and PlayStation VR, combined these components with improved head tracking. A parallel path used phone-based viewers such as Google Cardboard (2014) and Samsung Gear VR to bring cheap VR to a mass audience, and standalone headsets later removed the phone and the cable.
Two technical shifts defined this second wave. First, inside-out and room-scale tracking: instead of external cameras or lighthouses, headsets used onboard cameras and inertial sensors to track their own position and map the room, letting users walk and gesture in a bounded physical space. Second, foveated and perceptually driven rendering: because the eye sees sharply only near the center of gaze, systems could render the periphery at lower resolution and save the compute needed for high frame rates—an idea that became practical once eye-tracking hardware matured. These advances made the head-mounted display paradigm finally comfortable enough for sustained use.
Augmented and mixed reality
Alongside VR, a related line of work sought to overlay synthetic imagery on the real world. Early head-mounted see-through displays appeared in the 1960s and 1970s, and the term "augmented reality" was coined in the early 1990s by Boeing researchers working on aircraft assembly. The 2010s brought marker-based and then markerless tracking on phones, and head-worn systems such as Microsoft's HoloLens (2016) pursued mixed reality—virtual objects anchored in the physical room. Augmented and mixed reality share VR's tracking and rendering problems but add the harder task of registering synthetic content precisely against a real, moving scene.
Neural rendering and the present
The most recent turn comes from machine learning. Neural rendering methods learn to synthesize images from data—photographs, scans, or captured light fields—rather than describing scenes by hand. Neural radiance fields (NeRF), introduced in 2020, and later 3D Gaussian splatting made it possible to reconstruct photorealistic, view-dependent scenes from a handful of photographs, and to render them at interactive rates. For VR this is significant because it lowers the cost of building believable virtual environments and offers new ways to capture real places for immersive replay. The open questions are the ones the field has carried since Sutherland: latency, comfort, field of view, and the stubborn gap between a convincing picture and a convincing sense of being there.
References
- Virtual reality (20th century, Etymology, 1970–1990), Wikipedia
- Head-mounted display (History), Wikipedia
- VPL Research, Wikipedia
The frameworks of Virtual Reality
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1957-1970
Multisensory immersion
Can a machine surround a viewer with convincing synthetic senses?
- 1960-Present
Flight simulation and military training systems
How can simulation train people for dangerous real tasks?
- 1968-Present
- 1982-Present
- 1990-Present
- 1991-1999
Consumer VR product wave
Can immersive headsets be sold to ordinary people?
- 1992-Present
Cave automatic virtual environment
Can a room of screens immerse a whole viewer?
- 1992-Present
- 1993-Present
- 2014-Present
- 2015-Present
Room-scale and inside-out tracking
How can a headset know where it is in a room?
- 2016-Present
Foveated rendering
How can VR render fast enough without wasting pixels?
- 2020-Present
Neural rendering
Can learned models synthesize believable views from data?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Head-mounted display influenced by Flight simulation and military training systems
Military flight simulation supplied the wide-field visual displays, head tracking and helmet-mounted sights that Sutherland and later HMD builders drew on; the services funded head-tracked display work through the 1970s.
Augmented reality grew out of Head-mounted display
Augmented reality grew out of head-mounted see-through displays, keeping the headset and head-tracking of the HMD paradigm but overlaying imagery on the real world rather than replacing it.
Consumer VR product wave grew out of Head-mounted display
The 1990s consumer wave was the commercialization of the head-mounted display paradigm: Sega, Nintendo, Virtuality and low-cost goggles all sold headset-based VR to the public.
Cave automatic virtual environment rival of Head-mounted display
The CAVE was built as an alternative to the head-mounted display, trading the headset for a room of projection screens to give a wide field of view, low weight and shared viewing.
Carried by Carolina Cruz-Neira; Daniel Sandin; Thomas DeFanti.
Mobile and standalone VR grew out of Head-mounted display
Mobile and standalone VR are the head-mounted display paradigm rebuilt on phone components: cheap microdisplays, tiny inertial measurement units and mobile graphics.
Foveated rendering influenced by Head-mounted display
Foveated rendering is the HMD paradigm's answer to its graphics bottleneck: because the headset must render two eyes at high frame rate, it renders the periphery at lower resolution once eye-tracking hardware matured.
Room-scale and inside-out tracking grew out of Mobile and standalone VR
Inside-out and room-scale tracking grew out of mobile VR's onboard cameras and inertial sensors, letting headsets track their own position and map the room without lighthouses.
Beyond the field
Where Virtual Reality's ideas came from in other fields, and what they led to there: the strongest 12 of 33 links.
Room-scale and inside-out tracking influenced by Visual SLAM in Robot Perception And Slam
Inside-out tracking in consumer VR headsets was built directly on visual SLAM methods developed in robotics, using camera-based mapping and localisation to track the headset in a room. The SLAM algorithms and their engineering practice passed into VR tracking systems.
Room-scale and inside-out tracking influenced by Visual-Inertial Fusion in Robot Perception And Slam
Visual-inertial fusion, combining cameras with inertial measurement units, is the core technique of inside-out head and controller tracking in room-scale VR. The robotics and SLAM literature on visual-inertial odometry supplied the estimators used in these systems.
Neural rendering influenced by Neural radiance field in Computational Imaging
Neural radiance fields, introduced by Mildenhall and colleagues in 2020, became the central technique of neural rendering: a multilayer perceptron maps position and view direction to density and colour, and volume rendering integrates it. Neural rendering frameworks built directly on this representation and its follow-ups.
Carried by Ben Mildenhall; NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (2020).
Multisensory immersion influenced by Widescreen Stereophonic Sound in Film Sound
Widescreen stereophonic cinema of the 1950s established the technical and aesthetic model of wrapping an audience in coordinated image and sound, which Heilig's Sensorama and later multisensory immersion systems explicitly emulated. The passage was the multi-channel peripheral presentation of image, sound and motion as a single immersive experience.
Carried by Morton Heilig.
Flight simulation and military training systems influenced by Stability, Control, and Fly-by-Wire in Aerospace Engineering
Aircraft stability and control theory, including the fly-by-wire developments, supplied the aerodynamic and control equations that flight simulators implemented in real time. Simulator fidelity was defined against this body of control engineering, and simulator engineers drew their models from it.
Flight simulation and military training systems influenced by Simulation and wargame modeling in Game Studies
Military wargaming and simulation modeling provided the institutional model of training personnel through simulated scenarios, which flight simulators extended to continuous physical fidelity. The lineage runs from nineteenth-century Kriegsspiel through operations-research wargaming to the military flight-training programs of the 1960s.
Head-mounted display influenced by Hidden-surface determination in Computer Graphics
Sutherland's head-mounted display work at Harvard and Utah depended on real-time computer graphics, and hidden-surface determination algorithms developed in the same graphics community made the stereoscopic scenes renderable. The display was an application of the emerging interactive graphics pipeline, including visibility computation.
Carried by Ivan Sutherland.
Head-mounted display influenced Wearable computer in Ubiquitous Computing
Head-mounted displays developed in virtual reality and military research became a core component of wearable computing systems. The optics and display techniques passed directly into body-worn computing.
Carried by Ivan Sutherland.
Wired glove influenced by Remote manipulator in Robotics
Teleoperation and remote manipulator research, notably at Argonne and later NASA, developed hand-controller and force-feedback technology that the wired glove adapted for measuring and displaying hand motion. The glove's sensing and actuation concepts came from this master-slave manipulator tradition.
Augmented reality influenced by Intelligence amplification in Interaction Design
Intelligence amplification, especially Engelbart's framework of augmenting human intellect with interactive computing, directly shaped the conception of overlaying computational information onto the user's immediate environment. Augmented reality researchers cited this augmentation agenda as their intellectual starting point.
Carried by Douglas Engelbart.
Augmented reality influenced Virtual and Augmented Reality Performance in Dance And Technology
Virtual and augmented reality performance took augmented-reality display and tracking techniques from computer-science AR research, using overlays of virtual content onto live performance space.
Presence influenced by Postphenomenology in Technology And Society
Presence research in VR drew on postphenomenological accounts of embodiment and technological mediation, especially work on how instruments shape perception, to theorise the sense of being in a virtual environment. Scholars of presence explicitly cited postphenomenology when analysing mediated experience.

Virtual Reality is a planet in Nousfarer
Its city has a tower for each of these 13 frameworks. Land, walk in, and climb one concept at a time.
Land on Virtual RealityRelated fields
Fields its ideas are linked to
- Graphics Rendering 5 links
- Dance And Technology 3 links
- Ubiquitous Computing 3 links
- Robot Perception And Slam 3 links
- Aerospace Engineering 2 links
- Computational Imaging 2 links
- Computer Graphics 2 links
- Flight Dynamics 1 link
- Computer Vision 1 link
- Film Sound 1 link
- Perception 1 link
- Computer Animation 1 link
More in Computer Graphics
- Computational Imaging
- Computer Animation
- Computer Graphics
- Game Technology
- Geometric Modeling
- Graphics Rendering
- Physical Simulation
See where Virtual Reality sits on the Map of Human Knowledge