The basic idea
A virtual reality system tries to place a person inside a computer-generated or recorded scene. To do that convincingly, it needs three things: a display for each eye that fills much of the field of view, sensors that track how the head moves, and a computer that redraws the scene fast enough to match. Each of these was difficult for early engineers, which explains much of the history below.
Before headsets: immersive machines
Some early ideas used entire rooms or arcade-style machines rather than headgear. In the late 1950s and early 1960s, cinematographer Morton Heilig developed the Sensorama, a booth in which a person watched a film while experiencing vibration, wind and smells. It was not interactive and not head-mounted, but it aimed at the same goal of surrounding the senses. Heilig also patented a head-mounted display concept in the early 1960s.
Other early work was military and industrial. In the 1960s, a company developed a helmet-mounted display linked to a remote camera, so that an operator could look around by moving the head. Pilots and researchers also explored helmet-mounted displays for aircraft. These systems were related to virtual reality, though they showed camera feeds or simple symbols and not computer-generated worlds.
The Sword of Damocles
A landmark came in 1968, when Ivan Sutherland and his students built a head-mounted display that showed simple wireframe graphics drawn by a computer and updated as the viewer’s head moved. The headset was so heavy that it was suspended from the ceiling by a mechanical arm, which gave the project its nickname, the Sword of Damocles. The imagery was crude, with outlines of simple shapes. Even so, it combined a stereoscopic display, head tracking and real-time computer graphics, the same basic recipe that later headsets use.
Research labs in the 1980s
During the 1980s, research at government and university labs continued, often with expensive hardware. NASA investigated virtual environments for training and for remote control of robots, an idea that links to modern space robotics. Companies and researchers built gloves and tracking devices to let users reach into virtual scenes. Jaron Lanier and his company are often credited with helping popularize the term “virtual reality” at this time, and with selling some of the first commercial headsets and gloves, which cost far more than most people or businesses could spend.
The 1990s hype
In the early 1990s, virtual reality drew wide media attention. Arcade-style systems let people wear a headset and play a multiplayer game in a few minutes of virtual combat or exploration. Several companies announced consumer headsets. Films and television shows showed the technology as imminent.
In practice, the products fell short. Screens had low resolution, so the world looked blocky. Graphics hardware was slow, and tracking was imprecise. Many users felt discomfort or motion sickness, which often happens when the picture lags behind head movements or does not match what the inner ear senses. The most famous consumer flop was a game console from 1995 that used red monochrome displays and was sold for a short time, partly because it was uncomfortable and not truly immersive. By the late 1990s, interest had cooled, and VR retreated to research, military simulators and specialized training.
Why VR stayed niche
Several reasons are usually given:
- Display quality. Sharp, bright small screens with wide viewing angles were costly. The workings of human eyesight, discussed in our article on the optics of vision, set demanding requirements for resolution and field of view.
- Latency. If the image takes too long to update after a head movement, the experience feels wrong and may cause nausea.
- Computing power. Drawing detailed scenes twice, once for each eye, at a high frame rate needs powerful processors.
- Cost and bulk. Headsets were heavy, wired to computers and priced for institutions, not families.
- Lack of content. Without many applications to run, buyers had little reason to purchase.
What changed
The mobile phone boom of the 2000s had an unexpected side effect. It made small, sharp, high-resolution screens and compact motion sensors cheap and plentiful, while semiconductor chips designed for graphics became more powerful. In 2012, a crowdfunded headset project drew new attention, and larger technology companies soon invested in consumer products. Later headsets added tracking of the controllers and eventually of the surrounding room without external cameras. Some of that tracking uses techniques related to computer vision.
Today, headsets are used for games, training, design visualization and some therapy research, and they continue to improve. Yet adoption has remained smaller than many forecasts, and how widely they will be used in everyday life is uncertain.
A long arc
The first virtual reality headsets show a familiar pattern: the core idea appeared in the 1960s, but practical, affordable versions needed decades of progress in displays, sensors and processing. A related concept, augmented reality, overlays information on the real world, and has a similar history. Like early video games, VR advanced when the surrounding technology was ready, not when the idea first appeared.