Virtual reality: Theoretical basis, practical applications

Virtual reality (VR) is a powerful multimedia visualization technique offering a range of mechanisms by which many new experiences can be made available. This paper deals with the basic nature of VR, the technologies needed to create it, and its potential, especially for helping disabled people. It also offers an overview of some examples of existing VR systems. DOI:10.1080/0968776930010103


Introduction
An important property of computer systems is their ability to visualize information.A graphics program, for example, can turn a mass of numeric data into a digestible, pictorial representation of it.But faster processing speeds, and a good measure of human inventiveness, now allow us to take such visualizations much further, offering users many new types of experience.Such visual experiences form the basis of the virtual-reality (VR) systems described in this paper, which is intended to fulfil a four-fold objective: to explain the principles of VR to provide a perspective of the basic technologies on which VR environments depend to describe some current VR systems to discuss some potential applications of VR with reference to the problems of disabled people.

Basics of virtual reality
A VR system is a sophisticated multimedia environment in which users are exposed to, and can participate in, surrogate tacto-audio-visual experiences.These experiences are created by means of a computer system to which are attached special types of peripheral device, enabling users to interact with the real and artificial objects that exist within the interaction environment.

Degrees of realism
As individuals we exist in a world that continually presents us with new experiences, and one way of learning to deal appropriately with them is to generate them artificially.Many areas of education benefit from the use of pseudo-real situations which enable students to learn in a risk-free environment.Examples of available techniques include role-playing, models and simulations, surrogations, synthetic actors, microworlds, artificial-reality kits, and VR systems.Each of these techniques allows a different degree of realism to be achieved.Part of the attraction of artificial and virtual realities is that they allow us to go beyond the limits of what is real (Smith, 1986;Kelly, 1989;Fisher and Tazelaar, 1990;Magnenat-Thalmann and Thalmann, 1989).

Terminology and definitions
Books and other literature dealing with VR use a range of new terminology.Four important ones for the purposes of this paper are artificial reality, virtual reality itself, cyberspace and cyberspace deck.
Artificial reality, as defined by Krueger (1991), perceives a participant's action in terms of a body's relationship to a graphical world, and generates responses that maintain the illusion that his or her actions are taking place within that world, whereas (for Krueger) virtual reality is another term for artificial reality that applies only to systems implemented with goggles and gloves, the special peripherals that enable the user of VR to perceive the virtual world and to interact with it.
Again according to Krueger, cyberspace is a global artificial reality that can be visited simultaneously by millions of people.However, a more useful definition (Helsel and Paris Roth, 1991) is: an interactive simulation which includes human beings as necessary components.Walser (1991) suggests that cyberspace is a form of theatre which 'can be regarded as a computer-based simulation that enables groups of people to play roles of characters in cybernetic simulations of three-dimensional worlds: crucially, cyberspace gives the role players the ability to sense a virtual reality from the point of view of the characters they play.'A cyberspace deck, as defined by Helsel and Paris Roth, is simply 'a gateway through which people are transported to cyberspace'.But according to Walser, the term should be used to refer to 'the physical space containing the array of instruments which enable a player to act within, and feel part of, a virtual space.'This last definition is important since it introduces the need for access to suitable equipment in order to experience VR worlds.However, all the above the definitions can be used to make up the description of a 'systems model' on which to base a discussion of VR.

A systems model
An appropriate systems model has been proposed by Walser.The model is based on a cybernetic feedback loop between 'puppets' (which operate in virtual space) and 'patrons' (essentially, users who operate in physical space).Puppets within the virtual space embody intellects which may be either artificial or human.They acquire 'knowledge' of events that take place in physical space by means of devices (called sensors).Puppets also have a range of output devices (called effectors) with which they can influence physical space.A puppet's patron can influence virtual space by means of its sensors and can learn about events taking place in virtual space by means of its effectors.A cybernetic loop therefore arises because a puppet's sensors are a patron's effectors, and a puppet's effectors are a patron's sensors.
The feedback loop implicit in this systems model is a fundamental part of all the VR systems described later in this paper.

Equipment needs -cyberspace decks
For Walser, a cyberspace deck is the generic name given to the equipment needed to reach, and travel within, a cyberspace system, with most decks having a fundamental architecture usually consisting of seven basic components: (i) a cyberspace 'engine* for generating a simulated world and for mediating the patron's interaction with it; (ii) a 'control space' (a section of physical space) in which the patron's movements are tracked; (iii) a set of sensors to monitor the patron's actions and body functions; (iv) a set of effectors which can be used to produce various physical effects and stimulate the patron's senses; (v) a set of 'props' used to give the patron solid analogues of virtual objects and vehicles; (vi) a 'network interface' which can be used to admit other patrons to the simulated world; and (vii) an 'enclosure' (consisting of some sort of physical framework) which can be used to house all the components listed above.
Obviously, the exact nature of a cyberspace deck will vary from application to application, and the particular components of any given deck will also strongly influence the nature of the virtual realities that can be created.Some examples of cyberspace decks which have recently been mentioned in the VR literature are briefly outlined below.Autodesk Inc. in the USA have built a number of prototype cyberspace decks (such as HiCycle described below) that are oriented towards athletical applications.In one of their decks they have linked together three computer systems: a Compaq 386 PC (which runs the simulation), a Macintosh (used as a source of video images), and an Amiga (for sound effects).The computers are linked together in various ways using serial communication ports and an Ethernet local-area network.Each computer connects to the patron by means of an umbilical link which provides the high-speed data highways that are necessary for the patron's helmet and data glove.The prop (a sort of conveyor belt upon which the patron walks) is connected to the simulation machine by standard parallel and serial data links.
In the UK, two interesting VR systems have recently become commercially available: Vision and Virtuality.Vision depends essentially on the use of transputers for its computing power, but it also uses Intel i860 pipelined processors for image generation and rendering (Pountain, 1991).Vision is described as a VR server since it can be attached to a host computer such as an IBM PC, a Sun Sparcstation or a VAX.Virtuality, produced by W Industries, comes in two forms -a sit-down console, and a system based on a helmet and a glove which is designed to be used by a person standing or moving within a scanned area.
Of course, the availability of single-user decks marks only the beginning of what will eventually become much more complex multi-user systems -the ability to connect cyberspace decks via local-area or wide-area networks will enable multi-person spaces to be created.Furthermore, through networking it will also becomes possible for decks to have access to multiple cyberspaces, thereby enabling patrons to 'jump' from cyberspace to cyberspace.

Quality of experience
The quality of a VR system can be measured in terms of its 'distance' from a real environment from the point of view of its patron.Naturally, a patron's satisfaction or otherwise with a VR experience will depend on a large number of factors.However, two important considerations are deck ergonomics and psycho-semantics.The subject of deck ergonomics is concerned with issues such as how comfortable the VR equipment is to wear and the adverse physical and/or mental effects it might have on the human body.The subject of psycho-semantics is concerned with the quality and meaning of the artificial worlds generated, thus with issues such as the ways in which the brain handles illusion, and the problems of dissonance that may arise as a result of experiences in different cyberspace environments.However, because VR systems are so new, little work has yet been done in this area.

A historical perspective
Five major technologies have influenced VR, and in all likelihood will continue to do so.They are: television; remote control; 3D graphics and animation; display technology; and 3D human-computer interfaces.

Television
A major feature of television is that it enables us to observe remote events and so allows us to experience the phenomenon of 'tele-presence' -in other words, to be 'present' in any location in which a TV camera can be placed.A conventional TV broadcast does not of course involve a viewer in the kind of direct interaction required by a VR system.

Remote control
Closed-circuit TV, on the other hand, can offer levels of interactivity unavailable in conventional broadcast TV, for example in situations where the viewer can control the orientation of the TV camera.In order to achieve this, it is necessary to provide some sort of remote-control facility such as a joystick, a steering wheel, or -looking to VR systems -a device worn on the head, allowing the viewer to control the displayed view.The importance of remote-control technology in this context lies in the fact that when it is coupled with tele-presence it allows the implementation of tele-operation and tele-robotics (Iyengar and Kashyap, 1989).In many cases, through the design of suitable effectors, these techniques can enable a patron to interact with the events being viewed, and are therefore of importance to designers of VR systems.Remotely-controlled vehicles fitted with robot arms and TV cameras, which can be used in normally inaccessible or hazardous environments, are technically feasible with current technology.Remote-control technology thus enables us to create artificial experiences in which a person can participate interactively.

Graphics and animation
Clearly, although it might be technically feasible, it would not be a practical proposition to provide everyone who wants to explore the world outside their living-room with a remotely-controlled go-where-you-please robotized vehicle and TV camera.Thus if virtual worlds are to be generally available, some alternative mechanism has to be used.A number of approaches are possible.For example, video recordings of distant locations can be stored on videodisc or compressed and stored on CD-ROM.Indeed, this technique is quite widely used to create surrogate experiences, particularly for training purposes.Alternatively, images can be generated in real time using computer-graphics techniques (Magnenat-Thalman and Thalman, 1989; Upstill, 1990;Sharp, 1990), and most of the currently available VR systems depend very heavily on such computational graphics.Unfortunately, however, extremely powerful computers are needed in order to create images which have the quality of those we continually perceive through our natural visual system.The high-speed parallel computers needed to produce images of this sort would be far too expensive for widespread use in VR systems.Consequently, a compromise has to be achieved: image quality has to be sacrificed in order to meet realizable costs.Within a VR system the major function of the graphics and animation software is to generate 3D graphical worlds within the memory space of the host computer.These worlds will contain two basic types of object: static and animated.The static objects provide the contextual environment.The animated objects move around the 3D world interacting with each other and with the static objects.But unlike the objects we perceive in our real world, the objects in artificial realities can be given any special properties the VR designer may wish them to have.

Display technology
The commonest form of display is a simple flat screen of the everyday computer-monitor variety, but there are three major disadvantages associated with it.First, the range of colours and resolution that can be achieved is relatively low unless expensive equipment is used.Secondly, it is difficult to project to the viewer a feeling of three-dimensional realism.
Thirdly, extraneous events and objects within the environment in which the display is located can exert a distractive influence on the viewer, thereby reducing the ambience of the VR.
There are a number of ways of overcoming each of these problems through the use of special display technologies (Tello, 1988;Laurel, 1990;Krueger, 1991).In particular, various types of helmet and goggles can exclude the viewer's local environment, thus reducing the chance of interference between reality and artificial reality.Such head-mounted devices provide a stereoscopic display system that projects a wide-angle 3D panorama directly into the viewer's retina.The particular view observed can be controlled by head movements, voice commands and hand gestures.Equipment of this sort allows exploration of a spherical field of view which can be derived from either a simulated or a remotely-sensed environment.

Human-Computer Interfaces
In order, to interact with the real or the imaginary objects contained within a VR system, special types of 3D device are needed.The best known kinds of interaction peripheral for use in VR work are data gloves, body suits and various types of hand/foot control (Eglowstein, 1990;Laurel, 1990).
The function of a data glove and its supporting hardware/software is to provide data that will enable hand position and orientation, in 3D space, to be calculated.The glove must also provide data that reflects the relative position and crooking of the fingers.If two data gloves are used simultaneously (one on each hand), further data can be made available which gives the position of one hand relative to that of the other.The data produced by data gloves is usually stored and processed in the form of movement-pattern vectors (Morita, Hashimoto and Ohteru, 1991).Because of the amount and complexity of the data produced, neural networks are often used to analyse and interpret the meaning of hand gestures.
Several types of glove are currently available, the most common being the DataGlove produced by VPL Research in California.This glove embeds a 3D location sensor, and uses optical-fibre cabling to detect finger movement.Another well-known example is the Dextrous Hand Master produced by Exos, which uses an intricate exoskeletal arrangement of sensors that fit over the back of the hand.The sensors are held over each finger joint by light-weight pads and Velcro straps, and each sensor houses both a Hall-effect magnetic pick-up and a magnet.
The type of glove used in any given VR situation will depend on the particular functions to be performed.For example, in some of our work we have been exploring the use of low-cost data gloves for developing simple gestural control languages based on finger-touching operations involving the simultaneous use of two gloves.

Examples
Certain attempts to generate experiences of artificial realities were made decades ago.This section of the paper describes some operational VR systems, both old and new, built either for research purposes or for public use.

Sensorama
Sensorama is a multi-sensory simulation environment developed by Morton Heilig as early as the mid-1950s (Krueger, 1991;Laurel, 1990), though it is still operational today.It was designed as an arcade game in which the participant sat on a seat, held a pair of motorcycle handlebars and pressed his or her eyes against a pair of binocular-like stereo-mounted lenses.With this equipment it was possible for the user to see and listen to a participative 3D movie taken at eye-level.While the movie was playing, the seat and handlebars vibrated and wind was blown into the face of the 'rider', creating the illusion of driving around a city.This illusory experience could be reinforced by the introduction of aromas (the smell of exhaust fumes and pizzas) at appropriate points during the presentation.

Metaplay
Metaplay, dating from the early 1970s, was designed by Krueger (1991) as a multi-person experience within a 'responsive art' exhibit for use in a public gallery.As people flowed into the exhibition area, they were filmed using a closed-circuit video system.This enabled their moving images to be relayed back to them and displayed on a large screen by means of a back-projection system.While this was happening, an artist in another building could use a digital data tablet and stylus in order to sketch objects which could be super-imposed on the screen people were watching.By observing and monitoring the actions of particular people in the audience, the artist could give individuals the illusion that they could draw or create 'live graffiti' by moving their hands, head or body.

Videoplace
Videoplace is another example of an early artificial reality system designed by Krueger (1991).Fundamental to its design is the creation of a 'shared place' that two or more people can experience.Videoplace is similar to Metaplay in that participants' video images form part of a synthesized graphic world, but Videoplace is based on the use of two-way video to create a shared visual environment in which participant's images can interact with each other and with objects contained within the shared world.Since the visual world is computer-generated, the objects within it need not bear any relationship to objects encountered in the real world.The Videoplace system was used to explore all manner of interaction, and to create many different types of illusory experience based on the body gestures made by participants.Typically, these illusions could involve such activities as dancing, swimming, flying, and participating in various types of sporting activity.

Fluxbase
Fluxbase is another example of a virtual exhibit (Helsel and Paris Roth, 1991).Essentially, it is a virtual environment for generating contemporary dynamic and exploratory works of art.To the person using it, Fluxbase appears as a 'box' which can be opened by clicking a mouse button, and which contains a wide assortment of virtual objects such as postcards, poems, sketches, film-strips, slides; musical scores, a pack of playing cards, games, puzzles, audio-tape recordings, bits of string -indeed, anything that may be used to create a piece of contemporary art.Objects can be 'pulled out' of the box and explored by means of mouse-based pointing operations.Poems can be read, paintings can be examined in various ways, slides and film-strips can be watched, and audio recordings listened to.The artist's role in Fluxbase is to decide upon and create the objects which will go in the box, while that of the patron (the user) is to experience and explore the virtual objects contained in it.

HiCycle
Autodesk's HiCycle involves the use of a bicycle mounted within a rig connected to a control computer.The computer monitors the pseudo-motion of the bicycle and in response is able to generate various types of special effect.The cyclist wears a helmet similar to those described in section 3.5 above, which provides the field of view.If the cyclist pedals rapidly enough, the bicycle 'takes off and 'flies' over a graphic landscape.
A number of similar VR systems based on the bicycle approach have been created.For example, one of the participative exhibits at Panasonic's exhibition centre in Tokyo enables the cyclist to explore various cities by 'pedal', though in this system the cyclist watches images on a screen retrieved from videodisc rather than wearing a helmet, and the bicycle is incapable of 'flight'

VIEW
The Aerospace Human Factors Research Division of NASA's Ames Research Centre has developed VIEW (Virtual Interactive Environment Workstation) which is used in a range of experiments related to space research, including investigations into tele-presence, teleoperation, tele-robotics, and virtual data-display facilities for spaceships and space stations (Helsel and Paris Roth, 1991;Laurel, 1990).Fundamental to the VR environments created at the Ames Research Centre is the use of a data glove and a head-mounted wide-angle stereoscopic display system controlled by its wearer's position, voice and gestures.Using this basic equipment, VIEW is able to provide a multi-sensory interactive display for exploring a 360-degree synthesized or remotely-sensed environment.Space-research experiments in which this equipment has been used include a tele-robotic device which mimics its human controller's actions, and various simulations of different types of interaction environment for pilots of spaceships and lunar vehicles.

The potential of VR
The majority of VR applications described in the published literature rely heavily on a user's visual channels of perception.However, some years ago we became interested in exploring the possibility of generating virtual realities for blind people which rely on sound effects and tactile communication methods.As a result of our work, we introduced the concept of talking tactile books for blind people (Barker, 1991a).These books were of a multimedia nature and depended on two basic technologies for their fabrication: Braille and sound recorded on CD-ROM.The 'pages' of Braille could be mounted on various types of touch-sensitive surfaces to be used for information output and input.Depending on the effects to be produced, two pages could simultaneously be 'read' and touched -one with each hand.While tactile interaction was taking place, and in response to it, audio narrations and sound effects could be retrieved from the CD-ROM.An important outcome of this work has been the design and testing of a prototype data glove to facilitate the computer interpretation of simple gesticulative sign languages using a neural network system.Although there is yet much work to be done with this equipment, we feel that the progress made to date and its potential in supporting the generation of VR environments for the disabled is encouraging.
VR technologies will undoubtedly have a significant impact over the next decade in terms of the potential benefits they can bring to disabled people in areas such as learning, new kinds of sensory and cognitive experience, new kinds of communication aids, training facilities, and various kinds of spin-off development.

Learning environments
One of the attractive features of VR as a learning tool is its ability to display objects and situations not normally visible to humans, and to enable humans to interact with them -for example, by reaching out to 'touch' the atoms of a complex molecule.Just as easily, it could convert a wheel-chair into a vehicle capable of allowing its occupant to explore, for example, virtual safari parks or cities anywhere in the world.Equally, because they are based on sophisticated simulations, VR systems also enable us to explore learning domains by means of What-If experiments, the results of which will stimulate the development of cognitive models and mental skills that could be applied in many situations.Thus, future VR systems will undoubtedly offer exciting ways of learning, of course including experiential learning.

Generating new experiences
Normally, our ability to encounter new types of physical and cognitive experience depends on our ambulatory capability and the power of the perceptual tools we have available.But VR can provide mobility in situations where there may be immobility, and through the use of special peripherals and special effects, can generate sensation where previously there may have been none.In short, VR systems can produce a wide range of new experiences for people who would normally not be able to encounter them.Of course, the events that create the experiences will be of an illusory nature, but the effects they produce can be as stimulating and as exciting as real events.

New communication aids
The development of new types of technology for sensing body movement and stimulating the senses of sight, sound and touch are making possible many new types of communication aid.The data glove and body suit have already been referred to.Using this type of equipment it is possible to design an almost infinite variety of languages for communication based on almost any form of body movement, gesture, mime, touch or sonic utterance.Indeed, the technology on which VR is based makes it possible for each individual to have access to a 'private' language that can then be mapped to one or more commonly accepted public language systems.Our work with the low-cost tactile data glove illustrates how easily this can be done, and similar work has been carried out by Kramer and Leifer (1988) using a 'talking glove'.Although these are not direct applications of VR, they are important research spin-offs.Using VR itself as a total personal communication medium may be a future goal, though it has to be said that the concept of the VR phone is some way off.

Training facilities
We are increasingly turning to the use of computer-based technologies to provide training facilities (Barker, 1991b;Barker, 1993), and given the nature of the simulated environments they are capable of producing, VR systems are becoming an important knowledge and skill transfer tool for training people to perform complex and difficult tasks such as driving a car, piloting an aeroplane, and performing delicate surgical operations (using simulated patients).VR environments can be used to develop and practise inter-personnel skills such as public speaking, interviewing, participation in group discussion, and speaking, understanding and reading foreign languages.VR systems are also being used to provide rehabilitation environments in which to re-develop basic cognitive and motor skills after a debilitating accident or traumatic experience.And there is significant potential for training systems based on VR within the area of medical diagnosis.It would clearly not be sensible to say that VR offers limitless opportunities for training, but over the next decade, it will certainly be used in an increasing number of applications in this area.

Spin-off areas
A characteristic of technological research is that developments in one area often lead to spin-off developments in other areas, and such spin-off developments can sometimes be a way of broadening the application base for a product, thereby reducing the price at which it can be made available to end-users.The data glove that forms such an important part of many VR systems provides one example.The original DataGlove developed by VPL Research is a complex and expensive interaction peripheral (Eglowstein, 1990).However, a similar peripheral (the Power Glove) developed by Mattel (for use in home computers and video games) is far less expensive because of the much larger target market and the reductions in cost that can be accrued as a result of automated mass production.

Conclusion
Important, interesting and exciting research is currently being undertaken in the development and application of VR, much of which will lead to useful products.Currently, the majority of VR systems are expensive and therefore beyond the reach of ordinary users, However, low-cost, less functional systems could be and must be developed, in particular for disabled people, systems which will enable them to participate in the new and unusual experiences which VR promises.