New technology for interactive CAL: the Origami project

Origami is a three-year EPSRC project that forms part of a general research programme on humancomputer interaction. The goal of this research is to investigate and implement new methods for human-computer interaction, and to apply and evaluate their use. The research centres on the DigitalDesk, an ordinary desk augmented with a computer display using projection television and a video camera to monitor inputs. The DigitalDesk allows electronic and printed documents to be combined to give richer presentation and interaction possibilities than are possible with either separate medium. This paper examines the implications of such a system for CAL, and presents two prototype applications that demonstrate the possibilities.


Introduction
For many years, almost since the dawn of computing, people have speculated about the paperless society.Each new technology is heralded as the one that will finally take us to the fully electronic Utopia.However, the use of computers as document preparation tools has actually increased the number of paper documents.The latest mass-market development, the Internet, is unlikely to buck this trend even though the tools used seem to encourage on-screen browsing.The problems inherent with screen presentation have not gone away, while the amount of information available to the average computer user is increasing all the time.Education is no exception; students too are encouraged to create documents as part of more active learning programmes, building an awareness of the importance of presenting their own ideas to others as well as helping to meet targets for the integration of technology into teaching.But CAL is about more than just having better tools to create more attractive documents.
The arrival of multimedia-capable computers has provided an opportunity for the integration of new media types into documents.These media types range from sound, video and typed text to fully interactive computational tools such as virtual worlds.However, multimedia technology has its problems: (i) video frame rates and resolution are often low; (ii) audio quality is often poor; (iii) text is hard to read from computer displays; (iv) the document must be viewed with a suitable computer.
These are problems of perception, rooted in our experience of existing single-medium documents.We have got used to broadcast video, CD audio, good-quality colour imaging and high-resolution text rendering.It seems likely that the advances being made in software and hardware will fairly quickly remove the barriers presented by (i) and (ii) through better compression and more bandwidth.The typical display and speaker system of a modern multimedia PC are already good enough to deliver the fruits of these advances.On the other hand, solutions to (iii) and (iv) do not appear to be just around the corner.This threatens to divide multimedia documents into two types: those that are delivered in a virtual world (by computer) and those that are delivered in the real world (by paper).
The typical solution to this division is to create a metaphor which reduces the gap between the real and virtual worlds -hence the desktop metaphor.Many CAL experiments have involved placing material which is available to the student in printed form online in an attempt to take advantage of the added features that a computer text can have -for example hypertext links and search engines for navigation -as well as using this as the starting point for adding multimedia content to the original publication and linking it to computer-based instructional resources such as self-assessment engines.See Rush (1995) for a recent example.

DigitalDesk
The philosophy of the Origami project is that a significant improvement to this situation can be made by using a human-computer interaction technique known as computeraugmented environments (Robinson, 1995) to allow publications which sit uneasily in the computer domain to have the best of both worlds.Rather than create virtual books, we seek to unite the value-added functionality of computer-based texts to real paper documents and ultimately to the student's working environment itself via the DigitalDesk (Wellner, 1991;Wellner, 1993).
In the 1970s, Xerox PARC developed the desktop metaphor which made computers easier for many people to use by making them look and act like ordinary physical desks.Electronic documents could be manipulated like paper documents, but the computer added powerful new capabilities.Recently, a joint project between the Computer Laboratory at the University of Cambridge and Rank Xerox EuroPARC resulted in the development of BrightBoard (Stafford-Fraser and Robinson, 1996), a computer-augmented whiteboard, and prototype DigitalDesks (Wellner, 1994).Instead of making the computer act like a desk, the DigitalDesk makes the desk act like a computer.With a DigitalDesk, the user can do things with ordinary paper that normally can be done only with electronic documents on a screen.
The DigitalDesk is built around an ordinary physical desk and can be used as such, but it has extra capabilities.A video camera is mounted above the desk, pointing down at the work surface.This camera's output is fed through a system that can detect where the user is pointing (using an LED-tipped pen) and it can recognize the documents that are placed on it.The more advanced version also has a computer-driven projector mounted above the desk enabling electronic objects to be projected onto real paper documents, thus removing the burden of having to switch attention between screen and paper and allowing additional user-interaction techniques.It will be some time before such systems become widely available, but workstations with built-in support for video input have been available for many years and have now even found their way into the personal computer market. • The techniques employed by the DigitalDesk take advantage of the fact that most written materials are already created in electronic form.Sadly, much of the richness of this form is thrown away when the document is printed, but the DigitalDesk attempts to utilize this extra information to augment the paper copy.

Educational applications
The DigitalDesk is an ideal way to advance the computer-illustrated text (CIT) concept (Harding and Quinney, 1990) which sought to combine the strengths of the traditional printed textbook with those of the computer as an educational medium.The earliest CITs were printed books with software, the use of which was an integral part of the learning strategy of the package as a whole.They covered subjects in mathematics, computing and physics, areas where screen presentation is particularly difficult but where interactive graphic displays and dynamic computation can considerably enhance the learning activities.Of course, computers have come a long way since that series was published.The Renaissance project (Harding et al, 1995) experimented with electronic delivery but more recently Calculus Connections (Harding and Quinney, 1995), which involves video-clips, sounds and sophisticated computer animation, is still centred on the printed course book.
When graphical user-interfaces were first introduced to desktop computers they were principally aimed at document generation.In order to help the user transfer existing skills in handling paper documents to the computer, the desktop metaphor was invented.An integral part of the resulting systems was the printer which allowed the user to transfer the finished document back to the real world, throwing away properties (such as the ability to edit) which were no longer needed.
Although document generation is an important part of educational computing, the real explosion in computer use has come from the ability to interact with multiple media types, to gain access to information resources that exist outside the user's immediate 'desktop', and to participate in new styles of communication.As before, real-world metaphors have been applied in order to assist the use of these new applications.However, the problem with these new media types is that printing them causes them to lose properties which are still useful.The DigitalDesk is effective for these applications because its purpose is to retain these useful properties and, where possible, to add them to documents which were not even created on the computer.

Working prototypes
To illustrate the DigitalDesk's contribution, two illustrative scenarios are presented here.These scenarios represent possible uses of software that has already been developed for the DigitalDesk as part of the current Origami project.The first covers a broad range of computing applications and is used to set the scene for the second, an example aimed specifically at education.

Augmenting paper documents
Consider a user at the DigitalDesk browsing the World Wide Web, with the current page projected onto the desk surface in front of her.She selects links to follow by pointing with a special 'active' pen and pressing a button located near the tip.Having discovered a page of interest, she uses the projected print button to create a hard-copy with all the familiar and desirable qualities of paper: readability, tactile sensation and portability.She takes this paper copy away with her to read on the way home.The next day she returns to her desk with the paper copy of the page she printed the day before, marked up with her own notes.When she places it on the DigitalDesk it is recognized as the page that was printed the day before, suitable browsing software is launched and the same set of buttons that are available when browsing the electronic version are projected onto the header of the paper version and the links automatically become active.She can then continue to work with the document.

Computer-aided assessment
Now consider a mathematics student working with his course book at the DigitalDesk.The software which accompanies the course book is automatically launched when the book is .When he has finished, he clicks a projected button to ask the computer to assess his sketches.The sketches are analysed for features such as maxima, minima and axis crossings, and marked accordingly.

Future developments
Hardware and software have already been developed to meet the requirements of the above scenarios.For the future, the DigitalDesk provides exciting possibilities for the development of educational computing projects.However, it is a research project; it will be some time before computer-augmented environments make their way onto the average laboratory desktop.Ten years ago, the same could have been said for desktop video.
In the meantime, when designing applications, it is useful to break down the technology into several levels of increasing technical sophistication: • printed materials only; • printed materials augmented by traditional tools such as pen and paper, hand calculator, etc.; • printed materials augmented by a computer system (with keyboard and pointing device); • printed materials augmented by a computer system fitted with a page-recognition device (possibly driven by a hand-held device such as a simple bar-code reader); • printed materials augmented by a computer system with page recognition and a pen/finger-driven locator; • printed materials used in the context of the fully featured DigitalDesk with page recognition, locator tracking and projection.
Materials that are flexible enough to be used at any of these levels would be useful even if everyone had access to a fully functional DigitalDesk, since there are bound to be times when the complete system is not available.By designing computer-aided software packages with this in mind, it is hoped that a greater continuity can be provided between the student's experience in the computer laboratory and his or her own private time spent revising and reflecting on the subject matter.
figure /:The DigitalDesk is built around an ordinary physical desk figure 3:7?ie D/g/to/Desk brings paper publications to life first placed on the desk.During a section on curve-sketching, he reads about polynomials.The generic equation of a quadratic polynomial is given with spaces for the values of the coefficients and an empty box underneath for plotting the graph.The software projects default values and draws the graph into the box.However, it also projects controls alongside the coefficients to allow him to change these values while observing the corresponding change in the graph (perhaps in response to a rhetorical question like: What happens if you change c?) On the next page of his course book, he is faced with an assessment exercise.This time the polynomials are fixed and he must draw the curve into the box (the active pen does of course have a real nib for writing).When he has finished, he clicks a projected button to ask the computer to assess his sketches.The sketches are analysed for features such as maxima, minima and axis crossings, and marked accordingly.