Finely integrated media for language learning

FLUENT, an immersive foreign-language learning environment, was developed without recourse to hypermedia techniques. Nevertheless, if one accepts the premisses, proposed in this paper, on which the idea of hypermedia has been constructed, FLUENT shows a strong relationship with it. The paper discusses this relationship after attempting to distil the essence of educational hypermedia, and after presenting a taxonomy of media for language learning.

The essence of hypermedia for learning is the integrated use of multiple media, with associated activities in the various media that promote implicit learning and student initiative.
Close reading reveals four assertions in the foregoing, which I now take up briefly.
First, meaningfully integrated use should at least be possible, for otherwise the only benefit of making the media all available at once would be a slight improvement in convenience.For the whole system to exceed the sum of its media, there must be some kind of potential synergy, even if the responsibility of finding and exercising it falls to the student.
Next, technology-based activities for foreign-language learning range from computerization of traditional syntactic exercises to the literarily sophisticated, interactive videodisc story worlds of the Athena Language Learning Project (Murray, 1987).A middle ground is the hypertext translation environment of the TIGER project (Deepwell, 1991).
Third, implicit or indirect learning means getting away from explicit statements about the way things are and letting students pick up an understanding of the target subject matter in the context of a meaningful activity.The importance of this kind of 'situated' learning has been argued for education in general (Brown et al, 1989).In foreign language circles, the 'communicative' approach stresses the role of a meaningful situation, enabling the learner to acquire the regularities that grammars codify, without explicit mention of grammatical rules.
The last feature is student initiative.Allowing the student to get involved in deciding what to do may have the cognitive benefits of both active learning and the development of study skills, not to mention the personal benefit of self-expression.A cautionary note is in order, though: while the efficacy of active learning is well-established, it will not do for designers to abandon their responsibility to provide adequate guidance, leaving the student afflicted with 'cognitive agoraphobia' (Twidale, 1991) or bogged down in a corner of the curriculum.

Media for language learning
Turning from hypermedia to media, we present a simple taxonomy, summarized in Figure 1 opposite.Note that there is no reference to particular pieces of equipment, nor to modalities like music, for which there is at most secondary use in the foreign-language context.Our approach to foreign language immersion consists of conversations that take place in a supporting visual context that helps the student to understand remarks that would otherwise be slightly too difficult.In addition, the student can use interactive graphics for replies, while gaining exposure to new words and grammatical patterns.Briefly put, this means two-modality, two-directional communication.
The figure shows the two modes and the two directions, and indicates further subdivisions pertinent to each.One modality is language, of course, and the other is the spatial world that most of us (sighted people) receive by vision and which a computer may present in two dimensions by means of graphics algorithms.The two directions of language performance are comprehension and production, analogous to the familiar computerese terms input and output.The usefulness of this taxonomy will become clear in the final section of this paper, where it becomes a means to express design choices for our FLUENT immersion system, to which I now turn.The FLUENT immersion project

Language
The idea of immersion is to help the student understand and use the new language directly and automatically, not mentally to translate or compute grammatical formulae.To this end, we get the computer to provide some of the immersion activities that good language tutors often use, employing the student's own language only to set the stage, and leaving the explicit teaching of grammar to others.(On the reasonableness of this approach, see Richards and Rodgers, 1986.Also, note that it can be used with other systems -and teachers and booksthat complement it.)Immersion must be gradual, with exposure and comprehension preceding generation, and with new words and patterns introduced one at a time in situations where they can be figured out from the visual activity and the continuity of the conversation.
Let us see how these ideas play out in one of the microworlds that we have implemented (Hamburger and Maney, 1989;Hamburger and Hashim, 1991).Upon entering Washroom World, the student sees on the screen a drawing of a bathroom, suitably equipped.There is also a person with a movable hand that the student can manipulate with a mouse to pick up the soap or toothbrush, turn on the water, and so on.In one activity, the student's job is to make the figure go through the moves of washing its own face, while the tutor makes relevant comments and suggestions, all in Spanish.This activity promotes recognition of language, but other exercises check the student's language production ability, and leave it to the tutor (system) to make screen moves.In either case, the tutor is directly responsive to whatever the student has just done or said.The responses can vary somewhat, according to different levels of general achievement the student may have attained.
The most distinctive characteristic of FLUENT, is the very tight integration between the two modes (language and spatial) and between student and system.Another important aspect is that the screen actions influence and reflect the workings of an internal model, representing the current situation and the properties of objects.Having such a model enables the tutor to describe an action or its resulting state or even to critique an action in terms of a common-sense goal structure.Thus if the student tells the figure to wipe his face, the response may be: "But I don't have the towel."The current implementation does these things with limited flexibility.The next version, now under construction, is making more extensive use of the techniques of artificial intelligence, including reasoning and especially natural-language processing.It will also draw upon techniques of intelligent tutoring systems to guide the pedagogical decisions.

Bringing it together
I can now fulfil the promise to relate FLUENT to individual media and hypermedia, perhaps shedding some light on them all.
First, referring to the media taxonomy of Figure 1, I will mention a few design issues for FLUENT, touching on each modality and direction.Speech would be ideal for FLUENT, but on the input side, recognizers are expensive and cannot readily deal with students' poor pronunciation.On the output side, inexpensive generators have a robotic sound that one hardly wants a student to mimic, and canned speech prohibits generativity.We plan to continue with text but will experiment with a compromise of cutting and pasting recorded speech, using multiple prosodic patterns for each recorded word or phrase.
For spatial output, we have opted for generated images rather than recorded (photographed) ones, sacrificing visual fidelity in favour of close interaction between the student and the electronic tutor.Showing movement involves another compromise, one between still and moving graphics, a technique we call partial animation.This approach is exemplified by the screen figure whose hand is manipulable but who is otherwise stationary, and a scene where some objects are movable.Reaction from test users, even including a five-year-old, suggests that this approach can be fully engaging.
As for language structure, however, compromise is far less in order.Since the system's very purpose is to convey, albeit implicitly, the structure and vocabulary of a foreign language, it would be hobbled from the start if it did not handle language accurately and broadly.This entails using detailed, formalized language knowledge in a generative fashion.Therefore, although the current system gets by with simple pattern-based analysis of language input, and filling sentence templates with simple generated noun phrases on the output side, the new version is importing a very strong natural-language processing component.
In sum, although we do not bill FLUENT as a hypermedia system, it turns out to have all four characteristics proposed above as the essence of educational hypermedia.Certainly it thoroughly integrates the linguistic and the spatial media in a way that is essential to its goal.The second feature, the use of multiple and integrated activities, is also present.FLUENT provides activities for initial exposure to a microworld and its language aspects, as well as a variety of activities designed for comprehension, production, and assessment.Third, student initiative too is clearly in evidence.For example, in some activities the student controls the direction of conversation by making things happen with the mouse, while in others the system carries out student wishes expressed in language.Finally, the whole point of immersion is implicit learning.Abstaining from explicit statement of grammatical patterns, FLUENT instead provides a visual and linguistic environment in which those patterns should come to seem familiar and natural.

Figure 1 :
Figure 1: Summary of a simple media taxonomy