Possible versus desirable in instructional systems : who ' s driving ?

Although most instructional systems arise from enthusiasm for realizing an educational goal, all too often the educational problems the educator is trying to address are overshadowed by system design constraints. Often, the subordination of educational issues to implementation issues reflects an inadequate understanding of how pedagogic requirements map onto system design decisions and at what cost. It is all too easy to become engrossed in the challenges of implementation (in what is possible), especially if the pedagogic motivations (what is desirable) are not clearly established, and if their attendant instructional strategies are not firmly specified. Focusing on what is possible tends to lead to simple translations of existing materials into a new medium, whereas reflecting on what is desirable has the potential of identifying underlying goals and supporting them through transformation of practice into a form that exploits the particular advantages of an alternative technology.

This paper takes a pragmatic stance that the key to successful application of technology in education is good teaching: using technology only when it is a cost-effective servant of pedagogy.The paper discusses some fundamental issues in the production of computer-based materials, and considers them in the context of an on-going evaluation of an Internet courseware project.
Although most instructional systems arise from enthusiasm for realizing an educational goal, all too often the educational problems the educator is trying to address are overshadowed by system design constraints.Often, the subordination of educational issues to implementation issues reflects an inadequate understanding of how pedagogic requirements map onto system design decisions -and at what cost.It is all too easy to become engrossed in the challenges of implementation (in what is possible), especially if the pedagogic motivations (what is desirable) are not clearly established, and if their attendant instructional strategies are not firmly specified.Focusing on what is possible tends to lead to simple translations of existing materials into a new medium, whereas reflecting on what is desirable has the potential of identifying underlying goals and supporting them through transformation of practice into a form that exploits the particular advantages of an alternative technology.This familiar story is characterized in the experiences of the EONT project, described in this paper, which began as an initiative to transform paper-based materials into Internet courseware.These issues are examined in the context of a comparison of two generations of systems developed for an HCI course.The first system, although it set out with ambitions to address specific learning requirements, was overtaken by implementation issues.The specification and resulting product, a fairly direct translation of existing materials, were a realization of the possible.In contrast, the second system began with an analysis of pedagogic problems with a particular attempt to address structure issues such as access.These goals were analysed, categorized and prioritized to provide some coherence to the desirable instructional design driving the implementation.The conclusion from these experiences illustrate that particular attention to pedagogical design is not the only factor in improving Internet courseware: it is necessary to strive for an on-going dialogue between educator and developer to achieve a prioritized set of agreed goals which guards against the pragmatics of real-world development producing the least important and easiest options first.

Drawing back from technological opportunism
The popularity of the Internet has infected education, particularly distance education, with technological opportunism.Educators feel the 'technology pull' of Web applications and want to experiment.I have talked to colleagues from around the world, and their stories about new developments usually start with the technology, for example: 'We wanted to see what we could do with Java.' Reinhardt (1995, 55) predicts an optimistic future with improved educational productivity and qualitatively changing learning profiles -he predicts an improvement in learning due to technology alone.But do simple Internet translations of existing materials offer anything more to the student, or do we need other strategies for the effective exploitation of the Internet for teaching?One of the difficulties, both in developing and in evaluating Internet courseware, lies in the relationship between technology and pedagogy.We reassert the familiar argument that 'technology pull' should be subordinated to 'pedagogic push'.The rise of technology has not changed the fundamental educational issues: • How can we identify students' needs and address them effectively?
• How can we accommodate student diversity, particularly when we are remote from our students?
• How can we maximize individual feedback within the constraints of ever-decreasing resources?
The hope is that we can address such issues with new tools, that we can add value through new media.But we have yet to understand the costs of new tools.
Cognitive cost.We have been indoctrinated into book use over years; the book 'interface' is learned and made familiar through many instantiations throughout our lives.But Internet presentation carries a new cognitive overhead: the acquisition of the many-layered interface (operating system, network software, applications, 'documents', etc.).'Sexy' is not the same as 'effective'; sometimes simple, familiar mechanisms work better.Experience has shown that it is easy to propose an electronic solution that is more expensive and time-consuming than the paper-based system it is supposed to improve upon (Pilgrim and Leung, 1996).
Design cost.Although most instructional systems arise from enthusiasm for realizing an educational goal, all too often the educational problems the educator is trying to address are overshadowed by system design constraints.

The OU's interest in the Internet
As a distance-teaching university, the Open University (OU) is keen to investigate the use the Internet for teaching.We have been investigating its use as a communication medium for more than three years, including considering the impact of the Internet as a presentation medium (Thomas et al, 1998).OU course materials differ in nature from those of conventional universities: they typically involve a three-year development programme producing materials which have a five-to eight-year life cycle.Hence, they have a high initial production cost which is amortized over a number of years of large-scale dissemination.While this system produces high-quality materials, it also tends to be slowmoving and conservative.Superficially, the Internet appears to afford a feasible opportunity to produce courses more quickly and flexibly.Materials could be placed on the Web, allowing ready updates without the overheads of publishing and warehousing, and students could use them as and when required.However, merely using the Internet as a publishing medium is inadequate support for distance education.Where is the teaching?

When development issues dominate
As part of our on-going research into teaching via the Internet, we collaborated with European partners in a project to investigate whether there would be real gains for distance-learning students using Web-based learning materials (EONT: an Experiment in Open and distance-learning using New Technologies).Our research had already shown that using the Internet as a communication medium neither improved nor impaired learning outcomes despite anecdotal claims to the contrary (the research was presented in a paper given at the ACM SIGCSE/SIGCUE conference, ITiCSE '97, Uppsala, Sweden, June 1997), but its real and often understated value lay in the flexibility it provided students.Pursuing these observations, we were keen to establish what materials could and should be offered, and where any authentic benefits lay for the student.
The project aimed to address pedagogic issues; there was enthusiasm about the dual potential to provide something innovative for our students while addressing the publishing problem of quickly-dating material.However, in the absence of a clear and strongly-driven pedagogy design plan, the project yielded to the overwhelming push of the technology, and was shaped by its limitations and resource implications.These were twofold: • choice of tool -this was driven by project-political issues, thus the design was constrained by a tool rather than a tool chosen to fit the design • getting it done -it was more important to have the materials in this form than it was to get it right.
The resulting courseware consisted of basic HTML documents with some marginal improvements.It had lost the functionality and richness of a well structured book.Such a translation was not the initial aim; it became apparent that the implementation constraints had produced 'could be' instead of 'should be' Internet courseware.This outcome is an example of the possible, when implementation drives the design.
However, while this was a disappointment in design terms, it was useful as a research tool.This 'pure' translation provided a baseline for comparisons to the original paper-based course material and to a second phase of development.

What the students thought
While we understood that there were limitations to this basic book-to-internet translation, it was important to understand from a student perspective if there were any gains in this type of presentation.A group of typical users participated in an informal evaluation in which the interface was assessed and users performed set tasks in order to provide some insight into students' methods and requirements.The overwhelming feedback from students was their lack of preference for this format; they did not anticipate using the courseware very often.The only perceived value was as a source of reference and only then if they were near the computer.Some of their concerns are characterized below.
• Not optimized for usefulness.While the software was considered easy to use, it was not considered useful.It lacked the maturity of a well-structured book which could support a variety of access strategies through devices such as the headings, index, glossary, and table of contents.Students disliked reading text on screen.
• More cross-referencing required.Again, this appeared to be a structural issue; the crossreferencing would provide students with a variety of routes to the information.
• Cannot see different components simultaneously.When using a book, students had the ability to 'look' at different sections at one time, e.g., flicking between the sections 'book-marked' with their fingers.This sort of quick swapping was not supported by the Internet version.
• Not enough interactivity.The materials suffered from a general lack of interactivity (as is common in most courseware).However, the students viewed access as interaction: moving readily through relevant information would allow them to engage with the material.Hence, more sophisticated interactivity seemed to be less important than the provision of an adequate structure to support flexible and responsive access.
• Unexpected aesthetics.Students expect text, colours and screen layout to conform to a familiar range which they have experienced, i.e., they expect conformance to stereotype.This group of evaluators was experienced with Microsoft products and common Web browsers, and they expected features which looked like familiar widgets to behave like familiar widgets.Unusual colours and behaviour inconsistent with stereotype were found to be jarring.
• Inconsistent interface.This complaint was mentioned repeatedly throughout the navigation of the software.After coming to terms with the meaning and functionality of icons, students (predictably) experienced frustration and difficulty when the icons did not behave in a similar fashion in different contexts.However 'inconsistent interface' was also used to describe inconsistency with stereotype-students accustomed to a particular interface characterised the unfamiliar interface as inconsistent.
This review highlighted some interesting issues.First, little-to-no-benefit was identified by the evaluators as accruing from a basic translation of print to electronic form.The students indicated its likely usefulness as minimal.The Internet was not appealing as a publishing medium.Its main disadvantages were the overhead of understanding and becoming familiar with the interface, and the undesirable requirement to read volumes of text on screen (or alternatively to bear the cost of printing).
Second, making the structure of the material (both interface and content) evident is important.The students expected the Internet courseware to offer improved functionality.
Books have the advantages of portability and familiarity over a computer; students expected other added-value factors in compensation, such as more flexible structure, interactivity, and improved access to information.

Pedagogic push
We decided to discard the first piece of courseware and begin again, with a resolve to let the pedagogy drive the design.For the second system, we adopted a principled approach to the design of web-based materials by concentrating on student requirements, starting with the issues of structure and access identified in the first evaluation.The responses had emphasized that access should be flexible (supporting many routes and strategies) while using obvious, consistent mechanisms.
Supporting flexible access while providing clear insight into underlying structure appeared to be a sensible approach to addressing individual student needs.The Internet is heralded as a mechanism for flexible learning modes which can support student diversity (Steeples et al, 1996) (Watabe, Hamalainen and Whinston, 1995).Here, then, might be a point at which technology serves pedagogy.We reviewed the literature on learner traits and more particularly on cognitive controls and cognitive styles as a means to understanding how we could structure information for different types of learners.We examined these dimensions to provide insights into how learners process, reason, gather and organize information.

Cognitive controls
Cognitive controls are derived from our mental abilities and form patterns of human thinking or reasoning in individuals.Each cognitive control is a separate representation of a pattern of thinking.Cognitive controls differ from cognitive styles in that they are psychoanalytical and regulate perception.These influence an individual's perception of environmental stimuli (Jonassen and Grabowski, 1993).Cognitive styles are tendencies that define learner traits and reflect an individual's perceptual habits and are stated in terms of typical performance.Field dependence/independence is one cognitive control that provides information on how students differ in their preference for information structuring.

Field dependence/independence
This is the most extensively researched cognitive control and is the most prescriptive of learning and instructional outcomes.FD/I describes the degree to which a learner's understanding is influenced by the surrounding contextual or perceptual field (Liu and Reed, 1994).Field dependents find it difficult to separate important from unimportant information; what they are looking for is often masked by the surrounding information.They will store information and encode it without reordering, restructuring or revision.Field independents, on the other hand, are more likely to recognize, restructure, or reorganize information to their own needs.They have an articulated cognitive style as they prefer to create their own models for understanding the domain.They are better at expressing their own knowledge as they impose their own structures on it.Most learners fall somewhere in between, and share aspects of each in different concepts.FD/I is not a static trait and changes over a learner's life span from generally field dependent, as a child, to field independent, as an adult and gradually decreasing in field independence in later years (Jonassen and Grabowski, 1993).Field independence also seems to increase with the amount of formal education.However, this demonstrates that FDs have more difficulty in imposing their own structure on information and would likely benefit from some already provided structure upon which they could, at least initially, rely.

Cognitive styles
Cognitive styles can be described as consistent characteristics of the learner (which develop as a result of underlying personality trends) which indicate how they process information (Riding and Cheema, 1991).Learners differ in how they process information and interact with their environment; these are not cognitive controls and abilities (although they are linked to them) but preferences which reflect personality traits.For example, abilities are competencies which are stated in terms of maximal performance; styles are propensities (tendencies or inclinations) which are described in terms of typical performance.The value of such traits is in the recognition of individual differences and the preferences that learners may display in processing information.Such knowledge can help develop strategies to support appropriate structuring of information and flexibility in learning (Messick, 1984).There are two different cognitive styles which we will consider for different reasons.One (visualizer/verbalizer) indicates how learners gather information, and the other (seralist/holist) indicates how they organize information.
Visualizer/verbalizer.This ^describes learner preferences for processing information either by seeing (through diagrams, graphics, or illustration) or through words (either by reading or listening).Visualizers tend to think more concretely, preferring pictures, diagrams, etc. to augment text-based material.Verbalizers are more objective about information they are learning.Some learners are equally comfortable with both modes, and the differences in this cognitive style are not as great as others.There are alternative classifications for this dimension such as high and low imagery (Jonassen and Grabowski, 1993).This dimension also has a relationship with FD/I: visualizers are more field dependent, whereas verbalizers are more field independent.
Seralist/holist.This is a bi-polar dimension which describes the extent to which a learner selects and represents information.Holists use global, thematic approaches to learning and focus on several aspects of the same subject at the same time.They have many goals and working topics that span different levels of the hierarchical structure.They use complex links to relate different levels of information, and establish an ordering essentially top-down in approach.Pask (1976) describes a holist's deficiency as inability to focus on detail.Serialists use a bottom-up operations approach to learning; they focus narrowly on detail before conceptualizing an overall picture.They use simple links between concepts and use these to develop objective, logical arguments.Pask identified deficiencies in serial learning as ignoring important connections.Versatile learners will employ both serialist and holist learning strategies and will be able to resolve conflict over several opposing theories by using global and detailed approaches.They succeed in achieving a full or deep understanding that is comprised of both descriptions and procedures, and are proficient at learning from all or most modes of instruction.FD/I has a relationship with this dimension also: holists are related to field dependents; serialists are related to field independents.

The pedagogical goals
All three dimensions described are linked in several ways.First, they are connected by their association with information processing, reasoning, gathering and organizing.Second, the implications for improvement strategies for some of these types of learners are the structuring of information.And third, there is a relationship between the FD/I dimension and the serialist/holist dimension, and the FD/I dimension and the visualizer/verbalizer dimension (Ford, 1995).It is clear in examining these dimensions that some types of learners can benefit from structure.
But what kind of structure should be provided?By analysis, we derived three categories of information structuring: access, concept ordering, and information representation.We chose to focus on accessing information in the second phase of development; the main educational objectives were to develop strategies which would address the 'deficiencies' of field dependents, holists and visualizers by focusing on the access aspects of structuring information.

Instructional strategies for software implementation
Having identified the particular educational objectives of the second prototype, the next step was mapping them onto some kind of instructional design strategy: deciding what the structuring and access devices would look like and how they would operate.We brainstormed, and reviewed other courseware products for ideas -especially those that provided multiple information access routes such as Encarta.From this exploration, we drew a list of access mechanisms which could be mapped onto our pedagogical objectives.In addition, we planned to address the interactivity issue by providing a non-functional icon drawing tool, to allow HCI students to develop interfaces.
The developer was excluded from our initial planning as a deliberate tactic to debate our ideas without regard to technical constraints.The desire was to examine what it was we were trying to achieve, and how and why this might improve the learning materials for the students -to make pedagogy drive and technology serve.
Subsequently, after several meetings, the developer was involved and a specification was drawn up of the types of innovations we wanted to include in our design.More specifically, it enumerated the types of structure that would meet our pedagogical requirements: glossaries of terms, contents list, definitions, index, keyword searching, pop-up windows, annotations, navigational aids (with typical menus, etc.) to support flexible and speedy access to information, and searches specifically matched to topic or assignment.The plan was pedagogically-driven, supported by a good working relationship with the developer and agreed a set of goals.
However, while, superficially, communication with the developer was fine, at a deeper level there were major communication problems.The expectations of the 'woolly' educator were at odds with the developer's priority for demonstrable productivity.The problem lay not with the pedagogic plan, but with the failure to articulate that plan as prioritized implementation goals.As the project proceeded and deadlines approached for the completion of this phase, it became apparent that, while there were major improvements, all was not as we had hoped.The developer focused on capturing the content first and then began imposing the structure, whereas the innovations in structure and access mechanisms were the educators' priority.The developer's pragmatic approach meant that less important albeit high-volume aspects were completed first, leaving the innovations to last.Therefore, some goals were neglected because time and financial resources were exhausted.The typical problems could be summarized as: • the developer's desire to revert to familiar tried and tested solutions -the developer does not want to be credited with a failure; • the developer's desire to complete specific tasks first based on technical arguments -the educator lacks technical expertise or experience to know whether such arguments are valid, and usually succumbs to the developer's wishes; • the educator's inability to articulate a specification with a set of prioritized goalsdevelopers will make their own set of prioritized goals, which is to produce something that is tangible and works; • the educator lacks experience in developing teaching materials in this medium -this leads to confusion and a lack of confidence.
We had avoided the normal pitfalls of development seduction: trying to do everything at once, trying to produce a universal system, one-more-feature disease, doing things because one can (and mistaking that for progress).However, the project was jeopardized by the difference of perspective and of objectives between developer and educator.
The lessons are that educator and developer need to prioritize components of the specification and articulate that agenda as implementation priorities.The educator needs to 'hold onto the reins', especially ensuring that, when reality intervenes with constraints, decisions are based on educational priorities, rather than technical convenience.The developer's strategies tend to overlook pedagogic reassessment when compromises are required.

Discussion
Overall, the project provides an essential lesson on the importance of sustaining educational priorities through all phases of design and implementation.The first phase of the project was guided by what was technically and practically possible; pedagogy followed technology.This seems to be an illustration of one of the fundamental problems with Internet developments: people do things because they can within the constraints of budget, time, and experience.Instead of looking for solutions to things we want or need and really pushing the technology to come up with solutions (educational push), we accept available technical offerings and succumb to technology pull.
The first phase of the Internet courseware, largely a direct translation of the original paper-based materials, illustrated that there are real questions about the value of a simple translation of paper-based learning materials into HTML documents.Although it was a disappointment in terms of design, it gave us a baseline for comparison so that we could investigate the value of simple translation.Just moving material onto the Web incurs additional overheads for students; we have a culture of book use inculcated over years, but we do not yet have a comparable culture of Web-page access and Web-page usage.Students who come to this naively have the burden -the extra cognitive load -of developing new strategies for coping with accessing Internet-based material as well as having the material in this alternative form.The structure of the material, or lack of it, may increase the cognitive load further: students spend more time trying to understand how to access the information than using it (Molnar and Sharda, 1996).
The second phase of the project was intended to be pedagogically driven.The educational goals were identified through analysis and experience; the second phase aimed to address particularly the deficiencies of the previous Internet presentation.An analysis of the literature provided us with a framework for understanding what types of students could benefit from enhanced structuring and how that structure should be manifest.This match between the students' requirements and the literature forms a sound basis for design decisions and warrants further scrutiny.
The second phase of the project was notionally guided by what was desirable educationally.
Our difficulty was in mapping design decisions to implementation priorities.The difficulties here gave us reason to reflect on why so many high-minded educational projects seem dominated by technical convenience -and especially to consider the quality of the dialogue between the educator and the developer.The educator has some educational goals and notionally some technical aspirations.The developer has technical knowledge, including a knowledge of constraints.In the absence of a clear educational vision which is carried into the management of the technical development, the developer ends up leading the project.Inevitably the developer leads the project into decisions which are productivefrom the development perspective.Developers will aim for familiar, 'reliable' solutions, instead of reaching for the kind of infrastructure, support structure and tools that actually provide added value.The tendency to settle for safe-but-possible courseware may account for why so many projects which set out with enthusiasm and ambition turn out to be so mundane.
The EONT project demonstrates the importance of the educator-developer dialogue.The lesson is that the educator must develop the leadership to overcome the conservatism of developers, and to set and maintain a pedagogically-driven agenda.It is not sufficient to set the project aims from a pedagogical perspective: there is a need also to prioritize implementation objectives.All initial objectives may not be realizable.The advantage of setting priorities on implementation goals means that the aspects of the design most important from a pedagogical perspective can be tackled first, and if the project fails to complete all its intended goals then the result will at least include some of the more desirable and innovative features.

Conclusion
Fervour for the Internet is a catalyst to experiment with a new technology that may address some of our educational needs.Although we all set out with ambitions to address specific learning requirements, if we lack a clear pedagogical plan, implementation issues dominate.Given our inexperience in this area and the all-too-familiar seduction of 'sexy' technology, we produce the possible: translations which are mundane and offer little improvement over their printed counterparts.
While the EONT project was limited from a pedagogical perspective, its lessons are clear.We need to start from pedagogical aims and look for added value in areas that matter (that matter to students, that affect learning).A pedagogical plan which is founded on the literature, analysis of experience, and students' requirements is a principled framework for design.However, a pedagogical plan is not enough to ensure a wholly successful product.
In the absence of a prioritized set of implementation goals, the developer concentrates on the more realizable goals first, often the less innovative, mundanely high-volume aspects such as content.In the real world, with resource constraints, innovations left to the end may never actually be completed.Addressing this issue requires the development of an educator-led, continuing dialogue with the developer.
We need to be aware of development hazards and be sure that pedagogy drives the design.We need to develop a continuing dialogue with the developer and prioritize goals to ensure that the most desirable features of the design are completed first, not last.And we need to have respect for modest gains: not all our dreams are realizable in one project.
We must analyse our existing processes deeply and critically in order to provide fully -and appropriately -realized Internet teaching that serves learning at least as well, and at least as economically, as conventional methods.This paper takes the pragmatic stance that the key to successful application of technology in education is good teaching: making technology a cost-effective servant of pedagogy.