A case study of campus-based flexible learning using the World Wide Web and computer conferencing

This paper explores the use of the World Wide Web (WWW) integrated with computer conferencing as a teaching and learning tool. The aim of the study described was to evaluate the effectiveness of the use of online materials designed in a flexible learning format and integrated with a computer conference. It was hoped that this would create additional opportunity for group discourse between campus-based students. The paper is divided in the following way: a discussion of the context to new developments in teaching and learning is followed by an introduction to the case study. Finally the findings of the case study are discussed with reference to research from the field of collaborative systems (Orlikowski, 1992; Grudin, 1994) as a framework for reflection. Some tentative conclusions are made for future work.


Introduction The problem in context
In line with departments in many other universities, the IT Institute (ITI) has experienced an increase in student numbers which has not been matched by a commensurate increase in funding or staff numbers.Staff to student ratios have increased to the point where it has become impractical for small group tutorials to take place.This position has affected job satisfaction and caused increasing doubts over the effectiveness of large group lectures for student learning.Furthermore, it has been recognized that over the last ten years the makeup of the student population has radically changed, leading to a larger proportion of 'nontraditional' students such as mature students with family responsibilities.Related to this, many students are now in part-time or even full-time employment.It is recognized that both these groups require greater flexibility in the time and place of their learning.
Many universities are moving towards open or flexible learning to attract students from other parts of the country or abroad to study their courses locally.This is part of a wider trend towards the 'Virtual University' (Daniel, 1996) which can support learning across time and distance.The UK Open University has been a major influence in this area but the approach is entering into mainstream use as communications technology advances and becomes more widely available.For instance the University of Salford is involved in the GEMISIS 2000 project which has education as a major theme (details are available online, http://www.gemisis.co.uk/services-education.html) and the planned national 'Education Superhighway' is based on similar assumptions of using advanced communications technology to assist teaching and learning.
The above factors indicate the need for academics to engage with the various technologies and evaluate their effectiveness in a variety of teaching situations and the information systems community is ideally placed to do this.This paper introduces such an evaluation in the form of a case study of the use of the World Wide Web (WWW) and computer conferencing for campus-based teaching and learning.The paper discusses the findings of the case study using accepted research from the field of collaborative systems (Orlikowski, 1992;Grudin, 1994).This research is used as a framework for reflection and to make some tentative conclusions for future work.

Technological possibilities
'New technology is changing the way information is stored and transmitted.This has implications for both the skills which higher education needs to develop and the way in which it is delivered' (Dealing, 1997).
Computerized delivery mechanisms for distance learning have been on the higher educational agenda since the 1970s.With regard to computers and education, the teaching of children has always been more prolific in the educational literature.The limited value of early stand-alone computer-aided instruction systems was disappointing for many and led to disillusionment with the idea of Skinner's 'teaching machine'.The promise of the later developments in computer-aided learning based on the work of Seymour Papert did not cause the revolution in the classroom it promised (Boyle, 1996).There is now feverish optimism in both secondary and higher educational circles largely due to the rapid assimilation of mass networking involving the Internet and associated learning approaches using the WWW.Mason (1998) identifies four broad categories into which current educational technologies can be divided: • text-based systems, including electronic mail • conferencing, real-time chat systems and many uses of the WWW • audio-conferencing and audio extensions such as audiographics, and audio on the Internet • video-conferencing, one-way and two-way and other visual media such as video-clips on the WWW.
Education is viewed by many as the 'killer application' for interactive multimedia; when liberated from the CD ROM by the WWW it is seen as enabling mass, global access to interactive learning resources.Bates (1993) sums up the optimism: 'technological developments already available or in the pipeline have the potential to revolutionise education and training as we know it.'

Use of the WWW as a medium for teaching: a case study
Largely as a reaction to educational trends, a new ten-credit module was developed for the ITI undergraduate degree in IT entitled 'Developing Systems for Teaching and Learning'.This was broadly concerned with exploring the development of information systems for teaching and learning.This module was ideally placed to give students exposure to a wide variety of teaching and learning approaches and encourage reflection on their relative effectiveness.There was a desire to investigate the potential of emerging technology and reflect on experiences.

Aims and objectives
The following were considered to be the guiding objectives for the investigation: • To explore the feasibility and effectiveness of using web-based online interactive learning resources and conferencing to facilitate learning.
Leading to the following outcomes: • A prototype 'learning environment' for the new module, which would include a mixture of online interactive materials, a conferencing system, course materials and access to other open learning resources.
• A report exploring the feasibility of using web-based resources and conferencing to support student learning.
The overarching research question guiding the enquiry was concerned with: • How effective are advanced communication technologies in supporting campus-based student learning?

Evaluation approach
In order to evaluate the system and the process the author consulted Baric (1998), Boyle (1996) and Reeves and Harmon (1995).The emphasis was on qualitative evaluation which would lead to insights regarding the effectiveness of the module teaching and learning strategy.The web site was built around general systems development lifecycle principles with an analysis, design and implementation phase.In line with the rapid prototyping methodology, sections of the system were built and demonstrated.This led to an underpinning critique, new ideas and insights.For instance this process resulted in the design and development of several interactive animations.In this way evaluation commenced at an early stage with discussion over the appropriateness of the structure and screen designs.Prior to 'going live' the online materials were examined and tested by student groups.
Students were involved directly in evaluation activities as a result of their coursework components.Two student teams performed an evaluation of the module web site as part of their assessed coursework and all students were required to reflect on the success and appropriateness of the various teaching and learning approaches used in the module.Two focus group sessions were held during the course of the module and although these were not taped, copious notes were taken.Tutorials and workshops each week led to informal evaluation material and several semi-structured interviews took place with students at various stages throughout the course.The author also kept a diary of observations of the system in use, conversations and thoughts that occurred during the course of the module.
Finally, an online questionnaire was produced to gauge student opinion.The course team also carefully examined course work submissions to judge whether students had met the learning outcomes utilizing the teaching and learning approach adopted.

Components of the online delivery
This section will outline the components of the course and explain how the technology was used to support the teaching and learning.It is divided into the following aspects of the course: administration, interactive web-based learning, computer conferencing and assessment.The module is a third-year undergraduate option and was taken by 34 students.There were no formal classroom-based lectures although there were regular small group tutorials.

Administration of the course
The web site was instrumental in the administration of the course.It contained all of the syllabus and assignment details as well as times and venue information and was also used for other administrative matters such as recording names of group participants and group responsibilities for attendance, presentations and peer assessment.Peer assessment schedules and forms were stored online.Students found this useful as they 'knew where they stood' from the start with regard to expectations and did not need to worry about lost information or filing papers.The newsgroup became the official mechanism for communication about the course administration issues.

Web-based learning
The use of the WWW to deliver teaching and learning material is still in its infancy, yet two distinct major approaches can be seen to be developing: • WWW-based materials to support learning via access to resources.This takes the form of lists of links, resources, lecture notes, etc.This is meant to supplement other teaching and learning approaches such as lectures, etc.
• WWW-based materials as the main medium for teaching and learning.This could take the form of online open or distance learning materials designed to act as the main medium through which teaching and learning will take place.Other methods such as lectures or seminars would be supplementary or complementary to the online materials.
There was no desire to move to a distance learning approach as the course is campus-based and students are regularly on campus.However, the teaching and learning strategy was built around this strategy supplemented by face-to-face small group tutorials.The tutorials were designed to be used for evaluation and to engage with the issues discussed on the web site to reinforce understanding.There was also a need to evaluate the need for tutorials.Thus the course team was not attempting to emulate distance learning approaches but to supplement with a view to realizing deep learning and teach larger numbers of students more effectively.
The web site contained materials covering the key educational theories to design IT systems for teaching and learning.The detailed information was followed by questions and tasks for the students to complete in a similar flexible learning format to that recommended by Race (1993).In particular, the 'self-assessed questions' prescribed by Race were replaced by a request to submit responses to tasks and questions to the course newsgroup (the 'virtual seminar') for discussion with the rest of the group thus creating an 'online' seminar.To make this as seamless an experience as possible, a 'go to virtual seminar' icon was placed alongside the question or task to bring up the newsgroup window ready to accept discourse.This aspect of the course was designed to offer potential for online interactive discussion, thereby building on the static paper-based open learning approach as prescribed by Race (1993).

Computer conferencing
The use of asynchronous news group technology was chosen as a practical means to allow students to discuss the web-based learning materials and then to respond to the questions and tasks while separated by time and/or distance.Newsgroups were used due to the unavailability of synchronous communication software and the relative robustness of newsgroups.In parallel with the open learning resources, it was felt that this would provide greater flexibility of time and place for discussion and would facilitate more learning experiences and opportunities for reflection.The course newsgroup contained two 'rooms'one named the' 'virtual seminar' and another named the 'virtual coffee room'.The coffee room was intended for informal communication and the 'seminar' for discussion of course materials and matters directly linked to the module.

Assessment
The course web site was used for submission of student work.Students were invited to submit their evaluations as a series of linked web pages that would be accessible via the web site.The team evaluations were open to all and were peer-assessed by another team (who used the virtual seminar as a means of interrogating the team being assessed).

Analysis and discussion
The analysis of the results of the case study is divided under the general headings of the accessibility and functionality of resources, quantity and clarity, motivation and feedback issues and finally teamworking and co-operative interaction.The discussion refers to accepted research from the area of collaborative systems (Grudin, 1994;Orlikowski, 1992) to reinforce the validity of the findings.

Accessibility and functionality
The site was accessible via the WWW, creating independence from the confínes of the university and allowing access at any time.Every student had at least one method of accessing the pages, which could be online at the ITI, at home via an Internet Service Provider (ISP), off-line at home by copying the files onto a home machine and off-line by printing out the information and using the hardcopy for reference.The questionnaire and interview findings showed that most students were using the materials at home in an off-line mode, and only a small number accessed the materials online at the ITI.Critically, this meant that those students accessing the web pages off-line did not have instantaneous access to the virtual seminar to post queries and engage in discourse.This had a significant effect on the success of this element of the course.Interviews indicated that the virtual seminar debates had often finished before the 'off-line' students couid usefully contribute.Unless the seminar was 'close at hand' students forgot to submit or would lose the thread of their thoughts due to the time lag between working on the materials and actually inputting their response.Grudin (1994) makes the point that to be successful, accessibility to the features that support group processes must be unobtrusive.In this case, the very benefit of flexibility for the students via home off-line access led to the group discourse support process becoming impractical.This was a fundamental problem as it caused difficulties in achieving and maintaining the 'critical mass' of users to make it continuously beneficial.

Quantity and clarity
Issues of the quantity and the clarity of the information presented on the web pages were of relevance as a source of comparison and of indications of the success of the online learning.Issues of quantity were prevalent: We felt swamped by the amount of information.The voluminous amounts of information are overwhelming, Web-based material is overloaded with too much information.
Scaling them down to the main points and making it open for discussion would be more effective.
The qualitative section of the questionnaire indicates similar feelings when compared with other ITI module workloads.With regard to the virtual seminar, the newsgroup conference uses threaded discussions offering the opportunity to discuss topics and for several debates to take place at once.Those who did submit to the virtual seminar made impressive contributions with some students thoroughly enjoying submitting their ideas.A number of group submissions occurred in response to the online questions contained in the course online materials.However, students who did not use the resource regularly due to access differences (discussed above) found reading the 'voluminous' contributions to the virtual seminar impossible and this only contributed to the feelings of being 'swamped'.
This factor can be related to discussion of the adoption process (Grudin, 1994) in that the introduction of the materials did not take account of students being used to stage delivery of content.Students are socially accustomed to the idea of a regular timed meeting between lecturer and student.This is reflected in questionnaire responses that also indicate the importance of tutorials.Clearly the 'big bang' adoption process used in this case ignored the embedded custom and expectation of students and the critical effect of online / off-line access methods.

Motivation and feedback
The initial set of problems concerned access to the materials and technical difficulties.At least one student dropped out of the module stating the reason for this as feeling uncomfortable with the module being delivered by technological means.Students experienced technical difficulties getting access to the resources in some cases as well as navigation around the system.However there was a surprising lack of technical problems once the initial 'access denied' type difficulties had been overcome in week one.Comments during this period included: There was no reward for the effort of using the web-based resource and we were therefore not motivated to try hard.
I felt uncomfortable adjusting to the computer as teaching medium as opposed to computer as a tool.
I felt demotivated by the volume of information.
Although they were initially enthused by the novelty of using different teaching techniques, there were considerable problems with motivation regarding the virtual seminar.In order to encourage participation in the virtual seminar, an assessment incentive of 5 per cent was awarded for the online contributions.As students worked through the notes online there were numerous contributors with some excellent comments.However, it was found that the students were not engaging in dialogue in the virtual seminar but were simply responding to the questions in the online materials and posting the answers.After the initial period, 'active' students did not see any further benefit in continued posting of their ideas and usage of the newsgroup tailed off steeply after weeks three and four.During weeks four and five, there were still open-learning-type tasks and questions online but the readings were paper-based and by this time the virtual seminar activity had largely tailed off.The second half of the course moved to action learning with a number of continuous assessments driving the curriculum.This period saw use of the conference limited to pragmatic course information issues and a few subversive comments from anonymous students.
Interviews included the following: We posted an amount that we regarded to be sufficient for our marks.
Some students have said that they resented the use of this 5% (incentive) so much that they essentially boycotted the virtual seminar altogether.
The questionnaire results indicate that many students were 'lurking'.The conclusion appears to be that staff intervention was required to give the virtual seminar a sense of purpose and benefit in the student view.Interviews indicate that the students accepted the idea of interactive discussion but they wanted feedback on their submissions from the course team to keep participation up.There was a prevailing view that face-to-face tutorials were essential to learning, and that the virtual seminar experience would be insufficient to support sophisticated learning (which needs dialogue).
This general finding from the virtual seminar is in line with the work of Grudin (1994) and Orlikowski (1992) who state that groupware users must perceive a direct benefit for their additional work or participation will decline.Orlikowski (1992) discusses the importance of encouraging information sharing through reward structures.The 5 per cent motivation to 'kick start' the conference tailed off because of lack of perceived sustained benefit.After the initial novelty had worn off students were waiting for added value, which could have been in the form of input from the lecturers to validate and comment on the materials presented in the seminar.Thus there was a disparity in work and benefit and a critical mass of users was not in place.Additional work was required from individuals who did not perceive a direct benefit from the use of the application.

Teamworking and co-operative interaction
A high degree of teamworking was necessary within the module.This occurred in several different environments including lectures, tutorials, assignment groupwork and the virtual seminar.There was a mix of students accessing the materials alone and in their groups and due to the large volume of text-based background and reference material required for the course, some teams distributed workload between team members.The extent to which this was successful depended upon the level of organization within individual teams, breaking down sections of the site and sharing their experiences with the team.The idea of putting the student evaluation pages on a web site followed by an online critique using the virtual seminar available to all proved to be a very powerful learning device and a real success of the virtual seminar.However, interviews with students indicated three types of virtual seminar users.There are those who read but do not contribute in order to 'poach' extra marks, those who are genuinely interested and contribute readily and those who ignore it but submit something for the 5 per cent.
It is noteworthy that the ITI is a collaboration between business, industry and academia and therefore reflects a commercial competitive ethos.Lecturers often display a laissezfaire attitude and students are openly seen to 'sink or swim' in the environment.Interviews revealed that student teams often feel in competition for marks: 'The theme introduced by the course team of student teams working towards a common goal in a non-competitive environment was unfamiliar to many students.'Grudin (1994) suggests that groupware will be unsuccessful if it disrupts social processes, i.e. violates social taboos, threatens existing political structures or otherwise démotivâtes users.Interviews suggest the limited use of the virtual seminar is a result partly of typical student competition but also reflects a commercial ethos engendered by the ITI, which causes some teams not to share information.This is reinforced by the work of Orlikowski (1992) who encountered similar results with the use of Lotus Notes in a management consultancy.

Conclusions and implications for future work
In the author's opinion, the strategy of building educational networks using a flexible learning format is feasible and overcomes some of the problems over flexibility, time and place and large numbers of students.It also offers the possibility of incorporating interactivity via the virtual seminar and with multimedia, video-clips etc.There are fundamental issues from this case of use to those involved in building educational networks.The 'big bang' implementation in this case study did not take into account the social customs and expectations that students had regarding the delivery of content.
Introducing new methods of teaching and learning such as this need to be introduced gradually.The use of tutorial support to supplement learners should account for different learning styles.Students need to be given some clear guidance and education on how they should use the materials and the philosophy of flexible learning.
It is felt that the use of the web site for placing detailed information followed by questions and tasks to be submitted to the virtual seminar was an effective idea for some groups.
Critically, the various access methods affected the closeness at hand and seamlessness of posting to the virtual seminar thereby affecting the level of discourse.It is difficult to suggest solutions other than to make this issue plain to the students and discuss the advantages of the virtual seminar together with access options.
There was also little motivation to engage in discourse partly due to off-line access methods.The time lag between reading and posting meant many students did not bother to submit or felt swamped by the discourse or disappointed when a stream of discourse had already begun and ended.Students need to be encouraged to browse the seminar regularly and make it an accepted part of the teaching and learning.Competitive cultural factors and lack of staff involvement in the virtual seminar also contributed to motivation difficulties in this case.This issue may be resolved by the course team dedicating part of the contact to online tutoring or alternatively by setting up themed conferences lasting for a particular period of time.
There is also the issue of development time for courses delivered in this manner.The estimate for development time was roughly three hours' development time for every one hour of student exercises.This is costly but it is hoped that costs will be recouped over time due to efficiencies gained.
Clearly careful consideration of students' workload, time and staging of content is essential to avoid students feeling overloaded.One strategy is for early content to be structured in a week-by-week format rather than organizing material by topic, giving students more structure, until they become accustomed to the format.This structure should gradually diminish as the task drives the learning.