Activating Boxmind : an evaluation of a Web-based video lecture with synchronized activities

The aim of this study was to evaluate the use of synchronous computer-mediated communication activities in a video e-lecture. Previous research has reported that learning is facilitated when communication activities are added to a video lecture. Twelve postgraduate students participated in the study and they viewed a video e-lecture on the perspective-taking theory of communication. The lecture consisted of a video image of the lecturer, an audio track, slides, the transcript and a number of communication activities. They were given a pre-test a week before the lecture and a post-test a week after. They were also asked to rate the helpfulness of various aspects of the lecture. Students' post-test scores were statistically significantly higher than their pre-test scores. They found the audio track, transcript, slides and activities helpful. The most helpful aspects were the communication activities. The implications of these findings are discussed.


Introduction
The use of video tapes to support students' learning has a long history in higher education.Video case studies have been successfully used in teacher education (see Dolk, Hertog and Gravemeijer, 2002) and medical education (see Dequeker and Jaspaert, 1998).Similar success stories have been reported for video practicals (see Heniford, Backus, Matthews, Greene, Teal and Sing, 2001).However, the perspective on video lectures is not quite so positive.Gibbons, Kincheloe and Down (1977) found that learning can suffer when students watch a lecture video individually.Students watched the video passively and showed little engagement with the material.To overcome this problem, Gibbons et al. (1977) developed the method of Tutor Video Instruction (TVI).TVI involves groups of students watching video tapes of lectures in small groups with a tutor present to facilitate the discussion.Both the tutor and the students can stop the video tape and initiate a discussion or ask a question concerning parts of the lecture they did not understand.They found that students using TVI had higher course grades than students who attended the live lectures in the classroom; students who watched the videos individually and students who watched the lectures from remote locations.This finding was replicated by Stone (1990) who found the same superiority effects even when the students met without a tutor present.
Recently this approach has been adapted through the development of a Distributed Tutored Video Instruction (DTVT) (Smith, Sipusic and Pannoni, 1999;Dutra, Gibbons, Pannoni, Sipusic, Smith and Sutherland, 1999).DTVI involves groups of seven students watching a video of the lecture from a number of distributed locations.Students are connected with a high-quality audio and video connection.They found in a series of experiments involving several hundred students and six different university courses that the DTVI and TVI groups course grades were significantly better than those students who attended the live lecture.They attributed the success of DTVI and TVI to the discussion between the students.The finding was replicated by Herder, Subrahmanian, Talukdar, Turk and Westerberg (2003) in an experiment to co-teach an engineering course simultaneously in the Netherlands and the USA.Teams were formed which had members from both countries and the findings were positive.DTVI involves a tutor and a high-quality audio video communication.Cadiz, Balachandran, Sanocki, Gupta, Grudin and Jancke (2000) developed a similar approach without a tutor called CCTV.As with TV! and DTVI, the students can stop the video tape at any point and initiate a discussion about the lecture.Cadiz et al. (2000) compared four different communication channels: synchronous computer-mediated communication, audio conferencing, video conferencing and face-to-face.They found that synchronous computer-mediated communication was not a helpful medium to support the kind of open discussion that was thought to be beneficial in TVI and DTVI.This finding was not surprising.Previous research has reported that synchronous computer-mediated communication has a number of problems that limit its ability to support discussions.The lack of verbal, non-verbal and paraverbal feedback can be a hindrance to communication (Hron, Hesse, Cress and Giovis, 2000).Other problems occur because the time sequence of contributions differs from the logical sequence of argumentation due to input and transfer delays, and this impairs the coherence of the discussion (Murray, 1991).These problems hamper its ability to support discussion (Daley, 1993;Hiltz, Johnson and Turroff, 1986;Mckinlay, Procter, Masting^ Woodburn and Arnot, 1994;Straus and McGrath, 1994).However, a number of researchers have found that these difficulties can be overcome either by coaching (Cornelius and Boos, 2003;Veerman, Andriessen and Kanselaar, 2000) or by structuring the activities (Baker and Lund, 1997;Hron et al, 2000).In view of these findings, the aim of this paper is to re-examine the issue of the usefulness of synchronous computer-mediated communication for supporting video e-lectures by using structured activities.

Video e-lecture
The video e-lecture was part of a Masters' programme on human communication and computing.Although this was a campus-based course, we were finding it difficult to have face-to-face seminars and discussions.Therefore, we decided to examine whether these could be supported using synchronous computer-mediated communication.The students were able to take the lecture at any time of their choosing within a prescribed week of the course.The only constraint was that they had to participate in the lecture with a partner.
The video e-lecture was on the topic of perspective-taking models of communication (for example, Bakhtin, 1981;Clark and Carlson, 1982;Clark andMarshall, 1981, Krauss andFusell, 1991;Rommetviet, 1974;Volsinov, 1986).These models are based on the idea that speakers and addressees have different perspectives and that successful communication requires both speakers and addressees to take these different perspectives into account.A considerable amount of research on the role of perspective-taking in communication has examined how speakers refer to objects.To refer to an object, a speaker must first formulate a referring expression that will permit the addressee to identify the object.Acts of reference are complex.Most objects can be referred to in more than one way, which one is used depends on the context.For example the first author of this paper may be referred to as 'Richard', 'NB287 345' 'Richard Joiner', 'Dr Joiner', or 'the first author' depending on the situation.
The video e-lecture was implemented on the Boxmind proprietal system.Video e-lectures were delivered through the Web browser Internet Explorer (Version 4.0 or later) and Windows Media Player (Version 6.4 or later).The Web page is split into four areas or zones (see Figure 1).Each of these zones is designed to hold different types of content.Zone 2 (top right corner of Figure 1) was designed to contain supporting slides and images.These slides appeared automatically as the video plays.The size of the image was restricted to 400x300 pixels.
Zone 3 (bottom left corner of Figure 1) contained the lecture bookmarks and Web links.The user switched between these two facilities by selecting the appropriate button found in this area.Lecture bookmarks were the main method of navigating around the video electure.By selecting the desired bookmark the viewer was transported to the beginning of that section and was able to replay it if desired.
Zone 4 (bottom right corner of Figure 1) displayed a transcript of what the lecturer was saying.The transcript was divided into short paragraphs and as the lecturer spoke the corresponding paragraph in the transcript was highlighted.
The video, which held all commands for other content, was segmented into fifteen parts, each lasting between one and five minutes in length.The Boxmind framework was extended to include synchronized communication activities.They were embedded into the lecture by a series of instructions that told the Boxmind environment to open a new browser window.This' window contained instructions regarding the activity and a synchronous chat facility, which allowed the students to interact with each other online.
The e-lecture had five synchronous text-mediated activities and they were designed to illustrate points introduced in the lecture.Members of each pair worked on the activities at the same rate and used the synchronous chat facility to coordinate and discuss their activities.
Before the start of the video e-lecture, the students were first familiarized with the synchronous chat facility.The video e-lecture described the different components of perspective; outlined the types of experiment used to investigate perspective-taking and the key findings reported in research on the role of perspective-taking in the act of reference.These findings were exemplified in the activities.The first activity illustrated the way referential expression changes according to the situation.In this activity, the students were asked to refer to a course lecturer when the addressee was another student on the course, a student who is not on the course, a member of staff who teaches on the course and a complete stranger.The second activity exemplified a fundamental distinction, first identified by Mead (1934), between the self and another person and how referential expressions change according to whether they are for the same or another person.In this activity, the students were asked to refer to an object in an array so that they can pick it out later and refer to an object in an array so that another person can pick it out later.The third activity tried to demonstrate that the referential expression used to refer to an object varied with the age of the addressee.In this activity, the students were asked to refer to an object in an array so that a young person can pick it out later and refer to an object in an array so that an old person can pick it out.The fourth activity illustrated how referential expressions vary depending on the knowledge the speakers has of the addressee.The students were asked to refer to an object in an array so that their partner can pick it out later and refer to an object in an array so that another person can pick it out.The fifth activity was designed to replicate findings, which have reported that the referential expressions used to refer to objects between the same communicators change over time.In this activity, the students were asked to carry out a classic conversational referential task using the synchronous chat facility.One student was the director and the other was the matcher.The director must refer to an object in an array so that the matcher could identify it.They did this eight times.Its purpose was to illustrate how referential expressions shorten over time, because partners build up shared knowledge of each other.Directly after these activities the students discussed their findings with their partner using synchronous computer-mediated communication.

Method
The video e-lecture was part of the course unit on Communication, Interaction and Task, part of an M.Sc.course in Human Communication and Computing at the University of Bath.One of the aims of the course unit was to investigate the use of different communication technologies in education and thus the unit provided an ideal vehicle for examining the use of synchronous computer-mediated communication activities in video electures.Twelve postgraduate students took part in the study (seven females and five males, ranging in age from twenty-two to forty-eight).
The pre-test was administered a week before the video e-lecture and consisted of twelve questions on the perspective-taking model of communication.The lecture was scheduled in a particular week of the course and lasted approximately fifty minutes.All the students were expected to participate in the lecture during this week and were told to use the computers in the psychology laboratory.Each student had access to a computer and used headphones to listen to the audio track.Before the lecture the students selected a partner.A partner was necessary for the communication activities embedded in the video e-lecture.Members of each partnership sat in different parts of the room so they could not see each other.
The students completed a post-test a week later, which was exactly the same as the pre-test.The purpose of the pre-and the post-test was to check if the students had improved after participating in the e-lecture.The post-test also included a questionnaire that assessed students' opinions concerning the video e-lecture.The purpose of the questionnaire was to investigate what students thought of the different aspects of the e-lecture.Thirteen of the questions were in the form of statements and the students had to rate their agreement using a five-point Likert scale, ranging from strongly agree to strongly disagree.The thirteen statements are listed below: 1.The video e-lecture was a useful learning experience; 2. Visual image of the lecturer was helpful; 3. The audio track was helpful; 4. The transcript was helpful; 5.The slides were helpful; 6.The activities were helpful; 7. The video e-lecture was interesting; 8.The video e-lecture was difficult to understand; 9.The video e-lecture was too long; 10.The video e-lecture was too slow; 11.I participated more in the video e-lecture than in any other lectures; 12.I missed the opportunity to ask questions; 13.The video e-lecture had too much information.

Richard Joiner et at
Activating Baxmind an evaluation of a Web-based video lecture with synchronized activities The last three questions were open-ended questions and concerned the students' opinions of the embedded communication activities.These were: 1. What did you think of the activities in the lecture? 2. Do you think the activities helped you understand the lecture? 3. Do you think the activities helped make the lecture more interesting?

Results
The data collected from the questionnaire were of an ordinal level of measurement.Mitchel (1997Mitchel ( , 2000) ) recommends that data of this form should be analysed using nonparametric statistical tests.Therefore all the findings were analysed with non-parametric statistics.The mean score on the pre-test was 3.75 (SD = 1.5) and the mean score on the post-test was 7.5 (SD = 2.8).This difference was significant (Wilcoxan Signed Rank Test, z = 3.2, p < 0.005) indicating that the students had improved after the video e-lecture.
The findings were analysed from the fixed-response questions in the questionnaire.First of all examining the usefulness of the different aspects of the video e-lecture.Table 1 presents the mean Likert scores for the questions designed to assess the students' opinions of the video e-lecture.It also shows the percentage of students who agreed with the statements (collapsing the strongly agree and agree categories); the percentage who neither agreed nor disagreed with the statement and the percentage who disagreed with the statement (collapsing the strongly disagree and disagree categories).We used a binomial test to check if the students significantly agreed with the statements (Siegel and Castellan, 1988).We divided the students into those who agreed with a statement and those who did not (that is, collapsing the 'disagree' category and the 'either neither agreed nor disagreed' category).For example, for question thirteen, ten students agreed with the statement 'The video electure had too much information' compared with two students who did not agree with it (one disagreed and one neither disagreed or agreed).The two-tailed probability of this occurring by chance is 0.04.
Fifty-eight per cent of the students agreed that the e-lecture was a useful experience.The students were not positive about a number of aspects of the video e-lecture.As many as 92 per cent of students did not agree that the visual image of the lecture was helpful and 83 per cent of students did not agree that they participated more in the video e-lecture compared with other lectures.Eighty-three per cent agreed that they missed the opportunity to ask questions and the same percentage agreed that the video e-lecture had too much information.On the positive side, 92 per cent agreed the transcript was helpful; 83 per cent agreed the slides were helpful and 100 per cent agreed that the activities were helpful.
The helpfulness of the activities was compared with the other parts of the course.There was a statistically significant difference between the students' perceived helpfulness of the activities and the visual image (Wilcoxan Signed Rank Test z = 3.1, p < 0.005); activities and the audio track (Wilcoxan Signed Rank Test z = 2.8, p < 0.05); activities and the transcripts (Wilcoxan Signed Rank Test z = 2.3, p < 0.05); and activities and the slides (Wilcoxan Signed Rank Test z = 3.1, p < 0.005).The students reported that the activities were more helpful than the other four aspects of the course (for example, visual image, audio track, slides and transcripts).There were no other statistically significant differences.

Disagree Neither Agree 1
The video e-lecture was a useful Learning Experience 2 Visual image of the lecturer was helpful 3 The audio track was helpful 4 The transcript was helpful 5 The slides were helpful 6 The activities were helpful 7 The video e-lecture was interesting 8 The video e-lecture was difficult to understand 9 The video e-lecture was too Long 10 The video e-lecture was too slow 11 I participated more in the video e-lecture than other lectures 12 I missed the opportunity to ask questions 13 The video e-lecture had too much information

Table I: Responses to the questions
The activities were designed with the aim of illustrating and exemplifying key points of the lecture.The transcripts were examined and they showed that the activities successfully achieved that aim.Below is an example of the student's responses for Activity 1: concerning how referents change depending on the listener.They show how the referent to a lecturer changes depending on whether the listener is a student who is on the course or a student who is not on the course.Example 2 was a discussion between two students concerning the second activity and shows that the students recognized that the referential expressions used to refer to a person whom the speaker knows are shorter and more familiar.The quotes above show that the students thought the activities reinforced and helped them understand the content of the lecture.The quote from student eleven shows that activity five (tangram activity) achieved what it was designed to do -illustrate how referring expression changes with time.
In response to the question 'do you think they helped your understanding?', the quotes below show that students thought the activities made the lecture more interesting and stressed key points.Below are some of the students' responses to the last question, 'do you think the activities help make the lecture more interesting?'They show that they thought the activities helped.

Discussion
The aim of this study was to evaluate the use of synchronous computer-mediated communication activities to support a video e-lecture.These activities required the students to communicate with each other using a text-based chat communication system.We found that students' post-test performance was significantly higher than their pre-test performance.The communication activities were designed with the aim of exemplifying key points of the lecture.Analysis of the transcripts showed that they achieved this aim.
The students also reported that the activities were the most helpful aspect of the e-lecture.They were fun and enjoyable and they reinforced the points made in the lecture.
This study shows that students' understanding of lecture content improved after participating in the video e-lecture.Unfortunately with no control group, we cannot say conclusively that this finding was the result of the video e-lecture.It is possible that these improvements would have occurred just through participating in the pre-and post-test.Morris, Joiner and Scanlon (2002) have found this with an evaluation of software designed to support statistics learning.
The students reported that the communication activities were the most helpful aspect of the lecture.This finding suggests that the incorporation of synchronous computermediated communication activities in a video e-lecture could be useful for supporting students' learning.This finding contrasts with the study reported by Cadiz et al. (2000), who found no benefit of using synchronous chat facilities compared with audio and video conferencing.One possible reason for the difference could have been the highly structured nature of the communication activities and the limited nature of the communication in the study reported in this paper.A number of studies have also found that structured synchronous computer-mediated communication was beneficial for students' learning (Baker and Lund, 1997;Hron et al, 2000).
The transcripts showed that the students had some difficulties using the synchronous computer-mediated communication system implemented in this video e-lecture.It is possible that with a more sophisticated system some of these problems would have disappeared.
Another possible solution would be to train the students before they use the system.Training has been found to be successful in the past (Cornelius and Boos, 2003;Veerman et al, 2000).Alternatively, we could use synchronous audio-or video-mediated communication.Cadiz et al (2000) found both were superior to text-based communication.
This study could be extended in a number of ways.One extension would be to develop another video e-lecture and investigate whether the same benefits reported in this paper could be replicated using a different set of students and in a different domain.Another extension would be to compare video-mediated, audio-mediated and text-based communication channels.Following on from Cadiz et al. (2000), we would expect video-mediated and audio-mediated communication to be more beneficial than text-mediated communication.Finally, a training study could be carried out to investigate whether students who were trained to use synchronous computer-mediated communication would benefit more than students who were not trained.

Conclusion
In conclusion, the aim of the study was to evaluate the use of synchronous communication activities for supporting students' learning.The students' post-test performance was significantly higher than their pre-test performance, which shows that the students benefited from participating in the video e-lecture.The activities were designed to illustrate key points in the lecture and this was confirmed through analysing the transcripts.The students reported that the most useful aspect of the lecture were the activities.They found them fun and enjoyable, although they did have some difficulties with the text-based synchronous communication system.
These findings have important implications for the use of ICT.They show that synchronous, text-based, computer-mediated communication can support students' learning from a video e-lecture.The system potentially provides the students with a means of discussing issues arising in lectures, which is economical in terms of resources and is not constrained by time and space.
Figure I: Screen shot of the Web video lecture refer to Richard to another student on the course To a fellow student on the course That's easy I have just done ft 'Richard' Fellow student NOT on the course Richard Joiner; one of my professors in understanding the conduct of the lecture and the outcome of the studies 3 12 Most of the activities helped in understanding what was being described in the lecture to chat so I guess yes.
The students were observed having difficulties with the synchronous communication system.Example 3 presents a quote illustrating this point.The two students were finding it difficult because they were typing at the same time.The responses to the open-ended questions were examined and revealed that all the students reported that the activities were good, fun, and enjoyable: