Computer-assisted learning as an alternative to didactic lectures : a study of teaching the physics of diagnostic imaging

A computer-assisted learning (CAL) package entitled Physics of Diagnostic Imaging was developed in 1995 to replace five hours of didactic lectures at the University of Glasgow Faculty of Veterinary Medicine, and has been available as an additional learning resource for students in the other five UK veterinary schools for over three years. The package was reviewed by peer experts and the reaction of the students to its use gauged by post-task questionnaire administration, informal discussions and observation. To assess the effect of integration into the curriculum, analyses of fourth-year degree examination results over a six-year period were carried out. Analyses of students' examination results for preand post-CAL delivery of the diagnostic imaging course showed that performance in the CAL-based course was significantly higher than in other subjects. This confirmed that the courseware can be used to replace didactic lectures as part of a rich learning environment supported by other resources. Initial student resistance to lecture replacement with CAL occurred, but has lessened as the use of the package has become established in the curriculum.


Introduction
In recent years there has been an expansion in the number of undergraduate students recruited to the veterinary courses in the UK veterinary schools.This growth in student numbers has not been matched by an increase in academic staff devoted to teaching.In addition, higher education has in the 1990s suffered an approximate 20 per cent reduction in government funding if the effects of inflation and student numbers are taken into account.At the same time there has been a demand for the investigation of teaching quality, leading to the Teaching Quality Assurance process (Ellis, 1993).Thus a number of converging factors have stimulated the need to look for alternative and innovative methods of teaching in veterinary undergraduate education.
It is essential for veterinary students to understand the physical processes involved in diagnostic imaging; for example the importance of radiation safety, and the recognition and elimination of artefacts which might mar clinical interpretation of radiographs.Unfortunately the physics of diagnostic imaging was a relatively unpopular part of the fourth-year course because the comprehension and retention of facts delivered during presenter-paced, didactic lectures were hampered by the disparate pre-existing knowledge of physics among veterinary students.
The Computer-assisted Learning in Veterinary Education (CLIVE) Phase 2 Teaching and Learning Technology Programme (TLTP) project has enabled the authors to change the way in which diagnostic imaging is taught at the University of Glasgow (see http://www.clive.ed.ac.uk/ for more information).The diagnostic imaging course originally comprised five lectures on the physics of diagnostic imaging complimented by lectures on radiological interpretation and film-reading practicals.The lectures on the physics of diagnostic imaging have been replaced by the use of the.Physics of Diagnostic Imaging CAL package (Sullivan, Dale and May, 1998).
The pedagogical reason for replacing a component of the course with CAL was to introduce a student-centred, self-paced, independent learning resource as a potentially more enjoyable alternative to didactic lectures.The availability of the finished package provided an opportunity to assess whether students would learn as effectively from CAL material as they apparently do from lectures, and enabled the authors to record changing student attitudes towards computers in teaching over a three-year period.Although much of the course is theory-laden, ultimately it is in the clinical situation that veterinary students will apply their knowledge of the physics of diagnostic imaging, so many real-life examples are used in the program.
The package was supported by image-based question and answer problems delivered using the CLIVE QA template developed at the University of Edinburgh (Figure 1).Since its release in 1993, self-assessment materials created using this template have been used frequently by students at all the UK veterinary schools and its role as an effective revision j Vrtat 3 (estates m the 5»tona.'y»oseeidA t; A^cfcd ertodtf -> increased ii-g^t arve w e sman eSec&ve tecs* spot it) C&PP& anode condt-cfe he«twe!t «ij Oil svrfQfxlc& the tube figure /: The QA module dlowing students to test thdr knowledge of radiological physics tool throughout the five-year veterinary degree has been documented (Holmes and Nicholls, 1996).
The relevance of this study is highlighted by the anticipated need for veterinary students to develop the self-paced, independent learning skills they will need to avail themselves of the anticipated explosion in the provision of veterinary continuing professional development (Royal College of Veterinary Surgeons, 1996,1997).

Materials and methods
The Physics of Diagnostic Imaging package consists of fifteen modules designed and written using Authorware (Macromedia), deliverable over a NetWare 3/Netware 4 network (Novell) and able to run under MS Windows 3.1I95I98INT.The package is a multimedia CAL program which incorporates problem-solving activities and questions (Figures 2 and  3) to test student understanding of the concepts introduced throughout the package.Individual units in modules were designed to take students 15-30 minutes so that the entire package could take 10 hours with the student being given 10 weeks to complete the package before the degree examination in week 14 of the semester.The components of the package are shown in Table 1.The package was intended for use by fourth-year undergraduates.
The revised CAL-based physics of diagnostic imaging course includes the following elements (five lectures having been replaced by courseware): • timetabled use of courseware (10 hours allocated), plus free access; • student notes; • standard textbooks in library; • tutorials; • film-reading tutorial; • clinical experience.£» l.lr Production of X-ravs

Generation of high speed electrons
fa carers is passed through *e csSiode *e resistance in the f.temers causes tte f.lamer* to heat: up and give off electrons that SSher round the wire as a spacecharge .

Tiy changing the current passing through the cathode by clicking on the : acrow buttons next to the ftmmeteP ointslo note]
There are more electrons because of Use greater • current.

Exit-Clkt: on !1* words m bold lost* a* ciKuZ, then the ammeter ccarois to observe titc
Oilier modules All the students in each year followed the revised course.The first cohort to use the package was the 1995-6 fourth-year intake, followed by fourth-year students in the 1996-7 and 1997-8 sessions.In comparable studies control groups have been set up where one group of students continues to attend traditional classes, and their performance is compared with that of the group using the new technology (e.g.Rogers, Regehr, Yeh and Howdieshell, 1998).However the validity of such control groups has been questioned because of the number and complexity of factors involved in learning (Gunn, 1996), and control groups were not used in this study.

Evaluation
Before development the storyboard for the Physics of Diagnostic Imaging was approved in all six UK veterinary schools by subject specialists and members of the CLIVE consortium, to ensure compatibility with courses in all the schools.

Designer evaluation
Evaluation of the navigation system and the interface design was undertaken by the Design Manager for the CLIVE consortium, and other designers, who tested the program during its development.The designers expressed a common view that the navigation system was simple and easy to follow, and that the interface was clear and aesthetically pleasing.

Peer evaluation
Five Royal College of Veterinary Surgeons holding the Diploma in Veterinary Radiology with experience of teaching diagnostic imaging physics agreed to review the CAL package, as delivered to the students.There were no major criticisms of the package.Minor points made related to spelling mistakes or minor points of factual inaccuracies.The consensus was positive and that the material was well presented and easy to follow.

External evaluation
An external evaluation group, commissioned by the CLIVE consortium, used the courseware and provided a report of their findings.The group consisted of two

Table I: Components of the Physics of Diagnostic Imaging courseware
educational technologists with experience in CAL design and evaluation.The evaluation team noted that the large corpus of information was presented in an engaging and clear way, using a variety of interactions to engage the student, rather than allowing the user to be an entirely passive individual.Like the designers, they stated that the graphic design was extremely clear and easy to view.They did have queries about the appropriateness of some interactions, and suggested putting some pop-up information in separate screens.Other comments related to simplifying instructions and making some changes to animations.In response, a second version of the program was released which incorporated the changes and several other minor improvements.This was released to students in the 1996-7 session.

Student evaluation
A post-task questionnaire was developed by the courseware designer, and approved by the content author.This was distributed to the first cohort of students (1995-6) and again to students in the 1996-7 and 1997-8 sessions.The questionnaire was not exhaustive, to avoid discouraging the students from providing feedback, but was considered sufficient to provide an insight into students' expectations and experiences of the courseware.
Observation and informal discussions with students provided further information.For a more accurate and objective measure of learning gains, fourth-year examination results between 1993 and 1998 were statistically compared.

Questionnaire-based evaluation
Fifty-five of the 78 students in the first cohort returned evaluation forms (71 per cent).In the following year 48/69 forms were returned (70 per cent).The response of the third cohort was much greater with 81/85 forms returned (95 per cent).
Reasons quoted for using the program included: • 'As a backup to the notes'; • 'To help me understand diagnostic imaging'; • 'Images can be looked at repeatedly'; • 'It was examinable'; • 'More interesting and informative than the lecture notes'; • 'Because we were told to'.
The latter response appeared more frequently in the returns of the first cohort of students.Another noticeable difference between the first and second cohort of students was the strong reluctance of the first cohort to accept the courseware as a replacement for lectures, a typical comment being: 'The program should be used as a companion to lectures, not as a replacement.'In the forms collected from the second cohort, this sentiment was not expressed.Neither was it stated by any students in the third cohort, although two students expressed a preference for textbooks over computers, stating that they could be used in the comfort of their own homes 'with a coffee'.
For the first cohort of students, only 20.1 per cent of the students responding claimed to have enjoyed using the program, however 80 per cent of these students found some or all of the modules helpful.Some students even commented generally: 'I did not enjoy using the program but I found it helpful.'In the second cohort, 70.8 per cent of students claimed to have enjoyed using the program, and 87.5 per cent of the responding students found some or all of the modules useful.In the third cohort 50 per cent of students enjoyed using the program and 84.8 per cent of students found some or all of the modules useful (Figure 4).This reduced level of enjoyment may be explained by a network server crash disrupting the availability of Version 2 of the package, resulting in students accessing two slightly different versions.
Overall, 47 per cent of the modules accessed by the first cohort were thought to have helped the students to understand the topics (315 'helpful' modules accessed / 667 modules accessed).For the following cohort of students this proportion was 62.3 per cent (364/584), and in the third cohort the proportion was even higher at 67.3 per cent (641/952) (Figure 4).Some students from the first cohort preferred short modules -the duration of longer modules contributing to their lack of enthusiasm/enjoyment -while others preferred a more comprehensive and thorough account: • 'The ultrasound section was better than the section on X-rays (shorter, faster to work through)'; • 'The ultrasound section was not as useful as the section on X-rays -it was not as explanatory'; • 'Sometimes interactive screens are unnecessary -they slow the program down'.
Students in the second and third cohorts did not make comparable statements, although one student in the third cohort requested more information on CT and MRI (the smallest modules).
Analysis of the questionnaire responses also revealed that: • most students spent between IS and 20 minutes on each module; • students were divided about the usefulness of a bibliography; • some students thought that prior knowledge was assumed, but this response declined over the three years.
No significant and recurring problems were reported by the students relating to either screen design or navigation in the courseware program (Figure 5).

Statistical analysis of examination results
Statistical analyses were performed to establish whether or not the change in course structure would affect student performance in the 1996 degree examinations of the first cohort, given that the questionnaire responses indicated that students were reluctant to accept any modifications in the course structure.The data from the 1993, 1994 and 1998 professional degree examinations were also analysed.These examinations were chosen for analysis because they included diagnostic imaging questions.
A general one-way analysis of variance (ANOVA) test was used to analyse each data set to determine whether there was a significant difference between student performance in each category/subject (i.e.p < 0.05).Subjects included anaesthesia, diagnostic imaging, equine studies, ophthalmology, orthopaedics and reproduction.To establish which subjects the students performed better in, it was necessary to carry out multiple-range tests (pairwise comparisons).
Prior to the incorporation of CAL into the course, analysis of the 1993 examination results shows a significant difference (p < 0.001) between performance in different subjects, with diagnostic imaging appearing in the lowest of two groups of subjects.In 1994 the significantly different results (p < 0.001) fall into four performance groups.Diagnostic imaging falls into the lowest group of these subject groups.After the introduction of CAL the performance in 1996 between subjects is significantly different (p < 0.001), with diagnostic imaging appearing in the highest of two groups of subjects.In 1998 performance is again significantly different (p < 0.001) and performance in diagnostic imaging is significantly higher than in any other subject.The remaining four subjects were grouped within a single lower performance group.This enables us to draw the conclusion that the examination results have improved as a result of lecture replacement with CAL.
grouped within a single lower performance group.This enables us to draw the conclusion that the examination results have improved as a result of lecture replacement with CAL.

Discussion
The results of the study are discussed in terms of module design, screen design, the role of CAL in the curriculum, attitudes to CAL and examination performance.

Module design
In the diagnostic imaging course, core concepts must be understood to appreciate the clinical significance of radiographs and other images.The arrangement of the courseware modules was such that students were encouraged to work through the modules sequentially on a first outing, although students could elect to omit the foundation modules.Each module had embedded problems that referred to previous modules, which the student was expected to solve.Hooper, O'Conner and Cheesmar (1998) stress the importance of problem-solving exercises in medical education, emphasizing that this is the process by which clinical diagnoses are made.The reiteration of important concepts in different ways has been shown to help students integrate information, constructing knowledge by linking new information to information previously given (Grabinger and Dunlap, 1994).The use of the Edinburgh QA template to construct a self-assessment set of questions allowed the same knowledge to be tested, by providing visual clues, helping to reinforce the key points illustrated in the CAL package.

Screen design
The questionnaire analysis indicated that few students had problems using the interface.The main navigation controls in the program included a paging model which enabled the user to go backwards or forwards one screen, or to jump to the start or end of the current section.It is also possible to jump to a list of modules; progress being indicated by 'ticks' beside the title of each visited module, giving the student freedom to enter or exit the package as they see fit.

The role of CAL in the curriculum
One of the most frequent responses in the student questionnaire from the first cohort was that 'the program should be used as a companion to lectures, not as a replacement'.This response did not figure in the responses by the second cohort of students.This initial reaction to veterinary CAL has been observed previously in the United States (Weeks, Smith and Martin, 1992).
Where use has been monitored in the CLFVE institutions, it has been observed that students are motivated to use courseware if its use is timetabled, or recommended to coincide with particular lectures, practicals, or resource-based learning, i.e. integrated into the curriculum.Further scheduled CAL time, in addition to free access, was considered to be beneficial to students, as it has been argued that access to courseware in timetabled classes facilitates collaboration between students (Doughty et ah, 1995).
However, an increasing number of students are running CLIVE programs on home computers; this is shown by the increasing number of requests for overnight borrowing of CDROMs from the University of Glasgow veterinary library.The Dearing Report has recommended that all students should have access to their own portable computers by 2005/6 (National Committee of Inquiry into Higher Education, 1997).This will make CAL more accessible to the small number of students who commented that they did not like working in a 'noisy' computer cluster, and will satisfy more of those students who prefer to study at home.
Students generally prefer to use courseware for preparation and revision and the importance of CAL as an additional learning resource at the University of Glasgow has been exemplified by repeated requests for additional courseware covering a range of veterinary topics.The initial preference for CAL as a supplementary aid to learning may be explained by the feeling of deprivation induced by the lack of classical (didactic) lectures, and the novelty of CAL, where the onus is placed on the student rather than the teacher to drive the learning process.

Attitudes to CAL
Students were told at the start of the academic year that they would be examined on their knowledge of the subject.Assessment has long been recognized elsewhere as the sole motivation for using a resource which might be regarded as optional, but this may also engender negative feelings towards it (Scanlon, Jones and O'Shea, 1987).It is relevant to note that the initial negative opinions of the students in the first cohort diametrically opposed the responses from a peer review survey.Peers, compared to students, are likely to have a clearer idea of the educational imperatives and the desired outcome.This view is supported by Draper, Brown, Henderson and McAteer (1996) who noted that 'Students express quite strongly positive and negative views about a piece of courseware that often seem unrelated to their actual educational value'.
Other factors which may have influenced how CAL was received by the first cohort of students as a replacement for lectures are (i) the students in the first cohort had no formal training in IT, (ii) the subject was examinable, and (iii) the previous group of fourth-year students were given lectures.For the following two cohorts, CAL had already replaced lectures and the majority of students had more experience of using computers, although this was still variable.
A higher proportion of the second cohort of students enjoyed using the program, compared with the first cohort.This may be due to the fact that integration had been achieved and CAL was now accepted as part of the course.A smaller proportion of students in the third year claimed to have enjoyed the program than in the second year, but more students found the courseware modules useful.

Examination analysis
The investigation of the variance of scores indicates that in the pre-CAL years, student performance in diagnostic imaging was either on a par with other subjects, or significantly below the other subjects.In the post-CAL examinations, student performance in diagnostic imaging was significantly higher than in other topics, and exclusively the highest in the most recent examination.This would imply that the replacement of introductory lectures with CAL has improved student performance in this subject, despite initial reluctance by students to accept the change in learning methods.There are a number of possible reasons for the high diagnostic imaging scores in the post-CAL examinations.The most pessimistic is that given the perceived adverse change in course structure students compensated by working harder (Draper et al., 1996), using other resources in a remedial fashion.This might assume that all students considered the change adverse; however, this was not borne out by the questionnaire responses.The possibility that the questions were easier, or that the papers could have been marked more leniently, in the three years post-CAL is unlikely since the same person set and marked the examination papers each year.The most favourable explanation would be that the change in learning resources did help the students to learn at their own pace and fostered deep learning.This enabled them to apply their knowledge of facts derived from using the courseware to the rest of the course, e.g.radiopathology tutorials, and engage in collaborative dialogue with each other and the teacher during filmreading sessions.

Conclusion
The key issues to emerge for the future of CAL in the veterinary curriculum are overcoming initial student resistance, clear full integration into the curriculum, and making courseware available outside traditional scheduled timetables.The fact that design and navigation were not an issue indicated that a well designed package could be extended to CPD use, where users may not be particularly IT literate.The results of this study are very encouraging and indicate that CAL-based open-learning resources can provide an effective alternative to conventional lectures.

Figure
Figure 2: A saeenshot illustrating interaction fa the Physics of Diagnostic imaging

Figure 4 :
Figure 4: Analysis of the evaluation form responses