Online self-assessment materials : do these make a difference to student learning ?

During the last decade the teaching of students in first-year biology has changed from a teacher-centred focus to a student-centred focus. The change was designed to encourage students to take responsibility for their learning, develop team and communication skills and put in practice those skills required for lifelong learning. Students are introduced to small learning communities (in large classes) that give the students a sense of belonging and a peer support group. Activities have been devised and implemented to support student-centred learning, which in more recent years has included using computers. All these activities are integrated into the course design so that the students are offered an array of learning opportunities relevant to the course(s) learning outcomes. An important requirement for the development of student-centred learning is suitable and timely feedback that gives students guidance about their learning outcomes. Giving feedback to very large classes is an expensive commodity and one that is vulnerable in the current climate of reducing resources and increasing student numbers. First-year biology students receive online feedback from a weekly quiz (with both formative and summative components), from a mid-semester mock exam (formative only) and from a series of self-assessment modules (formative only). This paper will examine the use of such online self-assessment in a large first-year biology class, discuss current evaluations of the materials and propose further research into how students use these integrated learning opportunities.


Introduction
In our changing world where university students are demanding a greater say in their tertiary education, and in particular are demanding a greater flexibility in the way they receive their instruction, it is imperative that we investigate and experiment with course delivery strategies that fulfil these expectations.Online delivery of learning materials is not new, nor is it the panacea for all problems, but it does offer certain advantages for both teachers and students.The flexibility of using the Web may suit certain teaching activities, but more importantly may suit the learning styles and commitments of the students.In the current economic climate students may have to juggle university activities with employment, potentially missing some of the structured teaching and learning sessions.In addition they may not have time to use campus-based course materials or seek face-to-face assistance from staff.An Australian benchmark survey of the first-year experience (Mclnnis, James and McNaught, 1995), found the pressures of part-time work made it extremely difficult for some students to fulfil course expectations.A 1998 survey of firstyear science students at the University of Sydney revealed that 54 per cent of full-time students are undertaking some form of employment, with 31 per cent working ten hours or more per week during semester, and 14 per cent working over fifteen hours per week (Peat and Franklin, 1998).A small shift away from courses comprising all face-to-face activities to courses with a mix of face-to-face and online activities has the potential to help those very students who may otherwise give up when the pressure of time and other commitments seems too difficult to cope with.
Courses with a flexible structure that offer choice to the learner provide for the development of self-directed learning.This will enhance lifelong learning as students develop a student-centred approach to their studies, adopt deep learning strategies and develop contextual appreciation of content (Candy, Crebert and O'Leary, 1994).It has been argued that changes that would allow a more flexible approach include improved access to learning resources, provision of flexible student support systems (which should include counselling services, bridging, catch-up, remedial and study skills courses) and the development of learning resources and experiences that cater for different learning styles (Lewis, 1993).Computer learning resources delivered on the Web can fit these descriptors but, to be effective, courseware must offer an opportunity for learners to address a topic through a range of activities, appropriate for all learning styles (Valley, 1997).
Feedback on performance is a valuable tool in the learning process, especially feedback of a formative nature that provides suitable opportunities for students to gain a personal insight into their understanding of the course content (e.g.Zakrzewski and Bull, 1998;Macdonald, Mason and Heap, 1999).In the past early formative feedback was considered a normal component of science-based courses (as in the use of weekly quizzes, paper-based laboratory reports, solutions to questions, etc.) and much of it was followed up with faceto-face encounters between teachers and students.Now with increasing student numbers and decreasing staff resources, we need to come up with ways that help students help themselves and the whole issue of giving suitable and early feedback to first-year students needs to be revisited, it would seem, in most institutions.One way to help solve some of these problems is the judicious use of self-assessment on the Web which for many teachers is a viable option that can provide valuable information for students about their progress (Butcher, Stefani and Tariq, 1995;Edwards, 1989;Stefani, 1994).The paper will discuss self-assessment materials available on the Web to a large first-year class, and in particular will concentrate on a set of self-assessment modules designed, using Bloom's taxonomy, to test different cognitive levels of learning.

The Sydney problem
In Australia, first-year science courses have high student numbers, as these are also the courses that service the other sciences (e.g. medical, veterinary, agricultural, engineering etc).This large group of students (1,500 in biology at the University of Sydney) is also very heterogeneous, characterized by varied academic backgrounds (with a large range in the incoming entry grades, and some without biology qualifications), with varying interest in biology, and with a range of incoming generic skills (writing, computer, team work, etc.).In addition, many of the students arrive with an expectation of being spoon-fed, having been conditioned to using a surface approach to learning in high school.An additional problem, with an ever-increasing number of students, is a disinclination by staff to be involved in first-year teaching.For many staff, working with first years is now seen to be a less valuable activity than supporting the learning environments of higher-year students (which yield honours and postgraduate candidates), as previously recognized (Christopoulos, Rohwer and Thomas, 1987).In first-year biology, the Web is used to create a better learning environment for students that is more independent of teacher interaction, that is sustainable in the current economic climate and that encourages the development of lifelong learning strategies.

The Sydney biology solution
For over ten years now, within the first-year biology courses, there has been a move to change the philosophy from teaching students to facilitating student learning, especially within the laboratory experience.Early changes involved organizational and structural issues, such as the way the laboratory classes were designed or reviewing which activities best suited a student-centred learning approach.Small learning communities in large classes were created that offered group activities such as card and board games (based on familiar games strategies, e.g.Happy Families, Snakes and Ladders), group concept mapping tasks, group field work, group poster presentations and other tasks designed to facilitate learning.The introduction of computers in 1994 as a permanent feature in the laboratory, and the development of computer-assisted learning modules (CAL) to target specific learning difficulties, led to a focus on the use of computers in the learning process.In particular the use of revision and self-assessment materials was targeted.Anecdotal evidence from face-to-face contact with students had indicated that many of them (and we suspect those at the lower end of our intake) were finding it difficult to understand the concepts within our discipline and the linkages between concepts that are required for them to perform satisfactorily in examinations.Our approach to this was to develop a set of self-assessment modules that allows students to test themselves.
The aim of a self-assessment module (SAM) is to draw together related parts of the course to help students make connections between topics in biology and to promote a deeper learning strategy, whilst providing an enjoyable feedback and reinforcement session.These are additional, optional materials designed to let students identify their level of understanding.Whilst the courses are thematic, for example, the first semester course covers the requirements of life, basic ecology, role of microbes, cells and energy (metabolism), cells and chromosomes (mutations, evolution), the SAMs are organized around plants, animals and microbes (with genetics yet to come online).Thus in using a SAM, students are taken down a lateral pathway and so encouraged to see the relationship between the materials.
The development of SAMs in a non-course specific way allows for their constant reuse even when the course themes change.
Each SAM tests the students on four levels of increasing difficulty, using Bloom's taxonomy of educational objectives (Bloom, 1956) as the guide to develop the levels.Thus the content of the questions can be reused (from level to level) but with an increasing cognitive requirement and appropriate question types have been developed for each level of difficulty.Table 1 illustrates the question levels.Level 1 tests content and knowledge with the use of multiple-choice questions and drag and drop scenarios, but with the answer always on the screen.Level 2 tests application of content using some multiple choice, but mostly with a format that expects text input from the students.Level 3 tests analysis and uses question formats as for Level 2, but with the addition of two-part questions and formats requiring the building up of diagrams, flowcharts, etc. Level 4 tests synthesis of information and the most used format is free-flow prose where the student is expected to synthesize information in response to a question.This format is not computer-marked but marked by the students from sample answers, and with the option of self-scoring their own performance.SAMs are used by students individually or in groups (allowing for peer discussion, cooperative learning and team work).There is an entry point for each SAM which is designed to act as a barrier to progression if the student is unable to complete it successfully.It was envisaged that students unable to pass the entry point would revise further before attempting the SAM again.The entry point consists of a set of five true/false questions (selected from a small question bank), with questions of a Level 2-3 cognition requirement, allowing students with 80 per cent correct to proceed and denying further access to those students unable to achieve this level of performance.Once past the barrier students can choose to do any level in any order and the program allows the students to know where they have been and how they are performing.At any time they can ask for a progress report.The design of templates for the questions at different levels, discussed elsewhere (Peat, Franklin and Mackay-Wood, 1997), allows for the easy production of new SAMs.

Evaluations
Feedback on the use of the SAMs has been on a voluntary basis and all SAMs have an optional log-out feedback form, which is automatically generated after completion of the module, asking about the best aspects of the module, the worst aspects of the module and how we might improve the module for the students.Whilst this log-out form clearly irritates some students (who are not interested in giving us feedback), analysis of the 1998 and 1999 responses has given us some useful information which will help make improvements.With respect to the best aspects, the student comments are of a generic nature and not about the content.The students like the immediate feedback, the clarity of material, the variety of question types, the interactivity of the questions, the ease of use and the availability (any time, any place, any pace).In addition they indicate that such a package is motivational (not intended in the design), fun to use and good for revision.
Motivated me to work, showed me my weaknesses and what I have to revise.Can use these modules when I want... easy to use and understand.Clarity and immediate marking.Choice of level of difficulty.
The responses to the 'worst aspects' highlight two areas -technical problems (mostly to do with hardware problems, e.g.insufficient RAM, and not in our control), and details about content and in particular criticism about the restrictions imposed by an entry requirement (80 per cent pass for entry): It took a long time for questions to download.Some diagrams did not exist.The 'entry' questions -1 could answer the questions once I managed to get through these.
Many students do not see the entry test to have any value and these students are using the module for revision and not self-assessment and so want to go through the entire module even if they are not going to perform very well.The entry test may, therefore, be inappropriate.Some students indicated that they repeat the entry-level test until successful rather than doing any revision and that an 'inability' to pass the entry test does not mean they cannot answer the questions in the main test.This is to be expected from the design with four levels of increasing cognitive requirement and the entry test on a Level 2-3 degree of difficulty.The hardware problem will be addressed with additional information online for the students about the technical requirements of the SAMs.The entry requirement will be removed, especially as it appears students are using the SAMs as a revision aid as well as a performance indicator.

Discussion
It has been shown (Entwistle, Hounsell, Macaulay, Situnayake and Tait, 1989) that an important contributing cause of failure of first-year students is an absence of feedback on progress and this is also cited as a reason to discontinue (Mclnnis et al, 1995).In larger classes it is often difficult to provide individual feedback to students and it is recognized (Ramsden, 1982) that it is not always necessary for academic staff to give feedback if another option is available and self-assessment is a suitable option.The self-assessment materials in this study offer students a variety of opportunities to test their understanding and to identify their cognitive working level without the need to seek face-to-face assistance from the teaching staff.The students in the present study found the materials challenging, helpful and beneficial.This is supported by other studies on the use of selfassessment materials.For example, in a similar study with first-year physiology students 98 per cent of students thought that the self-assessment had made them think more and 71 per cent felt they had learned more (Orsmond, Merry and Reiling, 1997).
The design of the SAMs does not utilize the full range of Bloom's taxonomy (Bloom, 1956).Of Bloom's six categories, evaluation -the ability to evaluate the worth of material, theories, methods, information, etc. for a given purpose -was not included in the design of the SAMs as it was considered too difficult to portray using the computer and was best assessed by other means.A criticism of the SAMs is that we have not given the students sufficient support in using them.Fazey (1993) argues that students may need more instruction in the use of self-assessment materials and should be provided with guidance and given opportunities to practise these skills.A future development, therefore, will be to include a set of performance outcomes for students to identify with, based on some real data.For example a high performance at Level 1 and 2, with poor performance at Level 3 and 4 might indicate good rote learning skills but poor application and analysis.

Conclusion
In the first year, biology students are encouraged to develop collaborative learning strategies in their face-to-face encounters (by setting up small learning communities in large classes).They are offered guided learning scenarios as opposed to discovery learning, with a more flexible delivery for some of their materials, so that they can choose when they want to be engaged in these activities.Student responses to surveys show that the technology is being used in a way that is helping them to learn, in a way that suits their lifestyle and which we hope will enhance opportunities for participation in higher education.If we can present this within a learning style framework then we may be able to persuade the students to change their learning behaviours to include deep and meaningful learning.The mix of online materials available result in valuable and realistic learning opportunities for 1,500 students and these are empowering them to take responsibility for their learning and to develop the lifelong learning skills required of our graduates.
University teaching and student learning are moving through transition processes, driven by many factors including changing student requirements and economic forces.In the next few years we all need to adjust our mix of online and face-to-face teaching experiences and develop measures to evaluate the appropriateness of them.The next few years will show us the ways in which students will want to learn and we need to be flexible in accommodating their preferred learning strategies.

Table I :
Question levels in self-assessment modules.