Mobile phone voting for participation and engagement in a large compulsory law course

Chad Habel^a* and Matthew Stubbs^b

^aSchool of Education, University of Adelaide, Adelaide, Australia; ^bAdelaide Law School, University of Adelaide, Adelaide, Australia

(Received 14 August 2012; final version received 11 March 2014; Published: 8 April 2014)

Abstract

This article reports on an action-research project designed to investigate the effect of a technological intervention on the complex interactions between student engagement, participation, attendance and preparation in a large lecture delivered as part of a compulsory first-year law course, a discipline which has not been the focus of any previous study. The technology used was VotApedia, a form of mobile phone voting, and it was implemented in tandem with constructivist pedagogies such as explicit pre-reading and a prior context of interactive lecturing. Data were collected through observation, via mobile phone voting in class and by an online survey designed to specifically explore the relationship between attendance at VotApedia lectures and factors such as self-reported engagement, attendance and preparation. The findings indicated that student response systems (SRSs) are just as applicable to more Humanities-style disciplines which require divergent questioning, and supported complex interactions between engagement, attendance and preparation. Preliminary findings indicated that, although more work needs to be done, especially on the types of students who prefer to use these systems, there is a clear potential to increase student engagement in large law lectures through the use of SRSs.

Keywords: student response systems; pedagogy; VotApedia; constructivism; action research

^*Corresponding author. Email: chad.habel@adelaide.edu.au

Research in Learning Technology 2014. © 2014 C. Habel and M. Stubbs. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons CC-BY 4.0 License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Citation: Research in Learning Technology 2014, 22: 19537 - http://dx.doi.org/10.3402/rlt.v22.19537

Introduction

Education is not a spectator sport; it is a transforming encounter. It demands active engagement, not passive submission; personal participation, not listless attendance. (Rhodes 2001, p. 65)

Student engagement is an essential precondition for effective learning in Higher Education, especially in a context of increased transparency and quality assurance (Coates 2005). The challenges of encouraging student attendance and engagement (particularly in an era when lecture slides and audio/visual recordings are available online) are particularly acute in large, compulsory first-year courses, which have somewhat alarmingly been called ‘the bane of active learning pedagogy’ (Trees and Jackson 2007, p. 21). Encouraging active learning and engagement has also been acknowledged as essential for first-year law students (Brooman 2011, p. 110). More particularly, the relevance for Australian law schools who wish to enhance student attendance and engagement has been identified as an area for further study (Corbin, Burns, and Chrzanowski 2010, p. 40; Stevens et al. 2006).

Our research builds on previous work exploring the effect of mobile phone voting on international student engagement (Habel 2009). The aim was to measure student engagement and its interaction with other factors in a discipline which is not usually a candidate for such interventions or research. This project proposes a rigorous, iterative method for exploring this question by adapting Eric Mazur’s notion of ‘ConcepTests’ (Crouch and Mazur 2001; Lasry, Mazur, and Watkins 2008) and by collecting data during the implementation of the learning strategy. A recent parallel work has explored the potential of clicker technology in a class of around 200 in a North American chemistry course (Terrion and Aceti 2012), testifying to the current interest in the dissemination of student response systems (SRSs). We investigate audience response through the free mobile-phone voting system VotApedia, which does not require the purchase of any technological system. This is important in the current Australian context of widening participation from low socioeconomic groups (DEEWR 2009). We explored whether the documented benefits of SRSs would also apply in courses outside of the hard sciences and looked for relationships between peer learning, participation and engagement. Our findings support the use of SRSs in more diverse contexts and demonstrate that student responses are broadly positive, with some exceptions.

SRSs in higher education

Contrary to assumptions, the use of SRSs such as clickers is not new; they have been in use since at least the 1960s (Judson and Sawada 2002). There is a large body of research already on the use of SRSs in educational settings, which has been reviewed extensively (Caldwell 2007; Fies and Marshall 2006; Judson and Sawada 2002; Kay and LeSage 2009; Simpson and Oliver 2007). The vast majority of this research concurs that using SRSs in the classroom has a positive or at least benign influence, especially on factors such as student engagement, participation and peer learning potential (Caldwell 2007). A common thread through nearly all of the literature is the importance of combining SRSs with social constructivist pedagogies: ‘it is instructional method itself that is important when considering implementing an electronic response system’ (Judson and Sawada 2002, p. 178). The critical task is to deploy SRSs to ‘provide the tools for creating authentic learning environments and fostering the communication channels that support the social construction of learning and understanding’ (Massingham and Herrington 2006, p. 98).

One of the more thorough literature reviews, by Kay and LeSage (2009), identifies areas for further research. They report that further research needs to be undertaken in a broader range of discipline areas, as most of the implementation and research so far has been conducted in mathematics and hard sciences, a finding that has been echoed elsewhere (Judson and Sawada 2002; Simpson and Oliver 2007). This is now changing: Habel (2009) explored the use of SRSs in academic skills lectures for international students; and SRSs have been used in Marketing (Sprague and Dahl 2010), Management (Keough 2012) and Nursing (Patterson, Kilpatrick, and Woebkenberg 2010). This is about more than the challenge of applying an established pedagogical technology to a new context, though: non-scientific disciplines are more likely to contend with divergent rather than convergent questions, which is a challenge for technology and e-assessment more broadly (Crisp 2007). A related question which is largely unexplored by the literature is the type of students who find the most benefit from using SRSs; Sprague and Dahl (2010) discovered than both international students and students with low need for cognition particularly enjoyed its use: ‘This finding is important because it validates this technology as a valuable methodology to facilitate learning with students who typically do not enjoy more traditional methods inherent in a university learning environment’ (p. 97).

Furthermore, while Kay and LeSage (2009) indicate that several authors recommend that SRSs be used in conjunction with explicit student preparation for lectures, these pieces do not go much beyond this recommendation. D’Inverno, Davis and White (2003) do recommend the provision of lecture notes before (rather than after) lectures, but the outcomes and effects of providing skeletal notes prior to Engineering Mathematics classes are reported in only the most anecdotal fashion. In addition, much research into the use of SRSs seems to focus on specific pedagogical strategies in relatively isolated contexts. There is a strong consensus that ‘clickers are not, by themselves, sufficient to increase subjective reports of engagement’ (Morling et al. 2008, p. 49). According to Kay and LeSage (2009), three of the most important factors – engagement, peer-based instruction and student preparation – were hardly explored in conjunction (with the exception that three authors focussed on engagement and peer-based instruction). Finally, Kay and LeSage note that there are no accounts of why students find SRSs so engaging and recommend the collection of qualitative data to help explore this factor.

Our action research design developed directly from the gaps in the literature identified above. To begin with, our work is in a different context to most work in SRSs: although they are being used more in human sciences and business disciplines, there are to our knowledge no applications of their use in humanities or law-related fields. It is particularly important to explore the use of SRSs in the context of divergent questions where the emphasis is on interpretation, discussion and the use of evidence to support claims. This responds to ‘the need for a concerted research effort, one that rigorously explores conditions of use across diverse settings and pedagogies’ (Fies and Marshall 2006, p. 101). More specifically, the use of SRSs in this discursive context of divergent questioning supports the constructivist pedagogies that are essential to the implementation of SRSs.

Furthermore, SRSs were used explicitly in conjunction with both attendance and preparation activities. It is not yet common for pedagogical strategies to explicitly address preparation for large lectures (for an exception see Michaelsen and Sweet 2008), and this research both emphasised preparation to students and built it into the questions asked via the SRS. This was combined with an investigation into attendance, engagement and peer-based instruction to explore how these factors combined to support student learning in large lectures. Finally, qualitative data from surveys were drawn on to try and elicit reasons for enhanced engagement through SRSs and to attempt to explore the type of students for whom SRSs are particularly useful.

Our research questions

The research questions we sought to address were as follows:

1. Can the use of SRSs in a large first-year law lecture promote student engagement?
2. What combination of SRS, constructivist pedagogy and peer-learning is most effective to achieve increased student preparation, participation and engagement in a large first-year law lecture?

Intervention methodology

We conducted a trial of using mobile phone voting in three lectures in the undergraduate law course Principles of Public Law (enrolment 460, 2nd semester, 2011). VotApedia (a free mobile phone voting system) was employed for each of three lectures throughout the semester: this allowed an iterative and developmental approach to the implementation and avoided the selection bias and contingencies that would occur with a single implementation. It was decided to introduce three lectures using VotApedia in order to moderate the perception of ‘dumbing down’ or the reduction of content coverage that is so common in the use of such systems and to ensure that student expectations regarding lecture format were still being met (D’Inverno, Davis, and White 2003).

It is also very important to avoid contrived research design whereby the use of SRSs is simply compared with a traditional, passive lecture format. The lecturer regularly uses interactive formats by having groups of students work on small problems and linking this to the lecture material, so when students were asked to ‘compare the VotApedia lectures to Matthew’s normal lectures’ they were reflecting on comparable examples. In this way, students had the opportunity to compare the ‘treatment’ lectures with ‘control’ experiences which were still interactive but lacked the element of voting after group work.¹ This methodology allowed us to directly test student perceptions of engagement using SRSs so as to address our first research question of whether there is a benefit in terms of student engagement arising from our intervention.

Cyclical action research

Use of a cyclical action research methodology allowed us to interrogate our second research question, which is directed to the complex question of which combination of techniques is most effective to achieve our desired goal of increased student engagement. This project built on two separate implementations which created cycles of research with increasing levels of rigour. Habel (2009) used VotApedia with international students to enhance engagement with issues around academic integrity, plagiarism and the use of text-matching software. Although higher engagement was casually observed, it was not properly documented or analysed. In 2010, the second author began a preliminary implementation of VotApedia in Principles of Public Law. Again, heightened engagement was observed and responses in end-of-semester evaluations were positive, but this was not sufficient to properly establish the effect of the intervention. Therefore, the current study was designed to explicitly explore the effect of SRS usage on peer-learning, student preparation and engagement through the use of VotApedia in large first-year law lectures, a discipline that has not previously been used for such implementations.

The implementation was inspired by the work of Eric Mazur, who pioneered peer-learning pedagogies through using SRSs (Lasry, Mazur, and Watkins 2008). In each of the 3 weeks of our project, students were provided with a two-page pre-reading material and asked to complete that reading before attending the lecture. As in all weeks of the course, students also had advance access to the lecture slides, a topic guide and set readings which were designed to introduce them to the topic being examined, and which were built into the SRS activity.

In the lecture, we asked three substantive questions arising from the assigned pre-reading. For each question, students were first given a short lecture introduction to the topic (5–10 minutes), before being split into groups (of approximately three students) to discuss which answer was correct.² One student from each group would then vote (using their mobile telephone) for their group’s preferred answer from a choice of four given responses (see Figure 1). Once voting was completed, an interactive discussion would follow, during which student volunteers explained the reasoning behind their choices to the whole class, the results of the voting were revealed and discussed, and the strengths and weaknesses of each potential answer were examined in a group discussion led by the lecturer. We did not seek merely a ‘correct’ answer, but engaged in the processes of higher-level reasoning leading to a more complex understanding of the strengths and weaknesses of different approaches, thus pursuing the divergent exploration of issues necessitated by the research design. This process would then be repeated for the remaining two substantive questions.³

Figure 1.  Mobile phone voting in action.

In addition to the substantive questions posed, we also asked some questions whose purpose was purely to assess the benefits of our use of mobile phone voting to engage students through explicitly requiring preparation. Thus, at the beginning of each lecture, students were asked if they had completed the pre-reading, and how confident (on a 1–5 Likert scale) they were about their ability to correctly answer the questions posed. At the end of each lecture, students were asked if they found the pre-reading useful to their understanding of the issues addressed. The project implementation and research design is illustrated in Appendix.

This process was conducted in three 2-hour lectures during the course (in weeks 8, 10 and 11 of the semester). In addition, we undertook observational analysis of the learning environment during our classes: the primary author was available solely for observation, while the second author was engaged in delivering the lecture and simultaneously observing. At the conclusion of the final session, students were asked whether (on a 1–5 Likert scale) the lectures with mobile phone voting were ‘more engaging than Matthew’s regular lectures’ which, as explained above, regularly involved interactive question-based activities and group work.

After these lectures, we administered an optional on-line survey to elicit further information about students’ attitudes and perceptions of the use of mobile phone voting in lectures. This follows the approach of valuing self-report of engagement and satisfaction in the Australian Survey of Student Engagement by the Australian Council for Educational Research (ACER) (Coates 2008), as long as it is combined with other data. Students also provided feedback on mobile phone voting in their Student Experience of Learning and Teaching (SELT) surveys completed in the final class of the semester. It could be argued that these SELT results are overly contextual, in that they only focus on student experience in one specific course. However, because all these students have previous experience of different courses and lecturers, and the SELTs follow an institution-wide format, students do actually respond in the wider context of their experiences at university. This variety of data sources allowed for triangulation (as recommended by ACER) to effectively analyse the impact of VotApedia upon student preparation and engagement in a divergent questioning context.

Action research results

Group discussion increases student performance

In the first week of the project, we asked students to answer each question first individually, then to discuss their answer in small groups and vote again. For each of our three questions, as reported in Table 1, group discussion led to a measurable improvement in student performance.

These increases were statistically significant (p<0.05 for the first two questions and p<0.10 for the third),⁴ which substantiates the value of group discussions to student understanding, confirming the importance of constructivist pedagogies in the deployment of SRSs (Judson and Sawada 2002). This corroborates our suspicion that the benefits of peer-learning through SRSs that have been observed in the hard sciences also apply to more humanities-based disciplines, and should motivate educators to consider using SRSs in a broader range of disciplines.

Preparation is affected by engagement and facilitated by SRSs

The results for student preparation are presented in Table 2. Looking just at data for the first 2 weeks, there is encouraging evidence that students are prepared to put in the effort to do the pre-reading once they see the benefits that it can bring. Thus, in week 1 the pre-reading was done by 64% of students and valued by 70% of students. In week 2, this positive experience from week 1 prompted increased preparation – 74% of students did the pre-reading and 80% found it valuable. The negligible standard errors suggest that the results are valid.

This positive response in week 2 might generate an expectation of increased preparation in week 3. However, the extent of preparation in week 3 dropped to only 42%. In week 3, our lecture clashed with a major assessment in another course (in which most of our students were enrolled). This clash with assessment was a stronger (negative) influence than the (positive) influence of increased engagement.

In all cases, the standard error of the proportion was within acceptable limits, suggesting that students’ self-reports of completion and usefulness were valid. Given the informal and anonymous nature of the responses there is no reason to suspect that students would misrepresent their views.

The impact of the preparation of assessment items on student attendance has been previously noted (Corbin, Burns and Chrzanowski 2010). What this research adds is the impact of assessment items, even in other courses, on engagement – in the critical week where there was a clash between our lecture and assessment completion for another course, the rate of reading even our short two-page pre-reading was noticeably lower. This emphasises the importance of thorough planning across courses and even programs, where possible, or perhaps a stronger focus on time management or the management of expectations with students (Figure 2).

Figure 2.  Group discussion and voting.

Student perceptions of engagement

Evidence of increased engagement

A number of different data sources suggest that mobile phone voting led to increased student engagement in our course. Observational analysis from both authors indicated much more energy and liveliness in the room during mobile phone voting, and particularly sharp spikes when ‘correct’ answers were revealed. In addition, at the end of week 3 students were asked whether they found lectures with mobile phone voting ‘more engaging than Matthew’s standard lectures’. There was 84% broad agreement⁵ that they were, as reported in bold in Table 3.

This data is supported by the evidence of student perception gained from responses to SELT surveys at the end of the course. Responses to the prompt ‘This person encourages student participation’ (on a 1–7 Likert scale) are reported in (Table 4), comparing the 2011 results with those obtained in 2010, when the same course was taught by the same staff member.

Comparing the 2 years, there is a marked increase (8%, p=0.033 on a two-tailed t-test) in student perceptions of the lecturer’s encouragement of participation. The introduction of mobile phone voting was the only substantial intentional difference in course methodology between the 2 years, as previous years employed interactive, social constructivist pedagogies involving explicit preparation for lectures and group work and discussion.⁶ Indeed, in the previous year the lecturer even used a paper-based quiz to encourage engagement through questioning and so the inclusion of technology was part of an on-going pedagogical development. In this longer-term context there was still improvement via the use of the SRS. Furthermore, this was the only SELT measure of student experience to change dramatically between the 2 years, which suggests that it was mobile phone voting that caused this increase in student perception of the encouragement of participation by the lecturer.

Student self-report of increased engagement linked to attendance

At the end of the semester an online survey was administered to elicit students’ own perceptions of their engagement through the use of mobile phone voting. This survey went well beyond a simple evaluation of the teaching and learning experience: it was explicitly designed to explore relationships between attendance at lectures where VotApedia was used and subjective reports of engagement, using both quantitative and qualitative data. This relationship between attendance and engagement is an under-researched area. Students were asked how many VotApedia lectures they attended; since there were only three the margin of error is small and it can be assumed that their responses to this question were accurate. Based on their response, students were then asked how engaging they found Matthew’s ‘normal’ lectures and how engaging they found the VotApedia lectures. Results are reported in Table 5.

These results indicate significantly higher self-reports of engagement based on the use of VotApedia (p<0.05 for students who attended two or three VotApedia lectures; although the result for the small number of students who attended only one lecture was not statistically significant), but they must also be interpreted with caution. There is clearly some selection bias operating here; obviously students who are already engaged in learning are more likely to both attend more lectures (including VotApedia lectures) and to complete the survey. Here we encounter the common problem of eliciting results regarding students who are disengaged. Furthermore, it is clear that many students already feel that lectures are engaging, so the use of VotApedia is building on a base of strong pedagogical practice. Additionally, there were pockets of dissatisfaction with the use of VotApedia that must not be discounted, and appeared more strongly in the qualitative data. Nonetheless, these results are encouraging and demonstrate a clear link between attendance and engagement when SRSs are used to supplement an already interactive lecture style.

In addition to Likert-style questions, students were asked open-ended questions which provide more nuanced information. These comments were also grouped according to attendance at VotApedia lectures to further explore relationships between attendance and engagement and were subjected to a thematic content analysis based on an initial interpretation of the comments. The themes that emerged from the data included positive, negative and ambivalent assessments of VotApedia; comments that VotApedia made the lecture interactive, engaging or interesting; suggestions that it should be used more in lectures; and comments that it used too much time that would be better spent otherwise. To ensure complete representation of the responses, some responses were double coded: for example, if a statement had a generally positive comment followed by a comment that it took too much time, it was included in both categories (Table 6).

Clearly, the overwhelming qualitative response was that students felt that the use of VotApedia was engaging. For example, ‘I’m a student who often finds myself in ‘la la land’ in lectures, no matter how interesting the topic/lecturer, so for me it really helped to make me focus and actually learn something!’ This indicates that SRSs are particularly useful for students who are usually disengaged. Many respondents used adjectives such as engaging, interesting, or interactive, while others referred to group discussion or had more sophisticated ways of expressing a deeper relationship with the material or learning context. Many students had generally positive comments about the use of VotApedia, and some even suggested that the technology helped them retain information for quizzes and the exam, indicating that it may also serve traditional functions of helping memory retention. For example, ‘I have hence noticed a significant benefit in the use of VotApedia in regards to my exam revision—I have retained more information from these lectures in comparison to the ‘normal’ ones’. Several of these positive comments revolved around how VotApedia motivated preparation; for example, one responded that they only found it useful ‘after doing the pre-readings—I found out the hard way!’ This again reinforces the links between attendance and engagement, as facilitated by the use of SRSs.

Despite these overall positive responses, there was also notable ambivalence: several respondents combined positive comments (even about engagement) with criticisms of the time VotApedia took out of lectures. Others felt put upon (one student found it ‘Distracting and quite frankly, oppressive’), or that the material was being ‘dumbed down’, although often while simultaneously noting the positive aspects of VotApedia. A good example was the observation that: ‘Although the voting was ‘engaging’ and interesting, I felt that it took far too much time that would have been better spent simply giving students more information’. This sort of concern is indicative of some deeper student ambivalence about the purpose of lectures, which is becoming more widespread with increased use of technology in general. More particularly, this might be the reverse of a previous finding: that students who actively dislike SRSs are those who traditionally succeed in a didactic lecturing environment.

Indeed, this comment about inappropriate use of time was quite common. A total of 27 students (19% of survey respondents) commented that VotApedia took too much time and recommended fewer instances of it, or lamented the reduction of content or depth in the lectures. Nonetheless, our analysis demonstrates clearly that the vast majority of students (often including those who were dissatisfied with the use of time) found our use of mobile phone voting to be very appealing. Moreover, as we have already observed, part of the time taken was in asking questions necessary for our research (including whether the pre-reading had been done and whether it was useful). We are confident that some of the concerns about the time involved in obtaining and discussing the audience response would be addressed in a 2-hour lecture involving only three substantive questions.

Discussion and conclusion

Substantial prior research has demonstrated that SRSs, delivered via constructivist pedagogy, can substantially improve student engagement in learning activities. This research goes beyond the established claims for this impact in hard science (Judson and Sawada 2002; Kay and LeSage 2009; Simpson and Oliver 2007), business (Keough 2012; Sprague and Dahl 2010) and health science (Patterson, Kilpatrick and Woebkenberg 2010) disciplines to demonstrate the effects of SRSs and sound pedagogy on more humanities-style disciplines, such as law. Educators in ‘softer’ disciplines may doubt the usefulness of SRSs and indeed may be wary about using formative multiple-choice questions in areas that are more interpretivist and inclined towards discussing divergent questions rather than questions which have a simple, single correct answer. This research suggests that these areas could make good use of SRSs, especially if combined with the constructivist pedagogies that they may already be using.

Our trial also confirmed the value of peer-learning techniques in this context (Crouch and Mazur 2001; Habel 2009; Simpson and Oliver 2007). We also found close links between the use of SRSs and not only engagement but also attendance and preparation, relationships which have not been fully explored in the literature (Kay and LeSage 2009). However, although we demonstrated that engagement can have positive impacts on student preparation, we also conclude that the impact of engagement on preparation is not sufficient to overcome external factors such as a clash with a major assessment in another course, and wider contextual factors from the student perspective need to be considered when developing a pedagogy that makes use of an SRS.

Our research also confirmed that the use of SRSs is not universally endorsed by students, some of whom felt that the time could have been better spent delivering a larger amount of content – a sentiment which is not new (Van Dijk, Van den Berg and Van Keulen 2001). However, our findings explored an important issue regarding the types of students who are likely to prefer SRSs, along the lines of Sprague and Dahl (2010). The students who particularly reported the benefits of the SRS were those who had difficulty learning in the traditional lecture format: those who struggled to remain engaged or devote attention to the material being delivered. On the contrary, students who thought the SRS a waste of time were precisely those who wanted more coverage of content and would probably fare well no matter the pedagogy or delivery method. Of course, these findings are only preliminary, but suggest that further research into the learning styles and needs of those who engage in learning through an SRS is necessary. At the very least, these findings recommend a continued focus on the diversity of student needs and approaches to learning.

While research around the use of SRSs in pedagogically sound ways is continuing to develop in line with educational practice, this is only the beginning. More explicit exploration of the relevance of the pedagogy and technology for divergent-type questions and interpretivist disciplines is necessary, and a deeper understanding of the relationships between the various strategies and outcomes of the approach is also needed. In particular, we need a better understanding of how SRSs can help particular students – be they international, ‘non-traditional’, preparatory, or under-engaged – so that our combined use of technology and pedagogy can meet specific student needs.

Acknowledgements

The authors acknowledge the support of the Faculty of the Professions, University of Adelaide, through the Teaching Large and Diverse Classes Project, and thank the anonymous reviewers for their helpful comments and suggestions.

Notes

1. Clearly this is an action research design rather than an experimental one, but we have borrowed from the language of experimental research to describe the rigour built into the project.

2. In the first week, students were given an individual VotApedia question to compare it with their group response, but this was quickly discarded due to redundancy and time constraints. This meant that the peer-learning element of the research design was de-emphasised in the findings.

3. The research complied with institutional requirements for ethics clearance and was exempt from Human Research Ethics Committee review as it was ‘negligible risk’ research involving only non-identifiable data (in accordance with Australia’s National Statement on Ethical Conduct in Human Research (2007) [5.1.22]). At every stage at which student feedback was sought or obtained, the process was entirely voluntary – students who did not wish to vote in the lecture were told that they were welcome to choose not to vote, and students were instructed that they did not have to participate in the online survey unless they wished to do so. Further, at each stage student feedback was entirely anonymous – the VotApedia system itself ensured that in-lecture voting was completely confidential and online survey responses were collected anonymously.

4. The lesser significance of the final result may reflect survey fatigue on the part of the participants, or some regression to the mean, or some other unobserved factor, but our dataset does not permit further exploration of this issue.

5. ‘Broad agreement’ is a standard measure of analysis in student evaluation surveys at the University of Adelaide and incorporates responses which either strongly or somewhat agree. Since this unit of analysis is an institutional standard and simplifies the analysis of Likert-style responses, we have used it in this paper.

6. It is possible that there may be additional unobserved factors that contributed to the differences in student perceptions that we report, for which we have not controlled. However, our intervention was the only intentional change to course methodology, and our observations do not indicate to us the influence of any other factors.

References

Brooman, S. (2011) ‘Enhancing student engagement by building upon the “techtonic plates” of legal education’, Liverpool Law Review, vol. 32, no. 2, pp. 109–112. Publisher Full Text

Caldwell, J. E. (2007) ‘Clickers in the large classroom: current research and best-practice tips’, Life Sciences Education, vol. 6, no. 1, pp. 9–20. PubMed Abstract | PubMed Central Full Text | Publisher Full Text

Coates, H. (2005) ‘The value of student engagement for higher education quality assurance’, Quality in Higher Education, vol. 11, no. 1, pp. 25–36. Publisher Full Text

Coates, H. (2008) ‘Beyond happiness: managing engagement to enhance satisfaction and grades’, [online] Available at: http://research.acer.edu.au/ausse/11/

Corbin, L., Burns, K. & Chrzanowski, A. (2010) ‘If you teach it, will they come? Law students, class attendance and student engagement’, Legal Education Review, vol. 20, no. 1, pp. 13–44.

Crisp, G. (2007) The E-assessment Handbook, Continuum, New York.

Crouch, C. H. & Mazur, E. (2001) ‘Peer instruction: ten years of experience and results’, American Journal of Physics, vol. 69, no. 9, pp. 970–977. Publisher Full Text

DEEWR. (2009) Transforming Australia’s Higher Education System, [online] Available at: http://www.deewr.gov.au/HigherEducation/Pages/TransformingAustraliasHESystem.aspx

D’Inverno, R., Davis, H. & White, S. (2003) ‘Using a personal response system for promoting student interaction’, Teaching Mathematics and Its Applications, vol. 22, no. 4, pp. 163–169. Publisher Full Text

Fies, C. & Marshall, J. (2006) ‘Classroom response systems: a review of the literature’, Journal of Science Education and Technology, vol. 15, no. 1, pp. 101–109. Publisher Full Text

Habel, C. (2009) ‘VotApedia for student engagement in academic integrity education’, ergo, vol. 2, no. 1, pp. 15–25.

Judson, E. & Sawada, D. (2002) ‘Learning from past and present: electronic response systems in college lecture halls’, Journal of Computers in Mathematics and Sciences Teaching, vol. 21, no. 2, pp. 167–181.

Kay, R. & LeSage, A. (2009) ‘A strategic assessment of student response systems used in higher education’, Australasian Journal of Educational Technology, vol. 25, no. 2, pp. 235–249.

Keough, S. M. (2012) ‘Clickers in the classroom: a review and a replication’, Journal of Management Education, vol. 36, no. 6, pp. 822–847. Publisher Full Text

Lasry, N., Mazur, E. & Watkins, J. (2008) ‘Peer instruction: from Harvard to the two-year college’, American Journal of Physics, vol. 76, no. 11, pp. 1066–1069. Publisher Full Text

Massingham, P. & Herrington, T. (2006) ‘Does attendance matter? An examination of student attitudes, participation, performance and attendance’, Journal of University Teaching and Learning Practice, vol. 3, no. 2, pp. 82–103.

Michaelsen, L. K. & Sweet, M. (2008) ‘The essential elements of team-based learning’, New Directions for Teaching and Learning, vol. 2008, no. 116, pp. 7–27. Publisher Full Text

Morling, B., et al. (2008) ‘Efficacy of personal response systems (‘clickers’) in large, introductory psychology classes’, Teaching of Psychology, vol. 35, no. 1, pp. 45–50. Publisher Full Text

Patterson, B., Kilpatrick, J. & Woebkenberg, E. (2010) ‘Evidence for teaching practice: the impact of clickers in a large classroom environment’, Nurse Education Today, vol. 30, no. 7, pp. 603–607. PubMed Abstract | Publisher Full Text

Rhodes, F. H. T. (2001) The Creation of the Future: The Role of the American University, Cornell University Press, Ithaca, NY.

Simpson, V. & Oliver, M. (2007) ‘Electronic voting systems for lectures – research and practice’, Australasian Journal of Educational Technology, vol. 23, no. 7, pp. 187–208.

Sprague, E. W. & Dahl, D. W. (2010) ‘Learning to click’, Journal of Marketing Education, vol. 32, no. 1, pp. 93–103. Publisher Full Text

Stevens, E., et al. (2006) ‘Equity, diversity and student engagement in a law school – a case study approach’, Legal Education Review, vol. 16, no. 1, pp. 1–33.

Terrion, J. L. & Aceti, V. (2012) ‘Perceptions of the effects of clicker technology on student learning and engagement: a study of freshmen Chemistry students’, Research in Learning Technology, vol. 20, no. 2. pp. 1–11.

Trees, A. R. & Jackson, M. (2007) ‘The learning environment in clicker classrooms: student processes of learning and involvement in large university-level courses using student response systems’, Learning, Media and Technology, vol. 32, no. 1, pp. 21–40. Publisher Full Text

Van Dijk, L. A., Van den Berg, G. C. & Van Keulen, H. (2001) ‘Interactive lectures in engineering education’, European Journal of Engineering Education, vol. 26, no. 1, pp. 15–28. Publisher Full Text

Appendix: Methodology of intervention