Playing the game: a realist approach to evaluating online student access, retention, progression and attainment initiatives

Keywords: higher education, gamification, retention, student success

Abstract

This study presents an evaluation of an online game-based student access, retention, progression and attainment (ARPA) initiative at the University of Kent. The initiative, a narrative-based simulation of a condensed student journey from pre-enrolment to graduation, is designed to prepare and support students in their transition to and participation in Higher Education. Student retention continues to be a perennial issue across the Higher Education sector, and studies have indicated that the more knowledgeable and informed students are about their university environment, the less likely they are to leave before completing their studies. Many institutions have developed interventions with the express purpose of addressing these concerns. Recognising the contextualised and subjective nature of such interventions, a realist evaluative framework was adopted to better understand the initiative under scrutiny, asking what works, for whom and in what circumstances. Participant interviews were utilised to assess the efficacy of the initiative in supporting students and in helping them to navigate often unfamiliar institutional cultures, practices and expectations. A revised programme theory is presented, enabling deeper insight into the merit of the initiative and its overall worth as a mechanism for change within the ARPA paradigm.

Downloads

Download data is not yet available.

References


Advance HE. (2019) Essential Frameworks for Enhancing Student Success. York: Advance HE. Available at: https://www.advance-he.ac.uk/sites/default/files/2020-05/Enhancing%20Student%20Success%20in%20Higher%20Education%20Framework.pdf


Astbury, B. & Leeuw, F. L. (2010) ‘Unpacking black boxes: mechanisms and theory building in evaluation’, American Journal of Evaluation, vol. 31, no. 3, pp. 363–381. doi: 10.1177/1098214010371972


Billing, D. (1997) ‘Induction of new students to higher education’, Innovations in Education and Training International, vol. 34, no. 2, pp. 125–134. doi: 10.1080/1355800970340208


Bourdieu, P. (1977) Reproduction in Education, Society and Culture, SAGE Publications, London.


Brooman, S. & Darwent, S. (2014) ‘Measuring the beginning: a quantitative study of the transition to higher education’, Studies in Higher Education, vol. 39, no. 9, pp. 1523–1541. doi: 10.1080/03075079.2013.801428


Burke, P. J. (2012) The Right to Higher Education – Beyond Widening Participation, Routledge, London.


Buzzo, D., & Phelps, P. (2016) ‘JourneyMap: Visualising the time-bound student journey’, in EVA London 2016: Electronic Visualisation and the Arts (British Computer Society), eds N. Lambert, J. Bowen & G. Diprose, London, 12–15 July, pp. 178–185.


Carenys, J. & Moya, S. (2016) ‘Digital game-based learning in accounting and business education’, Accounting Education, vol. 25, no. 6, pp. 598–651. doi: 10.1080/09639284.2016.1241951


Cook, T. & Rushton, B. (2009) How to Recruit and Retain Higher Education Students – A Handbook of Good Practice, Taylor & Francis, London.


Crafter, S. & Maunder, R. (2012) ‘Understanding transitions using a sociocultural framework’, Educational and Child Psychology, vol. 29, no. 1, pp. 10–18.


Crosling, G., Heagney, M. & Thomas, L. (2009) ‘Improving student retention in higher education: improving teaching and learning’, The Australian Universities’ Review, vol. 51, no. 2, pp. 9–18. doi: 10.4324/9780203935453


Crozier, G. & Reay, D. (2008) ‘The socio cultural and learning experiences of working class students in higher education’, Teaching and Learning Research Briefing, (44) June. Teaching and Learning Research Programme, London.


Currant, B. & Keenan, C. (2009) ‘Evaluating systematic transition to higher education’, The Brookes Ejournal of Learning and Teaching, vol. 2, no. 4, pp. 1–12.


Dalkin, S., et al., (2021) ‘Using computer assisted qualitative data analysis software (CAQDAS; NVivo) to assist in the complex process of realist theory generation, refinement and testing’, International Journal of Social Research Methodology, vol. 24, no. 1, pp. 123–134. doi: 10.1080/13645579.2020.1803528


Dalkin, S. M., et al., (2012) ‘Understanding integrated care pathways in palliative care using realist evaluation: a mixed methods study protocol’, BMJ Open, vol. 2, no. 4. doi: 10.1136/bmjopen-2012-001533


Dalkin, S. M., et al., (2015) ‘What’s in a mechanism? Development of a key concept in realist evaluation’, Implementation Science, vol. 10, no. 1, p. 49. doi: 10.1186/s13012-015-0237-x


Davies, P. & Williams, J. (2001) ‘For me or not for me? Fragility and risk in mature students’ decision-making’, Higher Education Quarterly, vol. 55, no. 2, pp. 185–203. doi: 10.1111/1468-2273.00182


Dytham, S. & Hughes, C. (2017) Widening Participation Research And Evaluation: Where Are We Now?, University of Warwick, Coventry, UK, p. 8.


Emmel, N. (2013) Sampling and Choosing Cases in Qualitative Research: A Realist Approach, SAGE, London.


Formby, A., Woodhouse, A. & Basham, J. (2020) ‘Reframing widening participation towards the community: a realist evaluation’, Widening Participation and Lifelong Learning, vol. 22, no. 2, pp. 184–204. doi: 10.5456/WPLL.22.2.184


Francis, J. J., et al., (2010) ‘What is an adequate sample size? Operationalising data saturation for theory-based interview studies’, Psychology & Health, vol. 25, no. 10, pp. 1229–1245. doi: 10.1080/08870440903194015


Funnell, S. C. & Rogers, P. J. (2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models, Wiley, New York.


Guest, G., Bunce, A. & Johnson, L. (2006) ‘How many interviews are enough? an experiment with data saturation and variability’, Field Methods, vol. 2006, no. 18, pp. 59–82. doi: 10.1177/1525822X05279903


Hamshire, C., Whitton, N. & Whitton, P. (2012) ‘Staying the course - A game to facilitate students’ transitions to higher education’. In 6th European Conference on Games Based Learning, Cork, ed P. Felicia, Ireland, 4–5 October, pp. 624–630.


Harries, E., Hodgson, L. & Noble, J. (2014) Creating Your Theory of Change, NPC, London.


HEPI. (2021) A Short Guide to Non-Continuation in UK Universities, Higher Education Policy Institute, Oxford.


HESA. (2021) Non-Continuation: UK Performance Indicators | HESA. Available at: https://www.hesa.ac.uk/data-and-analysis/performance-indicators/non-continuation


Kapp, K. M. (2013) The Gamification of Learning and Instruction Fieldbook, John Wiley & Sons, New Jersey.


Keenan, C. (2009) ‘Stepping Stones 2HE: fresh thinking for introducing PDP to freshers’, in Enhancing Student-Centred Learning in Business and Management Hospitality Leisure Sport Tourism, eds J. Buswell & N. Becket, Threshold Press, Newbury, pp. 169–178.


Knowlton, L. W. & Phillips, C. C. (2012) The Logic Model Guidebook: Better Strategies for Great Results, SAGE, London.


Knox, H. (2005) ‘Making the transition from further to higher education: the impact of a preparatory module on retention, progression and performance’, Journal of Further and Higher Education, vol. 29, no. 2, pp. 103–110. doi: 10.1080/03098770500103135


Krause, M. & Williams, J. (2015) ‘A Playful Game Changer: Fostering Student Retention in Online Education with Social Gamification’. In Second ACM Conference on Learning @ Scale, Association for Computing Machinery, ed G. Kiczales, New York, 14–18 March, pp. 95–102. https://doi.org/10.1145/2724660.2724665


Laing, C., Robinson, A. & Johnston, V. (2005) ‘Managing the transition into higher education: an on-line spiral induction programme’, Active Learning in Higher Education, vol. 6, no. 3, pp. 243–255. doi: 10.1177/1469787405059575


Lefever, R. & Currant, B. (2010) ‘How can technology be used to improve the learner experience at points of transition’, Higher Education Academy, York, pp. 1–90


Lowe, H. & Cook, A. (2003) ‘Mind the gap: are students prepared for higher education?’, Journal of Further and Higher Education, vol. 27, no. 1, pp. 53–76. doi: 10.1080/03098770305629


Manzano, A. (2016) ‘The craft of interviewing in realist evaluation’, Evaluation, vol. 22, no. 3, pp. 342–360. doi: 10.1177/1356389016638615


Martin, C. & Benton, T. (2017) ‘Character creation: gamification and identity’, Teaching Media Quarterly, vol. 5, no. 2, pp. 1–8.


Money, J., et al., (2016) ‘Co-creating a blended learning curriculum in transition to higher education: a student viewpoint’, Creative Education, vol. 7, pp. 1205–1213. doi: 10.4236/ce.2016.79126


Morgan, M., et al., (2002) Risk Communication: A Mental Models Approach, Cambridge University Press, New York.


Moseley, A. (2008) ‘An alternate reality for higher education? Lessons to be learned from online games’. In ALT-C, ed S. Schmoller, Leeds, 9–11 September, pp. 92–95


O’Donnell, P., Kean, M. & Stevens, G. (2016) Student Transition in Higher Education – Concepts, Theories and Practices, Higher Education Academy, York.


Office for Students. (2018) Providing Information, Advice and Guidance for Students – Our Research. Available at: https://www.officeforstudents.org.uk/advice-and-guidance/student-information-and-data/providing-information-advice-and-guidance-for-students/our-research/


Office for Students. (2020) Information, Advice and Guidance for Prospective Students. Available at: https://www.officeforstudents.org.uk/publications/coronavirus-briefing-note-information-advice-and-guidance-for-prospective-students/


Pawson, R. (2013) The Science of Evaluation – A Realist Manifesto, SAGE Publications, London.


Pawson, R. & Tilley, N. (1997) Realistic Evaluation, SAGE Publications, London.


Pennington, C. R., et al., (2018) ‘Transitioning in higher education: an exploration of psychological and contextual factors affecting student satisfaction’, Journal of Further and Higher Education, vol. 42, no. 5, pp. 596–607. doi: 10.1080/0309877X.2017.1302563


Piatt, K. (2009) ‘Using alternate reality games to support first year induction with ELGG’, Campus-Wide Information Systems, vol. 26, no. 4, pp. 313–322. doi: 10.1108/10650740910984646


Pickering, N. (2021) ‘Enabling equality of access in higher education for underrepresented groups: a realist “small step” approach to evaluating widening participation’, Research in Post-Compulsory Education, vol. 26, no. 1, pp. 111–130. doi: 10.1080/13596748.2021.1873410


Reay, D., David, M. E. & Ball, S. (2005) Degrees of Choice: Social Class, Race and Gender in Higher Education, Trentham Books Ltd, Stoke on Trent, UK.


Robbins, D. (1993) ‘The practical importance of Bourdieu’s analyses of Higher Education’, Studies in Higher Education, vol. 18, no. 2, pp. 151–163. doi: 10.1080/03075079312331382339


Roberts, C., et al., (2003) ‘Supporting student ‘success’: what can we learn from the persisters?’, in Education in a Changing Environment: Inaugural Learning and Teaching Research Conference, eds L. Anderson, 17–18 September, University of Salford, UK, pp. 45–53.


Ryan, F. (2020) ‘A virtual law clinic: a realist evaluation of what works for whom, why, how and in what circumstances?’, The Law Teacher, vol. 54, no. 2, pp. 237–248. doi: 10.1080/03069400.2019.1651550


Sailer, M., et al., (2017) ‘How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction’, Computers in Human Behavior, vol. 69, pp. 371–380. doi: 10.1016/j.chb.2016.12.033


Tchen, P., et al., (2018) ‘Bridging the gap: an evaluation of self-paced online transition modules for advanced pharmacy practice experience students’, Currents in Pharmacy Teaching and Learning, vol. 10, no. 10, pp. 1375–1383. doi: 10.1016/j.cptl.2018.07.006


Turner, R., et al., (2017) ‘Easing the transition of first year undergraduates through an immersive induction module’, Teaching in Higher Education, vol. 22, no. 7, pp. 805–821. doi: 10.1080/13562517.2017.1301906


University of Kent. (2019a) ‘One Hour Degree proposal’. Internal University of Kent paper. Unpublished.


University of Kent. (2019b) Student Success Project – The One Hour Degree. Available at: https://www.kent.ac.uk/studentsuccess/onehourdegree.html


Webb, O. J. & Cotton, D. R. E. (2018) ‘Early withdrawal from higher education: a focus on academic experiences’, Teaching in Higher Education, vol. 23, no. 7, pp. 835–852. doi: 10.1080/13562517.2018.1437130


Weiss, C. (1997) ‘Theory-based evaluation: past, present, and future’, New Directions for Evaluation, vol. 1997, no. 76, pp. 41–55. doi: 10.1002/ev.1086


Whitton, N., et al., (2014) ‘Alternate reality games as learning environments for student induction’, Interactive Learning Environments, vol. 22, no. 3, pp. 243–252. doi: 10.1080/10494820.2011.641683


Yorke, M. & Longden, B. (2008) The First-Year Experience of Higher Education in the UK – Final Report, Higher Education Academy, York.


Zahedi, L., et al., (2021) ‘Gamification in education: a mixed-methods study of gender on computer science students’ academic performance and identity development’, Journal of Computing in Higher Educuation, 33, pp. 441–474. doi: 10.1007/s12528-021-09271-5
Published
2022-06-27
How to Cite
Clark D. (2022). Playing the game: a realist approach to evaluating online student access, retention, progression and attainment initiatives. Research in Learning Technology, 30. https://doi.org/10.25304/rlt.v30.2782
Section
Original Research Articles