Playing the game: a realist approach to evaluating online student access, retention, progression and attainment initiatives

Keywords: higher education, gamification, retention, student success


This study presents an evaluation of an online game-based student access, retention, progression and attainment (ARPA) initiative at the University of Kent. The initiative, a narrative-based simulation of a condensed student journey from pre-enrolment to graduation, is designed to prepare and support students in their transition to and participation in Higher Education. Student retention continues to be a perennial issue across the Higher Education sector, and studies have indicated that the more knowledgeable and informed students are about their university environment, the less likely they are to leave before completing their studies. Many institutions have developed interventions with the express purpose of addressing these concerns. Recognising the contextualised and subjective nature of such interventions, a realist evaluative framework was adopted to better understand the initiative under scrutiny, asking what works, for whom and in what circumstances. Participant interviews were utilised to assess the efficacy of the initiative in supporting students and in helping them to navigate often unfamiliar institutional cultures, practices and expectations. A revised programme theory is presented, enabling deeper insight into the merit of the initiative and its overall worth as a mechanism for change within the ARPA paradigm.


Download data is not yet available.


Advance HE. (2019) Essential Frameworks for Enhancing Student Success. York: Advance HE. Available at:

Astbury, B. & Leeuw, F. L. (2010) ‘Unpacking black boxes: mechanisms and theory building in evaluation’, American Journal of Evaluation, vol. 31, no. 3, pp. 363–381. doi: 10.1177/1098214010371972

Billing, D. (1997) ‘Induction of new students to higher education’, Innovations in Education and Training International, vol. 34, no. 2, pp. 125–134. doi: 10.1080/1355800970340208

Bourdieu, P. (1977) Reproduction in Education, Society and Culture, SAGE Publications, London.

Brooman, S. & Darwent, S. (2014) ‘Measuring the beginning: a quantitative study of the transition to higher education’, Studies in Higher Education, vol. 39, no. 9, pp. 1523–1541. doi: 10.1080/03075079.2013.801428

Burke, P. J. (2012) The Right to Higher Education – Beyond Widening Participation, Routledge, London.

Buzzo, D., & Phelps, P. (2016) ‘JourneyMap: Visualising the time-bound student journey’, in EVA London 2016: Electronic Visualisation and the Arts (British Computer Society), eds N. Lambert, J. Bowen & G. Diprose, London, 12–15 July, pp. 178–185.

Carenys, J. & Moya, S. (2016) ‘Digital game-based learning in accounting and business education’, Accounting Education, vol. 25, no. 6, pp. 598–651. doi: 10.1080/09639284.2016.1241951

Cook, T. & Rushton, B. (2009) How to Recruit and Retain Higher Education Students – A Handbook of Good Practice, Taylor & Francis, London.

Crafter, S. & Maunder, R. (2012) ‘Understanding transitions using a sociocultural framework’, Educational and Child Psychology, vol. 29, no. 1, pp. 10–18.

Crosling, G., Heagney, M. & Thomas, L. (2009) ‘Improving student retention in higher education: improving teaching and learning’, The Australian Universities’ Review, vol. 51, no. 2, pp. 9–18. doi: 10.4324/9780203935453

Crozier, G. & Reay, D. (2008) ‘The socio cultural and learning experiences of working class students in higher education’, Teaching and Learning Research Briefing, (44) June. Teaching and Learning Research Programme, London.

Currant, B. & Keenan, C. (2009) ‘Evaluating systematic transition to higher education’, The Brookes Ejournal of Learning and Teaching, vol. 2, no. 4, pp. 1–12.

Dalkin, S., et al., (2021) ‘Using computer assisted qualitative data analysis software (CAQDAS; NVivo) to assist in the complex process of realist theory generation, refinement and testing’, International Journal of Social Research Methodology, vol. 24, no. 1, pp. 123–134. doi: 10.1080/13645579.2020.1803528

Dalkin, S. M., et al., (2012) ‘Understanding integrated care pathways in palliative care using realist evaluation: a mixed methods study protocol’, BMJ Open, vol. 2, no. 4. doi: 10.1136/bmjopen-2012-001533

Dalkin, S. M., et al., (2015) ‘What’s in a mechanism? Development of a key concept in realist evaluation’, Implementation Science, vol. 10, no. 1, p. 49. doi: 10.1186/s13012-015-0237-x

Davies, P. & Williams, J. (2001) ‘For me or not for me? Fragility and risk in mature students’ decision-making’, Higher Education Quarterly, vol. 55, no. 2, pp. 185–203. doi: 10.1111/1468-2273.00182

Dytham, S. & Hughes, C. (2017) Widening Participation Research And Evaluation: Where Are We Now?, University of Warwick, Coventry, UK, p. 8.

Emmel, N. (2013) Sampling and Choosing Cases in Qualitative Research: A Realist Approach, SAGE, London.

Formby, A., Woodhouse, A. & Basham, J. (2020) ‘Reframing widening participation towards the community: a realist evaluation’, Widening Participation and Lifelong Learning, vol. 22, no. 2, pp. 184–204. doi: 10.5456/WPLL.22.2.184

Francis, J. J., et al., (2010) ‘What is an adequate sample size? Operationalising data saturation for theory-based interview studies’, Psychology & Health, vol. 25, no. 10, pp. 1229–1245. doi: 10.1080/08870440903194015

Funnell, S. C. & Rogers, P. J. (2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models, Wiley, New York.

Guest, G., Bunce, A. & Johnson, L. (2006) ‘How many interviews are enough? an experiment with data saturation and variability’, Field Methods, vol. 2006, no. 18, pp. 59–82. doi: 10.1177/1525822X05279903

Hamshire, C., Whitton, N. & Whitton, P. (2012) ‘Staying the course - A game to facilitate students’ transitions to higher education’. In 6th European Conference on Games Based Learning, Cork, ed P. Felicia, Ireland, 4–5 October, pp. 624–630.

Harries, E., Hodgson, L. & Noble, J. (2014) Creating Your Theory of Change, NPC, London.

HEPI. (2021) A Short Guide to Non-Continuation in UK Universities, Higher Education Policy Institute, Oxford.

HESA. (2021) Non-Continuation: UK Performance Indicators | HESA. Available at:

Kapp, K. M. (2013) The Gamification of Learning and Instruction Fieldbook, John Wiley & Sons, New Jersey.

Keenan, C. (2009) ‘Stepping Stones 2HE: fresh thinking for introducing PDP to freshers’, in Enhancing Student-Centred Learning in Business and Management Hospitality Leisure Sport Tourism, eds J. Buswell & N. Becket, Threshold Press, Newbury, pp. 169–178.

Knowlton, L. W. & Phillips, C. C. (2012) The Logic Model Guidebook: Better Strategies for Great Results, SAGE, London.

Knox, H. (2005) ‘Making the transition from further to higher education: the impact of a preparatory module on retention, progression and performance’, Journal of Further and Higher Education, vol. 29, no. 2, pp. 103–110. doi: 10.1080/03098770500103135

Krause, M. & Williams, J. (2015) ‘A Playful Game Changer: Fostering Student Retention in Online Education with Social Gamification’. In Second ACM Conference on Learning @ Scale, Association for Computing Machinery, ed G. Kiczales, New York, 14–18 March, pp. 95–102.

Laing, C., Robinson, A. & Johnston, V. (2005) ‘Managing the transition into higher education: an on-line spiral induction programme’, Active Learning in Higher Education, vol. 6, no. 3, pp. 243–255. doi: 10.1177/1469787405059575

Lefever, R. & Currant, B. (2010) ‘How can technology be used to improve the learner experience at points of transition’, Higher Education Academy, York, pp. 1–90

Lowe, H. & Cook, A. (2003) ‘Mind the gap: are students prepared for higher education?’, Journal of Further and Higher Education, vol. 27, no. 1, pp. 53–76. doi: 10.1080/03098770305629

Manzano, A. (2016) ‘The craft of interviewing in realist evaluation’, Evaluation, vol. 22, no. 3, pp. 342–360. doi: 10.1177/1356389016638615

Martin, C. & Benton, T. (2017) ‘Character creation: gamification and identity’, Teaching Media Quarterly, vol. 5, no. 2, pp. 1–8.

Money, J., et al., (2016) ‘Co-creating a blended learning curriculum in transition to higher education: a student viewpoint’, Creative Education, vol. 7, pp. 1205–1213. doi: 10.4236/ce.2016.79126

Morgan, M., et al., (2002) Risk Communication: A Mental Models Approach, Cambridge University Press, New York.

Moseley, A. (2008) ‘An alternate reality for higher education? Lessons to be learned from online games’. In ALT-C, ed S. Schmoller, Leeds, 9–11 September, pp. 92–95

O’Donnell, P., Kean, M. & Stevens, G. (2016) Student Transition in Higher Education – Concepts, Theories and Practices, Higher Education Academy, York.

Office for Students. (2018) Providing Information, Advice and Guidance for Students – Our Research. Available at:

Office for Students. (2020) Information, Advice and Guidance for Prospective Students. Available at:

Pawson, R. (2013) The Science of Evaluation – A Realist Manifesto, SAGE Publications, London.

Pawson, R. & Tilley, N. (1997) Realistic Evaluation, SAGE Publications, London.

Pennington, C. R., et al., (2018) ‘Transitioning in higher education: an exploration of psychological and contextual factors affecting student satisfaction’, Journal of Further and Higher Education, vol. 42, no. 5, pp. 596–607. doi: 10.1080/0309877X.2017.1302563

Piatt, K. (2009) ‘Using alternate reality games to support first year induction with ELGG’, Campus-Wide Information Systems, vol. 26, no. 4, pp. 313–322. doi: 10.1108/10650740910984646

Pickering, N. (2021) ‘Enabling equality of access in higher education for underrepresented groups: a realist “small step” approach to evaluating widening participation’, Research in Post-Compulsory Education, vol. 26, no. 1, pp. 111–130. doi: 10.1080/13596748.2021.1873410

Reay, D., David, M. E. & Ball, S. (2005) Degrees of Choice: Social Class, Race and Gender in Higher Education, Trentham Books Ltd, Stoke on Trent, UK.

Robbins, D. (1993) ‘The practical importance of Bourdieu’s analyses of Higher Education’, Studies in Higher Education, vol. 18, no. 2, pp. 151–163. doi: 10.1080/03075079312331382339

Roberts, C., et al., (2003) ‘Supporting student ‘success’: what can we learn from the persisters?’, in Education in a Changing Environment: Inaugural Learning and Teaching Research Conference, eds L. Anderson, 17–18 September, University of Salford, UK, pp. 45–53.

Ryan, F. (2020) ‘A virtual law clinic: a realist evaluation of what works for whom, why, how and in what circumstances?’, The Law Teacher, vol. 54, no. 2, pp. 237–248. doi: 10.1080/03069400.2019.1651550

Sailer, M., et al., (2017) ‘How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction’, Computers in Human Behavior, vol. 69, pp. 371–380. doi: 10.1016/j.chb.2016.12.033

Tchen, P., et al., (2018) ‘Bridging the gap: an evaluation of self-paced online transition modules for advanced pharmacy practice experience students’, Currents in Pharmacy Teaching and Learning, vol. 10, no. 10, pp. 1375–1383. doi: 10.1016/j.cptl.2018.07.006

Turner, R., et al., (2017) ‘Easing the transition of first year undergraduates through an immersive induction module’, Teaching in Higher Education, vol. 22, no. 7, pp. 805–821. doi: 10.1080/13562517.2017.1301906

University of Kent. (2019a) ‘One Hour Degree proposal’. Internal University of Kent paper. Unpublished.

University of Kent. (2019b) Student Success Project – The One Hour Degree. Available at:

Webb, O. J. & Cotton, D. R. E. (2018) ‘Early withdrawal from higher education: a focus on academic experiences’, Teaching in Higher Education, vol. 23, no. 7, pp. 835–852. doi: 10.1080/13562517.2018.1437130

Weiss, C. (1997) ‘Theory-based evaluation: past, present, and future’, New Directions for Evaluation, vol. 1997, no. 76, pp. 41–55. doi: 10.1002/ev.1086

Whitton, N., et al., (2014) ‘Alternate reality games as learning environments for student induction’, Interactive Learning Environments, vol. 22, no. 3, pp. 243–252. doi: 10.1080/10494820.2011.641683

Yorke, M. & Longden, B. (2008) The First-Year Experience of Higher Education in the UK – Final Report, Higher Education Academy, York.

Zahedi, L., et al., (2021) ‘Gamification in education: a mixed-methods study of gender on computer science students’ academic performance and identity development’, Journal of Computing in Higher Educuation, 33, pp. 441–474. doi: 10.1007/s12528-021-09271-5
How to Cite
ClarkD. (2022). Playing the game: a realist approach to evaluating online student access, retention, progression and attainment initiatives. Research in Learning Technology, 30.
Original Research Articles