Figure 1. Learning analytics data revealing total page views from 15/12/2015 to 23/06/2016.
ORIGINAL RESEARCH ARTICLE
James Alexandera,b*, Massimo Barcellonaa, Sarah McLachlana and Catherine Sackleya
aDepartment of Population Health Sciences, School of Population Health & Environmental Sciences, King’s College London, London, UK
bDepartment of Rehabilitation Sciences, Kingston & St George’s University of London, London, UK.
(Received 17 April 2018; final version received 26 November 2018; Published 06 February 2019)
Technology-enhanced learning (TEL) can engage students with learning and offer benefits in knowledge acquisition due to the flexibility of learning it provides. There is difficulty ascertaining best practice for the implementation of TEL in physiotherapy education. This study aims to address this through a case study with pre-registration students using a virtual learning environment (VLE) to supplement their learning. Seventy-nine students were enrolled onto a Movement & Exercise module and had access to the VLE resources. Data were captured by online survey, student focus groups, learning analytics data and comparison of examination results with a previous cohort who did not have access to all the resources. Survey data demonstrated that most students were satisfied with the resources and that they facilitated knowledge acquisition. Thematic analysis from the focus groups resulted in five higher order themes: (1) Content quality, (2) Interaction and accessibility, (3) Learning goal alignment, (4) Satisfaction with resources and (5) Suggestions for the future. Learning analytics data revealed students accessed the resources predominantly before examination periods. There were statistically significant improvements in mean examination marks compared to the previous cohort. In conclusion, satisfaction with the TEL resources was high, and there may be some positive effect on knowledge acquisition.
Keywords: technology-enhanced learning; curriculum development; education; higher education; online learning
*Corresponding author. Email: James.alexander@sgul.kingston.ac.uk
Research in Learning Technology 2019. © 2019 J. Alexander et al. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2019, 27: 2073 - http://dx.doi.org/10.25304/rlt.v27.2073
Technology-enhanced learning (TEL) has been described as using computer technologies to support learning locally or remotely (Gordon 2014). TEL is often used synonymously with e-learning but the variety of terms used to describe TEL has led to confusion and inconsistent findings over its benefits in higher education (HE) (Guri-Rosenblit and Gros 2011). For this article, the following definition is used, which incorporates the range of educational technologies that are common practice in HE today:
Any online facility or system that directly supports learning and teaching. This may include a formal VLE, e-assessment or e-portfolio software, or lecture capture system, mobile app or collaborative tool that supports student learning. This includes any system that has been developed in-house, as well as commercial or open source tools. (Universities and Colleges Information Systems Association [UCISA] 2016)
The implementation and use of TEL in HE has grown, driven by policy from UK Government (DfES 2004), student demand for HE and the growth in technology itself, which has been used as a catalyst for change in educational delivery (Laurillard 2008). This has led to greater use of virtual learning environments (VLEs): computer-based environments that enable teachers to upload learning material, customise the content and how it’s delivered, and allow students to interact with each other and access a wide variety of resources offering flexibility and autonomy for learning (Flavin 2016; Piccoli, Ahmad, and Ives 2001; Wilson 1996). VLEs should be designed to enable students of all learning streams to receive quality education and also reinforce their work in the classroom to develop deeper insight of the subject (Demain 2012). This overcomes the limitation of traditional face-to-face teaching and ensures that learning is neither confined to geographical location nor time.
TEL’s impact on learning through offering new forms of engagement, ownership of learning and knowledge acquisition is poorly understood (Laurillard 2008). Students seek efficiency and value ease of use and convenience, and therefore, online repositories of easily accessible resources are of great value for students (Flavin 2016; Lawrence 2015). Current drivers in HE for using TEL include meeting student expectations and improving satisfaction (UCISA 2016). UCISA identifies that blended learning delivery, through online and face-to-face methods (Margulieux, McCracken, and Catrambone 2015), using supplementary resources is the most common use of TEL in HE, but evaluation of the impact of TEL tools and systems remains poor (UCISA 2016).
Recent systematic reviews in physiotherapy education have demonstrated a minimally positive to neutral effect on knowledge acquisition, critical thinking and practical skills (Macznik, Ribeiro, and Baxter 2015; Veneri 2011). Systematic reviews considering blended learning in other health care professions have identified improved clinical competence (Rowe, Frantz, and Bozalek 2012) or no difference in clinical skills, knowledge and self-efficacy (McCutcheon et al. 2015). Both these reviews highlight a lack of studies and the need for further research. A meta-analysis into the effectiveness of blended learning studies concluded methods can either reduce time in class and maintain learning outcomes or maintain time in class and improve learning outcomes (Margulieux, McCraken, and Catrambone 2015). Inconsistencies with findings result from different study designs, outcome measures and technologies used, making it difficult to ascertain best practice for the implementation of TEL in physiotherapy education (Macznik, Ribeiro, and Baxter 2015).
In the United Kingdom, rising living costs and academic fees have led physiotherapy students to balance academic work, personal commitments and paid employment, alongside long commutes to university and clinical placements. TEL provides a flexible opportunity for students to access resources at their own convenience away from the classroom. If TEL is to be used in HE, the instructional design theory (IDT) (Gibbons and Rogers 2009) can be usefully applied to understand its merit. IDT argues that creating instructional experiences which make the acquisition of knowledge and skill more efficient, effective and appealing requires understanding of the state and needs of the learner (Gibbons and Rogers 2009). Identifying how TEL fits into students’ lives rather than expecting students to adapt to TEL will help to ensure that it meets the demands of HE in the UK today (Henderson, Selwyn, and Aston 2015; Stokoe, Benwell, and Attenborough 2013). This IDT and ethos encouraged the researchers to develop a study that was student-focused and informed by students themselves.
Considering the dearth of literature on optimal modes of TEL delivery within physiotherapy programmes and wanting to identify how TEL resources impact on our students, a series of student engagement groups were conducted by the lead researcher to capture feedback on how TEL could support them during their studies and help narrow the focus of our study. The feedback was mixed with students commenting that they did not feel fully engaged with TEL and wanted more interactive resources to facilitate their learning, especially whilst on clinical placements. The feedback directed the researchers to examine the effectiveness of TEL in a Movement & Exercise module, with specific focus on exercise medicine, a topic students use considerably on clinical placements, for first year Physiotherapy students at King’s College London (KCL). KCL is a world leading public research university with over 30 000 enrolled students: 24.4% of King's undergraduates are privately educated, and in the 2016/2017 academic year, the university had a domicile breakdown of 67:12:20 of UK:EU:non-EU students, respectively (HESA 2018a, 2018b, 2018c). It has a distinguished history for teaching and research in health and care and is a founding member of King's Health Partners, one of the largest academic health sciences centres in Europe. KCL uses its own VLE, the King’s E-Learning and Teaching Service (KEATS), which provides students access to course and module information, including learning and assessment resources. Students can log in remotely or by mobile app and access individual modules where resources are uploaded or created by module leads.
The research questions were as follows:
The study took place within the Academic Department of Physiotherapy at KCL. Ethical approval for this study was obtained from the Biomedical Sciences, Medicine, Dentistry and Natural & Mathematical Sciences (BDM) Research Ethics Panel at KCL (LRS-15/16-2727). All subject data were anonymous and students were informed that by either participating or not in the study would not jeopardise their position on the Physiotherapy programme.
All first year Physiotherapy, both BSc and MSc (Pre-registration), students (n = 79) were enrolled on the compulsory Movement & Exercise module. All students had access to the TEL resources and were invited to take part in an online survey and focus group. Participants were provided with an information sheet and gave written informed consent prior to participation in the focus groups. Implied consent was given through completion of the survey. The study was approved by the Biomedical Sciences, Medicine, Dentistry and Natural & Mathematical Sciences (BDM) Research Ethics Panel at KCL (LRS-15/16-2727).
A variety of TEL resources were created and made available for the students over the 2015/2016 academic year (Sept 2015–Aug 2016). These included interactive PowerPoints (PowerPoint 2016, Microsoft Corp, Redmond WA, USA), quizzes and screencasts, which required the student to complete case studies, click on hyperlinks, watch short video clips and answer questions related to the module. Other resources such as journal articles, lecture recordings, videos, past exam papers and condition-specific exercises in PDF format were also included as well as links to relevant pre-existing online resources. To follow a blended learning approach (Graham 2006), the resources were designed and created to match the learning outcomes for the individual sessions within the context of the module learning outcomes, and therefore, students would be able to access resources prior to or following a taught session to consolidate their learning.
A triangulation approach incorporating mixed methodology was used to improve the validity of results (Denzin 1978; Patton 1990). An online survey was designed using the Learning Object Review Instrument (LORI), a validated appraisal tool used, to assess the validity and efficiency of evaluation of online learning resources (Akpinar 2008). The survey was adapted to capture specific information about the variety of learning resources and address the study’s aims. The survey was developed using SurveyMonkey™ by the lead researcher and used a combination of a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree), multiple-choice ranked answers and free-text to allow for expression of thought. The survey was piloted with students not involved in the study, and amendments to the wording of some questions to improve clarity were made. The survey was open from 27 May until 18 July 2016 after the module assessment. Students from the BSc and MSc programmes were invited to attend one of three focus groups. The aims were to explore themes relating to student satisfaction and perception of usefulness for knowledge acquisition. Topic guides were informed by findings from the survey data. The focus groups were moderated by co-researchers and the principal researcher was seated at the periphery to answer questions and observe participant behaviour. The three focus groups lasted 84, 61 and 97 min, respectively. Learning analytics data, the capturing and analysis of data about student learning and the context in which that learning takes place (Sclater, Peasgood, and Mullan 2016), were captured from the KEATS VLE using the Moodle Learning Analytics Tool (v2.8 developed by Dr A Konstantinidis, Centre for Technology Enhanced Learning and Dr C Grafton, School of Arts & Humanities, KCL, London, UK), to understand which resources were used most frequently and when they were accessed. These data can be used to improve learning and teaching (Sclater, Peasgood, and Mullen 2016). To evaluate the impact on knowledge acquisition, practical and written examination results from the 2015/2016 study group cohorts were compared with the same examination results of the previous 2014/2015 cohorts, who only had access to limited resources (module handbook, lecture slides, lecture capture and past exam papers).
IBM SPSS Statistics for Windows Version 22.0 (IBM Corp., Armonk, NY, USA) was used for statistical analysis of examination results, with descriptive statistics calculated for each item of the survey. Learning analytics data were also analysed using descriptive statistics. Thematic analysis was completed using NVivo Pro 11 software (QSR International, Melbourne, Australia). Focus groups were audio and video transcribed by the lead researcher, and copies of transcripts were sent to randomly selected participants to verify the accuracy of the transcription. Inductive analysis was used for initial coding followed by axial coding using the LORI framework (Leacock and Nesbit 2007; Nesbit and Leacock 2009) to group codes into higher level themes. All coding was completed by the lead researcher, and a subsection of the transcripts was coded separately by a second researcher for independent analysis and confirmation of themes. Discussions took place to resolve discrepancies in coding.
Forty-three (55.1%) students responded to the survey, with 42 students completing all questions. All respondents had accessed the TEL resources KEATS page; 97.6% agreed or strongly agreed that they were satisfied with the resources, whilst 76.7% agreed or strongly agreed that the resources were engaging and interesting; 97.6% of respondents agreed or strongly agreed that the resources helped them with their practical and written exams, and 100% agreed or strongly agreed that the resources were useful in aiding knowledge of exercise medicine (see Table 1 for survey responses).
Fourteen students attended the focus groups (female, n = 12). Thematic analysis resulted in five higher order themes: (1) Content quality, (2) Interaction and accessibility, (3) Learning goal alignment, (4) Satisfaction with resources and (5) Suggestions for the future. Quotes are referenced with a number from 1 to 3, corresponding to focus group, and numbers 1–6, corresponding to individual students. Within the higher order themes were several lower order themes, the most prominent of which are discussed below.
Within the theme of content quality, students discussed the importance of the TEL resources having an appropriate volume of content that was manageable for learning as well as being interesting:
…the lectures on here the ones like ‘health benefits of exercise’ and ‘ageing and exercise’ because we had so many lectures…they would just tell us everything but these were just good for just summarising what we actually needed to know and fully understand. (FG 3; Participant 3)
It was just nice to like you know have an interactive quiz…it was interesting, it was different to what we usually like have in terms of educational resources. (FG 3; Participant 1)
Students also highlighted the diversity of resources being positive in terms of engagement and supporting knowledge and understanding. The students all agreed that the variety of TEL resources meant that different learning preferences were supported, with videos and quizzes emerging as preferred resources to facilitate learning. This was identified clearly by a student with dyslexia:
I think erm just having the diversity of all different resources that we could access…I’m dyslexic and I’ve found videos much easier, I just can’t keep looking at lectures, I struggle completely…I kind of think it accommodates er everyone’s different learning styles as well… (FG 3; Participant 4)
It’s nice to have them all there and like so you can use a combination of all of them. (FG 1; Participant 2)
Finally, students agreed that having TEL resources provided by the department meant that they could ‘trust’ that these were appropriate and helpful for their learning as they had been ‘vetted’:
Coz YouTube is a weird and wonderful world sometimes and you don’t want to be looking at something that’s wrong. So that’s what I mean by vetted. It’s come from something through you guys and you’re sort of going ‘yeah, stamped approval, that’s quite a good video’. (FG 3; Participant 4)
I think P1 or someone was saying because (lecturer) put them on there you feel like actually they’re right so you can trust that and use it and whereas if I just picked a video off YouTube I’d be worried it would be just some random person being like yeah do this or something like that. (FG 1; Participant 3)
Some students stated that easy-to-access and navigate module resources increased ease of use and willingness to use the resources:
I found it was quite good they had lists for everything it’s perfect to follow enough if you kind of know what you’re looking for; if it’s supplementary resources that’s what it says, if it’s the timetable that’s what it says…. (FG 2; Participant 3)
Not all students found the resources easy to use and three students identified issues relating to compatibility when using different platforms such as Apple™ products, and difficulties with use on mobile phones in areas with limited internet connection:
Erm, some of them I don’t think worked. From what I remember, I remember having difficulty, but that could have been the compatibility on my laptop. (FG 2; Participant 2)
If you’re on your phone its quite a lot to have to like log in coz it wants you to like log in to your King’s [internal account] and log into your module or something and by the time you’ve logged in a few times it’s just and it’s so small on your phone. (FG 3; Participant 3)
If you had an iPad which was like, had 3G network then that’s fine. When you’re on most of the trains here there’s not Wi-Fi is there? Obviously, there’s a bit on the tubes but it varies. (FG 3; Participant 3)
Many students stated that the resources gave them greater autonomy over their learning because they could access them at home or during long commutes to university or placement:
I could log onto the computer there [on placement] and log onto KEATS there and then go into the, go into the resource and then use it and I could do that on my lunch break and do that after work and it was really, really helpful. (FG 2; Participant 1)
Yeah, it’s useful if you’re ill as well if you have to have a few days off so you can don’t have to kind of like you don’t have so much pressure to catch up or learn off peers or contact your lecturer to try and go through stuff you can just kind of go on lecture capture or look at home. (FG 2; Participant 4)
Learning goal alignment pertains to the resources resonating with learning objectives for the module (Leacock & Nesbit, 2007). Students were generally positive when discussing the usefulness of the TEL resources for building their knowledge and understanding, and preparing them for exams. The previous exam papers, strength training quiz and ageing screencast were identified as particularly helpful for the written exam, whilst the videos and YouTube links were commonly cited as helpful for the practical exam:
And also through the ‘ageing and exercise screencast’ that was an exam, that was a question in our exam, erm not that we knew that at the time. So I found that really beneficial…coz I’d basically had written out an answer… (FG 2; Participant 2)
So yeah especially for OSCEs we’d be like, watch the videos and know exactly what was going on and what was expected and see other people do it, it really, really helped. (FG 2; Participant 3)
For students who had been on clinical placement, the TEL resources were discussed positively in terms of aiding preparation and facilitating knowledge acquisition, predominantly for musculoskeletal (MSK) placements. Students could also see how the resources were beneficial for other placement specialities. Students yet to go on placement saw potential for the resources to support them:
And I also used it [the resources] before erm placement, coz I went to MSK and there was a lot of stuff on there, the external stuff on there about like strength, training strength vs endurance and all those types of things so that exercise prescription for my patients I used it for things like that as well, so I found it really good for that. (FG 2; Participant 3)
…but if you have like MSK outpatient erm placement or just rehab in general that you can always kind of refer back to it, to what’s on there which I think will be helpful, especially coz I was in elderly and the ageing and exercise was really beneficial last time. (FG 3; Participant 2)
Students from all focus groups highlighted that having all the resources in one place enabled greater efficiency of learning, reducing time spent searching for appropriate resources elsewhere:
But I do think the online thing is great for accessibility because it’s not as time consuming. (FG 3; Participant 4)
…I like the pictures that they use on the PowerPoints and you can’t find them on google and so you can go onto the lectures and find them and saves you like hunting around for them. (FG 1; Participant 2)
Students identified positive aspects of specific TEL resources contributing to their overall satisfaction. Some students identified the quizzes and videos as most preferred, whilst others discussed the positive aspects of having the variety of resources:
I think they combine together to help overall. It was quite nice to just be able to flick between them all just to help your understanding which obviously is going to help in the exams… (FG 1; Participant 3)
Negative feedback from all groups highlighted a lack of consistency with the quality of lecture captures and a lack of clarity about using the Frequently Asked Questions Forum:
I think some of the lecture captures weren’t particularly great but I don’t think that was to do with the module I think that’s to do with the actual lecture capture. (FG 2; Participant 2)
Yeah the FAQ coz I remember looking at it and it’s like ‘you can’t access this unless you post a question’…so that was a bit of a confusing thing so I never used it because I didn’t know how to use it. (FG 1; Participant 1)
Students gave suggestions for specific conditions they would like further resources on and highlighted the need for a clearer link between the supplementary resources, lectures and learning objectives for the module:
…so obviously you’ve got all of the lectures in the order that they were given in but then maybe if they were grouped in terms of like topics instead. (FG 2; Participant 2)
And then linking, er like the lecture objectives up to the relevant supplementary resource. I think that’s quite beneficial and important. (FG 1; Participant 3)
The Moodle Learning Analytics Tool captured data from 15 December 2015 until the end of the academic year. Data identified that the module was accessed 10 548 times by the students and used predominantly between 11 and 5 pm with a smaller peak at 8 pm. The most accessed resources were the strength training quiz, module news, updates forum and module handbook. There were noticeable peaks in usage in the weeks preceding written and practical exams (see Figure 1).
Figure 1. Learning analytics data revealing total page views from 15/12/2015 to 23/06/2016.
The examination results demonstrated a statistically significant improvement in the mean mark for the practical and written exams as well as overall module grade (Table 2). The overall mean module grade improved from 67.6% to 75.1%, the mean practical examination mark increased from 72.9% to 81.8% and the mean written examination mark increased from 67.62% to 70.8% (see Figure 2).
| Mean mark 2015 no resources (SD) | Mean mark 2016 with resources (SD) | Mean difference (95% CI) | P-value (95% CI) | |
| Module grade | 67.6 (7.3) | 75.1 (7.57) | −7.570 (−9.992 to −5.147) | 0.000* |
| OSCE exam (practical) | 72.9 (10.64) | 81.8 (9.50) | −8.883 (−12.155 to −5.611) | 0.000* |
| Written exam | 67.6 (9.67) | 70.8 (9.06) | −3.141 (−6.185 to −0.096) | 0.043* |
| SD, standard deviation; CI, confidence interval; OSCE, objective structured clinical examination.
*Signifies statistically significant difference. |
||||
Figure 2. Mean examination results (95% CI) for both cohorts.
The study evaluated student satisfaction and knowledge acquisition using TEL resources on two pre-registration physiotherapy programmes at a London University. Results showed that all students accessed the TEL resources and the majority were satisfied with the resources. Satisfaction with the resources related to their role in the preparation for practical and written exams, and facilitation of autonomy over learning.
The ability to access the resources anytime, anywhere gave the students the opportunity to structure learning around other commitments. Students’ expressions of satisfaction with the TEL resources reflect findings from research regarding students’ perceptions of the ideal environment for a student in today’s world and managing the demands of the university alongside other commitments (Douglas et al. 2015). The diversity of the resources to meet individual learning needs was seen favourably and enabled students to utilise resources that they felt most suitable for them. All students felt the resources combined well to facilitate their knowledge and understanding of exercise medicine. Students reported enjoying quizzes and videos the most, and these were the most accessed resources. However, this contrasted with students’ perception that the most beneficial resources were exam papers and lecture slides. Learning analytics data showed that students accessed resources mostly prior to examinations, indicating that these served as revision tools to aid knowledge acquisition in the preparation for exams.
The TEL resources acted as a repository of information which students could easily access at any time and any place. Flavin (2016) has stated that the use of some VLEs as an online library has benefits to student learning. The focus group findings support this, as having a repository saved time for students when learning and revising for exams. Whilst the internet has a plethora of information for students, this overabundance often leaves students confused as to which resources are to be trusted (Khor et al. 2017). This links to the theme regarding resources being vetted, which enabled students to trust the information. Whilst other research (Buzzetto-More 2015) has encouraged the use of the internet and video sharing sites such as YouTube™, the content posted has not necessarily been assessed for quality, and thus, students may be unsure of its credibility.
Many comments in the focus groups related to the alignment of the resources with module learning objectives and facilitation of learning goal attainment through students passing the exams. This is consistent with previous research reporting that time, academic deadlines and personal priorities contribute to how students perceive TEL resources to be beneficial (Henderson, Selwyn, and Aston 2015; Woodhall, Hiller, and Resnick 2014). Findings suggest that accessibility of information to prepare for examination and assessment in a timely and efficient manner may be more important to students than the use of digital technologies to promote autonomy of learning (Entwistle and Ramsden 1983; Henderson, Selwyn, and Aston 2015; Virtanen and Lindblom-Ylanne 2010). It is important to recognise that examinations are an important aspect of HE study and may drive students’ use of the resources rather than the satisfaction with the resources themselves (Henderson, Selwyn, and Aston 2015).
There are several limitations to this study. A cohort design was used because it was ethically and logistically difficult to restrict access to the resources for some students within a cohort. Students in the previous cohort also had access to some resources such as lecture capture, lecture slides and past exam papers, as provision of these pre-dated the development of the TEL resources. The majority of participants in the focus groups were female. Whilst this may lead to the risk of a greater representation of female views, evidence is inconclusive regarding gender differences in attitudes towards TEL (Mayer-Smith, Pedretti, and Woodrow 2000; Ong and Lai 2004; Vale and Leder 2004; Yukselturk and Bulut 2009). Other limitations are confounding variables that may underlie the differences in examination results. These include analysing and comparing two different cohorts without accounting for the academic ability of each and different examination questions and weighting of examination components. The significant differences in examination results should therefore be interpreted with caution.
Considering the variety of educational settings, a standard pro forma for implementing TEL in health care education may be ineffective. Further cohort and case studies are needed with clear and focussed aims and methodology to understand the different ways TEL can be implemented in individual academic institutions, based on student wishes and needs. The learning analytics data provided insight into the type and frequency of resources used; however, future research should aim to capture how students utilise and access the online technologies within the module so that further use and development of TEL resources can be tailored to the individual student and module. The need also remains to evaluate the cost effectiveness of TEL in health care education.
Satisfaction with the TEL resources was high, and there may be some positive effect on knowledge acquisition but caution should be taken when interpreting these findings due to confounding variables. Nevertheless, findings indicate that with additional TEL resources, students could synthesise the information provided to support their knowledge acquisition of exercise medicine and achieve excellent grades. Students reported benefiting from the variety of TEL resources, which met individual learning preferences and facilitated autonomy of learning anytime, anywhere. The resources enabled students to meet learning objectives by guiding them to pertinent information to support knowledge acquisition and may be useful in other areas of health care education.
This study was supported by Professor C. Sackley’s NIHR Senior Investigator award. The authors would firstly like to thank the students who participated in this study and contributed to making it a success. They would also like to acknowledge and thank Dr Lindsay Bearne for running one of the focus groups, Dr Michael Flavin for his methodological advice and Mr Nabil Asif for his technological support.
Akpinar, Y. (2008) ‘Validation of a learning object review instrument: relationship between ratings of learning objects and actual learning outcomes’, Interdisciplinary Journal of E-Learning and Learning Objects, vol. 4, pp. 291–302. Available at: https://www.learntechlib.org/p/180953/.
Buzzetto-More, N. A. (2015) ‘An examination of undergraduate student's perceptions and predilections of the use of YouTube in the teaching and learning process’, Interdisciplinary Journal of E-Learning and Learning Objects, vol. 10, pp. 17–32. Available at: https://www.learntechlib.org/p/180931/.
Denzin, N. K. (1978) The Research Act: A Theoretical Introduction to Sociological Methods, McGraw-Hill, New York.
Demain, P. (2012) ‘The use of virtual learning environments and their impact on academic performance’, Engineering Education, vol. 7, no. 1, pp. 11–19. doi: 10.11120/ened.2012.07010011.
DfES (2004) Five Year Strategy for Children and Learners, The Stationary Office, Norwich.
Douglas, J., et al., (2015) ‘Understanding student satisfaction and dissatisfaction: an interpretive study in the UK higher education context’, Studies in Higher Education, vol. 40, no. 2, pp. 329–349. doi: 10.1080/03075079.2013.842217.
Entwistle, N. & Ramsden, P. (1983) Understanding Student Learning, Croom Helm, London.
Flavin, M. (2016) ‘Technology-enhanced learning and higher education’, Oxford Review of Economic Policy, vol. 32, no. 4, pp. 632–645. doi: 10.1093/oxrep/grw028.
Gibbons, A. & Rogers, P. (2009) The Architecture of Instructional Theory, (Vol. 3 Building a common knowledge base), Routledge, New York.
Gordon, N. (2014) Flexible Pedogogies: Technology-Enhanced Learning, The Higher Education Academy, York. Available at: https://www.heacademy.ac.uk/system/files/resources/tel_report_0.pdf
Graham, C. (2006) ‘Blended learning systems. Definition, current trends and future direction’, in The Handbook of Blended Learning: Global Perspectives, Local Designs, 1st edn, eds C. Bonk & C. Graham, Pfeiffer, San Francisco, CA, pp. 3–21.
Guri-Rosenblit, S. & Gros, B. (2011) ‘E-learning: confusing terminology, research gaps and inherent challenges’, Journal of Distance Education, vol. 25, no. 1. Available at: http://ijede.ca/index.php/jde/article/view/729/1206
Henderson, M., Selwyn, N. & Aston, R. (2015) ‘What works and why? Student perceptions of 'useful' digital technology in university teaching and learning’, Studies in Higher Education. vol. 42, no. 8, pp. 1567–1579. doi: 10.1080/03075079.2015.1007946.
HESA (2018a) HE Student Enrolments by HE Provider, Domicile, Level of Study, Mode of Study, First Year Marker and Sex, [Online] HESA, Available at: https://www.hesa.ac.uk/data-and-analysis/students/table-1
HESA (2018b) Widening Participation: UK Performance Indicators 2016/17, [Online] HESA, Available at: https://www.hesa.ac.uk/news/01-02-2018/widening-participation-tables
HESA (2018c) Where do HE Students Study?, [Online] HESA, Available at: https://www.hesa.ac.uk/data-and-analysis/students/where-study#provider
Khor, S., et al., (2017) ‘Consumer perception towards internet health information resources’, in Handbook of Research on Leveraging Consumer Psychology for Effective Customer Engagement, ed N. Mohd Suki, IGI Global, Hershey, PA, pp. 234–244.
Laurillard, D. (2008) ‘Technology enhanced learning as a tool for pedogogical innovation’, Journal of Philosophy of Education, vol. 42, no. 3–4, pp. 521–533. doi: 10.1111/j.1467-9752.2008.00658.x.
Lawrence, K. (2015) ‘Today's college students: skimmers, scanners and efficiency-seekers’, Information Services & Use, vol. 35, pp. 89–93. doi: 10.3233/ISU-150765.
Leacock, T. & Nesbit, J. C. (2007) ‘A framework for evaluating the quality of multimedia learning resources’, International Forum of Educational Technology & Society, vol. 10, no. 2, pp. 44–59. Available at: https://www.jstor.org/stable/jeductechsoci.10.2.44
Macznik, A. K., Ribeiro, D. C. & Baxter, G. D. (2015) ‘Online technology use in physiotherapy teaching and learning: a systematic review of effectiveness and users’ perceptions’, BMC Medical Education, vol. 15, p. 160. doi: 10.1186/s12909-015-0429-8.
Margulieux, L., McCracken, W. M. & Catrambone, R. (2015) ‘Mixing in-class and online learning: Content meta-analysis of outcomes for hybrid, blended, and flipped courses’, In O. Lindwall, P. Hakkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.) Exploring the Material Conditions of Learning: The Computer Supported Collaborative Learning (CSCL) Conference (pp. 220–227), 2. Gothenburg, Sweden: The International Society of the Learning Sciences.
Mayer-Smith, J., Pedretti, E. & Woodrow, J. (2000) ‘Closing of the gender gap in technology enriched science education: a case study’, Computers & Education, vol. 35, pp. 51–63. doi: 10.1016/S0360-1315(00)00018-X.
McCutcheon, K., et al., (2015) ‘A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education’, Journal of Advanced Nursing, vol. 71, no. 2, pp. 255–270. doi: 10.1111/jan.12509.
Nesbit, J. C. & Leacock, T. L. (Ed.) (2009) Collaborative Argumentation in Learning Resource Evaluation, Idea Group, Hershey, PA.
Ong, C. & Lai, J. (2004) ‘Gender differences in perceptions and relationships among dominants of e-learning acceptance’, Computers in Human Behaviour, vol. 22, pp. 816–829. doi: 10.1016/j.chb.2004.03.006.
Patton, M. Q. (1990) Qualitative Evaluation and Research Methods, Sage, Newbury Park.
Piccoli, G., Ahmad, R. & Ives, B. (2001) ‘Web-based virtual learning environments: a research framework and a preliminary assessment of effectiveness in basic IT skills training’, Management Information Systems Quarterly, vol. 25, no. 4, pp. 401–426. doi: 10.2307/3250989.
Rowe, M., Frantz, J. & Bozalek, V. (2012) ‘The role of blended learning in the clinical education of healthcare students: a systematic review’, Medical Teacher, vol. 34, no. 4, pp. e216–e221. doi: 10.3109/0142159X.2012.642831.
Sclater, N., Peasgood, A. & Mullan, J. (2016) Learning Analytics in Higher Education: A Review of UK and International Practice, Full report, Jisc, Bristol. Available at: href="https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education
Stokoe, E., Benwell, B. & Attenborough, F. (2013) ‘University students managing engagement, preparation, knowledge and achievement: interactional evidence from institutional, domestic and virtual settings’, Learning, Culture and Social Interaction, vol. 2, pp. 75–90. doi: 10.1016/j.lcsi.2013.01.001.
Universities and Colleges Information Systems Association (UCISA) (2016) 2016 Survey of Technology Enhanced Learning for Higher Education in the UK, University of Oxford, Oxford.
Vale, C. & Leder, G. C. (2004) ‘Student views of computer-based mathematics in the middle years: does gender make a difference?’ Educational Studies in Mathematics, vol. 56, pp. 287–312. doi: 10.1023/B:EDUC.0000040411.94890.56.
Veneri, D. (2011) ‘The role and effectiveness of computer-assisted learning in physical therapy education: a systematic review’, Physiotherapy Theory and Practice, vol. 27, no. 4, pp. 287–298. doi: 10.3109/09593985.2010.493192.
Virtanen, V. & Lindblom-Ylanne, S. (2010) ‘University students’ and teachers’ conceptions of teaching and learning in the biosciences’, Instructional Science, vol. 38, no. 4, pp. 355–370. doi: 10.1007/s11251-008-9088-z.
Wilson, B. G. (1996) Constructivist Learning Environments: Case studies in instructional design, Educational Technology Publications, Englewood Cliffs, NJ.
Woodhall, T., Hiller, A. & Resnick, S. (2014) ‘Making sense of higher education: students as consumers and the value of the university experience’, Studies in Higher Education, vol. 39, no. 1, pp. 48–67. doi: 10.1080/03075079.2011.648373.
Yukselturk, E. & Bulut, S. (2009) ‘Gender differences in self-regulated online learning environment’, Journal of Educational Technology & Society, vol. 12, no. 3, pp. 12–22. Available at: https://www.j-ets.net/ETS/journals/12_3/3.pdf