Just-in-time research: a call to arms for research into mobile technologies in higher education

Lucie Byrne-Davis^a*, Hilary Dexter^a, Jo Hart^a, Tim Cappelli^a, Ged Byrne^b, Ian Sampson^a, Jane Mooney^a and Colin Lumsden^a on behalf of the MMS Mtechnology Research Group

^aManchester Medical School, University of Manchester, Manchester, UK; ^bHealth Education England, Liverpool, UK

Abstract

Mobile technologies are becoming commonplace in society and in education. In higher education, it is crucial to understand the impact of constant access to information on the development of the knowledge and competence of the learner. This study reports on a series of four surveys completed by UK-based medical students (n=443) who received tablet computers (iPads) from their medical school during their 4th year of study. Students were surveyed prior to receiving the iPads and again regarding their usage and experiences at 2, 6 and 12 months post receipt of tablets. Findings indicate that students differed in their use of iPads but that the majority felt that tablets had impacted on their learning and the majority were using them frequently (at least once a day) during learning. Almost half of the students reported that clinical supervisors had raised the possibility of tablets changing patient care. These results, although only descriptive, raise important questions about the impact of mobile technologies on learning.

Keywords: clinical learning; just-in-time technology; mobile technologies; survey; tablet computers; undergraduate medical education

Introduction

The use of mobile technology (mtechnology) is growing. In the UK, OFCOM (the independent regulator and competition authority for the UK communications industries) reported in 2013 that 51% of adults owned a smart phone, using it to connect to the internet, and 80% of smartphone owners also owned a tablet device, such as an iPad. Twenty percent of adults own a tablet device, with tablet ownership doubling from 2012 to 2013 (OFCOM 2013). More than 30% of webpage traffic came from mobile devices in February 2013. Since the huge growth of the World Wide Web in the 1990s, access to knowledge has impacted on teaching and learning. Students entering higher education in the UK in recent years have been referred to as ‘digital natives’, i.e. people born after 1980 who have, therefore, been brought up in an era with personal computing (Prensky 2001). This has led to theories that current students think and process information differently to cohorts of students who were brought up before widespread computer technology (Prensky 2001). However, recent studies have found that, although students differ in their quantitative use of technologies, i.e. some students use more technology than others, there is no evidence that students who are ‘digitally native’ use technology in a different way from those who are not (Margaryan, Littlejohn, and Vojt 2011).

The availability of facts has undoubtedly coincided with, and perhaps contributed to, the shift in focus from teaching to facilitating learning, congruent with the fall of lecturing and the rise in enquiry-based forms of learning in medical education. The growth of mobile, so-called ‘just-in-time’ technologies, however, is set to change the face of medical education and practice even more profoundly. Now, more than ever, facts and complex processing capacity are readily and constantly available for students and healthcare professionals in high-income, and increasingly low- and middle-income, countries. It is the role of medical educators and medical education researchers to investigate the impact of these increasingly ubiquitous devices on the learning and practice of medicine.

Sparrow, Liu, and Wegner (2011) have proposed that people are using just-in-time technologies, including tablets, mobile phone and satellite navigation systems, as transactive memory stores. Transactive memory is the phenomenon whereby people learning and working in groups defer the learning of certain facts or processes to individuals within the group (Wegner 1986). One member, for example, might be charged with the responsibility of knowing how to complete certain complex processes, whilst another with remembering lists of numbers or facts. Wegner and colleagues saw this as a positive part of group membership as it capitalised on the cognitive processing power and memory available, avoiding duplication and increasing the efficiency of the group. Recently, a series of experimental studies of recall of facts using Internet search engines, found that people were significantly more likely to remember where to find information than the substance of that information (Sparrow, Liu, and Wegner 2011). They were more likely to remember the factual content if they believed that it was not going to be available later, however. This is evidence that, in these samples of undergraduate students in the USA, people can use, and are using, the Internet as an external memory store, as per transactive memory theory.

In medicine, the impact of computing on the doctor–patient relationship has long since been of concern, with studies considering both the impact of availability of information (Gerber and Eiser 2001) and the physical presence of the computer (Sullivan and Mitchell 1995). These studies illustrate that medical students having mobile technologies with them in their clinical learning environments could affect their relationships with patients, both in terms of their increased access to factual information and also their visible use of computing devices during or around consultations. This has yet to be fully explored in clinical learning or in clinical practice.

Recent research has shown that trainee doctors valued just-in-time technologies in their clinical practice and learning. In a study of trainee doctors’ use of a library of medical textbooks on smartphones, participants reported using the smartphones to access a wide range of information to assist with diagnosis and treatment (Hardyman et al. 2013). This study, however, necessitated participants to actively opt in to receive and use the smartphone. The authors reported that some of the participants did not use the smartphones, but they did not report how many of those agreeing to participate in the study had prior experience with just-in-time technologies. It is not possible, therefore, to generalise from their use and experiences to all trainees as it is possible that these were an unusual sample either keen on smartphone use, hence their participation or perhaps not owning a smartphone before and therefore participating to receive the technology. Indeed, a recent study in the US reported that medical students used devices when they saw a clear purpose and did not when the purpose of using the device was less clear (Ellaway et al. 2014). Institutions considering supplying devices, therefore, need information about how they are used by ready adopters and non-adopters. Another recent study, in a UK medical school (Davies et al. 2012), investigated the use of mobile devices containing preloaded medical information. Whilst interesting in terms of just-in-time learning, these stand-alone devices cannot inform us about the complex uses of devices which connect medical students almost constantly to the web.

Background

In 2011 there were no UK medical schools where tablet devices were routinely given to students to aid their learning. Medical school in the UK typically involves a 5-year course, in which the first 2 years are mainly focused on learning the underpinning sciences and the subsequent 3 years are spent immersed in clinical environments. In our medical school, students from year 3 onwards are based in one of four teaching hospitals and have infrequent contact with the central medical school. As a result of this, we were faced with a difficulty in providing all of our students with equal access to course resources. We also wanted to improve direct channels of communication to and from our students. An options appraisal was conducted and it was decided to trial a whole cohort pilot of iPads to our year 4 students. iPads were selected as the e-learning team decided that they were the most appropriate device for delivery of course resources at the time. Year 4 students were selected as students in this year group are the most dispersed, geographically. Medical students in clinical learning are engaged in a number of types of learning activities. Some of these take place in clinical environments, e.g. wards, outpatients’ clinics, GP surgeries. Anecdotally, students have reported that, due to the nature of clinical learning, they feel that their time is sometimes wasted, e.g. waiting for clinics to begin, waiting between patients, travelling to and from placements. Some of their learning takes place in lectures and small group learning, typically in education and research centres of hospitals. Our students take part in problem-based learning (PBL): a form of group learning in which students learn through creating and studying a particular ‘problem’ (Wood 2003). In medical schools the ‘problems’ are typically descriptions of a ‘case’ involving a patient. Our year 3, 4 and 5 students follow eight steps in their PBL (O’Neill, Willis, and Jones 2002) with private study in between the opening and closing of a case. In the closing of the case, students present the information about the problem that they have learnt during their private study. Anecdotally, there have been reports and concerns, from PBL tutors, that students could use their iPads to participate in the closing of PBL cases without having done the private study, since they would have ready and easy access to information about the case through their iPads, with use of search engines and medical applications.

As the first medical school in UK to trial the distribution of tablet devices to a complete cohort of our clinical students, we were in a unique position to study the use of an internet-connected mobile device by clinical medical students in the UK. In this initial pilot study, we were interested both in how the students used them and how they perceived their use. We wanted to investigate this in students with a wide range of previous experience of technologies. We were also interested in defining the questions that the medical education community should be trying to answer regarding the use of technology in medical students and we viewed this first large-scale evaluation as fertile ground to refine future research directions. Therefore, we aimed to explore: 1) how students reported using their iPads in their distributed, clinical environments, and 2) the perceptions of the students about using iPads for learning. Additionally, a third aim was to propose a list of questions arising from the experiences of the implementation of large-scale mtechnology in a medical school.

Methodology

This was a descriptive, baseline, pilot study using questionnaires and reporting frequencies. The questionnaires included both closed and open questions and closed questions had a variety of answer options (checked, yes/no, estimation of extent, estimation of frequency). As this was an evaluation of an educational intervention, questions were derived from the purpose of the intervention, i.e. to improve access to and use of resources and to improve communication. The majority of the questions were check box, yes/no answers and estimations of frequency about usage and did not, therefore, require validation. Questions around extent required judgement from the participants and, therefore, should be treated as pilot questions. Baseline survey had seven questions, 2-month survey had 74 questions, 6-month survey had 84 questions and 12-month survey had 80 questions. All questionnaires are included in the appendix.

Methods

Participants

All 4th year medical students in a large UK medical school (n=443) were given iPads in December 2011 at the end of their first semester in 4th year. iPads remained the property of the university. Full details of the practical challenges and solutions of implementation of the iPads is discussed elsewhere (Mooney et al. 2013). All students who received iPads were invited to participate in online surveys but these were not compulsory.

Materials

Questionnaires investigated:

1. What students were using their iPads for at home, during work (and during breaks at work) and during travel.
2. How often students were using their iPads.
3. Students’ perceptions of how iPads had impacted on learning, including communication with faculty and other students and whether they thought their learning had been altered.

Procedure

We conducted a survey at baseline, 2 months, 6 months and 12 months post distribution of iPads. Surveys were online and all students were sent a link and asked to complete the questionnaire. Completion was not mandatory but students were offered incentives (iTunes gift vouchers for five respondents selected at random). Students were allowed to complete the survey anonymously and questionnaire responses were not linked between each time period. As this was a routine evaluation of the implementation of an educational device, research ethics committee approval was not required. We adhered to ethical guidelines in the gathering, analyses and production of reports in that participation was voluntary, anonymous and non-participation did not result in any cost or consequences to students.

Analyses

We analysed data descriptively and have presented data as frequencies and percentages.

Results

Participants

A total of 419 students (95%) answered the baseline survey; 256 students (58%) responded to the 2-month survey; 141 students (32%) responded to the 6-month survey; 192 students (43%) responded to the 12-month survey.

iPad expectations

Of the 419 students who answered the baseline survey, only 17 (4%) already had an iPad but 149 (36%) had an iPhone. Almost all students expected to use the iPad to access email (91%) and their timetable (93%). Interestingly, 149 students (36%) reported that they had concerns about the iPad (see Table 1).

In the free comment part of the questionnaires, concerns reported, prior to receipt of the iPad, were the availability of Wi-Fi access, potential theft/safety issues, infection control issues, being able to use iPads on the wards/with patients, learning how to use it, losing the iPad, damage, cost of apps/covers, availability of non-Adobe^® Flash^®, i.e. device compatible, resources. Note, that during the pilots, the numbers of lost, stolen or broken iPads were three lost, two stolen, nine broken and four hardware problems replaced under warranty (4% of iPads in total had a problem).

Usage

At 2 months, 142 (55%) of the students were using their iPads in clinical areas with 87% reporting that it was convenient to carry at work. Despite this, only 61% reported that there was safe storage at work and 71% had concerns about the iPad being stolen.

At 6 months, over half of the students (54%) were using their iPads in clinical areas, with 87% indicating that they felt it was safe to carry at work. Despite the reduction in students answering the survey at 6 months, compared to 2 months, the percentages are the same. Only 61% of students indicated that they felt there was safe storage at work, 74% had concerns about it being stolen and only 69% indicated that they were using the iPad to the extent they had expected. Around half of the students indicated that they had required help with the iPad (see Table 2) and 92% reported that their familiarity with the iPad had increased. 65% of the students reported using the iPad daily or multiple times daily for work-related purposes (see Table 3).

At 12 months, the number of students using their iPads less than once a week increased from 7% at 2 months to 14% at 6 months to 21% at 12 months. This is interesting as it seems that there might be a minority of students who tried to engage with the iPad/engaged initially and then felt that engagement was not beneficial to them/easy enough/convenient enough and therefore stopped.

Perceptions of impact on teaching and learning

The majority of students, at all-time points, felt that using the iPad had become part of their workplace learning and that it had had an impact on their learning. At 12 months, 44% of students reported that the issue of mtechnology devices impacting on patient care had been raised by clinical supervisors. At 12 months, 16% of students reported that the iPad had had an impact on their interactions with patients. At 6 and 12 months, over 70% of students reported that the iPad had caused a positive change in use of previously wasted time. Also at 6 and 12 months, over 50% of students reported that the iPad had a positive effect on their use of time for extracurricular activities and relaxation (Tables 4 and 5, Figure 1).

Figure 1. Students’ perceptions about iPads changing learning (at 2 months).

Communication and engagement with others

Impact on communication was both within and outside of PBL groups, including communication, resource sharing, knowledge and experience, as reported by 62–80% of the students at all-time points (see Table 6).

Discussion

We aimed to discover how students were using their iPads, what the students’ perceptions were of using their iPads and to generate crucial questions for future studies. We had low to moderate response rates to two of our four surveys. In interpreting our results, we are assuming that the students who answered the survey are those who are more likely to be engaged in using their iPads or those who are very against the use of the iPads. In light of this assumption, we think that our results probably indicate an extreme of experiences with iPads in medical students, rather than the mainstream experiences.

We found that over a third of our students were initially concerned about the use of iPads. These were largely concerns about the processes of using iPads, rather than any concerns about the impact on teaching, learning, communication or patient experience. iPad use had changed over the 12-month period since students received their iPads. In particular, we found that there was a minority of students who appeared to use their iPads in the initial period, whose usage then reduced. Importantly, students were using their iPads to fill previously unused time in their clinical learning, i.e. using their iPads to store and access information during breaks during the day in their clinical placements. These medical students were not a homogeneous group in terms of their perceptions of iPads. Some students were highly motivated, ready and able to adopt the new technology whilst some were not motivated, were not ready and did not add the iPad into their toolkit of learning aids.

These findings have implications for the successful implementation and evaluation of just-in-time technologies in medical education. Firstly, there is a tendency, in medical education at least, to view students as a relatively homogeneous group, especially in respect of their understanding and use of technology. As a ‘digital native’ cohort, faculty often have expectations that students will be capable and motivated in terms of adopting new technology. In keeping with more recent research, we found that this was not the case. This has important implications if new technologies are adopted by higher education institutions (HEIs) with a view to increasing equality of availability of important technology. Our data shows that even with openly available technology, students will vary in their perception and use. Variations in use and perceptions might be influenced by students’ previous experiences of technology. HEIs, therefore, must be careful not to increase inequality, whilst attempting to decrease inequality. Secondly, iPad use and student perceptions changed over the 12-month period of our pilot. This is important in terms of the timing of evaluations of the adoption of new technologies.

We found that many medical students used their iPads frequently (more than once a day), that they used them to communicate, and to use previously wasted time and in clinical learning environments. We found that around half of the students were using their iPads in clinical areas and around half were using their iPads multiple times daily. If students are using their iPads frequently during clinical placements, it is possible that they are using them transactively, i.e. to store facts for later use. Clearly, these data do not definitively show transactive use but highlight the possibility for future study. We found that students perceived that iPad use had had an impact on their learning and on their communication with colleagues. Whilst most students reported that the iPad had not affected their interactions with patients, about half of the students, at 12 months, reported that the impact of iPads on patient care had been raised by their clinical supervisors. These results are important in that they indicate that mtechnology could be changing the way that students learn and that there are concerns, amongst medical professionals, about potential effects on patient care.

A final aim of our study was to determine the important questions facing medical educators in terms of mtechnology. Our results, and our experiences, indicate that the following two broad questions are crucial in understanding the impact of mobile technology on medical education:

• What is the impact of having constant, available access to ‘facts’ (of varying degrees of veracity) on the developing learner?

We currently don’t know to what degree the student-reported change in learning habits reflect the use of iPads as transactive memory stores. Certainly, it is possible that the deliberate non-remembering of factual information could affect the development of medical expertise. The job of the developing doctor was once to create a ‘fact repository’, from which knowledge could be drawn and synthesised throughout their medical career. This was in the context of libraries of written information that were not readily available at the point of care. With the increasing access to ‘facts’ at all times, from mobile technologies, it is crucial to understand how the role of medical education might evolve. This is important both in formal learning, e.g. students’ access to the internet during PBL and in informal learning. Future studies must focus on how the student and information interact through the medium of mtechnologies.

• How acceptable are mtechnologies to health professionals and patients and how will this affect how they can be used in clinical learning?

Future studies must include understanding in detail how learners interact with mobile devices: in clinical learning environments and in more prescriptive learning sessions, such as problem, or case-based learning groups. Crucial to the furthering of understanding of the impact of mobile technologies, is careful modelling of the important processes and outcomes of learning and how these might be influenced by mtechnologies, for better or for worse.

Conclusion

We studied the first large-scale implementation of iPads in a UK medical school. We found that students were heterogeneous in their use of iPads, supporting previous findings that rebut the concept of the ‘digital native’s’ acceptance and use of technology. We found that medical students were using iPads to make use of previously difficult-to-use time, e.g. waiting for clinics to begin. Students were positive about the impact on their learning but around half of them reported that clinical supervisors had raised concerns about use of iPads in clinical environments. This has implications for the ability of medical schools to make routine use of mobile technologies in clinical learning environments. We have highlighted two research areas: the impact of having constant access to information on the developing expertise in medicine and the acceptability of mobile devices in clinical learning.

References

Davies, B. S., et al., (2012) ‘Mobile Medical Education (MoMEd) – how mobile information resources contribute to learning for undergraduate clinical students – a mixed methods study’, BMC Medical Education, vol. 12, no. 1, pp. 1. doi: 10.1186/1472-6920-12-1. PubMed Abstract | PubMed Central Full Text | Publisher Full Text

Ellaway, R. H., et al., (2014) ‘Left to their own devices: medical learners’ use of mobile technologies’, Medical Teacher, vol. 36, no. 2, pp. 130–138. doi: 10.3109/0142159X.2013.849800. PubMed Abstract | Publisher Full Text

Gerber, B. S. & Eiser, A. R. (2001) ‘The patient physician relationship in the Internet age: future prospects and the research agenda’, Journal of Medical Internet Research, vol. 3, no. 2, pp. E15. Publisher Full Text

Hardyman, W., et al., (2013) ‘Mobile technology supporting trainee doctors’ workplace learning and patient care: an evaluation’, BMC Medical Education, vol. 13, no. 1, pp. 6. doi: 10.1186/1472-6920-13-6. PubMed Abstract | PubMed Central Full Text | Publisher Full Text

Margaryan, A., Littlejohn, A. & Vojt, G. (2011) ‘Are digital natives a myth or reality? University students’ use of digital technologies’, Computers & Education, vol. 56, no. 2, pp. 429–440. doi: 10.1016/j.compedu.2010.09.004. PubMed Abstract | PubMed Central Full Text | Publisher Full Text

Mooney, J., et al., (2013) ‘Implementing mobile learning with iPads in higher education: a large scale case study’, in Mobile Learning: How Mobile Technologies can Enhance the Learning Experience, eds J. Ahmed & S. Horrigan, Universities and Colleges Information Systems Association, Oxford, pp. 18–23.

OFCOM. (2013) The OFCOM Communications Market Report 2013. London [online] Available at: http://stakeholders.ofcom.org.uk/binaries/research/cmr/cmr13/UK_2.pdf

O’Neill, P. A., Willis, S. C. & Jones, A. (2002) ‘A model of how students link problem-based learning with clinical experience through ‘elaboration’’, Academic Medicine, vol. 77, no. 6, pp. 552–561. PubMed Abstract | Publisher Full Text

Prensky, M. (2001) ‘Digital natives, digital immigrants part 1’, On the Horizon, vol. 9, no. 5, pp. 1–6. Publisher Full Text

Sparrow, B., Liu, J. & Wegner, D. M. (2011) ‘Google effects on memory: cognitive consequences of having information at our fingertips’, Science, vol. 333, no. 6043, pp. 776–778. doi: 10.1126/science.1207745. PubMed Abstract | Publisher Full Text

Sullivan, F. & Mitchell, E. (1995) ‘Has general practitioner computing made a difference to patient care? A systematic review of published reports’, BMJ, vol. 311, pp. 848–852. PubMed Abstract | PubMed Central Full Text | Publisher Full Text

Wegner, D. M. (1986) ‘Transactive memory: a contemporary analysis of the group mind’, in Theories of Group Behaviour, eds B. Mullen & G. R. Goethals, Springer-Verlag, New York, pp. 185–208.

Wood, D. F. (2003) ‘ABC of learning and teaching in medicine: problem based learning’, BMJ, vol. 326, no. 7384, pp. 328–330. doi: 10.1136/bmj.326.7384.328. PubMed Abstract | PubMed Central Full Text | Publisher Full Text