ORIGINAL RESEARCH ARTICLE
Mixed reality results in vocational education: a case study with HoloLens 2
Jonathan Adamsa*, Fallyn Flavellb and Ramari Rauretic
aWhanake Ake | Academic Development, Toi Ohomai Institute of Technology, Te Pūkenga, New Zealand; bWhanake Ake | Academic Development, Toi Ohomai Institute of Technology, Te Pūkenga, New Zealand; cTiriti and Māori Relationships, Toi-Ohomai, NZIST | Te Pūkenga, New Zealand
(Received: 12 May 2022; Revised: 24 October 2022; Published: 22 December 2022)
Pedagogy in vocational education is challenged by the increasing availability of mixed reality (MR) technologies. Wearable technologies such as the Microsoft HoloLens are being embedded in nursing education programmes in Aotearoa New Zealand and Australia and explored in other subject areas such as construction, architecture, and engineering. Empirical research undertaken with staff and students in a nursing programme reinforces and expands on previous research findings about enhanced student motivation, learning and organisational change implications, highlighted in the 2020 special issue of Research in Learning Technology that provided a ‘state of the art’ review of mobile mixed reality (MMR) in education as at 2019. This new research also identifies a number of technical or design constraints that need to be overcome to optimise MR for vocational and higher education use. Importantly, it highlights the need for significant improvement in integration of MR platforms into the learner journey so that use of the technology is not confined to professional users and early adopter student cohorts. Finally, the authors note the portability that MR headsets offer with the potential to deliver training across multiple campus and work-based learning environments to a consistent standard.
Keywords: mixed reality; vocational education; tikanga Māori; education technology; HoloLens
*Corresponding author. Email: jonathan.adams@toiohomai.ac.nz
Research in Learning Technology 2022. © 2022 J. Adams et al. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2022, 30: 2803 - http://dx.doi.org/10.25304/rlt.v30.2803
Introduction
In Aotearoa New Zealand’s institutes of technology and polytechnics (ITP) sector, blended learning had been increasingly implemented, even prior to the onset of the Covid-19 pandemic in early 2020 with some fully online qualifications. In both face to face and blended learning, course content and theory delivery are typically mediated through web-based Learning Management Systems (LMSs), with formats that reflect to varying degrees, a conventional classroom delivery. All Aotearoa New Zealand universities and ITPs use LMS’s (Matear 2021; Nichols, cited in Nanayakkara and Whiddett 2005, p. 181) with the training aspects of vocational education performed either in classroom, or the skills-related environment such as a workshop, clinic or worksite in person by the teacher. In nursing education there is a recognised ‘theory–practice gap’ (Scully 2011) that occurs between the necessary course theory and practical clinical experience that also contrasts with learning in the context of real-life situations and alignment of motivation (Knowles, Holton, and Swanson 2015).
To explore the potential of Mixed Reality (MR) providing a stronger connectivity between theoretical and applied content, to ‘prioritise applying theory in context specific and workable ways’ (Greenway, Butt, and Walthall 2019, p. 4), research was undertaken at Toi Ohomai Institute of Technology / Te Pūkenga (Toi Ohomai / Te Pūkenga) using science-based content in nursing degrees. This leads on from the findings on measuring student progress in 2019 by Nichols, cited in Collins and Ditzel (2021, p. 181), and student success in Australia undertaken by the University of Canberra (Frost, Delaney, and Fitzgerald 2020), and desk-based research on the use of MR in nursing education by Kim, Choi and Kim (2021). The research was to serve as a reference point for the student experience and be available for comparisons with similar HoloLens implementations in three other Aotearoa New Zealand ITP nursing institutions. In addition, the research sought to explore the wider potential for using commercial MR technology in training beyond nursing and health sciences. This new empirical data was particularly timely given that the ITP sector is being merged into one new national entity with a significant scope and scale not previously found in the sector.
The MR in the reality-virtuality continuum
When discussing MR it is important to understand the differences in the visual and immersive experiences between MR, where it sits between Augmented Reality (AR) and Virtual Reality (VR) and their place on a Reality-Virtuality Continuum described by Milgram and Kishino (1994) that has different levels of immersion as perceived by the user. The Reality-Virtuality Continuum was revised by (Skarbez, Smith, and Whitton 2021) who proposed a three dimensional taxonomy comprising (1) Extent of World Knowledge (EWK); (2) Immersion and (3) Coherence.
Augmented Reality, adds information to augment or feedback into the real-world environment, common examples use head up displays and audio or haptic feedback (touch) in motor vehicles. These augmented feedback systems have knowledge about the real world but a low level of immersion (Skarbez, Smith, and Whitton 2021).
In Mixed Reality, information and objects are within the physical environment of a user, but the virtual objects also possess knowledge about the physical world, the (EWK) dimension (Skarbez, Smith, and Whitton 2021). The virtual objects can react to, give physical cues or be controlled by the user. This gives students ‘the potential to reduce cognitive overload by providing students with perfectly situated scaffolding’ (Bower et al. 2014, p. 1).
A Virtual Reality experience completely immerses the student in the artificial environment. These experiences can be used in education to simulate hard to access, or high risk real-world environments such as an operating theatre fire (Wunder et al. 2020). Virtual Reality (VR) can also create experiences unrelated to real-world environments, typically seen in entertainment and gaming.
Benefits of MR in training
The top-ranked benefits of MR identified by (Forrester Consulting 2021) were Training, Task Instructions and Visualisation, Remote Collaboration and Design, and Decision making. These benefits are directly relevant to the ITP and training sector where delivery of learning is through both applied hands-on experiences that require students to utilise spatial knowledge, and classroom delivery of theory components.
Key competencies that can be developed with MR include spatial reasoning and situational judgement (Papakostas et al. 2021). Research on the use of MR in the architectural design industry (Osorto Carrasco and Chen 2021) found a MR-based design can communicate over 20% more information to the client compared with two-dimensional (2D) media.
The MR experience has also been found to contribute to nursing student’s clinical reasoning and judgement skills in 2019 research with the HoloLens (Collins and Ditzel 2021). It also helped the educators to assess a student’s development so that they were able to customise the student’s learning to further develop those skills. These benefits were supported by (Frost, Delaney, and Fitzgerald 2020) who were also using the same HoloLens and GIGXR content for second year students while in a 3-year Bachelor of Nursing programme in Australia. All the students in their study stated that the experience with virtual standardised patients assisted them in their learning.
A scoping review of MR simulation in nursing education (Kim, Choi, and Kim 2021) found the HoloLens introduced by Microsoft in 2016 was the most commonly used device and had a number of advantages for nursing education which requires students to develop both clinical skills and professional judgement. The MR was also able to avoid some of the negative aspects of VR such as lack of spatial awareness that can lead to nausea, or headaches or blurred vision while fully immersed in VR. The review concluded:
Nursing education simulation requires programs that integrate both nursing skills and situational judgment, which are essential traits for nurses. MR can properly utilize and reproduce virtuality and reality. Thus, MR may be effective in nursing education simulations. (Kim, Choi, and Kim 2021, p. 8)
Method
Toi Ohomai / Te Pūkenga delivers vocational education to approximately 10 000 students in the Bay of Plenty and Southern Waikato regions of Aotearoa New Zealand. In 2021 Toi Ohomai / Te Pūkenga offered around 150 programmes from certificate to master’s degrees, across five campuses. Toi Ohomai / Te Pūkenga is one of 17 subsidiaries including all Aotearoa New Zealand’s ITPs, which from the beginning of 2023, will be merged into the new national entity Te Pūkenga, the New Zealand Institute of Skills and Technology.
The content from a range of vocational courses in Architecture, Engineering, Construction (AEC) and Nursing was tested in partnership with students and staff to assess its use with the HoloLens. From initial exploration, research was developed around Nursing education, as staff and students became familiar with the technology and its potential.
An example of MR in ITP construction courses is converting and presenting what were flat two-dimensional (2D) building plans for a student into three-dimensional (3D) form or a digital twin on a building site, where they can walk through the design at 1:1 scale in the building on its site and foundations (see Figure 1). This introduces the student to commercial applications for the HoloLens 2 that are already used for larger construction and civil works (BIM Holoview 2022). A review by Huang, Shakya and Odeleye (2019) observed that information (such as the building plans used by Toi Ohomai / Te Pūkenga) can be used to detect any conflicts in design in relation to the location or construction. The spatial presentation or presence, in context with the physical world presence provides an extension of the learning environment or place and is a key benefit in using MR in AEC subjects, in contrast to an immersive VR experience without the real-world context.
Figure 1. Student view of construction plans with the Bimholoview app on HoloLens 2 at Toi Ohomai / Te Pūkenga.
Source: Reproduced with permission from BIM Holoview.
HoloLens 2 headsets are also used in the nursing programme with students undertaking anatomy lessons and viewing standardised patients for Clinic Reasoning Cycle training (see Figure 2), using applications licensed from GIGXR (Putland 2021).
Figure 2. Students viewing a standardised holographic patient in clinic reasoning cycle training at Toi Ohomai / Te Pūkenga.
The study in this report involved quantitative data collection from 120 first-year Science students at Toi Ohomai / Te Pūkenga where students were enrolled as part of the Bachelor of Nursing degree. HoloLens headsets were used in on-campus tutorials within the nursing degree courses in Semester 1 (February–June) and Semester 2 (July–November) 2021. Student feedback on the impact on their learning was obtained through an online survey instrument where responses were collected after each tutorial. Responses were voluntary and anonymously recorded. Participants were asked to rate what their thoughts were about the HoloLens experience on a Likert scale of 1 (no change) through to 5 (strongly agree). The question scale displayed for the students is shown in Figure A1. An open-ended question at the end allowed capture of some qualitative data. The study provided baseline data about the first-year student experience with plans to survey the students through their 3-year course. Field notes (observations) were taken by the lead author who was the educational technologist who supported the nursing tutors with the MR technology.
Anatomy models (GigXR 2021) were accessed on the HoloLens 2 within scheduled science tutorial classes, with one tutorial in a laboratory and a second tutorial on the same topic in a classroom setting. Virtual anatomy models were viewed by the student that showed the position and structure of organs and systems. The headset displays a 3D model in the user’s real-world environment with a 100-degree field of view (Microsoft Corporation 2022a). The application models from GIGXR could be hidden, scaled and moved or manipulated by the student with a user menu, viewed at 1:1 scale and larger. A total of six headsets were used in each tutorial, typically five headsets were being used simultaneously. The student-controlled menu was virtually attached, showing anatomy sections and available body systems (see Figure 3).
Figure 3. Student view of the GIGXR HoloHuman app on HoloLens 2 at Toi Ohomai / Te Pūkenga.
Source: Reproduced with permission from GIGXR.
In Figure 3, the student user’s menu is on the right, and another student facing the viewer is also viewing the same anatomy model. The teachers lead the sessions with an appropriate andragogical model (Knowles 2015) by setting the guidelines for students who then take on responsibility for their learning. The students’ tutorial exercises required them to identify and locate a range of body systems and anatomy that were listed for them on a whiteboard. Students were able to see the room, the virtual models and other participants, and could discuss their experience and ask questions throughout each session with their colleagues and tutor as they proceeded in their self-directed learning objectives.
When using HoloLens 2 headsets the learner engages and manipulates the content with an illusion of presence for the student without being highly immersed (Slater 2018). Students can control and get feedback with audio and can still interact and collaborate with others in the class or remotely. Each tutorial also had at least one additional support person from the Toi Ohomai / Te Pūkenga academic support team for teachers to assist students to login and reset sessions to expedite the number of sessions that could run within the scheduled time. This close collaboration and support between the educational technology teams and nursing teachers was fundamental to the successful incorporation of MR into the nursing programme.
Embedding cultural competencies with learning technologies
Aotearoa New Zealand’s founding document, te Tiriti o Waitangi, requires public sector tertiary education organisations to respect and value tikanga Māori (the customs and practices of Māori). Meetings were held with Māori cultural advisors, nursing teachers and student support staff prior to the start of the academic year to discuss the implications of using the technology and appropriate tikanga when using the HoloLens with students. This was seen as particularly important given the nature of the subject of nursing with its close contact with the human body.
In addition, nursing education in Aotearoa New Zealand has included cultural safety as an important focus of the curriculum and pedagogy since the early 1990s (Hickey and Wilson 2017; Papps and Ramsden 1996; Ramsden 1993; Richardson and Carryer 2005). This includes respect for mātauranga Māori (Māori knowledge) and cultural beliefs and values, concepts such as ‘tapu’ (loosely interpreted as ‘sacred’) and noa (not tapu), and the tapu nature of the human body especially the head. Respect for the tūpāpaku or bodies of the deceased is also central to Māori cultural practices (Haami 2019; McClintock and McClintock 2018). Consideration was given to the impact and use of highly realistic models within a classroom setting, where previously human anatomy was accessed through plastic models, illustrations and 2D animations. A second consideration was around the HoloLens hardware and implications of touching another person’s head, placing headgear on tables that are used for food (not tapu) or seats used for sitting (not tapu) and care in handling the MR equipment in a classroom setting.
Lastly, there are four domains of competence for the Aotearoa New Zealand registered nurse scope of practice. The first domain, professional responsibility, includes competencies with legal, ethical and cultural safety responsibilities, reinforced with New Zealand Nurses Organisation Code of Ethics (New Zealand Nurses Organisation 2019, p. 10)
With these responsibilities in mind, each headset included a QR code to launch the session, and a printed karakia (incantation) so students are reminded to approach the learning experience with respect.
Teachers include mātauranga Māori in face-to-face and online lessons with the intention that the tikanga students practice in the classroom carries with them into their professional careers. Although students are interacting with a hologram, it simulates a real body, so we saw this as an opportunity to:
- provide both cultural safety and acknowledgement
- instill the same level of respect expected of the nursing students in their work environments
- demonstrate the expression of the nursing profession’s ethical values including the concept of Manaakitanga (care).
Finally, before the first session began, a short video was played for the class showing what the student would be seeing, including menus, models and opportunities for questions. This was to help students scaffold from the technology setup and into the new learning experience.
Question categories
The design of the online survey was guided by the Learning Object Evaluation Scale for Students (LOES-S) categories of ‘learning’, ‘quality or design’ and ‘engagement’ (Kay and Knaack 2009).
Results
During the sessions using MR it was observed that most students experienced a ‘lightbulb’ moment as they viewed what were once abstract or 2D models or plans presented spatially or as an illusion of presence (Slater 2018) when they wore a HoloLens headset (see Figure 4).
Figure 4. Using the GIGXR anatomy app on HoloLens 2 at Toi Ohomai / Te Pūkenga in tutorials.
There were some observable differences based on broad age groupings of the students. Mature age students had a relatively longer learning curve to become comfortable with using the headset, but then tended to express stronger feelings or results in using the technology, such as impact on motivation or interest. Younger students (those who typically enrolled directly from high school) were quicker to be comfortable using the headsets and then move on to explore the content and menu options. One inference is to examine the existing digital literacy of the participant as a dimension when measuring student receptivity or engagement with MR experiences.
Measuring the student experience
Of the 120 students who participated in the tutorials with the HoloLens, 78 unique online survey forms were completed.
Increased understanding
Just over half, or 51% (40 students) strongly agreed that the HoloLens session improved their understanding of the subject (anatomy), while 14% (11 students) reported no or slight change (see Figure 5).
Figure 5. Student survey results from a scale of 1 on the left (no change) through to 5 on the right (strongly agree).
Increase in motivation or interest
Students were then asked if the HoloLens session improved their interest or motivation in learning more about the subject, 71% (55 students) or participants strongly agreed, while 10% (8 students) reported no change or slight change in their motivation.
Relevance
All students reported some relevance and 67% (52 students) strongly agreed that the HoloLens session was useful in preparing them for their future work life, while the actual context for that use in a nursing career was not given or described.
Comfort or support in use
After a karakia and watching an introduction or scaffolding video prior to use only 73% (57 students) felt comfortable or very comfortable, while 9% (7 students) still felt uncomfortable or slightly uncomfortable or not well supported using the equipment, with remainder neutral.
Students were also asked to report on anything they experienced during the HoloLens session, or on anything that could be improved. Feedback was grouped into positive, negative and neutral categories (see Appendix 1).
Discussion
Impact on students
The aim of this study was to consider the impact of using MR in nursing education and in particular anatomy studies. As observed in the results section in Figure 5, participants in the MR condition reported increased understanding and motivation for the subject after they used the technology, alongside their existing course materials and practical experience tutorials.
However, the impacts of MR go beyond the student experience with opportunities for innovation in format and location of learning, and identifying improvements in the evolution of MR product design. Toi Ohomai / Te Pūkenga has run several hundred HoloLens sessions with students and teachers since 2020. The research with the 2021 first-year Nursing students provided an opportunity for systematic empirical data collection to supplement observations and implementation experience. That experience highlights some constraints with the HoloLens in the integration to campus networks, and on user ID access. The ecosystem of hardware and software (as of August 2022) needs to work together more seamlessly if the use of HoloLens in vocational or formal education settings is to become more widespread.
Impact on organisations
Technology provides the opportunity to change how an organisation operates, with the degree of change or impact depending on how it is used. Technology can be used to just bring efficiencies to existing operations with existing processes or it can be used to innovate and ultimately disrupt the status quo. These same strategic objectives and ambitions to disrupt and innovate are set out in the Toi Ohomai Teaching and Learning Strategy 2020–23:
Interventionist and disruptive change and innovation is needed to our teaching, learning and delivery if we are to be a viable and sustainable organisation able to meet future changes, growth and the adaptations that will certainly come with the advent of [the national Review of Vocational Education]. (Toi Ohomai Institute of Technology 2020)
Impact on learning design
Introducing HoloLens into ITP programmes provides the opportunity to contribute to these strategic objectives by creating significant change in how students learn in place and form, and what they can learn, shifting from consuming knowledge, to experiencing and creating knowledge. The Microsoft ecosystem provides clients with the capability of authoring MR training packages that can be delivered on campus or work-site environments. This de-couples training from a specific place or campus and opens the potential for less reliance on text-based publications as a student resource. However, there remains work to integrate these standalone learning experiences so accessing the technology is not a distraction from the learning experience itself.
Impact on collaboration
There are opportunities for wider collaboration in programme delivery at a consistent standard with the inclusion of MR in vocational education. Three of the core objectives for Aotearoa New Zealand’s largest tertiary education provider are to provide equitable and nationally consistent outcomes for all regions, and become a ‘provider driven by innovation, collaboration and teaching excellence’. The national ITP also aims to ‘partner with employers to deliver relevant work-integrated education and training that meets skills needs’ (Te Pūkenga n.d.).
Implementation journey
When considering adopting the HoloLens, the authors assessed it using the New Zealand Ministry of Education’s SAMR model (Ministry of Education, n.d.). We considered its alignment with the institute’s strategic objectives to create innovation in education delivery and contribute to student success. Table 2 presents the outcome of this decision-making process.
For the implementation of HoloLens the authors used the framework developed by Tyler-Smith et al. (cited in Marshall and Shepherd 2016, p. 10) with Capability building for teachers and support staff, and then Evaluation phases in 2021, including the results of this study.
At the end of 2021 the Evaluation stage of implementation was concluded, building a sustainable delivery model. Subsequently, the authors are undertaking testing of HoloLens MR authoring software, using MS Guides application (Microsoft Corporation 2022c) for training packages on campus. The next step on the implementation journey would be external Collaboration for consistency where MR applications are developed alongside other ITPs and employers for student learning outside of campus classrooms in workplace settings.
Constraints
Despite the clear evidence of the positive impact on student engagement, some technical limitations or constraints need to be overcome if MR is to be more broadly adopted as a new mode of learning. A seamless student journey from picking up the hardware to accessing the content needs to be designed so that adoption can move easily beyond professional users and early adopter student cohorts.
Being part of the Microsoft product suite, HoloLens headsets easily connected to Toi Ohomai / Te Pūkenga Azure AD and Business Intelligence applications, providing a well-supported technology platform to build employment-relevant projects within and beyond the campus. The technology was used on campus, and based on the authors experience of the configuration and student support required, we do not see this as suitable for distance learning environments with the HoloLens to date focused on enterprise and business applications.
Open interfaces and security
Independent software vendors and content developers are building learning content for the full capabilities of the HoloLens’s range of sensors and interactivity with peer-to-peer video and audio communication and remote-control capabilities. However, these capabilities and attributes of the HoloLens with its portability, flexibility and independent wearable computing can be at odds with enterprise IT security policies and controls. For education and corporate environments, the challenge is taking the innovation of a MR platform that can create content such as HoloLens, and integrating that capability and personalisation seamlessly within an organisation’s network.
User authentication
Our aim was to provide a personalised experience for students by utilising existing unique student IDs to differentiate their experience and measure results. The current designs in the HoloLens headset use a keyboard to enter a user ID and login, or you can choose a PIN and Iris biometric after registering the individual.
At the start of a student’s session, tapping away at a virtual keyboard is a poor user experience (UX) and a reminder of the links to enterprise desk-bound computing. The virtual keyboard metaphor to login contrasts with the remainder of the heads-up navigation conducted with hand, voice and eye movements.
The keyboard login also contrasts with the iris login experience for HoloLens, which is immediate and handsfree. It is this seamless personalised experience (with or without biometrics) that should be the goal in delivering the learning content for every user.
While teachers and support staff can use iris profiles, there is not enough capacity in the headsets for campus-wide student use, as it is limited to 64 iris profiles per headset, (as of August 2022) despite being connected to the organisation’s Azure (Profile) Directory network, and only 10 iris identities per headset are recommended (Microsoft Corporation 2022b). Beyond the hardware logins are the typical subscription content logins needed to get into licensed applications at the learning content or app stage. We found PINs are the best compromise between quick login and scalable use in the current format. The challenge is for both hardware vendors and content developers to improve the UX, balancing their commercial needs for controlling access without adding to a student’s cognitive load navigating to content (Buchner, Buntins, and Kerres 2022).
An anonymous or ‘kiosk’ mode configured in HoloLens profiles can allow access for anyone, but this reduces the value of personalised learning that the platform offers. An anonymous mode in the HoloLens situates the learner between passive consumption and the higher order learning afforded by individualised access. Unique logins and access give us the ability to customise learning for each student, report and measure progress, and leverage open network communications for collaboration.
The Mixed Reality (MR) design evolution
Currently users have to pre-select the type of experience they want to utilise, because VR is experienced in different hardware from MR or AR environments. Users have to first select a hardware vendor such as the Oculus brand, now Meta Platforms, Inc. for VR, or Microsoft with the HoloLens headset for MR, that then pre-determines the UX. These artificially imposed experience boundaries should be removed with newer hardware displays that vary their immersion with opacity, occlusion and content to suit the scenario or lesson, be it full VR, or MR across all of the Reality-Virtuality continuum (Milgram and Kishino 1994). At this stage of the MR design evolution the major technology companies such as Apple, Microsoft and Meta still largely control and mediate access to these separate worlds. From a user or student-centric view, the ideal experience would be a single display to deliver the appropriate content depending on the task or context, across the continuum that blends real and fully immersive environments. This would enable a seamless technology-enhanced learning continuum mirroring the technology integration for diverse learners, learner needs and digital literacies proposed by Aguayo, Eames and Cochrane (2020).
Early examples of this blended VR-AR mode are the pure MR vendors such as Magic Leap, which have pivoted from consumer to enterprise markets (Magic Leap n.d.), new ‘open’ platform entrants such as the Lynx-R1 (Lynx Mixed Reality n.d.), and potentially headsets from consumer technology companies such as Apple (Gurman 2022). In tandem with this hardware evolution are contributions to open-source software with toolkits (Microsoft Corporation 2022d) and Application Programming Interfaces (APIs) for interoperability (Khronos Group n.d.) to help build a consistent UX for navigating the immersive environment across platforms.
Conclusion
The MR technology continues to evolve and even the definitions of what is Augmented and MR need reviewing (Skarbez, Smith, and Whitton 2021) with opportunities for integrating MR into a learning continuum for diverse learning and learners (Aguayo, Eames, and Cochrane 2020). There is a significant increase in student motivation and engagement, with an improved student experience discussed in this review. The MR experiences also create significant change in how students learn, in place and format and what they can learn, shifting from consuming knowledge to higher order synthesis and evaluation (Bloom 1956). Given not only the positive impact on students, but also the commitment required by organisations to design, integrate and implement, then the priorities in research should be to determine to what extent MR contributes to students’ academic success. The results of this research can help course designers decide when to integrate MR experiences and support decision-making for education providers considering new modes of education and training for on-campus and work-based learning.
Acknowledgements
Thanks to the nursing students at Toi Ohomai / Te Pūkenga who participated in this study with the HoloLens 2. Thanks to current and former kaimahi including Kate Akers, Josh Burrell, Angela Cuff, Tina Dellabarca, Tepora Emery, Josh Hahunga, Jesse Henwood, Kieran Hewitson, Jacinda Hills, Philip Lopez, Marguerite Marsh, Tracy Narbey, Roseanne Sadd, Linda Shaw, George Stroud.
Thanks especially to our Participating Investigator Christine M. Cheyne, Research Office, Toi Ohomai Institute of Technology for assistance with the manuscript and preparation.
References
| Aguayo, C., Eames, C. & Cochrane, T. (2020) ‘A framework for mixed reality free-choice, self-determined learning’, Research in Learning Technology, vol. 28, pp. 1–19. doi: 10.25304/rlt.v28.2347 |
| BIM Holoview. (2022) Extended Reality for Construction, [online] Available at: https://www.bimholoview.com/ |
| Bloom, B. S. (1956) Taxonomy of Educational Objectives, Handbook I. The Cognitive Domain, David McKay Co Inc, New York, NY. |
| Bower, M., et al., (2014) ‘Augmented reality in education – cases, places and potentials’, Educational Media International, vol. 51, no. 1, pp. 1–15. doi: 10.1080/09523987.2014.889400 |
| Buchner, J., Buntins, K. & Kerres, M. (2022) ‘The impact of augmented reality on cognitive load and performance: a systematic review’, Journal of Computer Assisted Learning, vol. 38, no. 1, pp. 285–303. doi: 10.1111/jcal.12617 |
| Collins, E. & Ditzel, L. (2021) ‘Standardised holographic patients: an evaluation of their role in developing clinical reasoning skills’, Studies in Health Technology and Informatics, vol. 284, pp. 148–152. doi: 10.3233/shti210687 |
| Forrester Consulting. (2021) The Total Economic Impact™ of Mixed Reality Using Microsoft HoloLens 2. Business Benefits and Cost Savings Enabled by Mixed Reality Solutions Running on HoloLens 2 Devices, [online] Available at: https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RWQppN |
| Frost, J., Delaney, L. & Fitzgerald, R. (2020) ‘Exploring the application of mixed reality in nurse education’, BMJ Simulation and Technology Enhanced Learning, vol. 6, no. 4, pp. 214–219. doi: 10.1136/bmjstel-2019-000464 |
| GigXR. (2021) Extending Reality, [online] Available at: https://www.gigxr.com/ |
| Greenway, K., Butt, G. & Walthall, H. (2019) ‘What is a theory-practice gap? An exploration of the concept’, Nurse Education in Practice, vol. 34, pp. 1–6. doi: 10.1016/j.nepr.2018.10.005 |
| Gurman, M. (2022) Trademark Filings Suggest Apple May Be Securing ‘Reality’ Names for AR/VR Headset, Bloomberg, [online] Available at: https://www.bloomberg.com/news/articles/2022-08-28/what-will-apple-call-its-ar-vr-headsets-reality-one-reality-pro-names-emerge |
| Haami, B. (2019) Bringing Culture into Care. A Biography of Amohaere Tangitu, Huia Publishers, Wellington. |
| Hickey, H. & Wilson, D. (2017) ‘Whānau hauā: reframing disability from an indigenous perspective’, MAI Journal. A New Zealand Journal of Indigenous Scholarship, vol. 6, no. 1, pp. 82–94. doi: 10.20507/MAIJournal.2017.6.1.7 |
| Huang, Y., Shakya, S. & Odeleye, T. (2019) ‘Comparing the functionality between virtual reality and mixed reality for architecture and construction uses’, Journal of Civil Engineering and Architecture, vol. 13, no. 1, pp. 409–414. doi: 10.17265/1934-7359/2019.07.001 |
| Kay, R. H. & Knaack, L. (2009) ‘Assessing learning, quality and engagement in learning objects: the Learning Object Evaluation Scale for Students (LOES-S)’, Educational Technology Research and Development, vol. 57, no. 2, pp. 147–168. doi: 10.1007/s11423-008-9094-5 |
| Khronos Group (n.d.) OpenXR, [online] Available at: https://www.khronos.org/openxr/ |
| Kim, K.-J., Choi, M.-J. & Kim, K.-J. (2021) ‘Effects of nursing simulation using mixed reality: a scoping review’, Healthcare (Basel, Switzerland), vol. 9, no. 8, p. 947. doi: 10.3390/healthcare9080947 |
| Knowles, M. S., Holton, E. F. & Swanson, R. A. (2015) The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development, Routledge, [online] Available at: https://www.taylorfrancis.com/books/mono/10.4324/9781315816951/adult-learner-malcolm-knowles-elwood-holton-iii-richard-swanson |
| Lynx Mixed Reality (n.d.) Lynx R-1 Headset, [online] Available at: https://www.lynx-r.com/ |
| Magic Leap. (n.d.) Magic Leap 2: The Most Immersive AR Headset for Enterprise, [online] Available at: https://www.magicleap.com/solutions |
| Marshall, S. & Shepherd, D. (2016) e-Learning in Tertiary Education: Highlights from Ako Aotearoa Supported Research, [online] Available at: https://ako.ac.nz/assets/Reports/Synthesis-reports/SYNTHESIS-REPORT-e-Learning-in-tertiary-Education-Highlights-from-Ako-Aotearoa-supported-research.pdf |
| Matear, S. (2021) Good Practice Assessment of Online Teaching in Universities in Aotearoa New Zealand during the COVID-19 Pandemic and Lessons for the Future, Academic Quality Agency, [online] Available at: https://www.aqa.ac.nz/sites/all/files/GPA%20Report%20FINAL.pdf |
| McClintock, K. & McClintock, R. (2018) Tuku Iho, Tuku Iho, Culture in Māori Health Service Provision, Te Rau Matatini, [online] Available at: https://terauora.com/wp-content/uploads/2019/05/Tuku-Iho-Tuku-Iho.pdf |
| Microsoft Corporation. (2022a) About HoloLens 2, [online] Available at: https://docs.microsoft.com/en-us/hololens/hololens2-hardware |
| Microsoft Corporation. (2022b) Manage User Identity and Login for HoloLens, [online] Available at: https://docs.microsoft.com/en-us/hololens/hololens-identity |
| Microsoft Corporation. (2022c) Microsoft Mixed Reality / AR Guides | Microsoft Dynamics 365, [online] Available at: https://dynamics.microsoft.com/en-nz/mixed-reality/guides/ |
| Microsoft Corporation. (2022d) What Is the Mixed Reality Toolkit 2? [online] Available at: https://docs.microsoft.com/en-gb/windows/mixed-reality/mrtk-unity/mrtk2/?view=mrtkunity-2022-05 |
| Milgram, P. & Kishino, F. (1994) ‘A taxonomy of mixed reality visual displays’, IEICE Transactions on Information and Systems, vol. 77, pp. 1321–1329. |
| Ministry of Education. (n.d.) SAMR Model from Te Kete Ipurangi, by the Ministry of Education, [online] Available at: https://elearning.tki.org.nz/Professional-learning/Teacher-inquiry/SAMR-model |
| Nanayakkara, C. & Whiddett, R. (2005) A Model of User Acceptance of E-learning Technologies: A Case Study of a Polytechnic in New Zealand, International Conference on Information Systems Technology and its Application (ISTA’2005), Palmerston North. |
| New Zealand Nurses Organisation. (2019) Guideline – Code of Ethics 2019, [online] Available at: https://www.nzno.org.nz/Portals/0/publications/Guideline%20-%20Code%20of%20Ethics%202019.pdf |
| Osorto Carrasco, M. D. & Chen, P.-H. (2021) ‘Application of mixed reality for improving architectural design comprehension effectiveness’, Automation in Construction, vol. 126, Article 103677. doi: 10.1016/j.autcon.2021.103677 |
| Papakostas, C., Troussas, C., Krouska, A. & Sgouropoulou, C. (2021) ‘Exploration of augmented reality in spatial abilities training: a systematic literature review for the last decade’, Informatics in Education, vol. 20, no. 1, pp. 107–130. doi: 10.15388/infedu.2021.06 |
| Papps, E. & Ramsden, I. (1996) ‘Cultural safety in nursing: the New Zealand experience’, International Journal for Quality in Health Care, vol. 8, no. 5, pp. 491–497. doi: 10.1093/intqhc/8.5.491 |
| Putland, G. (2021) GIGXR Helps Nursing Programs Teach Clinical Reasoning, Meet Clinical Simulation Requirements, Healthy Simulation, [online] Available at: https://www.healthysimulation.com/31948/gigxr-mixed-reality/ |
| Ramsden, I. (1993) ‘Kawa whakaruruhau. Cultural safety in nursing education in Aotearoa (New Zealand)’, Nursing Praxis in New Zealand, vol. 8, no. 3, pp. 4–10. doi: 10.36951/NgPxNZ.1993.009 |
| Richardson, F. & Carryer, J. (2005) ‘Teaching cultural safety in a New Zealand Nursing Education Program’, Journal of Nursing Education, vol. 44, no. 5, pp. 201–208. doi: 10.3928/01484834-20050501-02 |
| Scully, N. J. (2011) ‘The theory-practice gap and skill acquisition: an issue for nursing education’, Collegian, vol. 18, no. 2, pp. 93–98. doi: 10.1016/j.colegn.2010.04.002 |
| Skarbez, R., Smith, M. & Whitton, M. C. (2021) ‘Revisiting Milgram and Kishino’s reality-virtuality continuum’, Frontiers in Virtual Reality, vol. 2, Article 64997. doi: 10.3389/frvir.2021.647997 |
| Slater, M. (2018) ‘Immersion and the illusion of presence in virtual reality’, British Journal of Psychology, vol. 109, pp. 431–433. doi: 10.1111/bjop.12305 |
| Te Pūkenga. (n.d.) About Te Pūkenga – New Zealand Institute of Skills and Technology, [online] Available at: https://xn--tepkenga-szb.ac.nz/about-us/ |
| Toi Ohomai Institute of Technology. (2020) Toi Ohomai 2020 – 2023 Teaching and Learning Strategy, Toi Ohomai Institute of Technology, Tauranga. |
| Wunder, L., et al., (2020) ‘Fire in the operating room: use of mixed reality simulation with nurse anesthesia students’, Informatics, vol. 7, no. 4, Article 7040040. doi: 10.3390/INFORMATICS7040040 |
Appendix 1
Open-ended question responses June 2021
Question: Open Feedback on anything you experienced during your session, good or bad, or is there anything we can improve?
| Feedback in survey | Classification |
| 1. Buttons were hard to press | Negative |
| 2. Hurts eyes | Negative |
| 3. The selection (baby shark) part was frustrating personally | Negative |
| 4. Aiding students individually through it rather than giving us something new that’s hard to use with no help. It was very claustrophobic and trapping with no help made something really cool be really awful and scary | Negative |
| 5. Better explain action of how to use | Neutral |
| 6. I think the equipment would be better if (the room) was blacked out apart from the virtual things. | Neutral |
| 7. I found it a little slow and confusing but I am sure with more use, I will work it out | Neutral |
| 8. We weren’t given much information or instructions as to how to use it and fully utilise the technology. Otherwise, it was a fun experience. | Positive |
| 9. Awesome | Positive |
| 10. Such a respectful tutorial by the presenters. I really enjoyed the experience of viewing human anatomy as if it was right in from of me. | Positive |
| 11. It was cool, seeing it like real. | Positive |
| 12. Reproductive system would be cool | Positive |
| 13. Interesting | Positive |
| 14. It was so cool seeing everything like that and easy to interact with | Positive |
| 15. Once I got the hang of it it was very easy to use. Accurate instructions from John also. | Positive |
| 16. I would like to see a female model, so I can view the female reproductive system. Otherwise really fun | Positive |
| 17. It was great to see all the different aspects of the anatomy in this way. I felt truly amazed by the model. | Positive |
| 18. I think this would be an incredibly beneficial resource | Positive |
| 19. Was a very exciting experience for me as it was the first time I have ever used a virtual reality. | Positive |
| 20. I thought it was really well done, easy to use and had a lot of help to further understand. | Positive |
| 21. Amazing experience | Positive |
Survey Questions
Your feedback on the HoloLens session