ORIGINAL RESEARCH ARTICLE

Analysing construction student experiences of mobile mixed reality enhanced learning in virtual and augmented reality environments

Nikolche Vasilevski and James Birt^*

Faculty of Society and Design, Bond University, Gold Coast, Australia

(Received: 30 August 2019; Revised: 12 November 2019; Accepted: 16 November 2019; Published: 16 January 2020)

Abstract

Mixed reality (MR) and mobile visualisation methods have been identified as important technologies that could reimagine spatial information delivery and enhance higher education practice. However, there is limited research on the impact of mobile MR (MMR) within construction education and improvement of the learners’ experience. With new building information modelling (BIM) workflows being adopted within the architecture, engineering and construction industry, innovative MMR pedagogical delivery methods should be explored to enhance this information-rich spatial technology workflow. This paper outlines qualitative results derived through thematic analysis of learner reflections from two technology-enhanced lessons involving a lecture and a hands-on workshop focussed on MMR-BIM delivered within postgraduate construction education. Seventy participants across the two lessons recruited from an Australian university participated to answer the research question: ‘Does applied mobile mixed reality create an enhanced learning environment for students?’ The results of the analysis suggest that using MMR-BIM can result in an enhanced learning environment that facilitates unique learning experiences, engagement and motivation. However, the study outcome suggests that to understand the processes leading to these learning aspects, further empirical research on the topic is required.

Keywords: mobile learning; AEC education; building information modelling; BIM; student engagement

This paper is part of the special collection Mobile Mixed Reality Enhanced Learning edited by Thom Cochrane, James Birt, Helen Farley, Vickel Narayan and Fiona Smart. More papers from this collection can be found here.

^*Corresponding author. Email: jbirt@bond.edu.au

Research in Learning Technology 2020. © 2020 N. Vasilevski and J. Birt. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Citation: Research in Learning Technology 2020, 28: 2329 - http://dx.doi.org/10.25304/rlt.v28.2329

Introduction

With growing project scale and complexity, the architecture, engineering and construction (AEC) industry is exploring innovative solutions to augment communication and information visualisation to enhance workflows and project management (Chan et al. 2018). Building information modelling (BIM) has been identified as a tool to improve the efficiency of design communication and collaboration among project participants, with many countries implementing governance measures to integrate BIM into their public projects (Antwi-Afari et al. 2018). Given this, there has been a rapid shift in AEC postgraduate education to integrate BIM into course content, which has led to integration difficulties with traditional didactic course delivery (Puolitaival and Forsythe 2016).

Today’s students are characterised as more oriented towards visual media than previous generations, preferring to learn visually by doing rather than by listening or reading (Thompson 2013). This coupled with ubiquitous mobile device ownership has led many educational institutions to explore a smartphone-enabled bring your own device (BYOD) approach to mobile enhanced learning (Crompton and Burke 2018). However, due to the relatively emerging nature of mobile technologies, the implications of these tools, especially within postgraduate education, are still largely being explored.

Mixed reality (MR) is a rapidly developing technology, first proposed by Milgram and Kishino (1994). It enables physical world enhancement through augmented reality (AR) and digital world interactivity through virtual reality (VR). More recently, Adria (2019) explored MR from the perspective of creating social environments, with Gugenheimer et al. (2019) identifying considerations for improving shared spaces using head-mounted displays (HMD). New definitions of MR have been proposed, including using immersion, interactions and information (Parveau and Adda 2018) and frameworks built on a number of environments, number of users, level of immersion, level of virtuality, degree of interaction, and input and output specific to senses (Speicher, Hall, and Nebeling 2019).

Combined with smartphones, mobile MR (MMR) enables the design of new and innovative BYOD learning environments that enable experiential learning in new dimensions, leading to more authentic experiences enhancing student learning outcomes, motivation and engagement (Cochrane, Smart, and Narayan 2018). This learning innovation is captured in the latest Educause Higher Education Horizon Report (Alexander et al. 2019) as a wicked challenge, requiring innovative research development and rethinking of higher education teaching practice.

There is a growing belief that active and immersive learning engagement and the use of emerging technology, especially in the form of MMR (Birt, Moore, and Cowling 2017; Birt et al. 2018; Cochrane, Smart, and Narayan 2018) and BIM (Puolitaival and Forsythe 2016), present an increasing opportunity to enhance higher education pedagogy and AEC design workflow (Birt and Cowling 2018). This study evaluates student experiences of BIM delivered through MMR enhanced learning, using a combination of mobile VR and AR, to postgraduate construction students at an Australian university. Specifically, the learners were given a lecture on innovation within the construction industry focussing on the applied use of VR, AR and artificial intelligence (AI) delivered using innovative MR technology and BIM models, and then split into small learning groups for hands-on experiential learning with mobile VR and AR technology.

The aim is to extend existing course learning outcomes of professional skill development in real-world environments, by focussing on strategic and analytic thinking capabilities using situated authentic learning, self-analysis and reflective learning skills (Wylie and Chi 2014) combined with emerging professional practices in BIM, VR and AR communication. This is achieved by analysing reflective student essays documenting the learning experience and addressing the research question: ‘Does applied mobile mixed reality create an enhanced learning environment for students?’

Background

Introduced in the early 2000s, BIM enables advanced forms of visualisation modelling through 3D models combined with object-related quantitative and qualitative data (Sacks et al. 2018). Used throughout the whole building lifecycle, BIM enables visualisation, scheduling, communication and collaboration among project participants to rectify design errors and implement changes before a project is developed. This combination of spatial modelling and information data enables optimised process and communication workflows (Miettinen and Paavola 2014) supporting the ever-changing standards and critical success factors (Antwi-Afari et al. 2018) around BIM compliance within the AEC industry (Chan et al. 2018).

With the massive information and data in AEC projects growing in scale and complexity, this can become overwhelming for teams if not managed correctly. Given this, there has been a rapid shift in the AEC education practice to integrate BIM into course content, which has led to difficulties in integration. Puolitaival and Forsythe (2016) identified these key difficulties in finding appropriate resources to balance theory, practice, process and emerging technologies, with emphasis placed on the best learning environment for students to facilitate collaboration and experientially driven approaches. It was noted that most courses use traditional didactic methods only, but a balance is required between traditional and emerging educational methods.

This challenge of balance is compounded with the learner’s expectation that they can participate and interact with their learning environment, leading to improved engagement through sensory-rich, experimental activities (either physical or virtual) (Jones et al. 2010) and higher expectations for input opportunities with individualised resources for productive and effective student outcomes (Sadler-Smith and Smith 2004). O’Brien and Toms (2008) suggested that in order for the learning to take place, learners should be engaged with the multimedia presentation and the technology within the learning environment. Engagement, referred to as a subset of the learning experience, is described by characteristics such as challenge, positive affect, endurability, aesthetics, sensory appeal, attention, feedback, variety/novelty, interactivity and perceived user control.

Today’s students are characterised as more oriented towards visual media than previous generations, preferring visual learning rather than reading, listening and recalling (Thompson 2013). This has shifted learning away from traditional face-to-face, didactic lectures and tutorials to self-direction, collaborative peer learning, technology-enhanced teaching and learning through multiple technology delivery modes (Birt and Cowling 2018) and multimedia coding methods (Clark and Mayer 2016).

This shift coupled with increased ubiquitous ownership of mobile devices has led many educational institutions to explore a smartphone-enabled BYOD approach to mobile enhanced learning (Crompton and Burke 2018). Mobile technology is disruptive, changing the world; therefore, it is imperative that pedagogy and teachers adapt and realise both the benefits and risks associated with this highly available form of learning technology (Akçayır and Akçayır 2017). However, mobile technologies are still emerging, with relatively little knowledge available regarding the immersive use of mobile technology in higher education settings. The implications of mobile tools, especially within postgraduate education, are still largely being explored (Crompton and Burke 2018).

Experiential education has been recognised as engaging for the holistic and supported pedagogical method used in all stages of learning (Kolb 2014). Schott and Marshall (2018) suggested that we can best understand experiential education as a philosophy that accepts the idea that in order to comprehend the world, learners need to interact with it. If immersive learning environments are achievable using technology, then the challenge is achieving high levels of interactivity as highlighted by the recent MR definitions by Parveau and Adda (2018) and Speicher, Hall and Nebeling (2019). Due to a combination of factors, such as constrained budgets, increasing concern about liability issues and increased workload pressures on the academic staff, we need to better understand the introduction of situated immersive learning environments (Bower, Lee, and Dalgarno 2017; Dalgarno and Lee 2010; Schott and Marshall 2018), while also understanding that ever more sophisticated opportunities are able to fill the gap and address the emerging issues.

MR is a rapidly developing technology, first proposed by Milgram and Kishino (1994), which enables physical world enhancement through AR and digital world interactivity through VR. Even though a real learning immersive environment will result in the widest range of cognitive, affective and skilled outcomes, there are numerous other reasons as to why one would use an MR immersive environment, such as cost, time, the logistics involved, staffing, ability to visit hostile or non-existent environments, repeating experiences and feedback in context (Birt, Moore, and Cowling 2017; Birt et al. 2018; Cochrane, Smart, and Narayan 2018; Speicher, Hall, and Nebeling 2019).

Dalgarno and Lee (2010) highlighted that virtual environments with increased immersion, fidelity and higher active learner participation have the potential to increase the quality of the learning experience. Fowler (2015) argued that immersion may emerge as a product of complex interactions within the virtual learning environments. This is further examined by Bower, Lee and Dalgarno (2017) by highlighting the importance of discussion and interaction within collaborative mixed reality or blended reality environments with learners both remote and face-to-face feeling increased effective communication, collaboration and copresence. Within AR, numerous studies are exploring applied AR with Quintero et al. (2019) providing a systematic review over the past 10 years highlighting the positive inclusion factors for integrating AR in education. This built on previous work of Bower et al. (2014) that discussed the pedagogical affordances of the technology and connections to thinking, creativity and analysis. There are however technical issues to overcome with virtual learning environments such as cognitive load and the sense of self through avatar personalisation within virtual environments (Steed et al. 2016). Other issues were associated with mixed reality hardware itself, including simulation sickness (Dziuda et al. 2014), specifically with mobile devices often lacking the six degrees of freedom, the latency of the scene refresh due to the relatively low processing power of mobile devices and other technical concerns (Akçayır and Akçayır 2017). There are also issues around the HMD and its impact on building shared social and immersive environments (Gugenheimer et al. 2019; Schott and Marshall 2018).

The 2019 Educause Higher Education Horizon Report (Alexander et al. 2019) captures this as a wicked challenge impeding higher education, requiring innovative research development and rethinking of higher education teaching practice. Specifically, new MMR technologies (comprising both mobile phone and mixed reality affordances) are identified as important technologies and drivers for rethinking higher education practice and learner engagement (Aguayo, Cochrane and Narayan 2017; Cochrane, Smart, and Narayan 2018). Dalgarno and Lee (2010) and Cochrane, Smart and Narayan (2018) also discussed the combination of virtual environment and games, and the increased engagement and the motivation of the learners.

Within the AEC industry, technology-enhanced design workflows are being explored through the use of MMR showing positive usability results of smartphone integration (Birt, Moore, and Cowling 2017; Birt et al. 2018). Birt and Cowling (2018) highlighted the effective use of smartphones in visualising both VR and AR with comparable results to expensive head-mounted solutions such as the HTC VIVE and HoloLens. This is further augmented with service enhancement through gamification and micro-location technologies, such as iBeacons (Vasilevski, Brand, and Birt 2018), implemented within AEC-MMR approaches (Vasilevski and Birt 2019).

However, currently, there is limited research into the effect that MMR workflows have within BIM education and how this can enhance and optimise the AEC industry. More detailed research is required with regard to the methods of coordination, communication, data management, analysis, simulation, productivity, design methods and facilities management, especially as it relates to BIM in construction education (Puolitaival and Forsythe 2016) and integration for learners.

Research design

This study outlines the results of smartphone-enabled MMR learning lesson, delivered through VR and AR experiences to postgraduate construction students at an Australian university. The aim of the lesson was to extend the existing construction course learning outcomes of professional skill development in real-world environments, as highlighted by Puolitaival and Forsythe (2016), by focussing on strategic and analytic thinking capabilities using situated authentic learning (Schott and Marshall 2018), self-analysis and reflective learning skills (Wylie and Chi 2014), combined with emerging professional practices in BIM, VR and AR communication (Birt and Cowling 2018). It is important to note that students that attended the lesson were referred to as students, and the students that submitted the reflective essays were referred to as participants. Two rounds of the study were run in the first (L1) and the second (L2) trimester of 2019 involving 90 students (L1: n=45, L2: n=45). Ethics clearance was granted by the institution’s ethics committee prior to running the lesson.

The student cultural backgrounds were diverse, including students from China (64), unknown countries (11), Australia (5), India (5), the United States of America (3), Norway (1) and South Africa (1), with a major presence of Chinese international students (71.1%). Regarding gender composition of the students, 40 were male and 50 were female students. The mean age was 27.5 years, with numbers distributed across three age groups, 18–24 (26), 25–34 (60), and 35–44 years (4), with the 25–34 years age group being the mode age, which represented 69% of the total students in the lessons. This reflects the broader postgraduate student population of the construction programme at the authors’ university.

The students self-evaluated their ‘competence when using technology’ on a 7-point Likert item scale with the following distribution numbers: Unknown (11), extremely incompetent (0), moderately incompetent (1), slightly incompetent (2), neither competent nor incompetent (9), slightly competent (16), moderately competent (35), extremely competent (16), resulting in 74.5% of the students being slightly to extremely competent when using technology. The primary mobile smartphone device platform was iOS (73.4%), followed by Android (26.6%).

The structure of the delivered lessons was a lecture followed by hands-on experiential learning. Both L1 and L2 lectures lasted 60 min and were on the impact of AI, VR and AR technology within the construction industry (see Figure 1). We gave a live presentation of HTC VIVE and Microsoft HoloLens technology as they applied to the construction industry and BIM visualisation reimaging the didactic delivery method (Puolitaival and Forsythe 2016).

Figure 1. Photos highlighting the lecture representing the AR and VR demonstration.

Following the lecture, the students were split into groups of 4–6 and were asked to take part in two hands-on learning activities using mobile VR (see Figures 2 and 3) and mobile AR (see Figure 4). Learners took part in the activities by exploring virtual BIM models and real-world locations enhanced through augmented overlays. This occurred simultaneously with one group partaking in the VR and the other in the AR experience. Students that were not part of the groups on hand were asked to work on their existing projects. This was due to the limited number of available mobile devices and headsets for the VR hands-on sessions.

Figure 2. Hands-on photos of the single user VR experience.

Figure 3. Hands-on photos of the multi-user VR experience.

Figure 4. Hands-on photos of the AR experience representing the tour and the gamified elements.

For the VR technology hands-on sessions, we asked the students to participate in two distinct mobile VR-simulated environment experiences that contained a fictitious BIM model of a pavilion constructed in Autodesk REVIT (modelling tool) and visualised using Unity3D (game simulation tool). The choice of using this model and the development environment was based on the results of Birt and Cowling (2018). In their study, they highlighted the positive usability affordances of using mobile smartphone technology for both existing real-world built environments and fictitious environments. Students were provided with a 5-min onboarding about the use of the technology, including attaching, navigation and methods of communication and the differences between the two experiences.

For the students, we provided Samsung Galaxy S8 smartphones running Android Pie and Samsung Gear VR headsets. The choice of smartphone and headset was based on technology availability and a desire to offer the same experience for all learners. The first experience consisted of a single user environment in the virtual BIM pavilion, and learners were given 10 min of self-exploration. The students were provided with their own self avatar camera representing a neutral blue figure and able to navigate the environment and observe simulation of the transition of the real-time lighting across 24 h (see Figure 2).

For the second experience, we situated the students in the same BIM environment with the same self-avatar, navigation and light system as the first experience. However, this time they had multiuser connectivity and voice chat enabled using the Photon Networking plugin for Unity3D. The plugin allowed for a freely available (maximum 20 simultaneous users) external, scalable cloud server accessible via Wi-Fi Internet. The students were able to see not only their own self avatar in keeping with Steed et al. (2016) but also of their peers in the virtual environment; the learners were given 10 min of collaborative learning. This setup facilitated enhanced communication between the students. The setup also provided the affordances, such as agency, perception and peer learning, which are associated with collaborative multiuser virtual environments (Bower, Lee, and Dalgarno 2017) (see Figure 3). In the remaining time, learners were able to use the HTC VIVE to explore higher fidelity forms of VR.

For the mobile AR technology hands-on, we used a mobile AR-enabled app, which was in the final phase of development (see Figure 4). Corrigan Walk Tour app first proposed in Vasilevski and Birt (2019) was used as an AR guide for the indigenous artworks collection at an Australian higher education institution. It included badges and hidden features as gamification elements offered in the form of a treasure hunt gamification scenario, which was highlighted as an element of micro-location implementations (Vasilevski, Brand, and Birt 2018). As the tour was taken within a built environment, enhancing the students’ knowledge beyond the specifics of the construction management, analysis and simulation were the main goals of the hands-on session, facilitated by inclusion of the factors, such as the place as a service system and its intangible facet, the sense of place, as the psychological bond we have with the place.

The learners were given 35 min in L1 and 45 min in L2 to complete the learning exercise. The difference between the versions of the app used in L1 and L2 were the number of indigenous paintings available for the tour and the inclusion of iBeacon Internet of things devices to support micro-location affording the students mapping and location awareness. This inclusion was to support the development of the Corrigan Walk Tour app and built environment changes at the university campus, but it did not significantly impact the results of this study as these features were not assessed by the participants and were outside the scope, mentioned only for publication transparency. The students’ experience with the app started with extensive onboarding. The learners were given a 5-min onboarding and informed on how to use the app, by a walkthrough of how the app flow worked and showing how to use the app optimally. Presented were the objectives and the goal of the app use but nothing about the artworks or indigenous culture was discussed at that point, to enable the students to uncover and interpret this information individually through the app.

L1 students were provided with Android Samsung Galaxy devices (S6, S7, S8) and L2 students could use their own devices, including iOS, by downloading the beta version of the app from the mobile phone app-stores. Due to the app’s multiplatform design using Unity3D, and equivalent appearance and functionality on both Android and iPhone devices, there was no added variance within the results. We provided over-ear headphones to the students to prevent sound pollution of the building and interference with any activities at the location.

For both L1 and L2, as a part of their assessment process, students were asked by their course convenor to complete a reflective essay (between 300 and 500 words) within 2 weeks after the session. The essay was a reflection on linking their lesson experience to the construction industry and made upgrades associated with their professional portfolio of work aligning with the learning outcomes of the course and was made available to the researchers to inform the study results.

Methodology

A qualitative research methodology was used in this study. We used thematic analysis (Braun and Clarke 2006), which saw emergent themes in the written reflective essays from the participants. As noted in the previous section, students are those that completed the lesson but participants were those that submitted reflective essays and part of the thematic analysis. We analysed 70 reflective essays submitted by the participants across the two iterations of the lesson. The coding included multiple themes that emerged from the data. These have focussed on issues such as sense of place, technology, app development and learning. For the purpose of this paper, we present three categories that arose in the data that focus on the mixed reality learning and the experiences with the mobile AR and VR technologies.

We categorised similar participant ideas under categories based upon our research question, ‘Does applied mobile mixed reality create an enhanced learning environment for students?’ We concurrently categorised emerging themes and sub-themes. We also noted if any of the students required additional help or guidance to complete the tasks.

Results

Below we present the emerging themes of the thematic analysis performed on the submitted essays representing participants’ reflections on the experiences they had with the mobile mixed reality technology. We have categorised the themes into three major categories: Learning experience, VR experience and AR experience. There is some overlapping between some themes within some categories and between categories. However, all of the coded themes express distinctive uniqueness and weight. Participants specifically state or imply these aspects in their reflections. Some of the quotations were corrected for grammatical errors; however, the meaning that was intended was not modified in any way. It is important to note that as described in the Methodology – demographic information, most of the students (71.1%) came from China with English as a second language and had some difficulties with the reflective essay in expressing their learning outcomes and reflective practice. A pie chart of the categories compared by the number of coding references is presented in Figure 5.

Figure 5. Chart of the categories and the themes compared by the number of coding references.

Learning category

Learning is the major category that featured in 71% of the reflections. This category contains four themes, namely, learning engagement, learning motivation, learning about AR/VR technologies and first-time experiences with the technologies (see Figure 6). Each theme and category are represented by a circle whose size represents the percentage of participants’ reflections where that theme emerged. The intermittent line circle represents 100%.

Figure 6. Visualisation of the learning category and the emerging themes.

The Learning engagement theme emerged in 57% of the reflections and relates to the engagement of the participants during the learning activities. Most of the participants found the activity to be very interesting, stating: ‘the session day was really interesting and gave me many ideas and hints in terms of new technologies in construction industry’ and ‘we have got some interesting activities about VR and AR including live augmented reality HoloLens demonstrations and virtual reality BIM walkthroughs’. Participants also talked about the enjoyment they had during the activity, for example: ‘I enjoyed the later AR activity so much…’, ‘My friends were all enjoying at that time’ and ‘This is an interesting activity, and we all enjoyed ourselves in this activity’. Moreover, many of the participants experienced fun, reflecting about it with comments like: ‘It’s such a fun experience to use the AR to visit the aboriginal paintings in the medical buildings’ and ‘As for AR, I have to say it is amazing. When I experience the gazebo, I [was] stuck [on] the tree, but it is fun’. Participants also talked about experiencing excitement, stating: ‘This week we experienced the VR and AR, which are interesting and exciting’. And, ‘It is really an exciting experience to use VR to feel what it is like to be in the proposed building…’. Some participants noted the influence of technology on the sensory appeal: ‘I noticed that both VR and AR technologies could alter our sense of reality, and they have lots of advantages in various ﬁelds in industry’ and ‘It was like dreaming and imagination’. The participants also wrote about having great experiences for the duration of the sessions, stating: ‘Experiencing the model at back side of architecture building with Virtual reality headset is amazing’ and ‘What I experienced in the session is outstanding’. Many of the participants were impressed with the ability to interact with the environment, and also between themselves: ‘But it still impresses me that I can interact with peers and study together…’ and ‘No matter how experienced people are, VR and AR give people the ability to “see” a project and interact with the environment’. One of the participants noted the importance of the feedback: ‘At the end of the tour, we provided several feedbacks in the survey, which reveals our thoughts and advice to the designer. In my view, this is really an effective way to interact with the audience of the work’.

The first-time experiences with the technologies theme emerged in 11% of the reflections and refers to the reflections from the participants that had no previous experiences with the used technologies in the terms of the first-time use of AR and/or VR technology. Out of the whole population, seven participants stated that they had no previous experience with AR or VR, and this was their first time to use these technologies: ‘That was my first time to experience the AR/VR glasses’ and ‘For the first time we have the opportunity to try the most edge-cutting Virtual Real[ity] and Augmented Reality technology’. Some participants also expressed content at being able to use AR and/or VR, by writing: ‘Since I am a tech-oriented people and have heard a lot about virtual-reality, I was so glad to have the chance to use this new construction technology in the class’ and ‘I have never used this type of apps before, so it is kind of interesting’. There was also a statement about the learning outcome from the first time use:

This activity introduced me [to] my first AR experience which I did not know the concept before, while this activity showed me that how the AR technology applied in daily life with a mobile phone instead of any high-tech equipment such as Google glasses, meaning it is easy to access and understand this technology.

The Learning motivation theme was coded in 7% of the reflections, and it refers to the motivation and inspiration to learn beyond the content introduced to the participants. All these participants expressed their inspiration for learning, stating that ‘This workshop is so inspiring for me’ and ‘We can understand the background and content of each painting, [the app] attracts people to learn more’. One participant specifically stated that: ‘For [the] learning purpose, it increases my learning interests’. Some participants stated the interest to further inquire new knowledge by talking about the impact of the new technologies, such as in this example: ‘It is just a beginning. To master skills of new technologies applied in construction, I need to learn more about operation, principles between virtual views and real works’.

Learning about AR/VR technologies theme emerged in 18% of the reflections and emphasised the importance to further learn more about AR and VR technologies and the benefit of the inquired knowledge. Most of the participants felt deeply the need to learn more about these technologies: ‘In this situation, we need to recognize that we need to spend time to learn these technologies and learn how to use them in our work to help us’ and ‘I think the most important is the ability to learn the technology efficiently for our participants to get involved in the construction field as soon as possible’. Some participants underline that learning about these technologies is not just proposed but a requirement if you want to stay relevant in the field, stating: ‘I believe that VR/AR will become the essential technology in the future, and I would like to follow on this technology and learn more about this technology’ and ‘therefore, we are required to learn and understand those [technologies] and use them to enhance our communication and improve audience engagement and outcomes. There are lots of advantages when applied them to construction industry’.

VR experience category

VR experience is the second largest category. It was coded in 57% of the references. This category refers to the themes that emerged in the reflections about the VR technology and the experiences that the participants had with it. It contains the following themes: VR Synchronous Communication, VR Interaction, VR Ability to change between day and night, VR Self-avatar, VR Simulation Sickness and VR Hardware Concerns (see Figure 7).

Figure 7. Visualisation of the VR experience category and the emerging themes.

Synchronous communication is the major theme in this category found in 34% of the reflections. This theme covers the participant opinions on the experience they had within the multi-user VR environment. Participants were impressed by the possibility to interact between each other, stating: ‘But it still impresses me that I can interact with peers and study together by ourselves control’ and ‘It was much fun than the single model because it created more communication between each other in the same model, and we can speak to each other by the microphone and see each other in the model’. Moreover, they recognise some benefits of VR multiuser communication. ‘Both AR and VR can help us to enhance communication amongst various project stakeholders prior to the construction work commence’ and ‘Effective communication is the essential part for any construction project. It makes me feel that it is convenient for clients, developers, and builders to communicate with each other’. The potential use of this type of communication in construction was also recognised by some of the participants, who stated: ‘I think these technologies provide a lot of benefits to the construction industry’ and

It was observed that while doing it individually it was less effective but when it was done in group it created a better understanding of collaborative use of this technology. As people working on the same project can discuss and observe the project details.

Two of the participants expressed their concerns about multiuser VR. The first one thought that being in the real world and not being able to see our real selves, we lose the ability to communicate as human beings:

I feel something are being stolen by technology. For instance, when we are in a multiuser mode, even we can talk to each other via headphone, we actually lost our ability to communicate with each other because everyone was masked with the cold equipment.

The other one is about the fast battery drain on the mobile device: ‘it seems that it takes [much] power [consumption] of phone after using once’.

The VR interaction theme was coded in 19% of the reflections, and it refers to the participant interaction or ability to interact with or within the VR environment. ‘We could also see other users and can reach to anywhere in that 3D world by some simple gestures’, commented one of the participants expressing the interaction with the virtual environment. Telepresence has also emerged in some reflections, evident in the comments like: ‘The first scene to enter the VR is to stand on the tree, the feeling is still relatively real…’. In the scene, we can walk around at will, jump up and down, and stand on the roof to see the panorama is also possible’. Many of the participants were amazed by the possibility of interaction with the environment, stating: ‘it is amazing to feel that we are interactive to the model and other group members’ and ‘I also can jump up or fall down by clicking buttons in my hand but I still can see the different views when my body jump up and down. That is exciting’. There was also amazement with the possibility to interact with their peers, as these two participants stated: ‘But it still impresses me that I can interact with peers and study together by ourselves’ and ‘… [I can] also interact with my groupmate in the virtual world under a multiple user mode’.

The VR Ability to change between the day and night themes is regarding the ability to switch between day and night within the virtual environment. It is a popular theme, and it was found in 17% of the reflections. Most of the participants noted the ability to switch between day and night, by commenting: ‘I was also able to change time and weather in the scene’ and ‘Moreover, we can take different screen captures of the environment and change the time of day’. Some participants wrote about the benefits for the users of having this possibility: ‘Besides, customers can stimulate the sunlight change during a day to observe the influence on the house’ and

It could change the daytime in the model, do the measurement, switch the layers, and check the specific information of the structure and structural elements, meanwhile, the experiencer could mark the error or any mistakes in the model through walking around the building.

However, some participants were very impressed with this feature, with comments like: ‘In my opinion, the function of changing day light is most impressive’ and ‘The very interesting thing is I can change the time of the day’.

The VR Self-avatar theme emerged in 7% of the reflections. It represents the codes where the participants mentioned or wrote about the avatars in the VR environment. Some of the participants simply reflected on their avatars, by writing: ‘When we put the headset on, we cannot see the world anymore, instead, we have tracking and can move our head and arms’. Some participants wrote about the simulated movement of the avatar, finding it interesting:

We can change our position in the game when we move our body in reality and I also can jump up or fall down by clicking buttons in my hand but I still can’t see the different views when my body jumps up and down.

and ‘Interestingly, we can “walk” in this VR environment by moving our feet in reality’. This referred to the ability for participants to rotate around and have the avatar simulate feet movement but due to the lack of 6 degrees of freedom, participants were unable to move their vertical or horizontal position. Most of the participants wrote about the ability to see their peers within the virtual environment, and not just hear them, by writing: ‘As a group, we can not only see ourselves stand inside the site of the 3D drawing, but also hear the sound of each other’ and ‘I also can see my colleagues were moving their bodies and change their position in the game’. One user commented that the avatars are not good looking, by writing: ‘we can also see other people around us, even though the body shape images are not beautiful’.

The VR Simulation Sickness theme is the first theme referring to a negative experience, and it relates to the simulation sickness within the VR environment. Fourteen per cent of the participants reported simulation sickness, which in most of the cases was dizziness, where the participants wrote the following: ‘This is kind of dizzying for me, and I can’t really get used to this 3D equipment’ and ‘However, for me, there is a big problem when using VR that it makes me feel dizzy’. Even though the participants felt dizzy, some of them still found the activity in VR interesting, by stating: ‘The [VR] activities, although I felt a little bit dizzy, the overall effects and video chatting were parts of this fascinating experience’ and ‘Although the experience with VR is very interesting, there are some uncomfortable things: The model is not so fine, this will cause 3D vertigo’.

The VR Hardware Concerns theme is the second theme that conveys negative experiences. It was coded in 11% of the reflections. It related to all negative experiences with the VR hardware. The main complaint was that of control. Participants found that control in the VR environment was very difficult to learn and requires onboarding process. They wrote the following: ‘I think the way of controlling is complicated with new users’ and ‘Need some training classes. This kind of technology needs some [instructors] to give clients a guideline about how to use this technology’. Some of the participants reported excessive heating of the devices: ‘…I found that most of our group members feel very hot after they were wearing the head set…’ and ‘Although I felt excited to take part in these two activities, I felt dizzy and hot’. The other complaints the participants had were with the fidelity of the mobile headsets: ‘image cannot move with our body movement and the quality of image is not good enough’, the comfort while wearing the headset: ‘The VR device is heavy and not comfortable for long-time wearing’ and the cost and the complexity of the VR setup: ‘The equipment is complex, expensive and limited in a small area’.

AR experience category

AR Experience is the third category, found in 31% of the reflections. This category embodies all emerging themes where the participants reflected on their experience with the use of AR technology during the lesson. It contains the following themes: AR Engagement, AR Gamification and AR Concerns (see Figure 8).

Figure 8. Visualisation of the VR experience category and the emerging themes.

The AR Engagement theme is the major theme in this category, and it was coded in 19% of the reflections. This theme relates to the engagement that the participants had during the AR hands-on session. Some participants noted the interaction with the augmented environment while scanning the paintings: ‘We could interact with the reality or environment to get more information and better experience in the building’ and ‘Open this app and focus the camera on the painting, and then the [narrators] voice about the story of [the painting] creation can be heard’. Some participants felt more connected with the environment through the use of the app, stated in the following: ‘The walk [AR] tour at medicine school enhances the connection between me and the place’. The awareness of the different meanings that the paintings hide is evident in some reflections, where it is stated that: ‘For example, we may have passed [by] these artworks every day before, and all of them have meanings that we don’t know’. Many of the participants showed interest about the indigenous paintings, by stating: ‘It hard to deny this really improve my interests and understanding of these displays’. One participant expressed the fun he had during the activity using AR, stating: ‘It’s such a fun experience to use the AR to visit the aboriginal paintings in the medical buildings’. ‘With the use of the simple device, which is our mobile phone, the artworks in the corridor became meaningful to the audience’, wrote one of the participants expressing the effect the AR app had on his perception of the indigenous paintings.

The AR Gamification theme featured in 14% of the reflections. It is coded in relation to the participant experience with the gamification of the AR app for the duration of the tour. While some of the participants simply recognised the use of the gamification, other participants found the app very interesting because it reminded them of a game, with comments like: ‘It is simple to operate but very interesting, because it is just like a game. Every time you found the mark on the picture or catch the hidden information, you can earn a reward’. Another participant stated that the combination of gaming elements and the real world inspired them to learn more: ‘Combing the gaming elements with the real-world objects made me feel eager to acquire more knowledge and stimulated my interests in discovering new things’. Some participants felt that the gamification had increased their motivation to learn more about the paintings, stating: ‘Combing the gaming elements with the real-world objects made me feel eager to acquire more knowledge and stimulated my interests in discovering new things’. Some of the participants noted the use of the mobile game PokémonGo (www.pokemongo.com) as an example of gamification: ‘Pokémon Go map, led the trend of mobile game into a different level, which combined the AR technology with the game and provided a new experience for game-players’.

The AR Concerns theme emerged in 4% of the reflections, and it refers to the participants’ negative experience or opinion about AR. Two participants complained about the speed and the precision of the AR technology on the smartphones: ‘The scan ability, although the scan function didn’t work on one painting’ and ‘Some paintings could not be scanned because of the light condition’. These two participants also had concerns about the battery drain. One of them complained that to scan a painting, ‘I had to raise my phone and aim the camera at it. I was a little tired of this because it really distracted my attention when I was appreciating a painting’. Moreover, this participant also commented that, ‘To me, the AR device makes a simple thing troublesome’. Another participant failed to find a meaning of the use of the AR, writing the following, ‘Because while I am scanning the paintings, I am observing on the phone screen how it matches the painting, but I failed to understand concept behind the technology’.

Discussion

The current study aims to make a contribution to the literature by analysing qualitative student learning reflection data that highlight that mobile mixed reality may lead to enhanced learning experiences by answering the research question: ‘Does applied mobile mixed reality create an enhanced learning environment for students?’ Based on the shared views from participants in the study, we identified four frequent themes that address the enhanced learning environments, namely, learning engagement, learning motivation, learning about AR/VR technologies and first-time experiences with the technologies.

The participants also reported positive experiences with all the aspects of the engagement, arguably as a consequence of being able to explore the virtual environment and experience disruptive technologies and gamification. Gamification is most likely the cause of the fun and enjoyment that the participants had within the learning environment. Students felt motivated to learn about AR and VR in general and even expand their knowledge beyond the lesson’s offering. We identified the use of the AR/VR and gamification to be the primary factors leading to increased motivation. This is supported by the literature highlighting that use of AR and VR (Cochrane, Smart, and Narayan 2018) and virtual environment in conjunction with games (Dalgarno and Lee 2010) can increase the engagement and the motivation of the learners.

The study revealed that applied MMR could result in the creation of an enhanced learning environment. It is not entirely clear from the data as to why the MMR had this enhancing effect. This is probably a future study topic and may relate to the cultural diversity within the study with predominantly Chinese participants. While the immersive learning environments might be steadily achievable (Schott and Marshall 2018), scholars emphasise the challenge of achieving high levels of interactivity. However, our results suggest that interactivity is strongly evident in the environment as experienced by the participants, most likely of the multi-user experience, which the aforementioned study lacked.

While students shared their positive experiences with the use of VR technology, the major aspect discussed was the ability for communication within the virtual environment. Many students highlighted the importance of discussion and interaction, for the purpose of the construction environment during all the phases of the process. This is comparable with the findings by Bower, Lee and Dalgarno (2017) that remote and face-to-face learners felt that effective communication, collaboration and copresence were supported by the blended reality environment. However, one student felt that by wearing the headset and being in a VR environment, she lost the human aspect of communication and the ability to effectively communicate. Gugenheimer et al. (2019) highlights significant challenges in the acceptable use of HMDs in shared, social environments with more work required on the design of shared experiences to reduce the feelings of isolation and improve communication within VR and AR environments.

Another major aspect of VR brought up in the data was the excitement and impressions the participants had while interacting with the environment and between their peers, evident in the literature as a learner interaction, which is a characteristic of the virtual learning environment (Cochrane, Smart, and Narayan 2018; Dalgarno and Lee 2010). A major interaction theme, which we even coded as a separate theme, was the ability to change the time of the day within the simulation. Students were impressed with this possibility, to be able to see the objects during the day and night and see how that influences the design, construction and the prospective users of those spaces. A very small number of participants were somewhat excited and wrote about the avatars in the VR environment, mostly about the ability to see their or somebody else’s avatar body movements and presence. The avatars were presented as blue mannequins, with no possibilities of customisation, due to the time limits and the difficulty of personalisation (Steed et al. 2016), no gender and no clothes or any physical details, except a general body shape, which was all done to stay culturally neutral. That might have had some negative effect on the participant’s experience; however, it was brought forward by only one participant, stating that the avatars were not ‘beautiful’.

Two major themes in the VR category talk about the negative side of the use of VR technology. The dizziness, also known as motion or simulation sickness in the literature, experienced by 14% of the participants, is a major shortcoming, specifically with the mobile device VR. It is due to the lack of the six degrees of freedom and also the latency of the scene refresh due to the relatively low processing power of mobile devices. Nevertheless, most of the participants who felt this did not disengage from the experience and finished the task, although feeling some degree of nausea. Only a few did not finish. This arguably might be a result of the short duration of each session (Dziuda et al. 2014). Eleven per cent of the participants reported negative experiences with the technology, mainly with the control. However, there were some concerns about the devices overheating, the discomfort and weight of the headsets, as well as the fidelity of the mobile screens in the headsets, which are issues of mobile approaches (Akçayır and Akçayır 2017).

Participants felt engaged with the AR environment. There is a high level of interaction reported in the reflections, due to the use of AR technology (Bower et al. 2014; Cochrane, Smart, and Narayan 2018; Quintero et al. 2019). The participants were impressed by the use of the app, and the knowledge they inquired about the indigenous artworks and culture. Many participants also felt connected with the place, with regard to sense of place. This is because of performing meaningful, engaging activities at the place.

A major part of the engagement is due to the use of gamification (Dalgarno and Lee 2010), which was coded separately and featured in 14% of the reflections. Many participants enjoyed gameful experiences and felt motivated to learn more. However, some participants reflected on their concerns about AR technology. Due to the changing lighting conditions in the hallways of the building where the paintings were on display, sometimes the AR scanning was unable to pick up the target, which led to frustration, even though this was carefully explained during the onboarding. A more traditionally oriented participant found the AR to be intrusive and would rather read about the painting on the little sign on the wall, which we see simply as the participant’s prerogative. They also found that the weight of the mobile device distracting. One student failed to grasp the whole concept of the use of AR on the paintings, which we surmise is due the language barrier. However, technical problems are expected with emerging technologies and MMR is no different (Akçayır and Akçayır 2017; Dziuda et al. 2014).

Conclusions

Both AR and VR as elements of the mixed reality continuum rely on the human–computer interaction, with both technologies combining the mapping of the environment and display of 3D virtual content anchored in space and time. Therefore, regardless of the MR definition used, all these aspects impact AR and VR and therefore affect MR. This is especially true with new and evolving definitions of the MR paradigm relying on principles of immersion, interactions and information input and output.

In this paper, it was found that the use of MMR in learning can result in an enhanced learning environment that provides unique learning experiences and engagement for the students throughout the learning process. Some of the core features of this delivery technique are found in the enhanced learning aspects, such as improved learning engagement and motivation, improved interaction, increased fun and enjoyment. The implementation of these techniques in the curriculum resulted in enhanced learning environments and consequently with advanced learning outcomes, especially as it applies to BIM and construction education. Based on the data, and to enable all these features, we propose deeper integrations of these techniques and technologies within the course offerings, beyond the scope of the AEC industry.

This study opens the door for future research to inform the causes of the emergence of enhanced learning environments and also how educators can optimise successful learning outcomes. For more conclusive results and a more in-depth understanding of the benefits and the optimisation of the use of technologies to enhance learning, future works should also include longitudinal studies with larger sample size and comparison to a traditional delivery control group. This will investigate how these techniques can influence the evolution of the students’ knowledge over a period of time.

References

Adria, M. (2019) ‘Reading the grand palimpsest of mixed reality’, Explorations in Media Ecology, vol. 18, no. 1–2, pp. 43–55. doi: 10.1386/eme.18.1-2.43_1.

Aguayo, C., Cochrane, T. & Narayan, V. (2017) ‘Key themes in mobile learning: prospects for learner-generated learning through AR and VR’, Australasian Journal of Educational Technology, vol. 33, no. 6, pp. 27–40. doi: 10.14742/ajet.3671.

Akçayır, M. & Akçayır, G. (2017) ‘Advantages and challenges associated with augmented reality for education: a systematic review of the literature’, Educational Research Review, vol. 20, pp. 1–11. doi: 10.1016/j.edurev.2016.11.002.

Alexander, B., et al., (2019) Horizon Report: Higher Education Edition, EDUCAUSE, Louisville, CO.

Antwi-Afari, M. F., et al., (2018) ‘Critical success factors for implementing building information modelling (BIM): a longitudinal review’, Automation in Construction, vol. 91, pp. 100–110. doi: 10.1016/j.autcon.2018.03.010.

Birt, J. & Cowling, M. (2018) ‘Assessing mobile mixed reality affordances as a comparative visualization pedagogy for design communication’, Research in Learning Technology, vol. 26, pp. 1–25. doi: 10.25304/rlt.v26.2128.

Birt, J., Moore, E. & Cowling, M. (2017) ‘Improving paramedic distance education through mobile mixed reality simulation’, Australasian Journal of Educational Technology, vol. 33, no. 6, pp. 69–83. doi: 10.14742/ajet.3596.

Birt, J., et al., (2018) ‘Mobile mixed reality for experiential learning and simulation in medical and health sciences education’, Information, vol. 9, no. 2, p. 31. doi: 10.3390/info9020031.

Bower, M., et al., (2014) ‘Augmented reality in education – Cases, places and potentials’, Educational Media International, vol. 51, no. 1, pp. 1–15. doi: 10.1080/09523987.2014.889400.

Bower, M., Lee, M. J. W. & Dalgarno, B. (2017) ‘Collaborative learning across physical and virtual worlds: factors supporting and constraining learners in a blended reality environment’, British Journal of Educational Technology, vol. 48, no. 2, pp. 407–430. doi: 10.1111/bjet.12435.

Braun, V. & Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative Research in Psychology, vol. 3, no. 2, pp. 77–101. doi: 10.1191/1478088706qp063oa.

Chan, A. P. C., et al., (2018) ‘Critical review of studies on building information modeling (BIM) in project management’, Frontiers of Engineering Management, vol. 5, no. 3, pp. 394–406. doi: 10.15302/j-fem-2018203.

Clark, R. C. & Mayer, R. E. (2016) E-learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning, 4th edn, Wiley, New Jersey, NJ.

Cochrane, T., Smart, F. & Narayan, V. (2018) ‘Editorial: special issue on mobile mixed reality’, Research in Learning Technology, vol. 26, pp. 1–5. doi: 10.25304/rlt.v26.2195.

Crompton, H. & Burke, D. (2018) ‘The use of mobile learning in higher education: a systematic review’, Computers & Education, vol. 123, pp. 53–64. doi: 10.1016/j.compedu.2018.04.007

Dalgarno, B. & Lee, M. J. W. (2010) ‘What are the learning affordances of 3-D virtual environments?’, British Journal of Educational Technology, vol. 41, no. 1, pp. 10–32. doi: 10.1111/j.1467-8535.2009.01038.x.

Dziuda, Ł., et al., (2014) ‘The effects of simulated fog and motion on simulator sickness in a driving simulator and the duration of after-effects’, Applied Ergonomics, vol. 45, no. 3, pp. 406–412. doi: 10.1016/j.apergo.2013.05.003.

Fowler, C. (2015) ‘Virtual reality and learning: where is the pedagogy?’, British Journal of Educational Technology, vol. 46, no. 2, pp. 412–422. doi: 10.1111/bjet.12135.

Gugenheimer, J, et al., (2019) ‘Challenges using head-mounted displays in shared and social spaces’, in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, Glasgow, Scotland, UK, pp. 1–8. doi: 10.1145/3290607.3299028.

Jones, C, et al., (2010) ‘Net generation or digital natives: is there a distinct new generation entering university?’, Computers & Education, vol. 54, no. 3, pp. 722–732. doi: 10.1016/j.compedu.2009.09.022.

Kolb, D. A. (2014) Experiential Learning: Experience as the Source of Learning and Development, 2nd edn, Pearson, New Jersey, NY.

Miettinen, R. & Paavola, S. (2014) ‘Beyond the BIM utopia: approaches to the development and implementation of building information modeling’, Automation in Construction, vol. 43, pp. 84–91. doi: 10.1016/j.autcon.2014.03.009.

Milgram, P. & Kishino, F. (1994) ‘A taxonomy of mixed reality visual-displays’, Ieice Transactions on Information and Systems, vol. 77, no. 12, pp. 1321–1329.

O’Brien, H. L. & Toms, E. G. (2008) ‘What is user engagement? A conceptual framework for defining user engagement with technology’, Journal of the American Society for Information Science and Technology, vol. 59, no. 6, pp. 938–955. doi: 10.1002/asi.20801.

Parveau, M. & Adda, M. (2018) ‘3iVClass: a new classification method for virtual, augmented and mixed realities’, Procedia Computer Science, vol. 141, pp. 263–270. doi: 10.1016/j.procs.2018.10.180.

Puolitaival, T. & Forsythe, P. (2016) ‘Practical challenges of BIM education’, International Journal of Building Pathology and Adaptation, vol. 34, no. 4/5, pp. 351–366. doi: 10.1108/SS-12-2015-0053.

Quintero, J, et al., (2019) ‘Augmented reality in educational inclusion. A systematic review on the last decade’, Frontiers in Psychology, vol. 10, no. 1835, pp. 1–14. doi: 10.3389/fpsyg.2019.01835.

Sacks, R, et al., (2018) BIM Handbook: A Guide to Building Information Modeling for Owners, Designers, Engineers, Contractors, and Facility Managers, 4th edn, Wiley & Sons, Hoboken, NJ.

Sadler-Smith, E & Smith P. J., (2004) ‘Strategies for accommodating individuals’ styles and preferences in flexible learning programmes’, British Journal of Educational Technology, vol. 35, no. 4, pp. 395–412. doi: 10.1111/j.0007-1013.2004.00399.x.

Schott, C & Marshall S., (2018) ‘Virtual reality and situated experiential education: a conceptualization and exploratory trial’, Journal of Computer Assisted Learning, vol. 34, no. 6, pp. 843–852. doi: 10.1111/jcal.12293.

Speicher, M., Hall, B. D. & Nebeling, M. (2019) ‘What is mixed reality?’, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, Glasgow, Scotland, UK, pp. 1–15. doi: 10.1145/3290605.3300767.

Steed, A., et al., (2016) ‘The impact of a self-avatar on cognitive load in immersive virtual reality’, IEEE Virtual Reality (VR), 19–23 March 2016, pp. 67–76. doi: 10.1109/VR.2016.7504689.

Thompson, P. (2013) ‘The digital natives as learners: technology use patterns and approaches to learning’, Computers & Education, vol. 65, pp. 12–33. doi: 10.1016/j.compedu.2012.12.022.

Vasilevski, N. & Birt, J. (2019) ‘Towards optimizing place experience using design science research and augmented reality gamification’, in Intersections in Simulation and Gaming: Disruption and Balance. ASC 2019. Communications in Computer and Information Science, Springer, Singapore. pp. 77–92. doi: 10.1007/978-981-32-9582-7_6.

Vasilevski, N., Brand, J. & Birt, J. (2018) ‘Analysing micro-location beacon gamification: scenarios, types and characteristics’, in Proceedings of the 30th Australian Conference on Computer-Human Interaction, ACM, Melbourne, Australia, pp. 484–489. doi: 10.1145/3292147.3292210.

Wylie, R. & Chi, M. T. H. (2014) ‘The self-explanation principle in multi-media learning’, in R. E. Mayer (ed.), The Cambridge handbook of multimedia learning, 2nd edn, Cambridge University Press, New York, NY, pp. 413–432.