ORIGINAL RESEARCH ARTICLE
Perceived educational usefulness of a virtual-reality work situation depends on the spatial human-environment relation
Martin Dobrickia*, Kevin G. Kimb, Alessia E. Coppia, Pierre Dillenbourgb and Alberto Cattaneoa
aSwiss Federal Institute for Vocational Education and Training, Learning Technologies Research Group, Zollikofen, Switzerland;
bSwiss Federal Institute of Technology Lausanne, Computer-Human Interaction for Learning and Instruction Group, Lausanne, Switzerland
(Received: 8 May 2020; Revised: 8 October 2020; Accepted: 27 November 2020; Published: 4 January 2021)
Abstract
Virtual reality (VR) may be useful for situating school-based vocational education in work-life by simulating a work situation such that learners viewing this VR work situation are located inside it. The reason for this assumption is that VR can fully spatially include its viewer. Research on the utility of viewer-including VR work situations for learners has, therefore, already started. However, no study has yet investigated their utility for teachers. This is particularly relevant for work situations involving environmental planning, as VR is expected to facilitate such a task. We, therefore, asked horticultural teachers to assess the educational usefulness of a VR work situation when they were located outside and inside it. For this purpose, we enabled them to plan a basic garden in the VR work situation when its environment was spatially excluding them and when it was including them. We found the teachers to perceive the viewer-including VR work situation as more useful for their teaching than its viewer-excluding version. This suggests that the perceived educational usefulness of a VR work situation depends on the spatial relation of its viewer and environment, that is, the spatial human-environment relation it involves.
Keywords: vocational education; situated education; virtual reality; perceived usefulness
*Corresponding author: Email: martin.dobricki@ehb.swiss
Research in Learning Technology 2021. © 2021 M. Dobricki et al. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2021, 29: 2453 - http://dx.doi.org/10.25304/rlt.v29.2453
Introduction
A common educational goal is to prepare learners for their work lives, which can be achieved by asking them to accomplish learning tasks based on examples of real-life situations (Bransford, Brown, and Cocking 2000; Lave and Wenger 1991). In vocational education, such situation-based didactics may be accomplished by using examples of work situations (Boldrini, Ghisla, and Bausch 2014; Dobricki, Evi-Colombo, and Cattaneo 2020). For this purpose, the affordance of digital technologies (Bower 2008) can be exploited (Schwendimann et al. 2015). One digital technology that may serve situation-based vocational education particularly well is virtual reality (VR) (Allcoat and von Mühlenen 2018; Dede 2009; Schott and Marshall 2018). The reason for this assumption is that VR can serve for the naturalistic and thereby practice-oriented simulation of work situations inside a classroom. Research on virtual learning environments (Concannon, Esmail, and Roduta Roberts 2019; Jensen and Konradsen 2018; Mikropoulos and Natsis 2011; Radianti et al. 2020) has accordingly already begun investigating the educational utility of VR work situations (Babu et al. 2018; Bharathi and Tucker 2015; Hafsia, Monacelli, and Martin 2018; Schild et al. 2018; Smith et al. 2018). Some studies have examined the utility of VR work situations for learners (Babu et al. 2018; Bharathi and Tucker 2015; Butt, Kardong-Edgren, and Ellertson 2018; Sacks, Perlman, and Barak 2013; Smith et al. 2018; Stone, Watts, and Zhong 2011), but none have investigated their utility for teachers. Because it is for teaching in vocational schools that VR work situations may serve best, it is necessary to start exploring the utility of VR work situations from the perspective of vocational teachers.
Educating learners through VR work situations requires them to be able to view the situations in a three-dimensional (3D) virtual environment (Dalgarno and Lee 2010). For this purpose, learners are most often asked to use a head-mounted or desktop display. A head-mounted display presents a virtual environment stereoscopically and enables its user to view the environment by moving the head (Carruth 2017). The spatial relation between the human viewer of a 3D virtual environment and this environment is, therefore, differential when the environment is viewed with a head-mounted or on a desktop display. When viewing a 3D virtual environment with a head-mounted display this ‘human-environment’ relation (Heft 2001) is such that the environment is spatially including its viewer (Johnson-Glenberg 2018). When viewing a 3D virtual environment on a desktop display the human-environment relation is instead such that the environment is spatially excluding its viewer. In contrast to such a viewer-excluding environment, a viewer-including 3D virtual environment may appear more relevant to viewers (Dobricki and Pauli 2016; Slater 2009). Hence, simulating a work situation with VR may serve situation-based vocational education better when the VR environment spatially includes its viewers instead of excluding them (Johnson-Glenberg 2018). This is particularly relevant for work situations involving environmental planning, as VR is expected to facilitate such a task (Portman, Natapov, and Fisher-Gewirtzman 2015). We, therefore, enabled horticultural teachers to plan and view a basic tree garden in a VR work situation. We asked them to assess the educational usefulness of this simulation and their sensation of being present in it when the environment excluded them and when it included them spatially. This served to examine whether they perceived the viewer-including or the viewer-excluding version of our VR work situation to be more useful for teaching. The teachers were also asked to freely express their opinions on the potential educational utility of the VR work situation in interviews.
Method
Participants
Ten male horticultural teachers (mean age = 48.3 years, SD = 6.9 years) with normal or corrected-to-normal vision participated. This sample size was suggested by a priori power analysis (Brysbaert 2019) specified as follows: f = 0.3, a error = 0.05, power = 0.8, corr. = 0.8. All participants had been teaching horticulture for at least 5 years. All of them had been using a personal computer multiple times per week. Seven teachers indicated that they had never used computer games or VR, and three said they had rarely used them. The subjects gave their written informed consent and could have withdrawn from the study at any time. The experimental procedure was performed in accordance with the 1964 Declaration of Helsinki and it was consistent with all relevant Swiss legislation.
Stimuli and apparatus
Participants were presented with a 3D virtual environment (Figure 1) from a first-person perspective. This was accomplished by using the graphics engine Unity3D on a Hewlett Packard OMEN computer with an NVIDIA GeForce GTX 1070 graphics card. The virtual environment involved trees and an outdoor environment. The virtual trees were created using the software SpeedTree (www.speedtree.com). The virtual outdoor environment was generated with the software DroneDeploy (www.dronedeploy.com) based on visual data from a real environment. These data were captured with the digital camera of a miniature unmanned aerial vehicle from DJI (www.dji.com/ch/spark). The virtual environment was presented either with the desktop display of the computer or with a stereoscopic motion-tracked Oculus Rift head-mounted display. This display consisted of dual OLED displays with a resolution of 1200 × 1080 pixels per eye displayed at 90 Hz. It had a 94° horizontal and 93° vertical field-of-view and was motion-tracked by two optical sensors. The user was enabled to switch between two perspectives on the virtual environment: a top view and a ground view. The top view served to plan a garden and involved a control panel for opting between different 3D virtual trees to place them in the environment, while the ground view served to explore the environment with the previously placed trees and involved another control panel. This panel (Figure 1) served to modulate the situation by changing the sizes of the trees, their shadows by moving the sun and the seasonal appearances of their leaves. Participants were enabled to operate the two panels by holding them with the Oculus Touch control devices in their left hands and making selections with them in their right hands. This was accomplished with a computer mouse when the virtual environment was presented on the desktop display.
Figure 1. Experimental setup. (a) Subject holding two wireless controllers and wearing the motion-tracked head-mounted display that was used to present the viewer-including version of (b) the 3D virtual environment of the VR outdoor work situation for garden planning, including a panel for modulating the properties of the situation with the controllers, such as the season, the sizes of trees and the sun’s position. (c) Subject looking at the desktop display that was used to present the viewer-excluding version of the 3D virtual environment.
Experimental design
We used a within-subjects crossover design with two experimental conditions. In one condition, the 3D environment of the VR work situation spatially included its observer, as it was viewed within the head-mounted display. In the other condition, the 3D virtual environment spatially excluded its observer, as it was viewed on the desktop display. All participants were exposed to both experimental conditions. The order of the conditions was determined by the crossover design.
Procedure
The procedure was the same in both experimental conditions: Firstly, the participants were asked to create a basic tree garden by placing and organising some trees in the virtual environment for 5 min. Secondly, they were asked to modulate the trees regarding the properties described in the stimuli and apparatus section. Subsequently, they were asked to respond to the two psychometric paper-and-pencil questionnaires described in the subsequent section. After accomplishing both of the experimental conditions described here, each teacher was interviewed.
Psychometric ratings
The participants were first asked to assess the perceived educational usefulness and ease of use of the VR work situation. They accomplished this by responding to a paper-and-pencil questionnaire. This 12-item questionnaire presented six self-report statements (Table 1) on perceived usefulness and six statements on ease of use adapted from the technology acceptance model (TAM) questionnaire from Davis (1989).
All statements had a visual analogue scale (VAS) of 100 mm, with the left pole labelled ‘not at all’ and the right pole labelled ‘very much’ below them. The participants were instructed to draw a small vertical line on the VAS to evaluate the VR work situation described in the stimuli and apparatus section. The participants were subsequently asked to assess their subjective experience in the 3D virtual environment by responding to a short version of the ITC-Sense of Presence Inventory (ITC-SOPI) from Lessiter, Freeman, and Davidoff (2001). This paper-and-pencil questionnaire involved 19 self-report statements (Table 2).
Four of these questionnaire items were on the experience of being present in the virtual environment, which was named ‘spatial presence’. Five statements concerned the engagement of the user, five concerned the perceived naturalness of the virtual environment and another five concerned the experience of negative effects. All statements had a bipolar 5-point Likert-type scale to their right (strongly disagree = 1, disagree = 2, neither agree nor disagree = 3, agree = 4, strongly agree = 5). The participants were instructed to rate their agreement or disagreement with a statement by making a mark on the corresponding category of this scale.
Interviews
The interviews were semi-structured and lasted about 10 min. The teachers were asked to express their opinions regarding the potential utility of our VR work situation simulation for horticultural education. All the interviews were digitally registered and were subjected to content analysis.
Data analysis
Firstly, the participants’ individual scores on the TAM scales named perceived educational ‘usefulness’ and ‘ease of use’, and on the ITC-SOPI scales, named ‘spatial presence’, ‘engagement’, ‘naturalness’ and ‘negative effects’, were determined by calculating their mean rating of the questionnaire items used to assess each of these psychometric scales. Subsequently, these scale scores were compared across the two experimental conditions by one-way repeated-measures analyses of variance (ANOVAs). For the purpose of a more detailed analysis, the ratings of the questionnaire items on perceived educational usefulness were further analysed with the Wilcoxon signed rank test (two-tailed). In addition to the ANOVAs, the effect size f, and the Wilcoxon tests, the effect size rcontrast (Rosnow and Rosenthal 2003) were calculated. All statistical analyses were conducted with the statistical software R (www.r-project.org). The contents of the interviews were analysed as follows: Firstly, the relevant excerpts were identified by determining all interview excerpts concerning the potential utility of the VR work situation. Secondly, it was determined based on the differences and similarities (Aveyard 2010) of these excerpts to which affordance categories they could be grouped together.
Data availability
The complete data set of the experimental study is available in the supplemental online material of this article. The interview data are available on request from the first author. These data are not publicly available because they contain information that could compromise the privacy of the research participants.
Results
Experiments
The teachers rated the educational usefulness of the viewer-including version of the VR work situation, M = 50.9, 95% CI [29.0, 72.8], significantly higher, F (1, 9) = 5.70, p = 0.041, f = 0.80, than its viewer-excluding version, M = 44.9, 95% CI [25.4, 64.4]. Both the viewer-including version of the VR work situation, M = 77.4, 95% CI [68.4, 86.4], and the viewer-excluding version, M = 79.9, 95% CI [68.5, 90.7], were easy to use, and there was no significant difference regarding this ‘ease of use’.
A more detailed analysis of the teachers’ ratings of the statements on perceived educational usefulness yielded the following. As depicted in Figure 2, the teachers rated the first two educational usefulness statements ‘I would find this system useful for my teaching’, Z = -2.19, p = 0.028, rcontrast = 0.69, and ‘Using this system would increase my teaching effectiveness’, Z = -2.25, p = 0.024, rcontrast = 0.71, significantly higher, and they rated ‘Using this system would make it easier to teach’ almost significantly higher, Z = -1.75, p = 0.080, rcontrast = 0.55, for the viewer-including version of the VR work situation than for the viewer-excluding version. There were no significant differences between the two experimental conditions regarding the other statements on perceived educational usefulness.
Figure 2. Perceived educational usefulness. Box-and-whisker plots of the teachers’ ratings of the perceived educational usefulness statements, which were higher for the viewer-including than for the viewer-excluding version of the VR work situation. Bold horizontal lines indicate the median rating score, boxes indicate the lower and upper quartiles and whiskers indicate the farthest data points within 1.5 times the lower and upper quartiles, respectively. The circles depict the individual rating scores of the 10 teachers.
The teachers’ ratings of their experiences being present in the virtual space were significantly higher, F (1, 9) = 12.94, p = 0.006, f = 1.20, for the viewer-including 3D virtual environment, M = 3.9, 95% CI [3.4, 4.3], than for its viewer-excluding version, M = 3.0, 95% CI [2.5, 3.5]. Moreover, they experienced themselves to be significantly more engaged, F (1, 9) = 12.10, p = 0.007, f = 1.16, in the viewer-including version of the VR work situation, M = 4.0, 95% CI [3.4, 4.5], than in its viewer-excluding version, M = 3.4, 95% CI [2.9, 3.8]. There was no significant difference in their ratings of the naturalness of the viewer-including 3D virtual environment, M = 3.7, 95% CI [3.1, 4.3], and its viewer-excluding version, M = 3.4, 95% CI [2.8, 4.0]. Finally, the teachers’ ratings of negative effects indicated that they experienced no such effects in either the viewer-including version, M = 2.3, 95% CI [1.7, 2.9], or the viewer-excluding version, M = 1.2, 95% CI [0.8, 1.5], of the VR work situation.
Interviews
Across the 10 interviews, we identified 45 excerpts concerning the utility of the viewer-including VR work situation. The content analysis of these excerpts indicated that they could be grouped into two affordance categories: ‘utility for naturalistic teaching’ and ‘utility for learning information integration’. The first category involved the ability of the viewer-including VR simulation to present educational content naturalistically and enable teaching ‘holistically’, that is, to teach different topics simultaneously by, for example, combining botany and environmental planning. This category was formed from 29 excerpts across 9 of the 10 interviews. The following quotation from one of the interviewed teachers exemplifies it well:
Assuming you could create a larger task of which one part is on plants and another part is on horticulture, then I see a great benefit … that, on the one hand, I could somehow make a section on garden design with trees in which we could look at shadows, autumn colours, flower colours or flowering times, things like that, and then at the same time I would also have a plan for asking: well, the path along which one can walk here, how should it be built? …such that one would have a complete task, something bigger … so that everything is combined into one profession.
This quotation highlights that the naturalistic teaching afforded by our viewer-including VR work situation may help avoid ‘chunking’ too much professional competence into separate knowledge or skills for the purposes of didactics, thus fulfilling an important requirement of situation-based didactics in vocational education.
The second affordance category, ‘utility for learning information integration’, involved the viewer-including VR simulation’s property to foster the training of learners in combining garden-planning information on plants regarding their organisation in space, size and change over time. This category was formed from 16 excerpts across 6 of the 10 interviews. The following quotation from one of the interviewed teachers exemplifies it well: It helps
learners to understand how a garden develops within a certain cycle, what the garden looks like after 10, 20, 30 years, what knowledge they can derive from this regarding planting distances, growth, seasons, sun shade, light conditions.
This quotation highlights that our VR work situation could help students learn integrating diverse task-related information despite the situation’s limited detail.
In summary, the 45 interview excerpts could be grouped into the following two affordance categories: the utility of the VR work situation for naturalistic teaching and its utility for learning information integration.
Discussion
We found horticultural teachers to experience themselves being more present in the viewer-including 3D environment of our VR work situation and to perceive it as more useful for their teaching than its viewer-excluding version. Thus, the perceived educational usefulness of the VR work situation appears to depend on the spatial relation of its viewer and environment, that is, the spatial human-environment relation.
Bharathi and Tucker (2015) found a viewer-including VR work situation to be more effective for vocational learning than a viewer-excluding version. The vocational teachers’ perspectives on the educational usefulness of our VR work situation are consistent with this finding. Moreover, both our experimental and interview findings point to the utility of the viewer-including version of our VR work situation for naturalistic teaching. Thus, consistent with current research on VR education (Jensen and Konradsen 2018; Johnson-Glenberg 2018; Radianti et al. 2020), our findings give rise to the following question: How can the viewer-including affordance of VR be exploited to advance instructional design? Considering that in a viewer-including 3D virtual environment, the consequences of one’s behaviour may appear relevant to the brain for one’s own well-being (Dobricki and Pauli 2016; Slater 2009), perhaps incomplete worked-out examples (Atkinson and Renkl 2007; Renkl 2005) may be implemented in VR as follows. They may be designed such that the behaviour by which these examples are completed appears to learners to be correct or wrong based on the positive or negative consequences it has for themselves. Thereby, worked-out examples may not only serve to effectively train cognitive skills (Atkinson et al. 2000) but potentially behavioural skills as well (Cattaneo and Boldrini 2016).
Our study has two main limitations. Firstly, the sample size was rather small. Secondly, we did not investigate whether our VR work situation could help actual vocational students learn garden planning, so our findings do not allow any conclusions in this regard. However, our interview findings point to the potential utility of the viewer-including version of our VR work situation to learn integrating task-related information. Hence, it may be worthwhile to investigate whether a viewer-including VR work situation can serve to learn information integration effectively. To date, no such investigation is available, and little experimental research has investigated whether a viewer-including VR work situation is more effective for learning than its viewer-excluding version (Bharathi and Tucker 2015; Smith et al. 2018). Hence, there is a general need for research on how and for what the spatial human-environment relation afforded by viewer-including VR work situations can serve learners, especially in vocational education.
Conclusion
We enabled vocational teachers to experience and operate a VR work situation with different relations between the viewer and environment. Our experimental findings indicated that the teachers perceived the viewer-including VR work situation as more useful for their teaching than its viewer-excluding version. This suggests that the ability of VR work situations to spatially include their viewers may advance vocational educational practice. Our interview findings indicate that this advance might consist of presenting educational content naturalistically. Hence, an important avenue for future research is to investigate what educational content could benefit most from the viewer-including property of VR work situations.
Acknowledgements
The authors would like to thank Joseph Vavalà for helping to prepare and program the virtual reality simulation.
Disclosure statement
No potential conflict of interest was reported by the authors.
References
Allcoat, D. & Von Mühlenen, A. (2018) ‘Learning in virtual reality: effects on performance, emotion and engagement’, Research in Learning Technology, vol. 26, no. 2018, pp. 2140. doi: 10.25304/rlt.v26.2140
Atkinson, R. K., et al., (2000) ‘Learning from examples: instructional principles from the worked examples research’, Review of Educational Research, vol. 70, no. 2, pp. 181–214. doi: 10.3102/00346543070002181
Atkinson, R. K. & Renkl, A. (2007) ‘Interactive example-based learning environments: using interactive elements to encourage effective processing of worked examples’, Educational Psychology Review, vol. 19, no. 3, pp. 375–386. doi: 10.1007/s10648-007-9055-2
Aveyard, H. (2010) Doing a Literature Review in Health and Social Care: A Practical Guide, McGraw-Hill, Maidenhead.
Babu, S. K., et al., (2018) ‘Virtual reality learning environments for vocational education: a comparison study with conventional instructional media on knowledge retention’, Proceedings of the IEEE International Conference on Advanced Learning Technologies, Mumbai, pp. 385–389.
Bharathi, A. K. B. G. & Tucker, C. S. (2015) ‘Investigating the impact of interactive immersive virtual reality environments in enhancing task performance in online engineering design activities’, Proceedings of the ASME 2015 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Boston, MA, pp. 1–11.
Boldrini, E., Ghisla, G. & Bausch, L. (2014) ‘Progetti di didattica per situazioni’, in Formazione. I metodi, eds G. P. Quaglino, Milano, pp. 337–360.
Bower, M. (2008) ‘Affordance analysis – matching learning tasks with learning technologies’, Educational Media International, vol. 45, no. 1, pp. 3–15. doi: 10.1080/09523980701847115
Bransford, J. D., Brown, A. L. & Cocking, R. R. (2000) How People Learn: Brain, Mind, Experience, and School, The National Academies Press, Washington, DC.
Brysbaert, M. (2019) ‘How many participants do we have to include in properly powered experiments? A tutorial of power analysis with reference tables’, Journal of Cognition, vol. 2, no. 1, pp. 1–38. doi: 10.5334/joc.72
Butt, A. L., Kardong-Edgren, S. & Ellertson, A. (2018) ‘Using game-based virtual reality with haptics for skill acquisition’, Clinical Simulation in Nursing, vol. 16, pp. 25–32. doi: 10.1016/j.ecns.2017.09.010
Carruth, D. W. (2017) ‘Virtual reality for education and workforce training’, Proceedings of the 15th International Conference on Emerging eLearning Technologies and Applications, Stary Smokovec, pp. 1–6.
Cattaneo, A. & Boldrini, E. (2016) ‘You learn by your mistakes. Effective training strategies based on the analysis of video-recorded worked-out examples’, Vocations and Learning, vol. 10, pp. 1–26. doi: 10.1007/s12186-016-9157-4
Concannon, B. J., Esmail, S. & Roduta Roberts, M. (2019) ‘Head-mounted display virtual reality in post-secondary education and skill training’, Frontiers in Education, vol. 4, pp. 80. doi: 10.3389/feduc.2019.00080
Dalgarno, B. & Lee, M. J. W. (2010) ‘What are the learning affordances of 3D virtual environments?’, British Journal of Educational Technology, vol. 41, no. 1, pp. 10–32. doi: 10.1111/j.1467-8535.2009.01038.x
Davis, F. D. (1989) ‘Perceived usefulness, perceived ease of use, and user acceptance of information technology’, MIS Quarterly, vol. 13, no. 3, pp. 319–340.
Dede, C. (2009) ‘Immersive interfaces for engagement and learning’, Science, vol. 323, no. 5910, pp. 66–69. doi: 10.1126/science.1167311
Dobricki, M., Evi-Colombo, A. & Cattaneo, A. (2020) ‘Situating vocational learning and teaching using digital technologies – a mapping review of current research literature’, International Journal for Research in Vocational Education and Training. vol. 7, no. 3, pp. 344–360. doi: 10.13152/IJRVET.7.3.5
Dobricki, M. & Pauli, P. (2016) ‘Sensorimotor body-environment interaction serves to regulate emotional experience and exploratory behavior’, Heliyon, vol. 2, p. e00173. doi: 10.1016/j.heliyon.2016.e00173
Hafsia, M., Monacelli, E. & Martin, H. (2018) ‘Virtual reality simulator for construction workers’, Proceedings of the Virtual Reality International Conference – Laval Virtual, ACM, New York, pp. 1–7.
Heft, H. (2001) Ecological Psychology in Context, Lawrence Erlbaum Associates, Mahawah, NJ.
Jensen, L. & Konradsen, F. (2018) ‘A review of the use of virtual reality head-mounted displays in education and training’, Education and Information Technologies, vol. 23, no. 4, pp. 1515–1529. doi: 10.1007/s10639-017-9676-0
Johnson-Glenberg, M. C. (2018) ‘Immersive VR and education: embodied design principles that include gesture and hand controls’, Frontiers in Robotics and AI, vol. 5, no. 81, pp. 1–19. doi: 10.3389/frobt.2018.00081
Lave, J. & Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation, Cambridge University Press, New York, NY.
Lessiter, J., Freeman, J. & Davidoff, E. K. J. (2001) ‘A cross-media presence questionnaire: the itc-sense of presence inventory’, Presence: Teleoperators and Virtual Environments, vol. 10, no. 3, pp. 282–297. doi: 10.1162/105474601300343612
Mikropoulos, T. A. & Natsis, A. (2011) ‘Educational virtual environments: a ten-year review of empirical research (1999–2009)’, Computers & Education, vol. 56, pp. 769–780. doi: 10.1016/j.compedu.2010.10.020
Portman, M. E., Natapov, A. & Fisher-Gewirtzman, D. (2015) ‘To go where no man has gone before: virtual reality in architecture, landscape architecture and environmental planning’, Computers, Environment and Urban Systems, vol. 54, pp. 376–384. doi: 10.1016/j.compenvurbsys.2015.05.001
Radianti, J., et al., (2020) ‘A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda’, Computers & Education, vol. 147, p. 103778. doi: 10.1016/j.compedu.2019.103778
Renkl, A. (2005) ‘The worked-out examples principle in multimedia learning’, in The Cambridge Handbook of Multimedia Learning, ed R. Mayer, Cambridge University Press, Cambridge, pp. 229–246.
Rosnow, L. R. & Rosenthal, R. (2003) ‘Effect sizes for experimenting psychologists’, Canadian Journal of Experimental Psychology, vol. 57, no. 3, pp. 221–237. doi: 10.1037/h0087427
Sacks, R., Perlman, A. & Barak, R. (2013) ‘Construction safety training using immersive virtual reality’, Construction Management and Economics, vol. 31, no. 9, pp. 1005–1017. doi: 10.1080/01446193.2013.828844
Schild, J., et al., (2018) ‘EPICSAVE – enhancing vocational training for paramedics with multi-user virtual reality’, Proceedings of the IEEE International Conference on Serious Games and Applications for Health, Vienna, pp. 1–8.
Schott, C. & Marshall, S. (2018) ‘Virtual reality and situated experiential education: a conceptualization and exploratory trial’, Journal of Computer Assisted Learning, vol. 34, pp. 843–852. doi: 10.1111/jcal.12293
Schwendimann, B. A., et al., (2015) ‘The “Erfahrraum”: a pedagogical model for designing educational technologies in dual vocational systems’, Journal of Vocational Education & Training, 67, pp. 367–396. doi: 10.1080/13636820.2015.1061041
Slater, M. (2009) ‘Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments’, Philosophical Transactions of the Royal Society B, vol. 364, no. 1535, pp. 3549–3557. doi: 10.1098/rstb.2009.0138
Smith, S. J., et al., (2018) ‘Effectiveness of two varying levels of virtual reality simulation’, Nursing Education Perspectives, vol. 39, no. 6, pp. 10–15. doi: 10.1097/01.NEP.0000000000000369
Stone, R. T., Watts, K. P. & Zhong, P. (2011) ‘Physical and cognitive effects of virtual reality integrated training’, Human Factors, vol. 53, no. 5, pp. 558–572. doi: 10.1177/0018720811413389