ORIGINAL RESEARCH ARTICLE
Simon N. Leonarda* and Robert N. Fitzgeraldb
aSchool of Education, University of South Australia, Adelaide, Australia; bCharles Darwin University, Darwin, Australia
(Received: 24 September 2018; final version received: 28 October 2018; Published 27 November 2018)
The technology supporting augmented and mixed reality educational environments is advancing with recent hardware including self-contained headsets that are able to simulate holographic additions to real spaces. These technical advances appear to offer greater capacity to actually realise the educational potential and promise of such technologies noted in the literature over the last decade. This article adds to this literature by reporting on the pilot phase of an educational design research project using the Microsoft HoloLens device in a secondary school setting in Australia. Consistent with previous research in this area, this project found ongoing technical and managerial limitations in implementing augmented and mixed reality, including a continuing concern by many participating teachers of a lack of control of the mixed reality environment. Notably, the pilot study also revealed different understandings of the potential for embodied learning between students, teachers and researchers that requires further research.
Keywords: mixed reality; augmented reality; embodied learning; expansive learning;
This paper is part of the special collection Mobile Mixed Reality Enhanced Learning, edited by Thom Cochrane, Fiona Smart, Helen Farley and Vickel Narayan. More papers from this collection can be found here
*Corresponding author. Email: simon.leonard@unisa.edu.au
Research in Learning Technology 2018. © 2018 N Simon et al. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2018, 26: 2160 - http://dx.doi.org/10.25304/rlt.v26.2160
Microsoft HoloLens is a mixed reality (MR) device that allows the user to engage with digital content and interact with both real-world artefacts and virtual artefacts in the form of holographic-like images. There is some contestation and confusion regarding the definition of MR; however, following Milgram and Kishino (1994), it is perhaps useful to see it, along with augmented reality (AR), as lying somewhere on the continuum between the natural world and a completely virtual environment. A particular feature of the Microsoft HoloLens device that relates to this special issue is its high level of mobility. The device places images onto a transparent near-eye screen so as to create the illusion of a holographic image in real space. Notably, the device accounts for the user’s movement in space and adjusts the image accordingly so that it appears to act like a real object. It does this in an independent stand-alone headset that is not tethered. In addition, and unlike tablet or phone-based AR solutions, it leaves the natural world completely visible, allowing the wearer to move safely and freely around a room or even larger area while engaging with virtual and natural objects. In providing such affordances, the Microsoft HoloLens appears to offer a significant step towards realising the educational potential long seen in MR, virtual reality (VR) and AR technologies. The pilot phase of an educational design research project reported in this paper confirms this potential, while also pointing to significant technical and theoretical challenges.
The pilot reported here was part of an educational design research project (McKenney and Reeves 2012). The project was conducted in a well-resourced Australian secondary school, with extensive support from the research team and a major academic publishing company. As is standard in educational design research, the pilot took a developmental stance. It was an opportunity for different actors in the design process – teachers, students, researchers and content designers – to work together to better understand the design and research problems. Much of what the pilot discovered was not unexpected: the technology was broadly engaging and has a ‘wow factor’; some teachers saw great potential; and there were numerous implementation issues, both technical and pedagogical, even in the ideal conditions in which the pilot took place. One thing that was not anticipated, however, was an uncertainty from both teachers and students in thinking about the ‘enacted’ rather than visualisation affordances that the mobility the Microsoft HoloLens device provided. That is, unlike the research team, the teachers and students involved in the pilot tended to use other visualisation technologies as their frame of reference for the use of this technology. In doing so, they tended to ‘bootstrap’ the technology to established classroom teaching and learning approaches and so restricted the capacity to think of the possibilities for the technology in its own right.
The purpose of this article is to argue the need of establishing an ‘expansive’ design framework when embarking on projects that offer new ways of doing teaching and learning. The argument will be made through reflecting on this pilot while engaging with relevant literature. In doing so, the article reflects on the messiness of the design process. It moves from data to discussion and back again. In broad terms, however, it starts by seeking to understand what happened in the pilot and then moves to exploring where the project ought to go next.
Before we move on to describing the pilot, it is worthwhile pausing to consider the theoretical orientation that the research team brought to the technology used in this project. Although both MR and AR provide visual simulations, they are different from completely virtual environments in ways that are educationally and theoretically important.
MR, VR and AR technologies are all systems that have the ability to alter human perception (Milgram and Kishino 1994). The technologies developed have generally targeted visual and auditory perceptions, although haptic (touch) systems have also been developed (Carijó, de Almeida, and Kastrup 2013). The technologies classified as VR have sought to create a completely artificial environment and provide the user with some capacity to interact with that environment. In doing so, they typically reduce or eliminate the capacity to interact with the real world. VR headsets, for example, allow an user to see only the virtual world created and make the user ‘blind’ to the real world around them. These technologies have been well developed for the video games industry and become relatively cheap through the adaptation of mobile phone screens to serve as VR headsets, such as the Samsung Gear. The terms ‘augmented’ and ‘mixed’ reality, on the other hand, are often used interchangeably to describe technologies that alter perception, but still allow interaction with the real world. Well-known examples of this are the abandoned Google Glass device and phone- or tablet-based solutions such as the game Pokémon Go.
The differences between VR and AR/MR have important practical and theoretical implications for learning design and there is a strong argument available to set aside the technical similarities of the technologies and treat them separately (Hugues, Fuchs, and Nannipieri 2011). In many respects, the affordances of VR have been well explored in the literature on the educational use of video games (Gee 2006; Waddington 2015), although the immersive nature of more advanced VR technologies appears to enhance these effects ( Clark, Tanner-Smith, and Killingsworth 2016). Merchant et al. (2014) conducted a meta-analysis of desktop VR applications (i.e. games, simulation and virtual worlds) and concluded the following:
The results of this meta-analysis are encouraging in that they provide evidence that virtual reality-based instruction is an effective means of enhance [sic] learning outcomes. Educational institutions planning to invest time and financial resources are likely to see the learning benefits in their students. (p. 37)
VR seems to have particular potential to improve spatial reasoning. In a recent study, for example, Pan and Niemeyer (2017) reported on how users’ experience of catching a real ball was impacted while they were immersed in VR. They showed that VR environment positively impacted the catching task and suggested that improved physical performance occurred because the physical task was able to be simplified through spatial VR affordances. Such studies connect to a growing body of multi-disciplinary research that highlights the importance of spatial reasoning, and especially spatial skills associated with spatial visualisation, mental rotation and spatial orientation (Newcombe 2010; Stieff and Uttal 2015) that show a positive correlation with mathematical competence and future success in Science, Technology, Engineering and Maths (STEM) fields (Lubinski 2010).
AR/MR, however, seems to provide something other than visualisation. Of particular relevance to this special issue is that they offer mobility, and therefore sensory–motor engagement in the real world. This was a standout affordance in the initial design thinking of our research team. The importance of such engagement to human cognition has become increasingly apparent within research in cognitive psychology over the past one or two decades, although it can be difficult to find because it goes under many names such as embodied, embedded, enacted, extended and situated cognition (Kiverstein and Clark 2009).
‘Extended’ cognitive science views cognition as being connected in a mind–body–world system (Clark 2011). This science tells us that the human brain is capable of incorporating tools into its motor and haptic systems; we don’t feel through the tool, rather the mind treats the tool as if it is an extension of the body. (Carijó et al. 2013; Froese et al. 2012). It shows that the mind can use the body in more abstract ways, such as the increasing evidence that gesture influences learning, both by influencing the learner’s environment and by influencing the learners themselves (Goldin-Meadow and Wagner 2005). Gesture appears to be particularly important when tasks are unfamiliar (Johnson-Glenberg and Megowan-Romanowicz 2017; Logan, Lowrie, and Diezmann 2014), through providing the cognitive system with a stable external and visual presence that can provide the means with which to think (Pouw et al. 2014).
An important finding in our pilot study that we will report below, however, was that the teachers and students we were working with did not appear to have access to extended models of cognition when thinking about educational design. They almost entirely drew on the visualisation affordance of the device when thinking about how the device might be used and gave no consideration at all to its mobility and interactivity. Although a tentative finding from a pilot study, this finding may have important implications for how researchers can work with teachers on the use of increasingly mobile technologies in education. It suggests that the teaching workforce may not have the extant conceptual models to take full advantage of the affordances of these technologies and that significant professional learning may be required. It supports questions such as those raised by Hamilton, Rosenberg and Akcaoglu (2016) on the capacity of current practice-based models, such as the popular Substitution, Augmentation, Modification, and Redefinition (SAMR) model (Puentedura 2006) to actually ‘redefine’ classroom practice.
The project reported here took an educational design research (Kelly, Baek, and Lesh 2008; McKenney and Reeves 2012) stance. This approach, also known as design-based research, blends the systematic development of solutions to educational problems with scientific research. Owing much to the approach of the design sciences such as engineering, it places a priority on developing useful solutions while also capturing theoretical understandings that can support the work of others. A common first step within educational design research is for researchers and practitioners to analyse the problem space together. Later phases of educational design research involve the development and evaluation of solutions leading to the documentation of design principles.
This article reports on the first phase of the project only, that is, it should be read as a collaborative exploration of the problem space, seeking to define more so than resolve the problems of practice and theory to be engaged. The project arose from a partnership between the publishing company, a large secondary school in Australia and our research team. The project also received some support from the manufacturer, although the manufacturer did not participate directly in the trial. In this pilot phase, each of the partners was seeking to understand more about the educational possibilities of the Microsoft HoloLens MR device in schools and an educational design research stance was taken to explore the researchable and designable aspects of the technology together. The publishing company was also interested in feedback on a number of apps they had developed for use with the device. The publishing company and the manufacturer had a clear commercial interest in promoting the use of this technology and the associated apps and entered the project seeking to develop high-quality products for the school sector.
The school involved in the study was an independent preschool-12 school in a metropolitan location. The project involved only the 7–12 sections of the school. The school was well resourced with a solid IT infrastructure and the financial capacity to invest in emerging technologies. The authors of this article and their university technical team worked with employees of the publishing company to facilitate design workshops on how the apps that had been developed by the publisher might be integrated into teaching and learning at the school. Teachers then trialled these initial designs with students from the school in subjects including chemistry, physics, music and health. Coincidentally, the trial was conducted in the first year the school had become a co-educational school and there were still many more boys than girls enrolled in the school, although no significant gender difference was discernible in the trial data.
Seeking to better understand the problem space from the diverse perspectives available even in this small trial, evaluative data were collected in the forms of researcher field notes, a student survey and a number of semi-structured focus group interviews with the teachers involved. Collected with the approval of the university’s ethics committee, these data were intended to support a process of reflection or developmental evaluation (Leonard, Fitzgerald, and Riordan 2016). Although the reflective process is the focus of this article, we will briefly provide some insight into the data that informed the process in the following section.
The first data source was a brief survey of students comprising 17 questions. The first 14 questions were rated on an 11-point scale (0–10), while the last three questions called for a written response. Seventy-three valid responses were received, representing a response rate of 50%. Table 1 summarises the means and standard deviations for each of the rated items.
Acknowledging that these scales were developed with certain dimensions in the minds of the researchers, we sought to test the meaningfulness of the implicit dimensionality. To achieve this, a principal components analysis (PCA) was conducted using SPSS software. PCA is a variable reduction technique that is used to identify the most useful variables in a data set and provides a way of exploring the common themes or constructs that they represent. In terms of this evaluation, the aim was to explore whether there were patterns in the way students responded to the apps that went beyond simply enjoyment and engagement. A PCA analysis with varimax rotation was performed in SPSS, which resulted in a three-component solution summarised in Table 2.
Even though this was a small sample, the results are challenging and suggest that the students are thinking about their experience with this technology in quite complex ways. This analysis shows that while the students quite uniformly found the technology engaging even if it was difficult to use (component 1), they believed it to have a positive impact on learning (component 2), but they showed some ambivalence on its spatial and perspective affordances (component 3). Notably, of all the factors measured, the ability to rotate objects was rated particularly poorly.
Another data source was a series of semi-structured focus group interviews carried out with the teachers involved in the trial. Guided by a predetermined set of questions, the focus groups included representatives from all parts of the partnership. Transcripts of the focus groups were analysed using a computer-assisted phenomenography (CAP) technique (Leonard and Roberts 2014) supported by the Leximancer software. Leximancer uses a corpus linguistic approach to textual analysis and identifies concepts used within text, mapping those concepts and the relationships between them. The result is not dissimilar to the PCA analysis in that it identifies concepts that ‘hang’ together. In practical terms, the analysis identifies themes or a cloud of concepts that are frequently discussed in relation to each other across the conversation. Table 3 summaries this analysis and shows the major themes identified (column 1) and an example of relevant text from the transcriptions (column 2).
Evident even in the limited quotes included in Table 3, the professional discourse here was striking. Across several hours of combined conversation, the teachers consistently sought to identify known approaches from within their current learning design repertoire to connect their students and the new technology. In doing so, they consistently took an incremental approach and identified ways to redesign the known learning environment. At one level this reflects the early iterative stages suggested by transformation models such as SAMR (Puentedura 2006). However, the teachers do not seem to show any signs of the reconceptualisation level of that model. As we can see in those quotes, the Microsoft HoloLens was seen as an advance on video and PowerPoint, allowing a student to engage in independent study while the teacher worked with her class, and offering an improved visualisation of content such as the human circulatory system.
There were exceptions to the overall trend of the discussion. For example, a music teacher saw the potential for new learning design through the use of an app on human anatomy:
… [W]e were very interested in reminding ourselves of, of our [human body] structure – we’ve been talking a lot lately of posture and when we play music and it was very useful for the students to be able to see these models. It is hard to sort of visualise what is going on inside your body, you know.
These exceptions, however, were rare and the overall shape of the professional discussion reflected the reports of the students via survey. That is, the way teachers and students in the trial engaged with the new technology was almost always with reference to current learning design and did not align well with the theoretical orientation of the research team.
This finding is based on limited data. As we have outlined, this is the nature of the pilot phase of educational design research and our purpose is to widen the problem space rather than offer conclusive solutions at this stage. However, it seems to be a valid problem to explore. The teachers involved in this project were highly successful and well-respected, and the students were highly able too. These factors highlight that creative pedagogical design is extremely challenging, that the culture of schools conforms to well-understood modes of working and that it is not overly surprising that it has proven to be difficult to scale and sustain changes in enacted teaching and learning through educational design research projects (Fishman et al. 2011).
The goal of this pilot phase of an educational design research project is to open up the problem space through researchers and practitioners working together. The previous section provided just a brief example of this and suggested areas for consideration on future design and research in this project. In this section, we will reflectively expand on these ideas through ‘conversation’ with the literature.
To begin this conversation, we note that educational research on the use of MR technologies is really in its infancy. Despite AR/MR technologies being around for some decades, the widespread and affordable availability of these technologies is still emerging (Melatti and Johnsen 2017). As commercial solutions have become available, a number of reviews have been undertaken (see, e.g., Akçayır and Akçayır 2017; Bacca, Baldiris, and Fabregat 2014; Cheng and Tsai 2013; Dunleavy, Dede, and Mitchell 2009; Radu 2014; Sung, Yang, and Lee 2017). To some extent these reviews highlight the immature nature of research in this area. In a review of 32 studies on AR published in the top five educational technology journals, Bacca et al. (2014), for example, found that nearly all research reported was of case or pilot studies, with only one study adopting a causal research method. This review also found that nearly all of the learning designs reported were quite basic and focussed on the use of AR or MR to explain a topic or to provide additional information about a topic.
Despite the immature nature of the studies considered, Bacca et al. (2014) found a consistent reporting of positive impacts of AR and MR on both learning and motivation . These positive findings were consistent with another review by Radu (2014). Considering 56 studies, this analysis found AR/MR to have a positive impact on the learning of spatial structures and language associations, as well as contributing to improvements in long-term memory retention, collaboration and motivation. Both reviews, however, identified usability and classroom integration as continuing challenges across the field.
The findings of our pilot phase are consistent with those in the literature, in that students and teachers reported a belief that the technology supports engagement and learning, but also that there are usability issues and areas for technical improvement. Some of our partners also spontaneously suggested some of the benefits found in these studies, including motivation and collaboration. This spontaneous discernment, however, did not go beyond the enhancement of existing student practice or activity. Our students and teachers did not ponder the possibilities of improved memory retention, language associations or – with the exception of the music example we quoted – spatial structures.
Our research is translational. We seek to understand how advances in basic research can be applied to innovation and change in real-world settings. In this context, we must be mindful of this apparent gap between professional teacher focus and the cutting edge of the research. At the same time, we must also be aware that, at the cutting edge, a range of opportunities are being conceived. In another review of 87 studies, for example, Santos et al. (2014) found that AR/MR was effective in providing real-world annotation, contextual visualisation and vision-haptic visualisation. Notably, this analysis found that majority of the studies reviewed called upon multimedia learning theory (Mayer 2014) and experiential learning theory (Kolb 1984) in understanding the educational benefits of the technology. This suggests that designers are seeing the potential for more complex and activity-oriented learning designs. The activity they suggest, however, is quite different from the existing modes of practice that our teachers and students were able to suggest. This points to a need in our project to design not only new activity but also ways to support a different understanding of the nature of student and teacher activity.
Santos et al. (2014) analysis also shows a growing appreciation that the motivational benefits of AR are not driven by the novelty value of the ‘wow factor’ of technology alone. A number of studies in the review pointed also to the motivational effects of easing cognitive load and providing support for situated rather than decontextualised understandings. These findings are promising and have been confirmed in a more recent review by Akçayır and Akçayır (2017), who found evidence of AR/MR leading to heightened levels of motivation, engagement and interest; increased opportunities for interaction; decreases in cognitive load; and enabling the visualisation of abstract concepts. Studies in this review also found evidence of AR/MR leading to stronger interaction among students (Kamarainen et al. 2013), between students and learning materials (Hsiao, Chen, and Huang 2012), and between students and teachers (Zarraonandia et al. 2013).
Enyedy et al. (2017) highlight the social affordances of MR. In moving beyond a primary focus on technical implementation, and by working with young children in first and second grades, this study suggests that educational research in this area is beginning to mature. The study reports on a project that uses MR to support children in ‘doing science like scientists do’ through asking questions, modelling phenomena and arguing from the evidence. The study concluded that it was the ‘social aspects of the space, not the technical aspects alone’ that had the greatest impact on student learning. Also notable within the learning design was that the students had ‘agency to pursue their emergent goals and to decide when they believed they had achieved these goals’ (p. 2104).
From our perspective as translational researchers, the fact that our student and teacher partners in this project simply did not see opportunities such as enactment and agency in the technology is instructive. It points to a strong need for a research understanding of innovation in context.
A significant point of tension evident in current educational policy and practice can be summarised as being between a defined outcomes approach to learning and an expansive approach to learning. This is not a new debate. In the 4th Century BCE, the philosopher Isocrates argued that ‘pedagogy’ should develop obedience, conformity and controllability, while Plato argued for ‘philosophy’ or education which was about searching for truth, value and the meaning of life (Jones 2013). The Isocratic approach calls for certainty and known outcomes, while the Platonic approach requires uncertainty.
Today the Isocratic approach, pedagogy, dominates practice in formal education institutions in most parts of the world. In these contexts, learning design is typically driven by the design parameter of supporting the acquisition of clearly defined and highly specified learning outcomes. In the world beyond formal education, however, there is a growing call for this to change due to a recognition that the skills required to approach the social, environmental and economic challenges of the age require something more (see, e.g., Commonwealth of Australia – Department of the Prime Minister and Cabinet 2015). The academic response to this has been to develop a focus on transverse or 21st Century skills (Centre for Educational Research and Innovation 2008; Jang 2016; Wan Husin et al. 2016) including collaboration, communication and life-long learning. As we have seen in our pilot study, however, teachers and students are not turning to such ideas when they engage in the possibilities of a potentially transformative technology. This suggests a design and research problem for the project ahead – how do we support teachers and students to engage in the possibilities of the technology in an expansive rather than ‘pedagogic’ way?
It is beyond the scope of this article to elaborate a design and research methodology that could answer this question, but we would like to point to a theoretical direction that others might explore in similar work.That direction is the use of various theories on the social nature of knowledge creation. Cultural historical activity theory (CHAT, Engeström 1987; Roth and Lee 2007) brings focus to the roles, rules and tools of the learning environment. Similarly, the theory of practice architectures (Kemmis et al. 2014; Lowrie, Leonard, and Fitzgerald 2018) calls for consideration of ways of saying, doing and interacting as they relate to both the individual and the social world. This theoretical ensemble offers a refinement to the research and design question – does a design brief calling for teachers and students to consider how a new technology can change and improve the roles, rules, tools and ways of saying, doing and interacting lead to a different response than a design brief that asks for how a new technology can be used for teaching and learning?
Akçayır, M. & Akçayır, G. (2017) ‘Advantages and challenges associated with augmented reality for education: a systematic review of the literature’, Educational Research Review, vol. 20, pp. 1–11. doi: 10.1016/j.edurev.2016.11.002
Bacca, J., Baldiris, S. & Fabregat, R. (2014) ‘Augmented reality trends in education: a systematic review of research and applications’, Journal of Educational Technology and Society, vol. 17, no. 4, pp. 133–149.
Carijó, F. H., de Almeida, M. C. & Kastrup, V. (2013) ‘On haptic and motor incorporation of tools and other objects’, Phenomenology and the Cognitive Sciences, vol. 12, no. 4, pp. 685–701. doi:10.1007/s11097-012-9269-8
Centre for Educational Research and Innovation. (2008) 21st Century Learning: Research, Innovation and Policy Directions from recent OECD analyses. Available at: http://www.oecd.org/site/educeri21st/40554299.pdf
Cheng, K. H. & Tsai, C. C. (2013) ‘Affordances of augmented reality in science learning: suggestions for future research’, Journal of Science Education and Technology, vol. 22, no. 4, pp. 449–462. doi:10.1007/s10956-012-9405-9
Clark, A. (2011) ‘Finding the mind: book symposium on supersizing the mind: embodiment, action, and cognitive extension’, Philosophical Studies, vol. 152, no. 3, pp. 447–461. doi:10.1007/s11098-010-9598-9
Clark, D. B., Tanner-Smith, E. E. & Killingsworth, S. S. (2016) ‘Digital games, design, and learning: a systematic review and meta-analysis’, Review of Educational Research, vol. 86, no. 1, pp. 79–122. doi:10.3102/0034654315582065
Commonwealth of Australia - Department of the Prime Minister and Cabinet. (2015) National Innovation and Science Agenda. Available at: https://www.innovation.gov.au/system/files/case-study/National%20Innovation%20and%20Science%20Agenda%20-%20Report.pdf
Dunleavy, M., Dede, C. & Mitchell, R. (2009) ‘Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning’, Journal of Science Education and Technology, vol. 18, no. 1, pp. 7–22. doi:10.1007/s10956-008-9119-1
Engeström, Y. (1987) Learning by expanding: an activity-theoretical approach to developmental research, Orienta-Konsultit, Helsinki.
Enyedy, N., et al., (2017) ‘Social affordances of mixed reality learning environments: a case from the Science through Technology Enhanced Play project (STEP)’, 50th Hawaii International Conference on System Sciences, Hawaii.
Fishman, B. J., et al., (2011) ‘What happens when the research ends? Factors related to the sustainability of a technology-infused mathematics curriculum’, Journal of Computers in Mathematics and Science Teaching, vol. 30, no. 4, pp. 329–353.
Froese, T., et al., (2012) ‘The enactive torch: a new tool for the science of perception’,. IEEE Transactions on Haptics, vol. 5, no. 4, pp. 365–375. doi:10.1109/TOH.2011.57
Gee, J. P. (2006) ‘Why are video games good for learning’, Nordic Journal for Digital Literacy, vol. 3, pp. 172–183.
Goldin-Meadow, S. & Wagner, S. M. (2005) ‘How our hands help us learn’, Trends in Cognitive Sciences, vol. 9, no. 5, pp. 234–241. doi:10.1016/j.tics.2005.03.006
Hamilton, E. R., Rosenberg, J. M. & Akcaoglu, M. (2016) ‘The Substitution Augmentation Modification Redefinition (SAMR) Model: a critical review and suggestions for its use’, TechTrends, vol. 60, no. 5, pp. 433–441. doi:10.1007/s11528-016-0091-y
Hsiao, K. -F., Chen, N. -S. & Huang, S. -Y. (2012) ‘Learning while exercising for science education in augmented reality among adolescents’, Interactive Learning Environments, vol. 20, no. 4, pp. 331–349. doi:10.1080/10494820.2010.486682
Hugues, O., Fuchs, P. & Nannipieri, O. (2011) ‘New augmented reality taxonomy: technologies and features of augmented environment’, in Handbook of augmented reality, ed B. Furht, pp. 47–63. doi: 10.1007/978-1-4614-0064-62
Jang, H. (2016) ‘Identifying 21st Century STEM Competencies Using Workplace Data’, Journal of Science Education and Technology, vol. 25, no. 2, pp. 284–301. doi:10.1007/s10956-015-9593-1
Johnson-Glenberg, M. C. & Megowan-Romanowicz, C. (2017) ‘Embodied science and mixed reality: How gesture and motion capture affect physics education’, Cognitive Research: Principles and Implications, vol. 2, no. 1, pp. 1–28. doi:10.1186/s41235-017-0060-9
Jones, B. (2013) ‘Education as commodity?: how creativity fell off the agenda and labour market factors took over’, Available at: http://math.haifa.ac.il/yair/The-Funneled-Web-archive-(2001-2013)RIP/PDF_Documents/The%20Hon%20Dr%20Barry%20Jones%20-%20Education%20as%20commodity-final-TFW.pdf
Kamarainen, A. M., et al., (2013) ‘EcoMOBILE: integrating augmented reality and probeware with environmental education field trips’, Computers & Education, vol. 68 no. C, pp. 545–556. doi:10.1016/j.compedu.2013.02.018
Kelly, A. E., Baek, J. Y. & Lesh, R. A. (2008) Handbook of design research methods in education : innovations in science, technology, engineering, and mathematics, Routledge, New York.
Kemmis, S., et al., (2014) Changing practices, changing education, Springer, Singapore.
Kiverstein, J. & Clark, A. (2009) ‘Introduction: mind embodied, embedded, enacted: one church or many?’, Topoi, vol. 28, no. 1, pp. 1–7. doi:10.1007/s11245-008-9041-4
Kolb, D. A. (1984) Experiential learning: experience as The Source of Learning and Development (1st ed.). Prentice Hall, Inc, Upper Saddle River, NJ.
Leonard, S. N., Fitzgerald, R. N. & Riordan, G. (2016) ‘Using developmental evaluation as a design thinking tool for curriculum innovation in professional higher education’, Higher Education Research & Development, vol. 35, no. 2, pp. 309–321. doi:10.1080/07294360.2015.1087386
Leonard, S. N. & Roberts, P. (2014) ‘Performers and postulates: the role of evolving socio-historical contexts in shaping new teacher professional identities’, Critical Studies in Education, vol. 55, no. 3, pp. doi:10.1080/17508487.2014.904808
Logan, T., Lowrie, T. & Diezmann, C. (2014) ‘Co-thought gestures: supporting students to successfully navigate map tasks’, Educational Studies in Mathematics, vol. 87, no. 1, pp. 87–102. doi: 10.1007/s10649-014-9546-2
Lowrie, T., Leonard, S. N. & Fitzgerald, R. N. (2018) ‘STEM practices: a translational framework for large-scale STEM education design’, EDeR. Educational Design Research, vol. 2, no. 1. doi:10.15460/eder.2.1.1243
Lubinski, D. (2010) ‘Spatial ability and STEM: a sleeping giant for talent identification and development’, Personality and Individual Differences, vol. 49, no. 4, pp. 344–351. doi:10.1016/j.paid.2010.03.022
Mayer, R. E. (2014) ‘Cognitive theory of multimedia learning’, The Cambridge Handbook of Multimedia Learning, 2 Ed., Ed R. E. Mayer, Cambridge University Press, Cambridge, pp. 43–71.
McKenney, S. & Reeves, T. C. (2012) Conducting educational design research, Routledge, New York.
Melatti, M. & Johnsen, K. (2017) ‘Virtual reality mediated instruction and learning’, 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality ( KELVAR), Los Angeles, CA.
Merchant, Z., et al., (2014) ‘Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: a meta-analysis’, Computers & Education, vol. 7, pp. 29–40. doi: 10.1016/j.compedu.2013.07.033
Milgram, P. & Kishino, F. (1994) ‘A taxonomy of mixed reality visual displays’, IEICE Transactions on Information and Systems, vol. E77-D, pp. 1321–1329. doi:10.1.1.102.4646
Newcombe, N. S. (2010) ‘Picture this: increasing math and science learning by improving spatial thinking’, American Educator, vol. 34, no. 2, pp. 29–35. doi: 10.1016/j.cobeha.2016.04.010
Pan, M. K. & Niemeyer, G. (2017) ‘Catching a real ball in virtual reality’, 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR), pp. 269–270, Los Angeles, CA.
Pouw, W. T. J. L., et al., (2014) ‘Toward a more embedded/extended perspective on the cognitive function of gestures’, Frontiers in Psychology, vol. 5, no. 359, pp. 1–14. doi:10.3389/fpsyg.2014.00359
Puentedura, R. (2006) Transformation, Technology, and Education. Available at: http://hippasus.com/resources/tte/.
Radu, I. (2014) ‘Augmented reality in education: a meta-review and cross-media analysis’, Personal and Ubiquitous Computing, vol. 18, no. 6, pp. 1–11. doi: 10.1007/s00779-013-0747-y
Roth, W. M. & Lee, Y. -J. (2007) ‘“Vygotsky’s neglected legacy”: cultural-historical activity theory’, vol. 77, no. 2, pp. 186–232. doi:10.3102/0034654306298273
Santos, M. E. C., et al., (2014) ‘Augmented reality learning experiences: survey of prototype design and evaluation’, IEEE Transactions on Learning Technologies, vol. 7, no. 1, pp. 38–56. doi:10.1109/tlt.2013.37
Stieff, M. & Uttal, D. (2015) ‘How much can spatial training improve STEM achievement?’, Educational Psychology Review, vol. 27, no. 4, pp. 607–615. doi:10.1007/s10648-015-9304-8
Sung, Y. T., Yang, J. M. & Lee, H. Y. (2017) ‘The effects of mobile-Computer-Supported collaborative learning: meta-Analysis and critical synthesis’, Review of Educational Research, vol. 87, no. 4, pp. 768–805. doi: 10.3102/0034654317704307
Waddington, D. I. (2015) ‘Dewey and video games: from education through occupations to educations through simulations’, Educational Theory, vol. 65, no. 1, pp. 1–20.
Wan Husin, W. N. F., et al., (2016) ‘Fostering students' 21st century skills through Project Oriented Problem Based Learning (POPBL) in integrated STEM education program’, Asia-Pacific Forum on Science Learning & Teaching, vol. 17, no. 1, pp. 60–77.
Zarraonandia, T., et al., (2013) ‘An augmented lecture feedback system to support learner and teacher communication’, British Journal of Educational Technology, vol. 44, no. 4, pp. 616–628. doi:10.1111/bjet.12047