ORIGINAL RESEARCH ARTICLE
Learning experience design of verbal prompts in virtual reality-based training for autistic children
Jewoong Moon*
Department of Educational Leadership, Policy, and Technology Studies, The University of Alabama, Tuscaloosa, Alabama, United States
(Received: 7 July 2023; Revised: 10 November 2023; Accepted: 14 December 2023; Published: 7 March 2024)
This study aimed to explore the design and development of verbal prompts in virtual reality (VR)-based social skills training for autistic children. Autism indicates a category with neurodiversity that influences individuals’ capability to engage in social and cognitive tasks. This complex neurodevelopmental condition manifests in a wide array of patterns, featuring unique experiences of each individual. This study explored both advantages and challenges encountered when autistic children interact with verbal prompts in multi-user, desktop VR-based social skills training. Our explanatory case study involved VR-based learning experiences of four autistic children. We used a qualitative thematic analysis to analyse the study participants’ interaction patterns with verbal prompts in the VR-based training. Our research can contribute to both theoretical knowledge and practical design guidelines for the creation of verbal prompts in desktop VR-based training programmes tailored for autistic children.
Keywords: autism spectrum disorder; virtual world, social skills; verbal prompts; learning experience design
*Corresponding author. Email: jmoon19@ua.edu
Research in Learning Technology 2024. © 2024 J. Moon. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2024, 32: 3129 - http://dx.doi.org/10.25304/rlt.v32.3129
Introduction
Autism is a diverse and complex neurodevelopmental condition that can affect how an individual communicates and interacts with others, as well as their behaviour patterns (Newman et al., 2000). Although there is an argument why autism occurs, research presumably considers that both genetics and the environment play a role (Bai et al., 2019). Research states that autistic learners generally undergo verbal and nonverbal communication difficulties (Yoder et al., 2013). This also relates to autistic learners’ social interaction challenges that involve both forming friendships and identifying social cues and norms (Cresswell et al., 2019; Heerey et al., 2003). In addition, autistic learners largely experience attention and focus deficits when gathering, collecting and synthesizing information (Allen & Courchesne, 2001).
In recent years, virtual reality (VR) has been widely used as a popular tool for supporting autistic learners, especially in the area of social and cognitive skills training (Herrero & Lorenzo, 2020; Lorenzo et al., 2023). VR has been advantageous to creating a safe and controlled learning space that allows autistic learners to practice their social skills (Dixon et al., 2020). Previous studies reported ways to integrate desktop VR into therapeutic and training interventions for autistic learners (Bradley & Newbutt, 2018; Didehbani et al., 2016; McCleery et al., 2023). Desktop VR involves exploring 3D environments via a standard computer screen. Such uses of VR have presented full potential in providing in-situ and meaningful learning experiences and support for autistic learners.
While VR-based social skills training has become widely adopted over time (Chen et al., 2022), there is still a need for empirical research (Barletta et al., 2023; Moon et al., 2023; Schmidt & Glaser, 2021) to explore the optimized learning experience design for autistic learners (Glaser et al., 2022) when they interact with verbal prompts during VR-based training sessions. Given that autistic learners often experience high levels of social anxiety and challenges in understanding social norms (Briot et al., 2020), instructional support that guides their attention and provides a clue of social interaction behaviours in social skills training is critical. Specifically, it is crucial to explore the optimized design of prompts that delivers and selects educational and therapeutic support for autistic learners. The use of prompts – instructional techniques designed to trigger desired actions and responses – has been widely adopted in autism studies (MacDuff et al., 2001). Prompts can take various forms, including visual, verbal and modelling prompts (Jones & Zarcone, 2014; Olson et al., 2015).
In comparison to other prompting approaches, verbal prompts have been shown to be especially effective in training interventions for individuals with autism, thanks to their flexibility and immediacy (Drasgow et al., 1998; Koegel et al., 1992). They allow for precise, adaptable instruction that can meet the complex and individual needs of learners. However, the specific application of verbal prompts in VR-based training settings remains underexplored. Given the individual differences among autistic children, understanding their unique developmental characteristics is crucial in designing effective verbal prompts for VR-based social skills training. This study aims to address this gap by asking the following questions: What influence does the use of verbal prompts within a VR-based social skills training programme have on the social interaction behaviours of middle school autistic children? How does it affect their understanding of social cues? Additionally, what impact does it have on their overall engagement? And how can these prompts be optimized to individual learner needs?
Literature review
Virtual reality-based training for autistic individuals
Social skills training, as part of empowering learning approaches, has been generally used to enhance autistic individuals’ social competence (Laugeson et al., 2012). Social competence involves individuals’ capability to understand complex or implicit social cues and perform responsive and appropriate social skills tailored to dynamic social circumstances and their change. This training aims at nurturing each person’s unique capacity to understand a variety of social cues and contexts that include the dynamic nature of social interactions. As a prominent theoretical account for autism research, theory of mind (ToM) explains that individuals have the ability to keep various mental states – beliefs, emotions and intentions that are different than their own. ToM suggests how to understand strategies and support that can empower autistic individuals to navigate social interactions tailored to their needs or preferences (Heerey et al., 2003; Leslie et al., 2004). Research suggests that autistic individuals may often experience social interactions differently (Heerey et al., 2003; Szumski et al., 2019), and these differences are important in shaping how social skills training is approached. The goal here is to construct a learning environment that highly values these differences, offering chances to practice and develop social behaviours and norms that are responsive to interactions with various people.
In decades, virtual reality (VR) has been widely serving as a digital platform for social skills training for autistic individuals (Didehbani et al., 2016; Ke & Im, 2013; Ke et al., 2022; Lorenzo et al., 2023; Schmidt & Glaser, 2021). The benefits of VR include the ability to customize and personalize interventions that can foster autistic individuals’ engagement and personal interests. This adaptability of VR not only aligns with each individual’s unique learning pace and preferences but also cultivates their engagement. 3D simulations through VR provide realistic social scenarios that allow autistic individuals to identify, practice and evaluate social interaction behaviours in safe and controlled settings. Due to its benefits, VR has been particularly used for various therapeutic settings. In autism research, social scenarios can be built and delivered to emulate real-life situations that autistic individuals could find challenging, such as crowded public places, social gathering that require interpersonal conversations under a low-stress and controlled environment.
Despite the wide use of VR in social skills training, there is a design challenge that autistic individuals are likely to undergo sensory overload that results in distractions (Jones et al., 2020). Accordingly, to overcome this challenge, additional support may be necessary to guide individuals’ attention and ensure they are able to perform social interaction skills as intended in the VR-based learning environment. In particular, VR has a great benefit in social skills training for autistic children. Autistic individuals often experience social communication difficulties, alongside restricted and repetitive behaviours. Lahiri et al. (2015) report that VR reduces the anxiety among autistic learners. They observed that a 3D virtual world may help learners tackle stressful scenarios without the fear of real-world consequences. Ke and Im (2013) also found that VR was useful to promote the social skills development of high-functioning autistic children.
Verbal prompts in social skills training for autistic individuals
In educational research, verbal prompts refer to verbal cues or hints given by an instructor to guide a student’s desirable behaviours (Green, 2001; Meadan et al., 2013). Prompts can take many forms, including questions, statements and reminders (Chi et al., 2001; Chi & Wylie, 2014). Prompts can be explicit statements that call for an individual’s specific action, such as ‘Please, sit down and share your toys with your friend’. Alternatively, prompts can be designed as open-ended questions to foster deeper thinking and problem-solving. For example, ‘How would you solve this problem? What might happen if you try option A?’ The aim of prompts is to provide a structured template and support for learners and keep them focused on the task at hand (Milley & Machalicek, 2012). Even in computer-based instruction settings, verbal prompts have been used to engage students in interactions with computers by guiding their attention during ongoing tasks (Goldsmith & LeBlanc, 2004; Lorah et al., 2015). Verbal prompts may be delivered through an instructor’s narration or text in a computer system that reminds students of the subsequent actions that should be performed.
In general, prompts have been widely utilized in training interventions for autistic learners (Aljehany & Bennett, 2020; Katz, 2021; Sundberg et al., 2000). Research reports that prompts can be delivered through methods such as video modelling and direct instruction in interventions for autistic learners (Bell, 2022; McCoy & Hermansen, 2007). Especially, among various types of prompts, verbal prompts have been widely used thanks to its benefit in effectively providing structured and immediate guidance on behaviours in various simulated social contexts. Despite the widespread use of verbal prompts in autism research, there appears to be limited empirical research on how such support mechanisms can be integrated into VR-based training for autistic learners (Maples-Keller et al., 2017). Existing literature primarily focuses on verbal prompts in in-person intervention settings, leaving a gap in understanding their application within a virtual world. Hence, research exploring how to effectively design, deliver and contextualize verbal prompts in VR-based training is needed. Understanding the specific interaction needs and learning preferences of autistic learners can enhance the quality of verbal prompts, thereby increasing the effectiveness of VR-based training.
Learning experience design framework
Learning Experience Design (LXD) is an interdisciplinary field that draws from various domains, including human-computer interaction, product and software design, and instructional design. Its core principles are derived from user experience design, a discipline that, despite its roots outside of academia, has significant implications for learning contexts (Law & Sun, 2012). With a strong focus on the learner’s background, preferences and instruction context, LXD aims to create meaningful and effective learning experiences. The LXD framework emphasizes the creation of learning experiences that intricately consider the practical realities and unique needs of learners. Unlike traditional Instructional Design (ID), which primarily focuses on achieving learning objectives, LXD extends beyond mere objective attainment to understand and enrich the entirety of learners’ experiences (Chang & Kuwata, 2020; Schmidt & Huang, 2022).
LXD takes into account the necessity for intuitive, user-friendly interfaces in both synchronous and asynchronous digital learning settings. It focuses on learner-centric design mitigating potential intimidation from technology, encouraging ease-of-use and fostering a more effective, self-guided learning experience. In the context of VR-based training for autistic individuals, by applying the principles of LXD, we can create personalized virtual environments that are engaging, relevant and accessible for autistic learners. For instance, LXD can guide the development of custom learning paths tailored to individual skill levels and interests. It can assist in the incorporation of multi-sensory elements that heighten engagement such as interactive tasks, auditory cues and vivid graphics. LXD can also facilitate the integration of support mechanisms like personalized verbal prompts that are adaptive to the learner’s responses. Lastly, LXD can ensure accessibility by designing controls and navigation that consider diverse motor abilities and cognitive features, thereby making the virtual environment truly inclusive for autistic learners. Glaser et al. (2022) designed and refined a 3D collaborative VR environment ‘Museum of Instructional Design (MID)’ supported by a Mozilla Hubs platform. The results showed the effectiveness of this iterative approach, with marked improvement in user engagement, usability and learning outcomes. These study findings highlight the effectiveness and usefulness of applying LXD framework to craft VR-based interventions for autistic learners (Glaser et al., 2022).
Methods
This study’s context, procedure and findings are organized based on the qualitative evidence synthesis framework SPIDER (Methley et al., 2014). This framework structures the analysis findings with following components: (1) sample, (2) phenomenon of interest, (3) design, (4) evaluation and (5) research type.
Research type
The present study adopted an explanatory case study design. This research design was chosen because it is advantageous to taking in-depth explorations of a specific phenomenon in a real-world implementation context (Ogawa & Malen, 1991). Considering previous studies that successfully applied LXD-related lens (Glaser et al., 2022; Schmidt & Huang, 2022; Vann & Tawfik, 2020), we conducted qualitative thematic analysis to yield implications on prompt design with following components: (1) engagement, (2) learning outcome, (3) usability and technology. Especially, this study aimed to uncover design implications on both challenges and promoting features on usability in using verbal prompts in VR-based training.
Participants
Aligned with a typical case sampling (convenience sampling), we recruited four school-age autistic children, aged 12–13 years and in grades 7–8, for this study. Purposive sampling was used to choose the participants, who were chosen from that satisfied the inclusion requirements for our study (Etikan et al., 2016). I intentionally included the study participants based on specific developmental characteristics they possess. Table 1 has a complete profile of each participant. All participants in this study were autistic children, characterized by being capable of literacy, despite facing challenges in social interaction performance. An orientation meeting took place before the study in order to secure parental consent and each participant’s agreement to participate in the study. The study was performed in accordance with the American Psychological Association’s (2002) ‘Ethical Principles of Psychologists and Code of Conduct’ and was approved by the Institutional Review Board. Each participant received a $25 monetary incentive as compensation for their participation.
Role of the researcher
The researcher of this study took on a pivotal role spanning from the design and execution of the study to the subsequent data analysis and interpretation. During the VR-based training activities, the researcher functioned primarily as an observer, tracking the performance of participants during social interaction tasks. To foster transferability, the researchers have provided detailed descriptions of the context and the participants in the study. To gain a comprehensive understanding of the participants’ activities, the observation notes were cross-checked with other training facilitator.
Procedures
This study began by setting up Opensimulator, a desktop-based VR platform, with several iterative adjustments via design-based research for autism studies (Ke et al., 2022; Ke & Im, 2013). We then established a VR-based social skills training programme within the VR platform, designed to present a naturalistic intervention tailored to individual interests. We designed the VR platform to allow participants to create, design and engage with 3D objects and simulations. In addition, they were exposed to 3D social interaction simulations with non-playable characters (NPCs) or facilitators. Training sessions were primarily focused on social interaction tasks, which were designed to enhance participants’ social skills. Two graduate students served as facilitators for these sessions, controlling NPCs and managing automated verbal prompts. Sessions usually occurred weekly, but some students attended twice per week. The intervention period ranged from 4 to 8 weeks. This structure was maintained consistently from the first to the eighth session, with the content and tasks evolving based on the learning objectives and progress of the participants.
Design
The researcher of this study designed a default interface of verbal prompts, which appeared as a blue dialogue window in the Opensimulator, as illustrated in Figures 1 and 2. The verbal prompts were triggered to appear at the top right corner of the study participant’s VR viewer screen and consisted of text messages and clickable buttons. The system-paced prompts were timed to appear at the start of each phase of the training session and included a preset number of prompt layers (n = 3). Study participants were able to access additional prompts by clicking the ‘More’ button if needed. The prompts were designed to stimulate the recall of appropriate social behaviours and verbal responses and delineate subsequent actions. Aligned with the particular social scenario under focus, these prompts were delivered across three stages, each limited to 5 min. Each training session embraced a cyclical design centred on a single social interaction task, fostering continuous engagement, learning and improvement.
Figure 1. An example of a verbal prompt: First to third layer.
Figure 2. A screenshot of a blue dialogue box as a verbal prompt.
Data analysis
For the purpose of data analysis, this study meticulously collected a rich variety of data sources from the participants. This encompassed behavioural observation records, carefully documented and analysed, as well as discourse data, facilitating a deeper understanding of the learners’ experiences and interactions within the VR environment. Study participants’ voice and character behaviours were screen- and video-recorded by Openbroadcaster. Qualitative thematic analysis was then conducted to collect and interpret evidence of design challenges and suggest design features that could be useful for the next round of verbal prompt design in VR-based training. Thematic analysis was chosen because it can identify and analyse data to uncover patterns and gain valuable insights from the data.
Assuring credibility and trustworthiness
In this study, the researcher meticulously described the participants, the setting, and the intervention process to facilitate an understanding of how the findings might transfer to other contexts. The credibility of the study was strengthened by a triangulation approach, utilizing varied data sources, including behavioural observation records and discourse data. A codebook was developed as a reference guide to ensure consistency in the interpretation of observed behaviours. The codebook provided clear definitions and examples of the behaviours of interest, serving as a standardized framework for the coding process.
Findings
In this study, the thematic analysis yielded enriched and detailed behaviour observation findings, focused on LXD evaluation components as follows: (1) engagement, (2) learning outcome and (3) usability and technology.
Engagement
Based on the observation records, all participants actively engaged with the verbal prompts provided throughout the study. Notably, each participant demonstrated an increased confidence in initiating conversations with NPCs across various social scenarios. For instance, Participant 1 initiated dialogues in 90% of the scenarios, closely followed by Participant 2 at 75%. Similarly, Participants 3 and 4 demonstrated a positive trend, initiating conversations in 70% and 60% of the scenarios, respectively. In terms of each participant’s behaviours, Participant 1, initially showed signs of social apprehension such as minimal eye contact (rotating the avatar against other characters) and reluctance to initiate conversations, underwent a significant transformation. By the study’s conclusion, this participant was initiating dialogues in 90% of the scenarios. This change was particularly evident in a virtual design project scenario, where Participant 1 confidently took on a leadership role. Participant 2, initially more reserved, exhibited a gradual increase in engagement within the virtual marketplace scenario, successfully navigating complex bargaining interactions, a task they previously found challenging. Their virtual avatar tends to become shifted from closed to more open and inviting postures, possibly indicating increased comfort and engagement in the virtual setting. Participants 3 and 4, starting from a higher baseline of virtual social confidence, demonstrated enhanced engagement in more nuanced ways. For example, Participant 3, who initially struggled with understanding virtual social cues and humour, was observed making jokes during the scenario. Participant 4 occasionally showed assertiveness in group discussions, often leading conversations. A majority, three out of four, actively relayed their messages to NPCs across the scenarios, illustrating the prompts’ effectiveness. Overall, the verbal prompts acted as behavioural aids, encouraging participants to respond appropriately to NPCs’ behaviours. For example, when NPCs exhibited a negative mood as a task challenge, these prompts effectively guided all participants in responding with comforting social cues. This highlights the potential of prompts in modulating social interaction behaviours.
Learning outcome
Attention shifting/inhibition control
According to the observation records, the data suggests that the verbal prompts may have played a pivotal role in guiding study participants (i.e. all autistic children) – in managing their attention shifting and inhibition, perhaps improving their social skills. The prompts appeared to foster an increased awareness of different social cues and environmental contexts. However, more analysis is required to corroborate this conclusion. One example of this was a situation in a virtual cafeteria, which illustrates a complicated social setting complete with multiple cues and interactions. Here, verbal prompts seemed to assist study participants in making better sense of the specifics of the issue, possibly assisting them in resolving future social problems. Tasks included joining a table, initiating conversation, correctly responding to social cues, and resolving a social conflict.
Among the four participants, for example, two appeared initially to find it challenging to engage their peers in conversation in the cafeteria scenario. However, the introduction of verbal prompts appeared to encourage more considered responses. They looked to think for a moment before responding, and their answers came across as more polite. Specifically, Participants 1 and 2 successfully performed social interactions responding to all phases of social interaction tasks, while Participants 3 and 4 managed most of them, respectively.
The prompts appeared particularly beneficial for those learners who had difficulties with verbal communication. Three out of the four participants suggested improvements in their communication and social skills during roleplay scenarios such as the waiter and interview roleplay. However, further investigations would be necessary in determining the extent of the verbal prompts’ effectiveness in complex social settings, a focus area of our learning outcomes.
Potential dynamics between verbal prompts and social scenarios
The effectiveness of verbal prompts in our study was observed to vary depending on the specific social scenarios. This variation highlights the nuanced relationship between the nature of the task and the effectiveness of the prompts. In scenarios that require active group discussions, or conflict resolution with NPCs, verbal prompts proved to be beneficial in fostering interpersonal communication skills for all four study participants. For example, in a virtual cafeteria scenario, the study participants were required to discuss conflicting arguments among peers and reach a consensus through negotiation approaches. Here, the verbal prompts guided the participants not only in being stepwise to conversations but also listening constructively to others’ thoughts. This was particularly evident in the way that the study participants used prompts to frame their action and responses aligned with the idea of turn-taking.
However, instances calling for complex cognitive processing – faced by two study participants across three separate scenarios – showed these prompts to be somewhat overwhelming. In the 3D gallery presentation scenario, for example, where study participants had to present their gallery ideas to a group of virtual visitor NPCs, verbal prompts appeared to successfully guide three out of four study participants in articulating design and engaging with the virtual audience. However, in the collaborative 3D design scenario, study participants performed tasks with coordinating with NPCs to design and construct structures within a virtual Legoland environment. In this case, verbal prompts were shown to be less useful since study participants needed to concentrate on the work at hand and found the prompts distracting.
These observations highlight the significance of adaptive verbal prompts to the task difficulty of different virtual scenarios. While prompts could largely aid social interaction and communication in certain contexts, their utility may be less effective in situations that require high levels of attention. This result echoes the potential need for adaptive prompting systems that can adjust the frequency or complexity of prompts based on the nature of tasks and learner engagement levels.
Usability and technology
Verbal literacy
Three of the four participants occasionally struggled with reading and processing text in verbal prompts, making it difficult for them to successfully react to verbal cues. This is especially difficult in the context of VR-based social skills training when learners are expected to participate in complicated social interactions with NPCs. For autistic individuals, who might often have unique needs, the difficulty of these challenges could be amplified. In certain circumstances, autistic individuals might choose to communicate with NPCs using text chat rather than speaking. Despite all NPC conversation being conducted via text chat, some participants struggled with understanding and comprehending the implicit meaning of the information presented. This was not necessarily a reflection of their overall literacy level, as all study participants were considered to have good literacy skills. However, those with slower processing rates, attention difficulties, or challenges with working memory might find it challenging to keep pace with the dynamic and sometimes unpredictable nature of virtual interactions. The delay in processing textual information often led to missed opportunities in the conversation tasks, which could potentially impact the effectiveness of the prompts and social-skills training.
Zoom functions in desktop-based VR settings
Another significant challenge observed in the study was related to the usability of zoom functions in the desktop-based VR environment. Some autistic participants, particularly those with less dexterity in mouse handling, faced difficulties in effectively navigating the zoom in and out features. While these features are designed to provide a comprehensive understanding of the virtual world, they can be counterproductive if not user-friendly for all learners.
Study participants who faced difficulty in fine motor skills found using the mouse to manage zoom features particularly problematic. This often resulted in frustration and disengagement from their learning experiences. For example, when a study participant attempted to zoom in for a closer view of the object or NPC, they sometimes found it challenging to navigate back to the broader view resulting in missing out on important verbal prompts or interactions that might happen in the virtual world. This issue was exacerbated under the remote training setting of the current study, where the facilitator’s assistance to provide technical support was limited. The inability to troubleshoot the zoom-related issues swiftly and appropriately might further interrupt the autistic children’s learning experiences.
Discussion and conclusion
Implications
Academic implications
The current study findings can contribute to the academic understanding of VR-based training for autistic learners, particularly in the context of using verbal prompts to enhance social interaction behaviours. This preliminary research’s findings are aligned with previous studies (Moon & Ke, 2023; Vedora & Conant, 2015), reinforcing the notion that verbal prompts are instrumental in aiding autistic learners to swiftly and properly respond to social cues in varied VR social simulations. The effectiveness of verbal prompts in activating proper social interaction behaviours across different social contexts (Jung & Sainato, 2015) is a notable observation, indicating the potential of adaptive instructional strategy design in an open-ended, immersive learning environment. In addition, the observation that found the increase in positive social interactions among study participants in the training suggests that verbal prompts function more than just triggering desirable responses; they may also shape the quality and outcome of the interactions. This finding is important because it presents the importance of structured and well-presented prompts in VR-based training environments for autistic learners. Additionally, the study finding also contributes to a broader understanding of autistic children’s learning processes, echoing the need for adaptive learning systems. The variability in participant responses to verbal prompts highlights the diverse learning needs, suggesting that further studies consider various design techniques to make training flexible and adaptable to individual learners.
Design implications
The study findings have significant design implications for VR-based training programmes. First, the study highlights the necessity of adapting verbal prompts to individual learners’ specific social contexts and developmental characteristics. The diversity inherent to autistic children necessitates careful consideration of several factors when delivering the prompts. These factors involve the complexity of simulated social context, task difficulty, and the learner’s individual needs and preferences (Parsons, 2015). These learners demonstrate a wide range of interests and levels of verbal literacy that could influence their comprehension of the training’s verbal and social cues. Therefore, verbal prompts’ design should be adaptable (Moon & Ke, 2023), aligning with each learner’s unique needs. Adopting such adaptive prompts, capable of harnessing learners’ diverse abilities, might be an effective strategy to boost engagement and effectiveness in a VR-based social skills training programme for autistic children. This approach could potentially pave the way for more inclusive and personalized learning experience designs and critical factors catering to neurodiverse learners (Moon et al., 2023; Parsons, 2016).
Secondly, the study raised the important technical considerations for remote VR training. The challenges faced by study participants in navigating technical features like zoom functions in desktop-based VR settings highlight the need for more user-friendly design and technical support. Trainers and facilitators should be equipped with comprehensive protocols, including alternative communication methods, to assist users in overcoming any technical challenges encountered during remote VR sessions (Adiani et al., 2022). This aspect, while a broader technical concern, is vital for ensuring the quality and accessibility of VR-based training programmes for autistic children.
Limitation and future research
While this study’s findings suggest that the delivery of verbal prompts in VR-based social skills training holds potential in fostering autistic individuals’ social skills practice, there are notable limitations to be considered in the future. First, one of the limitations of this study is the small sample size, specifically focused on four autistic children. This narrow age range limits the generalizability of our findings, as the effectiveness and impact of interventions like ours can vary widely across different developmental stages. Future studies should consider a broader and larger sample of study participants, encompassing various age groups. It is believed that the expansion of the sample can help to understand how age-related developmental differences of autistic children appear in VR-based training. Moreover, the study finding is limited in investigating the transfer effect of verbal prompts, which is connected to the concern of over-reliance on prompts (Van Laarhoven et al., 2018) in social skills training in autism research. In other words, this study primarily investigated the immediate impact of verbal prompts without thoroughly examining their long-term impact on the transfer of social skills to their daily lives. This is a crucial aspect, indicating the potential concern of prompt over-reliance in social skills training. Future research should aim to observe and examine the effects of verbal prompts in VR-based training on delayed transfer of social skills.
References
| Adiani, D. et al. (2022). Career interview readiness in virtual reality (CIRVR): a platform for simulated interview training for autistic individuals and their employers. ACM Transactions on Accessible Computing (TACCESS), 15(1), 1–28. https://doi.org/10.1145/3505560 |
| Aljehany, M. S., & Bennett, K. D. (2020). A comparison of video prompting to least-to-most prompting among children with autism and intellectual disability. Journal of Autism and Developmental Disorders, 50, 1714–1724. https://doi.org/10.1007/s10803-019-03929-x |
| Allen, G., & Courchesne, E. (2001). Attention function and dysfunction in autism. Frontiers in Bioscience-Landmark, 6(3), 105–119. https://doi.org/10.2741/allen |
| American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073. https://doi.org/10.1037/0003-066X.57.12.1060 |
| Bai, D. et al. (2019). Association of genetic and environmental factors with autism in a 5-country cohort. JAMA Psychiatry, 76(10), 1035–1043. https://doi.org/10.1001/jamapsychiatry.2019.1411 |
| Barletta, V. S. et al. (2023). Serious games for autism based on immersive virtual reality: a lens on methodological and technological challenges. In Temperini, M. et al. Methodologies and Intelligent Systems for Technology Enhanced Learning, 12th International Conference. MIS4TEL 2022. Lecture Notes in Networks and Systems (pp. 181–195). Springer, Cham. https://doi.org/10.1007/978-3-031-20617-7_23 |
| Bell, B. (2022). Using video modeling to teach abduction-prevention skills to children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 52(9), 3909–3918. https://doi.org/10.1007/s10803-021-05241-z |
| Bradley, R., & Newbutt, N. (2018). Autism and virtual reality head-mounted displays: a state of the art systematic review. Journal of Enabling Technologies, 12(3), 101–113. https://doi.org/10.1108/JET-01-2018-0004 |
| Briot, K. et al. (2020). Social anxiety in children and adolescents with autism spectrum disorders contribute to impairments in social communication and social motivation. Frontiers in Psychiatry, 11, 710. https://doi.org/10.3389/fpsyt.2020.00710 |
| Chang, Y. K., & Kuwata, J. (2020). Learning experience design: challenges for novice designers. In Schmidt, M., Tawfik, A. A., Jahnke, I. & Earnshaw, Y. (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. Retrieved from https://edtechbooks.org/ux/LXD_challenges |
| Chen, J. et al. (2022). Virtual reality enhances the social skills of children with autism spectrum disorder: a review. Interactive Learning Environments, 1–22. https://doi.org/10.1080/10494820.2022.2146139 |
| Chi, M. T. et al. (2001). Learning from human tutoring. Cognitive Science, 25(4), 471–533. https://doi.org/10.1207/s15516709cog2504_1 |
| Chi, M. T., & Wylie, R. (2014). The ICAP framework: linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823 |
| Cresswell, L., Hinch, R., & Cage, E. (2019). The experiences of peer relationships amongst autistic adolescents: a systematic review of the qualitative evidence. Research in Autism Spectrum Disorders, 61, 45–60. https://doi.org/10.1016/j.rasd.2019.01.003 |
| Didehbani, N. et al. (2016). Virtual reality social cognition training for children with high functioning autism. Computers in Human Behavior, 62, 703–711. https://doi.org/10.1016/j.chb.2016.04.033 |
| Dixon, D. R. et al. (2020). Evaluation of an immersive virtual reality safety training used to teach pedestrian skills to children with autism spectrum disorder. Behavior Analysis in Practice, 13, 631–640. https://doi.org/10.1007/s40617-019-00401-1 |
| Drasgow, E., Halle, J. W., & Ostrosky, M. M. (1998). Effects of differential reinforcement on the generalization of a replacement mand in three children with severe language delays. Journal of Applied Behavior Analysis, 31(3), 357–374. https://doi.org/10.1901/jaba.1998.31-357 |
| Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11 |
| Glaser, N. et al. (2022). Formative design and development of a three-dimensional collaborative virtual learning environment through learning experience design methods. Journal of Formative Design in Learning, 6, 63–76. https://doi.org/10.1007/s41686-022-00072-2 |
| Goldsmith, T. R., & LeBlanc, L. A. (2004). Use of technology in interventions for children with autism. Journal of Early and Intensive Behavior Intervention, 1(2), 166–178. https://doi.org/10.1037/h0100287 |
| Green, G. (2001). Behavior analytic instruction for learners with autism: advances in stimulus control technology. Focus on Autism and Other Developmental Disabilities, 16(2), 72–85. https://doi.org/10.1177/108835760101600203 |
| Heerey, E. A., Keltner, D., & Capps, L. M. (2003). Making sense of self-conscious emotion: linking theory of mind and emotion in children with autism. Emotion, 3(4), 394–400. https://doi.org/10.1037/1528-3542.3.4.394 |
| Herrero, J. F., & Lorenzo, G. (2020). An immersive virtual reality educational intervention on people with autism spectrum disorders (ASD) for the development of communication skills and problem solving. Education and Information Technologies, 25, 1689–1722. https://doi.org/10.1007/s10639-019-10050-0 |
| Jones, A. S., & Zarcone, J. R. (2014). Comparison of prompting strategies on two types of tasks with children diagnosed with autism spectrum disorders. Behavior Analysis in Practice, 7, 51–60. https://doi.org/10.1007/s40617-014-0010-7 |
| Jones, E. K., Hanley, M., & Riby, D. M. (2020). Distraction, distress and diversity: exploring the impact of sensory processing differences on learning and school life for pupils with autism spectrum disorders. Research in Autism Spectrum Disorders, 72, 101515. https://doi.org/10.1016/j.rasd.2020.101515 |
| Jung, S., & Sainato, D. M. (2015). Teaching games to young children with autism spectrum disorder using special interests and video modelling. Journal of Intellectual and Developmental Disability, 40(2), 198–212. https://doi.org/10.3109/13668250.2015.1027674 |
| Katz, S. (2021). Utilizing GoogleMeets™ to implement prompting strategies during instruction for students with autism spectrum disorder. Journal of Special Education Technology, 36(2), 113–117. https://doi.org/10.1177/0162643421992702 |
| Ke, F., & Im, T. (2013). Virtual-reality-based social interaction training for children with high-functioning autism. The Journal of Educational Research, 106(6), 441–461. https://doi.org/10.1080/00220671.2013.832999 |
| Ke, F., Moon, J., & Sokolikj, Z. (2022). Virtual reality–based social skills training for children with autism spectrum disorder. Journal of Special Education Technology, 37(1), 49–62. https://doi.org/10.1177/0162643420945603 |
| Koegel, R. L., Koegel, L. K., & Surratt, A. (1992). Language intervention and disruptive behavior in preschool children with autism. Journal of Autism and Developmental Disorders, 22(2), 141–153. https://doi.org/10.1007/BF01058147 |
| Lahiri, U. et al. (2015). A physiologically informed virtual reality based social communication system for individuals with autism. Journal of Autism and Developmental Disorders, 45, 919–931. https://doi.org/10.1007/s10803-014-2240-5 |
| Laugeson, E. A. et al. (2012). Evidence-based social skills training for adolescents with autism spectrum disorders: the UCLA PEERS program. Journal of Autism and Developmental Disorders, 42, 1025–1036. https://doi.org/10.1007/s10803-011-1339-1 |
| Law, E. L. C., & Sun, X. (2012). Evaluating user experience of adaptive digital educational games with Activity Theory. International Journal of Human-Computer Studies, 70(7), 478–497. https://doi.org/10.1016/j.ijhcs.2012.01.007 |
| Leslie, A. M., Friedman, O., & German, T. P. (2004). Core mechanisms in ‘theory of mind’. Trends in Cognitive Sciences, 8(12), 528–533. |
| Lorah, E. R. et al. (2015). A systematic review of tablet computers and portable media players as speech generating devices for individuals with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 3792–3804. https://doi.org/10.1007/s10803-014-2314-4 |
| Lorenzo, G. G., Newbutt, N. N., & Lorenzo-Lledó, A. A. (2023). Designing virtual reality tools for students with Autism Spectrum Disorder: a systematic review. Education and Information Technologies, 28, 1–49. https://doi.org/10.1007/s10639-022-11545-z |
| MacDuff, G. S., Krantz, P. J., & McClannahan, L. E. (2001). Prompts and prompt-fading strategies for people with autism. In Maurice, C., Green, G. & Foxx, R. M. (Eds.), Making a Difference: Behavioral Intervention for Autism (pp. 37–50). PRO-ED. |
| Maples-Keller, J. L. et al. (2017). The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harvard Review of Psychiatry, 25(3), 103. https://doi.org/10.1097/HRP.0000000000000138 |
| McCleery, J. P. (2023). The Oxford Handbook of Autism and Co-occurring Psychiatric Conditions (Oxford Library of Psychology) (p. 443). New York, NY: Oxford University Press. |
| McCoy, K., & Hermansen, E. (2007). Video modeling for individuals with autism: a review of model types and effects. Education and Treatment of Children, 30(4), 183–213. https://doi.org/10.1353/etc.2007.0029 |
| Meadan, H. et al. (2013). How can I help? Prompting procedures to support children’s learning. Young Exceptional Children, 16(4), 31–39. https://doi.org/10.1177/1096250613505099 |
| Methley, A. M. et al. (2014). PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Services Research, 14(1), 1–10. https://doi.org/10.1186/s12913-014-0579-0 |
| Milley, A., & Machalicek, W. (2012). Decreasing students’ reliance on adults: a strategic guide for teachers of students with autism spectrum disorders. Intervention in School and Clinic, 48(2), 67–75. https://doi.org/10.1177/1053451212449739 |
| Moon, J., & Ke, F. (2023). Effects of adaptive prompts in virtual reality-based social skills training for children with autism. Journal of Autism and Developmental Disorders, 1–21. https://doi.org/10.1007/s10803-023-06021-7 |
| Moon, J., Choi, G. W., & Seo, J. Y. (2023). Revisiting multimedia learning design principles in virtual reality-based learning environments for autistic individuals. Virtual Reality, 27(4), 3101–3113. |
| Newman, B., Reinecke, D. R., & Meinberg, D. L. (2000). Self–management of varied responding in three students with autism. Behavioral Interventions, 15(2), 145–151. https://doi.org/10.1002/(SICI)1099-078X(200004/06)15:2%3C145::AID-BIN50%3E3.0.CO;2-3 |
| Ogawa, R. T., & Malen, B. (1991). Towards rigor in reviews of multivocal literatures: applying the exploratory case study method. Review of Educational Research, 61(3), 265–286. https://doi.org/10.3102/00346543061003265 |
| Olson, A. J., Roberts, C. A., & Leko, M. M. (2015). Teacher-, student-, and peer-directed strategies to access the general education curriculum for students with autism. Intervention in School and Clinic, 51(1), 37–44. https://doi.org/10.1177/1053451214546406 |
| Parsons, S. (2015). Learning to work together: designing a multi-user virtual reality game for social collaboration and perspective-taking for children with autism. International Journal of Child-Computer Interaction, 6, 28–38. https://doi.org/10.1016/j.ijcci.2015.12.002 |
| Parsons, S. (2016). Authenticity in Virtual Reality for assessment and intervention in autism: a conceptual review. Educational Research Review, 19, 138–157. https://doi.org/10.1016/j.edurev.2016.08.001 |
| Schmidt, M., & Glaser, N. (2021). Investigating the usability and learner experience of a virtual reality adaptive skills intervention for adults with autism spectrum disorder. Educational Technology Research and Development, 69(3), 1665–1699. https://doi.org/10.1007/s11423-021-10005-8 |
| Schmidt, M., & Huang, R. (2022). Defining learning experience design: voices from the field of learning design & technology. TechTrends, 66(2), 141–158. https://doi.org/10.1007/s11528-021-00656-y |
| Sundberg, M. L., Endicott, K., & Eigenheer, P. (2000). Using intraverbal prompts to establish tacts for children with autism. The Analysis of Verbal Behavior, 17, 89–104. https://doi.org/10.1007/BF03392958 |
| Szumski, G. et al. (2019). Examining the effectiveness of naturalistic social skills training in developing social skills and theory of mind in preschoolers with ASD. Journal of Autism and Developmental Disorders, 49, 2822–2837. https://doi.org/10.1007/s10803-017-3377-9 |
| Van Laarhoven, T. et al. (2018). Comparing mobile technologies for teaching vocational skills to individuals with autism spectrum disorders and/or intellectual disabilities using universally-designed prompting systems. Journal of Autism and Developmental Disorders, 48, 2516–2529. https://doi.org/10.1007/s10803-018-3512-2 |
| Vann, S. W., & Tawfik, A. A. (2020). Flow theory and learning experience design in gamified learning environments. In Schmidt, M., Tawfik, A. A., Jahnke, I. & Earnshaw, Y. (Eds.), Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. Retrieved from https://edtechbooks.org/ux/flow_theory_and_lxd |
| Vedora, J., & Conant, E. (2015). A comparison of prompting tactics for teaching intraverbals to young adults with autism. The Analysis of Verbal Behavior, 31(2), 267–276. https://doi.org/10.1007/s40616-015-0030-6 |
| Yoder, P. J. et al. (2013). Social communication intervention effects vary by dependent variable type in preschoolers with autism spectrum disorders. Evidence-based Communication Assessment and Intervention, 7(4), 150–174. https://doi.org/10.1080/17489539.2014.917780 |