Teaching social skills to students with autism spectrum disorder through augmented, virtual and mixed reality

Maggie A. Mosher; Adam C. Carreon

doi:10.25304/rlt.v29.2626

Maggie A. Mosher Department of Special Education, University of Kansas, Lawrence, KS, USA
Adam C. Carreon Department of Elementary and Special Education, Georgia Southern University; Statesboro, GA, USA https://orcid.org/0000-0001-9677-8025

DOI: https://doi.org/10.25304/rlt.v29.2626

Keywords: social validity, systematic review, virtual environment, social and emotional learning, technology delivered intervention

Abstract

This systematic literature review was conducted to explore the social validity of augmented reality (AR), virtual reality (VR), and mixed reality (MR) as a means of providing social skill instruction to students with autism spectrum disorder (ASD). Forty-one articles met the inclusion criteria, including five studies utilizing AR and the remaining 36 utilizing VR for social skill interventions. No studies implemented MR. The targeted skills of the studies included emotion recognition, relationship skills, social awareness, cooperation, and executive functioning. The intervention was considered effective in 63% of studies, not effective in 10% of studies, and mixed results in 27% of studies. The social validity indicators reported by researchers ranged from two to 14 of 17 determined categories. Findings indicate the primary socially valid reasons for utilizing AR/VR for social skill instruction were high student motivation toward the intervention and a positive attitude toward the technology. Findings indicate that increasing the role of parents, educators, and students as both social skill selectors and treatment agents and adding valid and reliable skill measures may improve the effects of an intervention. Sustainability may increase by providing training to both treatment agents and participants. AR has the potential to improve generalization and VR provides a practice environment for performance deficits. Combining these technologies may provide a more effective social skill intervention.

Downloads

Download data is not yet available.

References

Adjorlu, A. & Serafin, S. (2018) ‘Head-mounted display-based virtual reality as a tool to teach money skills to adolescents diagnosed with autism spectrum disorder’, in Interactivity, Game Creation, Design, Learning, and Innovation, Springer, Cham. pp. 450–461.

Akçayır, M. & Akçayır, G. (2016) ‘Advantages and challenges associated with augmented reality for education: A systematic review of the literature’, Educational Research Review, vol. 20, no. 2016, pp. 1–11.

Alcorn, A., et al., (2011) ‘Social communication between virtual characters and children with autism’, In international conference on artificial intelligence in education, Springer, Berlin, Heidelberg, pp. 7–14.

Armstrong, K. J., Ehrhardt, K. E., Cool, R. T. & Poling, A. (1997) ‘Social validity and treatment integrity data: Reporting in articles published in the Journal of Developmental and Physical Disabilities, 1991–1995’, Journal of Developmental and Physical Disabilities, vol. 9, no. 4, pp. 359–367.

Bekele, E., et al., (2014) ‘Assessing the utility of a virtual environment for enhancing facial affect recognition in adolescents with autism’, Journal of Autism & Developmental Disorders, vol. 44, no. 7, pp. 1641–1650. doi: 10.1007/s10803-014-2035-8

Bellani, M., et al., (2011) ‘Virtual reality in autism: state of the art’, Epidemiology and Psychiatric Sciences, vol. 20, no. 3, pp. 235–238. doi: 10.1017/S2045796011000448

Bernardini, S., Porayska-Pomsta, K. & Smith, T. J. (2014) ‘ECHOES: an intelligent serious game for fostering social communication in children with autism’, Information Sciences, vol. 264, pp. 41–60. doi: 10.1016/j.ins.2013.10.027

Cai, Y., et al., (2013) ‘Design and development of a virtual dolphinarium for children with autism’, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 2, pp. 208–217. doi: 10.1109/TNSRE.2013.2240700

Callahan, K., et al., (2017) ‘Social validity of evidence-based practices and emerging interventions in autism’, Focus on Autism and Other Developmental Disabilities, vol. 32, no. 3, pp. 188–197. doi: 10.1177/1088357616632446

Campbell, D. T. & Stanley, J. C. (2015) Experimental and quasi-experimental designs for research, Ravenio Books, Boston.

Carreon, A., et al., (2020) ‘A review of virtual reality intervention research for students with disabilities in k-12 settings’, Journal of Special Education Technology, pp. 1–18. doi: 10.1177/0162643420962011

Carter, S. L. & Wheeler, J. J. (2019) The social validity manual: Subjective evaluation of Interventions, Academic Press, Caimbridge.

Catania, A. C. (1998) Learning, 4th edn., Prentice Hall, Upper Saddle River, NJ.

Chelkowski, L., Yan, Z. & Asaro-Saddler, K. (2019) ‘The use of mobile devices with students with disabilities: a literature review’, Preventing School Failure: Alternative Education for Children and Youth, vol. 63, no. 3, pp. 277–295. doi: 10.1080/1045988X.2019.1591336

Chen, C. H., Lee, I. J. & Lin, L. Y. (2015) ‘Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders’, Research in Developmental Disabilities, vol. 36, pp. 396–403. doi: 10.1016/j.ridd.2014.10.015

Chen, I. & Lin, L. Y. (2016) ‘Augmented reality-based video-modeling storybook of nonverbal facial cues for children with ASD to improve their perceptions and judgments of facial expressions & emotions’, Computers in Human Behavior, vol. 55, pp. 477–485. doi: 10.1016/j.chb.2015.09.033

Cheng, Y., et al., (2010) ‘Enhancing empathy instruction using a collaborative virtual learning environment for children with autistic spectrum conditions’, Computers & Education, vol. 55, no. 4, pp. 1449–1458. doi: 10.1016/j.compedu.2010.06.008

Cheng, Y., Huang, C. L. & Yang, C. S. (2015) ‘Using a 3D immersive virtual environment system to enhance social understanding and social skills for children with autism spectrum disorders’, Focus on Autism and Other Developmental Disabilities, vol. 30, no. 4, pp. 222–236. doi: 10.1177/1088357615583473

Cheng, Y. & Ye, J. (2009) ‘Exploring the social competence of students with autism spectrum conditions in a collaborative virtual learning environment-the pilot study’, Computers & Education, vol. 54, no. 4, pp. 1068–1077. doi: 10.1016/j.compedu.2009.10.011

Cook, B., et al., (2014) ‘Council for exceptional children: standards for evidence-based practices in special education’, Teaching Exceptional Children, vol. 46, no. 6, p. 206. doi: 10.1177/0040059914531389

Dabholkar, P. A. & Bagozzi, R. P. (2002) ‘An attitudinal model of technology-based self-service: moderating effects of consumer traits and situational factors’, Journal of the Academy of Marketing Science, vol. 30, no. 3, pp. 184–201. doi: 10.1177/00970302030003001

Didehbani, N., et al., (2016) ‘Virtual reality social cognition training for children with high functioning autism’, Computers in Human Behavior, vol. 62, pp. 703–711. doi: 10.1016/j.chb.2016.04.033

Dobbins, N., et al., (2010) ‘An analysis of social skills instruction provided in teacher education and in-service training programs for general and special educators’, Remedial and Special Education, vol. 31, pp. 358–367. doi: 10.1177/0741932509338363

Finney, J. W. (1991) ‘On further development of the concept of social validity’, Journal of Applied Behavior Analysis, vol. 24, no. 2, p. 245. doi: 10.1901/jaba.1991.24-245

Fox, J. J. & McEvoy, M. A. (1993) ‘Assessing and enhancing generalization and social validity of social-skills interventions with children and adolescents’, Behavior Modification, vol. 17, no. 3, pp. 339–366.

GovTrack.us. (2020) H.R. 850, 114th Congress: Academic, Social, and Emotional Learning Act of 2015 (ASELA), Available at: https://www.govtrack.us/congress/bills/114/hr850

Grynszpan, O., Martin, J.-C. & Nadel, J. (2008) ‘Multimedia interfaces for users with high functioning autism: an empirical investigation’, International Journal of Human – Computer Studies, vol. 66, no. 8, pp. 628–639. doi: 10.1016/j.ijhcs.2008.04.001

Halabi, O., et al., (2017) ‘Design of immersive virtual reality system to improve communication skills in individuals with autism’, International Journal of Emerging Technologies in Learning (iJET), vol. 12, no. 05, pp. 50–64. doi: 10.3991/ijet.v12i05.6766

Hansen, D. J., St. Lawrence, J. S. & Christoff, K. A. (1989) ‘Group conversational-skills training with inpatient children and adolescents: social validation, generalization, and maintenance’, Behavior Modification, vol. 13, no. 1, pp. 4–31. doi: 10.1177/01454455890131001

Herrera, G., et al., (2008) ‘Development of symbolic play through the use of virtual reality tools in children with autistic spectrum disorders’, Autism: The International Journal of Research and Practice, vol. 12, pp. 143–157. doi: 10.1177/1362361307086657

John Wiley & Sons, Chichester. (2011) Cochrane Handbook for Systematic Reviews of Interventions, Vol. 4, John Wiley & Sons, Chichester.

Horace, H., et al., (2016) ‘Virtual reality enable training for social adaptation in inclusive education settings for school-aged children with autism Spectrum disorders (ASD)’, in Blended Learning: Aligning Theory with Practices, eds S. Cheung et al., 9th International Conference on Blended Learning, Springer, Cham, pp. 94–102.

Howard, M. C. & Gutworth, M. B. (2020) ‘A meta-analysis of virtual reality training programs for social skill development’, Computers & Education, vol. 144, p. 103707. doi: 10.1016/j.compedu.2019.103707

Ip, H. H., et al., (2018) ‘Enhance emotional and social enhance emotional and adaptation skills for children with autism spectrum disorder: a virtual reality enabled approach’, Computers & Education, vol. 117, pp. 1–15. doi: 10.1016/j.compedu.2017.09.010

Jarrold, W., et al., (2013) ‘Social attention in a virtual public speaking task in higher functioning children with autism’, Autism Research, vol. 6, no. 5, pp. 393–410. doi: 10.1002/aur.1302

Ke, F. & Im, T. (2013) ‘Virtual-reality-based social interaction training for children with high-functioning autism’, The Journal of Educational Research, vol. 106, no. 6, pp. 441–461. doi: 10.1080/00220671.2013.832999

Ke, F. & Lee, S. (2015) ‘Virtual reality based collaborative design by children with high-functioning autism: design-based flexibility, identity, and norm construction’, Interactive Learning Environments, vol. 24, no. 7, pp. 1511–1533. doi: 10.1080/10494820.2015.1040421

Ke, F., Whalon, K. & Yun, J. (2018) ‘Social skill interventions for youth and adults with autism spectrum disorder: a systematic review’, Review of Educational Research, vol. 88, no. 1, pp. 3–42. doi: 10.3102/0034654317740334

Kern, L. & Manz, P. (2004) ‘A look at current validity issues of school-wide behavior support’, Behavioral Disorders, vol. 30, pp. 47–59. doi: 10.1177/019874290403000102

Kim, K., et al., (2015) ‘A virtual joy-stick study of emotional responses and social motivation in children with autism spectrum disorder’, Journal of Autism and Developmental Disorders, vol. 45, no. 12, pp. 3891–3899. doi: 10.1007/s10803-014-2036-7

Koegel, R., Kim, S. & Koegel, L. (2014) ’Training paraprofessionals to improve socialization in students with asd’, Journal of Autism and Developmental Disorders, vol. 44, no. 9, pp. 2197–2208. doi: 10.1007/s10803-014-2094-x

Lan, Y.-J., Hsiao, I. Y. & Shih, M.-F. (2018) ‘Effective learning design of game-based 3d virtual language learning environments for special education students’, Journal of Educational Technology & Society, vol. 21, no. 3, pp. 213–227.

Ledford, J. R. & Gast, D. L. (Eds.). (2014) Single case research methodology: Applications in special education and behavioral sciences, Routledge, Milton Park.

Lee, I.-J., et al., (2018) ‘Augmented reality plus concept map technique to teach children with asd to use social cues when meeting and greeting’, The Asia-Pacific Education Researcher, vol. 27, no. 3, pp. 227–243. doi: 10.1007/s40299-018-0382-5

Liu, D., et al., (Eds.). (2017) Virtual, Augmented, and Mixed Realities in Education, Springer, Singapore.

Lorenzo, G., et al., (2019) ‘Preliminary study of augmented reality as an instrument for improvement of social skills in children with autism spectrum disorder’, Education and Information Technologies, vol. 24, no. 1, pp. 181–204. doi: 10.1007/s10639-018-9768-5

Lorenzo, G., et al., (2016) ‘Design and application of an immersive virtual reality system to enhance emotional skills for children with autism spectrum disorders’, Computers & Education, vol. 98, pp. 192–205. doi: 10.1016/j.compedu.2016.03.018

Lorenzo, G., Pomares, J. & Lledó, A. (2013) ‘Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with asperger syndrome’, Computers & Education, vol. 62, pp. 88–101. doi: 10.1016/j.compedu.2012.10.028

Maggin, D. M., et al., (2017) ‘Quality indicators for systematic reviews in behavioral disorders’, Behavioral Disorders, vol. 42, no. 2, pp. 52–64. doi: 10.1177/0198742916688653

Mantziou, O., Vrellis, I. & Mikropoulos, T. A. (2015) ‘Do children in the spectrum of autism interact with real-time emotionally expressive human controlled avatars?’, Procedia Computer Science, vol. 67, pp. 241–251. doi: 10.1016/j.procs.2015.09.268

Merchant, Z., et al., (2014) ‘Effectiveness of virtual reality-based instruction on students’ learning outcomes in k-12 and higher education: a meta-analysis’, Computers & Education, vol. 70, pp. 29–40. doi: 10.1016/j.compedu.2013.07.033

Mikropoulos, T. A. & Natsis, A. (2011) ‘Educational virtual environments: a ten-year review of empirical research (1999-2009)’, Computers & Education, vol. 56, no. 3, pp. 769–780. doi: 10.1016/j.compedu.2010.10.020

Miller, H. & Bugnariu, N. L. (2016) ‘Level of immersion in virtual environments impacts the ability to assess and teach social skills in autism spectrum disorder’, Cyberpsychol Behav Soc Netw, vol. 19, no. 4, pp. 246–256. doi: 10.1089/cyber.2014.0682

Mitchell, P., Parsons, S. & Leonard, A. (2007) Using virtual environments for teaching social understanding to 6 adolescents with autistic spectrum disorders. Journal of Autism & Developmental Disorders, vol. 37, no. 3, pp. 589–600. doi: 10.1007/s10803-006-0189-8

Modugumudi, Y. R., Santhosh, J. & Anand, S. (2013) ‘Efficacy of collaborative virtual environment intervention programs in emotion expression of children with autism’, Journal of Medical Imaging and Health Informatics, vol. 3, no. 2, pp. 321–325. doi: 10.1166/jmihi.2013.1167

Moore, D., et al., (2005) ‘Collaborative virtual environment technology for people with autism’, Focus on Autism and Other Developmental Disabilities, vol. 20, no. 4, pp. 231–243. doi: 10.1177/10883576050200040501

Mosher, M. A., et al., (2021) ‘Immersive technology to teach social skills to students with autism spectrum disorder: a literature review’, Review Journal of Autism and Developmental Disorders, pp. 1–17. doi: 10.1007/s40489-021-00259-6

Parsons, S. (2015) ‘Learning to work together: designing a multi-user virtual reality game for social collaboration and perspective-taking for children with autism’, International Journal of Child-Computer Interaction, vol. 6, pp. 28–38. doi: 10.1016/j.ijcci.2015.12.002

Parsons, S., Leonard, A. & Mitchell, P. (2006) ‘Virtual environments for social skills training: comments from two adolescents with autistic spectrum disorder’, Computers & Education, vol. 47, no. 2, pp. 186–206. doi: 10.1016/j.compedu.2004.10.003

Parsons, S. & Mitchell, P. (2002) ‘The potential of virtual reality in social skills training for people with autistic spectrum disorders’, Journal of Intellectual Disability Research, vol. 46, no. 5, pp. 430–443. doi: 10.1046/j.1365-2788.2002.00425.x

Parsons, S., Mitchell, P. & Leonard, A. (2004) ‘The use and understanding of virtual environments by adolescents with autistic spectrum disorders’, Journal of Autism & Developmental Disorders, vol. 34, no. 4, pp. 449–466. doi: 10.1023/B:JADD.0000037421.98517.8d

Plavnick, J. B. & Hume, K. A. (2014) ‘Observational learning by individuals with autism: a review of teaching strategies’, Autism, vol. 18, no. 4, pp. 458–466. doi: 10.1177/1362361312474373

Radu, I. (2014) ‘Augmented reality in education: a meta-review and cross-media analysis’, Personal and Ubiquitous Computing, vol. 18, no. 6, pp. 1533–1543. doi: 10.1007/s00779-013-0747-y

Reichow, B., et al., (2011) Evidence-Based Practices and Treatments for Children with Autism, Springer, New York, NY. doi: 10.1007/978-1-4419-6975-0

Saadatzi, M. N., et al., (2018) ‘Small-group technology-assisted instruction: virtual teacher and robot peer for individuals with autism spectrum disorder’, Journal of Autism & Developmental Disorders, vol. 48, no. 11, pp. 3816–3830. doi: 10.1007/s10803-018-3654-2

Schmidt, M., et al., (2011) ‘Developing methods for understanding social behavior in a 3D virtual learning environment’, Computers in Human Behavior, vol. 28, no. 2, pp. 405–413. doi: 10.1016/j.chb.2011.10.011

Self, T., et al., (2007) ‘A virtual approach to teaching safety skills to children with ASD’, Topics in Language Disorders, vol. 27, no. 3, pp. 242–253. doi: 10.1097/01.TLD.0000285358.33545.79

Shih, C. H., Chang, M. L., Wang, S. H. & Tseng, C. L. (2015) ‘Assisting students with autism to actively perform collaborative walking activity with their peers using dance pads combined with preferred environmental stimulation’, Research in Autism Spectrum Disorders, vol. 8, no. 11, pp. 1591–1596. doi: 10.1016/j.rasd.2014.08.011

Stahmer, A. C. (2007) ‘The basic structure of community early intervention programs for children with autism: Provider descriptions’, Journal of autism and developmental disorders, vol. 37, no. 7, pp. 1344–1354.

Stichter, J., et al., (2014) ‘Isocial: delivering the social competence intervention for adolescents (sci-a) in a 3d virtual learning environment for youth with high functioning autism’, Journal of Autism & Developmental Disorders, vol. 44, no. 2, pp. 417–430. doi: 10.1007/s10803-013-1881-0

Taryadi, B. R. & Kurniawan, I. (2018) ‘The improvement of autism spectrum disorders on children communication ability with pecs method multimedia augmented reality-based’, Journal of Physics: Conference Series, vol. 947, p. 012009. doi: 10.1088/1742-6596/947/1/012009

Tobin, M. C., Drager, K. D. R. & Richardson, L. F. (2014) ‘A systematic review of social participation for adults with autism spectrum disorders: support, social functioning, and quality of life’, Research in Autism Spectrum Disorders, vol. 8, no. 3, pp. 214–229. doi: 10.1016/j.rasd.2013.12.002

Tsiopela, D. & Jimoyiannis, A. (2014) ‘Pre-vocational skills laboratory: development and investigation of a web-based environment for students with autism’, Procedia Computer Science, vol. 27, pp. 207–217. doi: 10.1016/j.procs.2014.02.024

Vasquez, E., et al., (2015) ‘Virtual learning environments for students with disabilities: a review and analysis of the empirical literature and two case studies’, Rural Special Education Quarterly, vol. 34, no. 3, pp. 26–32. doi: 10.1177/875687051503400306

Wang, C., et al., (2016) ‘The combined effects of teacher-child and peer relationships on children’s social-emotional adjustment’, Journal of School Psychology, vol. 59, pp. 1–11. doi: 10.1016/j.jsp.2016.09.003

Wang, X., Xing, W. & Laffey, J. M. (2018) ‘Autistic youth in 3D game–based collaborative virtual learning: associating avatar interaction patterns with embodied social presence’, British Journal of Educational Technology, vol. 49, no. 4, pp. 742–760. doi: 10.1111/bjet.12646

Welsh, M., et al., (2001) ‘Linkages between children’s social and academic competence’, Journal of School Psychology, vol. 39, no. 6, pp. 463–482. doi: 10.1016/S0022-4405(01)00084-X

Yuan, S. N. V. & Ip, H. H. S. (2018) ‘Using virtual reality to train emotional and social skills in children with autism spectrum disorder’, London Journal of Primary Care, vol. 10, no. 4, pp. 110–112.