Empowered learning through microworlds and teaching methods: a text mining and meta-analysis-based systematic review

  • Joana Martinho Costa Instituto Universitário de Lisboa (ISCTE-IUL), ISTAR-IUL, Lisboa, Portugal
  • Sérgio Moro Instituto Universitário de Lisboa (ISCTE-IUL), ISTAR-IUL, Lisboa, Portugal https://orcid.org/0000-0002-4861-6686
  • Guilhermina Miranda Instituto de Educação, Universidade de Lisboa, Lisboa, Portugal https://orcid.org/0000-0003-4423-4522
  • Taylor Arnold Department of Mathematics and Computer Science, University of Richmond, Richmond, VA, USA
Keywords: meta-analysis, microworlds, simulations, teaching methods, technology-enhanced learning


Microworlds are simulations in computational environments where the student can manipulate objects and learn from those manipulations. Since their creation, they have been used in a wide range of academic areas to improve students learning from elementary school to college. However, their effectiveness is unclear since many studies do not measure the acquired knowledge after the use of microworlds but instead they focus on self-evaluation. Furthermore, it has not been clear whether its effect on learning is related to the teaching method. In this study, we perform a meta-analysis to ascertain the impact of microworlds combined with different teaching methods on students’ knowledge acquisition. We applied a selection criterion to a collection of 668 studies and were left with 10 microworld applications relevant to our learning context. These studies were then assessed through a meta-analysis using effect size with Cohen’s d and p-value. Our analysis shows that the cognitive methods combined with microworlds have a great impact on the knowledge acquisition (d = 1.03; p < 0.001) but failed to show a significant effect (d = 0.21) for expository methods.


Download data is not yet available.


Alice Project. (2020) Alice – Tell Stories. Build Games. Learn to Program, [online] Available at: http://www.alice.org

Anderson, J. R. (1993) Rules of the Mind. Hillsdale, Erlbaum, NJ.

Ausubel, D. (2000) The Acquisition and Retention of Knowledge: A Cognitive View, Dordrecht: Kluwer Academic Publishers.

Baddeley, A. (1997) Human Memory. Theory and Practice, Psychology Press, East Sussex, UK.

Bloom, B. (1956) Taxonomy of Educational Objectives: The Classification of Educational Goals, Allyn and Bacon, Boston, MA.

Borenstein, M., et al., (2009) Introduction to Meta-Analysis, Wiley, Chichester, UK.

Bransford, J. D., Brown, A. L. & Cocking, R. R. (2000) How People Learn: Brain, Mind, Experience and School, Academic Press, Washington, DC.

Bruner, J.S. (1965) The growth of mind. American Psychologist, 20(12), pp. 1007–1017

Bruner, J. (1966) Toward a Theory of Instruction, The Belknap Press of Harvard University Press, Cambridge, MA.

Coe, R. (2002) It’s the Effect Size, Stupid: What Effect Size Is and Why It Is Important, [online] Available at: http://www.leeds.ac.uk/educol/documents/00002182.htm

Cooper, H. & Hedges, L. (2009) ‘Research synthesis as a scientific process’, in The Handbook of Research Synthesis and Meta-Analysis, eds H. Cooper, L. Hedges & J. Valentine, Russel Sage Foundation, New York, pp. 4–15.

Cortez, P., et al., (2018) ‘Insights from a text mining survey on expert systems research from 2000 to 2016’, Expert Systems, vol. 35, no. 3, p. e12280. doi: 10.1111/exsy.12280

Costa, J. M. (2019) ‘Microworlds with different pedagogical approaches in introductory programming learning: effects in programming knowledge and logical reasoning’, Informatica, vol. 43, pp. 145–174. doi: 10.31449/inf.v43i1.2657

Costa, J. M. & Miranda, G. L. (2017) ‘Relation between Alice software and programming learning: a systematic review of the literature and meta-analysis’, British Journal of Educational Technology, vol. 48, no. 6, pp. 1464–1474. doi: 10.1111/bjet.12496

Costa, J. M. & Miranda, G. L. (2019) ‘Using Alice software with 4C-ID model: effects in programming knowledge and logical reasoning’, Informatics in Education, vol. 18, no. 1, pp. 1–15. doi: 10.15388/infedu.2019.01

De Corte, E. (1993) ‘Toward embedding enriched LOGO-based learning environments in the school curriculum: retrospect and prospect’, Proceedings of the Fourth European LOGO Conference Greece, Athens: University of Athens, pp. 28–31.

De Jong, T. & Lazonder, A. W. (2014) ‘The guided discovery learning principle in multimedia learning’, in The Cambridge Handbook of Multimedia Learning, ed R. E. Mayer, 2nd edn, Cambridge University Press, Cambridge, pp. 371–390.

DiCerbo, K. E., et al., (2010) ‘Individual practice and collaborative inquiry: instructional configurations in a simulation environment’, Sixth International Conference on Networking and Services, Cancun, Mexico: IEEE, pp. 335–339.

Fargas-Marques, A. & Costa-Castelló, R. (2006) ‘Using interactive tools to teach and understand MEMS’, IFAC Proceedings Volumes, vol. 39, no. 6, pp. 589–594. doi: 10.3182/20060621-3-ES-2905.00101

Gagné, R. (1985) The Conditions of Learning and Theory of Instruction, Holt, Rinehart and Winston, New York.

Hattie, J. (2009) Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, Routledge, London, UK.

Hedges, L. & Pigott, T. (2004) ‘The power of statistical tests for moderators in meta-analysis’, Psychological Methods, vol. 4, pp. 426–445. doi: 10.1037/1082-989X.9.4.426

Hedges, L., Shymansky, J. & Woodworth, G. (1989) Modern Methods of Meta-Analysis: A Practical Guide, National Science Teachers Association, Washington, DC.

Hoyles, C., Noss, R. & Adamson, R. (2002) ‘Rethinking the microworld idea’, Journal of Educational Computing Research, vol. 27, no. 1, pp. 29–53. doi: 10.2190/U6X9-0M6H-MU1Q-V36X

Jenkins, C. (2015) ‘A work in progress paper: evaluating a microworlds-based learning approach for developing literacy and computational thinking in cross-curricular contexts’, Proceedings of the Workshop in Primary and Secondary Computing Education, Potsdam Germany: ACM, pp. 61–64.

Kilpatrick, W. (2007) O método do projeto [The Project Method], Pedago Editions, Viseu.

Kirsnher, P. (2019) Constructivist Pedagogy Is Like a Zombie that Refuses to Die. An Interview with Professor Paul A. Kirschner by Isak Skogstad, [online] Available at: https://3starlearningexperiences.wordpress.com/2019/03/26/constructivist-pedagogy-is-like-a-zombie-that-refuses-to-die/

Kluge, A. (2008) ‘What you train is what you get? Task requirements and training methods in complex problem-solving’, Computers in Human Behavior, vol. 24, no. 2, pp. 284–308. doi: 10.1016/j.chb.2007.01.013

Lehrer, R., Lee, M. & Jeong, A. (1999) ‘Reflective teaching of logo’, The Journal of the Learning Sciences, vol. 8, no. 2, pp. 245–289. doi: 10.1207/s15327809jls0802_3

Littlefield, J., et al., (1988) ‘Learning LOGO: method of teaching, transfer of general skills, and attitudes toward school and computers’, in Teaching and Learning Computer Programming. Multiple Research Perspectives, ed R. E. Mayer, Erlbaum, Hillsdale, NJ, pp. 111–135.

Mayer, R. (2014) The Cambridge Handbook of Multimedia Learning, 2nd edn, Cambridge University Press, Cambridge, MA.

McDougall, A. (2002) ‘Technology-supported environments for learning through cognitive conflict’, Research in Learning Technology, vol. 10, no. 3, pp. 83–91. doi: 10.1080/0968776020100307

Mendelsohn, P. (1991) ‘LOGO: Quest-ce qui se développe’, in LOGO et apprentissages, eds J.-L. Gurtner & J. Retschitzki, Delachaux et Niestlé, Neuchâtel, pp. 29–37.

Merrill, M. D. (2002) ‘First principles of instruction’, Educational Technology Research and Development, vol. 50, no. 3, pp. 43–59. doi: 10.1007/BF02505024

Moons, J. & De Backer, C. (2013) ‘The design and pilot evaluation of an interactive learning environment for introductory programming influenced by cognitive load theory and constructivism’, Computers & Education, vol. 60, no. 1, pp. 368–384. doi: 10.1016/j.compedu.2012.08.009

Moro, S., et al., (2019) ‘A text mining and topic modelling perspective of ethnic marketing research’, Journal of Business Research, vol. 103, pp. 275–285. doi: 10.1016/j.jbusres.2019.01.053

Nelson, L. D., Simmons, J. & Simonsohn, U. (2018) ‘Psychology’s renaissance’, Annual Review of Psychology, vol. 69, pp. 511–534. doi: 10.1146/annurev-psych-122216-011836

Oliveira, O. L., Monteiro, A. M. & Roman, N. T. (2013) ‘Can natural language be utilized in the learning of programming fundamentals?’, IEEE Frontiers in Education Conference, Oklahoma: IEEE, pp. 1851–1856. doi: 10.1109/FIE.2013.6685157

Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas, Basic Books, Inc., New York.

Papert, S., et al., (1979) Final Report of the Brookline LOGO Project – Part II: Project Summary and Data Analysis, LOGO Memo N.º 53, MIT – AIL, Cambridge, MA.

Pea, R. & Kurland, D. M. (1984) ‘On the cognitive effects of learning computer programming’, New Ideas in Psychology, vol. 2, no. 2, pp. 137–168.

Pfahl, D., Koval, N. & Ruhe, G. (2001) ‘An experiment for evaluating the effectiveness of using a system dynamics simulation model in software project management education’, Seventh International Software Metrics Symposium, London, England: IEEE, pp. 97–109. doi: 10.1109/METRIC.2001.915519

Rieber, L. P. (1996) ‘Seriously considering play: designing interactive learning environments based on the blending of microworlds, simulations, and games’, Educational Technology Research & Development, vol. 44, no. 2, pp. 43–58. doi: 10.1007/BF02300540

Santos, M. R., Laureano, R. M. & Moro, S. (2019) ‘Unveiling research trends for organizational reputation in the nonprofit sector’, vol. 31, VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, pp. 56–70. doi: 10.1007/s11266-018-00055-7

Sarama, J. & Clements, D. H. (2002) ‘Building blocks for young children’s mathematical development’, Journal of Educational Computing Research, vol. 27, no. 1, pp. 93–110. doi: 10.2190/F85E-QQXB-UAX4-BMBJ

Schneider, B., et al., (2013) ‘Preparing for future learning with a tangible user interface: the case of neuroscience’, IEEE Transactions on Learning, vol. 6, no. 2, pp. 117–129. doi: 10.1109/TLT.2013.15

Schofield, J. W. (1995) Computers and Classroom Culture, Cambridge University Press, Cambridge.

Scratch. (2020) Scratch – Imagine, Program, Share, [online] Available at: https://scratch.mit.edu/

Simonsohn, U., Nelson, L. D. & Simmons, J. P. (2014a) ‘P-curve: a key to the file-drawer’, Journal of Experimental Psychology: General, vol. 143, no. 2, p. 534. doi: 10.1037/a0033242

Simonsohn, U., Nelson, L. D. & Simmons, J. P. (2014b) ‘P-curve and effect size: correcting for publication bias using only significant results’, Perspectives on Psychological Science, vol. 9, no. 6, pp. 666–681. doi: 10.1177/1745691614553988

Skinner, B. (1954) ‘The science of learning and the art of teaching’, Harvard Educational Review, vol. 24, no. 2, pp. 86–97.

Somekh, B. (1996) ‘Designing software to maximize learning’, Research in Learning Technology, vol. 4, no. 3, pp. 4–16. doi: 10.3402/rlt.v4i3.9974

Sweller, J., Ayres, P. & Kalyuga, S. (2011) Cognitive Load Theory, Springer, New York.

Van den Noorgate, W., et al., (2013) ‘Three-level meta-analysis of dependent effect sizes’, Behavior Research Methods, vol. 45, no. 2, pp. 576–594. doi: 10.3758/s13428-012-0261-6

van Merriënboer, J. (1997) Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training, Educational Technology Publications, Englewood Cliffs, NJ.

Vosinakis, S., Anastassakis, G. & Koutsabasis, P. (2018) ‘Teaching and learning logic programming in virtual worlds using interactive microworld representations’, British Journal of Educational Technology, vol. 49, no. 1, pp.30–44. doi: 10.1111/bjet.12531

Wang, S. Y., et al., (2018) ‘A microworld-based role-playing game development approach to engaging students in interactive, enjoyable, and effective mathematics learning’, Interactive Learning Environments, vol. 26, no. 3, pp. 411–423. doi: 10.1080/10494820.2017.1337038

Yuen-Kuang, C. & Brigth, G. (1991) ‘Effects of computer programming on cognitive outcomes: a meta-analysis’, Journal of Educational Computing Research, vol. 7, no. 3, pp. 251–268. doi: 10.2190/E53G-HH8K-AJRR-K69M

Yurtseven, M. K. & Buchanan, W. W. (2012) ‘Educating undergraduate students on systems thinking and system dynamics’, Proceedings of PICMET’12: Technology Management for Emerging Technologies, Vancouver, British Columbia, Canada: IEEE, pp. 1837–1844.

Zohar, A. & Peled, B. (2008) ‘The effects of explicit teaching of metastrategic knowledge on low-and high-achieving students’, Learning and Instruction, vol. 18, no. 4, pp. 337–353. doi: 10.1016/j.learninstruc.2007.07.001
How to Cite
CostaJ. M., MoroS., MirandaG., & ArnoldT. (2020). Empowered learning through microworlds and teaching methods: a text mining and meta-analysis-based systematic review. Research in Learning Technology, 28. https://doi.org/10.25304/rlt.v28.2396
Original Research Articles