Measuring what matters: a systematic review and VOSviewer-based bibliometric approach to digital literacy assessment instruments, competency dimensions and challenges in education

  • Natia Afriana Suri Doctoral Program of Science Education, Universitas Negeri Padang, Padang, Indonesia https://orcid.org/0009-0007-7556-4619
  • Festiyed Department of Science Education Doctoral Program, Universitas Negeri Padang, Padang, Indonesia
  • Minda Azhar Department of Science Education Doctoral Program, Universitas Negeri Padang, Padang, Indonesia
  • Yerimadesi Department of Science Education Doctoral Program, Universitas Negeri Padang, Padang, Indonesia
  • Yuni Ahda Department of Science Education Doctoral Program, Universitas Negeri Padang, Padang, Indonesia
  • Heffi Alberida Department of Science Education Doctoral Program, Universitas Negeri Padang, Padang, Indonesia
DOI: https://doi.org/10.25304/rlt.v33.3413
Keywords: digital literacy, measurement instrument, digital competency

Abstract

Digital literacy is a critical competency in education across all levels, from primary to higher education. It includes skills such as technical proficiency, information evaluation, online collaboration, creativity and ethical technology use. This study conducts a Systematic Literature Review (SLR), following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, to examine types of instruments used to assess students’ digital literacy, the competencies targeted and the methodological challenges in their development. A total of 23 peer-reviewed articles published between 2014 and 2024 were selected from Scopus, PubMed, Crossref and ERIC. This review shows that assessment instruments include Likert scale-based questionnaires, framework-aligned tools (DigComp and DQ Framework) and digital performance-based methods. These instruments are applied across diverse educational settings: primary, secondary, tertiary and adult education with varying emphases based on age and learning context. Whilst core competencies are addressed, several limitations persist, such as reliance on self-reporting, limited cross-cultural validation and lack of authentic performance assessment. This study highlights the need for more comprehensive, validated and context-sensitive instruments that integrate digital safety, ethics and practical digital skills. The findings offer insights for researchers, educators and policymakers to improve digital literacy measurement across education sectors.

Downloads

Download data is not yet available.

References


Afandi, A. N. H., Kusumaningrum, S. R., Dewi, R. S. I., & Pristiani, R. (2024). Digital literacy questionnaire instrument: Based on the integration of elementary school students’ characteristics. International Journal of Elementary Education, 8(2), 344–353. https://doi.org/10.23887/ijee.v8i2.76773




Agormedah, E. K., Quansah, F., Ankomah, F., Hagan, J. E., Srem-Sai, M., Abieraba, R. S. K., Frimpong, J. B., & Schack, T. (2022). Assessing the validity of digital health literacy instrument for secondary school students in Ghana: The polychoric factor analytic approach. Frontiers in Digital Health, 4, 968806. https://doi.org/10.3389/fdgth.2022.968806




Baharuddin, M. F., Masrek, M. N., Shuhidan, S. M., Razali, M. H. H., & Rahman, M. S. (2021). Evaluating the content validity of digital literacy instrument for school teachers in Malaysia through expert judgement. International Journal of Emerging Technology and Advanced Engineering, 11(7), 71–78. https://doi.org/10.46338/ijetae0721_09




Cabero-Almenara, J., Gutiérrez-Castillo, J.-J., Barroso-Osuna, J., & Palacios-Rodríguez, A. (2023). Digital teaching competence according to the DigCompEdu Framework. Comparative study in different Latin American universities. Journal of New Approaches in Educational Research, 12, 276–291. https://doi.org/10.7821/naer.2023.7.1452




Cabrera, W. R. R., & Sosa, J. E. P. (2024). Validation of an instrument for assessing the digital literacy of Mexican students. Edutec, 88, 200–219. https://doi.org/10.21556/edutec.2024.88.3131




Choi, J., Choi, S., Song, K., Baek, J., Kim, H., Choi, M., Kim, Y., Chu, S.H., & Shin, J. (2023). Everyday Digital Literacy Questionnaire (EDLQ): Development and validation for older adults in South Korea (Preprint). Journal of Medical Internet Research, 25, e51616. https://doi.org/10.2196/51616




Churchill, N. (2020). Development of students’ digital literacy skills through digital storytelling with mobile devices. Educational Media International, 57, 271–284. https://doi.org/10.1080/09523987.2020.1833680




DQ Institute. (2023). The 2023 DQ Global Standards Report: Defining digital literacy, digital skills, and digital readiness. Retrieved from https://jointresearchcentre.ec.europa.eu/digcompedu_en




Economou, A., Vuorikari, R., & Punie, Y. (2023). DigCompEdu: European framework for the digital competence of educators – SELFIEforTEACHERS adaptation and implementation. Publications Office of the European Union. Retrieved from https://op.europa.eu/en/publication-detail/-/publication/3a69fc0c-2e00-11ee-a9ce-01aa75ed71a1




Febliza, A., & Okatariani, O. (2020). Pengembangan Instrumen Literasi Digital Sekolah, Siswa Dan Guru. Jurnal Pendidikan Kimia Universitas Riau, 5(1), 1. https://doi.org/10.33578/jpk-unri.v5i1.7776




Forzani, E., Leu, D. J., Yujia Li, E., Rhoads, C., Guthrie, J. T., & McCoach, B. (2021). Characteristics and validity of an instrument for assessing motivations for online reading to learn. Reading Research Quarterly, 56(4), 761–780. https://doi.org/10.1002/rrq.337




Guo, J. (2024). Approaches to improve digital literacy of higher vocational teachers. Occupation and Professional Education, 1(3), 25–30. https://doi.org/10.62381/O242304




Harlanu, M., Suryanto, A., Ramadhan, S., & Wuryandini, E. (2023). Construct validity of the instrument of digital skill literacy. Cakrawala Pendidikan, 42(3), 781–790. https://doi.org/10.21831/cp.v42i3.59703




Harutyunyan, G., Manucharyan, M., Muradyan, M., & Asatryan, H. (2024). Digital literacy of the Armenian society: Assessment and determinants. Cogent Social Sciences, 10(1), 2398652. https://doi.org/10.1080/23311886.2024.2398652




Hernández-Marín, J. L., Castro-Montoya, M. D., & Figueroa-Rodríguez, S. (2024). Media, information and digital literacy: Assessment instrument analysis. Investigacion Bibliotecologica, 38(99), 55–73. https://doi.org/10.22201/iibi.24488321xe.2024.99.58865




Herro, D. (2014). Techno Savvy: A Web 2.0 curriculum encouraging critical thinking. Educational Media International, 51(4), 259–277. https://doi.org/10.1080/09523987.2014.977069




Herzog-Punzenberger, B., Altrichter, H., Brown, M., Burns, D., Nortvedt, G.A., Skedsmo, G., Wiese, E., Nayir, F., Fellner, M., McNamara, G., & O’Hara, J. (2020). Teachers responding to cultural diversity: Case studies on assessment practices, challenges and experiences in secondary schools in Austria, Ireland, Norway and Turkey. Educational Assessment, Evaluation and Accountability, 32, 395–424. https://doi.org/10.1007/s11092-020-09330-y




Hwang, H. S., Zhu, L. C., & Cui, Q. (2023). Development and validation of a digital literacy dcale in the artificial intelligence era for college students. KSII Transactions on Internet and Information Systems, 17(8), 2241–2258. https://doi.org/10.3837/tiis.2023.08.016




Irhandayaningsih, A. (2022). Digital literacy assessment model in learning management system: A self-directed learning perspective. E3S Web of Conferences, 359, 03017. https://doi.org/10.1051/e3sconf/202235903017




Jashari, X., Fetaji, B., & Guetl, C. (2024). Assessing digital literacy skills in secondary and higher education: A comprehensive literature survey of validated instruments. 2024 47th ICT and Electronics Convention, MIPRO 2024-Proceedings (pp. 688–692). https://doi.org/10.1109/MIPRO60963.2024.10569399




Jisc. (2023). Digital capabilities definitions: Individual digital capabilities framework. Jisc. Retrieved from https://digitalcapability.jisc.ac.uk/what-is-digital-capability/individual-digital-capabilities/




Lawson, T.K., Knox, J., Romero, E., Palacios, A.M., & Fallon, L.M, (2024). Check yourself! Exploring current culturally responsive teaching assessment measures. Psychology in the Schools, 61(6), 2649–2667. https://doi.org/10.1002/pits.23189




Lukitasari, M., Murtafiah, W., Ramdiah, S., Hasan, R., & Sukri, A. (2022). Constructing digital literacy instrument and its effect on college students’ learning outcomes. International Journal of Instruction, 15(2), 171–188. https://doi.org/10.29333/iji.2022.15210a




Magnago, W., Baiôcco, L.V., Lima, R.C.M., Junior, A. B., Soprani, L. C. P., Azevedo, J. D. F., Pinheiro, R.B., & Nunes, P.D.C. (2024). Digital education and the role of technological literacy in cognitive and socioe-motional development. Lumen et Virtus, 15(42), 6548–6553. https://doi.org/10.56238/levv15n42-002




Mainz, A., Nitsche, J., Weirauch, V., & Meister, S. (2024). Measuring the digital competence of health professionals: Scoping review. JMIR Medical Education, 10, e55737. https://doi.org/10.2196/55737




Mattar, J., Ramos, D. K., & Lucas, M. R. (2022). DigComp-based digital competence assessment tools: Literature review and instrument analysis. Education and Information Technologies, 27(8), 10843–10867. https://doi.org/10.1007/s10639-022-11034-3




Mieg, H.A., Klieme, K.E., Barker, E., Bryan, J., Gibson, C., Haberstroh, S., Odebiyi, F., Rismondo, F.P., Römmer-Nossek, B., Thiem, J., & Unterpertinger, E. (2023). Short digital-competence test based on DigComp2.1: Does digital competence support research competence in undergraduate students? Education and Information Technologies, 29, 139–160. https://doi.org/10.1007/s10639-023-12251-0




Momdjian, L., Manegre, M., & Gutiérrez-colón, M. (2024). Digital competences of teachers in Lebanon: A comparison of teachers’ competences to educational standards. Research in Learning Technology, 32(1063519), 1–18. https://doi.org/10.25304/rlt.v32.3203




Nalaila, S., & Elia, E.F. (2024). Students’ digital literacy skills for learning in selected Tanzania’s public universities. Cogent Education, 11(1), 2355350. https://doi.org/10.1080/2331186X.2024.2355350




Newland, B., & Handley, F. (2016). Developing the digital literacies of academic staff: An institutional approach. Research in Learning Technology, 24(1063519), 1–12. https://doi.org/10.3402/rlt.v24.31501




Nguyen, L. A. T., & Habók, A. (2023). Tools for assessing teacher digital literacy: A review. Journal of Computers in Education, 11, 305–346. https://doi.org/10.1007/s40692-022-00257-5




Nguyen, L. A. T., & Habók, A. (2024). Tools for assessing teacher digital literacy: A review. Journal of Computers in Education, 11(1), 305–346. https://doi.org/10.1007/s40692-022-00257-5




Nishikawa‑Pacher, A. (2022). Research Questions with PICO: A Universal Mnemonic. Publications, 10(3), 21. https://doi.org/10.3390/publications10030021




Oh, S. S., Kim, K. A., Kim, M., Oh, J., Chu, S. H., & Choi, J. (2021). Measurement of digital literacy among older adults: Systematic review. Journal of Medical Internet Research 25(2), e26145. https://doi.org/10.2196/26145




Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., …Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71




Park, Y., & Gentile, D. (2019). Establishing digital intelligence (DQ) as a universal standard for digital literacy, skills and readiness. OECD Education Working Papers, No. 193. OECD. https://doi.org/10.1787/19939019




Ramirez-Rodriguez, L. T., Sanchez-Pimentel, J. I., Osorio-Galvan, R. C., & Perez-Ortiz, J. O. (2022). Design and validation of an instrument to measure digital skills in university students of the first cycles of health careers. Proceedings of the 2022 IEEE 2nd International Conference on Advanced Learning Technologies on Education and Research, ICALTER 2022. 16–19 November 2022, Lima, Peru: ICALTER.




Restrepo-Palacio, S., & De María Segovia Cifuentes, Y. (2020). Design and validation of an instrument for the evaluation of digital competence in Higher Education. Ensaio, 28(109), 932–961. https://doi.org/10.1590/S0104-40362020002801877




Ristiyana Puspita Sari, A., Sidauruk, S., Meiliawati, R., & Anggraeni, M. E. (2021). Development of Digital Literacy Assessment Scale to measure student’s digital literacy. Jurnal Ilmiah Kanderang Tingang, 12(02), 137–143. https://doi.org/10.37304/jikt.v12i02.128




Rubach, C., & Lazarides, R. (2021). Addressing 21st-century digital skills in schools – Development and validation of an instrument to measure teachers’ basic ICT competence beliefs. Computers in Human Behavior, 118, 106636. https://doi.org/10.1016/j.chb.2020.106636




Ruffini, C. (2022). La comprensione del testo digitale e cartaceo in età scolare: il ruolo delle Funzioni Esecutive. In S. Ulivieri (Ed.), Studies on Adult Learning and Education (pp. 305–317). Firenze University Press. https://doi.org/10.36253/979-12-215-0081-3.20




Sari, C. M. W., Huda, I., Pada, A. U. T., Rahmatan, H., & Samingan. (2020). Construct validity of digital media literacy instrument for student teachers. Journal of Physics: Conference Series, 1460(1), 012053. https://doi.org/10.1088/1742-6596/1460/1/012053




Shin, S. K. (2015). Teaching critical, ethical and safe use of ICT in pre-service teacher education. Language Learning & Technology, 19(1), 181–197. https://doi.org/10.10125/44408




Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective by learning from the past – A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84. https://doi.org/10.1016/j.edurev.2016.05.002




Snyder, I., Angus, L., & Sutherland-Smith, E. (2002). Building equitable literate futures: Home and school computer-mediated literacy practices and disadvantage. Cambridge Journal of Education, 32(3), 367–383.




Son, M., & Ha, M. (2024). Development of a digital literacy measurement tool for middle and high school students in the context of scientific practice. Education and Information Technologies, 30, 4583–4606. https://doi.org/10.1007/s10639-024-12999-z




Tajuddin, S. N. A. A., Bahari, K. A., Al-Majdhoub, F. M., Maliki, N. K., & Baboo, S. B. (2024). Developing and measuring an assessment instrument for media literacy among digital natives using Digital Intelligence (DQ) framework. Journal of Media Literacy Education, 16(2), 29–45. https://doi.org/10.23860/JMLE-2024-16-2-3




UNESCO-UNEVOC. (2023). ICT competency framework for teachers (ICT-CFT) Version 3. UNESCO International Centre for Technical and Vocational Education and Training. Retrieved from: https://unevoc.unesco.org/home/ICT-CFT




Ussarn, A., Pimdee, P., & Kantathanawat, T. (2022). Needs assessment to promote the digital literacy among students in Thai community colleges. International Journal of Evaluation and Research in Education, 11(3), 1278–1284. https://doi.org/10.11591/ijere.v11i3.23218




Ventivani, A., Ul Muyassaroh, L., Sakti, K. F. L., Masrur, M. F., & Kharis, M. (2024). A measuring tool for assessing digital literacy competence among Mandarin language students. International Journal of Engineering Trends and Technology, 72(10), 336–343. https://doi.org/10.14445/22315381/IJETT-V72I10P132




Villoria-Mendieta, M. (2023). Teacher digital competences: Formal approaches to their development. In Digital education outlook.




Yasa, A. D., & Rahayu, S. (2023). A survey of elementary school students’ digital literacy skills in science learning. AIP Conference Proceedings, 2569, 060015. https://doi.org/10.1063/5.0113483




Yasa, A. D., Rahayu, S. D., Handayanto, S. K., & Ekawati, R. (2024). Investigating the effects of digital literacy on primary student attitudes in Indonesia. International Journal of Elementary Education, 8(1), 11–19. https://doi.org/10.23887/ijee.v8i1.70413




Yuan, C., Wang, L., & Eagle, J. (2019). Empowering English language learners through digital literacies: Research, complexities, and implications. Media and Communication, 7(2), 128. https://doi.org/10.17645/mac.v7i2.1912


Published
2025-08-05
How to Cite
Suri , N. A., Festiyed, Azhar , M., Yerimadesi, Ahda , Y., & Heffi Alberida. (2025). Measuring what matters: a systematic review and VOSviewer-based bibliometric approach to digital literacy assessment instruments, competency dimensions and challenges in education. Research in Learning Technology, 33. https://doi.org/10.25304/rlt.v33.3413
Issue
Vol. 33 (2025)
Section
Original Research Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors contributing to Research in Learning Technology retain the copyright of their article and at the same time agree to publish their articles under the terms of the Creative Commons CC-BY 4.0 License (http://creativecommons.org/licenses/by/4.0/) allowing third parties to copy and redistribute the material in any medium or format, and to remix, transform, and build upon the material, for any purpose, even commercially, under the condition that appropriate credit is given, that a link to the license is provided, and that you indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.


Crossref
Scopus
Google Scholar
Europe PMC