ORIGINAL RESEARCH ARTICLE
Alessio Suriana* and Andrea Sciandrab
aUniversity of Padova, Padova, Italy; bUniversity of Modena and Reggio Emilia, Reggio Emilia, Italy
(Received: 8 September 2018; Revised: 28 June 2019; Accepted: 25 July 2019; Published: 23 August 2019)
Recent studies indicate that Internet skills have a positive impact on academic achievement. This article presents a national study that seeks to validate an Internet skills scale that was already tested in other EU countries (the Netherlands and the United Kingdom) to understand the competence level of the population as a whole as well as across population sectors. The scale questionnaire was completed by a sample of the Italian population stratified by gender, age and geographical area. The result is globally consistent at the empirical level as well as at the cross-national level. All the five scales showed excellent internal consistency.
Keywords: digital inclusion; Internet skills; skill recognition; survey research; reliability; confirmatory factor analysis
*Corresponding author. Email: alessio.surian@unipd.it
Research in Learning Technology 2019. © 2019 A. Surian and A. Sciandra. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2019, 27: 2155 - http://dx.doi.org/10.25304/rlt.v27.2155
This Italian national study seeks to validate an Internet skills scale (ISS) (van Deursen et al. 2017) that has previously been tested in other EU countries (the Netherlands and the United Kingdom) with a view to determining Internet competence level of the population as a whole as well as across population sectors. The debate and the definition of competencies and skills are of fundamental importance to educational policies. Both 21st-century skills and digital skills are considered crucial to future socio-economic and educational scenarios, albeit this combination has been neither sufficiently addressed nor defined. To this purpose, van Laar et al. (2017, p. 578) introduce the concept of ‘21st-century core and contextual digital skills’. Their framework includes seven core skills (technical, information management, communication, collaboration, creativity, critical thinking and problem-solving) and five contextual skills (ethical awareness, cultural awareness, flexibility, self-direction and lifelong learning). Such skills are defined as critical for both people and organisations because they determine technological developments and innovating products and processes. Utilising van Laar et al. (2017) framework, the present study is concerned with the specific skills that relate to a competent use of the Internet. Gui, Fasoli, and Carradore (2017, p. 169) claim that individual ‘digital well-being’ is not a function of personal ‘digital well-being skills’ alone, but it is also dependent on the norms, values and expectations of the social context.
The specific skills that relate to a competent use of the Internet need to respond to the development of Internet and digital technologies. Such development is having an impact on the demand for skills and the design and implementation of key policies to foster skills development that take into account the digital world and related educational and vocational policies. The Internet requires specific facilities and competencies (van Deursen, Helsper, and Eynon 2016; Weinert et al. 2011, p. 82), including the identification and management of the appropriate tools and information, the ability to access contextual operations, the ability to assess and to communicate data and information according to specific contexts. A better awareness of these issues by educational environments enables such organisations to improve the way they address inequalities in terms of Internet access (first-level digital divide); Internet skills and use (second-level digital divide and the focus of this paper) and third-level digital divide which highlights the tangible outcomes of Internet use (Scheerder, van Deursen, and van Dijk 2017). This is particularly relevant in the learning and education domains where online learning management systems, massive online open courses and open educational resources expand opportunities for learning. In addition, online educational games offer opportunities to develop collaborative and communication skills. Recent studies indicate that Internet skills have a positive impact on academic achievement, with school performance being positively associated with longer Internet use for study (Kim et al. 2017; Leung and Lee 2012). This effect is found to be stronger for students with low academic performance or poor socio-economic background and, conversely, for students in technical or vocational schools (Pagani et al. 2016).
According to OECD (2016), the increasing use of online digital technologies is reshaping everyday activities as well as the business models and organisational policy, suggesting that digital literacy skills are becoming increasingly important. Based on these observations, the European Union has defined a Digital Competence Framework (Carretero, Vuorikari, and Punie 2017). The framework is articulated in five competence areas each of which incorporates Internet skills. The present article focuses on one EU country, Italy. An interesting aspect of the study is the validation of the ISS in a country that, according to the Italian Digital Agency (OECD 2017), lies behind EU average in terms of digital skills, as well as in the number of Internet users. Specific aims are twofold: (1) an exploration into how digital exclusion may be affecting vulnerable groups in society (Helsper and Reisdorf 2017), and (2) to determine the types of information that might be relevant to inform learning policies and actions. It is argued that this is especially important should it be necessary to address biases in the provision of learning and training opportunities for low-skilled adults (OECD 2017, p. 108).
Based on the principle that Internet skills represent a fundamental element of digital inclusion, van Deursen, Helsper, and Eynon (2014, 2016) proposed a definition centred on individual abilities because Internet skills are required to access the key features of the information society. Such skills relate to a socially accepted and effective use of the Internet in accordance with established standards such as netiquette. Internet skills are distinct from computer skills because Internet use implies particular actions such as searching for information, online communication and creating online content. van Deursen, Helsper, and Enyon (2014, 2016) include theoretically informed survey measures of individual’s digital skills, as their conceptualisation accounts for technical aspects (medium-related skills) and substantial aspects (content-related skills). In this way, a technologically deterministic interpretation of Internet skills is avoided by enhancing the role of content-related skills along with technology (i.e. medium)-related skills.
Through the development of operationalisation and validation of the ISS in many studies conducted, particularly in the Netherlands, van Deursen, Helsper, and Eynon (2014, 2016) determined a framework consisting of five skill sets: (1) operational skills are the basic technical skills required to access on the Internet (e.g. opening a Website, navigating forward and backward by using browser buttons, opening, sending or receiving email, using a search engine, managing different file formats); (2) information navigation skills relate to searching, selecting and evaluating information on the Internet or, in other words, to maintain the orientation in navigation; (3) social skills include several abilities among which communicating and interacting online to understand and exchange meaning; managing contacts online; attracting attention online and profiling; (4) creative skills are essential to create content to be published or shared with others on the Internet (e.g. text, photo, video) and (5) mobile skills relate to the ability to install and to monitor mobile devices newer applications. Consistently with the work by van Deursen et al. (2017), the Italian validation of ISS provides an overall framework including both Web 1.0 activities implying Operational and Information Navigation skills, and Web 2.0 activities implying social and creative skills. Finally, as the skills in using mobile devices are currently considered very crucial, it was decided to keep mobile skills as a separate scale.
The five-dimension ISS uses the Likert format for each response item, ranging from ‘Not at all true of me’ to ‘Very true of me’ in terms of the respondents’ behaviour related to Internet skills truth claims. In this way participants indicate the extent to which they believe each item to be true of them, or they can choose the ‘I do not understand what this means’ option. The complete ISS includes 35 items, while a shorter version includes 23 items only. We elected to validate the 35-item scale for the Italian population. The higher number of items provides a higher probability to identify cultural diversity issues and therefore to control for potential cross-cultural issues as well.
The ISS implementation in the United Kingdom and the Netherlands indicated promising results in terms of reliability and external, convergent and discriminant validity (van Deursen, Helsper, and Eynon 2014, 2016), so we chose to validate the questionnaire1 through a representative Italian sample, as explained below. Moreover, we added one new Internet skills item addressing Open Source Software (OSS) to allow further investigations and questions about respondents’ profiles to check potential groups comparison variables.
The study has been structured into three main phases: pre-test, pilot survey and full test validation.
During the pre-test phase, our research team translated the 35 items from English to Italian. The first step of the process of cross-national validation began with a group discussion to verify the understanding of the Italian translation, especially to detect issues of cultural diversity, that is, items that could not be understood as intended by the survey developers. Before starting the pilot survey, we conducted interviews aimed at checking item understanding with subjects representing different segments of the Italian population according to age, gender and education. In this way we explored whether there was any problem regarding item understanding and question formulation. We did not detect any issue of cultural adaptation. Two minor changes were required to achieve better comprehension: we replaced Tumblr with Instagram in an example within the Social Subscale; smartphone and tablet were added as examples of ‘mobility’ within the Mobile Subscale. As recommended by van Deursen, Helsper, and Eynon (2016), the information navigation skill scores were reversed because this scale contained negatively worded items. The study’s second phase included a pilot survey to validate the Italian translation of the instrument, in particular by testing the reliability of the five scales. In the last phase of the study, we conducted a full test validation through a representative sample survey of Italian Internet users, including reliability analysis and general fit of the model to the data (Cronbach’s alphas, confirmatory factor analysis [CFA]); external validity analysis (checking the averages across the scales for different socio-demographic groups; convergent and discriminant validity of the scales) and scale consistency (stability of the factor solution; correlations analysis between factors within different socio-demographic groups).
After the pre-test, we conducted pilot tests with an online questionnaire in May 2017. The aim of the pilot tests was to verify the Italian translation and to check the reliability of the five ISS scales. A total of 61 respondents completed the online survey. From a demographic perspective, we determined the respondents’ age, gender, diversity of professional and academic backgrounds.
The Operational, Social and Mobile scales showed significantly high means, but these values are very close to the ones of the Dutch survey (van Deursen, Helsper, and Eynon 2016).
As shown in Table 1, out of five alpha coefficients only one (Social) was below 0.80, the other four scales show alphas equal to 0.85. Only three items, if deleted, would increase their scale, Cronbach’s alphas. Only one of these three items belongs to the questionnaire short form, so our decision to use the full-scale questionnaire is corroborated by this result. These results made us very confident about the Italian translation and the five ISS scales.
| Mean | Variance | a | |
| Operational | 4.52 | 1.05 | 0.85 |
| Information Navigation | 2.01 | 1.25 | 0.85 |
| Social | 4.35 | 1.27 | 0.78 |
| Creative | 2.95 | 2.15 | 0.85 |
| Mobile | 4.32 | 1.40 | 0.85 |
This section explores whether the scales constructed and validated in the Netherlands show reliability, internal and external validity across the Italian population of Internet users. As suggested by van Deursen, Helsper, and Eynon (2016), by including other cultures in further investigations it should be possible to truly test the cross-cultural validity of the instrument.
Therefore we examined the consistency of the ISS with a representative sample survey of Italian Internet users. For this purpose, we used the professional services of SWG, a research institute that created a 60 000-people online panel that is believed to be a largely representative sample of Italian population. As members respond to some surveys, they accumulate points that allow them to receive rewards. SWG panel includes users with different Internet skill levels, so it should fit for our cross-national validation framework. On July 2017, a questionnaire comprising ISS was completed by a sample stratified by gender, age and geographical area (data taken from the National Institute of Statistics – Istat) of 1067 adults, for a confidence level of 95% and a margin of error of 3%. Each of the participants took approximately 8–9 min to complete the questionnaire. Table 2 shows the characteristics of the respondents.
The following statistical analyses recall those carried out by van Deursen, Helsper, and Eynon (2016). When necessary, we added more analyses to explore critical results further.
We tested ISS reliability and fit on the Italian population survey by looking at Cronbach’s alphas and CFA.
Table 3 shows that all five scales have excellent internal consistency because no alpha is below 0.90 and each of them is systematically higher than the equivalent scale of the Dutch sample (van Deursen, Helsper, and Eynon 2016).
| Mean | Variance | a | |
| Operational | 4.53 | 0.93 | 0.92 |
| Information Navigation | 4.38 | 0.89 | 0.90 |
| Social | 4.46 | 0.95 | 0.91 |
| Creative | 3.43 | 1.90 | 0.92 |
| Mobile | 4.25 | 1.36 | 0.91 |
A CFA was carried out on the whole sample to test the validity of the five-factor model. Mobile scale is not analysed separately in Table 4 because it is made up of three items, thus this model is saturated. Fit assessment was based on several indices. As the χ2 statistic is dependent on sample size, we considered also the Comparative Fit Index (CFI), as it performs well with large samples. Values above 0.95 are usually considered satisfactory (Schermelleh-Engel, Moosbrugger, and Muller 2003). The root mean square error of approximation (RMSEA) was also used. This is an absolute fit index assessing approximation of parameter estimates to true parameters in the population. Values below 0.08 reflect good fit (Schermelleh-Engel, Moosbrugger, and Muller 2003).
The theoretical model provided an excellent fit to the observed data: all the subscales and the overall scales model show optimal CFI and RMSEA values, very similar to those deriving from Dutch survey. Furthermore, all estimated factor loadings were significant and generally high (>0.50) confirming very good levels of internal factor structure validity.
Correlations of the subscales are presented in Table 5. All the correlations between the scale constructs were significant in the Italian population sample and higher than the Dutch sample. We must note that the large samples boost levels of significance.
In this section we examined the averages across the scales for different socio-demographic groups and tested for convergent and discriminant validity of the scales. Next, we tested the convergent and discriminant validity of the scales and whether the scale characteristics would be consistent through random resamples of the population using bootstrapping techniques.
Reliability analysis shows that all the scales have rather similar alphas across different socio-demographic groups (Table 6). Most of the Cronbach’s alphas are above 0.90, while the lowest values were found in the student group. We must stress that there were only 58 students in our adults’ sample, their ages ranging from 18 to 36 (with 23.04 as mean age). All the differences between groups were significant, as predicted by the literature and confirmed in the Dutch sample: males estimated their own skills higher than females; younger generations estimated their skills higher than older generations; those with higher educational levels were more confident and the employed; and students were more confident having the highest skill levels.
Composite reliability (CR), average variance extracted (AVE), maximum shared variance (MSV) and average shared variance (ASV) tests were run to understand whether the factor models show convergent and discriminant validity. In other words, we wanted to test whether the factors were internally coherent and could be externally distinguished from other factors. The analyses reported in Tables 7 and 8 provided evidence for the convergent validity because all reported CR values are above 0.70 and AVE values for the variables are above 0.50 (Fornell and Larcker 1981; Hair et al. 2010). The above analyses suggest that the latent variables have high convergent validity, suggesting that the items on the scales highly correlate with each other, based on their factor loadings.
The Social, Creative and Mobile dimensions exhibited discriminant validity, that is, the AVE exceeded the squared correlation for the specific paired constructs. Therefore, in these three scales the items correlated more highly with each other than they did with items that were on other factors. Yet, the AVE for Operational and Information Navigation scales is smaller than maximum shared squared variance (MSV) and average shared squared variance (ASV). Moreover, the square root of AVE for Operational and Information Navigation scales is less than the absolute value of the correlations with another factor. These results show that there is a discriminant concern for the first two scales, so further testing was undertaken, as we conducted a χ2 difference test for the overall scales fitted model with a model with four factors (Operational and Information Navigation scales had been merged in a unique scale). By using a scaled difference χ2 (Satorra and Bentler 2001), we assumed that a large change in χ2 compared to the difference in degrees of freedom indicates that the freed parameters constitute a real improvement. The χ2 difference was significant (χ2=406.19, df=4, p<0.01), indicating that the five-factor solution was significantly more accurate in its reconstruction of the observed data than the four-factor solution (Bagozzi and Yi 1988) and supporting the discriminant validity hypothesis.
To examine whether the factor solution was stable, we conducted a Bollen and Stine (1992) test for the overall scales model with the ‘lavaan’ package of R software (Rosseel 2012). With this package, the standard p-value of the χ2 test statistic is supplemented with a bootstrap probability value, obtained by computing the proportion of test statistics from the bootstrap samples that exceed the value of the test statistic from the original sample. We chose 2000 bootstrap draws and in all samples the fit was better than in the original model (p <0.001), so we can be confident that the overall scales model is a stable solution for the Italian population.
Following the results of the discriminant analysis for Operational and Information Navigation scales, we focused on examining correlations between these two factors within the different socio-demographic groups, using Fisher’s r-to-z transformation test. In summary, in comparing the correlations among the different skills scales across age groups, the Operational and Information Navigation skills scales seem to imply a ‘digital exclusion’ dimension as they present higher correlations with the older, the primary education and the retired people.
In this article we examined the consistency of the ISS when measured in a representative sample survey of Italian Internet users. The result is globally consistent at the empirical and cross-national levels. All the five scales showed excellent internal consistency, with Cronbach’s alpha higher than 0.90. Theoretically, the subscales and the overall scales model proved to have optimal fit indexes in the CFA. The analyses provided evidences for external and convergent validity as well. Some concerns appeared in the discriminant validity for the Operational and the Information Navigation scales, even though the five-factor solution fit was significantly more accurate than the four-factor solution. van Deursen, Helsper, and Eynon (2016) stress that several items which are often used in Information Navigation scale loaded on different factors and recommended to include additional items for further investigation. In the Italian survey we included an additional item concerning OSS, that is, ‘I am able to identify proprietary software vs open source software’. This responded to a specific concern. In the first instance, because we noticed that the Dutch survey and the Italian pilot survey showed significantly high mean scores in three scales, the new item was introduced as a more difficult benchmark to achieve for an Internet user. In fact, the OSS question mean score was lower than each scale mean score, except for the Creative scale. Similarly, the Italian survey identified a significantly high mean score for the ISSs, as shown in Table 3. This suggests that in future studies it might be useful to include updated items (Litt 2013) such as those that address relevant social issues.
Moreover, reliability comparison between socio-demographic groups indicated higher correlations between the Operational and Information Navigation skills for the older class (61+), with primary education and retired people. This result is consistent with previous studies that highlight that digital exclusion is becoming more concentrated in (economically and socially) vulnerable groups, as it was found in Sweden and the United Kingdom (Helsper and Reisdorf 2017). In the Netherlands, the elderly and those with lower levels of education had fewer skills, engaged less and achieved fewer outcomes (van Deursen et al. 2017). Litt (2013) reviewed several studies finding strong evidence of a positive relationship between education level and Internet skills, while mixed (positive and negative) associations were identified in the relationship between age and Internet skills. In future studies it would be important to address Litt’s (2013, p. 621) distinction between technical versus evaluation Internet skills. This should allow a better understanding of the relationship between technology (i.e. medium)-related skills versus content-related skills and their evolution not just in terms of age but in terms of educational dynamics as well.
According to Helsper and Reisdorf (2017), patterns of digital exclusion persist even when access is almost universal and many services are only available online. Thus, further research would be important to understand why specific groups of people are not taking advantages of existing digital opportunities and what type of learning opportunities could be made available to support them.
Looking at the whole sample, the five average scale scores are systematically higher in the Italian sample than in the Dutch one. The possible explanations may concern a more generous self-perception of their skills by the Italians and/or the possibility that after 3 years since the Dutch survey, some skills have improved in the general population. The misalignment between self-perceived skills and actual performances is typical in proxy survey measures because they represent one of the most prevalent ways of measuring Internet skills (Litt 2013). However, survey measures are usually less time- and budget-consuming and more suitable for researching large samples. Therefore, we argue that a regular assessment and improvement of the ISS would be essential for the instrument durability.
The misalignment between self-perceived skills and performance-based skills indicates a limitation of this study. In performance-based and observation Internet skills studies, participants are asked to complete a set of tasks while the researcher observes and evaluates their behaviours. Clearly, performance/observation methods present higher validity, despite the known issues concerning reliability and generalisability (Litt 2013). We believe that a combination of self-reported data and performance-based measures (Gui and Argentin 2011) would be a possible improvement of the Internet skills collection, as the diffusion of smartphones and related apps could make it easier to record performances.
Another limitation refers to the sample population because we used an online panel. Arguably, however, we were seeking to determine Internet skills and would have utilised an alternative research design and instrument of inquiry if Internet non-users were to be included.
Globally, in the Italian validation we did not detect any issue of cultural diversity because only minor changes were required to achieve better comprehension. The scores of the Information Navigation scale were reversed because these items were all negatively formulated. As recommended by van Deursen, Helsper and Eynon (2016), future research should use positively formulated items to measure these skills. No other item needed to be adjusted for cultural difference, so we suggest that future Italian research about Internet skills should use a faithful translation of the original ISS, including some updated items.
Tools such as the ISS can be instrumental in monitoring key aspects of digital inclusion and to provide potential benchmarks for educational and inclusion policies. Pagani et al. (2016) suggest that an increase in digital skills can be specifically helpful in the case of population groups lacking significant cultural stimuli from their family and social context. Their rationale is that digital skills can act as a substitute for family background when the latter is poor and they should be a key element in designing both digital inclusion and broader educational policies. ISS data offer an opportunity to design activities in at least three areas that are considered crucial in digital inclusion policies by both the European Commission (Cedefop 2017) and OECD (2017): (1) skills assessment to enable adults to identify their existing skills and ways to upskill; (2) learning offer, aiming at boosting digital skills and/or at allowing progress towards higher qualifications and (3) validation and recognition of adults’ skills. Further research is needed to determine the development of Internet skills and digital competencies not just as skills that are functional to the job market but also as overall personal growth opportunities, that is, in relation to the role that education plays in supporting the ‘socialisation’ and ‘subjectification’ of individuals in the broadband society (Biesta 2015; UNESCO 2017).
Bagozzi, R.P. & Yi, Y. (1988) ‘On the evaluation of structural equation models’, Journal of the Academy of Marketing Science, vol. 16, no. 1, pp. 74–94.
Biesta, G. (2015) ‘What is education for?’, European Journal of Education, vol. 50, no. 1, pp. 75–87. https://doi.org/10.1111/ejed.12109.
Bollen, K.A. & Stine, R.A. (1992) ‘Bootstrapping goodness-of-fit measures in structural equation models’, Sociological Methods & Research, vol. 21, no. 2, pp. 205–229. https://doi.org/10.1177/0049124192021002004.
Carretero, S., Vuorikari, R. & Punie, Y. (2017) DigComp 2.1: The digital competence framework for citizens with eight proficiency levels and examples of use (No. JRC106281), Joint Research Centre (Seville site), EUR 28558 EN, Publications Office of the European Union, Luxembourg. https://doi.org/10.2760/38842.
Cedefop. (2017) On the Way to 2020: Data for Vocational Education and Training Policies. Country statistical overviews – 2016 update, Publications Office, Luxembourg, Cedefop research paper; No 61. https://dx.doi.org/10.2801/414017
Fornell, C. & Larcker, F.D. (1981) ‘Evaluating structural equation models with unobservable variables and measurement error’, Journal of Marketing Research, vol. 18, no. 1, pp. 39–50.
Gui, M. & Argentin, G. (2011) ‘Digital skills of Internet natives. Different forms of digital literacy in a random sample of northern Italian high school students’, New Media & Society, vol. 13, no. 6, pp. 963–980. https://doi.org/10.1177/1461444810389751.
Gui, M., Fasoli, M. & Carradore, R. (2017) ‘“Digital well-being”. Developing a new theoretical tool for media literacy research’, Italian Journal of Sociology of Education, vol. 9, no. 1, pp. 155–173. https://doi.org/10.14658/pupj-ijse-2017-1-8.
Hair, J., et al., (2010) Multivariate Data Analysis: A Global Perspective, Pearson Education, Singapore.
Helsper, E. J. & Reisdorf, B. C. (2017) ‘The emergence of a “digital underclass” in Great Britain and Sweden: Changing reasons for digital exclusion’, New Media & Society, vol. 19, no. 8, pp. 1253–1270. https://doi.org/10.1177/1461444816634676.
Kim, S.Y., et al., (2017) ‘The associations between internet use time and school performance among Korean adolescents differ according to the purpose of internet use’, PLoS One, vol. 12, no. 4, pp. e0174878. https://doi.org/10.1371/journal.pone.0174878
Leung, L. & Lee., P. (2012) ‘Impact of Internet literacy, Internet addiction symptoms, and Internet activities on academic performance’, Social Science Computer Review, vol. 30, no. 4, pp. 403–418. https://doi.org/10.1177/0894439311435217.
Litt, E. (2013) ‘Measuring users’ internet skills: A review of past assessments and a look toward the future’, New Media & Society, vol. 15, no. 4, pp. 612–630. https://doi.org/10.1177/1461444813475424.
OECD. (2016) Skills for a digital world, Panel 4.2 background document, Ministerial Meeting, The Digital Economy: Innovation, Growth and Social Prosperity, Cancún, Mexico, 21–23 June 2016.
OECD. (2017) Skills strategy diagnostic report: Italy, OECD, Paris.
Pagani, L., et al., (2016) ‘The impact of digital skills on educational outcomes: Evidence from performance tests’, Educational Studies, vol. 42, no. 2, pp. 137–162. https://doi.org/10.1080/03055698.2016.1148588.
Rosseel, Y. (2012) ‘lavaan: An R Package for structural equation modeling’, Journal of Statistical Software, vol. 48, no. 2, pp. 1–36. https://doi.org/10.18637/jss.v048.i02.
Satorra, A. & Bentler, P. M. (2001) ‘A scaled difference chi-square test statistic for moment structure analysis’, Psychometrika, vol. 66, no. 4, pp. 507–514.
Scheerder, A., van Deursen, A.J.A.M. & van Dijk, J.A.G.M. (2017) ‘Determinants of Internet skills, uses and outcomes. A systematic review of the second- and third-level digital divide’, Telematics and Informatics, vol. 34, pp. 1607–1624. https://doi.org/10.1016/j.tele.2017.07.007.
Schermelleh-Engel, K., Moosbrugger, H. & Muller, H. (2003) ‘Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures’, Methods of Psychological Research Online, vol. 8, no. 2, pp. 23–74.
UNESCO Broadband Commission. (2017) Working Group on the Digitalization Scorecard: Which Policies and Regulations Can Help Advance Digitalization, Paris, Available at: http://www.broadbandcommission.org/Documents/publications/WG-Digitalization-Score-Card-Report2017.pdf
van Deursen, A.J.A.M., Helsper, E.J. & Eynon, R. (2014) Measuring Digital Skills. From Digital Skills to Tangible Outcomes project report. Available at: www.oii.ox.ac.uk/research/projects/?id=112
van Deursen, A.J.A.M., Helsper, E.J. & Eynon, R. (2016) ‘Development and validation of the Internet Skills Scale (ISS)’, Information, Communication & Society, vol. 19, no. 6, pp. 804–823. https://doi.org/10.1080/1369118X.2015.1078834.
van Deursen, A.J.A.M et al., (2017) ‘The compoundness and sequentiality of digital inequality’, International Journal of Communication, vol. 11, pp. 452–473.
van Laar, E., et al., (2017) ‘The relation between 21st-century skills and digital skills or literacy: A systematic literature review’, Computers in Human Behavior, vol. 72, pp. 577–588. https://doi.org/10.1016/j.chb.2017.03.010.
Weinert, S., et al., (2011) ‘Development of competencies across the life span’, Zeitschrift für Erziehungswissenschaft, vol. 14, pp. 67–86. https://doi.org/10.1007/s11618-011-0182-7
1. The ISS items are available at http://alexandervandeursen.nl/Joomla/Articles/Journal/2015%20-%20ISS.pdf.