ORIGINAL RESEARCH ARTICLE
A cross-sectional study of video game play habits and graduate skills attainment
Matthew Barr
School of Computing Science, University of Glasgow, Glasgow, Scotland
(Received: 24 August 2019; Revised: 26 December 2019; Accepted: 31 January 2020; Published: 28 February 2020)
Abstract
Using a survey of higher education students (N = 2145), correlations between game play habits and the attainment of certain graduate skills or attributes (communication skill, adaptability and resourcefulness) are presented. Correlations between graduate attribute attainment and a range of demographic and educational factors, including age, gender, level of study and year of study, are also calculated. While it is shown that there is no significant relationship between existing game play habits and graduate attribute attainment, several broad observations are made. Students who do not play video games tended to score best, while those students who play games in a variety of modes (online and local cooperative play, team-based and other cooperative play) also scored better on measures of graduate attribute attainment. Assumptions about the development of graduate attributes over time are also challenged by the data presented here, which suggest there is little correlation between attribute attainment and years spent at university. The work suggests that, while video games may be used to develop graduate skills on campus, there is no strong correlation between existing game play habits and the attainment of certain transferable skills.
Keywords: video games; game-based learning; graduate skills; graduate attributes; employability; communication; resourcefulness; adaptability
*Corresponding author. Email: Matthew.Barr@glasgow.ac.uk
Research in Learning Technology 2020. © 2020 M. Barr. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2020, 28: 2326 - http://dx.doi.org/10.25304/rlt.v28.2326
Introduction
Evidence produced by a randomised experimental study, Barr (2017), suggests video games may, under certain conditions, be used to develop generic skills in a fun and engaging way. Gains were reported in students’ self-reported communication skill, resourcefulness and adaptability – skills commonly referred to as ‘graduate attributes’ (Hughes and Barrie 2010) – following an 8-week experiment, wherein students played an average of 2 h of specified video games per week under lab conditions. As Barrie (2006) has suggested, it is not generally agreed how best the attributes of our students may be developed at university. The study above pointed to one potential solution to this problem, especially where there is student resistance to the inclusion of specific skills development activities in the curriculum (O’Leary 2016) or students are faced with competing commitments such as part-time or voluntary work (Gbadamosi, Evans, Richardson, and Ridolfo 2015). As explored in Barr (2018), students involved in the experimental study were broadly positive about the idea of playing games on campus to develop their graduate attributes, suggesting that game-based skills development activities may be met with less significant resistance.
While it has been suggested that playing video games may improve communication skill (Griffiths 2003), there is little empirical evidence for the efficacy of game-based approaches to the development of such skills in higher education prior to Barr (2017). An inconclusive picture emerges from the literature on the effects of video game play on academic attainment, with some studies finding that improved performance is associated with video game play, and others finding the opposite to be true. For example, Posso (2016) found that playing online video games was positively correlated with students’ academic performance, while use of online social networks was seen to have the reverse effect. In a longitudinal study of the effects of Internet use and video game playing on children’s academic performance, Jackson et al. (2011) found that game play was negatively related to grade-point average (GPA), but only for children with initially average GPAs. Meanwhile, Anand (2007) also found that as video game usage increased, students’ GPA decreased.
However, while this work is not concerned with academic performance, as represented by GPAs, several authors have suggested that video games may help develop ‘21st century skills’, akin to the graduate attributes considered here. Romero, Usart, and Ott (2015), for example, highlight social skills, critical thinking, problem-solving and productivity as abilities that have been shown to be exercised by ‘serious games’: games developed for purposes other than pure entertainment but not necessarily, as Romero et al. note, to develop 21st century skills. More recently, Alonso-Díaz, Yuste-Tosina, and Mendo-Lázaro (2019) have suggested that playing video games, particularly role-playing games, promotes the acquisition of ‘key competencies for a globalised society’. And several studies have suggested that playing video games can provide perceptual, attentional and cognitive benefits to players (Green and Bavelier 2015) and help develop cognitive and non-cognitive abilities, including problem-solving, spatial skills and persistence (Shute, Ventura, and Ke, 2015). Indeed, the work described by Shute et al. also involves one of the games (Portal 2) used in the Barr (2017) experimental study that suggested playing games in a university lab can develop related attributes such as resourcefulness.
So, if games may be used to develop graduate attributes on campus, what is the relationship between students’ attribute attainment and their existing, extracurricular game play habits? Building on the 2017 experimental study and using the same previously validated instruments to measure certain attributes, a cross-sectional survey was conducted to collect demographic and gaming-related data from a sample of students (N = 2145) drawn from the student population of a single Scottish university. By surveying the game play habits of a broader student cohort, in conjunction with collecting scores on attribute-measuring instruments, it was possible to determine if there is any correlation between the existing game play habits and the self-reported graduate attribute attainment. It was also possible to assess if there is any correlation between the level of university study (undergraduate [UG], postgraduate taught [PGT], postgraduate research [PGR]) or year of study (first, second, third, etc.) and the attainment of these attributes.
Methods
In this study, four measures were used: two related to communication and one each related to resourcefulness and adaptability. It was thought important to use the same measures as those used in the previous experimental study to facilitate comparisons between results. The measures were originally selected on the basis that they addressed graduate attributes claimed to be developed by the host university, were straightforward to administer at scale, were suitable for use in a pre-/post-test design (as used in the previous experimental study) and were published with some indications of their validity.
Duran’s Communicative Adaptability Scale (CAS) (1983, 1992) is a self-report measure of communication ability. The scale has been used in several studies and produced Cronbach’s alpha coefficients between 0.74 and 0.84, indicating good reliability (Duran 1992). The communication competence that the Self-perceived Communication Competence Scale (SPCCS) is intended to measure is defined by the authors as ‘adequate ability to pass along or give information; the ability to make known by talking or writing’ (McCroskey and McCroskey 1988). Building on the work of Pulakos et al. (2000, 2002), Ployhart and Bliese (2006) suggest a self-report measure of adaptability based on their own Individual ADAPTability (I-ADAPT) theory, in which they define adaptability as an ‘individual difference construct that influences how a person interprets and responds to different situations’. The measure has been validated by means of a confirmatory factor analysis, and a construct validity study of the original 40-item measure ‘found strong support for convergent and discriminant validity’ (Ployhart and Bliese 2006). Zauszniewski, Lai, and Tithiphontumrong (2006) offer a 28-item Resourcefulness Scale, developed and validated in a two-phase study with chronically ill elderly patients. The authors found that the scale had acceptable internal consistency (Cronbach’s alpha = 0.85).
The survey also included additional questions derived from the university definitions of the three attributes being measured, based on the stated personal and transferable dimensions of each123. Each of these dimensions was arranged as a statement, and the participants were asked to indicate on a five-point Likert scale the extent to which they agreed or disagreed with the statement. For example, ‘I communicate clearly and confidently, and listen and negotiate effectively with others’. This provided an alternative means of measuring these attributes – incontrovertibly tied to the university’s conception of each – that may also be correlated with the measurements obtained via the instruments outlined above. The motivation for including these otherwise unvalidated measures was to determine if the university-defined graduate attributes were mapped to the validated instruments. For example, the instruments used to measure communication would be expected to correlate most closely with the university definitions of Effective Communicators. The measures based on university definitions are not used in any primary analysis; they serve only to provide confidence that the validated measures are appropriate.
In summary, 10 measures were used: four intended to measure communication (the previously validated CAS and SPCCS, plus two measures based on the university definition of Effective Communicators), three intended to measure adaptability (I-ADAPT-M plus two based on the university definition) and three to measure resourcefulness (the Resourcefulness Scale plus two based on the university definition).
The study population was drawn from students at the host university. Ideally, the study sample would have been selected at random from the study population (e.g.a random sample of all students at the institution) to ameliorate sampling errors (Coggon, Rose, and Barker 2013). In practice, since the study relied on volunteers, the study sample was, to some extent, self-selecting and therefore prone to bias. It is reasonable to assume, for example, that students with an active interest in video games are more likely to volunteer to take part in a study that concerns games. To address this potential bias, volunteers from each of these groups were recruited by means of advertisements that encouraged non-players to participate. Further, background data collected on each participant included a set of items that pertained to gaming habits in addition to the essential demographic information (age, gender, etc.) and that relating to their university studies (subject, year of study, etc.). These data allowed observations to be made about groups of participants with varying characteristics and exposure to video games.
Data were collected by means of an online survey that reproduced the attribute-measuring instruments and additional questions described above. Students across all levels at the host institution were recruited by email, and a prize draw for Amazon vouchers was offered as a means of incentivising respondents, including those for whom video games hold no appeal. In total, 2145 responses were collected via the online survey, accounting for 8.4% of the total student population at the host institution (University of Glasgow 2015).
Considering quantitative and qualitative data obtained in the previous experimental study, questions relating to preference for multiplayer and cooperative play were added to the survey. The survey, then, was designed a priori to determine:
- Is there a correlation between game play habits and self-reported communication, adaptability and resourcefulness scores? Specifically, are there correlations with:
a. Hours played per week (including non-players),
b. Preference for multiplayer gaming,
c. Preference for co-op gaming?
- Do students believe video games might help develop any useful skills or provide any valuable experience? And, if so, what skills or experience?
As noted by Anand (2007), however, demonstrating any causal relationship ‘remains difficult because of the complex nature of student life and academic performance’, noting that ‘video game usage may simply be a function of specific personality types and characteristics’. Such limitations are inherent in any study of similar cross-sectional design, including that described here, regardless of whether such limitations are made plain. Ethical approval for carrying out the survey was granted by the College of Arts Ethics Committee.
Description of survey respondents
A total of 72% of respondents indicated that they were pursuing an UG programme of study, with 15% on a taught PG programme and 14% pursuing a PGR degree (29% total PG population). The proportion of UG and PG respondents closely mirrors those figures for the university, which reports a 71% UG population and 29% PG population (University of Glasgow 2015); 59% of respondents were female and 40% of respondents were male, with <1% of respondents not identifying as female or male. This matches the 59% female population reported by the university for the relevant academic session (University of Glasgow 2016). In terms of these simple demographic data, then, the sample is remarkably representative of the university population. Participants’ year of study initially seems skewed towards first-year students, who account for 39% of responses. However, this is largely explained by the fact that most Master’s degrees are 1 year in duration, and, of course, a proportion of UG students will drop out of their studies after the first year. Compared with internal institutional data for the academic year in question, which reveals that 32% of students were enrolled in the first year of their programme, the figures are still not quite representative, but they are not as skewed as may initially appear to be the case. The mean age of respondents was 22.76, the median age was 21 and the age range was 16–65.
Around 63% of respondents stated that they play video games, while the majority of those who do play video games report that they do so for between 1 and 4 h per week. These figures are somewhat dissimilar to those released by GameTrack (GameTrack [ISFE/Ipsos Connect] 2016), which indicate that just 40% of the UK population played video games in Q1 2016, albeit for an average of 8.8 h per week. However, UKIE (the Association of UK Interactive Entertainment) also notes alternative figures produced by Newzoo, which suggest that 57% of the UK population plays games (‘The games industry in numbers | Ukie’ 2016).
The proportion of respondents who selected the option ‘I don’t play video games’ on subsequent questions varied slightly. For example, in answer to the question ‘What kind of games do you like to play?’, 35% of respondents stated they did not play video games, compared to the 37% of respondents who claimed they did not play games in the earlier question. In subsequent analysis, such contradictory responses are removed from game play totals and treated as an ‘unknown’ group.
Results and discussion
As described above, additional questions derived from the university definitions of the three attributes being measured were included in the survey. These related to the personal and transferable dimensions of each of the Effective Communicators, Adaptable, and Resourceful and Responsible attributes. Calculating correlations between the published measures and simple questions based directly on institutional definitions of the attributes under consideration allows a broad assessment of the relevance and applicability of the chosen measures to be made. Such assessments are instructive when considering the differences between results for the two communication measures, for example. To this end, Spearman’s rank correlation coefficient (Spearman’s rho) was calculated using R statistical software to determine the correlation between CAS and SPCCS scores and responses to the above attribute-based questions. It is notable that CAS correlated more strongly with scores on the attribute definition-based questions (transferable dimension, rho = 0.52; personal dimension, rho = 0.48) than those for the SPCCS measure (transferable dimension, rho = 0.48; personal dimension, rho = 0.33). Thus, the measure of communication skill that showed the greater post-test improvement for intervention group participants in the earlier experimental study is also that which correlates more closely with the university definitions of effective communication. The correlogram in Figure 1 summarises the degree of correlation between all measures. Variables were clustered based on their co-variance and are ordered based on this clustering (see Friendly 2002, for an explanation of the formula used by the R statistical package to calculate clustering).
Figure 1. Correlations between graduate attribute measures. Correlation coefficients are Spearman’s rank correlation coefficient. Variables were clustered based on their co-variance and are ordered based on this clustering.
Perhaps the most striking observation that may be made about the correlogram is that all the correlations are positive, as the scores on any one attribute measure increase, so too do the scores on every other measure, to varying degrees. This overall relationship suggests that the graduate attributes measured here are all related or that they could be facets of the same phenomenon. Looking more closely, however, it is notable that the Resourcefulness Scale scores are generally those that correlate most weakly with all other scores, including measures based on the university definitions of this attribute. This is evidenced by the appearance of a lightly shaded (low correlation) cross that emerges from the intersection of the row and column depicting correlations with the Resourcefulness scores. As noted above, the plotting of the correlogram also attempts to group variables that are most closely related; that is, the order of variables along the axes is not random. It is apparent, then, that all four measures of communication are closely related, positioned here in a contiguous block (or cluster) at the top left of the correlogram. The dendrogram in Figure 2 illustrates this clustering more clearly.
Figure 2. Hierarchical clustering dendrogram. Horizontal distance on the tree represents distance of correlation between branches (based on correlation coefficients shown in figure above).
Much like a family tree, the dendrogram illustrates the degree of relatedness between ‘family members’, which is expressed by the horizontal distance between elements. Elements may be grouped at any level in the hierarchy, but the level selected here (indicated by the dashed line) results in three distinct groups of measures. Communication measures fall into one group, and adaptability and resourcefulness measures fall largely within another, with the Resourcefulness Scale scores forming a distinct group of their own. There are some further subtleties – the personal dimension of the university’s Effective Communicators attribute is slightly more distantly related to the other communication measures, for example – but overall it is clear that a high degree of correlation exists between communication measures and that adaptability and resourcefulness, as defined by the university, are closely related. Furthermore, all three measures of adaptability are closely related, suggesting that I-ADAPT-M is a suitable instrument for measuring adaptability in this context. Correlations with the personal and transferable dimensions of the university definition are moderately strong (0.59 and 0.54, respectively). However, the Resourcefulness Scale does not correlate closely with the university definition, and these scores are something of an outlier if each attribute is thought of as a component of a larger notion of ‘graduateness’.
When asked ‘Do you think playing video games might help develop any useful skills or provide any valuable experience?’, nearly half of those surveyed (48%) responded in the affirmative. A large proportion (32%) of respondents weren’t sure if games could be helpful in this regard, while around 20% of those surveyed were certain that games could not develop useful skills or provide valuable experience.
Tables 1 and 2 summarise the categorical and continuous survey data, respectively, and treat contradictory data in the manner outlined above.
Results by gender
Female students (N = 1271) scored slightly higher than male students (N = 857) across three of the four measures, with male students recording a somewhat higher score on the I-ADAPT measure. However, for students who did not identify as male or female, scores were consistently lower than those for other genders. This pronounced difference has resulted in very significant p-values for the difference between genders, but the absolute number of students in this category (N = 17) represents less than 1% of the total cohort. Figure 3 clearly illustrates these relative scores (in all but I-ADAPT-M, there is a clear downward trend in median scores from female, to male, to other) but the insufficient number of data points for the other (non-binary) category is also highlighted by the truncated shape of the violin plots.
Figure 3. Distribution of graduate attribute scores by gender. Scatter plots show individual students plotted as points and arranged to show overall distribution. Box plots show range, interquartile range and median.
Results by level of study
Graduate attribute scores do not vary consistently across levels of study. Only the I-ADAPT-M and SPCCS scores demonstrate the expected pattern, with scores increasing from UG, to PGT, to PGR students. Looking at measures of communication, CAS scores somewhat contradict those for SPCCS, with UG students scoring best, but the difference across all three levels is slight and not significant (p = 0.311). Taught PG students scored best on the Resourcefulness Scale, although the differences here are not highly significant (p = 0.08). Therefore, it may be observed that the most significant differences between levels of study are those that conform to the expected upward trend from UG to PGR.
Results by year of study
As year of study may also be viewed as a continuous variable, at least theoretically, these data are analysed in terms of correlation (see Table 2). These analyses revealed that the apparently negative correlation between the year of study and resourcefulness is so weak (Spearman’s rank correlation coefficient = −0.0004) and statistically insignificant (p = 0.987) that it is meaningless. The positive correlations between the year of study and the other three measures are statistically significant in all cases, suggesting that adaptability and communication skill improve over time at university, but the correlation is extremely weak. However, there is reason to examine the UGs as a distinct cohort because of the variable nature of PG study (see below).
It is notable that there is not a strong correlation between years spent at university and the attainment of graduate skills. While this does not demonstrate that higher education is failing to develop these skills, as implied in the somewhat controversial work, entitled Academically Adrift, by Arum and Roksa (2011), this finding suggests that we should not take such development for granted.
Results by form of multiplayer
This question asked whether respondents played multiplayer games, or if they played only single player games. If the former, they were asked if they played local multiplayer (playing with others in the same room) or online multiplayer (playing with others over the Internet), or both. For two of the four measures (Resourcefulness and the SPCCS communication measure), respondents who do not play video games scored best. On the other two measures (I-ADAPT-M and the CAS measure of communication), those who played both local and online multiplayer scored more highly.
It becomes apparent here that there is no significant relationship between the existing game play habits and graduate attribute scores, supporting the idea that the significantly improved scores recorded for participants in the previous experimental study were the result of a combination of factors. It seems plausible that being adept at both online and face-to-face interaction would be beneficial, and it is perhaps not surprising that scores for adaptability and communicative adaptability were positively correlated with playing both online and local multiplayer. However, given the one-time, cross-sectional nature of the survey, it is not possible to say that there is a causal relationship between multiplayer gaming and these scores. This must also be borne in mind when considering the generally higher scores for non-players, but it is notable that these relatively high scores may be observed for non-players across all three gaming-related items (multiplayer, cooperative play and hours played per week) – in general, non-players score more highly than players.
Results by form of cooperative play
This question asked whether respondents played cooperative games. If so, they were asked if they played cooperative or team-based shooters, of the sort played in the experimental study (e.g. Team Fortress 2); or if they played other games in cooperative mode (e.g. Portal 2); or both. There is no consistent pattern in how these responses relate to attribute scores, but some observations may be made. Non-players continue to score relatively well in these measures; however, the violin plots for adaptability (I-ADAPT-M) and communication (CAS and SPCCS) exhibit a slight U-shape in the distribution of median scores across categories, with non-players on the left and those players who engage in both types of cooperative play on the right. This suggests that those players who play a more diverse set of games may be more adaptable than those who play, for example, only team-based shooters. The relationship is most pronounced in scores for adaptability where both the mean and the median scores are higher for players in this category than for non-players.
Results by hours played per week
While non-players again score well here, the most striking feature of these data is that those who play video games for more than 8 h per week score worst across all measures, except for adaptability, where there is no significant difference in mean scores across categories (p = 0.754). Thus, moderate and non-players score better than ‘excessive’ players do.
Results by player versus non-player
Based on the observation that non-players appeared to score better across most measures, data were collapsed into two categories, player and non-player, to reflect respondents’ game playing status. Summary survey data for these two categories are provided in Table 3. The table data shows that non-players are disproportionately female (83%, against 59% for the overall cohort). It also highlights that players score less well on all measures of graduate attribute: this difference is real, as indicated by most of the associated p-values but is very small when absolute scores are compared to the standard deviation for each measure. For example, there is a difference of 1.88 between players and non-players for the CAS measure of communication, but the standard deviation in absolute scores for players and non-players is 13.49 and 13.85, respectively.
The four main attribute measures are visualised as a set of violin plots, shown in Figure 4. These plots highlight the small but observable difference between the two groups on measures of communication (CAS and SPCCS) and resourcefulness. As suggested by the more granular plots, however, adaptability (I-ADAPT-M) is largely unrelated to game play.
Figure 4. Distribution of graduate attribute scores by player versus non-player. Scatter plots show individuals plotted as points and arranged to show overall distribution. Box plots show range, interquartile range and median.
Player versus non-player results disaggregated by gender
To allow an assessment to be made if the association between game playing status and attribute scores are independent of gender, Figure 5 shows attribute scores disaggregated by both game playing status and gender. This highlights that the largest gender effect is observed for ‘other’ (non-binary) scores and that male and female scores are similar within each strata of game playing status.
Figure 5. Distribution of graduate attribute scores by player versus non-player, disaggregated by gender. Scatter plots show individuals plotted as points and arranged to show overall distribution. Box plots show range, interquartile range and median.
Results by age
It might be expected that the attributes measured here would generally increase with age, as a proxy for experience. The graphs below show that this is not necessarily the case, with Resourcefulness Scale and CAS communication scores remaining flat when plotted against age, while the other communication measure (SPCCS) and I-ADAPT-M show clear increases with age, as depicted by the line of best fit. However, caution must be exercised when interpreting such increases, or the absence thereof, as the median age for the study was 21: there is little data for older students, and small variations at the upper end of the age range may therefore skew the line of best fit.
As continuous variables, it is more appropriate to examine these data in terms of correlation, as shown in Table 2. The Spearman’s rank correlation coefficient for the two most positive correlations, SPCCS and I-ADAPT-M, is 0.12 and 0.16, respectively, with p-values of <0.001. This indicates a highly significant but very small positive correlation between age and these attribute measures. Correlation coefficients for the other two measures are negligible (0.0258 for the Resourcefulness Scale and −0.0222 for CAS), which confirms the above observation that there is no obvious relationship between these measures and age, bearing in mind the limitations imposed by a relative lack of data at the upper end of the age range.
Results by college
The host university comprised of 52 subject areas, arranged into four colleges (Arts, Medical, Veterinary & Life Sciences, Science & Engineering, and Social Sciences). Examining 52 categories of study is too broad an approach to be useful, so subjects were mapped to their respective college for analysis. An ‘Other’ category was created to accommodate the 4.2% of respondents who stated they were not studying any of the subjects offered by the university.
Mean absolute scores for all measures across all four colleges are shown in Table 4. Differences in graduate attribute scores are not pronounced at college level. Despite the logical grouping that a college might imply, there are, in reality, large differences in attribute attainment at subject level. For example, within the College of Arts, which boasts the highest mean CAS score for communication (108.69, SD = 4.52), the Translation Studies subject area has a mean CAS score of 100.2 (SD = 15.09) while French has a mean CAS score of 118.76 (SD = 10.08).
At subject level, some general observations may be made. For example, language-based subjects generally score well on communication. Table 5 shows the 10 highest CAS scores by subject. Not only is the table entirely populated by the colleges of Arts and Medical, Veterinary and Life Sciences, it is dominated by subjects that involve the study of language, with five of the 10 falling into this category.
Undergraduates only
As noted above, the inclusion of PG students in analyses may introduce a degree of undesirable variability, not least because PGs may be at university for as short a period as 1 year or as long as 6 years, and they may come to the institution from a range of backgrounds. Therefore, focusing on the typical 4-year UG degree, over which graduate attributes are said to develop, might provide more useful insight. Table 6 summarises categorical variable data for UGs only. Table 7 shows the relevant continuous variable, year of study.
Drilling down to UG level does not reveal any more striking relationships. The violin plots shown in Figure 6 again indicate that non-players tend to score best on graduate attribute measures. However, a U-shape may be observed on several of the plots for multiplayer, indicating that playing multiplayer games is more positively associated with communication and adaptability than single player games. Cooperative play is only better than non-cooperative play on SPCCS and I-ADAPT-M scores, although cooperative play is associated with higher scores on adaptability than non-play. ‘High’ weekly game play of greater than 8 h per week is associated with the lowest scores on all four measures, while ‘moderate’ play (up to 8 h per week) shows a weaker negative correlation.
Figure 6. Distribution of undergraduate scores on graduate attribute measures by multiplayer play, where ‘non-multiplayer’ represents those respondents who do not play multiplayer games and ‘multiplayer’ combines both local and online multiplayer. Scatter plots show individuals plotted as points and arranged to show overall distribution. Box plots show range, interquartile range and median.
Correlations between the year of study and resourcefulness and the two communication measures are marginally more positive, but adaptability is slightly less so. Only the correlations between UG year of study and the SPCCS and I-ADAPT-M measures might be considered significant (p < 0.05). Furthermore, collapsing multiplayer and cooperative play data into broader categories (e.g. both types of multiplayer are treated as one) has little or no effect on the outcome. This brief analysis of UG data, therefore, does not indicate a strong correlation between video game play and graduate attribute attainment.
That players who do not play video games generally fare better on attribute measures than those who do is important to consider, not least because the experimental study described in Barr (2017) showed that, under certain conditions, playing games can have a positive effect. It is worth remembering, however, that the direction of causality is by no means evident in these data: there is no evidence that students playing video games on their own time depresses attribute scores.
Conclusion
This work has provided insight into the relationship between video game play habits and graduate attribute attainment, as well as the attainment of graduate attributes more widely. While non-players generally score best across measures of graduate attributes, those students who play games in a variety of modes (local and online multiplayer; cooperative and team-based) also tend to score highly and are shown here to score more highly than non-players in terms of adaptability. Players with a narrower repertoire of preferred play modes fare less well, and those who play video games most frequently (more than 8 h per week) score worst. The difference in graduate attribute attainment scores between players and non-players, when considered as two homogeneous groups, is small in terms of significance but is real: non-players generally score better on these graduate attribute measures.
However, a cross-sectional study of this nature cannot show causality, and any causal link between personal game play habits and graduate attribute attainment should not be implied. There is a weak and somewhat complex relationship between game play habits and graduate attribute attainment, but it is not possible to say, for example, that playing more than 8 h of video games per week has a detrimental effect on graduate attributes. Certainly, playing video games under normal conditions does not appear to have a positive effect on graduate attribute attainment. The picture is complicated by indications that the mode of play – and variety of modes in which players elect to engage – may be important.
At a more granular level, the work has revealed some interesting possible avenues for further investigation. For example, self-perceived communication competence and adaptability appear to improve slightly with age, but there is no general improvement in resourcefulness or communicative adaptability in older students. This somewhat counter-intuitive finding bears further investigation: do not all such graduate attributes improve with increased life experience? More intuitive was the finding that attribute scores broadly increased with the level of study, from UG to PGR, but this increase was not consistent. When considered separately, the UG data, which perhaps captures the most consistent picture of progression through university, did not reveal a strong correlation between video game play and graduate attribute attainment. More notable is the generally positive correlation between all the measured attributes, which suggests that there may be some underlying phenomena that underpin all of these graduate attributes, whether this is ‘critical evaluative experience’ (Nicol 2010) or some more nebulous notion of ‘graduateness’ (Coetzee 2014). While exploring such concerns was not central to this work, it is also notable that the correlation between the year of study and attribute attainment is so weak, implying that attending university does not necessarily develop students in the manner we expect. If the measures used here – despite their published validity – are not capable of detecting the gains in attribute attainment that universities claim to encourage, then, at the very least, we must identify measures that do. Otherwise, it is difficult for universities to make any claims about the attainment of graduate attributes in higher education.
Finally, the effects of gender on graduate attribute attainment should also be considered, and further work on the relationship between graduate attribute attainment and subject of study might be undertaken. For example, are language-based subjects really associated with better communication skill? This makes intuitive sense, but, again, it is not possible, based on the data presented here, to determine causation – better communicators may simply excel in the study of languages and thus choose to study one or more languages at university, rather than the experience of studying languages at university enhancing communication skill.
If the results of the previous experimental study challenged assumptions about the role video games might play in higher education, perhaps the data presented here might cause us to consider more carefully our assumptions about the development of graduate attributes in higher education. Certainly, the picture revealed here is far from straightforward.
References
Alonso-Díaz, L., Yuste-Tosina, R., Mendo-Lázaro, S. (2019) ‘Adults video gaming: key competences for a globalised society’, Computers & Education, vol. 141, pp. 103616. doi: 10.1016/j.compedu.2019.103616
Anand, V. (2007) A study of time management: the correlation between video game usage and academic performance markers’, CyberPsychology & Behavior, vol. 10, pp. 552–559. doi: 10.1089/cpb.2007.9991
Arum, R., & Roksa, J. (2011) Academically Adrift: Limited Learning on College Campuses, University of Chicago Press, Chicago, Illinois.
Barrie, S. C. (2006) ‘Understanding what we mean by the generic attributes of graduates’, Higher Education, vol. 51, pp. 215–241. doi: 10.1007/s10734-004-6384-7
Coetzee, M. (2014) ‘Measuring student graduateness: reliability and construct validity of the graduate skills and attributes scale’, Higher Education Research & Development. vol. 23, pp. 887–902. doi: 10.1080/07294360.2014.890572
Coggon, D., Rose, G., Barker, D. J. P. (2013) Epidemiology for the Uninitiated, 4th edn, BMJ Publishing Group, London.
Duran, R. L. (1983) ‘Communicative adaptability: a measure of social communicative competence’, Communication Quarterly, vol. 31, pp. 320–326. doi: 10.1080/01463378309369521
Duran, R. L. (1992) ‘Communicative adaptability: a review of conceptualization and measurement’, Communication Quarterly, vol. 40, pp. 253–268. doi: 10.1080/01463379209369840
Friendly, M. (2002) ‘Corrgrams: exploratory displays for correlation matrices’, The American Statistician, vol. 56, pp. 316–324. doi: 10.1198/000313002533
GameTrack (ISFE/Ipsos Connect). (2016) GameTrack Digest: Quarter 1 2016.
Gbadamosi, G., Evans, C., Richardson, M., & Ridolfo, M. (2015). ‘Employability and students’ part-time work in the UK: Does self-efficacy and career aspiration matter?’, British Educational Research Journal, vol. 41, no. 6, pp. 1086–1107. doi: 10.1002/berj.3174
Granic, I., Lobel, A., Engels, R. C. M. E. (2014) ‘The benefits of playing video games’, American Psychologist, vol. 69, pp. 66–78. doi: 10.1037/a0034857
Green, C. S., & Bavelier, D. (2015) ‘Action video game training for cognitive enhancement’, Current Opinion in Behavioral Sciences, Cognitive Enhancement, vol. 4, pp. 103–108. doi: 10.1016/j.cobeha.2015.04.012
Griffiths, M. (2003) ‘The therapeutic use of videogames in childhood and adolescence’, Clinical Child Psychology and Psychiatry, vol. 8, pp. 547–554. doi: 10.1177/13591045030084012
Hughes, C., & Barrie, S. (2010) ‘Influences on the assessment of graduate attributes in higher education’, Assessment & Evaluation in Higher Education, vol. 35, pp. 325–334. doi: 10.1080/02602930903221485
Jackson, L. A., et al., (2011) ‘A longitudinal study of the effects of Internet use and videogame playing on academic performance and the roles of gender, race and income in these relationships’, Computers in Human Behavior, Current Research Topics in Cognitive Load Theory, vol. 27, pp. 228–239. doi: 10.1016/j.chb.2010.08.001
McCroskey, J. C., & McCroskey, L. L. (1988) ‘Self-report as an approach to measuring communication competence’, Communication Research Reports, vol. 5, pp. 108–113. doi: 10.1080/08824098809359810
Nicol, D. J. (2010) The Foundation for Graduate Attributes: Developing Self-Regulation through Self and Peer-Assessment, The Quality Assurance Agency for Higher Education, QAA Scotland, Glasgow.
O’Leary, S. (2016) ‘Graduates’ experiences of, and attitudes towards, the inclusion of employability-related support in undergraduate degree programmes; trends and variations by subject discipline and gender’, Journal of Education and Work, vol. 30, pp. 84–105. doi: 10.1080/13639080.2015.1122181
Posso, A. (2016). ‘Internet Usage and Educational Outcomes Among 15-Year Old Australian Students’, International Journal of Communication, vol. 10, pp. 26.
Pulakos, E. D., et al., (2000) ‘Adaptability in the workplace: development of a taxonomy of adaptive performance’, Journal of Applied Psychology, vol. 85, pp. 612–624. doi: 10.1037/0021-9010.85.4.612
Pulakos, E. D., et al., (2002) ‘Predicting adaptive performance: further tests of a model of adaptability’, Human Performance, vol. 15, pp. 299–323. doi: 10.1207/S15327043HUP1504_01
Ployhart, R. E., & Bliese, P. D., (2006) ‘Individual Adaptability (I-ADAPT) theory: conceptualizing the antecedents, consequences, and measurement of individual differences in adaptability’, in Understanding Adaptability: A Prerequisite for Effective Performance within Complex Environments, Advances in Human Performance and Cognitive Engineering Research, Emerald Group Publishing Limited, Bingley, England, pp. 3–39.
Romero, M., Usart, M., Ott, M. (2015) ‘Can serious games contribute to developing and sustaining 21st century skills?’, Games and Culture, vol. 10, pp. 148–177. doi: 10.1177/1555412014548919
Shute, V. J., Ventura, M., Ke, F. (2015) ‘The power of play: the effects of Portal 2 and Lumosity on cognitive and noncognitive skills’, Computers & Education, vol. 80, pp. 58–67. doi: 10.1016/j.compedu.2014.08.013
Sumsion, J., & Goodfellow, J. (2004) ‘Identifying generic skills through curriculum mapping: a critical evaluation’, Higher Education Research & Development, vol. 23, pp. 329–346. doi: 10.1080/0729436042000235436
University of Glasgow – About us – Facts and figures – Student numbers [online]. (2015) Available at: http://www.gla.ac.uk/about/facts/studentnumbers/.
The games industry in numbers | Ukie [online]. (2016) Available at: http://ukie.org.uk/research#Who%20plays
University of Glasgow—Services A-Z - Planning and Business Intelligence—QlikView Student Headcount Profiles—Gender—Headcount of Students by Gender 2011-12 to 2015-16.. (2016) Available at: http://www.gla.ac.uk/services/planning/qv/gender/g1/.
Barr, M. (2017). Video games can develop graduate skills in higher education students: A randomised trial. Computers & Education, 113, 86–97. doi: 10.1016/j.compedu.2017.05.016
Barr, M. (2018). Student attitudes to games-based skills development: Learning from video games in higher education. Computers in Human Behavior, 80, 283–294. doi: 10.1016/j.chb.2017.11.030
Zauszniewski, J. A., Lai, C.-Y., & Tithiphontumrong, S. (2006) ‘Development and testing of the resourcefulness scale for older adults’, Journal of Nursing Measurement, vol. 14, pp. 57–68. doi: 10.1891/jnum.14.1.57
Footnotes
1Effective Communicators attribute definition, [URL removed], accessed 7 July 2019
2Adaptability attribute definition, [URL removed], accessed 7 July 2019
3Resourcefulness attribute definition, [URL removed], accessed 7 July 2019