Co-creation of knowledge using mobile technologies and digital media as pedagogical devices in undergraduate STEM education

Research in Learning Technology 2020. © 2020 J. Reyna and P. Meier. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.


Introduction
Students as co-creators of knowledge is an emerging theme in higher education (Gros and López 2016). The concept involves engaging students in their learning process by having them contribute using digital media to the design of learning activities, to the development of marking rubrics and to the creation of content, thus enabling their agency as 21st-century learners (Browne et al. 2017;Gros and López 2016). Research from multiple sources highlights the fact that the issue with learning and technology is not ownership of devices, but fluency in their use (Alexander, Adams, and Cummins 2016;Colbert, Yee, and George 2016;Savin-Baden 2015;Tchoubar, Sexton, and Scarlatos 2018). The concept of digital natives has been discussed previously and is still contentious (Bennett, Maton, and Kervin 2008;Creighton 2018). Simply owning a device does not guarantee the fact that the owner can use the device to its greatest effect. Statistics on technology ownership at Australian universities collected just over a decade ago suggested that most students owned a smartphone and/or a tablet, and most had access to desktop computers and laptops (Oliver and Goerke 2007).
Digital media assignments are considered in the early stages of development (Baepler and Reynolds 2014; Hoban, Nielsen, and Shepherd 2015;Potter and McDougall 2017). There is a lack of published theoretical frameworks that identify how to use and embed digital media skills into the curriculum, which likely reflects the fact that empirical research in this area is still in its formative stages . In the discipline of Teacher Education, digital media has already been used as a reflective tool for pre-service teachers (Kearney 2013;Kearney, Pressick-Kilborn, and Maher 2012;Kearney and Schuck 2005) and digital media has recently been used in science education for inquiry-based learning approaches (Nielsen, Hoban, and Hyland 2017;Nielsen et al. 2018). It appears that the implementation of digital media assignments in educational settings often tends to overlook key aspects such as training students in basic digital media production principles (Reyna et al. 2019a). Neglecting such aspects is problematic because the effective production of digital media requires an understanding of discipline-related principles and approaches that cannot be developed just by using technology to create digital media (Arvidsson and Delfanti 2019;Musburger and Kindem 2012). To effectively learn digital media skills, which are a key component of 'the grammar of the 21st century', students require clear frameworks to understand the principles of media design. These frameworks will allow them to communicate effectively using digital technologies. Moreover, educators need frameworks to help implement digital media learning and design effective marking rubrics as well as support evaluation and research of these technology-based pedagogies (Hoban, Nielsen, and Shepherd 2015).
The literature on digital media assignments in science education is restricted. It appears that many previous studies have not used theoretical models to guide their implementation of digital media assignments . It is relatively uncommon for educators, many of whom implement digital media assignments, to have an adequate understanding of digital media production workflow. One of the consequences of this is that, in many cases, students are not provided with the necessary guidance on how to complete their assignments. This involves the instances like: how to write a storyboard, how to use the software and which digital media principles to consider for effective communication with their audiences. As such, it is perhaps not surprising that the literature on multimedia assignments reports that students are often apprehensive and anxious about digital media assignments and that many students indicate that they lack adequate digital media production skills (Coulson and Frawley 2017;Pearce 2014;Pearce and Vanderlelie 2016). Another issue is that research on the development of marking rubrics appropriate for digital media assignments is in its infancy (Reyna 2019). Arguably, the most obvious limitation is that digital media is only being used as an opportunistic agent or vehicle to learn subject content (Buckingham 2007). This is evident in science education, where the use of digital media assignments has largely been restricted to digital explanations. Digital media assignments can and should be used in more advantageous ways, such as developing student graduate attributes such as time management, group work and collaboration, conflict resolution, creative agency and understanding of diversity. Using the whole potential of digital media assignments could lead to transformative learning 3 (page number not for citation purpose) experiences for students. This project sought to exploit the widespread ownership of digital devices and innovatively repurpose mobile technologies in the classroom as pedagogical devices to learn subject content mediated by digital media creation.

Literature review
Recent literature in science education has highlighted the need for new scientists to use a multimodal approach to communication when promoting science and their discoveries (Nielsen et al. 2018). Scientists will often learn the technical aspects of science, but not necessarily the skills to communicate scientific ideas, concepts and findings to a broad audience. The rationale of a systematic approach to digital media assignments in science education is to: • Help students to learn complex scientific knowledge using a multimodal approach mediated by digital media (Hoban, Nielsen, and Shepherd 2015; Nielsen, Hoban, and Hyland 2017); • Develop problem-solving, critical thinking and research expertise while creating digital media (Pearce and Vanderlelie 2016); • Develop digital media literacies through formal training (Reyna et al. 2019a); • Expose students to collaborative work, giving them the opportunity to exercise their interpersonal skills and solve possible conflicts when creating digital assignments (Coulson and Frawley 2017); and • Assist students to exercise communication, development of cultural sensitivity and diversity .
In order to design, implement and evaluate digital media assignments in the classroom, a set of guiding theoretical frameworks is required. Recent literature on digital media assignments has articulated several frameworks briefly discussed further.

The Digital Media Literacies Framework
The Digital Media Literacies Framework (DMLF) (Reyna, Hanham, and Meier 2018a) is designed to identify and address student training needs in digital media production in the classroom. This framework helps to define the elements of digital media literacies required to produce digital media assessments effectively. In the framework, it is proposed that the effective production of digital artefacts has three domains: conceptual, functional and audio-visual. The conceptual domain relates to the creation of a storyboard and researching evidence-based information. The functional domain is related to the skills required for the students to use digital media design software and applications. Finally, the audio-visual domain informs the principles to create effective digital media. Students require an understanding of the three domains to communicate effectively using digital artefacts.

The taxonomy of digital media types
The Taxonomy of Digital Media Types (Reyna, Hanham, and Meier 2017) incorporates and further extends the DMLF. The model identifies different media types including audio podcasts, digital stories, animation or video. The framework covers the skills required to produce a digital media artefact. The theoretical underpinning of this framework uses the notion of 'technological proxies' (TP), which posits that certain technologies can be conceptualised as proxy agents for learners to carry out tasks on their behalf (Hanham et al. 2014). For instance, when students use their mobile phones to capture videos to explain a natural process, the phones act as pedagogical devices. By recording the process in video format, the device takes on the learning task on behalf of the user. Students can then watch the video as many times as needed and reflect on how the process occurs. They can watch and edit the footage and reflect on the theory behind it, which they previously learnt through the creation of a storyboard.

The digital media principles framework
This framework is designed to help students to apply digital media principles in their assignments (Reyna, Hanham, and Meier 2018b). For educators, it helps to create training materials and marking rubrics. These digital media principles include layout and how design elements are distributed on-screen, the colour theory and how it engages users with online content, typography and fonts for maximum legibility, the C.R.A.P principles of graphic design (contrast, repetition, alignment and proximity) for effective communication and image and video principles. The application of these principles ensures the creation of effective digital media artefacts. The framework guides student training in digital media production in the classroom.

The digital media implementation framework
The three frameworks described above complement the Digital Media Implementation Framework , especially regarding student training and marking schemes. This student-centred practical framework was designed to assist in the implementation of digital media as an assessment tool in the classroom. The framework focuses on what students do with the digital media task and helps them to understand the assessment workflow and the rationale behind learning with digital media. It includes frequently asked questions that are discussed and clarified in the tutorials and inside the Learning Management System (LMS) with custom-developed online resources.
The study aimed to explore student perceptions of the co-creation of knowledge using mobile devices and digital media assignments following the described frameworks. The research question was: what are students' attitudes towards digital media assignments for learning in STEM disciplines?

Materials and methods
The research project used a mixed-methods approach to data gathering (Tashakkori and Teddlie 2010) and included two validated questionnaires and open-ended questions to gauge student perspectives on digital media assignments for learning in STEM disciplines.

Participants
Students participating in the study were enrolled in the Faculty of Science between 2015 and 2018 in different programs, including Science (Neuroscience), Technology (Programming), Engineering (Medical imaging technology) and Maths (Calculus).

(page number not for citation purpose)
All the students needed to produce a digital media assignment (5-min video), and the assessment task weight was between 20% and 30% of total subject marks. The cohorts comprised first, second and third-year students (Table 1). Participation in the study was voluntary. Two previously validated questionnaires were used; the first gauging student attitudes to digital media assignments (2015-2018, N = 1751) and the second capturing student attitudes to digital media for learning and career (2018, N = 524). This questionnaire was developed and validated at the end of 2017 and used the following year. Four subjects were dropped between 2015 and 2018, and new subjects joined the digital media assessment task because subject coordinators rotated across the programs each year and brought different approaches to assessment tasks.

Digital media task learning design
The digital media assessment design followed the frameworks presented in the literature review. The first step was to plan the training topics for the students. The digital media literacies framework was used to design lecture materials and online content using the three domains: conceptual (storyboard), functional (use of software and applications) and audio-visual (the digital media principles). Subject coordinators used the taxonomy of digital media types to choose the digital media for the assessment task (digital story, animation, video or blended media) and to develop the marking rubric. The digital media principles framework informed which topics were covered in the audio-visual domain training (layout design, colour theory, typography, use of images, video principles). Finally, the digital media implementation framework was used to guide the assignments' deployment in the classroom and to communicate to students the rationale for learning science using digital media.
All the cohorts participating in the digital media assessment task had the same type of assessment design: (1) production of a 5-min video to communicate a scientific concept; (2) working in groups of 3-5; (3) receiving training in digital media production; (4) marking rubrics were similar, with criteria covering quality of content and application of digital media principles and (5) assessment weightings were between 20% and 30% of the total subject mark.

Digital media training
Students were trained face-to-face, in blended mode, and in some cases, fully online. The different modes of delivery were based on time-slot availability in some subjects. Students in all cohorts received tuition on digital media principles for online communication at the beginning of the session (weeks 1-3). In this lecture, the rationale for learning science via mobile devices and digital media was communicated and discussed. The lecture covered the digital media principles for the effective use of media to communicate science. The practical session included a workshop on how to create a storyboard to produce digital media and was delivered across all cohorts between 2015 and 2018. The learning designer developed online resources on these topics, and also curated content (YouTube videos) with different applications to produce a video using mobile technologies across platforms (Windows, Macintosh, iOS and Android).

The questionnaires
The study used two previously validated questionnaires, one on student attitudes towards digital media assignments (2015-2018) (Reyna and Meier 2018c), and one on digital media for learning and career presented in Table 2 (2018). Both questionnaires used a Likert scale from 1 to 4, which corresponded to strongly disagree, disagree, agree and strongly agree. At the end of the questionnaires, open-ended questions were asked (issues with the assignment, like most, least and feedback to improve the assignment). Demographic data were gathered, including gender, age, education and English as an additional language at the end of the questionnaire.

Data analysis and interpretation
Questionnaire data were analysed using SPSS Statistics for Windows (Version 24.0, Armonk, NY, USA: IBM Corp. ). For open-ended questions, NVivo (Version 11, QSR International, Melbourne, Australia, 2016) software was utilised. Questionnaire data were analysed per cohort and compared across years (2015-2018). They were able to be combined because they showed no significant statistical differences. The data were presented depending on the theme rather than for individual items due to the high reliability of the questionnaire items. Responses to open-ended questions were analysed using thematic analysis, with 10 themes emerging. Table 3 presents the demographic characteristics of the entire cohort between 2015 and 2018. Most students were under 30 years of age and females. A high percentage of the students were from a non-English speaking background (29%-42%).

Student attitudes to digital media assignments
Student perceptions of digital media assignments in STEM subjects have been overwhelmingly positive since 2015. The validated evaluation questionnaires showed positive student outcomes for digital media support, attitudes towards technology, understanding of the assignment, knowledge construction (Table 4, Figure 1) and digital media for learning and careers (Table 5, Figures 2 and 3).
Although the cohorts were different in size, there were no significant statistical differences between the years, and so all the data were analysed together. Figure 1 presents student attitudes towards digital media assignments for the four constructs measured.
Student perceptions of using digital media to learn scientific concepts and its importance in their future careers were highly positive. These data were captured only for 2018. Table 5 presents descriptive statistics, while Figures 2 and 3 give a visual representation of the results.

Open-ended questions
Qualitative data gathered since 2015 support the data presented in Figures 1-3. The total of open-ended responses received during the study was 456. Only 14 responses were negative (no learning, waste of time and group issues), representing 3% of the responses received. The students described the transformative nature of their learning when producing digital media assignments across different STEM subjects. Thematic analysis of open-ended questions revealed a positive student attitude towards learning using digital media, groupwork enjoyment, learning engagement, development of critical-thinking skills, creativity and collaboration, improvement of time management and the development of independent learning skills. Additionally, engaging with the needs of society, getting motivated and development of cultural awareness were themes that emerged from the analysis.  Figure 1. STEM students' attitudes towards digital media assignments. Overall evaluation data from 2015 to 2018 were collected by using a validated questionnaire. The total sample includes first-, second-and third-year subjects (n = 1751). Results presented in percentage ranged from strongly disagree, disagree, agree to strongly agree.  Figure 2. STEM student perceptions of digital media for learning science. N = 524, Years 1-3, STEM students, 2018. Results are presented in percentage ranging from strongly disagree, disagree, agree to strongly agree.

9
(page number not for citation purpose)

Discussion
Results from datasets collected from STEM students (2015-2018), measuring their attitudes towards digital media assignments, were extremely positive. Descriptive statistics of overall results by year show a high mean of 3 on the four-point scale (Table 4). When the data across the 4 years were combined and analysed based on category (knowledge construction, understanding of the assignment, attitude towards technology and digital media support), 70% or more of the students agreed or strongly agreed with the statements, which confirmed a positive attitude to these constructs ( Figure 1). In 2018, with the second questionnaire added, student perceptions of digital media for learning and for their future careers were also highly positive, again with a mean of 3 on the four-point scale (Table 5, Figures 2 and 3). Responses to open-ended questions reinforced the results from the questionnaires. These results agree with what researchers have previously reported in a small-scale study (Reyna et al. 2016). In the current study, only 3% of responses to open-ended questions were negative, and they were related to students' perceptions of task value and group work issues. Previous research reported student anxiety and apprehension about digital media assignments (Anderson 2013;Coulson and Frawley 2017;Pearce and Vanderlelie 2016). Unlike the current study, those studies did not implement a rigorous approach that included scaffolding for the digital media task, which could explain the different results of our study. The quantitative and qualitative data from the current study showed that students had positive attitudes to digital media assignments and digital media for learning and their future careers, providing an answer to the research question: What is the student attitude towards digital media assignments for learning in STEM disciplines? Gender data for the 4 years of the study showed that 59% of students were female. The literature on gender and attitudes towards technology reports inconsistent findings. Some studies have found significant gender differences, with females showing more positive attitudes to technology compared to males (Chen and Tsai 2007;Johnson 2011;Price 2006). Some studies have reported the opposite (Chou, Wu, and Chen 2011;Hasan 2010), while other trials found no significant differences (Bain and Rice 2006;Imhof, Vollmeyer, and Beierlein 2007;North and Noyes 2002). A recent meta-analysis of studies of gender and attitudes to technology use reported that males held more favourable attitudes than females, attributing the difference to a small effect size (Cai, Fan, and Du 2017). However, attitude is a multidimensional concept that can be influenced not only by gender, but also by age, culture, socioeconomic status and other variables (Osborne, Simon, and Collins 2003). It is unlikely that merely having 9% more females in the study could affect the results presented. In terms of age, 83% of students were between 19 and 29 years old and had likely been exposed to digital media assignments during high school. They may, therefore, have been more confident with the digital media assessment task. Student interviews could help to explore this hypothesis in future studies. Digital media production skills are considered 'the grammar of the digital age', and they are the foundation of multimodal online communication (Arvidsson and Delfanti 2019;Reyna, Hanham, and Meier 2018a;Shen et al. 2018). New graduates should have the skills to tackle with the needs of society by developing innovations in STEM and communicating that progress to a diverse audience. Understanding how to effectively repurpose existing mobile technologies as pedagogical devices and create engaging digital media is fundamental to that purpose. Our digital media assignment intervention was innovative for its systematic approach to the design, implementation, evaluation and large-scale analysis of digital media assignments. It represents the first time in the published literature that students have received a comprehensive scaffolding on how to produce digital media, including conceptual skills (storyboarding), functional skills (using software and applications) and audio-visual skills (digital media principles).
Data gathered for this study presented a positive student perception of digital media assignments, but there were also beneficial outcomes reported by educators. Although the focus of this study was students, we also found that educators successfully integrated technological tools into existing curricula and teaching contexts. For example, subject coordinators used Turnitin to provide students with feedback on their writing skills (storyboards). Digital media assignments are now 'business as usual' in the Faculty of Science and are sustained by subject coordinators.
The currently available research on digital media assignments does not have a rigorous approach informed by theoretical frameworks. The first contribution of this research lies in the proposed workflow and theoretical frameworks for digital media assignments. The second contribution is the use of mobile technologies, not only to learn subject content but also to learn the digital media principles for effective communication in the digital space, as well as developing student ability to collaborate and work in groups. The third contribution of this study is the use of the validated questionnaire tools in a longitudinal study. For instance, the questionnaire on attitudes to digital media can be used at the end of the session to gauge student feedback on the digital media task. The data can then be used to improve the assessment task in the next iteration. Also, the questionnaire on digital media for learning and career can be used at the beginning of the session to assess student understanding of learning with digital media. The responses can help to design a strategy to engage students in co-creation of knowledge and get them to see their mobile gadgets as pedagogical devices to learn subject content mediated by digital media production. Lastly, our thematic analysis revealed that digital media assignments could be used to improve student time-management skills, group work and conflict resolution, engagement with their learning, development of critical thinking skills and understanding of cultural diversity. The use of digital media assignments solely to learn subject content and explain 11 (page number not for citation purpose) the science behind it has not fully exploited the potential of these digital technologies to engage students in a transformative learning experience. Educators can align the skillsets revealed in our thematic analysis with the digital media assessment task, as they all represent desirable graduate attributes for 21st-century scientists.

Limitations of the study
Limitations of the study include a lack of understanding of how students organise group work and their different roles, and how they adjust to this new way of assessing knowledge. A second limitation of this research is that it is unknown which digital media workflow students used. Whether using their mobile phones, tablets, laptops or a combination of these, the digital media workflow that students use is essential data to gather in the next iteration. Improvement of student training and support can be further customised by understanding the tools and applications students use to create their digital media assignments. A third limitation was that the researchers did not follow-up with the students, so it is unknown if first-year and second-year cohorts engaged in producing digital media in the following years. It is likely to be the case that some students bring more skills to creating digital media as an assessment task. A fourth limitation is the lack of data on group contribution and marks attained for the digital media task. Informal conversations with subject coordinators gauged that group issues affected between 6% and 10% of participants, which is low in comparison to other group assignments observed at our institution (15%-25%). The last limitation is that the study was student-focused and there is a lack of data from educators representing their perspectives, even though the fact that educators have continued using digital media assignments since 2015 suggests that they value them. It is recommended that in-depth interviews be conducted to further investigate educators' experiences with digital media assignments. A second recommendation is to use these questionnaires across several institutions and different disciplines, with or without the systematic approach using theoretical models described in this study, to compare student attitudes to digital media assignments.

Conclusion
Longitudinal data gathered for this research showed that STEM students had a positive perception of the co-creation of knowledge using mobile devices and digital media and following the systematic approach presented here. Students enjoyed learning subject content using digital media, learning digital media principles and digital media production and having the opportunity to exercise their creativity in learning how to repurpose their mobiles, tablets, laptops or cameras as pedagogical devices. There have also been the additional benefits of students gaining exposure to group work and collaboration, learning to deal with conflict resolution and developing a better understanding of diversity. These positive results could be explained by the systematic approach and theoretical frameworks to develop the assessment task, the support provided to students and the communication of the importance of learning digital media as future STEM professionals. To date, this is one of the first longitudinal large sample-size studies of student attitudes to digital media assignments that use a systematic approach as well as validated questionnaire tools.