Analysing the impact of e-learning technology on students’ engagement, attendance and performance

In higher education


Introduction
This article presents the findings of an initial study that was undertaken to examine how the use of Blackboard could identify the extent to which students' utilised online materials on their engineering courses.The study was undertaken to explore if regular links to the online resources was related to their attendance and supported their engagement with their studies.
The findings contribute to the literature on how the development of web-based technologies can enhance students' performance in higher education.

Literature review
The Development of Web-based technologies.
The ease of access to online materials has also become more common due to advancement in information technology via e-devices such as Desktops, Laptops and Smartphones.Although new e-learning resources are appearing all the time in education, one of the most prolific VLEs, is Blackboard (Bb).Bb is a system that allows users to access it via a unique username and password to 'log-in' to their subject modules/programme.It started as "one of leading commercial learning management systems and then shifted to wide use as a course management system software package in educational institutions (Guo, Zhang and Guo 2016;Zidan 2015, p.230).
Electronic-learning through the World Wide Web, or e-learning via the internet, as it is now more commonly known, has become possible because of the advancements in communication, networking and broadcast technologies.The use of electronic materials is heavily researched from a number of different perspectives (Flavin and Quintero, 2018).For example, research by Hewitt and Stubbs, (2017) examined how learning technology could help address law students' anxiety about their studies and improve their self-efficacy.A study by Young and Nichols (2017) examined how academics embedded digital learning approaches into the curriculum.Throughout this extensive research, the debates surrounding the use of the internet and related advanced technologies have acquired a number of different terms, which are frequently used interchangeably in the literature.For example, blended learning, distance education/learning; online environment learning; web-based instruction and more recently Virtual Learning Environments (VLEs) (Young and Nicols, 2017) VLEs such as Blackboard (Bb), Canvas and WebChat (Web CT) can be available twenty-four hours per day, all year around.
Universities may have many national and international students studying on their programmes, so within this heavily competitive marketisation and internationalisation of higher education, they have to ensure they stay up-to-date with the latest e-learning technologies to improve communications, and student engagement and performance.This technology also helps to improve student engagement in terms of the time spent on a task, quality of effort and student involvement.The challenges and benefits of e-learning have been discussed in many articles (see inter-alia Altuna and Lareki 2015;Raab, Ellis and Abdon 2002;Bouhnik and Marcus 2006;Liaw et al, 2007), but a common thread throughout the research is the importance of e-learning technologies as a support mechanism for helping students to engage in their studies.Starting with a discussion of what is meant by engagement, the following section highlights some of the issues surrounding the use of elearning technologies.

What is engagement?
The generic term engagement employed throughout the literature on higher education, depicts students' study patterns, how they use their time, resources, relationships and communications with their tutors, peers and the organisation (Kahn 2014;Trowler 2010).
Theories of how best to do this, however, vary across and within disciplines.From, the behavioural perspective, it is defined as the 'time and effort students devote to educationally purposeful activities' (ACER 2010b), but from the psychological perspective, cognition incorporates individual characteristics such as motivation, self-efficacy and expectations as part of student engagement (Jimerson, Campos and Greif 2003).Researchers in the UK have proposed a more holistic definition: 'The conception of engagement encompasses the perceptions, expectations and experience of being a student and the construction of being a student' (Bryson, Hardy and Hand 2009).Whichever definition is postulated, research into improving students' engagement in their studies embraces all the quality enhancement and quality assurance processes, ensuing in the improvement of the educational experience (The UK Quality Code for the Higher Education, 2012).Some studies have examined the students' feelings and emotions surrounding the process of engagement.According to Harper and Quaye (2009a), student engagement is more than just involvement or participation.It requires a positive frame of mind and 'mood' and 'sense making' in addition to the physical active involvement in different types of activities within the academic environment.Acting without sentiment, engagement is just like participation; or feeling engaged without acting is known as dissociation.Fredricks, Blumenfeld and Paris (2004)  • Emotional engagement: Students who engage emotionally would experience affective reactions such as interest, enjoyment, or a sense of belonging.
• Cognitive engagement: Cognitively engaged students would be invested in their learning, would seek to go beyond the requirements, and would relish challenge.
A study by Stewart et al (2011) about the relationship between student engagement in terms of attendance, online learning and performance was inconclusive, but their findings did demonstrate the importance of attendance as a predictor of performance and argues it is influenced by the study behaviour rather than time spent on accessing the resources (Bb clicks/hits), particularly online resources.They also suggested that an integrated blended learning approach could help to improve the student performance.

How does technology enhance engagement?
While research into helping students engage with their studies has shown the importance of good communications; starting with clear guidance to students about what it is they will study, assessment and feedback (HEA 2017;Kahu 2013, Thomas 2012), the complexity of this process is articulated in research findings across both the general and specialist literature on higher education (Zepke 2014).The emerging research into how students' think and feel about their studies has also added to the intricacies of the debates whilst contributing to how different resources might be used in various ways to positively enhance the students' experience and performance (Hewitt and Stubbs, 2017).The stronger the engagement, the better the student is seen to perform (Trowler 2010).
The student profile in higher education has changed considerably over the past two decades, not only with the internationalisation of the curriculum (HEA, 2017) but with the attendance patterns of students.While the traditional, full-time student remains, many students now work part-time, or combine distance learning with course attendance.This change in study patterns has necessitated the use of web-based technologies.

The research study and limitations
The study concentrated on exploring the relationship that might exist between student engagement, attendance and performance.The study was limited to one undergraduate course module in a civil engineering programme, over two levels: level 4 and level 6.Whereby, as part of their programme studies, students are normally required to search the learning and teaching materials for the coursework assignment and exam purposes.It is not possible therefore, to argue that Bb clicks/hit rates have any impact on students' engagement and progression with their learning, but it is possible to see how the clicks/hits linked with attendance and final performance, and this is useful to the module tutors to help them design the more effective online materials.While the insights from the study are limited to the exploration of the interaction with Bb on two engineering modules, and without further examination across other subject areas, no claim to generalisability of the findings can be made; nonetheless, the approach to the data collection and the findings may help to assist tutors and programme managers when designing module guidelines and structuring course materials.

The research methodology
A quantitative approach to data collection was employed.According to Aliaga and Gunderson (2000), the quantitative method is defined as 'explaining phenomena by collecting and analysing the numerical data through mathematically based methods in particular statistics.' Quantitative methods are frequently described as deductive in nature, in the sense that inferences from tests of statistical hypotheses that lead to general inferences about characteristics of a population (Bryman, 2015).

Hypothesis
In this study, "it is hypothesised that student engagement via Bb 'hits' rates has a significant relationship or correlation with class attendance, engagement and performance" The study's aim was to explore any connection between the students' Blackboard clicks/hits, their attendance on their programme of study, engagement and performance.A statistical analysis test for the correlation between students' online activities via Bb clicks/hits and class attendance were performed to understand the depth of the relationship between student engagement and their impact on student performance.This relationship will help inform further research into how best to enhance teaching and learning practices through the redesign of the module structure, inform guidelines and understand the way students utilise online learning resources via the Bb system.
To minimise the impact of subject-type and student-cohort, two different course modules were included in the study with two levels of student performance in the civil engineering programmes.The study was based on secondary data analysis, which was gathered from the university Bb system and attendance records to reflect student use of online resources and physical participations in the class rooms.
The design of the study contained two aspects.The first aspect of the study aimed to examine the correlation between student engagement via online activities measured through the Bb 'hits' rates and student module performance.The online activities/hits were recorded under the course evaluation tool in the Bb system, based on the use of electronic resources, over the whole academic year in a course module "construction practice" at level 4 and the "risk management" module at level 6 in the civil engineering programme.The aim of the second aspect of the study was to identify any correlation that could exist between class attendance and module performance of the student at the same modules at level 4 and level 6 respectively.

Data collection for research
The total number of students included in the study, were 82 and 88 at level 4 and 6 respectively.The details of Bb clicks/hits, records of class attendance and the final grade of each student on the module, at both levels, are shown in Appendix-A.The secondary data for statistical analysis in the study was collected under three aspects/attributes of student engagement as detailed below: 1. Performance.For each module, performance data in terms of the final grade of each student were collected at both levels (last Column 'D' of each module, appendix-A).

Student engagement and performance Level 4
After analysing the sampling data, the results are presented in the tables and graphs Similarly, Figure 2 shows that there is slightly different frequency between attendance and performance, but the line graph does not identify any type of existing correlations between them.Hence, a T-test was then conducted to identify the positive or negative correlation between student performance and engagement at both levels.The results of the paired sample statistics, that is the paired sample correlation and paired sample test are presented in Tables 2, 3 and 4 respectively below.
The T-test results of a paired sample correlation analysis reveals that there is significant positive correlation between the Bb hits and the final grade (0.516, P=0.00 <0.05), and between attendance and final grade (0.590, P=0.00 <0.05) (see Table 3).However, when the paired sample test was conducted at a 95% confidence level, it was found that student engagement in terms of Bb 'hits' has highly significance on performance with positive tvalue (t=9.99,P=0.00<0.05).Whereas, the pair sample test between students' attendance and final grade reveals an insignificant result with negative t values (t = -1.32,p=0.19>0.05).The details of pair test results are shown in Table 4.Moreover, the results confirm that student performance has positive correlation with student engagement in terms of Bb 'hits/click' compared to class attendance, as an initial findings from the study.An additional regression analysis using SPSS was conducted to understand the importance and effect on student performance from student engagement aspects.A regression analysis with automatic linear modelling was then conducted to analyse the linear effect on student performance (final grade) from the aspect of student engagement indicators such as (Bb clicks/hits and attendance).The results of the regression analysis are shown in Figure 3 below.The student performance on the module at Level 4 (mean = 55.74,SD= 20.566 and N= 77) shows the linear effect with respect to Bb hits and attendance (see Figure 3).The linear modelling result reveals that online activities related to exam preparation has the most important consequence compare to the online activities associated with coursework.Figure 4 reveals that the estimated mean has significant effect on the final grade (student performance) from the engagement aspects of (Bb clicks/hits and attendance).
This supports the argument that there exists a positive linear relationship between student engagement and performance.The linear relation of student performance with respect to Bb hits has highly sensible than class attendance at level 4 module of the programme.

Student engagement and performance at Level 6
After analysing the research data, study results are presented in tables and graphs below.
At first, student engagement aspects in term of Bb 'clicks/hits', attendance and student performance aspect in terms of final grade at Level 6 was analysed using SPSS.Two frequency graphs with student ID and mean values of Bb hits, attendance and final grade were drawn and presented them in Figures 5 and 6 respectively.Figure 5 reveals that there is a similar trend of fluctuation between student engagement aspects and their performance, but the line graph is unable to identify the types of correlation that exists between them.Similarly, Figure 6 shows a slightly different frequency between attendance and performance and the line graph does not show the correlations between them.Therefore, a T-test was conducted to identify the correlation between student performance and engagement at both levels.Statistical analysis of T-test with paired sample correlation are performed and results are presented in Tables 6, 7 and 8 respectively.T-test results of the paired sample show that there is significant correlation exists between the student engagement aspect of Bb hits and the final grade (0.244, P=0.022 <0.05) but insignificant correlation exist between student attendance and the final grade (0.056, p= 0.00 <0.605) (see Table 7).On other hand, when a paired sample T-test was conducted at 95% confidence level, it was found that student engagement and performance is highly significant with positive t-value (t= 16.93, P = 0.00<0.05),whereas pair test between student attendance and the final grade showed significance results but negative t-value (t=-4.157,p=0.00<0.05)(see Table 8).
The above results confirm that student performance has some relationship with the Bb 'hits' compared to the student attendance.A regression analysis was also conducted with automatic linear modelling using SPSS to understand the importance and consequence on student performance from engagement aspects.The results are shown in Figures 7 and 8 below.Moreover, the results and discussions about the regression analysis, which was conducted with linear modelling with the aim of identifying type of relationship between student performance and engagement aspects.

Discussion of the findings
From the statistical analysis of the research data, the findings are significant at both levels 4 and 6.Firstly, it was recognised that student performance has a positive correlation with student engagement from the aspect of Bb clicks/hits' at both levels 4 and 6, but the types and levels of correlation are different at both levels.One of the results showed that the class attendance at level 4 is significant on student performance but it is insignificant at level 6.
Secondly, the other key finding found was that student engagement from Bb hits' aspects have a significant and positive connection in improving the student performance at both levels, but it was insignificant on performance from the attendance aspect.The study results also confirmed that student engagement has a linear effect on the student performance from the regression analysis.This exposed the issue that students need to be involved more in online activities in order to improve their performance in a course module.From the above results, it could be argued that the results might be different in other subject areas due to the nature and complexity of different modules, where different levels of online activities take place.For example, lab-based or field-based module must need active participation compare to class-based modules, however online activities can help to improve student understanding and performance.

Conclusion
A review of the literature illustrates the range and complexity of advancements in web-based technologies and reveals the equally diverse ways that students utilise the e-resources available to them (Wang, 2015).In this study, the findings showed that student performance has a positive and significant correlation with student engagement at both levels 4 and 6 in the civil engineering programme, however, both types and level of correlation were found to be diverse at both levels.While class attendance was significant with student performance at level 4, it was shown to be insignificant at level 6; however, from the regression analysis test, the results also confirm that student engagement shows a linear relationship.This suggests that students' involvement in online activities could help to improve their performance on a module.Of course, when various levels of online activities take place in the programmes of study, it can be argued that the results might be variable in other modules due to the nature and complexity of different subject areas, Since Marton and Säljö (1976) first introduced the concept that students take different approaches to how they learn the subject, the extensive and rich literature on all aspects of the student learning experience has contributed to the knowledge of the intricacy of students' relationship with their own learning.Across and within different subject domains, students employ a range of deep, surface and strategic approaches to their studies (see inter-alia, Gibbs 1992; Bryson and Hand 2007;Fielding 2006;Holmes 2015).Emerging research on the use of digital technologies now explores the intersection between the convergence of learning theories and digital technologies (Altuna and Larek, 2015) and implementing blended learning frameworks could be one of the ways forward in research into the advantages and challenges of e-learning (Adekola, Dale and Gardiner, 2017).The advancements in technology-enhanced learning and teaching (TELT) over the past decade adds another dimension to this complex relationship, so how best to utilise the electronic material to encourage students' engagement with their studies remains an ongoing area for further research.

2.
Attendance: Class attendance was used as an indicator of levels of student engagement with teaching and tutorials.Both modules comprised a mix of class-based and lab/fieldbased teaching (third column 'C', appendix-A).3.Bb (Hits): Access to the online learning resources was collected using the course evaluation-reporting tool via Bb.The magnitude of intended usage of e-resources held on the Bb system was considered as indicative levels of student's online engagement.Both the modules had a distinctive design structure holding a wide range of e-learning resources, course administration, information, announcement, discussion blogs and assessment tools on Bb.These comprise of folders containing lecture-supporting resource items, mostly PowerPoint slides, lectures notes, worksheets from practical and tutorial classes, links to other e-resources and other reading online materials links to the relevant websites.The course reporting tool logged as a click/hit each time a folder, page or item (uploaded e-resources or website URLs) was accessed by a student within these areas.It was assumed that the total volume of 'log-ins' is largely used for productive purposes in their study rather than getting information of hits' rates, which is determined by the site design structure.The numbers of clicks/hits of each student recorded by 'Bb' are shown in appendix-A (Column A show student ID and Column B show Bb 'hits').
below.At first, student engagement in terms of the Bb clicks/hits and performance in relation to the final grade of students at Level 4 in a course module was analysed using SPSS.Two frequency graphs of student ID and mean values of Bb hits, attendance and final grade were drawn and presented in Figures 1 and 2 respectively.Figure 1 reveals that there is similar trend of fluctuation between student engagement and student performance but fails to identify what types of correlation exists between them.

Fig. 1 :
Fig. 1: Line graphs of total hits and final grade of a module at level 4.

Fig. 2 :
Fig.2: Line graphs of total hits, final grade & attendance of a module at level 4

Fig 3 :Fig. 4 :
Fig 3: Effect on the final grade from exam and coursework at level 4

Fig. 5 :
Fig. 5: Line graphs of total hits and final grade of a module at level 6

Fig 7 :
Fig 7: Effect on the final grade from exam and coursework at level 6

Fig 8 :
Fig 8: Estimated means chart of student performance and engagement aspects Bb'hits' and attendance at level 6

Table 1 :
classified student engagement into three dimensions (see Table 1 below).Examples of positive, negative engagement and non-engagement

Table 2 :
Paired Samples Statistics at level 4

Table 4 :
Paired Samples Test at level 4

Table 6 :
Paired Samples Statistics at level 6

Table 7 :
Paired Samples Correlations at level 6

Table 8 :
Paired Samples Test at level 6