Virtual learning environments – help or hindrance for the ‘ disengaged ’ student ?

The introduction of virtual learning environments (VLEs) has been regarded by some as a panacea for many of the problems in today’s mass numbers modular higher education system. This paper demonstrates that VLEs can help or hinder student engagement and performance, and that they should be adapted to the different types of learner. A project is described that aimed to investigate whether the introduction of a VLE can assist ‘disengaged’ students, drawing on click count tracking data and student performance. The project took place in the context of two very large undergraduate modules (850 and 567 students) in a Business School of a new university in the UK. In an adaptation of a model of learner engagement in Web-enhanced environments, four distinct learner types have emerged: model, traditionalist, geek and disengaged. There was evidence that use of the VLE exacerbated, rather than moderated, the differences between these learner types.


Introduction
The business world is not a global village but a global technological jungle, where survival depends on the ability to use IT in new and innovative ways.(Brady, Saren, and Tzokas 1999, 764) In a mass numbers modular higher education system, students may learn rapidly that their attendance is not closely monitored nor their absence penalised.Such students risk becoming 'disengaged', failing to participate in academic life.This is a common characteristic among the 22% of students in England who fail to complete their higher education course at their original institution (House of Commons Committee of Public Accounts 2007).
The introduction of a virtual learning environment (VLE) changes the relationship that a student has with their course of study.The Joint Information Systems Committee (JISC) (2007) defines a VLE as a set of components in which learners and tutors participate in 'online' interactions of various kinds, including online learning.VLE activity data, sometimes called click count tracking, despite its imperfections, provides an easily available, yet rudimentary, measure of student 'learning' activity.Although such systems may support flexible study, there is a concern that they may also foster a belief among those students already distanced by the system that learning can be delayed to some vague future date and diminishes further the perceived need to attend face-to-face sessions.
This project seeks to establish what relationships, if any, exist between undergraduates' use of a VLE and academic achievement on two very large modules (Level 1 Organisation Studies, 850 students; and Level 2 Marketing, 567 students).The broader purpose of this is to see whether such measures can be used to identify disengaged students (Lee 2001) so that early interventions can be made that will build a stronger sense of self-efficacy (Bandura 1986) in an at-risk group.

The influences and effects of e-learning
Effective use of new technology requires an evaluation of current pedagogy and a move towards interactive and collaborative teaching and learning activities (Mason 2002;Salmon 2006;Stiles 2000).Failure to do so, according to Stiles (2000), could result in VLEs compounding the mistakes of the past, leaving the learner with a passive, unengaging experience leading to surface learning.Entwistle (2001) explains that a deep approach describes active engagement with the content, leading to extensive elaboration of the learning material, which contrasts with the surface approach of using routine memorisation to reproduce those aspects of the subject matter expected to be assessed.
VLEs are NOT neutral.Like any technology they embed underlying values about teaching and learning, promote certain affordances and reduce other choices.(Salmon 2002, 8) The concept of affordance, according to Salmon (2002, 2) means the properties of a system which allow certain actions to be performed and which encourage specific types of behaviour.
The affordance of publishing inherent within the commercial VLE used encourages staff to use it to provide easy access to lecture slides and other teaching materials.However, Steinbronn and Merideth (2003) comment that copying and pasting lecturer notes into a Web page may reproduce the material, but not the interactive communication and engagement that can be promoted in a live lecture, with non-verbal expression and body language.Furthermore, over-dependence on content-driven models results in the same 'boring' material that was provided in lectures becoming boring Web pages (Cronje 2001).
While many academics focus on learning in relation to the use of VLEs, Selwyn (2000) argues that researchers need to be aware of the social, cultural, political and economic aspects of educational computing; the soft as well as the hard concerns.Alexander (2006) develops this point, arguing that today's diverse student body means that it is important to take cultural factors in account.The implication of this is that opportunities should be created to ensure that individuals and minorities are allowed to develop mental models within their own context.This has been described as the 'salad' model of diversity, where the richness of the mixture is seen as valuable, instead of the 'melting pot' where uniformity is encouraged (Alexander 2006).
It is also important to consider the motivational, affective and cognitive factors (Jackson et al. 2000) that might explain why some students are more willing to use VLEs than others.According to Mackie and Beeby (2002), unwillingness may be due to anxiety or a sense that change is imposed and may detrimentally affect learning.
Resistance may also be about risk adversity and an unwillingness to jettison old (successful) learning habits in order to try something new (Akerlind and Trevitt 1999).These concerns highlight the importance of understanding students' conceptions of themselves in relation to study.

The importance of self-efficacy in learners
An important element of students' self-conception, particular in relation to their classification as being 'at-risk', is the concept of self-efficacy (Bandura 1986).This is seen as a factor in student willingness to engage with a course of study and a determinant of whether there is a positive relationship with achievement.Self-efficacy refers to people's judgments of their capabilities to organise and execute the courses of action required to attain designated types of performances.In other words, it is the belief that one can do a task and can transfer the learning of that task to similar tasks.Of particular importance to our project is Willis' (1993) reference to the point made by Entwistle and Tait (1990) that it is students' perception of their environment rather than its objective reality that impacts on learning.Steinbronn and Merideth (2003) add that self-efficacy supplies motivation to persist and, since motivation enhances problem solving, it influences later success.When applying the concept to online teaching and learning they emphasise the need to design online support that should positively impact the psychological and physical environment of teaching and learning in an electronic area and influence the retention rate of students.An individual's perceived self-efficacy, according to Houghton, Neck, and Manz (2003), will affect whether they are successful in performing a task.They state that individuals who believe in their ability to complete a given task exert more effort and persist longer, thereby sustaining performance levels until success is achieved.Mackie's (2001) study supported this view: Committed students have positive, realistic expectations of university and a long germ goal, they are highly motivated and determined to stay despite any difficulties.(Mackie 2001, 270) However, the poorly committed student experiences a lack of control over events and feels helpless.
Furthermore, feelings of self-efficacy relate to student engagement of the two types as distinguished by Willis (1993): • institutional engagement, the integration of the student into campus life; and • academic engagement, connected to factors directly linked with learning.This concept of engagement has become a current buzzword in today's educational environment.Kearsley and Shneiderman (1999) propose engagement theory as a model of technology-based teaching and learning.The idea underlying engagement theory is that students must be meaningfully engaged in learning activities through interaction with others and worthwhile tasks.Their theory posits three primary means to accomplish engagement: (1) an emphasis on collaborative efforts involving communication, planning, management and social skills; (2) project-based assignments which make learning a creative, purposeful activity; (3) non-academic focus e.g. when the project has an outside customer in the community.It is proposed that such a model of engagement pre-supposes self-efficacy, hence the focus here on this concept as a means of supporting engagement.

Building self-efficacy
Early interventions and feedback build students' notions of self-efficacy.Simpson (1990) described the impact of an early intervention (a phone call prior to submission of the first piece of assessed work) on building the confidence of at risk students in an online environment.Levy's (2007) study into dropout rates found that students who withdrew reported significantly lower satisfaction with e-learning than students who successfully completed the same course.These findings have played a major role in the development of the intervention processes in our own project.
The VLE offers opportunities for early and frequent formative feedback through online assessment opportunities, and it has been argued that feedback has an impact on the process of engagement as learning theory recognises the important function of feedback.Biggs' (1999) concept of constructive alignment is also relevant, with desired behaviour being shaped by formative feedback.
While we have been discussing students as a homogenous entity, a study involving mass modularisation of courses must recognise the changing and diverse student population.Many authors highlight the changing attitudes and circumstances of students and how this impacts on their motivation and academic performance (e.g.McInnis 2001;Kuh 1999).A study by Smart et al. (1999) discovered that in comparison to undergraduates surveyed 10 years before, undergraduates of the day were more bored in class, more often over-slept and missed classes or appointments and were less likely to study or do homework for six or more hours a week.We need, therefore, to take into account that today's students may be different to those who attended university before the advent of e-learning.
Of particular interest was the finding by Smart et al. (1999) that, when examining 'at risk' students (those in poor academic standing), marketing majors were slightly more likely to experience academic troubles than general students at-large.They offered one potential explanation for this unfavourable comparison, marketing students typically work more hours per week for money than the other students, which would leave less time to study.This situation is likely to be more widespread now, with the introduction of fees for study in the UK.
With e-learning enabling students to spend more time off campus, Morgan (2001) argues that students need the motivation of feeling that they are a valued member of a learning community.He refers to the experiences of institutions that have employed liberal open learning practices, concluding that transferring unrestricted power to students may not be in their best interests: The reality of providing unfettered freedom for the student has resulted in unacceptably low completion rates.(Morgan 2001, 2) Salmon (2006, 8) supplies an explanation for this finding by proposing that learners cannot affect their own learning until they have fundamental skills and maturity.
As well as differentiation between students in terms of their paid work commitments, other factors may also shape their experiences.Lee (2001) developed a model that proposes distinctions between types of learners.The model learner scores highly on both academic achievement and satisfaction and is a frequent user of Web-based learning resources, while the maladaptive (here, 'disengaged') student scores low on both counts and is a low user of technology.This project was interested in establishing whether it can confirm any correlations between learner type, the use of technology and academic performance.The suspicion is that the students disengaged with the VLE ('maladaptors') (Lee 2001) constitute the 'at risk' group.

Case context
'New' universities are polytechnics, central institutions or colleges of higher education granted university status by the Further and Higher Education Act 1992.Student numbers on modules are much greater in many of these institutions compared with more traditional universities.Many practice a cross-faculty modular scheme so students interact with a great many students but may not form the more traditional support network of key peers.
Since September 2003, the two modules considered here have used the VLE in a new university.The core of learning remains face-to-face, but pre-and post-session learning activities increasingly take place online -preparatory work, readings, links to a wide range of resources, advice, formative assessment and revision.It is valued by staff as a 'one-to-many' electronic notice board and filing cabinet; an easy-to-update mechanism for course content distribution; and as a portal to a wide range of internal and external resources that blurs the boundaries of the module.

Methodology
A key part of the literature which informed the research design of the present study was the importance of early interventions and feedback in building students' notions of self-efficacy.Of particular relevance was Simpson's (1990) description of the impact of early interventions in building the confidence of at risk students in an online environment.
The objectives were to investigate the following questions: • What does the tracking data reveal about the online learning behaviours of students?
• Is there any relationship between level of use, as measured by tracking data, and student performance?• Is the relationship the same for all forms of assessment and at all times of the academic year?
It should be stated at the outset that, for ethical reasons, it was impossible to have a control group for comparison that would not be exposed to the modules' VLE.Tracking statistics indicating the amount of student access to the system were recorded at several points throughout the year and the non-users and low users were contacted personally by module leaders.At the beginning of the academic year, straightforward induction tasks linked to the use of the VLE were incorporated on the website to emphasise the importance of early and frequent use.
The key monitoring and intervention points during the investigation were: (1) Induction tasks linked to use of the VLE to emphasise the importance of early use. (

Is VLE tracking ethical?
All research activities need to take ethical issues into account but the use of VLE tracking data in this project highlights particularly complex issues.The surveillance tools within VLEs enable lecturers and management to generate records for each user indicating their level of accesses to the module, which particular areas they access and the time of day or, more surprisingly, time of night at which they access resources (Maltby and Mackie 2003).Land and Bayne (2002) stated that these tools are far more than the electronic equivalent of the attendance sheet since they provide intimate details of every student's working hours and patterns of study.Cyber-surveillance theory, building on the panopticon metaphor highlights the need to understand how the virtual environment works.(The panopticon was a prison design created by Jeremy Bentham in 1785, where prisoners were able to observed at all times without seeing the observers (Bentham 1962;Foucault 1979)).
Our chances of developing effective pedagogies for online learning will be greatly enhanced if we are prepared to recognise and work with the new modes of identity formation and new articulation of power/knowledge that cyberspace technologies represent.(Land and Bayne 2002, 137) Further research needs to be conducted on how students feel about these ethical issues of surveillance.Results may confirm Schlossberg and Cebrzyninski's (1993) observation that most people believed that they had already lost the right to control their personal information because new technologies and large databases facilitate the increasingly effective use of information about consumers and citizens.However, a recent study by Dawson (2006) revealed that surveyed students indicated that their online behaviour was influenced by the degree to which they perceived it to be surveyed by both the institution and teaching staff, suggesting that it remains an issue for students.
In our study, all the students on the two modules appeared in the tracking data.They were advised that their online behaviour would be monitored and that they were unable to opt out of the process.Where individual students were contacted by the module leaders their anonymity was protected in the research findings (and this was made clear to them).

Findings
Firstly, the relationship between tracking and performance data is presented for the two modules.This is followed by the findings suggesting the importance of early monitoring.Thirdly, we discuss the group of students who prefer traditional face to face teaching.

An overview of student use
Firstly, we were interested in knowing whether there were any particular patterns in student use or examples of optimum usage.
The range of student use is highly variable, throughout the year (September to May).For Organisation Studies level 1 students the average number of clicks per student was 189, but the range is from 0 to 1432.However, for Marketing level 2 students, the average per student is higher, 222, and the range is wider from 0 to 2429.

The relationship between tracking and performance data
This is a key part of the project as it indicates whether there is any relationship between assessment performance and the use of the VLE.
For Organisation Studies Level 1 there was a steady increase in assessment performance in relation to the amount of times students accessed the VLE. Figure 1 shows Total tracking into five groups that those who only registered 0-32 clicks failed the module (average 34%) and those in the high use group (385-1432 clicks) averaged 55%.However, the results for Marketing Year 2 students were not as straightforward.There seemed to be an optimum level of use i.e. between 277 and 437 clicks when students achieved a mean of 52%. Figure 2 shows that those students registering more than 440 clicks dropped slightly to 50%.

Can tracking data predict performance and identifying 'at risk' students?
The literature suggested the value of identifying 'at risk' students early on in the academic year.The results demonstrate that the online behaviour patterns of potentially 'at risk' students are formed surprisingly early in their university life.The Organisation Studies Level 1 findings reveal that by the end of October, 10% of students were not enrolled as users of the VLE and 20% of students had not yet used the VLE.These students performed less well (at 39% or 40% overall module mark) than students with higher use patterns at that stage (56% overall module mark).Despite interventions in October to the Organisation Studies students, non-users/low performers failed to change their early online learning intentions.The Marketing Year 2 results did not show such clear differences.
We must also take into account the responses to a questionnaire sent to non-and low users which revealed a group ('the traditionalists') who were still engaged as they respond to emails/letters and were not 'lost' to the system, yet these students found the written resources (e.g.textbook) more helpful and expressed a preference for traditional face-to-face teaching over online learning.

Discussion
This section will mirror the three strands discussed above, namely, the current thinking on e-learning, including whether e-learning can benefit students, the concept of self-efficacy and the ethical issues in the use of VLE tracking to monitor student behaviour.

Can e-learning benefit students?
There are many possible reasons for the different levels of use between the groups, which would have to be researched further.One confounding variable is the fact that some students accessed the VLE using their password on their friend's behalf to download lecture slides etc.Since for ethical reasons, it is impossible to have a control group, for comparison purposes, who have no access to the e-learning environment, we have no way of knowing whether use of the VLE improves students' performance overall.
Another possible issue relates to the finding by Smart et al. (1999) that marketing students typically work more hours per week for money than other students, which would leave them with less time for study.We could suggest that since students in this position are away from the university more, they need to use the VLE to make up for what they are missing through poor attendance.However, the opposite may be the reality, possibly because according to Morgan (2001), transferring unrestricted power to students may not be in their best interests.Similarly, Salmon (2006) suggests that learners cannot affect their own learning until they have fundamental skills and maturity.
There is no doubt that the analysis of the project's findings could have been much richer if the researchers had been able to interview the 'non-participating' students but since they did not respond to any form of communication, or attend classes, they were impossible to reach.Mackie and Beeby (2002) suggest that unwillingness to use VLEs may be caused by anxiety.Similarly, Akerlind and Trevitt (1999) refer to risk adversity as being a potential factor.Solely using tracking statistics cannot illuminate these issues.

Relationship between tracking and performance data
These results were not straightforward and showed further discrepancies between the two modules.While the Organisation Studies results showed a positive correlation between the amount of VLE access and assessment performance, the Marketing results were more complicated.The optimum level of access was between 277 and 437 clicks.Increased access showed a detrimental effect on assessment performance.
An interesting explanation is proposed by Mason (2002, 7), who suggests: The flexibility of the medium easily leads learners to allow other priorities to come before logging on to the course or group work.The much vaunted interactivity easily leads to overload.The ability to jump from one resource to the next on the Web (i.e.hypertext) can be over-used so that relatedness becomes an end in itself, and meaning is lost.
Many other factors identified in the literature review could enrich the findings with further research.These include the social, cultural, political and economic aspects of educational computing (Selwyn 2000;Alexander 2006); the motivational, affective and cognitive factors (Jackson et al. 2000); and students' prior knowledge (Corredor 2006).

The importance of self-efficacy in learners
The rationale for this research was to use the VLE as an aid for identifying students who can be deemed 'at risk'.It had been assumed that early intervention would be important for some students in their first year.However, in contrast to this assumption, it was found that patterns of online learning appear to be fixed early on in students' university careers.Despite the email interventions in October to the Organisation Studies students, non users/low performers failed to change their early online learning intentions.
We have already defined self-efficacy as a factor in student willingness to engage with a course of study and in having a positive relationship with student achievement.Our results highlight the need to investigate further those students whose behaviour in the first October defines their university career.In particular, we need to study those factors highlighted by Selwyn (2000), Alexander (2006), Jackson et al. (2000) and Corredor (2006).

The benefits and risks of online surveillance
We have reported that the results demonstrate the importance of monitoring early online behaviour to be able to identify possible 'at-risk' students, which supports the findings of Simpson (1990).Similarly, Morris, Finnegan and Wu (2005) demonstrated the relationship between online user-behaviour and academic performance.The employment of online surveillance thus enables the identification of students who may require additional scaffolding.
This complicates the ethical situation, since surveillance may be in the students' best interests -it could be considered unethical for a tutor not to attend to students' behaviour in this way.However, we must question the reasons and usage of online surveillance.An ongoing discussion in the academic community centres on how we can encourage independent learners who engage in deep learning (e.g.Entwistle 2001).Further research needs to be conducted into whether by monitoring and intervening we may be unintentionally encouraging surface learning and more student dependence.

Lee's model of learner engagement in Web-enhanced environments
We have used this model to validate our own research findings.Lee's (2001) model suggests the emergence of four distinct learner types, shown in Figure 4, each with a different combination of levels of use and academic performance:

Model students
High use and high performance.Students' expectations of the VLE are in line with tutor intentions (i.e. the VLE as Web-enhanced learning).These students understand what is expected of them and therefore learn appropriately as they make best use of all of the resources and learning opportunities available to them.

Traditionalists
Low use and high performance.These students favour face-to-face and print-based learning.They are clear about this preference and learn appropriately from it.This group also objects to correspondence that implies a link between their lack of use of the VLE and poor motivation and performance.As the core of the learning is still faceto-face for both modules they are not significantly disadvantaged by their low use of the VLE as they are part of the learning community.This group would, however, be disadvantaged by any move along the blended learning continuum towards a distance learning model.

Geeks
High use and low performance.There is a mismatch between student and tutor intentions for the VLE.Geeks may interpret the VLE as a distance learning course or simply be motivated by the technology rather than the course content or learning activities.

The disengaged
Low performance and low use of the VLE.Mañana learners (and tomorrow never comes).For reasons not understood at present, by the end of October this group has started on a downward spiral of non attendance.They start off badly and never recover and increasingly operate at the periphery of the organisation.They learn inappropriately and fail at assessment points as they do not access the information available to them about how to succeed.
While some of our findings concur with Lee's groupings, further investigation needs to take place into the comparison of online behaviour statistics, face-to-face attendance patterns and assessment performance.

Conclusions
This paper aimed to investigate whether VLEs help or hinder student engagement and performance.It compared and analysed click count tracking data with student assessment grades on large Level 1 and Level 2 modules.The ultimate aim was to assess whether VLE tracking could identify the 'disengaged' student allowing the opportunity for staff interventions to help rectify the situation.
Tracking data revealed that students varied greatly in the amount that they access the VLE.At best, as witnessed by 'model' learners, the VLE promotes a positive learning climate and thereby student engagement as students benefit from flexibility and access to a wider range of resources and 'Martini' (anytime, any place, anywhere) 'just in time' learning.
At worst, as witnessed by 'disengaged' learners, the VLE promotes a negative learning climate and thereby student disengagement in that when used inappropriately it may encourage 'Mañana' learning, a sense that learning does not have to be captured in the moment and can be put off to some vague future date.
At this stage the learner typology is only tentative and requires further testing and development, most specifically with regards to the most problematic non-user/ poor performer disengaged group.We have already highlighted the problem of  communicating with this group and the necessity of investigating whether anxiety and risk-adversity to using VLEs is relevant to their lack of engagement.
The literature review has revealed many other factors which were not included in the research design but which could warrant future investigation.These include the social, cultural, political and economic aspects of educational computing (Selwyn 2000;Alexander 2006), the motivational, affective and cognitive factors (Jackson et al. 2000) and students' prior knowledge (Corredor 2006) and whether or not online surveillance nurtures surface and dependent learners (Dawson 2006;Land and Bayne 2002).
) Monitoring point one at end of October.Tracking statistics taken of number of times students accessed the VLE.A first email was sent to all non-users and low users to remove any technical or personal barriers to use and to promote beliefs of self-efficacy.(3)Monitoring point two (a) March.Tracking data were recorded.An email plus questionnaire were sent to low and non users.(4) Monitoring point two (b) April.Letter plus questionnaire were sent 22 April to those who had not replied to the email.(5) Overall summaries of course usage were calculated for -October, term 1, term 2, and overall usage.(6) End of year Field board reports were analysed -marks for the coursework component, examination component and overall module were noted, plus the pass/fail outcome.

Figure 1 .
Figure 1.Correlation between online behaviour and assessment performance (MOB students).

Figure 1 .
Figure 1.Correlation between online behaviour and assessment performance (MOB students).

Figure 2 .
Figure 2. Correlation between online behaviour and assessment performance (Marketing).

Figure 3 .
Figure 3.The value of early tracking data for MOB students.

Figure 2 .
Figure 2. Correlation between online behaviour and assessment performance (Marketing).

Figure 3 .
Figure 3.The value of early tracking data for MOB students.

Figure 4 .
Figure 4. Adaptation of Lee's (2001) model of learner engagement in Web-enhanced environments.

Figure 4 .
Figure 4. Adaptation of Lee's (2001) model of learner engagement in Web-enhanced environments.