Relationships between conceptions of learning approaches to study and students ' judgements about the value of their experiences of networked learning

This paper reports on an enquiry into relationships between students' views of their experiences of participating in networked learning courses and data on their conceptions of learning and approaches to study. It has been suggested in the literature on networked learning that students with more sophisticated conceptions of learning and students who take a deep approach to learning are more likely to benefit from, and have positive experiences of, networked learning. Drawing on a sample of almost 180 undergraduate social science students on four networked learning courses, we established that there were no strong links between students' judgements about their experience of networked learning and either their conceptions of learning or their approach to study. Further research is needed, but a practical implication of this study is that it is reasonable to expect all students to have positive experiences on well-designed and well-managed networked learning courses not just those students with more sophisticated conceptions of learning or deep approaches to study.


Introduction
Very large numbers of students and teachers in UK higher education are now taking part in some form or other of online education.Email and the World Wide Web are now used for study purposes on an almost daily basis by the great majority of staff and students (Goodyear, Jones, Asensio, Hodgson and Steeples, 2001).Very many universities have invested in 'Virtual Learning Environments' or 'Managed Learning Environments' (VLEs and MLEs) which allow integrated delivery to the student of a range of electronic services.
In this paper, we are particularly concerned with a genre of online education which we call 'networked learning' (Steeples and Jones, 2002).We define 'networked learning' as: learning in which information and communications technology (ICT) is used to promote connections: between one learner and other learners, between learners and tutors; between a learning community and its learning resources.Some of the richest examples of networked learning involve interaction with online materials and with other people.But, in our view, use of online materials is not a sufficient characteristic to define networked learning.Human-human interaction (computermediated communication or CMC) is an essential part of networked learning.It is an important part of the pedagogical rationale for engaging 'conventional campus-based' undergraduate students in online education.
Much of the pioneering work in networked learning at the university level has taken place in the context of postgraduate courses (see, for example, Hartley, Tagg, Garber, Barry and Fitter, 1991;Steeples, Johnson and Goodyear, 1992;Hodgson and McConnell, 1992; and also the collections by Mason and-Kaye, 1989, andKaye, 1992).There are some case studies of undergraduate use of networked learning (such as Nicholson, 1999 andpapers in Banks, Goodyear, Hodgson andMcConnell, 2002).There is also a small literature on students' learning styles, personality measures and participation in networked learning (e.g.Atkins, Moore, Sharpe, and Hobbs, 2001;Dewar and Whittington, 2000;Smyth and Buckner, 2000).But, other than in the Open University, there has been relatively little research into networked learning on undergraduate courses jn the UK and a lingering suspicion that it may be better suited to the postgraduate level and/or to areas of higher education in which students can be relied upon to be self-organizing, self-motivating and reflective (see for example, Romiszowski and Mason, 1996;Fredericksen, Pickett, Shea, Petz and Swan, 2000;Laurillard, 2001;Klassen and Vogel, 2001;Knight, 2002: 119).
Two studies which investigate connections between 'campus-based' undergraduates' participation in networked learning and their approaches to study should be mentioned.Both were single institution studies and both used instruments developed by Noel Entwistle and his colleagues to represent aspects of students' approaches to study (Tait and Entwistle, 1996;Entwistle, Tait and McCune, 2000).Gibbs (1999) reports data from a second-year course module in philosophy at the University of Huddersfield, for which he was also the module leader.The course used the Co Mentor virtual learning environment.As part of the end-of-course evaluation, the 106 students who took the module were asked to complete a version of the ASSIST approaches to study inventory (Entwistle et al., 2000).Students could choose between (a) having six of their online contributions count towards their module assessment or (b) taking an end-ofcourse exam.Gibbs' data show that students who took option (a) -students he regards as making regular use of CoMentor -scored higher on deep and strategic approaches to study than did students who chose option (b).Using students' own reports of their frequency of use of CoMentor, the relationship between frequency of use and a deep approach weakens but a strong association with a strategic approach remains (Gibbs, 1999: 226-8).Interpretation of the data needs to be tempered by the knowledge that only half the students completing the course also completed the end of course evaluation and only half of these (29 students) took option (a).Light and Light (1999) also report research from a single institution (Southampton University).They asked students who had and had not been active participants in CMC discussions on their course to take a version of Entwistle's Approaches to Study Inventory (ASI - Tait and Entwistle, 1996).They found that those who had participated actively in the discussion scored higher on the 'deep approach to study' subscale while the nonparticipants scored higher on the 'surface approach to study'.However, (i) levels of participation in the CMC discussion had been low and unevenly distributed, with 60 per cent of the students having submitted one or no contributions, (ii) most student contributions were questions directed to the lecturer, rather than part of a sustained group discussion, and (iii) only 24 students were involved in the study (12 participants; 12 nonparticipants) (Light and Light, 1999: 168-9).
These two studies are suggestive of some associations between approach to study and involvement in networked learning.However, the limitations of the studies -in the context of a relatively unexplored field -mean that further research is needed to determine whether or not there are significant connections between students' approaches to study and their experience of networked learning.

Research aims
We set out to test the following conjectures: 1. Students with more elaborate conceptions of learning will report more positive end-ofcourse judgements about the value of their experiences of networked learning than will students who have less elaborate conceptions of learning.
2. There is an association between approach to study and judgements about the value of networked learning experiences: deep and strategic approaches will correlate positively and surface approaches negatively with such judgements of worth.

Methods
Much of our research in the field of networked learning has made use of interview-based data sources, allowing us to come closer to the personal experience and sense-making of the participants (see, for example, Steeples and Jones, 2002).However, there is also value in complementing these studies with broader-based survey material.The present study makes use of self-completion questionnaire techniques.These have the virtue of allowing comparison across large numbers of students, though the imperatives of the survey method remove nuances from the data.

Instruments
The research reported here makes use of three main instruments.Two of these have already been described in the literature, so we account for them only briefly here.The third is new and warrants a more detailed description.
The most familiar instrument is Entwistle's Approaches to Study Inventory (ASI).The ASI has been a linchpin of research in higher education since the late 1970s and has been tested in a very large number of studies (Entwistle and Ramsden, 1983;Entwistle et al., 2000).The instrumentation for the ASI has evolved over the years and, like Gibbs (1999) above, we used the version called ASSIST.ASSIST exists in a number of forms.We used the 51-item ASSIST at the start of each course and the shortened 18-item ASSIST at the end of each course.ASSIST allows one to characterize students' reports of their approaches to study in terms of Deep, Strategic and Surface/Apathetic approaches.' The second instrument is also based on Entwistle's work with ASSIST, as well as on the ideas of Marton, Saljo, Beaty, Dall'Alba and others (see for example, Marton, Dall'Alba, and Beaty, 1993;Beaty, Dall'Alba and Marton, 1997).The focus here is not on approach to study but conception of learning.Research on conceptions of learning among university students is not as highly developed as research on approaches to study.However, there is a general view that students adopt more or less elaborate conceptions of learning and that the more elaborate conceptions include rather than displace the less elaborate conceptions.Students might typically move from a conception which is very focused on acquiring and recalling factual information towards one which is concerned with developing as a person.Our questionnaire presented students with eight statements about learning.These are reproduced in Table 2. Students in our study were asked to record their response to each statement on a 5-point Likert scale (ranging from 'very close' to 'very different' to/from my own way of thinking about learning).
We developed the third instrument ourselves.The aim of this instrument was to capture students' overall judgements about the value of their experiences of networked learning, at the end of their course.In particular, we wanted them to focus on whether the course's use of technology seemed worthwhile and enhanced their engagement in study activity.The end-of-course questionnaire contained forty Likert statements relating to the students' thoughts about the use of networked learning on the course they were just completing.These statements were presented in two sets.A set of seventeen statements opened the questionnaire.Then came the eight statements about conceptions of learning, followed by the eighteen-item ASSIST inventory.After this came the second subset of twenty-three statements about networked learning.We used the same set of twenty-three statements on both the start-of-course and end-of-course questionnaires, adjusting only the tense of each statement so that it was in the future tense at the start of the course and the past tense at the end of the course.We have investigated the change in students' responses to these twenty-three items from the start to the end of each course and have established that though students' views moderate over time, the structure of expressed beliefs represented by the twenty-three statements remains fairly constant (Goodyear, Jones, Asensio, Hodgson and Steeples, 2002).Consequently there is good reason to think that these twenty-three statements are robust in combination as a tool for capturing some important dimensions of students' reported experiences.
To construct a single measure tapping students' overall judgements about the value of their experiences of networked learning, we took their responses to the forty Likert statements and subjected them to principal components analysis.This is a technique for reducing the complexity of data by revealing underlying patterns of association between variables -in this case the forty Likert statements.One very strong component emerged from this analysis.It accounted for 24 per cent of the variance in the set of forty statements.(Each subsequent component added only 1-2 per cent more in explained variance.)This caused us to focus on those Likert statements which were strongly associated with the first component: that is, those twenty Likert statements which proved to have component loadings greater than plus/minus 0.5.The twenty statements are listed in Table 1, together with their mean scores for the whole sample of students.We then constructed a measurement scale by summing the scores on each of these twenty statements (reversing the values on the nine negatively phrased statements) and dividing the result by twenty.Thus a high mean value denotes a positive judgement about networked learning on all twenty items.We carried out a reliability analysis of this twenty-item scale.The results were very good indeed.The most commonly used measure of the reliability of this kind of scale is Cronbach's Alpha.The Alpha for this scale was 0.913, which is high.We also calculated a split-half reliability measure, using Guttman's technique, and again found a reassuringly high value of 0.865.This leads us to conclude that the scale is highly reliable.We examined each of the twenty items in turn to see whether the Alpha for the scale as a whole would be improved by the deletion of the item.For none of the items was this the case, so all twenty items are retained in our overall measure.It is always a little dangerous to try to capture the flavour of such a measure in a short title, but for convenience in reporting results in the rest of this paper, we refer to the scale measure as JNLjudgements about networked learning.Note that we do not claim that this is a stable or general predisposition of the students.It is a simplification of their reports of their experience, made in the context of (the end of) a specific course.

Statement
I enjoy working with the technology on this course I think the technology is helping me learn I believe the technology is helping me to achieve my personal aims on the course I think I am able to study more effectively using the technology I find I am working with others more easily using this technology I would like to take another course taught using technology like this Technology has been particularly important in the running of this course The technology was easy for me to use I was confident when using the technology on this course Using the technology on this course suited the way I did my work I was excited by the technology on this course I feel isolated working on this course The technology makes it difficult for me to know what I am expected to do I feel I would be happier doing this course without the technology The technology did not suit the way I managed my time I thought using technology was second best to traditional methods The technology distracted me from the course content I wondered whether using the technology on this course was really worthwhile I needed more help on this course because of the technology I wasn't really interested in technology

Sample
We gathered data from 178 students who were involved in one of four networked learning courses in UK universities.The courses all involved a significant element of online communication, as described in the definition above, and also involved some timetabled face-to-face activity.None of the courses was being run for the first time.All the students were undergraduate.Some were part-time, some full-time.All the courses were in an area of the social sciences and lasted either a term or a semester.The gender balance was 59 per cent female, which is not unusual in social science courses.Mean age was 24.7 years, with a median age of 22.All students were given a questionnaire at the start and towards the end of their course.The end-of-course questionnaires were completed before the students knew their end-of-course results.Students completed the questionnaires in timetabled face-toface meetings so response rates were high and there are no grounds to presume that the responses disproportionately represent students with strong views about their experience (positive or negative).Although tutors were able to see collated results, they could not link any questionnaire to the student who had completed it.
Most of the data reported in this paper comes from the end-of-course questionnaire.However, we also wanted to be able to draw on the 51-item version of the ASSIST approaches to study inventory.This was completed by students when they filled in the start-of-course questionnaire.The sample of students for whom we had both start-ofcourse and end-of-course questionnaires was smaller: 119 cases, of whom 106 provided a full set of data.

Conceptions of learning
The first part of the analysis is concerned with conceptions of learning and the students' overall judgements about the use of networked learning technology on their course.In each case we show the number of students (the n column) who said that the statement was 'very close' to how they thought about learning and the number who did not say the statement was very close to how they thought about learning.(Note that it was open to students to rate any or all of the statements as 'very close' to the way they think about learning.)The column headed 'mean JNL' shows the mean score on the overall judgement about networked learning scale for each of the two groups of students ('very close' and otherwise), in relation to each of the eight conceptions of learning.
One-way analysis of variance (ANOVA) tests were run on each of the eight pairs of means.None of the ANOVAs showed a statistically significant difference between the means.The F values and significance levels are shown in the columns headed F and p respectively.Only statement 2, at p=0.066, comes anywhere near approaching a conventionally accepted level of statistical significance.
The interpretation of Table 2 is clear.At least in this sample, a student's conception of learning has no effect on their overall judgements about their experience of networked learning.
To check the robustness of this finding, we ran two more tests.First, we looked at the small subset of students who said that both statement 1 and statement 3 came very close to their way of thinking about learning.These 24 students could be said to be strongly oriented towards a conception of learning which is concerned with building knowledge by acquiring facts and information and ensuring that they remember things well.This is towards the least developed end of the hierarchy of conceptions of learning.We checked to see whether the mean JNL score for these 24 students differed significantly from the mean JNL score of the other 153 students.It did not.The means were identical.
Finally, in case the way we had split students into the 'very close' and 'other' categories had had an effect on the relationship with JNL score, we calculated Pearson product-moment correlation coefficients between the JNL score and scores on each of the eight 'conceptions' variables.None of the correlations was significant at the 0.05 level.The nearest, again, was statement 2 (r= 0.176; p=0.065).
This leads us to conclude that there is no significant relationship between a student's conception of learning and their judgements about the use of networked learning technology.Those who believe that students with more elaborate conceptions of learning are likely to be more positive about networked learning may need to revise their views.

Approaches to study
The second part of the study moves from conceptions of learning to approaches to study.We had calculated, for each of the students in the sample, their scores on the Deep, Strategic and Surf ace/Apathetic approaches to study.These were based on the 18-item version of the ASSIST inventory.We calculated Pearson product-moment correlation coefficients to measure the association between the JNL score and each of these three ASI scores.We found statistically significant correlations between JNL score and each of the three measures (see the left hand half of Table 3).The correlation coefficients themselves were modest in size, with the strongest correlation being between the Surface/Apathetic Approach and JNL.Note this correlation is negative, meaning that students with a Surface/Apathetic approach to study make more negative judgements about networked learning.However, some researchers have raised doubts about the strength of the 18-item ASI as a research tool (for example, Richardson, 2000: 112-123).Consequently we set out to replicate the correlations within the smaller student sample for which we had ASI results on the 51-item ASSIST inventory.
The resulting correlations are shown in the right hand half of Table 3.In each case, the correlation coefficients are lower, though they are in the same direction as the correlations for the 18-item ASSIST.The lower correlations, coupled with the smaller sample size, place each of these associations on the margins of statistical significance (p>0.05 in each case).
This must cause us to view the correlations from the 18-item ASSIST with some scepticism.At best, there is tentative evidence of a weak association between approach to study and judgements about networked learning.Deep and strategic approaches show a weak positive association with a positive judgements about networked learning while the surface/apathetic approach shows a weak negative association.
Given the volume of literature underpinning the ASI as a predictor of positive learning outcomes in conventional undergraduate courses, the uncertainty surrounding our results is worthy of some further investigation.What we can say, however, is that we have found no convincing evidence of a strong association between approach to study and judgements about networked learning.

Conclusions
This study has a number of implications for practice and for further research.First, we would argue that there are no good grounds for assuming that only students with sophisticated conceptions of learning, or who adopt a deep approach to study, are likely to benefit from engagement in networked learning.If networked learning courses are welldesigned and well-managed, then all students are likely to experience benefits.Second, good design and management needs to be informed by an understanding of how students experience networked learning.The JNL scale developed in this study foregrounds the use of technology.In some ways, this is a shorthand for 'the networked learning approach' in which the students found themselves to be involved.Our in-depth interviews with students show that issues of collaboration and co-operation in learning were also in the foreground (Goodyear et al, 2001).Students showed particular concerns about the dependence of their assessment results on the efforts of others -an observation which is by no means specific to networked learning contexts.Further work aimed at representing students' experience of networked learning needs to help them articulate these and other concerns about efficient, effective and enjoyable group-based learning.Finally, research and design need to help us clarify expectations about the degree and pattern of involvement in online activities.A theme in the literature on networked learning reflects disappointment about lower than expected levels of student engagement in online activities (for example, Jones, Asensio and Goodyear, 2000).We have little to help benchmark expectations -to help students, teachers and researchers ground their expectations about educational benefits in a realistic set of heuristics about the level and degree of activity required.In further work, we aim to explore these connections more thoroughly, looking particularly at the nexus between task design, online activity, student experience and learning outcome.

Table 1 :
20 items in the JNL scaleThis completes the description of our main instruments.We also gathered data on the sex, age and year of study of each student, together with information on their course and university.

Table 3 :
Approaches to study (after Entwistlejait and McCune, 2000)and judgements about networked learning