Grounded Theory as an approach to studying students ' uses of learning management systems

A 'Grounded Theory' methodology (Glaser and Strauss, 1967) was selected to model the students' framework, as independently as possible from the views of other stakeholders in the LMS. The approach still offered the potential for generalization despite working with a small group of students. In Grounded Theory, data collection and theory-building drive each other reciprocally until a stable 'theory' is reached.


Introduction
The study was designed to examine issues relevant to planning the full-scale implementation of the Learning Management System (LMS) across the university.Two aspects of usability were of particular interest: whether the system could be improved through changes in the design of the interface and whether a student's ability to use an LMS successfully was determined by technical abilities developed either at, or before, university.Students from two courses had experience of using both WebCT and Blackboard.A study using these two groups offered the opportunity to investigate how differences in the interface had affected the usability of the system for students with a relatively high degree of technical competence.
An LMS offers different benefits to the various stakeholders in the system.For management an LMS can provide operational benefits, such as integration with other management systems (student records and finance, etc.) as well as strategic opportunities, for example to increase student numbers by decreasing the time spent by each student within the university.Such benefits do not necessarily offer educational advantage to the students.
For a student an LMS would appear to offer few technical advantages beyond those that are available free through other Internet providers offering support for more generic online communities.Both WebCT and Blackboard provided similar functions as learning environments accessed through a Web portal.Each offered links to a number of integrated functions supporting document management, both synchronous and asynchronous communication, and integration with email services.Any educational advantages were essentially mediated by the design of the interface.
The arrangement and labelling of icons and links to the various facilities controls the students' expectations of what activities they could be engaged in.If the learning management systems were substantially the same, then differences would only be realized if the students could anticipate the benefits that might be available and could successfully navigate to those different services.Simple differences in the usability of the interface could provide significant educational gains.The researchers' own experiences with Blackboard suggested that there was considerable potential for redesign.This research focused on students studying Geographical Information Systems (GIS) in their final year of two courses, either for a B.Sc. or an HND.These students had all used Blackboard during 2001 and WebCT during 1999-2000.WebCT and Blackboard were both offered with the normal suite of services from within the university.Only Blackboard was made available through remote access (with password protection).Although the students were supported by an LMS at each stage, the courses were still structured as conventional undergraduate courses with attendance required for lectures and other workshop activities.

Methodology
The researchers were aware that their perception of what was critical in the design of an LMS might not be seen as important by the students.The selection of a methodology that was relatively immune to the expectations of the researchers was critical.

Selection
The first stage in the research was the selection of a methodology appropriate to the outcomes that were required and the constraints of the particular study.The most critical requirement was the importance of capturing the students' own perceptions and understanding of the systems that they were using.
Many research methodologies expect the research to be conducted within an objective, conceptual model that is framed by others external to the subject group.Several stakeholders, other than the students, have an interest in the performance of the LMS: university managers, system designers, system implementers, course lecturers and, of course, ourselves as researchers.Research that is based on an external framework may well provide evidence that a system is not working -but will seldom provide answers as to why or how the system fails to work.This requirement implied that a qualitative approach, grounded in the students' own conceptual frameworks, would be essential.
The second requirement was that the methodology had to support the potential of building a theory about what was being studied.This would enable the results of the research to inform the next stage of the development in the use of the LMS.This requirement would exclude any methodologies, such as ethnography, which did not conform to this.
A third class of methodologies was excluded by the constraints of the study.Some qualitative approaches use an initial subject-centred qualitative phase as grounds to generate relevant questions for a larger statistical analysis.The timing of the research in two phases would not produce any results over the time-scale available as all the students with experience of both systems would have left by the end of the year and the group would not have been large enough for a statistical analysis.
Our principle interest lay in providing a rich description of the range and variety of student perceptions and the factors that could be altered to produce different behaviours.The overall research model selected was Grounded Theory (Glaser and Strauss, 1967;Glaser 1978;Strauss and Corbin, 1990;Miles and Huberman, 1994).A more detailed discussion of this selection process is provided elsewhere (Alsop and Tompsett, 2002).

Grounded Theory
The Grounded Theory method requires that the full research process is grounded in the data that is collected.As with most qualitative methods it accepts that the researcher cannot remain external to the collection and interpretation of data but is inherently bound up in the process of research.The process of collection and interpretation is cyclical, with low-level analysis of data providing grounds for simple levels of interpretation, which then provide the basis for reinterpretation of existing data and collection of additional data.It is an iterative process that focuses on the subject's perspective but also accepts that the researchers cannot approach the subject without bias or personal viewpoints.The implication that the results will be dependent on the researcher's implicit involvement is managed through careful recording of the process of interpretation, where the researcher's initial interests, subsequent comments and possible interpretations are treated not just as information external to the analysis, but are recorded as data within the project itself.The possibility that different researchers might not agree on results is as a logical consequence of a constructivist epistemology.The rigour of any resulting 'theory' is established by the ability of other researchers to audit the stepwise sequence of interpretation, whilst recognizing that individual bias and interpretation would not necessarily be the same.The degree to which the researchers share (or should share) the same values and frameworks will determine the extent to which different researchers would accept the same conclusions.
The development of a theory is cyclical too, using existing data to build a theory and using the developing theory to guide data collection, increasing the depth of interpretation and the coverage of the theory across subjects.The first stage is to generate a catalogue of the terms and concepts used by the subjects (termed 'open coding')-This catalogue is then used to generate an 'axial model' (Strauss and Corbin, 1990: 96), a time-sequenced framework that establishes the interrelationships between the concepts within the open coding.This develops the interpretation of the subjects' perception of the problems that were being solved, for example the conditions controlling what was possible, the choices that were open to them and their assessment of what was successful and what was not.When this appears to be effective in structuring the data collected this model is then summarized by the identification of a single bounding concept, termed the 'core category'.This core category encompasses the framework within which all of the subjects' perceptions can be included.In reality there is a constant cycle between the last two stages; the intention is to develop a theory for which the core category provides a summary that allows all the data to be interpreted within the axial model and hence the core category.
The axial model and core category are validated as a theory through a recursive process termed 'theoretical sampling'.In this phase concepts and issues that are already relevant in the current axial model are used to search for new cases to be integrated.The new data that is collected may well introduce new concepts or variables that enrich the open coding stage, provide changes and complexity to the axial model or even require a reconsideration of the core category.When this process stops, a process of uncertain length, the research is considered to be complete and the model is said to have reached 'theoretical saturation'.
The outline of the process suggests that existing literature in the field is ignored but this would be illogical if the theory is, itself, to have predictive value.Concepts and models from literature are treated here as frameworks that have been created by other stakeholders outside the immediate focus of the research.There is no direct implication that they should apply to the conceptual framework used by subjects.Existing literature is treated as a further source of material to be used in developing theoretical saturation.Awareness of possible frameworks from the literature, termed 'theoretical sensitivity', allows possible issues to suggest areas for data collection, as long as there is sufficient evidence from what has been collected already.

The first phase
This paper covers the first stages of the analysis: the open coding, early axial coding and considerations for the core category.Whilst the research process is not complete, the convergence of the cyclical method to a stable theory offers the opportunity to comment on aspects of the developing theory at a stage at which those aspects become sufficiently well defined to be of significance.
In this case the requirement to establish a definitive theory must be balanced against the potential utility of that information to stakeholders.Even at this stage of the research, it is clear that the students' conceptions of an LMS are categorically different from those held by the other stakeholders in the system.A delay in publication to develop and refine a 'theory' will have little value if decision-makers proceed to implement large-scale systems unaware that their expectations of how students will use the system is far from reality.

Data collection
The particular method for generating data was derived from an earlier qualitative study (Bliss and Ogborn, 1977) that used semi-structured interviews to collect and analyse the experiences of first-year physics undergraduates.
Each session was carried out during time allocated to a normal lecture or workshop.An introduction was given to the purpose of the research and why the students had been chosen.The method was described as a technique for collecting their approaches to using the LMS that was designed to avoid interference from the views of other stakeholders in the university's system.Students were given details of the process by which their contributions would remain anonymous and were allowed the option of withdrawing from the process.Students were then asked to write an account, with as much detail as they could remember, of the occasion on which the use of one particular LMS had been 'the most rewarding educational experience'.As they completed their own individual account they were asked to summarize this account with a single word or short phrase.
As four or five students finished they were formed into a group and completed the first phase by reviewing, without discussion, the accounts of the other students in the group, summarizing each in turn.They then discussed the accounts between themselves to agree on a common word or phrase to summarize the set of accounts within the group.This is less constrained than a semi-structured interview, but earlier experiments with a similar topic by one of the researchers has demonstrated that it can produce remarkable consistency between individuals and groups.Each of the groups then repeated the cycle with a different starter question -the occasion on which the use of one particular LMS had been 'the worst educational experience'.In general the method worked as anticipated, though not all students confined themselves to single examples.
If time allowed, students were then asked to contribute in two further ways to building the model.The first asked for a specific comparison of the different interfaces.The second asked them to identify any issues to which they or their group had failed to refer but which might indicate key issues that should be included in the model within the data collection process.
This open structure to the collection of qualitative data was designed to elicit responses that would be framed within language understood by other students.Descriptions written directly for us could be filtered or adapted to make them appropriate for lecturers.Requesting a narrative account supported a simpler transfer to an axial model.The request for individual summaries checks that the accounts make sense to the students and, with a collective discussion, provides a starting point for developing a core category rooted in the subjects' perceptions.The intention was for students to produce an account that was acceptable or understandable by the student community.
The data collection process was supplemented with field notes from the researchers throughout the process.It is not possible to collect data and interpret it in separate phases, as interpretation of what is collected cannot be 'turned off': for example the comment 'not exclusively about the technology' was recorded after the first session of data collection even before transcription began.Contemporaneous notes on ideas that occur provide a mechanism for identifying these interpretations and, by recognizing the source, of tracking and auditing the influence of individual bias on the research.Notes are also kept on any _ extraneous issues affecting data collection.In the first session, for example, it was noted that the offer of tea and coffee to students appeared to remove tension.
The discussion below focuses on the language used by the students in these accounts.Issues of 'theoretical sensitivity' were excluded at this stage.

Open coding
The analysis of the full sample collected so far (49 accounts) was conducted using the software package Nudist 4. This provided effective facilities for tracking the progressive searching and coding of accounts, the sorting and resorting of coding groups along with the parallel recording of coding notes.Some of the students followed closely the request to describe a single event with some detail whilst others provided wider but shallower accounts.A typical account (transcribed with minor changes for legibility but with original spellings and expressions recorded) is presented in Figure 1.
When using blackboard, groups can be set up so you can communicate with people in group work.This involves talking to the people directly using chat or emailing them, but also being able to exchange files.The consistency within the summaries of the account in Figure 1 was repeated in almost all cases and gave a clear indication that the individual accounts were well understood within the student community (see Figure 2 for an additional example).The consistency of language and terminology used by the students in their individual accounts also provided a strong indication that the particular model of data collection was working effectively.
Even the simplest reading of the accounts reveals a terminology and language focused on being able to (or alternatively failing to) access resources in a remote-access mode when the students needed to or preferred to.One researcher specifically notes at this stage: 'stories seem to be about flexibility and choice -students are working in semi-distance mode.' Within this common theme, the different resources available seem equally significant (for example, lecture notes, assignment details, etc.), and the 'reasons' for access were varied.

Coding and notes
Interpretation (and recording of notes) inevitably occurs as soon as a researcher has access to the data (as with the comment above).Open coding progresses from that point through a structured approach, interleaving analysis and coding of the data collected with recording of notes and commentary.
Interpretative comments were attributed to a specific point in the analysis and elaborated using explicit terminology from the cases (for example, 'several students use "choice" words -need, have to, useful, unable, wasting, access (choice), repair (missing lectures)').This generated, in turn, a further analysis of the cases -for example a search through all the cases for examples where that specific terminology is used and extending the classification to a wider set of cases than those that triggered the initial comment.It was then possible to re-examine these cases in more detail, or juxtapose the new coding with a previous classification -or to review the cases where 'choice' did not seem to play a role.
Absence of specific language, where the researcher's personal expectation was not met in reviewing text, was also considered a valid starting point for search, such as, 'No student mentions learning specifically -why?' In such cases searches with no results were also recorded (as with: 'August 1st 2001 searched for "learning" nothing found; August 15th 2001 searched for "learn*" nothing found').
At this stage the study was most remarkable for the absence of significant categories of language.Little comment or reference was made on specific issues that were assumed to be important in the original design, either interface issues or the relevance of technical skills.Remote access, the result of a policy decision at Kingston University, overrode any other consideration of implementation or design.

What the data did not say
As the students' accounts failed to produce anticipated commentary on either aspect of usability, careful consideration was given to whether this resulted from the design itself.As commented above, the positive and negative accounts (described in what follows as 'open' accounts), omitted almost all terms or issues relating directly to human-computer interface design; only one, triggered by the 'worst experience' question, could be interpreted as a lack of closure (commenting on a lack of confirmation after an assignment was delivered electronically through Blackboard).The extent to which this could be a consequence of the questions that were asked was a consideration but was rejected after analysis of some of the additional comments that were requested.When asked for comments comparing the two systems, some of the students readily demonstrated an understanding of humancomputer interface issues (see Figure 3).
Blackboard is much better than WebCT, it has the modules separate in Buttons, calendar to show the submission date for the assignments or Bb [Blackboard] better than webct -visually a mess -Bb very ordered Clearly students are aware of the impact of an interface and can express preferences.There is, of course, indirect evidence from their accounts that they did use the interfaces successfully.
The disregard of specific interface issues within the open accounts provides strong evidence that changes in interface design are of low priority for the students at this stage in the use of the LMS.Arguing that the research process has failed to capture the relevance of the interface would be to devalue what the students choose to report -apparently devaluing a comment if it is not within a researcher's chosen category.
The second aspect of usability, understanding the relationship between the students' technical abilities and their use of an LMS, was also only reflected to a minor extent in the accounts and is not yet sufficient to feature in our models.The justification for this position follows the same structure as before but is too detailed to present in this paper The discussion of these two specific issues does not imply that technical ability and interface design do not have an influence on the usability of the system -such a view would clearly contradict evidence from the general field of HCI design.What is called into question is the relative significance of these issues, from the students' point of view.
The open coding reveals a significant divergence in the issues and language used by the students, compared with the language anticipated in the original design of the research (usability) or the researchers' own personal interests in the potential of technology to improve the learning process.The openness of the research process to capturing students' views has redefined the focus of this research to what appears critical to the students, the decision process by which they choose to exploit the facilities offered by an LMS.
To view this as a weakness in the research design would be to ignore what is fundamental to the principles of Grounded Theory methodology.The immunity of the data collection process to the expectations of the researchers is valuable evidence of a well 'grounded' study..This divergence allows the range of subjects available to the study to be increased and studied over a longer period of time, but does not imply a change in methodology.Indeed, the effectiveness of a fully 'grounded' approach to capture the views of users of an LMS seems clearly established.

Axial model
A storyline is constructed to provide an account of how the individual in each case perceives the problems that they encounter.This includes the causal conditions that led to the problem, the particular context of each problem, the intervening events and behaviours that provide a dynamic sense to the case, the strategies applied by the individuals themselves and the eventual outcomes (Strauss and Corbin, 1990: 61-2).
One possible formulation of the account from Figure 1, represented here without a strict typology of concepts for simplicity, could be as in Figure 4.With further analysis of similar accounts a model is developed that provides more subtlety to the analysis and an understanding of the consequences that result when options fail.Figure 5, for example, includes an assessment to consider the possibility that a virtual meeting is not sufficient.
Even without a more rigorous approach there are evident gaps in the network that need to be explored (theoretical sampling).What, for example, determines the choice of mode for a virtual meeting?Or what could be the consequence, to match with 'avoid coming into university', when a real meeting cannot be organized?
Completing an axial model to integrate all student accounts in the same, single model is the key step in 'understanding' the students' behaviour.There will be contexts in which students have little opportunity to make decisions that affect the consequences for themselves.Equally there will be contexts and circumstances when students can make choices that they anticipate will produce greater benefits in their own terms.
Understanding the relationships between these circumstances, actions and benefits helps us to understand why students perceive some patterns of behaviour as the rational pattern to choose.Understanding these relationships can, ideally, allow the university to make _ changes to the context or circumstances in which the students find themselves, so that they change their decision.

Core category
A core category should provide an overriding conceptual framework within which the decisions taken in each case can appear rational to the individuals (Strauss and Corbin, 1990:116).
The terms that were identified by students as summaries in the positive accounts were intended to provide a catalogue of concepts that could be considered as the candidate phenomena for a core category, with reinforcing evidence taken from the summaries of the negative accounts.This has not directly applied to the accounts that have been collected.
Within the positive accounts the concept of 'accessibility' (including variations such as working from home or remote access) was most consistently used and almost all the negative accounts reflected failures of the system to support this.The summary terms used by the students appear to be chosen in terms of their instrumental value, identifying the significant change in circumstances that has most influence on outcome (as also with 'communication' in the example used above).
Remote access did not confer direct benefits to the students.It was the way in which that access is exploited that was of benefit.This suggested that the summaries should primarily be treated as intervening conditions rather than as possible core categories.Where the system works as students expect, the benefits cover a wide range of issues: avoiding journeys into university (as in the example), the direct saving of time, starting academic work sooner, catching up on study when behind, being aware of new and relevant information, etc.It was these stated benefits that became potential core categories.
The benefits themselves, as taken from the accounts of the students, were too varied to provide a single 'core phenomenon' and generally could not be expected to show such consistency.A stated benefit, such as 'avoiding travel to the university', could be stated in a more general or specific form or be seen to have direct consequences that occur in other accounts, such as 'saving time' or 'having more time to work'.
The development of an axial model and core category is not yet complete.It is clear that the relationships between the benefits can be constructed into a limited set of possible lattice structures which closely limit the range of what could act as a core category (see Figure 6, for example).Strauss and Corbin (1990) suggest that it is more effective to produce alternative models where the balance between more than one type of benefit occurs, but this does not appear to be the case in this study.Please note that some of the connections may not be obvious -this is a representation of the students' perceptions and not our own interpretation.As would be predicted from the minimal impact of usability in open coding, the network above makes little reference to either technical skills or to aspects of interface design.The actions and approaches to comparing alternative solutions are functional in terms of life management and ignore educational issues of effectiveness, learning style, etc.Even where educational terms are used (such as revision, catching up) the benefits are realized in more functional life-style gains.
An initial reaction to the students' use (or non-use) of the LMS might be to suggest that students are merely using it as a passive filing or messaging system, but this would fail to reflect that the students are actively engaged in using (or attempting to use) the different systems to manage their learning.Remote access, rather than on-campus access, provides them with a choice of how and where they can study.
It is interesting to note from this analysis that these students have provided a more critical assessment than most students who work within a full distance-learning framework.These students choose to meet their objectives by working remotely, by following a traditional model of full attendance at the university, or by managing a combination of both.From the evidence in the accounts there is no simple solution; costs, benefits and reliability, rated in a number of different ways, have to be weighed against each other.

Continuing research
This study is now continuing within the revised focus identified by the students.With the continuing promotion of the LMS across the university, the range of courses to be included covers six faculties with some cohorts of more than 400.Changes in the LMS software and its use by staff should not invalidate the existing findings -it should provide access to actual cases rather than hypothetical cases in the progress towards theoretical saturation.
The overall consistency in the accounts that have been collected will now support consideration of 'theoretical sensitivity' to provide additional direction to this phase of the study.Two aspects will need to be considered in introducing existing research, the issues that are considered and the findings themselves.
Issues that are of potential interest can be identified across a range of studies.Bacsich, Ash, Boniwell, Kaplan, Mardell and Caven-Atack (1999) identify the same financial issues but suggest (p.47) that students are not aware of the savings.Our study suggests that students are actively balancing the finance and time in how they choose to study.The 4E model (Collis and Pals, 2000;Sikkel, Gommer and van der Veen, 2000) also provides a useful summary of issues.Although the model includes four 'dimensions', the resulting measures are not at the same level of granularity as used by students in the current study.
The lack of similarity in findings, as opposed to issues, is unsurprising in view of the nature of the research.Both survey techniques and mathematical modelling will cover the same aspects of a problem but produce results on a different scale that are relevant to other stakeholders but not to individual students.
More challenging research comes, naturally, from Activity Theory (AT) (Nardi, 1996a), since the epistemological model is consistent, even if Grounded Theory is considered by those in AT to lack 'underlying principles' (Nardi, 1996b: 236).There are clear parallels between the fundamental concepts of AT and the structure of an axial model.The difference arises, of course, in whether accounts of the students must fit with AT, or whether they actually do.Assuming that the accounts must fit appears to tackle the underlying principle of Grounded Theory.If the model from AT fits, we may unwittingly have provided an investigative model (at this stage at least) for researchers who start with AT as an underlying first principle.
The role for AT, perhaps more specifically as functional pragmatics (Tetscher, Meyer, Wodack and Vetter, 2000), will be to provide clarity to the potential meanings of goals, aims and actions where the narrative accounts appear unclear and where further discussion with the subjects is not possible.In this context there are other theoretical approaches for text analysis that claim equivalent respectability, even if from different principles.provide a useful summary with particular attention paid to underlying principles.

Conclusions
The research from this study is published at this early stage for three reasons that are as relevant, or more relevant, at this stage as they will be at the end of the process.
Firstly, a research process that set out to be grounded in the perceptions of the student community has demonstrated, by confounding any expectations of the researchers, that it is able to achieve that objective.
Secondly, a grounded approach has revealed that the underlying decision process used by students to determine how they use an LMS is considerably more complex than reported in positivist accounts.The categories used, and level of granularity with which students consider them, are not those that arise within the design of most educational technology.
Students make case-by-case decisions whether to replace existing university-based activities with equivalent activities based off-campus.If an LMS is to support change in student behaviour, the models used by designers must change.Such models must include an operational model of how students currently evaluate their choices in learning at university.It must also ensure that the educational benefits that are anticipated translate into benefits that can be recognized by the students.
Thirdly, decisions that extend the use of an LMS in further and higher education are being taken daily, in a process that appears unstoppable.It is critical that we understand the impact that these changes will have on students' learning before the possibility of revoking any decision is overridden by the scale over which the change would need to take place.This argument appears similar to Patton's utilization-focused evaluation (Oliver, 2000) but there is a potential conflict between the design of this research and the values of the stakeholders who take the decision.Will those who promote the use of an LMS be prepared to accept research that suggests that their models are not valid?We hope so! Entwistle provides a detailed analysis of some approaches to designing courses to 'encourage student learning directed towards personal understanding' (Entwistle, 2000:1).
The models and constructs that he reviews are significantly different from those that are used by these students.If we are to include both sets of models into the design of online courses then we have much to learn about how students achieve this.The answers on this aspect, however, must depend on the further stages in the research.

Figure 2 :
Figure 2: A further example of a student account

Figure 3 :
Figure 3: Examples of students' comments addressing human<omputer interface issues

Figure 4 :
Figure 4: A formulation of the account in Figure I

Figure 5 :
Figure 5: A formulation of the account incorporating an assessment of sufficiency Figure 6: A possible lattice structure This is very useful especially as people work in different places and times to others.It also enables people to communicate easily without having to find the ID numbers.It also means people can continue to develop work without actually having to meet up to exchange files, which is very useful for people who don't need to come into the building.
Personal summary: communication with others Summaries of this account by other members of group: communication facilities, easy communication, communication tool Group summary (combining all four accounts): communicationFigure I: A typical student account Perhaps the best experience of blackboard I have had is that I can get my lecture notes /course materials from home (internet) which has been useful when I have not managed to get a print-out of the materials when in university.This remote access proved useful when I needed some lecture notes to do a piece of work and I was able to get hold of, and print the course materials out at home thus meaning I didn't have to go into university to do it Personal Summary: Access from home Summaries of this account by other members of group: Remote access of notes, copy and print the notes, lecture notes Group summary (combining all four accounts): communication