Costing the Lifecycle of Networked Learning: Documenting the Costs from Conception to Evaluation

This paper is based on the lifecycle development work done by the authors as part of the JISC funded "Costs of Networked Learning" project which originally ran over a six month period (January to June 1999) under the direction of Professor Paul Bacsich at Sheffield


Introduction
This paper is based on a six month research study funded by JISC, the Joint Information Systems Committee of the UK Funding Councils. The study aimed to produce a Planning Document and Financial Schema through which the full costs of Networked Learning could be documented, thus bringing to light the hidden or unrecorded costs involved. This paper will concentrate on the development and consequent evolution of the lifecycle model developed by the research team for inclusion in the final study report. The paper falls neatly into four sections detailing: the original five-stage model arrived at during the literature review; the testing and development of this model; the evolution of the final model and lastly an example of its usage using two fictitious scenarios.
The views expressed in this paper are those of the authors and the study team, who wish to acknowledge the support of the JISC and Sheffield Hallam University. More details about the project in general can be found on the web site at http://www.shu.ac.uk/virtual_campus/cnl. The web site also hosts an order form -activated once the report has been published -copies of conference papers and presentations, and links to other projects in the field.

Early Methodology
The study team conducted an extensive literature review during the first phase of the project. The aim of the review was to update and extend the understanding of the team in the field of costing of learning (including Distance Learning and Networked Learning) and to extract "cost items" as mentioned by other authors. Over 100 cost items were identified during the project, many of which do not usually exist as University cost codes and can therefore be regarded as hidden costs which are normally subsumed into other, possibly unrelated, budgets. (A full list of these cost items is contained within an Annex to the project report.) Part of the project brief was to build a framework within which all of these cost items could be documented. Therefore the team analysed currently available models which attempted to provide a framework or methodology for recording the costs of technology-enhanced learning systems. The team had already discounted traditional financial management accounting procedures as inappropriate for exposing the full range of the hidden costs of Networked Learning, but three traditional accounting insights were useful: • Distinct organisations have separate budgets -thus in particular University-borne costs are distinct from student-borne costs.
• There are fundamental accounting distinctions between people and things -staff and nonstaff -and between capital and recurrent costs.
• Organisations have different kinds of people -in Universities, academic and nonacademic staff.
In addition, one modern insight is fundamental: • People do different kinds of tasks, and the balance of their tasks changes even within the accounting period -this is the basis of Activity-Based Costing.
Using these four basic insights, we embarked on an analysis of the best of the traditional educational costing literature. Orivel (1987) states that the most common classifications of costs associated with educational media are: administration costs; production costs; diffusion costs; and reception centres. In our view this mixes activities with standard accounting categories. Rumble (1989) believes costs can be classified by type as: human resource costs; costs of developing, producing and delivering; capital equipment costs; consumables and expenses; and space and accommodation costs. In our view Rumble blurs the boundaries between staff and non-staff costs and capital and recurrent costs. It seems that Rumble is addressing the issue from a University planning viewpoint (which will have its place in our analysis) rather than taking a University accounting viewpoint. Crabb (1990) used headings of: human resources; premises-related costs; equipment; consumables and expenses; central resources; and overheads; in the two stages of development and delivery costs. In our view Crabb presents a coherent accounting framework on which to build. Cukier (1997) follows in similar vein to Rumble by categorising the costs of educational technology as: human resources; general administration; development; production and delivery; capital equipment and start-up; consumables and expenses; and space and accommodation. In our view Cukier mixes fixed and variable costs and capital and recurring costs. Moonen (1997) summarises costs as: personnel costs; equipment costs; facilities costs; material costs and other costs, calculated in a 'costs per activity phase' breakdown of a development phase, and a delivery, operation and maintenance phase. Moonen has clearly separated costs into terms understandable to finance administrators and also produced a lifecycle model. However, it is clear from a more detailed reading that he is considering the lifecycle of the course not that of its supporting infrastructure.
Having looked at a number of such analyses, the study team decided that there were serious omissions when looking at all the current frameworks with a view to making apparent hidden and unrecorded costs: • Students are not considered, in terms of their income or expenditure.
• Staff are not considered, except as employees of the Institution.
• Courses grow from nowhere and vanish into oblivion -the models do not take planning, evaluation or maintenance into account.
• There is no obvious connection between the learning module and the environment in which it is studied.
All the models appeared inadequate when analysed against such criteria. Consequently the project team decided that they were implausible in terms of documenting hidden or unrecorded costs and/or unsuited to the UK Higher Education sector. Therefore a number of short consultative meetings were held with select academics at the University about the lifecycle of course development and delivery and also where staff believed the greatest hidden costs to be.
Based on these discussions, the varying experience of the study team in different areas, and the desire to illustrate the relationship between people and activities and therefore expose possible areas of hidden costs, a new "five-phase, three stakeholder" working model was created. This was in diagrammatic rather than spreadsheet form, as shown below.

Testing and Development
The five-phase model was subjected to a two-stage testing process. In the first stage participants at interviews conducted during seven Institutional case studies (see below for more details) were questioned about various aspects of the model. In the second stage, a small workshop was held to instigate discussion between a number of experts in the area (both in costing and in online learning) from the educational and the industrial training sectors. The purpose of such thorough testing was to assess the viability of the model in a sector which is traditionally antipathetic to imposed methods of working -the model needs to resonate with academic working reality rather than impose an impractical straight-jacket.

Test Group One
One of the main components of the study was to assess the extent of Networked Learning taking place in UK Higher Education Institutions and how these activities were costed, if at all. Seven Institutions were selected (on the basis of their replies to a sector-wide survey) to "represent" the sector, using such criteria as research or teaching bias, geographical split and Students current status of Networked Learning activities. Teams of interviewers visited the Institutions over a two-day period and interviewed academics, administrators and senior management, often receiving in addition a tour of the facilities and activities taking place. (Full details of this exercise can be found in the study report.) Over fifty interviewees were shown a copy of the working model and asked to comment on several aspects.
One clear point was that of omissions. Staff in Institutions thought that Senior Management (Deans and above) and Evaluation should be included in the working model. Others thought that more general issues such as Flexibility and Sustainability should be included. Still others believed that additional stakeholders such as Employers and Parents should be incorporated.
Yet on the whole, the response to the working model was positive. Staff were pleased to see Course Maintenance and Support Staff being properly recognised in an educational setting. The "lifecycle" approach to the issue was also commended. One interviewee expressed the view that simple methods would have more impact on non-financially aware academics; however another considered that even in its simplest form the model would need serious staff training and "hearts and minds buy-in" at a grassroots level in order to succeed.
Some academic staff interviewed were concerned that any framework of this nature would be too rigid, since organisational change is slower than the pace of technology. Some others believed that scenarios would be a good way of illustrating its function. On balance, on further analysis it seems as though most academics are negative about the very idea of a framework to document the costs of Networked Learning, while senior management were in favour of a methodology which would help record the costs of any learning system.
Much concern was expressed about whether such a model could stifle creativity and innovation, thus hindering the development of Networked Learning.

Test Group Two -Experts' Workshop
In addition a workshop was held at Sheffield Hallam University which was attended by several prominent educators and financial experts. The aim of the workshop was to facilitate discussion on costing Networked Learning and to explore ideas which would assist the study team to develop the project outcomes.
The experts agreed that the model -in whatever form it finally takes -should be equally as appropriate for costing conventional teaching systems as for costing Networked Learning. If the model did not also apply for alternative systems then fair comparisons could not be made and the sector would be no closer to determining whether and in what circumstances Networked Learning is more cost-effective than conventional teaching and learning (an issue which was agreed to be more important than the issue of whether Networked Learning costs less in absolute terms).
Strategic Planning was notably missing, but on discussion it was resolved to take place outside of the existing model. Quality Assurance and Evaluation were also noted to be missing and it was decided that these two aspects definitely needed inclusion in the main model.
Further discussion on the model revealed that the industry-based experts were actually not comfortable with the five-phase working model. They proposed a helical four-phase Business Process Re-engineering (BPR) model which revolved through the following phases: Develop, Market, Provide, and Assure. However, when the identified costs were categorised under these headings it was found that some categories were quite sparse (Assure) while others (Provide) would need to be broken down into more than one subcategory. Given that the aim of the exercise is to categorise costs in a way understandable to policy makers, academics and financial managers -and also to do this at a level that does not allow for the hiding of hidden costs -the four-phase suggestion did not work, and therefore was rejected as unsuitable for this purpose.

The Final Model
Both test groups noted that there were a number of omissions from the working model, including Evaluation, Quality Assurance, and Strategic Planning. Another vital finding to arise from this testing process was that simple methods have more resonance with academic staff especially when they impact on costs, time and workload.
The team decided to accept the Experts' Workshop view that the original five-phase model was too complex, but to reject the industry experts' four-phase BPR model. Therefore a new three-phase model was proposed.
The new model includes Evaluation and Quality Assurance but assumes that Strategic Planning takes place on the periphery. (One could argue that this is a valid assumption from the viewpoint of the "working academic".) Staff and Students have been removed from the model, since the study team decided, after much research and discussion with colleagues, that their place is secure now that the stakeholder analysis approach is becoming more accepted in educational circles. The three-phase model is shown below.
The three-phase model follows classic course planning frameworks from the distance education sector but also incorporates the increasingly accepted need for Quality Assurance and Course Maintenance.
The evolution from the five-phase to the three-phase model is shown below: the student). In his later opus Rumble includes the costs incurred by academic staff and student -a serious omission in his earlier work, and a key component when documenting hidden costs. Orivel (1987) reported on Production costs; Diffusion costs; and Reception centres, all three of which fall into the second phase of the model as all being dependent on student numbers. Crabb (1990) used a two-phase Development and Delivery model, which neglects Planning, Production (perhaps), Maintenance and Evaluation. Cukier (1997) follows Rumble with phases: Development; Production and Delivery. We would add Planning to Development and call Production and Delivery one phase, but we would highlight the omission of any form of Evaluation or Maintenance. Moonen (1997) distinguishes two phases: a Development phase and a Production, Delivery, Operation and Maintenance phase. We would split his phase 2 into two parts, leaving Maintenance in phase 3.

5-phase 4-phase 3-phase
It became more apparent in the later stages of our study project that work on costing in the training sector was closely related to the Activity-Based Costing systems we were working towards. However, although they were more in touch with costs incurred by staff and students and the growing opportunity cost of activities, their course models proved no more comprehensive than those above. Stahmer (1995) proposes a model of course development comprising three phases: Research and Planning; Development; Delivery. In this instance we would group 1 and 2 together in our "Planning and Development" phase. Hunt and Clarke (1997) have a four-phase model: Research and Development; Initial Investment; Operation and Support; Disposal and Salvage. We would group 1 and 2 together. Their phase 4 is rather odd, and hard at first sight to equate with our "Maintenance and Evaluation". One assumes they were thinking of courses delivered primarily by CD-ROM.
The table below contains a summary showing how the most significant models compare.

Production & Delivery
Make more material available on Web, such as topical items on Philosophy, moderate discussion groups online, receive and mark essays sent in by e-mail. Set up "real" office hours for those students who live nearby.

Maintenance & Evaluation
Students want some "synchronous" online events; so must get technician to find out about RealAudio and record some lectures for next year. Also worried about the new OU global course in this area; how can she differentiate her course? (What about her students at the BBC?) Can she write something about this in a journal and count it for the next RAE?

Issues
We would not contend that all problems are solved with our model. At present we have tried to ensure that all aspects of a course whose costs depend overtly and substantially on student numbers are mapped into the "Production and Delivery" phase. This includes Assessment. We note the argument that Assessment is not part of the "teaching" of a course (and indeed in some work planning models used in Institutions, time for assessment is not explicitly counted) -however, there is a counter-argument that Assessment, or at least formative assessment, is part of the teaching, or at least the learning; and that is the view of this paper.

Conclusions
In project terms the lifecycle model was developed to assist Institutions who do not have their own model but would like to use our proposed Planning Document and Financial Schema. In the wider context it illustrates the lack of consensus in the sector about the best way to pursue course development. Adequate costing throughout the course lifecycle, and realistic attribution of overhead costs, are needed in Universities today to inform resource planning decisions and aid the effective management of organisational change.