Computer-simulated experiments and computer games : a method of design analysis

This paper describes a new research programme to design computer-simulated experiments in the field of fuels and combustion, and describes a method of categorization based on a taxonomy proposed by Gredler. The key features which enhance science content and process skills are identified The simulations are designed to be as realistic as possible, and are built using threedimensional computer-aided design, rendering and animation tools, with the intention of creating an interactive virtual laboratory on the computer screen. A number of computer games are also categorized against the computer simulations and the same taxonomy for comparison. The paper then describes how designers of computer simulations can add to their own learning by retrospectively analysing their own simulations.


Background
Through the new modularization of the undergraduate science degree at the University of Brighton, larger numbers of students are choosing to take some science modules which include an amount of laboratory practical work.Indeed, within energy studies, the fuels and combustion module, for which the computer simulations were written, has seen a fourfold increase in student numbers from twelve to around fifty.Fitting out additional laboratories with new equipment to accommodate this increase presented problems: the laboratory space did not exist; fitting out the laboratories with new equipment would involve a relatively large capital spend per student for equipment that would be used infrequently; and, because some of the experiments use inflammable liquids and gases, additional staff would be needed for laboratory supervision.
However, the option of running a science-based module such as fuels and combustion with students doing fewer practicáis was not desirable from an educational view, since experimentation plays a key role in enabling students to develop science process skills (Gibbs & Jaques, 1990), and a reduction in the amount of experimentation would have a consequential effect on students developing those process skills.The necessity of teaching the maximum amount of science content and science process in a cost-effective manner leads to consideration of computer simulations as a supplemental means of instruction.Indeed, the opportunity was seen that the total number of experiments that an individual student performed could be increased with the use of computer simulations.
Much research has already confirmed that simulations are no less a viable means of instruction than any other form of instructional approach (for example, Hedlund et al, 1986;Sherwood et al, 1984), and in fact studies have shown that students achieved as well, if not better, on the post tests through computer simulations.Magín and Reizes (1990) suggest there is considerable unexploited potential for computer simulations in education in the field of laboratory experiments, but in reality there is little computer simulation software available for undergraduate teaching in the field of science.In 1992, this dearth lead to the instigation of a research programme by the author of this paper in computer simulations within the University (Leary, 1994).The latest three-dimensional computeraided design, rendering and animation tools would give the experiments a sense of realism.

The experiments
It was initially planned to develop eight computer-simulated experiments in the field of fuels and combustion.This phase is now complete and the simulations are currently being tested on second-year energy studies students.The students use the simulations to: • determine the gross and net calorific values of natural gas using a Boy's Calorimeter; • determine the upper and lower flash points of liquid hydrocarbon fuels using the Pensky Martens apparatus; • measure the relative density of liquid hydrocarbon fuels by the balance method and with hydrometers; • determine the viscosity of liquid hydrocarbon fuels by using a Redwood Viscometer; • determine the fractions of light hydrocarbons in gasoline using a distillation column; • measure the pollutant emissions from a Wankel engine; • determine the constituents of coal and efficiency of combustion by measuring the products of combustion in a fluidized bed boiler; • investigate diesel-engine efficiency and diesel knock by varying fuel injection profiles.

Classification of simulations
Before undertaking any redesign or refining of the initial eight computer-simulated experiments, or indeed developing any further simulations, it was felt necessary to perform an analysis to identify the factors involved in optimizing the success of the learning outcome and, thereby, add to the designer's own learning.As a comparison, some popular computer games would also be analysed on the same basis.The model chosen for the analysis was Gredler's taxonomy (Gredler, 1986) of computer simulations which identified four categories of simulation: 1. Structured questions with associated graphics.The learner views the simulation, then answers specific questions.These can be either drill-and-practice or tutorial type exercises.
2. Assignment of variables with results exercises.One or more independent variables are set by the learner and the results on the dependent variable or variables are observed through several trials.The nature of this simulation is cyclic, feedback being related to variables affecting other variables.
3. Diagnostic simulations.This type of simulation usually involves applied decisionmaking in a professional content area leading to the development of 'expert knowledge'.
A decision variable is selected by the learner after several episodes, and there is usually one large scenario.
4. Group-interactive.This is applied decision-making in a real social context.Distinguishing the correct approach in context is the principal learning outcome.There is no right or wrong answer in this case, and the results of decisions are difficult to determine.None of the laboratory experiments falls into this category, so the category is not considered further.

Computer games
The following computer games were chosen arbitrarily on the basis of being commercially successful and representing a cross-section: Anal/sis of the taxonomy Table 1 identifies the fit between the eight computer-simulated experiments described above, the four computer games, and the taxonomy proposed by Gredler (1986).The descriptions of type, or categories, are listed on the left, and the shaded boxes identify those items which match each of the four classifications.If there is a match between a computer simulation or a computer game and a category, a star is used to indicate this.Then, in looking at the data in each of the columns, it is a matter of identifying a match between each column and each of the columns with shaded boxes.If the'pattern is the same, the category of simulation is clearly identified as being either of the four types.The taxonomy splits the category-structured questions with associated graphics into two other categories: drill and practice, and tutorial.
The conclusions that can be drawn from the analysis of the simulations in the table are that: • there is a weak but marked correlation between both the tutorial and variable assignment categories and the majority of the simulations;  • there is strong use of visuals in both demonstration of items and presentation of the running experiments; • the simulations provide only weak feedback and do not identify errors in the student's selections; • simulation number 3 shows marked characteristics of the drill-and-practice class; • the nature of the instructional process of simulations 7 and 8 is akin to the diagnostic class.This analysis provides an extremely useful agenda for further design work on the simulations with the issue of weak feedback heading the list.It also provides an insight into the timing of the use of the simulations within the fuels and combustion module as a whole, as it is apparent from the taxonomy that simulations 1, 7 and 8 are partly diagnostic in nature and require some prior knowledge, that simulations 2, 4, 5, and 6 are mainly variable assignment and require minimal prior knowledge, while simulation 3, as a drill-and-practice type, can be used without any prior knowledge of measuring relative density.Lastly, the analysis also provides a level of comfort as it confirms that about half of the simulations fall into the assignment of variables class, which Clariana (1989) indicates do result in improved learning of specific content and also of more generalized cognitive processes.
The games were also analysed using Gredler's taxonomy and the results are also shown in Table 1.It is quite interesting that there is a strong correlation in all cases with variable assignment and diagnostic categories.The emphasis in these games is providing an environment in which players can test their skill, be it landing an aircraft, playing ping pong, managing a city or doing battle, and although the teaching of these skills is not a primary aim of the software -commercial success is -the development of the skills does occur, and the players in all cases do improve their performance and increase their knowledge.
However, although the analysis using the taxonomy provides much useful information, it leaves unanswered some questions concerning the effectiveness of the simulations.This is because these questions primarily concern learner understanding rather than typological issues of context and feedback, and are, therefore, outside the ambit of Gredler's taxonomy.When addressing issues of learner understanding there is, unfortunately, no single reference point like Gredler's taxonomy, and a comprehensive view of the issues is gained only by studying relevant literature, some of which is discussed below.They include areas such as enhanced reality, learner guidance or learner control, level of questioning, degree of fascination, and transfer.

Further issues
Enhanced reality.The eight simulations developed for the fuels and combustion module are, thanks to the latest 3D graphics tools, quite realistic, and this is borne out by student feedback.Realism is considered important as many of the undergraduates on the module have no prior experience of what some fuel-testing apparatus looks like.Thus, important features have been enhanced using animations and cut-away views.For example, simulation number 1, which uses a Boy's Calorimeter (Figure 1) to determine the gross and net calorific values of gaseous fuels, shows a cut-away view of the inner workings of the calorimeter, allowing the student to view the direction and magnitude of water flows, gas flows and the combustion flame.This view conveys additional information which would not be available in a conventional laboratpry.Furthermore, where possible, taskirrelevant cues have been eliminated or reduced.Again using simulation number 1 as an example, the experiment would, in reality, involve a mass of wiring, connecting thermocouples and other components to temperature meters, whereas the simulation shows only simple digital thermometers.The simulations also highlight features that may not be visible in reality, such as heat flows, and show magnified views, such as a single oil droplet burning, and slow down time so that fast processes, such as combustion occurring in a diesel engine, can be studied in detail in slow motion (Figure 2).Successful implementation of these features was a key design consideration, and the ability of the eight simulations to aid the students' understanding more effectively than traditional laboratory-based experiments is therefore deemed to be high.
Learner guidance or learner control.It has been shown, for example by Vockell & Rivers (1984), that guided discovery computer simulations are generally better than the exploratory computer simulations, particularly in the development of general problemsolving skills.This is contrary to the commonly held view that free exploratory discovery is best for learning science process, and that guided discovery is best for learning science content.In any case, simulations of a diagnostic nature require prerequisite knowledge and provide a test environment to try out that acquired know-how.In other types, it is more difficult to determine in advance exactly how much learner-guidance to include in each of the simulations, so the method for deciding this is an interactive one based on student feedback.The amount of student questioning is monitored and used to include or remove additional information.Level of questioning.This is aimed at stimulating reflection on what has occurred during each experiment.A correct level of questioning is important because reflection is needed to create a deeper understanding, and in order to develop cognitive processes in the learner.In a study by Merrill (1985), the use of higher-level questions in computer simulations was supported, but not lower-level questions where individual attributes of a concept are treated in isolation.In the simulations described in this paper, students are asked a set of higher-level questions at the end of the experimentation phase to encourage reflection.In diagnostic simulations and computer games, students are already working at a deeper level, testing and refining their mental model continuously, and so additional questioning is less appropriate.
Fascination.The level of difficulty of a simulation is important.Ideally, there should be an exact match between the learner's ability and the difficulty of the simulation to stimulate interest.If a simulation is too easy it will result in boredom and poor skill development, whereas if a simulation is too difficult it will cause anxiety in the learner and a failure to understand or complete the task.However, if there is an exact match, the learner is more likely to proceed with maximum concentration, or even become fascinated with the task, at which point learning occurs with consummate ease (Mayhew 1994) as the student becomes addicted.With computer simulations, it is much easier to match the difficulty of the task with the level of competence of the learner.Some computer games do this particularly well with different skill levels.With real experiments, this process is limited by physical constramts.Student feedback suggests that the level of difficulty for one or two of the experiments is too low, in particular simulation number 3. Table 1 confirms that this simulation belongs to the drill-and-practice class, which are routine in nature, and therefore we can conclude that simulation number 3 needs to be redesigned to make it more challenging.
Validation.One desirable learning outcome from any experimentation is development of skill in validation of results.Rather than trusting implicitly the output of the experiment, as students are more likely to do with computer generated results, in these experiments students are made aware that they should treat the measurements from the computer simulations with the same degree of caution as they would measurements from equivalent real experiments.Several processes are incorporated into the simulation to ensure that good results can be obtained only with careful procedure.For example, a mathematical model which is dynamic, with built-in time delays, thermal inertias and so on; a measurement variation or 'noise' added to measurements; and, thirdly, variations introduced into the parameters of the model so that it changes gradually over time in a random manner.The level of these contingencies actually built in determine the development of experimentation skills and lead to a fuller appreciation of the nature of empirical investigation.
Transfer.The above issues directly affect transfer, that is, the ability of a student to use knowledge in either a similar context (near transfer) or a different context (far transfer).This is tied in with whether the student is gaining a surface knowledge or a deeper understanding (Entwistle, 1993).To perform far transfer, the student must acquire a deeper understanding, and in general simulations of a diagnostic class are better at this than drill and practice.However, other issues are also important, for example enhancing reality, providing an optimum level of learner guidance, generating higher level questions, and making the simulations fascinating all help the student achieve a deeper understanding and obtain skills in both near and far transfer.

Conclusions
It is natural for designers of simulations to want to undertake some form of analysis of their work to identify how their own learning can be improved.More effective designers lead to more effective simulations.Educational software also has a lot to learn from computer games in their potential to support that old adage that learning can be fun.Gredler's taxonomy is a useful tool for assessing the context and feedback characteristics of simulations and comparing them to other types of software, such as computer games, and it can provide useful pointers for retrospective design work.An assessment of the effectiveness of characteristics beyond typological considerations is, however, much more difficult as there is no single reference point as yet that provides a model for comparison.The analysis of issues such as enhanced reality, learner control or program control, level of questioning, fascination, and transfer, is, therefore, judgmental and often subjective.As designers' judgement about the effectiveness of their own simulations may be clouded by their close proximity to the development work, other judgements such as the students' feedback should be taken into account.Despite the problems associated with the assessment of learner understanding, the exercise outlined in this paper encourages the designer to take a holistic view, and provides criteria which will tend to make his or her work more effective.
of discrete, specific problems Identify new examples of concepts or rules Disco ver the optimum strategy to solve a problem 1 Discover the set of values identified as optimum for several variables I Implement the knowledge acquired in a particular subject

Figure
Figure I: Boy's Calorimeter simulation

Table I :
Classification of computer simulations and computer games