A program to help students of life sciences prepare a research protocol

This paper describes a Web-based program that has been written in JavaScript to help honours-level students to design a protocol for their research project. It consists of several screens which contain text-input boxes and options, that students can select to determine what statistical test is appropriate. On each page students can obtain information concerning the particular step in the procedure for defining the research protocol. Some links to useful pages on the Internet are also provided. After the student has navigated the screens, the program writes a protocol outline, depending on the information entered by the student. The program was introduced to fourth-stage students of Nutrition and Dietetics at the beginning of the Research Project module. Students were asked to complete an anonymous questionnaire containing attitude statements after they had discussed the protocol with their supervisors. Students generally rated the usefulness of the Internet links highly and considered the program to be user-friendly, clearly laid out and visually appealing. Most students agreed that the program helped to clarify what is needed for the protocol, provoked them to think of aspects of the project, the importance of which they might have failed to realize early enough, and helped them to ask appropriate questions of their supervisors.


Introduction
A professional research protocol is a complex document that is needed prior to application for research grants and for submission to ethical committees (Food and Drug Administration, 2000).Important aspects of a research protocol include a definition of the research problem, sufficient detail about the plan of work, reasons for selecting the specified approach, and information related to how research funds will be allocated.The scientific method implies a logical derivation of conclusions by means of inference correctly drawn from reliable data.The analysis of experimental data is, therefore, a critical stage in every research study and statistical tests are required for most research in the life sciences and this should form an important part of the overall method described in the research protocol.Some reasons why research protocols are rejected by funding bodies can be instructive at the writing stage (Gordon, 2000).Some have an error in design or feasibility such that it is unlikely that the researcher would be able to draw valid conclusions from the data.Some do not contain enough detail on the measurement of variables, the way data will be analysed, or the conceptual framework or rationale guiding the research for reviewers to gauge the potential quality of the results.Sometimes there may appear to be possibilities for the existence of problems with feasibility, for example the time required for data collection, the lack of background of the researchers in the area of proposed research, or unacknowledged problems in obtaining or keeping an adequate number of research participants.This makes reviewers sceptical that the research can be conducted as proposed.Sometimes the research is technically sound but the problem being investigated is not very important or does not appear likely to advance significantly the particular field of research.Reviewers may see problems with the logic of the research or the failure to consider important alternatives to the methods proposed.A common failing in protocols is a lack of clarity about important aspects of the research, such as why certain variables are chosen to be measured, a sketchy indication of how study subjects will be chosen or where they will come from, and a vague and quite non-specific account of how data will be analysed.Lack of clarity in protocols often stems from inclusion of too many variables in the research.Applications that propose to collect data measuring a large number of variables often do poorly in peer review, especially if there are no specific hypotheses about expected group differences or a discussion of how to reduce the number of variables to a manageable number for statistical analysis.Such a proliferation of variables often indicates a lack of thoughtful selectivity about what is really important to investigate in an area.
At honours level, students have to design their own research project and write a simple research protocol to convince the supervisor that the student has sufficient understanding to design a study that is likely to be successful.It is useful practice for the possibility that the student will later be involved in actual research, to make this exercise as realistic as possible.In order to prompt the student to include the most important aspects of a research protocol, especially the statistical ones, a computer program has been developed.Because many parts of the protocol are very dependent on the actual problem and method to be used, it is not clear how they can be included in a simple fashion.Some parts of the protocol therefore have been excluded from this program.

Method
Bearing in mind the comments of reviewers of protocols it seems that the research hypothesis itself is a critical area for the student to focus on, and this is therefore the starting point in the simulation (see Figure 1 for a flow chart of the program).At this stage, it seems important to bring this part of the protocol to the student's attention by providing some explanation of what is involved and simply providing a box in which to type the hypothesis.Although this may be further developed in the final version of the protocol, a relatively small box on the screen may help to focus the student's attention better than a blank sheet of paper.The box for typing in the hypothesis is placed in the upper of these two frames, at the top of which appears a menu bar.The whole program is written in JavaScript, which permits relatively advanced programming (Wise, 1999).The data entered by the student is stored in variables that are defined in the parent window to the two lower frames so that they are not lost as the student navigates through the site.The menu bar (Hall, 2000) uses dynamic positioning and is based on a library of functions written by Isaacs (1997).Although menu bars are common in Windows programs, they are difficult to achieve in Web browsers and some are specific only for Microsoft browsers; this version was designed to work in both Netscape and Internet Explorer.The menu item 'Explain' has submenus that load pages into the lower frame to explain the need for a research hypothesis, links to sites explaining research and statistics, and the help screen that explains how to navigate through the program.It is only on this first screen that a 'File' menu item appears.This gives access to a procedure that saves all the variables in a cookie that can be retrieved later when the student comes back to the program.
Empty boxes are also provided on the subsequent two screens, with explanations about selecting the population and sampling.The students are encouraged to consider the population for which the study is intended to be valid in as brief a description as they had to provide the hypothesis, again to focus their minds.Sampling techniques can also be relatively simply described and a supervisor should be able to judge the appropriateness of the method without requiring a long description.
The next screen provides an explanation of three different types of study along with three graphical 'radio' buttons, only one of which can be chosen at a time.Although scientists are commonly thought by the lay person to undertake experimental studies in which different treatments are given, there are also two other types of research that may be involved in the work of our students.Many are involved in the design of surveys involving questionnaires, and here the main question that needs to be resolved before statistical tests can be chosen is the type of data involved.This is explained to the students on this screen and they are also provided with an explanation of the null hypothesis to remind them of the way in which results are statistically evaluated.All students have undertaken a course in research methods in the second stage to enable them to understand studies in the literature, so the protocol program is not intended to teach the students new concepts.The program writes a protocol on the screen by incorporating the information already given by the student on the previous screens and choosing an appropriate test for the type of data involved.

Research Protocol Simulation Help Screen
This program is designed to simulate the design of research.You can edit as many times as you like and design different types of research.On the first page, you can save your work and retrieve it when you next return to this page.After answering all the questions, you can automatically print a research protocol, which is not in a finalised state, but shows the steps needed in the writing of a real protocol.Use the » and « signs to move forwards and backwards, respectively and «« to return to the first page.
In order to design a research protocol there are several steps that are required: Another type of research involves the investigation of relationships between measurements.These involve correlation and as in the previous screen, the type of data, and presence or absence of known confounding factors is explained on the screen to aid the student.When confounding is present, the student is advised to use partial correlation.
The experiment is a very common research tool in the life sciences and it was necessary to construct several more screens than for the other types of research.Experiments are factorial in design, in that one or more factors will be involved, usually by altering the treatment given, but sometimes by simple inclusion in the experiment of different individuals based on some factor that classifies them, for example sex.Students can not pass this screen until they have successfully entered at least the name of one factor, and they can design experiments with up to three factors.For each factor, on the next screen, students have to decide how many levels of the factor they intend to include.For gender, this can only be two, but when the amount of a drug is to be varied, any number could be chosen; again advice about this decision is provided on screen.When a professional research protocol is written, it should be an important subject to consider how many units are needed.On the next screen, units are defined and a simple account of how the number of units can be calculated to provide sufficient power for the study is given.For honours students, this section generally provides the most complex aspect of statistical analysis that they will meet and mostly they are constrained by time and circumstances rather than being able to include the number of units they would genuinely need for a powerful study.This screen may therefore be read, but not taken too seriously by many students.It is provided for completeness and to remind the student that this issue is important, especially in the ethics of human experimentation (Altman, 1982).Nobody should be asked to undergo any but the most innocuous of procedures in student projects in the name of science if the experiment is so badly designed that is has no hope of a successful conclusion.
Prior to the screen that writes the protocol for factorial experiments, there are screens in which students are expected to consider the important issues of whether the design could benefit from blocking to reduce unexplained variation and some procedure to randomize or provide an unbiased order of treatments.Here again, students are provided with boxes in which to type a simple explanation of the procedure to be used, such as a Latin square.
The independence of data is discussed on the screen and the student has to decide whether data fulfils this requirement or not.The final screen explains the issues surrounding the normality of the data, whether it can be transformed, or whether it is best to consider it as non-parametric data.When all these issues have been resolved, the program decides what is the most appropriate statistical test and includes this decision in the research protocol.The student should then be able to read up on the test and justify why it should be used in the final written protocol.
The program was introduced to fourth-stage students of Nutrition and Dietetics at the beginning of the Research Project module and instructions were included in a handout that was used for practical classes that included advanced computing related to aspects of their project.The instructions were brief, but included an example of what a student from a previous year might have written into the boxes for her project as an example.The handout contained an evaluation questionnaire with statements asking students to rate them on a scale of 1-5 from strongly disagree to strongly agree, with neutral in the centre.
Questionnaires were anonymously returned after students had discussed the protocol with their supervisors.The results were subjected to analysis by Chi-square test to determine whether there were significant differences between answers to the questions.

Results
There were 24 students in the class and 21 returned questionnaires.One of these, however, had considered the program irrelevant to the project as shown by the answer to the first question and had not completed the other questions.Two other students also considered it irrelevant, but had nevertheless used the program and gave feedback on it, so their results are included in the analysis.Few students used the boxes for 'strongly agree' or 'strongly disagree', so they were combined with 'agree' and 'disagree', respectively, and the results are given as the ratio of answers for agree: disagree; neutral scores can be calculated by difference from 20.A Chi-square test showed that there were significant differences between the answers to each question (x^odf ^3-3, P < 0.001).The most positive feature of the program seen by the students was that it had 'links to useful material on the Internet' (17 agreed: 0 disagreed).They generally considered it 'user-friendly' (13:1) and 'clearly laid out and visually appealing' (13:2).There was also agreement, though less strong, that the program 'helps to clarify what is needed for my protocol' (14:4), 'provoked me to think of aspects of the project, the importance of which I might have failed to realize early enough' (12:4) and 'provoked me to ask appropriate questions of my supervisor about the project' (10:3).Students were less likely to consider that the program 'surprised me by selecting a statistical test that I had not considered' (7:4).There was also general disagreement with statements that the program 'is daunting because you have to fill in empty boxes' (3:8), 'appears to be an academic exercise that is not directly helpful for my work' (3:12) and 'merely confirmed that I had already chosen the right statistical test' (2:10).

Discussion
Although JavaScript is frequently considered to be a language that provides simple functions for use in Webpages, for example for sending information back to the site originating the page, it is clear that it can perform complex tasks and deserves more use in education (Wise, 1999).The present program includes a wide range of functions, including menu bars, selecting various options in the flow of ideas and selecting an appropriate statistical test.Perusal through a large bookshop shows that there are many books written specifically for students in each discipline area to teach them research methods.The authors clearly feel that it is important to use examples that will appear relevant to their students, whereas most statistical textbooks that may be sold to a wide range of students are more mathematical and provide less discipline-specific examples.Although the program was written by a nutritionist for students in that subject area, few examples in the text provided to the student are overtly nutritional and students undertake a relatively wide range of projects.These include surveys and laboratory experiments.From the comments received from the students, at least three of the four students undertaking projects in food microbiology considered the program irrelevant to their projects.Two of them had explored the program and answered the questions, but as they still did not consider it relevant, this suggests that some more steps are needed to make its relevance to microbiology clearer.In fact, when one of the students had completed the project, it was clear that she had to use an analysis of variance and had not realized that statistics was involved in microbiology and had hence not looked at the program.Students had been given an example to fill in during their practical class that had involved an experiment requiring analysis of variance, but one of the students commented that more examples including those for relationships between measurements and for questionnaires might be beneficial.The feedback therefore suggests that more subject-specific examples, including microbiological ones, might improve the usefulness of the program.
Although the number of students using the program was relatively small, some of the results were clear enough to suggest that this approach is useful to students.It may be necessary to edit it for students in different disciplines, or it might be possible if several subject specialists combined their efforts to increase its breadth by including more research possibilities.

Postscript
Several months after completing the work described in this paper it was discovered that Netscape had dropped layers from its version of JavaScript so that the latest version of the program (version 6) would no longer display the menu bar.In order to maintain the capacity of the protocol program to run in both browsers, it was necessary to rewrite it to use simpler navigation techniques.At the same time, the opportunity was taken to tidy up the program by using the facility of JavaScript to write material to each frame rather than require a large number of separate HTML files for each frame.The program is available for readers to try at http://www.rgu.ac.uk/windiets and the author will welcome comments and suggestions for future collaborative research.

Figure 2
Figure I: Flowchart of the programFigure2shows a screenshot of the opening screen of the program.The upper frame contains navigation aids employing JavaScript for selecting different parts of the notes provided by lecturers in the course and other interactive material.The research protocol program appears in the lower two frames.The box for typing in the hypothesis is placed in the upper of these two frames, at the top of which appears a menu bar.The whole program is written in JavaScript, which permits relatively advanced programming(Wise, 1999).The data entered by the student is stored in variables that are defined in the parent window to the two lower frames so that they are not lost as the student navigates through the site.The menu bar(Hall, 2000) uses dynamic positioning and is based on a library of functions written byIsaacs (1997).Although menu bars are common in Windows programs, they are difficult to achieve in Web browsers and some are specific only for Microsoft browsers; this version was designed to work in both Netscape and Internet Explorer.The menu item 'Explain' has submenus that load pages into the lower frame to explain the need for a research hypothesis, links to sites explaining research and statistics, and the help screen that explains how to navigate through the program.It is only on this first screen that a 'File' wasting our time.In the space on .the'rightyon."should type" the research' hypothesis for this study or yon can continue, through the other screens and come back to fin in (his information.

Figure 2 :
Figure 2: The program help screen