Issues in student training and use of electronic bibliographic databases

In an article in this journal Ottewill and Hudson (1997) raised a number of issues concerning students' use of electronic bibliographic databases. They emphasized the need for co-operation between academics and librarians in database training and in coursework where databases would be used. We report a project on students' use of bibliographic databases. Our findings reveal that access to these databases, whilst solving many of the problems students experience in sourcing reference material for coursework and research, raises new intellectual problems due to the sheer breadth and depth of their coverage of subject matter. Typically database training programmes focus on search skills and the use of different interfaces. However, our findings demonstrate that students should be encouraged to develop a more critical perspective on databases since these can be seductive, time-consuming and, in certain circumstances, counterproductive resources. Students would benefit from more guidance on the quality cues that academics and librarians employ when evaluating different databases and their contents.


Introduction
Access to bibliographic databases, online and on CD-ROM, is advantageous for the academic community (East, 1993;Squires, 1993).Developments in communications and information technology mean that students, as well as academic staff, now have direct access to these databases (Tenopir and Ennis, 1998).This permits them to access a wider range of journal articles for research and project work than is usually available in a university library.There is an additional pay-off for business studies students since they have opportunities to develop the database search and retrieval skills they will need when they enter business and the professions (Taylor, 1996;Wisendanger, 1994).In some professions, such as law and public relations, access to specialist databases is essential to undertaking certain types of professional work (Taylor, 1996).Graduates who know where to locate information and how to access, retrieve and use information are valuable assets to employers.
Traditionally the main drawbacks of online databases have been the costs and the skills required to use them.As a result universities have limited access to those trained in database search skills, usually subject librarians.As more electronic information services are provided on CD-ROM and via the World Wide Web, direct access to databases can be granted more widely throughout the organization and time spent on individual and intuitive searches is no longer a key direct cost issue.
Database training initiatives aimed at students are reported in the literature, mainly from library professionals and in the context of North American universities (Neuman, 1995;Tenopir and Ennis, 1998;Tenopir, 1998).Librarians and information specialists offer most of the training for students and they tend to concentrate on issues of access and retrieval.Specifically, training programmes concentrate on how to use the different interfaces, identify keywords, employ Boolean operators and so on (Azzaro and Cleary, 1994;Webb, 1998).Whilst these basic skills are important, students need to be made aware of other issues involved in using bibliographic databases.
At least two additional sets of issues need to be considered.The first set relates to the efficacy of the databases themselves.The availability of electronic bibliographic databases for academic research is often perceived as offering such clear advantages over traditional hard copy abstracting services and journals that there is a temptation to view them uncritically.Their convenience and speed at performing searches tend to deflect attention away from both the ease of using them and their contents.There is considerable variation between databases on factors such as the user-friendliness of their interfaces, journal coverage, coverage of subject matter, incorrect citations and typographical errors (Grzeszkiewicz and Hawbaker, 1996).
The second set of issues relates to the skills required to use electronic databases effectively.
Most training programmes focus on search and retrieval skills.Arguably, these skills are the means to achieving a particular end, namely the evaluation and appropriate use of the information retrieved.As students move from manual to electronic searches they encounter a new situation, information overload (Tenopir, 1998).We know little about how this impacts on their intellectual decisions such as the criteria that they employ to evaluate the information retrieved.Furthermore, most problems in academic, business and professional spheres do not come labelled as problems and considerable problem definition and delimitation may be required.The difficulties involved in delimiting problems can be exacerbated when there is a surfeit of information.
In this article we report on a project designed to provide opportunities for students to identify and reflect on the benefits and drawbacks of using electronic bibliographic databases.We describe the project briefly below and go on to discuss the findings.The findings from the project have forced us to rethink the training and support students need to use databases effectively.They have also influenced the ways that we set guidelines for assessing coursework when bibliographic databases are used.

Project description
The project was designed to encourage students to become more critical users of electronic bibliographic databases; a need that is recognized by many library professionals (Tenopir and Ennis, 1998) and academic staff (David and Zeitlyn, 1996).The project was incorporated into the, coursework requirements for market research modules.Market researchers place considerable emphasis on using secondary data sources, including databases, to help solve clients' problems.Thus, coursework designed around the use and assessment of bibliographic databases was appropriate to the aims and content of the module.
Students were asked to develop a bibliography for an essay that they were writing.Specifications were set for the bibliography.It should be developed employing a variety of sources including OPACs, current and past volumes of printed journals, electronic bibliographic databases (primarily ANBAR and ABI INFORM) and abstracting services in hard copy format (including Market Research Society Abstracts and ANBAR).They received a brief training programme delivered by library staff before commencing work on the bibliography.This programme was designed specifically for business studies students and focused on the range of databases available, their different interfaces and some basic search skills.
Each student was asked to submit a written account of their experiences with all the sources they employed.They were asked to include a descriptive account of the activities involved in compiling the bibliography including which sources of information were used and in what order, search terms employed, and hits achieved.They were requested to discuss any problems they encountered and how these were addressed.Finally, they were asked to reflect on the benefits and drawbacks of using databases and to compare the usefulness of the various sources they used.We reported back to students on our analysis and interpretation of their written accounts and asked for their comments.This feedback and the discussion it stimulated provided further information for us and students found it useful to compare experiences.
The first cohort to participate in the project in 1994 consisted of 49 full-time students (29 female and 20 male) in their final year of a business studies degree programme.These students did not have access to electronic bibliographic databases prior to their final year.During 1995 the project was extended to include part-time students and postgraduate marketing students.The design of the project remained largely unaltered until 1998 when our teaching responsibilities for undergraduate classes ended.
Since the project commenced in 1994 we have had an opportunity to identify how students' use of these resources alter as they gain more experience of using them.Monitoring change was important, as access to databases has improved over the years.When the project began the range of electronic databases available to our students was limited to abstract (ABI INFORM) and citation databases (FirstSearch).These were available in CD-ROM format and could only be accessed in the university library.During 1997 the library subscribed to full text bibliographic databases including European Business ASAP and, later, ProQuest.These are accessed via the World Wide Web, which means that students can use them from any networked university computer.

Project findings
Students' written accounts provided considerable detail relevant to a range of issues.We report the findings that relate to problems that can result from a surfeit of information and judging the relevance and quality of this information.We begin by reporting how the first cohort of students tackled the project and then go on to discuss how this has altered with successive student cohorts.

Search and evaluation process
Our analysis of written accounts revealed considerable similarities in the search procedures students employed.The typical search process consists of various stages and there is considerable iteration between and within these stages.
Stage 1: Initial forays Students searched the electronic databases first in preference to other sources such as current and past volumes of journals and abstracting services in hard copy format.Whilst students described these first searches as unproductive in locating suitable references, it was clear that they were productive in terms of learning how to use databases and in identifying the need to clarify exactly what you are looking for.
Keywords for these early searches were drawn from the essay titles, usually resulting in searches that were either too broad, resulting in huge numbers of references, or, too narrow resulting in no references at all.A search using the keyword 'ethics' resulted in hundreds of hits from ABIIFORM.By contrast the keywords 'group discussion ethics' resulted in no hits at all.When unfamiliar with the essay topic, as most students are at this stage, keywords are exhausted very quickly.Whilst hits that are definitely irrelevant to the topic can be spotted quite easily, students are not in a position to evaluate the relevance of those that remain.It is at this point that they identify the need to obtain more information on the essay topic before undertaking further searches.
Stage 2: Defining problem parameters and focused searching Students read the relevant chapters from textbooks to obtain a better understanding of the essay topic and to identify potential keywords.Armed with a better understanding of the topic and new keywords, they engage in more focused searching.These more focused searches produce hits that, in turn, suggest new avenues for further searches.The point at which to stop searching and make some decisions on which of the hits to retrieve in full text was not considered to be problematic.Some students reported that diminishing returns set in; searches revealed few hits that had not been obtained during earlier searches or from another database.Others stopped when they considered that they had a sufficient volume of hits, most of which they would probably not use anyway.

Stage 3: Decision time
Hits that were definitely irrelevant to the essay topic had already been eliminated at Stage 1 and the more focused searches at Stage 2 generated fewer of these.Even so, students may still be left with a large volume of hits to be evaluated, all of which may be relevant to the essay topic.Two approaches to evaluation emerged.Some students made decisions about their essay plan including which issues to make the focus of the essay and which ones to leave out.This approach considerably reduces the number of hits and may require further, even more focused, searches.However, most students make their evaluation decisions based solely on whether or not they can obtain the full text of the article.The availability of the article in the library is the key factor in determining whether or not it will be accessed and read.This search and retrieval protocol remained largely unaltered even though subsequent cohorts of students embarked on the project with some prior experience of using databases.Their written accounts offer some insight into the reasons why the search protocol remained unaltered even after the introduction of full text databases.

Defining and delimiting problems
Searching databases for information is a means to an end and unless the user has a clear end in sight, or a well-defined problem, information overload can be a significant problem.However, it is also the case that databases can provide the information that helps the user specify the problem in the first place.Our description of the search protocol revealed this interplay between searching and problem definition and delimitation.Thus, experienced users of electronic databases still engage in the initial foray stage described above and knowingly employ keywords that will result in large numbers of 'unproductive' references.This is because the titles of the hits generated can provide cues to the nature and scope of the topic they are researching.These broad-brush searches can highlight the issues or themes that are the subject of the most recent publications on a topic.They also provide cues to which journals to search and authors who publish frequently on the topic.
Of course students still report problems with regard to problem definition.Some students referred to their 'pre-database' essay preparation in terms such as 'blissful ignorance', 'naive' and 'superficial'.Scouring the library shelves for essay material can be a hit and miss affair.
Finding any material at all by this type of browsing was considered a small victory and the material that was retrieved was employed liberally in the essay, even though its quality may not deserve such focus.With access to bibliographic databases these problems reduce, but different problems arise.In our experience, few undergraduate students employed abstracts in hard copy before we initiated this project.Usually, they would identify material for essays from references in textbooks and in any journal articles they accessed.In turn, these sources would be used to identify yet more references.The availability of databases has eased much of the manual burden and time involved in searching for references; however, the intellectual burden has increased.Access to abstract and full text databases has encouraged students to use these resources.At the same time, these databases have revealed the sheer volume, range and diversity of reference material on a topic.

Judging database quality
Databases can be evaluated on the ratio of relevant to irrelevant hits they generate on the problem at hand and also on their inherent quality.Students' written accounts raised some interesting questions on both these issues.
Students reported that the most useful references for their essays were found in the printed volumes of a specialist market research abstracting service.Few references listed in this source were eliminated as irrelevant.This is hardly a surprising finding since they were writing a market research essay.A specialist market research database would be expected to outperform more generalist databases such as ABI INFORM on the criterion of relevance.This led some students to question whether using electronic databases did in fact result in timesaving.Of course the key learning point for students is that there is a variety of databases available and students need to select the one(s) most appropriate to the problem at hand.This was an important point given that students preferred to use ABI INFORM, which was considered easier to use than ANBAR.
Students make their selection decisions based largely, and even solely, on the availability of the full text versions of articles.When only abstract and citation databases were available, the decision was based on whether they could obtain the full text of the article in the library.This reduced considerably the number of references that were considered.As full text databases became available, selection decisions altered somewhat.Most databases report the results of searches by listing the most recent publications on a topic first.Students work back through the abstracts marking those that seem most relevant to their topic before downloading or printing the full text versions.If there is a large number of relevant hits, students work back through the abstracts until they judge they have a 'manageable' number to work with.They operate on the basis that even the most recent publications are likely to include references to important earlier works, and these can be accessed at a later stage if necessary.
Students were asked to compare the various databases they used.Thus, it was not unusual to find the same reference abstracted in different databases.Some abstract writers would emphasize issues not mentioned by others and, more significantly, some abstracts bore very little relation to what was contained in the full text of the article.This prompted questions on the extent to which users can rely on abstracts.
Students also identified and questioned issues of editorial policy and coverage.Some articles are referenced in one abstract database but not in another that abstracts the same journal.In a similar vein, comparisons of full text databases revealed that items available in one journal issue were not available in the same issue in another database, raising questions as to what exactly full text means.Indeed one of the main findings to emerge from this ongoing project is that students who had access only to abstract databases on CD-ROM demonstrated a more critical attitude to all of the sources that they used.The most recent cohorts of students who had access to full text databases on the World Wide Web were also able to identify some weaknesses in the various sources but, for the most part, did not consider them a problem since they are often spoilt for choice of material.

Discussion
What we have learned from students' accounts of using bibliographic databases raises issues on the content and delivery of database training and support for students who use them.It also raises issues for teachers who set and assess coursework for these student users.We discuss these issues in more detail below.

Delivery of training
The literature on this subject reveals that the responsibility for training rests largely with library and information technology staff.University librarians are already under considerable pressure in their instructional role.Tenopir and Ennis (1998) found that over the past decade university reference librarians had increased substantially the time they spent on formal and informal instructional sessions with students as a result of the rapid and widespread adoption of electronic media.The authors point out that this situation is exacerbated by the fact that student needs have become increasingly diverse with some students requiring very basic instruction and other more experienced users requiring advanced instruction, say, on synthesizing the results of searches.
The potential problems involved in relying mainly or only on library professionals are highlighted by Meadow, Wang and Yuan (1995).They found that search process specialists and subject matter specialists differ in the ways they implement database searches.Similarly, David and Zeitlvn (1996) found that even in domains where databases are heavily used for research, academics often rely more on specialist sources than the more generalist sources available in university libraries.Indeed some of their respondents claimed that electronic databases encouraged a 'scatter-gun' approach to searching.They were useful only for stimulating ideas and could in fact be counterproductive.Thus, students could be receiving mixed messages about databases from academics and library professionals.Ottewill and Hudson (1997) made a case for both library staff and academic staff to be involved and to co-operate in training students on how to use databases, and the research reported here supports their case.Academics could co-operate with and support the efforts of library staff in a number of ways that would benefit both parties and, most importantly, their students.

Content of training
The content of training programmes tends to focus on what databases are available, how to use the various interfaces and the development of search and retrieval skills.Electronic databases with the sheer volume of content and their apparent advantages in convenience and speed of searching can be very seductive, leading students to reject other more (apparently) timeconsuming sources of material.For this reason it is important that training initiatives incorporate a more critical assessment of them which, at a minimum, should include discussion of the criteria against which all sources of material might be judged.Here the criterion of relevance is of prime importance.Ideally, training initiatives will include some practical exercises to illustrate relevance problems and how these might be addressed.This is especially important since many of the electronic databases that university libraries subscribe to are generalist rather than specialist and aimed at multiple users.FirstSearch and European Business ASAP are aimed at users from all disciplines and all management disciplines respectively.Teachers could help in the design of exercises relevant to their subject areas.
Other evaluation criteria need to be discussed in training programmes.As university libraries subscribe to a greater variety of databases, we need to make more explicit to our students the differences in database efficacy.This would include factors such as their journal coverage, what the provider means by full-text, proneness to errors and so on.Here library staff and academics could co-operate by sharing knowledge on databases that can subsequently be passed on to students.Most electronic databases provide abstracts or full text articles from the mid-late 1980s.Some seminal articles on a topic may predate these databases.Although this may seem a very obvious observation, many of our students discovered it only after they had tried unsuccessfully to access some reading list articles dated prior to the late 1980s.Since students prefer to work only or mainly with full text databases, many academics are expressing concern about 'the loss of knowledge caused by the great electronic divide' (Peek, 1998).Teachers may need to emphasize the message that recent is not always best.

Coursework
Direct access to databases has revealed the sheer volume of material available on a topic and, more significantly, the range and complexity of issues involved in a topic.A key issue for students is how this range and complexity can be represented or reduced to meet the requirements of a word-limited essay.Is it better to select one or a few of the issues associated with the essay topic and discuss these in depth or to consider the range of issues, perhaps sacrificing depth?Faced with increasing numbers of students asking these questions, we recognized at an early stage in this project that we would need to give more consideration to how we select and design essay titles and specifications.
For some students, particularly postgraduates, setting coursework assignments that do not define too precisely the problem to be addressed provides them with opportunities to experience the delimitation problems that academic researchers and decision-makers face.This can be built into the assignment specification that students receive.For example, it can be listed as one of the assignment objectives and as an assessment criterion.For other students, particularly undergraduates, the assignment set may need to be more tightly specified in terms of its focus and scope.To illustrate, we could indicate that we were looking for a range of issues to be discussed, e.g.'discuss the issues companies need to consider when designing a test market', or a single issue in depth, e.g.'select and discuss in detail one of the issues facing companies when designing test markets'.
Large generalist bibliographic databases include refereed journals, non-refereed journals and trade and professional publications.Marketing News, a trade journal, is available alongside the Journal of Marketing, a high-quality research journal, and both are publications of the American Marketing Association.As teachers we need to alert students to quality cues we employ as academic researchers.Often these cues are so taken for granted that we assume our students will know them.For example, if the full text article does not contain any references then it may not be worth downloading or printing it.We noted that when faced with a large volume of 'relevant' references our students start with the most recently published and work back until they consider they have retrieved a sufficient volume of material.In certain areas of marketing, students may have to work back through many pages of hits before locating the first refereed journal article.
Of course, it is not always the case that articles from high-quality refereed journals are the most useful for a particular piece of coursework.However, we do need to inform students of the kind of references we expect them to source for individual pieces of coursework.To illustrate, nonreferred and trade publications such as Campaign may be appropriate where students are to undertake a review of current advertising practice, but not for a critical review of advertising theories.By contrast, a comparison of the current issues puzzling advertising practitioners and academics may require the perusal of a wider range of sources.

Conclusions
Electronic databases, online, CD-ROM and the World Wide Web are a useful and necessary resource for academics and for those working in business and the professions.The ability to use them well will be advantageous for students as they seek employment in business and the professions.However, they need to be aware of the wider issues associated with the use of these seductive resources: to learn to choose databases carefully and to use them critically.The wider issues associated with database use, relevance and coverage (or lack of it) are issues as important as how to use particular interfaces or construct lists of keywords.We have tended to see training as something that is the responsibility of library and information specialists.However, our own and other research suggests that academic staff need to play a greater role here.
Direct student access to a wider range and diversity of journals means that teaching staff need to rethink implications of this for student coursework.Students would benefit from more guidance on the quality cues that academics and librarians employ when assessing the usefulness of different databases and when evaluating the individual references they contain.The evaluation of individual references refers not only to their inherent quality, such as the quality of the journal, but also their relevance to the problem.Students using databases can find themselves with huge lists of references without having thought through issues of scope and focus for the problem at hand.The process of defining and delimiting problems is an important intellectual skill in academic research and in business and professional practice.Access to electronic databases, particularly the generalist databases, may complicate rather than simplify problems.
The power and potential of electronic databases are not in question.However, if they are to be used effectively by all our students greater care must be given to their integration into the curriculum.