The Application and Evaluation of Adaptive Hypermedia Techniques in Web-based Medical Education

This article discusses the design issues involved in delivering Web-based learning materials. An existing application in the medical domain - JointZone - is used to illustrate how personalization and an interactive environment can be incorporated into Web-based learning. This work applies the combination of an adaptive hypermedia, situated-learning approach and hypermedia linking concepts to facilitate online learning. A usability study was carried out on the work described and an evaluation was undertaken to measure the effect of personalization on various learning factors. The evaluation outcome was analysed subjectively and objectively. The results proved to be contradictory but, nevertheless, the work gives new insights into the use of technology to support learning DOI:10.1080/0968776020100303


Introduction
The web has a profound influence on teaching and learning today especially in higher education.The evidence can be seen from simple online lecture notes to more complicated tutoring applications on the web.The popularity of web-based learning lies in the strengths of its distributed nature and the ease of its browsing facility.On the other hand, emerging technology such as adaptive hypermedia brings new flexibility and the capability of personalization into the conventional static presentation of materials on the web.Adaptive hypermedia is a recent direction of research at the crossroads of hypertext/hypermedia and user modeling (Brusilovsky 2001;Debra et. al. 1999b).The goals of this research are to improve hypermedia usability and achieve personalization by bringing the right content to the right person at the right time and the right place.
A survey of medical training in the United Kingdom [Towler, 1991] proposed replacing the conventional theory based teaching methods with the emphasis on the development of medical skills.In addressing this issue, a number of computer-based systems have been developed, such as MR Tutor (Sharples et. al. 1995) for radiology training and ATLAS (Garlatti and Sharples, 1998) in neuroradiology.
In this article, we outline the collaborative experience and strategies taken in developing a webbased learning application in Rheumatology -namely JointZone.JointZone (Armstrong et. al., 2002;Ng et al., 2002) was developed primarily for university medical students.The content exists in the form of an "electronic online textbook" which is liberally illustrated.A number of video clips and over 700 illustrations have been included.In an additional section, there are a total of 30 case studies.The "virtual patients" in the case studies present rheumatic disorders set in a variety of clinical scenarios, most of which arise frequently in the typical district general hospital rheumatology clinics.It was a joint project involving a medical expert generating the content and an educational developer tackling the pedagogical issues of the website.The website shows an example of how adaptive hypermedia can be used to present a personalized learning experience.An evaluation has been carried out with medical students to look specifically at how the integration of adaptive hypermedia supports learning in this environment.
To provide some background, this article will first describe the pedagogical rationale and the approach to hypermedia linking that was fundamental to the development of the application.Then, it explains more about adaptive hypermedia and how various adaptive features were integrated into JointZone.Finally, the article presents the results of the evaluation undertaken to assess the effectiveness of the adaptive features, and concludes with a discussion including suggestions for future work.

Background Pedagogical rationale
Figure 1: The advance case study gives wider scope of user choices.
The theory of situated learning promotes the acquisition of knowledge in a meaningful context (Brown et. al, 1989) (Herrington and Oliver, 1995).In the situated mode of learning, students are put through authentic life experiences where knowledge gained is embedded in a social or physical context.The theory suggests that learning through real life experiences produces more usable and robust knowledge (Brown et. al. 1989) as compared to inert knowledge (Whitehead, 1929) that is only useful in school exams.In recent research, this situated learning approach began to be adopted as a model of instruction in computer based learning environments (Bransford et. al., 1990).In supporting this situated learning approach, hypermedia is capable of presenting engaging interactions with the students and at the same time provide a database of information to support learning (Harley, 1993) (Reeves, 1993).Hypermedia can be used to provide an alternative for the real life settings by enriching the learning experience through the availability graphics, videos, audio and text simulation.The emergence of adaptive hypermedia further ensures that the provision of learning materials can be adapted to different needs and knowledge background of the users.
The approach of situated learning prompted the integration of interactive case studies in JointZone to simulate the clinical diagnosis process of a Rheumatologist.The case studies, which are developed by a rheumatologist, provide a real life clinical context where students can actively engage in problem solving rather than being passive recipients of information.Students are able to develop procedural knowledge which includes interpretation of clinical information and decision-making.The case studies are graded into three levels: beginner, intermediate and advanced.For the beginner or intermediate cases, the scope of patient scenarios is narrower and the diagnosis choices are more restricted compared with advanced cases (see figure 1).At every stage in a case study, relevant feedback and responses are given to users so that they can benefit from the expert's experience and opinion (see figure 2).In order to relate the case studies (procedural knowledge) to the factual contents of the application (declarative knowledge), a list of reading pages are given at the end of every case study for the purpose of further reading on a particular disease or fact (see figure 3).The number of pages proposed for further reading depends on the level of knowledge of the users.The motivation for this is to expose more learning concepts to users of higher competency and to avoid overloading users of lower competency.
Figure 2: The beginner case study gives immediate response to user input and relevant feedback from an expert point of view

Hypermedia linking
It is important that information is well cross-referenced to ensure the ease of navigation through learning materials on the web.Easy accessibility to pieces of information such as text, video, audio or images is important to avoid the user being frustrated during the learning process.The issues of how and what to link are nontrivial in order to present learning materials more effectively to the users.In JointZone, three types of links are used (Lowe and Hall 1999, p.33) and they each serve a different purpose.The three types of links are: • Structural links which provide navigation by connecting pages according to the structural design of the site.

• Referential links (or glossary links) which provide answers to the question of 'what is
x?' They link a word to its explanation, which can be a mixture of text, image or video.
• Associative links provide learners with the opportunity of to find out more about a concept.They interlink pages of related concepts according to the semantics of the domain.
Structural links, which are widely used in the site map and table of contents, are the most common links followed by the users to navigate from one place to another within the website.Referential links (see figure 4) are used to check out the meaning of a word or a phrase that appears in a predefined glossary database.They are found everywhere in the site where a glossary word appears.These links are implemented based on the Microcosm generic linking mechanism (Fountain et. al. 1990) which enables the links to be generated automatically.Lastly, associative links are used to interlink pages, either internally or with an external site, that are related by a common context.For example, when a user is looking at the context of a disease -'osteoarthritis of the kne'e, they could follow the associative links to find out more about the general concept of 'ostearthritis' or the disease management on 'knee surgery'.All links except structural links are stored in link databases.This separation of links from the documents enables more flexibility in link management.For example, any update to a referential or associative link need only be done once in the database and the changes will be propagated to the instantiation of links in the domain.
The details of how the three types of links are created in JointZone can be found in (Ng et al., 2002).

Implementation of adaptive hypermedia features in JointZone
Adaptive hypermedia The paradigm of web publishing has shifted from static to dynamic presentation.A web site no longer displays static information but rather is capable of adapting its content to perceived user needs and individual knowledge.The relatively new idea of adaptive hypermedia (AH) leads to applications that reflect knowledge about the components of a learning environment.Adaptive applications model the domain knowledge they represent and actively capture information about users as they interact with the application.The users' cognitive burden regarding what links to follow is transferred to the system, thus reducing the problems of information overload and 'being lost in information space' (Thuring and Haake, 1995).
An AH system normally consists of a user model (that describes information about the user such as their knowledge, preference etc), a domain model (describes the domain subject at the conceptual level) and an adaptation model (describes how the adaptation can be done) (De Bra et. al. 1999;De Bra and Brusilovsky, 1998).Users' individual and needs are met through two levels of adaptation: adaptive navigation support (ANS) and adaptive presentation (Brusilovsky, 2001;De Bra, 1999).Adaptive presentation adapts content to users by presenting different versions of the same piece of information based on their needs.ANS adapts to users by tailoring the presentation of links.Techniques in this area include link hiding, link sorting, link annotation, direct guidance and map adaptation (Brusilovsky 2001).Link hiding is achieved by disabling the functionality of irrelevant links in an attempt to limit users to only the set of links relevant to their goals or background knowledge.Link annotation is achieved by annotating links with extra information about the links.For example ELM-ART (Brusilovsky et. al., 1996) uses a traffic light metaphor where different colour balls are used to annotate links in order to indicate whether the user is ready for the links.

The basis of adaptation in JointZone
The literature shows that in most educational AH systems, the idea of curriculum sequencing is used to model the domain into prerequisites, remedial and outcome topics before the aggregated learning materials are presented adaptively to the students based on their current level of knowledge (Stern and Woolf, 2000;Brusilovsky et. al., 1996;Brusilovsky et. al., 1998;De Bra and Calvi, 1998).From the perspective of authoring, this method increases the burden on the domain expert to manually design a prerequisite network of the contents.In some cases this problem is solved because the application designers are experts in the information domain themselves, a reason why many educational AH applications are built in the domain of computer science (Stern and Woolf, 2000;Brusilovsky et. al., 1996;De Bra and Calvi, 1998).
Domain Model -The content of JointZone is in the medical domain and the domain experts do not have enough time to manually sequence the content.Hence we adopted the idea of text processing from the area of information retrieval to identify the semantics of the contents (Ng et. al. 2002).This method freed us from requiring prohibitive effort from the domain expert in organizing and labeling the content.Text processing is applied to the contents in order to extract keywords that can be used as indexing terms for the documents.Indexing terms are (document) words whose semantics reflect the document's main themes.They are used to index or summarize a document (Baeze-Yates and Ribeiro-Neto, 1999).
Knowledge-based Adaptation Model -The availability of keywords to conceptually represent the documents enables link-level adaptation based on the user's background knowledge in rheumatology.This is achieved using link hiding.For example, for the topic of 'Milwaukee shoulder', advanced students are provided with access to documents about the concepts: 'Milwaukee shoulder, hydroxyapatite, crystal, polymyalgia' (four concepts) while intermediate students are directed to only three concepts namely 'Milwaukee shoulder, hydroxyapatite, crystal' and beginners are simply shown the single concept of 'Milwaukee shoulder'.The domain expert identifies these three sets of concepts for three different user groups (advanced, intermediate and beginner), which is an easier task for them compared to delivering a prerequisite network of documents.From the users point of view, this incremental way of linking through link hiding helps reduce information overload since less reading pages are directed to less well-equipped students.However, in order to be consistent with the philosophy of an unobtrusive learning environment, there is no restriction for any user group should they wish to investigate further, i.e. the reading documents allocated for the advanced level are still accessible to a beginner.
User Model -As for user modeling, the user's knowledge levels are drawn from a 'prior knowledge online test' given at the first login to the application.This test categorizes users into one of three knowledge levels and on subsequent access, the users are transferred automatically from one user group to another based on their performance in previous case studies.JointZone also captures the user's browsing history (pages visited) in order to provide another type of linklevel adaptation -namely history-based link annotation.The system keeps a record of a user's individual effective reading speed (Jackson and McClelland, 1979) which is assigned when they first use the application.An 'effort index' (Ng et. al. 2002b) is then calculated for each document visited.A page will have a higher effort index if the page display time approximates an optimal reading time.The optimal reading time is measured based on the length of the page and the individual effective reading speed.
History-based Adaptation Model -The effort index stored in the user model is used to construct a reading gauge (see figure 5), which is then used to annotate links.This provides history-based link annotation which gives users additional information on how much of a page has been read and helps them to focus on pages where they have spent little or no time on.• a personalized reading room (see figure 3) that adapts to each interactive case study to provide further reading on concepts related to the cases; • a personalized site map (see figure 6) that serves as a navigational tool; • a personalized topic map (see figure 7) that encourages goal-oriented reading by adapting pages to the user's short-term reading goal (a self-selected reading topic), browsing history and knowledge level.
For all the above AH features, the combination of adaptation mechanisms based on the user's browsing history and knowledge level helps users to focus on the number of documents that are suitable to their knowledge background, and at the same time helps users obtain extra information on what they have already read and how much they have read during previous visits to the site.

Usability study
To assess the usability of the application prior to the evaluation of the adaptive hypermedia features, a usability study on JointZone was carried out with 29 undergraduate students who were taking a course module on Human-Computer Interaction (HCI).The subjects consisted of 3 females and 26 males with an average age of 19 (stdev = 2.06).
In the study, the students were first asked to fill in an online demographic questionnaire.Then they were shown a list of online instructions to guide them to the site.Based on these instructions, users had to explore JointZone by observing its structure, navigation facilities and presentation styles.They were given a set of tasks such as: look up the meaning of a medical term using the site, locate a document and attempt an interactive case study.Finally, they were asked to complete an online feedback questionnaire (see appendix A) created using a list of guidelines suggested by (Preece, 2000) for evaluating websites.The questions in the questionnaire were categorized into navigation (the ease of getting around the site), accessibility (the ease of the authentication procedure and page loading time), presentation (site layout and graphical design) and subjective feedback (the users' opinion of the site).The usability result was calculated by dividing the total feedback score on the Likert scale by the number of users and the number of questions for each category.
As seen from the chart below (see Figure 8), the result shows no major concern with the overall usability of the website since all four categories achieved an average score of above 3.The accessibility perspective of the website gained the highest score reflecting the ease of using the web site.This is so even though most subjects were using a relatively slow connection (accessed from home using a standard 56k (or slower) telephone modem) during the study.

Evaluating adaptive hypermedia
Educational hypermedia was the first application area to see the introduction of adaptive hypermedia techniques before the research extended to other areas such as tourism, e-commerce and news portals.Classical projects such as ELM-ART (Brusilovsky et.al. 1996) and AHA! (De Bra and Calvi, 1998) are web-based systems used to develop adaptive online courses.However there is a significant lack of evaluation in this area and much debate as to how an AH applications should be evaluated.The conventional method of comparing an adaptive and non-adaptive version of an application is debatable (De Bra, 2000) as most adaptive systems are developed with particular adaptive techniques in mind and removal of these techniques renders causes a lost of the system's basic functionality.Therefore, any evaluation using AH vs non-AH techniques will not result in a fair comparison.The basic non-adaptive design of JointZone was optimized based on the usability study before any personalized mechanism was integrated into the site.This was to make sure that the basic functionality of the website was not overshadowed by the advantages given by the personalized mechanisms.This provided a fairer comparison between the adaptive and non-adaptive version of JointZone.
A recent approach has been proposed by Karagiannidis et. al. (2001) which is known as the layered evaluation.This approach suggests two layers of assessment: • Interaction Assessment Layer -Assess whether the assumption drawn by the system concerning characteristics of the user-computer interaction is valid, e.g. is the inference of the user's knowledge level accurate?• Adaptation Decision Layer -Assess whether the adaptation decision is valid and meaningful, e.g. is the set of links presented based on the user model meaningful?
This approach provides researchers with the opportunity to properly identify the problems behind a failed implementation and to target the solutions accordingly (Brusilovsky et. al. 2001).For example, it can be the case that the adaptation decisions are reasonable, but they are based on incorrect system assumptions; or that the system assumptions are correct but the adaptation decision is not meaningful, or both cases can happen at the same time.The evaluation of JointZone followed this approach in order to properly target a solution should any problem arise.
In previous work, some of the comprehensive evaluations that were performed in educational AH have shown encouraging results.For example, the empirical studies performed by Boyle and Encarnacion in MetaDoc (Boyle and Encarnacion, 1994) have indicated that adaptive presentation can improve content understanding.Also, an empirical study of ISIS Tutor (Brusilovsky and Pesin, 1998) has shown that adaptive navigation support can increase the speed of navigation in learning for certain groups of users.

Experimental evaluation of JointZone
The purpose of the evaluation of JointZone was to find out whether: • The personalized features facilitate the process of learning in JointZone by reducing the time taken to look for materials on the site and reducing the number of navigational steps.1: The distribution of subjects in the experiment according to their year of study and their average score on the prior knowledge test.
During the evaluation, the students were randomly assigned into two groups: the first group used the personalized version of JointZone and the second group used the non-personalized version of the site.A series of predefined tasks were setup for the students asking them to look up information using the topic map and to try out the case studies.Pre and post-tests were used to assess the user's improvement in learning.Time taken and the number of navigational steps required to complete a task were logged in order to assess whether the personalized features affected either of these metrics.

Results
The results are grouped into objective and subjective feedback.The subjective feedback is based on the user's opinion collected from the questionnaire whereas the objective feedback is based on the dependant factors such as time taken, performance in the pre and post-tests, and navigational steps.

Subjective feedback on navigational features
Figure 9 shows the general subjective feedback on the various navigation features in JointZone.
As indicated in Table 2, all except one of the subjective user feedback results are significantly positive, above a confident level of p < 0.01.This result shows that students welcomed the idea of using a variety of navigational support mechanisms to help them to get to a piece of information faster and easier.In the case of users' preferences between a personalized site map and a table of contents, 50% of the users preferred the former and 31% had no preference.This indicates a browsing style where users were still not prepared to give up the conventional table of contents as a navigational aid but at the same time they welcomed the idea of a personalized site map.

Subjective feedback on adaptive link hiding and adaptive link annotation
There was inconsistent subjective feedback from the students regarding the various adaptive features integrated into these navigation mechanisms.As shown in Figure 10, a significant number of students felt that there were too many links suggested in the adaptive reading room.However, in the case of the non-adaptive reading room, no significant conclusion can be drawn to show that users are satisfied in terms of the number of links suggested (see Table 3).This scenario suggests that users needed some sort of filtering mechanism in the reading room but the current implementation of adaptive link hiding is not capable of delivering the right set of links.
The reason for the negative opinion on adaptive link hiding in the reading room could be due to the incorrect adaptation decision-making by the system.For example, a student might be wrongly assigned to a user group that is of a higher level than their knowledge level; hence he or she is overwhelmed by the number of links suggested.The evidence of incorrect assignment of user group can be seen from the poor performance of subjects in the case studies, as will be shown later in this section.In the case of the feedback on the history-based link annotations using the reading gauge, users found them useful on the personalized site map but not significantly useful in the case of the personalized topic map (see Figure 11).The main reason could lie in the nature of the task being assigned to the students when using the topic map in the experiment.The task is concerned with looking up information, hence the focus of the students was more on finding answers regardless of whether they have or have not read a page.In addition, the presence of the reading gauge might not have the initial expected impact since the browsing history might not have been sufficiently built up due to the short period of experiment time.The personalized site map received better feedback as it shows all pages, hence changes in the reading gauge are more significant.At the same time, the personalized site map serves a different purpose, i.e. revision or general reading, thus the importance of the reading gauge is more significant in tasks of this nature.This might explain why users find the reading gauge more useful in the personalized site map as compared to the personalized topic map.

Objective feedback on topic map
In the experiment, the users' objective feedback was captured and analyzed through various learning factors such as the time taken to complete a learning task, the learning performance gathered from pre and post-tests and the number of navigation steps required.Two groups with two different experimental conditions were set the same task of learning about 'psoriatic arthritis'.The first group used a topic map (without any adaptive features) and the second group had the aid of the table of contents for free browsing.The effect of using navigation tools in supporting learning can be seen from this experiment.As compared to free browsing, the results (see Table 5 and Figure 12) indicate that the topic map significantly reduces the time taken and the number of navigation steps required to complete a task.The learning performance of students in the topic map group was also significantly better.

Objective feedback on personalization
In analyzing the effect of personalization on learning factors, two groups of users were asked to use the personalized and non-personalized topic map respectively for a task of learning about Rheumatoid Arthritis.As seen from the results (Figure 13 and Table 6), the integration of personalized features on the navigation tool (topic map) did not result in any significant improvement in terms of the time factor and of learning performance.However, it did significantly reduce the number of navigational steps required to complete the learning task.
Hence, it can be concluded that the personalization features on the topic map in JointZone do not improve learning (performance) but partially facilitate the process of learning since they do help in reducing the number of navigational steps significantly but not in the time taken.

Adaptive assignment of case study on learning performance
In evaluating the adaptive assignment of the case studies, the system's decision-making was put under scrutiny.The results revealed the sensitivity and accuracy of the system in inferring the user's knowledge level and assigning case study levels (beginner, intermediate, advance).As usual, two groups of users were involved.The first group (system assign) was asked to do a case study from a study level assigned by the system whereas the second group (self assign) selected a study level of their own choice.Their performance in the case studies was then compared.
In order to find out whether the assignment of case studies was accurate, a metric we called 'allocation fitness' was calculated by using the case study scores.For example, if a user scores between 0 to 30% in a case study, the allocation fitness is set to 'too difficult' to indicate the level of case study assigned was higher than the user's knowledge level.If the scores fall between 30 to 80%, they will be considered 'optimal' indicating the right level of case study and a score of above 80% will give an allocation fitness of 'too easy'.Figure 16 shows the comparison between the allocation fitness of the 'system assign' group and the 'self assign' group.The result revealed that more than half the students in the 'system assign' group were assigned case studies that are 'too difficult' a level for them (Chi Square Test = 3.86, p= 0.05, significant level of 5%).In this group, only 30% of the users were correctly assigned the right level of case studies.This indicates the need for the tuning of the system's adaptated decision-making mechanism by reducing the current assignment threshold.On the other hand, the result of the 'self-assign' group revealed the majority of the users know the correct level of case studies for them.Nine students are in the 'optimal' category and six students fall in the 'too difficult' category.However this result is not significant (Chi-square = 0.6, p= 0.439), so we cannot claim that the users know better than the system in terms of the right level of case studies for them.

The Analysis of System Adaptation Decision Making in Assigning Case
Studies -A comparison between system-assign and self-assign groups

Discussion
From the pedagogical perspective, this evaluation has given insights into whether the adaptive features really benefit the students or whether they were mere technology-driven.Before any conclusion can be made, a few issues need to be taken into consideration.
Firstly, the outcome of this evaluation has shown that the effectiveness of the adaptive hypermedia features relies on what types of AH techniques are being used (i.e.link annotation or link hiding), where the features are applied to (i.e.site map or topic map) and the nature of the tasks involved (i.e.general browsing or information lookup).Hence, the effectiveness of the adaptive hypermedia features could be both application and task-dependant.This is shown by the contradictory subjective feedback where the history-based link annotation (reading gauge) proved to be very useful in personalized site map but is not the same case for the personalized topic map.
Secondly, the accuracy of the adaptated decision-making by the system also proves vital, for example in the case of adaptive link hiding in the personalized reading room, the outcome is disappointing due to the inaccurate user group assignment.The same problem arises with the adaptive case study assignment.The system's adaptation model needs fine tuning in order to better map the current prior knowledge test score to different user groups.
Thirdly, the current system might not be sensitive enough in inferring the user's knowledge level.The prior knowledge test used, which consisted of multiple-choice questions, might be too general to capturing the user's knowledge of the domain as compared to the level of complexity in the case study.Unfortunately, no test has been carried out thus far to verify the accuracy of the prior knowledge test.If the test proves inadequate, it might not be possible to develop a good set of questions to correctly test the user's knowledge level.Questions like this one involve addressing cognitive issues which are beyond the scope of this work.

Conclusions
JointZone represents a medical web-based learning application that embodies both declarative knowledge (domain knowledge) and procedural knowledge (interactive case studies) through a personalized and interactive learning environment.It allows users to engage with the learning materials more effectively through various adaptive linking strategies.
To simplify the adaptive hypermedia authoring, the idea of keyword indexing was used to model the domain, hence avoiding the need for prohibitive effort and time from the domain expert.A simple student model was constructed based on the user's knowledge level and browsing history.This contributed to the novel idea of using effective reading speed to better gauge if a student has read a page.The work has applied the combination of knowledge-based link hiding and historybased link annotation on a fully functional website to present an adaptive web-based learning environment.
The evaluation of various adaptive features on the web site has provided feedback from the users both subjectively and objectively.Overall, users welcomed the various navigational support features built into the application.The use of the topic map as navigational aid significantly improved the students learning performance and reduced the time and navigation steps required compared to free browsing.Users' opinions revealed that they like the personalized features that were integrated into the website; particularly the personalized site map which used the historybased link annotation.However, from the objective analysis, there is no evidence to show that the integration of the personalization features has improved learning or facilitated the learning process except to reduce the number of navigational steps.This might be due to the inaccurate system adaptation decision-making.
Based on the evaluation, it can be concluded that the potential of the personalized learning environment is promising as the students welcomed the personalized features in the learning environment.However the implementation of the adaptive mechanisms still needs refinement, particularly in system adaptated decision-making before it can make a significant impact in webbased learning.
JointZone has been officially published online and is currently in use with the medical students (Armstrong et. al., 2002).Further evaluations are planned over a longer period of time to obtain more reliable feedback from real time usage.

Figure 3 :
Figure 3: Adaptive reading room for intermediate level.

Figure 4 :
Figure 4: An example of a glossary (referential) link in JointZone

Figure 5 :
Figure 5: An example of set of links annotated by the reading gauge, which indicates high (H), medium (M) or low effort index (L)

Figure 6 :
Figure 6: A screen shot of the personalized site map where links are annotated by the reading gauge.

Figure 7 :
Figure 7: A screenshot of the personalized topic map, which suggests reading pages based on the user's short term goal (reading topic), knowledge level (e.g.intermediate) and browsing history (reading gauge)

Figure 8 :
Figure 8: Usability Result on the Likert Scale

Figure 12 :
Figure12: The difference of using topic map and free browsing on time taken, performance and navigational steps

Figure 14 :
Figure14: The analysis of allocation fitness from the system adaptation decision making in assigning case studies • The personalized features improve learning in JointZone in comparison to learning without the presence of any personalized mechanism.The evaluation on JointZone was performed with medical students from a university.A number of sessions were carried out involving a total of 36 medical students (see table1) with the same level of prior knowledge (T-test, t=0.298, sig.=0.768).Before each session the subjects were given an overview of the website and were taught how to use the personalized topic map.After each session, the students were interviewed individually to find out if they have any problems or comments on the website.

Table 2 :
Summary of the general subjective feedback from users on the various navigation mechanisms in JointZone.

Table 3 :
The subjective feedback on the number of links suggested in the reading room, with and without adaptive link hiding.Summary of subjective feedback on adaptive link hiding in JointZone.

Table 4 :
Summary of subjective feedback on adaptive history-based link annotation in JointZone.

Table 5 :
Summary of the effect of using topic map and free browsing on time taken, performance and navigational steps

Table 6 :
Summary of the effect of personalizing navigation tool in learning