Embedding Accessibility and Usability: Considerations for E-learning Research and Development Projects

This paper makes the case that if e-learning research and development projects are to be successfully adopted in real-world teaching and learning contexts, then they must effectively address accessibility and usability issues; and that these need to be integrated throughout the project. As such, accessibility and usability issues need to be made explicit in project documentation, along with allocation of appropriate resources and time. We argue that accessibility and usability are intrinsically inter-linked. An integrated accessibility and usability evaluation methodology that we have developed is presented and discussed. The paper draws on a series of mini-case studies from e-learning projects undertaken over the past 10 years at the Open University.


Introduction
The Open University (OU) is Europe's largest educational establishment, delivering mainly distance learning courses.The OU currently has 180,000 active students, of which about 9900 declare a disability (∼5.5%).Approximately one-half of all disabled students receive some form of support from the university to enable them to participate in their studies.The OU has made extensive investment in e-learning particularly since the late 1990s.Investment continues with an ongoing multi-million pound virtual learning environment (VLE) programme.The OU has a commitment to widening access to higher education, to providing high-quality, interactive educational materials that meet students' needs and operating within the mission of 'openness to all'.The OU is committed to making its online educational content and student services accessible to disabled students and usable by all; a considerable challenge given the size of the OU student population.

M. Cooper et al.
The paper draws on examples from various projects undertaken within the OU over the past 10 years, presented as a series of short case studies.The purpose is not to highlight any failings in the way accessibility and usability was managed in any particular project, but rather to draw lessons from a wide range of past and ongoing work to inform the design of future projects.These case studies are used as the basis of identifying a set of lessons learnt, along with recommendations to enable future elearning projects to successfully integrate and embed accessibility and usability considerations.

Definitions of accessibility and usability
Usability is the extent to which a system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (Karat, 1997).Usability, in an e-learning context, can thus be defined as the effectiveness, efficiency and satisfaction with which users can achieve specified learning (or learning related) goals in a particular environment or with a particular tool or learning resource.Some regard usability as synonymous with 'ease of use'.The IMS Accessibility SIG defined 'accessibility' as the ability of the learning environment to adjust to the needs of all learners (IMS Global Learning Consortium, 2002).Accessibility is thus determined by the flexibility of the e-learning system or learning resource to meet the needs and preferences of all users.These needs and preferences may arise from their environment (e.g.working in a noisy environment), the tools they use (e.g.assistive technologies such as screen-readers, voice-recognition tools or alternative keyboards, etc.) or a disability in the conventional sense.
Accessibility and usability are intrinsically linked.The lower the level of accessibility of a resource for an individual, the less usable it will be for them.In the worst case they will not be able to use it at all.Conversely, improved accessibility for disabled users promotes usability for all.Usability should play an important role in accessibility testing, since a resource presenting usability difficulties will generally present significant accessibility problems for disabled users (Sloan et al., 2002).Even sites with a high level of accessibility can nevertheless have usability problems that may prevent people with disabilities from using them efficiently.
Most projects developing or exploring the use of technologies in education wish to see their approaches adopted in teaching and learning practice beyond the term of the project.Adequately addressing accessibility and usability in their developments enhances the possibility of achieving this.One important reason for this is that in most countries there is now anti-discrimination legislation 1 relating to disabled people's access to education.The introduction of e-learning technologies should not put barriers in the way of disabled students accessing their learning.Further, in the terms of the UK legislation, reasonable adjustments should be made to meet the needs of disabled students in accessing the curriculum.One such reasonable adjustment is addressing accessibility and usability issues in e-learning developments.If this is not adequately done, then a judgment has to be made as to whether that particular development can in fact be deployed.

Embedding accessibility and usability 233
Accessibility and usability impact directly on the pedagogical effectiveness of elearning systems or resources for all learners, but particularly for disabled learners.This should be reason enough for them to be addressed in all e-learning projects.It is invariably the case that addressing accessibility in the development phase is far more cost-effective than any retrospective accessibility response and is usually less costly and better pedagogically than the provision of an alternative but comparable learning experience for disabled students.The main challenge in accessibility is responding to the diversity of the ways different users interact with a computer environment.As well as being encouraged to follow established accessibility guidelines, developers need to be encouraged to always bear in mind that people interact with computers in different ways.There is globally a lack of in-depth knowledge and expertise in accessibility and usability available to project teams.

Evaluation methodologies
Evaluation is key to any research and development effort.If we do not evaluate, how do we know whether our developments achieved what we set out to achieve?Evaluation is also needed to enable iterative improvement.A range of different evaluations is required during a research and development project, including technical and functional evaluations.However, in reviewing how best to embed accessibility and usability in research and development projects, this paper focuses on evaluation of the end-user experience.This section discusses issues of designing and integrating accessibility and usability evaluation methods generally, and then outlines a methodology developed by the authors.Comments are made based on the experience of using and refining this methodology in different projects.

Methods
When planning an evaluation it is important that the associated aims are clearly articulated, as these will directly impact on the selection of methods and the associated design of the evaluation.Different methods are likely to be appropriate at different stages of the project depending on the objectives of the evaluation and the status of the prototype being evaluated at that stage.This section discusses a range of different approaches that are common in evaluation work looking at accessibility and usability.
In our work we define an expert evaluation as one undertaken by an accessibility or usability expert in which they make an assessment of probable issues for users in interacting with the prototype being evaluated.They will usually try and replicate a user's interaction with the website or software by working through scenarios or typical tasks expected of the users.In assessing the accessibility of a web site or software application, the expert will interact with the interface using a range of assistive technologies.Routinely in our work that will be a screen-reader, a screen magnifier and voicecontrol software.The expert will attempt to perform all user actions without the use of a mouse and test the response of the software to changes in browser or operating system accessibility settings, such as text size and colour contrast.
There are automated accessibility evaluation tools available such as the desktop application Bobby™ (Watchfire®) 2 and the online service A-Checker. 3These are designed to check web sites against the criteria of existing accessibility guidelines, including Section 508 of the US Rehabilitation Act of 1973 4 and the W3C's Web Content Accessibility Guidelines. 5 These tools have their place in supporting expert evaluations but, in our view, come with a 'handle with care' label.Such tools evaluate technical accessibility.They can be very useful in rapidly discovering certain accessibility problems, but they cannot assess from the user perspective.It is perfectly possible for an educational web site, say, to meet all the Web Content Accessibility Guidelines accessibility criteria as assessed by such tools but not to give a disabled student access to the learning.This follows from issues relating to usability, learning design and how these interact with accessibility.A simple illustration of this concerns the use of alt-texts 6 and images.Such technically focused tools will highlight where 'alt' attributes on image elements are empty but do not assess whether any text in an 'alt' attribute is pedagogically meaningful.Similarly, a learning resource may be technically accessible but its design means a screenreader user takes so long to navigate around it that they give up before achieving or even discovering the learning objective.This illustrates one of the central points of this paper that accessibility, usability and pedagogic issues are all interrelated in an e-learning context.
We believe that end-user engagement is vital, and we would strongly recommend that projects build in the opportunity to undertake end-user evaluation as part of their project plan.A range of methods is available for engaging end-users in an evaluation of prototype software, web resources or applications.These methods are designed to enable the researcher to elicit the user's experience including their behaviour, their perceptions and cognitive changes, their affective responses and their views.They are normally devised so that comparisons between the experiences of different users are possible.It has been our experience in evaluations undertaken that we always gain additional insights and reveal further usability and accessibility issues when we conduct evaluations with users, including users with disabilities.
Asking direct questions of the participants is a basic way of eliciting information about their interaction with a prototype under evaluation, although such instruments need careful design.In our work we make extensive use of semi-structured interviews straight after an observational session; with the former considering usability and accessibility criteria, and the latter focusing on validating whether the established criteria have been met.When undertaking observational studies, some means of recording the students' interaction with the object of evaluation is required together with their reaction to the experience.At our own institution we have a dedicated 'Data Capture Suite' for this purpose.The facilities allow for the synchronous recording of the user actions and the software behaviour on the screen together with video of their facial expressions and body language and audio recording of anything they say.Other tools may be useful; some researchers also use keystroke recorders and eye-trackers, for example.

Making research methods more inclusive
This section discusses a range of issues around promoting the effective engagement of disabled people in research including evaluation studies.As with most types of research, in accessibility and usability evaluations framing the right research questions is essential to the detailed design of the method, running of the evaluation sessions and subsequent data analysis.Research questions focus on the following aspects: 'to what extent is the learning resource, web site or software application accessible to disabled students?','how does this compare with usability for nondisabled students?','what are disabled students' opinions on …?' and 'how does this compare with those of non-disabled students?'However, it is important to note that some care needs to be taken in phrasing of these questions.For example, the question 'is this web site/software accessible?' is inappropriate because a given web site may be accessible to some people but not to others.Similarly there are different degrees of accessibility, which will also be influenced by additional tools the user might use to facilitate their access.
This section gives some of the findings from our own reflection on adopting an integrated usability and accessibility methodology.We try to apply a philosophy of continued improvement to our own methodology.Disabled participants often needed more time to complete a task; this is important in designing a learning resource and how it is to be used, but is also important in the evaluation design.There are noticeable different levels of interaction in an evaluation task between disabled and non-disabled users.Disabled people are often aware of the need for accessibility and focus at a 'technical' level; that is, whether they can get to the functionality.In other words they look for accessibility issues.Non-disabled people typically focus at a 'personal use' level (e.g.why would I want to use this website or software?).It seems to be the case that non-disabled users are more reluctant to be negative in their comments about the web site or software under evaluation.
There is very little literature on inclusion of disabled people in research.Accessibility research often focuses on questions of ease of access for disabled users; however, they need also to take account of the accessibility of the evaluation methods they use.Sampling is important, but can be challenging in this area.In any research into users' experiences, the more representative of the general target population the sample of users, the more likely the research is to reveal issues reflecting a diversity of users.Sampling of the population is a particular challenge in accessibility research.There is a wide diversity in the ways people with disabilities choose or need to interact with the computer.It is worth noting that it is the way an individual elects to interact with a computer environment that is important in accessibility research, not any medical classification of disability per se.Although we have undertaken studies with about 100 users, more typically we work with about 10 disabled users in a study.This is not sufficient to cover the functional requirements of all disabled people in their interactions with the computer; however, we select users who are likely to be particularly challenged by the application under evaluation.We supplement user evaluations with expert judgements and we believe that it is better to engage with 10 disabled people in an evaluation than none; even engaging with five disabled people can provide valuable insights.
Different methods raise different issues when undertaking accessibility research.An illustration of some of these is provided here.Observational evaluations raise a number of issues, particularly in terms of ensuring that an appropriate environment is provided for the end-user and any necessary adaptations for disabled students should be organised in advance.For example, assistive technologies and/or adjustable furniture may be needed, facilitators (sign-language interpreters or note-takers) may be required, written instructions may be required for hearing impaired participants or large print for visually impaired participants.It is often necessary for the researcher to be in the same room as the participant for communication with hearing impaired and speech impaired participants.Finally, it may be necessary to factor in more time for tasks depending on the needs of those taking part in the study.Focus groups raise additional issues; for example, it can be difficult to include people with visual/hearing/ speech impairments, and access issues for other disability groups (e.g.those with dyslexia if flip-charting/post-it techniques are used) need to be thought through.Surveys provide a different set of issues to those involving face-to-face interaction.The primary one is whether alternative methods can be offered (e.g.telephone/paper/ online surveys) to ensure maximum accessibility.Similar questions arise with interviewing-are telephone/email/conferencing alternatives possible?With technology field trials, can adaptations for mobile technologies be used or are participants allowed to use their own technology?Newer techniques also raise interesting issues in terms of the possibility they offer.For example, how might a method that includes eye tracking be adapted to be inclusive with visually impaired participants?In some cases tracking mouse trails or the screen-reader focus might be a way of collecting comparable data, but this is not yet a method evaluated by the authors.Remote evaluations over the Internet using virtual screen software in conjunction with various Internet communication tools are becoming increasingly popular.This approach is being adopted by some as a way of making disabled testers more readily available to researchers (e.g.Usability Exchange Service 7 ).
End-user evaluations can yield a body of rich and diverse data.Analysing this requires application of appropriate qualitative data analysis techniques.Accessibility and usability problems can be identified by end-users not being able to complete a set task, by unexpected behaviour of the system or the user, by direct reporting from the users, and so on.A more detailed discussion of qualitative data analysis techniques is outside the scope of this paper.However, evidently planning the analysis should be an integral process in devising the evaluation methodology for a given study.
When first seeking to integrate and further develop established accessibility and usability methodologies in our work at the OU, the following objectives were established in the approach we adopted: 1.To use, as far as possible, the same methods with disabled and non-disabled participants.2. To clearly identify usability and accessibility issues.
3. To produce a report in which there is no conflict between the recommendations for usability and accessibility improvements.4. To produce a report that is useful and usable by the developers in their subsequent work.
To a large extent all these have been met as our methodology has evolved.Objective one is often achieved by devising comparable methods rather than identical ones.By involving both disabled and non-disabled people in integrated studies of accessibility and usability, objective two has been achieved.By having accessibility and usability experts working together in the studies, objective three has been met.In fact, since adopting the integrated methodology no such conflicts have arisen.However, others have reported to us that in commissioning separate usability and accessibility studies they have received conflicting recommendations from each.Objective four has been achieved by producing, in the main succinct, bullet-point reports with clear numbered recommendations (so developers can treat them like a bug report), suggestions of techniques (so developers are not left with the question 'so what do I do about it?')and offers of clarification that promote an onward working relationship between the developers and the accessibility and usability experts.We are currently working on ways of enhancing the data collection opportunity in the precious time that we have the user in front of a prototype.We are seeking to reduce the dependence on video records to speed up data analysis and also to allow us more readily to extend our methodology to remote evaluations.We are also seeking to reduce non-disabled users' 'need to please' and encourage critical review.

Brief case studies of past and present projects
This section describes five projects either internal to the OU or in which the OU has participated, and reviews how accessibility and usability were addressed in each of them.The authors either were directly involved in or were called upon to advise and evaluate prototypes in all of these projects.

The Lyceum tool
Lyceum is a synchronous groupware communication tool that was developed by the OU and subsequently used widely in course delivery.It facilities online group working and was designed specifically for an educational context.It began life as a research project within the OU's Knowledge Media Institute 8 that ran from 1995 to 1998.Following successful trials in a course context in 1999, the system was handed over to Learning and Teaching Solutions, 9 who have developed and maintained it ever since.Lyceum supports interactions between tutors and students and between students and students in a variety of capacities, such as tutoring, group work, negotiation, collaborative writing and peer-to-peer communication.It facilities voice conferencing, virtual rooms, shared concept mapping and collaborative white-boarding.The authors have been involved in the evaluation of various applications of Lyceum in different course and project contexts but were not involved in evaluation of the original research project prototype.Lyceum is coming to the end of its life now and a tendering process is underway for a replacement.Lyceum is a good example of an e-learning project that then gets embedded into teaching and learning practice.This take up of the tool was because many of the course teams (particularly in languages) could see the pedagogic potential for their subject area.This move from research to practice, however, has not been without its challenges, particularly with respect to accessibility issues.The original Lyceum project did not include work on accessibility issues, as the researchers were concentrating at the time more on the pedagogic potential of the technology.Furthermore, the Lyceum development work occurred before the WCAG 1.0 Web Content Accessibility Guidelines were published, at a time when accessibility had a much lower profile.The unfortunate consequence of not considering accessibility in the initial project was that considerable re-engineering had to be done when it was handed over for mainstream use in order to meet even basic accessibility criteria.This substantially increased the development costs of Lyceum and delayed the release date of the software.However, improved accessibility has been one of the objectives of subsequent software upgrades.

OpenMark-S for e-assessment
OpenMark-S was an internal OU project concerned with the development of online formative and summative assessment.In particular, it supports the creation and delivery of interactive questions that go beyond the constraints of simple multiple-choice questions.The project has made accessibility a high priority, and the second author was involved in conducting evaluation of prototypes with disabled students.An important lesson from this project was that the point when software is being upgraded provides a good opportunity to address any accessibility deficits in the versions.In this project the second author was involved as an accessibility expert from the early stage of the development phase of the project.A close working relationship was fostered between the developer and the accessibility expert with frequent contact.Because of this and because of the project design, frequent iterations were possible throughout the developments of the project.

The DiVA project
DiVA, the Digital Video Applications research project, ran from 2000 to 2004. 10 The project investigated the use of digital video within various academic contexts.This included using the DiVA system to locate clips from existing television programmes for re-use in other media, the pedagogical impact of the use of video and whether the DiVA system could improve access to video material for hearing impaired users.Central to the project was the evaluation of the use and impact of digital video in these contexts.The evaluation findings were incorporated into a final report delivered in May 2005 that made recommendations for the future of digital video within the OU.DiVA was a good example of a project that included research into the needs of disabled Embedding accessibility and usability 239 students from the outset.Included in its research agenda was the objective of exploring how best to present transcripts, especially for people with a hearing impairment, alongside video material.A key lesson to be drawn is the value of formulating specific research questions relating to the needs of disabled students from the outset.

The OU-VLE (Moodle) programme
The ongoing OU-VLE Development Programme is developing a VLE 11 for the OU integrating a range of computer-mediated services around a courseware server based on the Open Source 12 (OS) course management system Moodle. 13It consists of a range of specific projects focused on different aspects of developing the VLE.These include the Integrated Online Experience Project (portals and personalisation), ePortfolios, 14 eAssessment (online formative and summative computer-marked assessment), online collaboration and communications (asynchronous and synchronous tools such as Blogs, Wikis, web conferencing, instant messaging, etc.), mobile learner support and mathematical and scientific content.The system is now in use by courses across the OU.A second major production release is scheduled for February 2008.Accessibility has been a key priority, and the first author has been charged with overall responsibility for accessibility across all the projects within the programme.The second and third authors are undertaking a series of expert and end-user accessibility and usability evaluations of different components of the VLE as they reach usable prototypes.Moodle is an OS community development and the OU is investing considerable resources and effort to enhance the accessibility and usability of the system.There are advantages and disadvantages in addressing the accessibility agenda in an OS context as opposed to a propriety one; when a user detects an accessibility problem in an OS application, a fix will often occur more rapidly than with propriety software; however, the inherent nature of OS development-making software publicly available in early versions so the community can contribute to its further development-often means that products are released with accessibility poorly addressed.Key accessibility benefits follow from consistency of interface design and behaviour across an application.With a more loosely coordinated and distributed team of developers contributing to an OS product there is a risk that this consistency is not achieved.Where an OS development results in the rapid release of updated versions of the software, any changes in interface design and behaviour can be more problematic for some disabled users as it may take them longer to discover and understand such changes.
The OU undertook a detailed review of many of the then available VLEs prior to selecting Moodle.It found that most addressed accessibility poorly.In selecting Moodle, although there were known accessibility problems, because it was an OS community project, the OU felt it could contribute to addressing these.The first two authors and a developer formed the core of a working group to draw up a Moodle Accessibility Specification. 15This document gives general information to facilitate developers addressing accessibility and gives a prioritised list of specific issues to be addressed to improve to the accessibility of Moodle.It was based on a detailed expert evaluation of many (but not all) features in Moodle version 1.6.The document was posted to the openly accessible Moodle Docs 16 site (so that the knowledge could be shared across the Moodle community), and developers at Moodle.com were then commissioned to undertake the software changes necessary to meet most of the highpriority issues identified in the specification.The second author has been undertaking reviews of these changes to confirm that the accessibility benefits sort have indeed been achieved.This and other accessibility and usability enhancements will, subject to evaluation, be incorporated into future versions of Moodle released and will thus benefit all Moodle users.
The OU-VLE Development Programme represents a highly complex set of interrelated projects.Addressing accessibility across these has required action at both a strategic and project level, as well as at the fine detail of the specific developments.There is still much to do but the current status in terms of accessibility and usability issues is summarised here.Steady progress is being made on making Moodle more accessible, although there still are accessibility issues.Aspects of the VLE are not currently as usable as they might be, which is an issue for all students, but particularly affects students with disabilities, and usability studies have shown that some interfaces are not sufficiently intuitive.There is also a lack of consistency across the different component web sites that have very different interaction styles.With this in mind we recognise that we need more student testing, on a more regular basis, in a more timely fashion, in order to ensure that we have a fully accessible and usable VLE.A key challenge has been how to integrate development work and evaluations in such as large-scale venture.Development work regularly over-runs its target delivery date.However, there is a significant lead-time (typically six to eight weeks) required to plan a set of end-user evaluations.This is because of the need to recruit participants in advance and to plan the detailed methodologies.An end-user evaluation requires a working prototype without too many bugs but further requires appropriate content to populate the prototype to design meaningful tasks for the participants to undertake.Delays in the development of prototypes and the release of appropriate content has resulted in evaluation sessions that have gained less insights than might otherwise have been possible.The experience of scheduling issues in earlier phases of the OU-VLE work has meant that we are now leaving much more time for testing in the forthcoming development phase.

The EU4ALL project
The European Unified Approach for Accessible Lifelong Learning EU4ALL 17 project started in October 2006.This is a major, four-year project with overall funding of US$10 million (US$1.6 million allocated to the OU).The project addresses systemic issues in providing access for disabled learners to Lifelong Learning. 18It sets forward the concept of Accessible Lifelong Learning uniting three key strategies: • Technology that mediates lifelong learning does so in an accessible way.
• Technology is used to bring specialist support services to disabled learners.

Embedding accessibility and usability 241
• Support services and a technical infrastructure are provided to enable staff at educational institutions to more readily offer their teaching and services in a way that is accessible to disabled learners.
The aim of EU4ALL is to improve the efficiency and efficacy of implementing these strategies by developing an open service architecture.To achieve a wide impact, the approach taken is not to develop a single EU4ALL system but a standards-based framework that facilitates the integration of the approach with a wide range of elearning systems.This will be validated by integration with two OS VLEs: dotLRN 19 and Moodle.EU4ALL is in the early stages so comments here are confined to issues associated with project design.Our prior experience of projects of this kind suggests usability and accessibility should be integrated across the different phases of the project (requirements capture, specification, design, development and validation).In such a large-scale project this means integrating this work in the planning stage between many of the subprojects.This is a challenge in project design, and subsequently project management.The EU4ALL project is addressing many issues of accessibility at a systems level rather than just at the interface level.This includes personalisation to the needs of different users with disabilities and the automated serving of alternative resources where required.Designing end-user evaluations that can contribute early to the iterative development of these is challenging.This follows from the fact that many technical issues need to be addressed to make these systems work before a meaningful user experience can be created for evaluation.

Recommendations
This section gives a set of recommendations for future projects based on our analysis of the experience in mini-case studies described in this paper.We have argued that evaluation has an essential role in embedding of accessibility and usability into a project plan.Specific numbered recommendations are listed here under different areas for consideration.

Project proposals
1.The rationale for usability and accessibility should be clearly stated in the proposal.
2. There should be appropriate allocation of how the usability and accessibility work will be resourced in the project.3. Usability and accessibility should be reflected in the project research questions, which should include something that involves an investigation of usability for all and accessibility for disabled students.

Project design, development and evaluation
4. Accessibility and usability criteria need to be built into the specification. 5. Accessibility and usability experts need to be brought in the earliest possible stage.
6.If a use-case-based approach for arriving at a specification is adopted, then usecases should be developed that include disabled people as 'actors'.7. If integrating with third-party software, its accessibility and usability needs to be reviewed.8.The project should address the fact that accessibility and usability are system design issues, not just interface design issues.9.An integrated evaluation plan should be adopted that includes accessibility, usability and educational evaluation.10. Outline methodologies for the planned evaluations need to be produced earlythis will effect the detailed planning of the developers work.11.Accessibility issues associated with any data collection methods used need to be fully considered and accommodations devised to meet the meets of disabled people as necessary.

Iterative development
Iterative design is essential to any software development project; however, making this effective requires a close integration of the work carried out by the developers and the evaluators involved in the project.Therefore, the following issues need to be addressed early in any project: 12.Where in the project plan are expert and end-user evaluations most appropriate?13.Sufficient time and development resource allocation after an evaluation is required to respond to any recommendations that arise from the evaluation.14.To be effective end-user evaluations, the prototypes under investigation needs to be populated with appropriate content and activates for the users to interact with in a realistic way.15.End-user evaluations require a reasonable lead time to facilitate the recruitment of participants and the preparation of the detail of the evaluation methods, which will be dependent on the content and activities.16.Development work almost inevitably takes longer than anticipated.Contingency plans need to be in place to deal with this.

Successfully project managing for usability and accessibility
How successfully issues relating to the end-user experience are addressed in a project depends on how accessibility and usability are managed within the project.This is because accessibility and usability issues transcend many areas of a project.It is appropriate to liken responsibility for accessibility in a project to health and safety at work policies, where everyone has responsibility for health and safety but specific duties and areas of responsibility are delegated to named individuals.Similarly, everyone in an e-learning research and development project team has to pay due regard to accessibility and usability issues.However, experience in past projects has shown that it is often advantageous to have a named member of the project team with Embedding accessibility and usability 243 overarching responsibility for accessibility and usability.This person should have the permission of the project team to repeatedly ask 'what are the accessibility and usability implications for this?' The problem of integrating evaluation work with development schedules that often slip has been discussed above.Probably the only way to address this is by detailed initial planning and periodic progress and planning reviews.Close communications between development and evaluation teams is important so that early notice is given if slippages are anticipated and plans can be adjusted accordingly.There is inevitably a tension between developers and evaluators of software.The former perceive the latter of criticising work in which they have usually invested significant amounts of effort and professional expertise.The evaluators see the developers as reluctant to respond to their recommendations.Better relations and a better outcome for the project are promoted if an integrated team approach is fostered; with developers and evaluators seeing each other as partners with a common goal, calling on each others' areas of expertise as necessary.To achieve this, evaluators should be involved throughout the project, participating in the specification work from the outset and being available to developers at any point.For example, a developer may ask a question of the form: 'I was thinking of implementing <a function> this way, what do you think will be the accessibility and usability implications of that?' Conversely, an evaluator might ask: 'I have been looking at how we have implemented < a function>, can you advise me whether it can be adapted to incorporate <an approach> that I think will promote accessibility?'.

Concluding comments
In summary, a few of the more important points from the discussion of this paper are highlighted here.We argue that accessibility, usability and pedagogic issues are all interrelated in an e-learning context.Accessibility and usability issues need to be addressed throughout a project's lifecycle to ensure its developments are subsequently adopted in educational delivery.Evaluations are key to ensuring accessibility and usability issues have been addressed in a project's developments.Valuable insights can be gained when conducting evaluations with a range of users leading to overall improvements in the system being developed.We believe that integrating accessibility and usability evaluations yields distinct benefits over treating them as separate areas for study.We have described how care is needed when drawing up evaluation methodologies, to ensure that they are inclusive and consistent for both disabled people and non-disabled people acting as participants.However, we have also discussed how accessibility and usability raise fundamental issues for how projects are organised and run.
We believe that enhancing the end-user experience of an e-learning system or resource, by comprehensively addressing accessibility and usability, impacts positively on the effectiveness of the learning, and hope that we have illustrated why this should be a key reason for giving them due consideration, effort and resources in any e-learning project.