Assuring best practice in technology-enhanced learning environments

This paper documents the development and findings of the Good Practice Report on Technology-Enhanced Learning and Teaching funded by the Australian Learning and Teaching Council (ALTC). Developing the Good Practice Report required a meta-analysis of 33 ALTC learning and teaching projects relating to technology funded between 2006 and 2010. This report forms one of 12 completed Good Practice Reports on a range of different topics commissioned by the ALTC and Australian Government Office for Learning and Teaching (OLT). The reports aim to reduce issues relating to dissemination that projects face within the sector by providing educators with an efficient and accessible way of engaging with and filtering through the resources and experiences of numerous learning and teaching projects funded by the ALTC and OLT. The Technology-Enhanced Learning and Teaching Report highlights examples of good practice and provides outcomes and recommendations based on the meta-analysis of the relevant learning and teaching projects. However, in order to ensure the value of these reports is realised, educators need to engage with the reports and integrate the information and findings into their practice. The paper concludes by detailing how educational networks can be utilised to support dissemination.


Context
The increasing availability and access to new technologies has focussed attention on technology-enhanced learning (TEL) and the possibilities now open to educators.As noted by Weller (2011), new technologies have the potential to reshape all scholarly areas.The area of teaching holds great opportunities for radical change to emerge in regards to the application of new technologies.With access to online content and new modes of interaction and engagement, traditional forms of teaching are being challenged and new models are emerging such as MOOCs (massive online open courses) that are generating discussion across the sector.However, despite increasing hype and interest there remains a disconnect between technologies, research, design and practice (Wang and Hannafin 2005).This has meant that whilst the potential for radical change to teaching persists, issues in disseminating good practice, research findings and effective e-learning designs continue to impede the rate of successful transformation in teaching and learning.Although dissemination remains an ongoing issue, 'the field is beginning to recognise that teachers need to help each other to discover how best to organise the mix of learning technologies in support of learning' (Beetham and Sharpe 2013, p. xvii).Whilst this may be recognised, common models for promoting and sharing good teaching practices (such as workshops and awards) are impeded by the problematic nature of dissemination across the sector; a necessity if good practices are to have widespread uptake (Southwell et al. 2005).
In an investigation of Australian and international learning and teaching grant schemes, Southwell et al. (2005) explored how dissemination might be improved to bring about change.They identified the following five conditions for successfully implementing, embedding and up-scaling an innovation: (1) Effective, multi-level leadership and management (2) Climate of readiness for change (3) Availability of resources (4) Comprehensive systems in institutions and funding bodies (5) Funding design that demands, encourages and supports risk-taking, change and dissemination (Southwell et al. 2005, p. 3) The findings from this report and another by McKenzie et al. in 2005 informed the design of the Australian Learning and Teaching Council (ALTC) Dissemination Framework which was in turn reviewed for effectiveness by Gannaway et al. (2011).
The following findings emerged from the Gannaway et al. review: (1) Clearer definition of key terms is required if the sector is to move forward in the quality and effectiveness of dissemination (2) The ALTC Dissemination Framework ( 2006) is not a sufficient mechanism for supporting an understanding of effective dissemination (3) The most popular method of communication of project outcomes remains traditional academic modes such as conference presentations, book publications and publication in academic journals and conferences (4) A web presence is a common dissemination activity, but its potential impact is limited due to maintenance issues after the project concludes (5) There is an espoused understanding amongst grant recipients that dissemination activities occur during the life of the project, rather than after the project concludes (6) Dissemination is commonly described as a collection of atomistic activities, rather than as a clearly planned strategy designed to achieve a particular purpose (7) ALTC grant holders tend to equate the end of the grant with an end of their involvement in that topic, possibly as a result of ALTC project work being viewed as an additional workload (8) It is not possible to accurately measure or determine evidence of long-term changes that may have occurred as a result of the project based on the current reporting mechanisms (9) Projects that have successfully embedded and upscaled have identified and engaged with potential adopters from the outset (10) Project leaders grapple with identifying, articulating, and responding to or developing a climate of readiness for change (11) The ALTC is perceived as having an obligation to support the dissemination of project outcomes through providing a searchable repository for project deliverables and facilitating opportunities for making links between projects (12) Successful dissemination strategies have multiple layers of change enablers who facilitate dissemination (Gannaway et al. 2011, p. 2) Thus, it is acknowledged that the dissemination of project innovations, experiences and findings from learning and teaching projects has an important role in improving sector-wide practice.Within this context and with the aim of providing the sector with a concise and accessible means of engaging with the outcomes, innovations and experiences from ALTC and Australian Government Office for Learning and Teaching (OLT) funded learning and teaching projects, a number of Good Practice Reports have been commissioned since 2011.These reports provide syntheses and reviews of learning and teaching projects and fellowships funded by the ALTC and OLT.
Twelve reports had been published on different topic areas between 2011 and August 2014.These reports were commissioned to support the sharing of educational practices, resources and experiences from numerous projects through the development of palatable topic-bound reports.

Defining TEL and teaching
The first step in the process of writing the Technology-Enhanced Learning and Teaching Report involved developing a common understanding of what the concept of TEL meant to us as the authors.Our perspective of TEL and teaching aligned with that of Laurillard et al. (2009) who viewed technology as a means of supporting new types of learning experiences as well as enhancing existing learning contexts.Moreover, they held the view that 'interactive and cooperative digital media have an inherent educational value as a new means of intellectual expression and creativity' (Laurillard et al. 2009, p. 289).Building on these understandings, Laurillard et al. (2009, p. 304) suggested that 'the route from research to innovation, then to practice, through to mainstream implementation requires the following: An understanding of the authentic professional contexts that will influence the curriculum, pedagogy and assessment practices that need technology enhancement

Congruence between innovation and teacher values
Teachers having time to reflect on their beliefs about learning and teaching because TEL requires a more structured and analytical approach to pedagogy Teachers and practitioners need a sense of ownership through their involvement in co-development of the TEL products and environments TEL research must be conducted to reflect the interdependence between researchers and users Education leaders need more support for the radical change of institutional teaching and learning models needed, if technology is to be exploited effectively Teachers need to be more closely engaged in the design of teaching that uses technology, collaborating with peers and exchanging ideas and practices' This understanding of TEL and teaching underpinned the development of the Good Practice Report on TEL and Teaching.

Approach
The approach to developing the report involved the construction of a matrix which facilitated the comparison and thematic analysis of project information including final project reports and online resources.The 33 projects reviewed included 22 complete projects and three completed fellowships as well as seven on-going projects and one on-going fellowship on TEL and teaching.The authors examined the projects individually over sustained periods of time and became 'saturated' in each project from which the key themes and outcomes were developed.Subsequent analyses determined 10 outcomes representing best practice in TEL and teaching which were detailed in the Good Practice Report (Keppell, Suddaby, and Hard 2011).These outcomes represent the key themes related to best practice that surfaced during the analysis of the 33 two-year, multi-institutional projects.Critical reflection and triangulation of the views of the three authors strengthened the outcomes developed.
Ten best practice outcomes 1.A focus on learning design allows academics to model and share good practice in learning and teaching Learning design has been defined in numerous ways.In this context it was understood as 'a methodology for enabling teachers/designers to make more informed decisions in how they go about designing learning activities and interventions, which are pedagogically informed and make effective use of appropriate resources and technologies.This includes the design of resources and individual learning activities right up to curriculum-level design' (Conole 2013, p. 8).In line with this definition, the process of reviewing the projects found learning design to be primarily reflected in the design of activities, subjects, assessment and curricula.For example, one project developed a database which provided open-access to a large number of technologybased learning activities (Oliver 2008).Projects varied in scope from discipline specific to multi-disciplinary and generic approaches to design.Projects which effectively implemented approaches focused on learning design understood the importance of supporting academics to share and model good practices.Projects which considered these elements from the beginning were also more successful in meeting their goals.
2. Authentic learning provides a means of engaging students through all aspects of curricula, subjects, activities and assessment Authentic learning is meaningful as it engages and immerses students in real world activities, subjects, assessment and curricula that are directly relevant for their profession and allows a transition to the workplace setting.In examining such projects the specific focus was on the four key principles of authentic learning based on the work of Herrington, Reeves, and Oliver (2010): Providing an authentic context that mirrors the way the knowledge will be used in real life

Supports collaboration
Provides coaching and scaffolding by the teacher Provides authentic assessment The importance of developing authentic learning experiences through the assistance of technology to enrich student learning was evident in the projects reviewed.Projects focused on authentic learning immersed students in process 3D VR engineering plants (Cameron et al. 2009); used e-simulations and role plays (Cybulski et al. 2010;Wills et al. 2009); and designed exemplars of Web3D environments for educational use that mirrored real-life activities (Albion and McKeown 2010).Authentic learning was aligned to real-world applications in each of these projects and attempted to bridge the gap between theory and practice.
Motivating and engaging students were key priorities in the design and approach of these projects.

Successful academic development in projects focuses on engaging academics over sustained periods of time through action learning cycles and the provision of leadership development opportunities
There are numerous possible approaches to academic development, ranging from traditional seminars and workshops through to sustained involvement in action learning, leadership development projects and fellowships.The changing global context of learning and teaching in higher education, the nature of students, and the impact of technology on learning and teaching all influence the need for effective professional and academic development.Whilst many approaches to academic development were illustrated, action learning including cycles of action and reflection relating to professional practice appeared to offer a successful means of supporting academic development.One project in particular successfully adopted the use of mobile technologies to engage participants in adopting an action learning approach to professional development (Herrington et al. 2008).Another project adopted a viral leadership development model in building a network centred on a range of web-based resources and networking activities (Cluett 2012).However, the challenge remains to embrace sustained and relevant academic development, collegial networks, and leadership development opportunities to ensure staff are engaged and supported in technology-enhanced practices.

Engaging teaching approaches are key to student learning
Student engagement is a critical factor in good teaching and learning.Krause (2005) suggested that student 'engagement refers to the time, energy and resources students devote to activities designed to enhance learning at university' (p.3).Whilst Coates (2007, p. 122) (Christie 2010).However, aligning pedagogical, technical and administrative issues remains a necessary condition of success in creating an engaging learning environment.
5. Technology-enhanced assessment provides flexible approaches for academics to provide feedback to students Assessment plays a key role in teaching and learning.Assessment should be designed to identify the quality of students' learning and how teaching can be more effective.
A core element of assessment is that it must engage students and provide feedback that can be acted upon by the student to improve future learning.Moreover, students and teachers must become partners in both learning and assessment processes which encourages life-long learning practices (Boud and Associates 2010).The addition of technology provides opportunities for flexible, diverse and interactive approaches to student assessment.In the online environment, assessment-centred activities should focus on formative and learning-oriented assessment activities (Keppell and Carless 2006).Assessment was an important element of the projects reviewed and the explicit focus of many.For example, projects explored the use of e-Assessments (Crisp 2008(Crisp , 2011)); assessment using 3D virtual worlds (Gregory et al. 2013); online clinical assessment (Engstrom et al. 2011); and the development of templates for mathematical questions and worked examples (Adams et al. 2008, p. 3).Colbram (2009Colbram ( , 2010) ) developed a method for providing feedback on electronic assessment submissions and Crisp (2008Crisp ( , 2011) ) enhanced approaches to e-Assessment through the articulation of an e-Assessment design model and a collection of disciplinary examples.
6. Integrating TEL and teaching strategies across curriculum, subjects, activities and assessment resulted in major benefits to the discipline Wang (2008) suggests a model of ICT integration that includes pedagogy, social interaction and technology focusing on the pedagogical affordances, social affordances and technological affordances when designing TEL and teaching environments.
In combination with the elements, we found that effectively integrating TEL and teaching across curriculum, subjects, activities and assessment was critical to the sustainability and efficacy of projects.Successful projects were able to embed their initiative across the curriculum, IT infrastructure and professional practice of academics which led to more sustainable change than if the initiative was confined within a course or subject.For example, the virtual microscopy project (Kumar et al. 2009) worked with course coordinators to maximise integration across courses, whilst the Web3D project (Albion and McKeown 2010) adopted an action learning approach with staff which proved highly effective in empowering academic staff in the use of virtual worlds.The eSimulation project (Cybulski et al. 2010) integrated simulations into the curriculum of three universities, and the engineering project (Cameron et al. 2009) focused on integration with a focus on curriculum review, graduate attributes and capabilities in line with industry expectations.
7. Knowledge and resource sharing are central to a vibrant community of practice Wenger (2009, p. 1) describes communities of practice as '. . .groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly'.To create a vibrant community of practice within higher education a culture of knowledge and resource sharing needs to be cultivated.Whilst networks and communities have many forms, localities and methods of interaction there is considerable interest is the use of technology to enhance communication and engagement in developing such communities and networks (JISC 2009).Many of the projects reviewed sought to foster such communities of practice with the aid of technology-enhanced methods of communication.Projects such as EnROLE (Wills et al. 2009) and Learning to Teach Online (McIntyre 2011) demonstrate how effectively educational resources can be developed and disseminated in ways that support a strong and engaged community of practice which readily shares skills, knowledge, expertise and pedagogies.The Learning to Teach Online project successfully used a range of online resources as catalysts for engagement and was able to see their effectiveness in the resulting Twitter and blog activity (McIntyre 2011).Meyer (2011) demonstrated how the development of an online repository for histology materials could bring together histology teachers from across the globe to access as well as develop and share related learning materials.However, a project on the adoption, use and management of open educational resources (OERs) discusses the on-going resistance to the OER movement, indicating that whilst the sharing of resources, particularly OERs, may offer many potential benefits to academics and the sector, it remains in its early stages (Bossu, Brown, and Bull 2014).
8. Academics need sophisticated online teaching strategies to effectively teach in higher education environments Teachers need to be conversant in a wider range of skills to teach effectively in online environments.These skills focus on being able to engage the learner in an effective learning environment that is learner-centred, knowledge-centred, assessment-centred and community-centred (Bransford, Brown, and Cocking 1999).A learning-centred online environment involves understanding the learner in terms of their attitudes, knowledge, perspectives and preconceptions about learning in an online environment.Additionally, teachers need to have 'technological pedagogical knowledge', 'an understanding of how teaching and learning change when particular technologies are used' (Harris, Mishra, and Koehler 2009, p. 398).A number of projects implemented targeted online teaching strategies which aimed to foster self-directed learning and improved learning outcomes.For example, one project developed, implemented and promoted an online learning environment (website) dedicated to improving student writing in the sciences and engineering (Drury and Jones 2009).The development of Deutsch e-rkla ¨rt (German e-xplained) sought to support German language students to become more autonomous learners Á providing essential elements of an introductory language course and strategies to support students to take control of their own learning (Dunne et al. 2009).The VirtualPREX project helped prepare pre-service teachers for professional experience opportunities and professional life through the use of 3D virtual worlds (Gregory et al. 2013).
9. Academics need a knowledge of multi-literacies to teach effectively in contemporary technology-enhanced higher education Rapidly evolving communications media and the abundance of different platforms for learners to share and shape meaning necessitates new forms of digital, cultural  (Pullen, Gitsaki, and Baguley 2010, p. xiii).Beetham (2010) defines being digitally literate as possessing 'the functional access, skills and practices necessary to become a confident, agile adopter of a range of technologies for personal, academic and professional use'.Educating the Net Generation (Kennedy et al. 2009) explored the skills and experience of both students (net generation) and lecturers with the use of established and emerging technologies.They found that there was little empirical evidence to support the notion that students are digital natives and staff are digital immigrants.They also noted that new technologies provide a range of new learning possibilities that can be beneficial to student outcomes Á however, these possibilities typically require new skills in TEL and teaching for both educators and students.
10. Exemplar projects focused on multiple outcomes across curricula integration, sustainable initiatives, academic development and community engagement Ten exemplar projects were identified as demonstrating best practice.These projects clearly exhibited at least two of the nine outcomes for good practice identified in the report.

The good practice reports
The Technology-Enhanced Learning Report was published in late 2011 and forms part of a collection of 12 Good Practice Reports commissioned by the ALTC and subsequently by the OLT (see Table 1 for a complete list of reports).The Good Practice Report format affords the sector and individual educators value by: Providing easy access to relevant literature, syntheses of key concepts and innovative approaches through grouping them relevant to specific topic areas Highlighting key challenges faced in the successful design, implementation and maintenance of learning and teaching projects Highlighting effective solutions to common problems faced by educators and those leading learning and teaching projects as well as insightful recommendations for consideration in the design and delivery of future programs and interventions (opportunity to learn from the experiences of others) Being compact and accessible, providing individuals with a rich and practical resource which can act as a gateway for exploring the range of projects funded through the ALTC/OLT.

Dissemination
Despite the clear value of these Good Practice Reports and their applicability across disciplines and the sector in general, evidence of the widespread awareness and uptake of the reports by educators has remained limited.Anecdotal evidence collected during webinars, conference presentations and discussions with colleagues indicated a relatively low level of engagement with these resources by the sector.Despite numerous efforts to better understand the complex issues associated with effective dissemination, raising awareness of ALTC and OLT learning and teaching projects and their outcomes through the publishing of the Good Practice Reports remained an on-going challenge.The Network of Australasian Tertiary Associations (NATA) was developed in 2011 in response to the closure of the ALTC and sought to continue the critical roles of sharing best practice, promoting collaboration and collegiality and engaging academic staff in the scholarship of learning and teaching.This role was considered important in light of the work by Gannaway et al. (2011) which highlighted the problematic nature of dissemination in this area at the time.The NATA sought to facilitate a sustainable collaborative network or 'supranet' between established higher education associations in order to foster best practice in networks by engaging members more strongly with learning and teaching in Australasian higher education (Network of Australasian Tertiary Associations 2014).A key component of the network's role was to utilise its diverse networks and communication channels to promote and disseminate the outcomes of ALTC and OLT funded projects including the Good Practice Reports.In addition to drawing upon its partner associations to support dissemination, the project team established an active website, conducted conference presentations and developed asynchronous eResources which emphasised the key elements and value of individual Good Practice Reports (Network of Australasian Tertiary Associations 2014).

Conclusion
In examining the projects and investigating the literature, it became evident that a range of conditions and factors contributed to successful project outcomes.Analyses of the 33 ALTC TEL projects identified a number of inspiring approaches and pedagogies that have informed or been developed by the projects themselves.These factors provided the basis for the development of a set of 10 outcomes representing a framework for success for implementing TEL initiatives.These outcomes highlighted the importance of focusing on learning design, authentic learning, academic development, engaging teaching, flexible assessment, widespread integration, communities of practice, academic multi-literacies and strategies for teaching online in supporting successful TEL and teaching projects.
To successfully implement TEL strategies, leaders need to ensure that a range of relevant and practical engagement techniques are in place so that such strategies become standard practice.TEL strategies and practices should be integrated and not confined to specific subjects or projects in order to support widespread and sustainable change.Academics, teachers, learners and researchers all have the ability to influence the effectiveness of TEL strategies and thus, must be actively engaged.Our findings illustrate the need for academic practitioners to be competent in the use of technology if the boundaries of best practice in learning and teaching in higher education are to be extended.This highlights the importance of academics being equipped with multi-literacy skills.Consequently, institutions need to provide authentic training and professional development opportunities for staff, and provide strong incentives for staff to attend and develop skills to support TEL.
The paper showcases the potential benefits of individual projects and seeks to encourage readers to engage with the range of Good Practice Reports published.Whilst the sector is beginning to value these resources and the need for academic sharing, it is clear that the current percolation-type approach to dissemination is hindering the realisation of this potential.Although the dissemination of learning and teaching resources has been the explicit focus of several ALTC and OLT projects, it appears to still remain a key challenge in realising the full value of learning and teaching projects across the sector.Adopting a network-based approach to dissemination, NATA engaged professional organisations in establishing a collaborative network.By building collaborative networks such as the NATA it is hoped that project outcomes may be more readily shared, rely less on changing government structures and more on communities of practice to facilitate the integration of good practice principles into learning and teaching to benefit tertiary education students Á our future leaders.It is also essential that curriculum renewal is accompanied by appropriate and authentic professional development that integrates the good practice principles and empowers teachers.
provides a more practice-based description, defining student engagement as 'active and collaborative learning, participation in challenging academic activities, formative communication with academic staff, involvement in enriching educational experiences, and feeling legitimated and supported by university learning Lowe et al. 2008)9n 2010;Cybulski et al. 2010;Wills et al. 2009)pproaches including the development of strategies and exemplars of socially-oriented technologies(Fitzgerald and Steele 2008;Herrington et al. 2008); the use of gaming, virtual worlds and simulation approaches(Albion and McKeown 2010;Cybulski et al. 2010;Wills et al. 2009); the use of remote laboratories and virtual microscopy(Kumar et al. 2009;Research in Learning TechnologyCitation: Research in Learning Technology 2015, 23: 25728 -http://dx.doi.org/10.3402/rlt.v23.25728Loweetal. 2008); as well as supporting indigenous teachers to teach from 'in country' McLoughlin 2011).Thus, being 'literate is vital for learning and working, possibly more so in the digital age than in the industrial age, given society's reliance on digital technologies' Citation: Research in Learning Technology 2015, 23: 25728 -http://dx.doi.org/10.3402/rlt.v23.25728 and communicative literacies (

Table 1 .
Good practice reports