aCenter for Learning Sciences and Technologies (CELSTEC), Open University of the Netherlands, Heerlen, The Netherlands; bTwente University, Enschede, The Netherlands
(Received 11 February 2012; final version received 10 June 2013; Published 16 September 2013)
Abstract
Internationally, society is increasingly demanding that the relevance and practical applicability of research be made transparent. Despite intentions to the contrary, insights on pedagogically appropriate uses of educational technology for representative teachers in everyday school settings are severely limited. In part, this is because (design) research is often conducted at the bleeding edge of what is technologically possible – exploring innovative uses of new and emerging technologies. There is no disputing that such work is greatly needed to seek out new ways to potentially enhance the quality of teaching and learning. However, in the excitement of exploring what is possible, tomorrow, insufficient research and development work focuses on what is practical, today. This leaves a problematic gap between what could be effective technology-enhanced learning (TEL) in theory, and what can be effective TEL in practice. This paper calls for designers/researchers of TEL to devote attention to not only fine-grained issues of pupil learning and instruction but also to broader factors that determine if and how innovations are understood, adopted and used by teachers and schools, by designing innovations to align with their zone of proximal implementation. Methodological considerations are given for designing and studying interventions that are prone to implementation by being: value-added, clear, harmonious and tolerant.
Keywords: learning design; implementation; innovation
*Corresponding author. Email: susan.mckenney@ou.nl
Research in Learning Technology 2013. © 2013 S. McKenney. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) Licence (http://creativecommons.org/licenses/by/3.0/) permitting use, reuse, distribution and transmission, and reproduction in any medium, provided the original work is properly cited.
Citation: Research in Learning Technology Supplement 2013, 21: 17374 - http://dx.doi.org/10.3402/rlt.v21i0.17374
Society in general and research foundations around the globe such as the European Research Council and the (American) National Science Foundation are increasingly calling for the relevance and practical applicability of research to be made transparent. Despite intentions to the contrary, research on technology-enhanced learning (TEL) that truly serves current educational practice is more rare than it is common (Reeves, Herrington, and Oliver 2005). Insights on pedagogically appropriate uses of educational technology for representative teachers in everyday school settings are severely limited. In part, this is because (design) research is conducted at the bleeding edge of what is technologically possible – exploring innovative uses of new and emerging technologies. There is no disputing that such work is greatly needed to seek out new ways to potentially enhance the quality of teaching and learning. However, in the excitement of exploring what is possible, tomorrow, there is insufficient research and development work focusing on what is practical, today. This leaves a problematic gap between what could be effective TEL in theory, and what can be effective TEL in practice. With the aim of generating “usable knowledge” (Lagemann 2002) and creating innovations that truly serve learning in practice, this paper calls for designers/researchers of TEL to devote attention to not only fine-grained issues of pupil learning and instruction, but also to broader factors that determine if and how innovations are understood, adopted and used by teachers and schools. Allowing these issues to steer the design of TEL innovations is necessary to yield innovations that can feasibly be implemented outside of (often highly enabling) research and development trajectories. Accepting that radical innovation in education is not possible (Berliner 2002), largely due to its high degree of uncertainty (Kenny 2002), doing so constitutes a form of incremental innovation. Throughout this paper, incremental innovation targeted at what teachers and schools can implement with realistically sustainable amounts of guidance or collaboration is referred to as innovation within the zone of proximal implementation.
There is no shortage of literature critically assessing the educational impact of TEL (or lack thereof), and why innovations tend to fail. Common problems in the implementation and/or maintenance of TEL innovations demonstrate that, with regularity, insufficient attention is given to anticipating and designing for educational realities. Within the classroom, common problems stem from: poor alignment between innovations and classroom curricula such as textbooks and attainment targets (Cuban 2001); downplaying, or flat-out or ignoring key system factors that, within the scope of the innovation, cannot be manipulated such as assessments, technology policies and infrastructure (McKenney, Nieveen, and van den Akker 2006); over-estimating the interest and expertise of teachers, not just related to technology and or (pedagogical) content knowledge, but also related to the orchestration that is often involved in giving students access to/guidance on the technology (e.g. Knezek and Christensen 2008); insufficient attention to practitioner understanding and ownership of the innovation and its underpinning ideas (cf. Tebbutt 2000); focus on delivery and not on pedagogy (Reeves 2011). Looking beyond classroom innovations alone, researchers at the Open University of the Netherlands identified six “sure-fire causes of failure” for technology innovations (OUNL 2005). These are:
The shortcomings of TEL (design) research are not only measured in terms of innovation failure. They can also be measured in terms of innovation focus. Often, good-willed technology enthusiasts, seeking to design, develop, and try out new possibilities, conceive of technology-based innovations. The problems tackled are often ones of missed opportunity, e.g. not making optimal use of iPads, or failing to incorporate mobile technologies in new ways. However, in so doing, opportunities are frequently missed to address more urgent issues in schools, such as tackling levels of student learning, transfer of learning to daily use, or teacher turnover. This issue plagues much educational research and is especially applicable to that involving technology. The words of Schön (1995, p. 28) are applicable here:
In the swampy lowlands, problems are messy and confusing and incapable of technical solution. The irony of this situation is that the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern. The practitioner [or in this case, designer /researcher] is confronted with a choice. Shall he remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?
While mucking it up in the “swampy lowlands” can certainly present methodological challenges to research, rigor and relevance are not mutually exclusive (McKenney and Reeves 2012; Reeves 2011). However, as Schön points out, commitment to relevance is a matter of choice. Given all the time, energy and resources being pumped into developing and studying educational technologies, it would seem we are behooved to identify ways to design, develop and try out new possibilities that speak not to quasi-needs but to urgent ones. For example, “our teachers need ideas for how to use the iPads we gave them,” “technology coordinators require training in intra-personal skills,” and “mathematics learning should be more practical,” do not point to urgent problems worth solving. Rather, they are proposed semi-solutions or, in some cases, solutions in search of problems.
Much current research on TEL ultimately benefits only a small fraction of learners and practitioners, because it is conducted through high-intensity boutique projects (McKenney 2006). Such projects tend to feature substantial levels of researcher/facilitator involvement and often lack attention for gradually withdrawing implementation scaffolds or creating/shifting ownership of an innovation into the hands of those who would continue its use. As stated above, such projects are necessary, but not sufficient to develop the understanding and tools that can yield improvements in everyday practice. To seriously explore the viability and effectiveness of TEL, research is also needed that seeks to understand the perceptions, behaviours and motives that shape the varied experiences of teachers and learners in different settings. This can be particularly problematic for TEL researchers, who often thrive on the voluntary participation of early adopters (Rogers 2003). Yet working with this group can be misleading or even counter-productive (Bereiter 2002). To understand how to address representative concerns, we need the majority, and sometimes even working with laggards can be particularly insightful:
Studying the status quo of teaching, learning and settings, and designing TEL such that it gradually bridges from the current situation to the desired situation, is essential in developing both the knowledge and the tools required to address real needs in today's classrooms. This perspective is referred to here as the zone of proximal implementation. Vygotsky's concept of the zone of proximal development – the distance between what learners can accomplish independently and what they can accomplish through guidance or collaboration – has previously been applied to large-scale reform (Rogan 2007; Rogan and Grayson 2003); school leadership (McGivney and Moynihan 1972); and the mediation of educational partnerships (Oakes et al. 1998). Similarly, others have referred to the need to pursue certain innovation goals in stepwise fashion, gradually moving from the current situation toward what is desired (e.g. Sullivan 2004). Here, the basic concept is applied to the design of TEL; but rather than focusing on what can be achieved by learners, it focuses on what can be implemented by teachers and schools. The zone of proximal implementation refers to the distance between what teachers and schools can implement independently and what they can implement through guidance or collaboration. Designing for the zone of proximal implementation means explicitly tailoring products and processes to fit the needs of not only learners, but also of teachers and schools. It additionally means planning for implementation scaffolding (e.g. honoraria or researcher co-teaching) to fade away in a timely fashion, while simultaneously developing the ownership and expertise among practitioners that will engender the desire and ability to sustain innovation. This is done, in part, through responsive (and sometimes participatory) design, fed by insights concerning learners, practitioners and context.
In their book on conducting educational design research, McKenney and Reeves (2012) identify four characteristics of innovations that are prone to successful implementation; such innovations are: value-added, clear, compatible and tolerant. During the inception, creation and testing of TEL innovations at the zone of proximal implementation, these characteristics may be considered criteria to be met. These concepts are briefly summarized below (please see McKenney and Reeves (2012) for full descriptions and justification).
Value-added innovations offer something better than what is already in place. Similar to Rogers’ (2003) notion of the relative advantage, the potential benefits of value-added innovations visibly outweigh the investments required to yield them. Clear innovations enable participants to easily envision their involvement. Innovations may be clear through high levels of explicitness (cf. Fullan and Pomfret 1977) through a priori specifications of procedures (cf. Doyle and Ponder 1978) and/or interactive mechanisms whereby developers and users co-define (elements of) the innovation. Compatible innovations are congruent with existing values, cultures, practices and beliefs (cf. Doyle and Ponder 1978; Fullan and Pomfret 1977; Rogers 2003; Zhao et al. 2002). They are still innovative, but the innovations and/or their underlying assumptions do not violate or reject fundamental concerns and principles of those involved. Compatible innovations are also aligned with non-changeable aspects of the educational system, such as assessment frameworks or policies (McKenney, Nieveen, and van den Akker 2006). Finally, tolerant innovations are those that “degrade gracefully” (Walker 2006) as opposed to yielding “lethal mutations” (Brown and Campione 1996) during the natural variation in enactment that inevitably comes along with differing contexts, resources, expertise, acceptance levels and so on. Tolerance refers to how precisely core components must be enacted for the innovation to be true to its goals, and how well an innovation withstands local adaptations.
If designing for the zone of proximal implementation includes creating innovations that are value-added, clear, compatible and tolerant, then it makes sense to consider how these characteristics can be embodied in designed innovations. Table 1 offers considerations of what would need to be studied in order to derive design inputs related to each of these characteristics. For each characteristic, the focus of inquiry is defined (with slight variations depending on the stage of TEL innovation development). In addition, methodological recommendations are given for studying each characteristic (grey cells). Though beyond the scope of this article to discuss in detail, it may be useful to mention that technologies for data collection and analysis – both qualitative (Onwuegbuzie, Leach, and Collins 2011) and quantitative (Bryman and Cramer 1997)—are rapidly on the rise, as is literature on the warrants and risks of these new approaches (Beddall-Hill, Jabbar, and Al Shehri 2011; Garrett 2013). Technology-supported data collection includes use of social and mobile-based applications for collection in the field (e.g. Mendeley®, Facebook®, Evernote®), interview recording and transcriptions tools (e.g. Dragon Naturally Speaking®, Skype®); and observation recording and analysis tools (e.g. Morae®), as well as more generic tools (e.g. Atlas-ti®, SPSS®).
The importance of understanding where teachers and schools are, and framing innovations to be within a reachable distance from that, has been described in TEL literature previously (e.g. Bielaczyc 2006; Blumenfeld, Fishman, Krajcik, Marx, and Soloway 2000; McKenney and Voogt 2012). This paper emphasizes that more work is needed to help TEL designers and researchers do so. Research is needed to develop and refine understanding that can feed design (e.g. design principles, patterns and heuristics); and examples are needed to demonstrate how these ideas can be embodied in actual TEL scenarios. Moreover, choices are needed which focus research and development efforts on exploring new possibilities that address urgent – and not merely quasi – needs in existing classrooms, at various levels (e.g. poor learner motivation; teacher attrition; insufficiently aligned curricula and exams)
Focusing research where it is needed most remains difficult for a host of reasons, but two factors are especially powerful in terms of shaping research agendas: financial resources and academic reward systems. In particular, TEL researchers rely on research and development financing from third party support, which is often tied to political bodies (e.g. European Union). Under these circumstances, funding is designed to advance the political and economic agendas, not scientific understanding. Thus, for many TEL researchers, there is precious little funding available which would support state-of-practice innovations, because those funding schemes are, by definition, created to support start-of-the-art advancements.
A second factor that powerfully influences the focus of TEL research is the academic reward system. Increasingly, academic advancement is becoming contingent on numbers of publications in high impact journals, sometimes also using the H-index (statistical representation of impact based on a scientist's citations in other papers). Each of these metrics (publication counts, journal impact factors and the H-index) originated from worthy intentions: to evaluate if and to what extent researchers are contributing to knowledge production in their field. However, as scholars have indicated (Togia and Tsigilis 2006), they are crude at best because, e.g. there is no distinction between positive and negative citations; the calculation for impact factor is not corrected for self-citations; and citation indices favour English language journals. Moreover, the tendency to use them as the primary basis for assessing research contributions, as opposed to supplemental verification, stimulates perversion. Specifically, societal and scientific interests are usurped by survival concerns, and researcher priorities are set first by what is deemed publishable, and second by what addresses a real scientific and/or practical concern. Speaking of this very issue, authors recently published a paper (in a journal which, on their website, denounces the use of journal tier systems as a determinant for career advancement) entitled “Publishing and perishing: The critical importance of educational design research.” This paper argues for increased use of design approaches in educational technology research. Central to this approach is carefully identifying problems and framing inquiries that are both scientifically valuable (i.e. they address a real knowledge gap) and practically relevant (i.e. they address urgent concerns experienced today).
Conducting (design) research at the zone of proximal implementation inherently involves collaboration with practitioners (not only taking concerns seriously, but also drawing on their expertise), and appreciation of the reach and limitations of their role in determining what actually happens in classrooms. For some researchers, this may mean revisiting existing stances and possibly questioning them. Both researchers and practitioners could be better prepared than is often the case to actively shape the cooperative relationships they undertake. One important step in that direction is acknowledging that there are different forms of cooperation, and that the form should be chosen based on the research questions being asked, the people involved, and the context in which the study is being carried out. Wagner (1997) identifies three different forms of researcher-practitioner cooperation: data-extraction agreements (researchers are outside the schools and engaged reflection; practitioners are inside the schools and engaged in action); clinical partnerships (researchers and practitioners remain in their usual spaces, but engage in reflection together, usually aimed at improving practitioner effectiveness); and co-learning agreements (researchers and practitioners collaborate on processes of action and reflection).
For other researchers, working within the zone of proximal implementation can require fundamental changes in the researcher-practitioner relationship (e.g. Confrey, Castro-Filho, and Wilhelm 2000). It may also mean learning to accept what Barab et al. (2007, p. 297) refer to as “a life of compromises”:
… several interrelated tensions also emerged as problematic for our efforts yet illuminative of critical design work more generally, including: (a) tensions among preexisting biases and supporting local needs, (b) tensions between empowering teachers and empowering children, and (c) tensions between local design work and more general products and theories. Further, related to the three of these is a more global tension recurrent in the prior discussion of the process of critical design work, namely, the critical design researcher's responsibility to understand the local concerns and use an appreciation of the literature to characterize the local context in a way that considers local problems but with broader significance.
While it may take time for researchers to adjust to different relationships, or to make peace with the tensions that come along with pursuing the dual aims of generating theoretical understanding while developing TEL scenarios for use in specific practical settings, the benefits of such pathways seem to warrant the effort. If we truly care about the relevance and practical applicability of research, then, alongside investments in research and development of what might be technically possible, we must invest in understanding and designing for what is realistically feasible: in the zone of proximal implementation.
Barab, S. et al., (2007) ‘Our designs and the social agendas they carry’, Journal of the Learning Sciences, vol. 16, no. 2, pp. 263–305. Publisher Full Text
Beddall-Hill, N. L. Jabbar, A. & Al Shehri, S. (2011) ‘Social mobile devices as tools for qualitative research in education: iPhones and iPads in ethnography, interviewing, and design-based research’, Journal of the Research Center of Educational Technology, vol. 7, no. 1, pp. 67–89.
Bereiter, C. (2002) ‘Designing social infrastructure: critical issues in creating learning environments with technology’, Journal of the Learning Sciences, vol. 15, no. 3, pp. 301–329.
Berliner, D. (2002) ‘Educational research: The hardest science of all’, Educational Researcher, vol. 31, no. 8, pp. 18–20. Publisher Full Text
Bielaczyc, K. (2006) ‘Design research for sustained innovation’, Cognitive Studies, Bulletin of the Japanese Cognitive Science Society, vol. 9, no. 3, pp. 321–327.
Blumenfeld, P. et al., (2000) ‘Creating usable innovations in systemic reform: scaling up technology-embedded project-based science in Urban schools’, Educational Psychologist, vol. 35, no. 3, pp. 149–164. Publisher Full Text
Brown, A. L. & Campione, J. (1996) ‘Psychological theory and the design of innovative learning environments: on procedures, principles, and systems’, in Innovations in Learning, eds L. Schauble & R. Glaser, Earlbaum, Mahwah, NJ, pp. 289–325.
Bryman, A. & Cramer, D. (1997) Quantitative Data Analysis with SPSS for Windows: A Guide for Social Scientists, Routledge, London.
Confrey, J. Castro-Filho, J. & Wilhelm, J. (2000) ‘Implementation research as a means to link systemic reform and applied psychology in mathematics education’, Educational Psychologist, vol. 35, no. 3, pp. 179–191. Publisher Full Text
Cuban, L. (2001) Oversold and Underused: Computers in the Classroom, Harvard University Press, Cambridge.
Doyle, W. & Ponder, G. (1978) ‘The practicality ethic in teacher decision-making’, Interchange, vol. 8, no. 3, pp. 1–12. Publisher Full Text
Fullan, M. & Pomfret, A. (1977) ‘Research on curriculum and instruction implementation’, Review of Educational Research, vol. 47, no. 2, pp. 335–397. Publisher Full Text
Garrett, J. (2013) ‘Technology to support data collection and management in the public schools’, Perspectives on School-Based Issues, vol. 14, no. 1, pp. 10–14. Publisher Full Text
Kenny, J. (2002) ‘Managing innovation in educational institutions’, Australian Journal of Educational Technology, vol. 18, no. 3, pp. 359–376.
Knezek, G. & Christensen, R. (2008) ‘The importance of information technology attitudes and competencies in primary and secondary education’, in International Handbook of Information Technology in Primary and Secondary Education, eds J. Voogt & G. Knezek, Springer, New York, pp. 349–365.
Lagemann, E. (2002) An Elusive Science: The Troubling History of Education Research, University of Chicago Press, Chicago.
McGivney, J. & Moynihan, W. (1972) ‘School and community’, Teachers College Record, vol. 74, no. 2, pp. 209–224.
McKenney, S. (2006) ‘Book review: internet environments for science education’, International Journal of Science Education, vol. 28, no. 1, pp. 95–98. Publisher Full Text
McKenney, S. Nieveen, N. & van den Akker, J. (2006) ‘Design research from the curriculum perspective’, in Educational Design Research, eds J. van den Akker et al., Routledge, London, pp. 67–90.
McKenney, S. & Reeves, T. (2012) Conducting Educational Design Research, Routledge, London.
McKenney, S. & Voogt, J. (2012) ‘Teacher design of technology for emergent literacy: an explorative feasibility study’, Australasian Journal of Early Childhood, vol. 37, no. 1, pp. 4–12.
Oakes, J. et al., (1998) ‘Norms and policies of enquiry-minded change: reaching the zone of mediation’, in International Handbook of Educational Change, eds A. Hargreaves et al., Kluwer, Dordrecht, pp. 952–975.
Onwuegbuzie, A. J. Leach, N. J. & Collins, K. M. T. (2011) ‘Innovative qualitative data collection techniques for conducting literature reviews/research syntheses’, in The Sage Handbook of Innovation in Social Research Methods, eds M. Williams & W. P. Vogt, Sage, London, pp. 182–204.
OUNL. (2005) Determinants for the Failure and Success of Innovation Projects: The Road to Sustainable Educational Innovations, Open University of the Netherlands Sponsored by SURF, Heerlen.
Reeves, T. C. (2011) ‘Can educational research be both rigorous and relevant’, Educational Designer, vol. 1, no. 4. http://www.educationaldesigner.org/ed/volume1/issue4/article13/.
Reeves, T. Herrington, J. & Oliver, R. (2005) ‘Design research: a socially responsible approach to instructional technology research in higher education’, Journal of Computing in Higher Education, vol. 16, no. 2, pp. 97–116. Publisher Full Text
Rogan, J. (2007) ‘How much curriculum change is appropriate? Defining a zone of feasible innovation’, Science Education, vol. 91, pp. 439–460. Publisher Full Text
Rogan, J. & Grayson, D. (2003) ‘Towards a theory of curriculum implementation with particular reference to science education in developing countries’, International Journal of Science Education, vol. 25, pp. 1171–1204. Publisher Full Text
Rogers, E. (2003) Diffusion of Innovations, 5th edn, Free Press, New York.
Schön, D. (1995) ‘The new scholarship requires a new epistemology’, Change, vol. 27, no. 6, pp. 27–34. Publisher Full Text
Sullivan, M. (2004) ‘The reconceptualisation of learner-centred approaches: a Namibian case study’, International Journal of Educational Development, vol. 24, no. 6, pp. 585–602. Publisher Full Text
Tebbutt, M. (2000) ‘ICT in science: problems, possibilities and principles’, School Science Review, vol. 81, no. 297, pp. 57–64.
Togia, A. & Tsigilis, N. (2006) ‘Impact factor and education journals: a critical examination and analysis’, International Journal of Educational Research, vol. 45, no. 6, pp. 362–379. Publisher Full Text
Wagner, J. (1997) ‘The unavoidable intervention of educational research: a framework for reconsidering researcher–practitioner cooperation’, Educational Researcher, vol. 26, no. 7, pp. 13–22. Publisher Full Text
Walker, D. (2006) ‘Toward productive design studies’, in Educational Design Research, eds J. Van den Akker et al., Routledge, London, pp. 8–13.
Zhao, Y. et al., (2002) ‘Conditions for classroom technology innovations’, Teachers College Record, vol. 104, no. 3, pp. 482–515. Publisher Full Text