RLT Research in Learning Technology 2156-7077 Co-Action Publishing 17374 10.3402/rlt.v21i0.17374 Designing and researching technology-enhanced learning for the zone of proximal implementation Designing and researching technology-enhanced learning for the zone of proximal implementation McKenney Susan 1 2 * Center for Learning Sciences and Technologies (CELSTEC), Open University of the Netherlands, Heerlen, The Netherlands Twente University, Enschede, The Netherlands Corresponding author. Email: susan.mckenney@ou.nl 16 09 2013 2013 21 10.3402/rlt.v21i0.17374 11 02 2012 10 06 2013 © 2013 S. McKenney 2013

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Internationally, society is increasingly demanding that the relevance and practical applicability of research be made transparent. Despite intentions to the contrary, insights on pedagogically appropriate uses of educational technology for representative teachers in everyday school settings are severely limited. In part, this is because (design) research is often conducted at the bleeding edge of what is technologically possible – exploring innovative uses of new and emerging technologies. There is no disputing that such work is greatly needed to seek out new ways to potentially enhance the quality of teaching and learning. However, in the excitement of exploring what is possible, tomorrow, insufficient research and development work focuses on what is practical, today. This leaves a problematic gap between what could be effective technology-enhanced learning (TEL) in theory, and what can be effective TEL in practice. This paper calls for designers/researchers of TEL to devote attention to not only fine-grained issues of pupil learning and instruction but also to broader factors that determine if and how innovations are understood, adopted and used by teachers and schools, by designing innovations to align with their zone of proximal implementation. Methodological considerations are given for designing and studying interventions that are prone to implementation by being: value-added, clear, harmonious and tolerant.

learning design implementation innovation
Purpose

Society in general and research foundations around the globe such as the European Research Council and the (American) National Science Foundation are increasingly calling for the relevance and practical applicability of research to be made transparent. Despite intentions to the contrary, research on technology-enhanced learning (TEL) that truly serves current educational practice is more rare than it is common (Reeves, Herrington, and Oliver 2005). Insights on pedagogically appropriate uses of educational technology for representative teachers in everyday school settings are severely limited. In part, this is because (design) research is conducted at the bleeding edge of what is technologically possible – exploring innovative uses of new and emerging technologies. There is no disputing that such work is greatly needed to seek out new ways to potentially enhance the quality of teaching and learning. However, in the excitement of exploring what is possible, tomorrow, there is insufficient research and development work focusing on what is practical, today. This leaves a problematic gap between what could be effective TEL in theory, and what can be effective TEL in practice. With the aim of generating “usable knowledge” (Lagemann 2002) and creating innovations that truly serve learning in practice, this paper calls for designers/researchers of TEL to devote attention to not only fine-grained issues of pupil learning and instruction, but also to broader factors that determine if and how innovations are understood, adopted and used by teachers and schools. Allowing these issues to steer the design of TEL innovations is necessary to yield innovations that can feasibly be implemented outside of (often highly enabling) research and development trajectories. Accepting that radical innovation in education is not possible (Berliner 2002), largely due to its high degree of uncertainty (Kenny 2002), doing so constitutes a form of incremental innovation. Throughout this paper, incremental innovation targeted at what teachers and schools can implement with realistically sustainable amounts of guidance or collaboration is referred to as innovation within the zone of proximal implementation.

Current shortcomings of TEL (design) research

There is no shortage of literature critically assessing the educational impact of TEL (or lack thereof), and why innovations tend to fail. Common problems in the implementation and/or maintenance of TEL innovations demonstrate that, with regularity, insufficient attention is given to anticipating and designing for educational realities. Within the classroom, common problems stem from: poor alignment between innovations and classroom curricula such as textbooks and attainment targets (Cuban 2001); downplaying, or flat-out or ignoring key system factors that, within the scope of the innovation, cannot be manipulated such as assessments, technology policies and infrastructure (McKenney, Nieveen, and van den Akker 2006); over-estimating the interest and expertise of teachers, not just related to technology and or (pedagogical) content knowledge, but also related to the orchestration that is often involved in giving students access to/guidance on the technology (e.g. Knezek and Christensen 2008); insufficient attention to practitioner understanding and ownership of the innovation and its underpinning ideas (cf. Tebbutt 2000); focus on delivery and not on pedagogy (Reeves 2011). Looking beyond classroom innovations alone, researchers at the Open University of the Netherlands identified six “sure-fire causes of failure” for technology innovations (OUNL 2005). These are:

Lack of balance between investments and output (e.g. high investment with low output);

Information politics (power is abused and information is not transmitted);

Lack of responsibility (uncertainty about the responsibility of people inside and outside the project);

Culture gap (the gap between technology specialists and the rest of the organization, as well as between those who plan and enact education);

Over-commitment (not knowing when to cut losses and stop a project); and

All-in-one solutions (trying to do everything at once instead of using multiple projects, steps, and phases).

The shortcomings of TEL (design) research are not only measured in terms of innovation failure. They can also be measured in terms of innovation focus. Often, good-willed technology enthusiasts, seeking to design, develop, and try out new possibilities, conceive of technology-based innovations. The problems tackled are often ones of missed opportunity, e.g. not making optimal use of iPads, or failing to incorporate mobile technologies in new ways. However, in so doing, opportunities are frequently missed to address more urgent issues in schools, such as tackling levels of student learning, transfer of learning to daily use, or teacher turnover. This issue plagues much educational research and is especially applicable to that involving technology. The words of Schön (1995, p. 28) are applicable here:

In the swampy lowlands, problems are messy and confusing and incapable of technical solution. The irony of this situation is that the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern. The practitioner [or in this case, designer /researcher] is confronted with a choice. Shall he remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?

While mucking it up in the “swampy lowlands” can certainly present methodological challenges to research, rigor and relevance are not mutually exclusive (McKenney and Reeves 2012; Reeves 2011). However, as Schön points out, commitment to relevance is a matter of choice. Given all the time, energy and resources being pumped into developing and studying educational technologies, it would seem we are behooved to identify ways to design, develop and try out new possibilities that speak not to quasi-needs but to urgent ones. For example, “our teachers need ideas for how to use the iPads we gave them,” “technology coordinators require training in intra-personal skills,” and “mathematics learning should be more practical,” do not point to urgent problems worth solving. Rather, they are proposed semi-solutions or, in some cases, solutions in search of problems.

Toward relevant TEL research within the zone of proximal implementation

Much current research on TEL ultimately benefits only a small fraction of learners and practitioners, because it is conducted through high-intensity boutique projects (McKenney 2006). Such projects tend to feature substantial levels of researcher/facilitator involvement and often lack attention for gradually withdrawing implementation scaffolds or creating/shifting ownership of an innovation into the hands of those who would continue its use. As stated above, such projects are necessary, but not sufficient to develop the understanding and tools that can yield improvements in everyday practice. To seriously explore the viability and effectiveness of TEL, research is also needed that seeks to understand the perceptions, behaviours and motives that shape the varied experiences of teachers and learners in different settings. This can be particularly problematic for TEL researchers, who often thrive on the voluntary participation of early adopters (Rogers 2003). Yet working with this group can be misleading or even counter-productive (Bereiter 2002). To understand how to address representative concerns, we need the majority, and sometimes even working with laggards can be particularly insightful:

Representative/diverse teachers: Through working with diverse and representative teachers over time, it is possible to move beyond innovative one-off pilots and study how to bring about and sustain (even modest) advancements in pedagogically appropriate uses of technology.

Representative/diverse learners: Remembering that technology constitutes a mode of delivery and not, in and of itself, pedagogy, working with different kinds of learners can yield insights into the different ways that learners respond to TEL environments and different implementation choices made by teachers.

Representative/diverse settings: Rather than working around the (for researchers often frustrating) realities and limitations of classroom and school infrastructures, this view tackles head-on the work in average settings where, for example: the costs of printing are prohibitive; the school's internet firewall acts more like a prison than a filter; the teachers have extremely little curricular autonomy to make decisions about when/how to integrate technology in their classes; the location of computers (e.g. three in the classroom vs. eight in the lab) plays a determining role in how things are implemented; or “covering” the examination content almost singularly drives the allocation of learning time.

Studying the status quo of teaching, learning and settings, and designing TEL such that it gradually bridges from the current situation to the desired situation, is essential in developing both the knowledge and the tools required to address real needs in today's classrooms. This perspective is referred to here as the zone of proximal implementation. Vygotsky's concept of the zone of proximal development – the distance between what learners can accomplish independently and what they can accomplish through guidance or collaboration – has previously been applied to large-scale reform (Rogan 2007; Rogan and Grayson 2003); school leadership (McGivney and Moynihan 1972); and the mediation of educational partnerships (Oakes et al. 1998). Similarly, others have referred to the need to pursue certain innovation goals in stepwise fashion, gradually moving from the current situation toward what is desired (e.g. Sullivan 2004). Here, the basic concept is applied to the design of TEL; but rather than focusing on what can be achieved by learners, it focuses on what can be implemented by teachers and schools. The zone of proximal implementation refers to the distance between what teachers and schools can implement independently and what they can implement through guidance or collaboration. Designing for the zone of proximal implementation means explicitly tailoring products and processes to fit the needs of not only learners, but also of teachers and schools. It additionally means planning for implementation scaffolding (e.g. honoraria or researcher co-teaching) to fade away in a timely fashion, while simultaneously developing the ownership and expertise among practitioners that will engender the desire and ability to sustain innovation. This is done, in part, through responsive (and sometimes participatory) design, fed by insights concerning learners, practitioners and context.

How to design and study TEL at the zone of proximal implementation?

In their book on conducting educational design research, McKenney and Reeves (2012) identify four characteristics of innovations that are prone to successful implementation; such innovations are: value-added, clear, compatible and tolerant. During the inception, creation and testing of TEL innovations at the zone of proximal implementation, these characteristics may be considered criteria to be met. These concepts are briefly summarized below (please see McKenney and Reeves (2012) for full descriptions and justification).

Value-added innovations offer something better than what is already in place. Similar to Rogers’ (2003) notion of the relative advantage, the potential benefits of value-added innovations visibly outweigh the investments required to yield them. Clear innovations enable participants to easily envision their involvement. Innovations may be clear through high levels of explicitness (cf. Fullan and Pomfret 1977) through a priori specifications of procedures (cf. Doyle and Ponder 1978) and/or interactive mechanisms whereby developers and users co-define (elements of) the innovation. Compatible innovations are congruent with existing values, cultures, practices and beliefs (cf. Doyle and Ponder 1978; Fullan and Pomfret 1977; Rogers 2003; Zhao et al. 2002). They are still innovative, but the innovations and/or their underlying assumptions do not violate or reject fundamental concerns and principles of those involved. Compatible innovations are also aligned with non-changeable aspects of the educational system, such as assessment frameworks or policies (McKenney, Nieveen, and van den Akker 2006). Finally, tolerant innovations are those that “degrade gracefully” (Walker 2006) as opposed to yielding “lethal mutations” (Brown and Campione 1996) during the natural variation in enactment that inevitably comes along with differing contexts, resources, expertise, acceptance levels and so on. Tolerance refers to how precisely core components must be enacted for the innovation to be true to its goals, and how well an innovation withstands local adaptations.

If designing for the zone of proximal implementation includes creating innovations that are value-added, clear, compatible and tolerant, then it makes sense to consider how these characteristics can be embodied in designed innovations. Table 1 offers considerations of what would need to be studied in order to derive design inputs related to each of these characteristics. For each characteristic, the focus of inquiry is defined (with slight variations depending on the stage of TEL innovation development). In addition, methodological recommendations are given for studying each characteristic (grey cells). Though beyond the scope of this article to discuss in detail, it may be useful to mention that technologies for data collection and analysis – both qualitative (Onwuegbuzie, Leach, and Collins 2011) and quantitative (Bryman and Cramer 1997)—are rapidly on the rise, as is literature on the warrants and risks of these new approaches (Beddall-Hill, Jabbar, and Al Shehri 2011; Garrett 2013). Technology-supported data collection includes use of social and mobile-based applications for collection in the field (e.g. Mendeley®, Facebook®, Evernote®), interview recording and transcriptions tools (e.g. Dragon Naturally Speaking®, Skype®); and observation recording and analysis tools (e.g. Morae®), as well as more generic tools (e.g. Atlas-ti®, SPSS®).

Methodological considerations for researching TEL innovations at the zone of proximal implementation.

Before design (needs/context analysis) During design (prototyping and formative evaluation) After design (summative evaluation)

Value-added (better than status quo) Learning practices, problems, outcomes in the baseline situation Learning practices, problems, outcomes during use Learning practices, problems, outcomes with all implementation scaffolds removed
Observation, learner work/assessments, document analysis, brief questionnaires (e.g. learning environment rating scales) to study enacted curriculum; focus groups and interviews to get teacher perceptions

Clear (participants can envision their involvement) Mindsets, habits and conventions within the classroom/school in the baseline situation Mindsets, habits and conventions within the classroom/school during use Mindsets, habits and conventions within the classroom/school that are sustained or changed after the innovation
Interviews, observations, and logbooks to track how clearly professionals understand their role and how actively they engage in it

Compatible (compatible with values, beliefs, surrounding educational context/system) Values, cultures, beliefs, priorities, and contextual /system factors in the baseline situation Values, cultures, beliefs, priorities, and contextual /system factors that help or hinder implementation Values, cultures, beliefs, priorities, and contextual /system factors that are sustained or changed after the innovation
Observation, interviews, document analysis to understand and track how alignment between the innovation and other determinants of implementation

Tolerant (withstands the natural variation of actual use) Actual behaviours of teachers and learners and reasons for them in the baseline situation Actual behaviours of teachers and learners and reasons for them during use Actual behaviours of teachers and learners and reasons for them with all implementation scaffolds removed
Observation, interviews and document analysis to understand what teachers and learners actually do and why

Discussion and conclusion

The importance of understanding where teachers and schools are, and framing innovations to be within a reachable distance from that, has been described in TEL literature previously (e.g. Bielaczyc 2006; Blumenfeld, Fishman, Krajcik, Marx, and Soloway 2000; McKenney and Voogt 2012). This paper emphasizes that more work is needed to help TEL designers and researchers do so. Research is needed to develop and refine understanding that can feed design (e.g. design principles, patterns and heuristics); and examples are needed to demonstrate how these ideas can be embodied in actual TEL scenarios. Moreover, choices are needed which focus research and development efforts on exploring new possibilities that address urgent – and not merely quasi – needs in existing classrooms, at various levels (e.g. poor learner motivation; teacher attrition; insufficiently aligned curricula and exams)

Focusing research where it is needed most remains difficult for a host of reasons, but two factors are especially powerful in terms of shaping research agendas: financial resources and academic reward systems. In particular, TEL researchers rely on research and development financing from third party support, which is often tied to political bodies (e.g. European Union). Under these circumstances, funding is designed to advance the political and economic agendas, not scientific understanding. Thus, for many TEL researchers, there is precious little funding available which would support state-of-practice innovations, because those funding schemes are, by definition, created to support start-of-the-art advancements.

A second factor that powerfully influences the focus of TEL research is the academic reward system. Increasingly, academic advancement is becoming contingent on numbers of publications in high impact journals, sometimes also using the H-index (statistical representation of impact based on a scientist's citations in other papers). Each of these metrics (publication counts, journal impact factors and the H-index) originated from worthy intentions: to evaluate if and to what extent researchers are contributing to knowledge production in their field. However, as scholars have indicated (Togia and Tsigilis 2006), they are crude at best because, e.g. there is no distinction between positive and negative citations; the calculation for impact factor is not corrected for self-citations; and citation indices favour English language journals. Moreover, the tendency to use them as the primary basis for assessing research contributions, as opposed to supplemental verification, stimulates perversion. Specifically, societal and scientific interests are usurped by survival concerns, and researcher priorities are set first by what is deemed publishable, and second by what addresses a real scientific and/or practical concern. Speaking of this very issue, authors recently published a paper (in a journal which, on their website, denounces the use of journal tier systems as a determinant for career advancement) entitled “Publishing and perishing: The critical importance of educational design research.” This paper argues for increased use of design approaches in educational technology research. Central to this approach is carefully identifying problems and framing inquiries that are both scientifically valuable (i.e. they address a real knowledge gap) and practically relevant (i.e. they address urgent concerns experienced today).

Conducting (design) research at the zone of proximal implementation inherently involves collaboration with practitioners (not only taking concerns seriously, but also drawing on their expertise), and appreciation of the reach and limitations of their role in determining what actually happens in classrooms. For some researchers, this may mean revisiting existing stances and possibly questioning them. Both researchers and practitioners could be better prepared than is often the case to actively shape the cooperative relationships they undertake. One important step in that direction is acknowledging that there are different forms of cooperation, and that the form should be chosen based on the research questions being asked, the people involved, and the context in which the study is being carried out. Wagner (1997) identifies three different forms of researcher-practitioner cooperation: data-extraction agreements (researchers are outside the schools and engaged reflection; practitioners are inside the schools and engaged in action); clinical partnerships (researchers and practitioners remain in their usual spaces, but engage in reflection together, usually aimed at improving practitioner effectiveness); and co-learning agreements (researchers and practitioners collaborate on processes of action and reflection).

For other researchers, working within the zone of proximal implementation can require fundamental changes in the researcher-practitioner relationship (e.g. Confrey, Castro-Filho, and Wilhelm 2000). It may also mean learning to accept what Barab et al. (2007, p. 297) refer to as “a life of compromises”:

… several interrelated tensions also emerged as problematic for our efforts yet illuminative of critical design work more generally, including: (a) tensions among preexisting biases and supporting local needs, (b) tensions between empowering teachers and empowering children, and (c) tensions between local design work and more general products and theories. Further, related to the three of these is a more global tension recurrent in the prior discussion of the process of critical design work, namely, the critical design researcher's responsibility to understand the local concerns and use an appreciation of the literature to characterize the local context in a way that considers local problems but with broader significance.

While it may take time for researchers to adjust to different relationships, or to make peace with the tensions that come along with pursuing the dual aims of generating theoretical understanding while developing TEL scenarios for use in specific practical settings, the benefits of such pathways seem to warrant the effort. If we truly care about the relevance and practical applicability of research, then, alongside investments in research and development of what might be technically possible, we must invest in understanding and designing for what is realistically feasible: in the zone of proximal implementation.

References Barab S ‘Our designs and the social agendas they carry’. Journal of the Learning Sciences 2007 16 2 263 305 Beddall-Hill N. L Jabbar A Al Shehri S ‘Social mobile devices as tools for qualitative research in education: iPhones and iPads in ethnography, interviewing, and design-based research’ Journal of the Research Center of Educational Technology 2011 vol. 7 no. 1 67 89 Bereiter C ‘Designing social infrastructure: critical issues in creating learning environments with technology’ Journal of the Learning Sciences 2002 vol. 15 no. 3 301 329 Berliner D Educational research: The hardest science of all Educational Researcher 2002 31 8 18 20 Bielaczyc K ‘Design research for sustained innovation’ Cognitive Studies, Bulletin of the Japanese Cognitive Science Society 2006 vol. 9 no. 3 321 327 Blumenfeld P ‘Creating usable innovations in systemic reform: scaling up technology-embedded project-based science in Urban schools’ Educational Psychologist 2000 vol. 35 no. 3 149 164 Brown A. L Campione J Schauble L Glaser R ‘Psychological theory and the design of innovative learning environments: on procedures, principles, and systems’ in Innovations in Learning 1996 Mahwah, NJ Earlbaum 289 325 Bryman A Cramer D Quantitative Data Analysis with SPSS for Windows: A Guide for Social Scientists 1997 London Routledge Confrey J Castro-Filho J Wilhelm J ‘Implementation research as a means to link systemic reform and applied psychology in mathematics education’ Educational Psychologist 2000 vol. 35 no. 3 179 191 Cuban L Oversold and Underused: Computers in the Classroom 2001 Cambridge. Harvard University Press Doyle W Ponder G ‘The practicality ethic in teacher decision-making’ Interchange 1978 vol. 8 no. 3 1 12 Fullan M Pomfret A ‘Research on curriculum and instruction implementation’ Review of Educational Research 1977 vol. 47 no. 2 335 397 Garrett J ‘Technology to support data collection and management in the public schools’ Perspectives on School-Based Issues 2013 vol. 14 no. 1 10 14 Kenny J ‘Managing innovation in educational institutions’ Australian Journal of Educational Technology 2002 vol. 18 no. 3 359 376 Knezek G Christensen R Voogt J Knezek G ‘The importance of information technology attitudes and competencies in primary and secondary education’ in International Handbook of Information Technology in Primary and Secondary Education 2008 New York Springer 349 365 Lagemann E An Elusive Science: The Troubling History of Education Research 2002 Chicago. University of Chicago Press McGivney J Moynihan W ‘School and community’ Teachers College Record 1972 vol. 74 no. 2 209 224 McKenney S ‘Book review: internet environments for science education’ International Journal of Science Education 2006 vol. 28 no. 1 95 98 McKenney S Nieveen N van den Akker J van den Akker J ‘Design research from the curriculum perspective’ in Educational Design Research 2006 London Routledge 67 90 McKenney S Reeves T Conducting Educational Design Research 2012 London Routledge McKenney S Voogt J ‘Teacher design of technology for emergent literacy: an explorative feasibility study’ Australasian Journal of Early Childhood 2012 vol. 37 no. 1 4 12 Oakes J Hargreaves A ‘Norms and policies of enquiry-minded change: reaching the zone of mediation’ in International Handbook of Educational Change 1998 Dordrecht Kluwer 952 975 Onwuegbuzie A. J Leach N. J Collins K. M. T Williams M Vogt W. P ‘Innovative qualitative data collection techniques for conducting literature reviews/research syntheses’ in The Sage Handbook of Innovation in Social Research Methods 2011 London Sage 182 204 OUNL Determinants for the Failure and Success of Innovation Projects: The Road to Sustainable Educational Innovations 2005 Heerlen Open University of the Netherlands Sponsored by SURF Reeves T. C ‘Can educational research be both rigorous and relevant’ Educational Designer 2011 vol. 1 no. 4 http://www.educationaldesigner.org/ed/volume1/issue4/article13/ Reeves T Herrington J Oliver R ‘Design research: a socially responsible approach to instructional technology research in higher education’ Journal of Computing in Higher Education 2005 vol. 16 no. 2 97 116 Rogan J ‘How much curriculum change is appropriate? Defining a zone of feasible innovation’ Science Education 2007 vol. 91 439 460 Rogan J Grayson D ‘Towards a theory of curriculum implementation with particular reference to science education in developing countries’ International Journal of Science Education 2003 vol. 25 1171 1204 Rogers E Diffusion of Innovations 2003 New York. Free Press 5th edn Schön D ‘The new scholarship requires a new epistemology’ Change 1995 vol. 27 no. 6 27 34. Sullivan M ‘The reconceptualisation of learner-centred approaches: a Namibian case study’ International Journal of Educational Development 2004 vol. 24 no. 6 585 602. Tebbutt M ‘ICT in science: problems, possibilities and principles’ School Science Review 2000 vol. 81 no. 297 57 64. Togia A Tsigilis N ‘Impact factor and education journals: a critical examination and analysis’ International Journal of Educational Research 2006 vol. 45 no. 6 362 379. Wagner J ‘The unavoidable intervention of educational research: a framework for reconsidering researcher–practitioner cooperation’ Educational Researcher 1997 vol. 26 no. 7 13 22. Walker D Van den Akker J ‘Toward productive design studies’ in Educational Design Research 2006 London. Routledge 8 13 Zhao Y ‘Conditions for classroom technology innovations’ Teachers College Record 2002 vol. 104 no. 3 482 515.