RLT Research in Learning Technology 2156-7077 Co-Action Publishing 20057 10.3402/rlt.v21i0.20057 Learning design Rashomon II: exploring one lesson through multiple tools Learning design Rashomon II: exploring one lesson through multiple tools Prieto Luis P. 1 * Dimitriadis Yannis 1 Craft Brock 2 Derntl Michael 3 Émin Valérie 4 Katsamani Mary 5 Laurillard Diana 2 Masterman Elizabeth 6 Retalis Symeon 5 Villasclaras Eloy 7 GSIC-EMIC group, School of Telecommunications Engineering, University of Valladolid, Valladolid, Spain London Knowledge Lab, Institute of Education, London, UK Advanced Community Information Systems (ACIS), RWTH Aachen University, Aachen, Germany S2HEP, Institut Français de l’Éducation, ENS Lyon, Lyon, France Department of Digital Systems, University of Piraeus, Greece IT Services, University of Oxford, Oxford, UK Institute of Educational Technology, Open University, Milton Keynes, UK Corresponding author. Email: lprisan@hotmail.com 16 09 2013 2013 21 10.3402/rlt.v21i0.20057 © 2013 L.P. Prieto et al. 2013

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

An increasing number of tools are available to support the learning design process at different levels and from different perspectives. However, this variety can make it difficult for researchers and teachers to assess the tool that is best suited to their objectives and contexts as learning designers. Several of the tools are presented elsewhere in this issue. In this article, the aforementioned tools are used as lenses to view the same learning design narrative – an inquiry-based learning lesson on healthy eating aimed at secondary-school students – from different perspectives, in a manner inspired by the plot structure of Kurosawa's film “Rashomon”. In modelling the lesson on five tools, we uncovered similarities and differences in relation to the challenges posed by modelling a particular learning scenario, the ease of implementation of the computer-interpretable products’ output by the tools and their different target audiences and pedagogical specialities. This comparative analysis thus illustrates some of the current underlying issues and challenges in the field of Learning Design.

learning design authoring tools inquiry learning personal inquiry
Introduction

As Information and Communication Technologies (ICT) permeate every aspect of education, supporting the preparation of technology-enhanced teaching and learning activities – the process of learning design – is becoming increasingly important. A growing number of tools have been developed to assist teachers in planning learning outcomes, activities, assessment and other aspects of learning, both inside and outside the classroom.

Learning design authoring tools generally have the common goal of facilitating the sharing, adaptation and reuse of teachers’ pedagogical ideas, and they are also useful as tools for reflection on practice (Laurillard 2012). However, as Conole (2010) notes, different tools may employ various representations and operate at different levels of granularity, from simply capturing the essence of a design to aiding in its semi-automated enactment with students. For example, LAMS (Learning Activity Management System: Dalziel 2003) facilitates the micro-level planning and automation of the learning activities, whereas the Learning Designer (Laurillard et al. 2013) facilitates design work at the macro level, aiding the design of whole learning sessions and modules, and supporting more strategic pedagogical thinking. These different foci of support require different visual representations to abstract the individual elements of a design such as learning activities and resources, so that they can be easily defined and manipulated.

As the number of learning design authoring tools continues to grow, it becomes increasingly difficult to keep track of them or to engage with each one in depth. Few researchers or practitioners have the time to try multiple tools or to compare how different tools model a particular learning scenario in order to see which one best suits their needs. This, therefore, is the motivation for this article: to capture a single learning design using several authoring tools, in order to reflect on the different perspectives they provide and to learn about the design process itself from the various methods that the tools represent. Our aim is not to compare the authoring tools to see which one reflects the original conceptual design best. Indeed, such a comparison would be inappropriate as the research teams involved in the exercise did not have access to the designers’ original conceptions, instead they only had access to the design artefacts generated by them.

Inspired by Akira Kurosawa's film “Rashomon” (1950), in which an incident is portrayed through four alternative points of view, we reveal a design narrative from the perspectives of five different learning design tools. Persico et al. (2013) perform a similar exercise using different learning design conceptual frameworks, and related exercises previously performed in the learning design community include Botturi et al. (2007) and Vignollet et al. (2008).

In the following sections, we describe the rationale and method followed in this comparative study, before outlining the processes entailed in modelling an inquiry-based learning scenario using the different learning design tools. Finally, we discuss some of the main lessons learned from these complementary views of the common design narrative and report briefly the reactions of teachers who have engaged with the tools.

Methodology

The common learning scenario that we modelled was the “healthy eating” lesson from the Personal Inquiry project (Anastopoulou et al. 2012), which guided students aged 11–14 in an inquiry-based investigation into their eating habits, the food that they ate and its nutritional value.1 This learning scenario was chosen as it explored a different set of pedagogical concerns from previous exercises: the studies by Botturi et al. (2007) and Vignollet et al. (2008) were based on project-based and game-based learning situations, respectively.

The learning scenario was modelled using five of the learning design tools that were presented at the “Art and Science of Learning Design” workshop in 2011. They are all publicly available and well known within the learning design community of practice for the different levels of authoring support that they provide. We selected the tools that reflect the most recent evolution of research in the field, which has moved away from a concern with data modelling issues and now focuses on providing greater usability and conceptual support to a wider range of practitioners. The tools used were:

The Learning Designer: a tool for teachers to plan and reflect on their pedagogy at a more strategic level;

OpenGLM: an authoring toolkit to provide comprehensive graphical support for the design of IMS-LD compliant units of learning;

CADMOS: a graphical tool based on the “separation of concerns” principle, which is also compatible with IMS-LD;

Web Collage: a graphical tool for teachers aimed at the design of collaborative learning, which also supports IMS-LD;

ScenEdit: an intention-oriented tool for designing learning scenarios with an emphasis on the orchestration of interactional situations.

The study followed a three-step procedure:

Each of the research teams that had originally developed the tools received a copy of the design narrative and lesson plans for the scenario (in document form), together with an example of the original implementation of the scenario in the nQuire2 tool.

The teams modelled the scenario in their respective tools. They sought to remain as faithful as possible to the design representation, while also capitalising on the specific features of their tool.

The teams shared the resulting formalisations of the scenario with each other in the computerised representation outputs by the tools (e.g. XML files) and through screen captures of the design. These materials serve as the basis for the descriptions and reflections that appear in the following sections.

The initial lesson plans, together with the resulting formalisations and screenshots, are included in the supplementary material for this article.

The Learning Designer

The Learning Designer3 (Laurillard et al. 2013) is a community knowledge-building tool that supports teachers in creating, modifying, finding, sharing and reusing learning designs. Its aim is to support the teaching community in collaborating on how to design and express its pedagogical ideas, especially in relation to the use of technology-enhanced learning to improve learners’ experience.

The software environment represents learning designs using formal learning concepts (e.g. Bloom's taxonomy of learning outcomes and a typology of forms of learning), underpinned by semantic technologies. This kind of computer-interpretable representation enables the tool to provide visual analytics of the learning designs (for example, in the form of pie charts representing the character of the designed learning experience) and intelligent recommendations using community knowledge. The community knowledge is expressed in terms of pedagogical patterns for different learning outcomes, a wiki of advice and guidance on ways to introduce technology-enhanced learning and alternative design ideas generated by context-aware artificial intelligence techniques. Finally, learning designs can be exported as document templates or as files that can be shared with, and imported and adapted by, colleagues in their version of the tool. The files can even be shared with students for their comment. A storage site is available for this purpose.

To model the “healthy eating” lesson in the Learning Designer, we took the lesson descriptions and transformed them into properties and “teaching–learning activities” (TLAs) in a block of sessions.

Although a detailed description of the modelling process exceeds the scope of this article, certain aspects of the use of the Learning Designer are worth highlighting. Figure 1 shows the Properties page, which describes the overall features of the design.

The Learning Designer properties page. Text boxes enable the teacher-designer to insert his/her own free text and select a learning outcome type (categorised in terms of Bloom's taxonomy) from a drop-down menu, or input their own learning outcome (which is therefore not categorised). The floating window on the left shows part of the advice and guidance wiki that can be consulted at any time.

The detailed pedagogical features of the learning design are expressed in the Timeline screen, where the teacher-designer4 can drag and drop pre-defined TLAs, and customise them in terms of name, activity notes, group size, duration, and so on (Figure 2). TLAs are sized on the basis of their duration, and they contain an editable analysis of the likely learning experience, in terms of acquisition, inquiry, discussion, practice and/or production. The software offers suggestions for alternatives on the basis of the properties of the currently highlighted TLA (Figure 2, bottom left).

The Learning Designer timeline screen, showing the ordered sequence of TLAs. The highlighted “Homework” activity, described in the Activity Notes, is done individually and lasts 20 minutes. The “Alternative ideas” window (bottom left) suggests that each learner might keep a blog of their project development.

At any stage, the teacher-designer can check the Analysis page of the design (shown in Figure 3) to see the overall proportions of types of learning afforded by the design so far. This page also displays the likely time that the teacher will need for preparation and presentation.

The Learning Designer analysis screen. This provides feedback to the teacher-designer on the balance of the learning experience designed so far, in terms of the proportion of time spent learning through different cognitive activities. These proportions are calculated from the properties of the TLAs designed on the timeline.

When modelling the scenario with the Learning Designer, we found several gaps in the information in the scenario description, in particular the learning time supported, learning outcomes, student feedback, group size and duration of activities. The modelling of resources linked to activities had to be edited, since currently the tool only offers three resource types (local files, URLs and notes). Some of the activities were not described in sufficient detail to clarify what exactly learners were doing, and so they could not be categorised by the Learning Designer's taxonomy of activity types. Other activities which were grouped together as one activity in the scenario are actually several different kinds of learning activities in the Learning Designer.

OpenGLM

The Open Graphical Learning Modeller5 (OpenGLM: Derntl, Neumann, & Oberhuemer 2011) is a learning design authoring toolkit that supports the creation of IMS Learning Design (IMS-LD) units of learning at levels A and B (IMS Global Consortium 2003). Its main goal is to provide comprehensive and intuitive IMS-LD support for teacher-designers by interpreting a graphical representation of a learning design and converting it to the required XML format as specified in the IMS-LD standard. These IMS-LD units of learning can then be run on IMS-LD-enabled learning management systems (LMS).

We modelled the “healthy eating” lesson using OpenGLM by defining the teaching and learning activities and the roles that enact those activities. These activities can in turn be connected to each other, forming a sequence. Learning materials and other resources (e.g. a discussion forum) can then be linked to the activities, so that they are available to the actors who perform them.

OpenGLM does not impose a prescribed sequence of steps for modelling a learning scenario. We started by using the lesson description to prepare a simple list of tentative activity labels and roles performing those activities (e.g. “Watch introductory short video – students” and “Split up the class – teacher”). However, this activity list was not directly translated to IMS-LD activities in OpenGLM. Instead, some teacher and student activities were merged. Teacher activities that were actually part of a student activity, such as giving instructions about the activity, were incorporated into the description of the student activity. Similarly, the presentation of content or the sharing of artefacts was integrated into the subsequent learning activity: for example, the distribution of pictures for discussion was represented as a resource in the discussion activity.

The graphical representation of the resulting activity sequence is shown in Figure 4. There are three roles – student, group and teacher – and their activities are represented using different colours along vertical “swim lanes” (the visual arrangement has no effect on the underlying IMS-LD executable package). Learning activities are represented with a solid outline, and support activities (e.g. teacher activities) have a dashed outline. Resources and tools are represented as icons within each activity box, which also includes activity descriptions, add-ons and, for teacher activities, the roles to be supported.

The learning design sequence modelled with OpenGLM. Connecting arrows indicate synchronisation points (i.e. all previous activities need to be finished before the outgoing flows can continue). The two small circles represent the start (top) and end (bottom) of the sequence, respectively.

The resulting OpenGLM model can be shared with other designers using OpenGLM for viewing, editing and exporting to IMS-LD. The exported IMS-LD package is a valuable artefact that enables the underlying learning design to be reused through being shared with other IMS-LD authoring tools and deployed in IMS-LD runtime environments (Derntl, Neumann, & Oberhuemer 2011). However, the fact that the “healthy eating” lesson was to take place primarily in a face-to-face setting led to certain peculiarities during modelling, as IMS-LD was conceived primarily for the deployment of computer-managed learning opportunities (Derntl et al. 2012). In general, it is difficult to model non-software resources and services in IMS-LD, such as the “device” described in the lesson plan (which can be represented by an environment of services and learning objects) and other physical artefacts (e.g. pictures on a whiteboard, which have been modelled as a digital picture pool). Face-to-face activities such as a classroom discussion are equally difficult to model accurately in IMS-LD. Indeed, the “healthy eating” lesson was not described in a way that suggested computer-managed activities. Therefore, we tried to imagine the activities taking place in a virtual environment. This required thinking about virtual equivalents for the physical objects used in the scenario. For example, where shared pictures were to be annotated and discussed, we imagined an online picture pool (e.g. Flickr) as the virtual equivalent of the pictures on a whiteboard. Of course, this modified the original scenario and added interpretation bias to the formal representation of the scenario in OpenGLM.

CADMOS

CADMOS6 (CoursewAre Development Methodology for Open instructional Systems) is a graphical editor aimed at teachers with basic computer skills and knowledge of learning standards (Katsamani & Retalis 2011). Its main purpose is to make it easy for teachers who are not experts in learning design to create and share learning designs and to enact them in a real environment. CADMOS proposes a “separation of concerns” approach for the learning design process, which stems from the principles of Web engineering (Papasalouros, Retalis, & Papaspyrou 2004; Rossi et al. 2008). In accordance with this approach, the designer creates two different models:

A conceptual model, which describes the learning activities in which the different roles are involved, and the corresponding learning resources and services;

A flow model, which describes the orchestration of the learning activities.

CADMOS guides the teacher-designer through an initial screen about metadata for the learning design as a whole (title, description, learning goals, prerequisites and roles). Next, in the main design canvas screen, the teacher uses drag-and-drop methods to create the two models. Finally, they can export the resulting design in IMS-LD level B format (Katsamani & Retalis 2011).

In CADMOS, the teacher can define simple and composite activities; however, in modelling the “healthy eating” lesson, we only defined simple activities. The conceptual model of the lesson, which is shown in Figure 5, has 14 activities: for example, “Each student watches a video and asks questions”, “The teacher presents different inquiry methods” and “Each group chooses a question and sends it to an expert”. We defined metadata for each activity, including title, description, goal, the roles performing it and the activity type (e.g. informative, communicative or assessment). At any time, the teacher-designer can view statistics of the different kinds of activities that have been used so far. We also associated resources with each activity (see Figure 5) and specified metadata for each one, including title, author, description and resource file.

CADMOS conceptual model of the “healthy eating” lesson, showing activities (middle row) and resources (bottom row).

The flow model of the lesson, which we created next, is shown in Figure 6. In this model, activities are set in vertical “swim lanes” in chronological order and have been divided into seven phases (i.e. sets of activities), as suggested by the original description of the lesson.

CADMOS flow model of the “healthy eating” lesson, showing the flow of activities for each role.

The CADMOS tool was developed primarily for designing computer-supported lessons. Thus, although we were able to represent the lesson using CADMOS, we faced a number of constraints, generally relating to the description of face-to-face situations. Certain resources (e.g. whiteboard, email and face-to-face discussion) are currently omitted from CADMOS’ list of resource types. The length of each phase was modelled using comments, as CADMOS only permits users to specify timings at the activity level. There were also certain issues in presentation: even though CADMOS allows unlimited space for modelling a design, the length and complexity of the “healthy eating” design meant that the resulting model was not very readable.

Web Collage

Web Collage7 (Villasclaras-Fernández et al. 2011) is an authoring tool developed to support teachers who are not experts in learning design in creating computer-supported collaborative learning (CSCL) scenarios. Its main conceptual and visual metaphor is design patterns: general solutions to recurrent design problems (Alexander, Ishikawa, & Silverstein 1977).

Web Collage integrates a set of Collaborative Learning Flow Patterns (CLFPs: Hernández-Leo et al. 2006), which describe validated collaborative pedagogy techniques such as “Jigsaw” and “Pyramid”. These CLFPs serve as ideas for suitable collaborative learning structures, and Web Collage also uses them to generate the computer-interpretable components of the design that are needed to implement the collaborative techniques. The current version of Web Collage also includes assessment design patterns in order to encourage the teacher-designer to think about assessment and to design appropriate plans (Villasclaras-Fernández et al. 2011). If a teacher-designer wishes to split a class into groups, Web Collage can retrieve a list of the students from an LMS such as Moodle, which the teacher-designer can then use to allocate individual students to their groups. When the design work is complete, Web Collage, working alongside with GLUE!-PS (Prieto et al. 2011), can implement the learning design directly in the LMS, automatically creating the required activities, groups, resources and so forth.

Our model of the “healthy eating” lesson in Web Collage is shown in Figure 7. The left side shows the “learning flow” modelling of the learning and support activities. Some of the activities (in particular, individual tasks) did not follow any of Web Collage's CLFPs, and were modelled using generic “phases”; however, others could be represented using CLFPs. For example, the activities “Propose ideas for a healthy diet” and “Compile a list of food items” were modelled as “Brainstorming” CLFPs. The group activity of selecting a number of photos and the whole-class discussion which followed it comprised an example of a Pyramid pattern. The graphical representations of the Pyramid and Brainstorming CLFPs are intended to help make the teacher-designer's intentions easily understood, including the type of collaboration and outcomes expected for those phases. The left side of Figure 7 shows only the coarse-grained phases; specific learning and support activities can be described inside each phase (see, for example, Figure 7A). Within each pattern or phase, the tool can also model the number of groups and the students’ flow across the different groups (Figure 7B), as well as the assessment plan, which here includes a quiz with subsequent feedback from the teacher (Figure 7A).

Screenshots of the Web Collage model of the proposed lesson. They include detailed views of specific phases/patterns depicting assessment flow (A, C) and students’ flow (B) within a phase.

However, Web Collage's representation of the lesson also has limitations. There is no way to specify the duration of an activity, and there is only limited provision for modelling resources. For example, although the teacher-designer can define the URL of a certain Web-based resource, or specify the use of an interactive whiteboard, they cannot describe how the resource will be used, or how to access the whiteboard system. Another limitation is that, in activities other than assessment, Web Collage models the kind of task to be performed using natural language only in the description of each activity. Also, as Figure 7 shows, the model of the scenario can quickly become complex. Web Collage currently models activity sequencing, group creation and assessment. However, the scenario used in this article also included other aspects such as timings and the sharing of resources. Whether these aspects need to be modelled depends on how the scenario will be enacted: for example, to what extent the LMS will be used to manage the activities. Indeed, it is possible that these other aspects, which are interrelated, might be more easily configured using design patterns.

ScenEdit

ScenEdit8 (Emin, Pernin, & Aguirre 2010) is a Web-based authoring tool developed to structure and support teachers and trainers in the design and reuse of blended pedagogical scenarios in formal learning. It combines two approaches to the design process; namely:

Organise the scenario by eliciting formally the intentions of the teacher-designer and representing explicit learning strategies;

Facilitate the exploration of reusable components in the form of libraries adapted to specific communities of teachers.

ScenEdit is based on the Intentions, Strategies and interactional Situations (ISiS) intention-oriented conceptual model (Emin, Pernin, & Guéraud 2009). ISiS aims to capture teachers’ intentions and strategies so as to understand better the learning scenarios written by others, and to encourage sharing and reuse. ISiS describes a scenario in terms of three dimensions:

Intentional: teacher-designers’ intentions, which are linked to targeted knowledge items such as competences and abilities;

Strategic: related to teaching methods and strategies;

Tactical: related to the interactional situation: that is, the solution which will implement the intentions and strategies.

Interactional situations are interactions among a set of roles, tools, resources and locations: for example, “a certain role-in needs a resource-in to produce certain resource-out at a location, for a certain role-out”.

ScenEdit supports the visual description of learning scenarios according to these three dimensions of the ISiS model, including the locations where the activities take place, and the tools and resources to be used by the actors. This visual representation can be exported as an XML or PDF file and shared with other teachers. ScenEdit also provides templates or patterns of different kinds (i.e. intentions, strategies or interactional situations), which have been derived from examples of good practice uncovered in the literature or in communities of practice.

The “healthy eating” lesson, as modelled in ScenEdit, is shown in Figure 8 and 9. As can be seen in Figure 8, the different types of components in an ISiS scenario are represented by symbols: triangles denote steps, squares denote intentions, circles denote strategies and clapperboards denote interactional situations. The main aim of the lesson was for pupils to develop skills in undertaking inquiry-based learning tasks. The teaching method comprised a sequencing strategy in eight phases, modelled on the activities in the nQuire tool (e.g. “Collect my evidence”). Each of these eight activities was modelled through one or more interactional situations (e.g. “Collect data from the food diary meal” and “Organise data collected”).

The “healthy eating” lesson modelled using ScenEdit.

Detail of the “Collect data” activity in ScenEdit.

Figure 9 shows the ScenEdit screen on which the interactional situations are described. In terms of tools, each student needs a laptop, a camera and an Internet connection in order to access the nQuire project. In terms of resources, the nQuire website contains the data, either collected by students or provided by the teacher (diagrams, notes, description of objectives, links and so forth). Five roles were identified (pupil, group, class, teacher and nutritional expert), as well as a number of locations (including home and restaurant).

One of the main difficulties in modelling the lesson was that the scenario allowed activities to be carried out in any order, something that could not be modelled easily with ScenEdit. Using interactional situation design components, ScenEdit was able to model a sketch of the activities, specifying the kinds of resource, tool, location and role, together with the URLs linking to online materials such as the documents describing the activity, the tasks to be performed by the students and the resources they could access. However, this description, which is stored in an XML file, needs conversion tools in order to be implemented in, for example, an LMS.

Discussion: what can we learn from expressing the same learning design with multiple tools?

From the five design representations described above, we can see how different tools emphasise different aspects of a learning design, ranging from the abstract (e.g. the “learning experience” types of the Learning Designer or CADMOS) to the specific (e.g. the URLs of resources that can be defined in the IMS-LD compliant tools). We can also note how the use of different tools – and the abstractions and processes that they represent – to specify a learning design in computer-interpretable form can shape the resulting learning design in very marked ways. For example, learning designs specified using the Learning Designer contain more detail about the pedagogical composition of lessons or their timing, while designs created with CADMOS, Web Collage and OpenGLM provide more detail about the digital resources to be used. The same is true of other tools and formalisations, such as the textual and abstract descriptions of the “healthy eating” scenario used as the common base for this study. Thus, the transformations described in this paper are merely examples of the kinds of transformations which a teacher's pedagogical ideas may undergo when they are implemented in digital form (for a further discussion on this topic, see Muñoz-Cristóbal et al. 2012).

Throughout the preceding five sections of this article, we have noted the difficulties encountered when modelling the “healthy eating” scenario in the authoring tools. These difficulties, which we have synthesised in Table 1, were normally associated with face-to-face activities (e.g. classroom discussions) and physical resources (e.g. interactive whiteboards), and were more prominent in tools compliant with the IMS-LD standard (i.e. OpenGLM, CADMOS and Web Collage). When using these tools to model the “healthy eating” scenario, the research teams often had to circumvent their limitations by modelling activity and resource types in slightly counter-intuitive ways, in contrast to the tools that rely more on natural language descriptions and teaching–learning activity types derived from teaching practice (e.g. the Learning Designer). This apparent shortcoming stems from the fact that the IMS-LD standard was devised mostly with online education in mind and, more specifically, with a view to the automation of activities in IMS-LD compliant enactment environments.

Key features of the five learning design authoring tools and issues arising from modelling the “healthy eating” lesson in them.

Authoring tool Method of implementation Primary target audience/goal Pedagogic speciality Issues arising during modelling

Learning Designer • Manual • Teachers/expressing pedagogical ideas and collaborating in designing TEL Any approach (online, F2F, blended) • Difficult to visualise activities of greatly differing durations• Cannot model physical resources• Activities in scenario may correspond to more than one type of activity
OpenGLM • Semi-automated, in IMS-LD environments• Semi-automated in other environments via GLUE!-PS Teachers experienced in learning design/intuitive graphical IMS-LD support Mostly virtual activities • Need to merge small-scale activities into larger ones• Cannot model physical devices easily• Cannot model face-to-face activities easily
CADMOS • Exports directly to Moodle• Semi-automated, in IMS-LD environments• Semi-automated in other environments via GLUE!-PS Teachers with basic computer, learning standards skills/create, share and enact learning designs Mostly online activities • Cannot model face-to-face resources easily• Cannot model phase duration (only activity duration)• Difficult to model long or complex scenarios• No pedagogical analysis or advice
Web Collage • Semi-automated, in IMS-LD environments• Semi-automated in other environments via GLUE!-PS Teachers/design of computer-supported collaborative learning scenarios by non-experts Mostly online collaborative activities • Not all activities follow a collaborative pattern• Cannot specify activity duration• Cannot model access to hardware or physical resources• Tasks modelled with natural language descriptions only
ScenEdit • Semi-automated, in IMS-LD environments through XSL transformation Teachers/design and reuse blended pedagogical scenarios Any approach (online, F2F, blended) • Cannot represent unordered tasks• Cannot specify activity duration• Cannot model physical resources (only abstract resources)

Another issue that is closely related to the previous one – and, indeed, could be seen as its converse – is the ease with which a learning design can be implemented in a digital environment such as an LMS. As Table 1 shows, the tightly modelled elements of IMS-LD compliant tools are at an advantage in this respect, since the learning designs expressed with those tools can be implemented and managed with relative ease by execution environments that also comply with that standard (e.g. Coppercore and.LRN).9 In addition, CADMOS’ learning designs can also be exported to LMS such as Moodle (Boloudakis et al. 2012). On the contrary, designs produced with tools that take less account of the execution environment (such as the Learning Designer and ScenEdit) might be harder to implement for teachers without advanced ICT skills. This is because these tools require an ad-hoc mapping of their XML output either to the aforementioned IMS-LD standard10 or to the functionalities available within an LMS (as suggested by Bower et al. 2011). Interestingly, proposals have recently been made for the semi-automatic implementation of learning designs using applications that are agnostic with respect to the authoring tools used to create the designs (e.g. GLUE!-PS: Prieto et al. 2011). The deployment process could therefore be semi-automated even for non-IMS LD tools. Of course, purely face-to-face elements that do not use digital tools or resources would remain outside this “modelling for deployment” and would be implemented by teachers manually or through socially mediated operations.

We should also note that the five tools presented in this article are intended for different target audiences and stakeholders. For example, OpenGLM is more oriented towards specialists in learning design, while the others are aimed at teachers with varying degrees of technical and learning design knowledge (teacher-designers), as shown in Table 1. Learning design authoring tools typically have at least two different audiences: those who create the learning designs using the tool and those who enact (or realise) the learning designs with learners. Depending on the educational context, these two kinds of users might be the same or different individuals: some institutions may have specialist learning designers who produce, share and reuse the designs, which are then enacted by teachers, while in other institutions, both the production and enactment roles may be fulfilled by teacher-designers. Indeed, additional audiences may have a stake in these tools, such as managers and those responsible for the institutional teaching and learning strategy (Derntl et al. 2011, Masterman & Manton 2011).

Similarly, some of the tools are intended to support particular pedagogical approaches and/or educational contexts (see Table 1). For example, OpenGLM and CADMOS lend themselves specifically to the design of online courses in an IMS-LD environment, whereas the patterns in Web Collage and ScenEdit are intended to support collaborative learning. Other tools may support designing for multiple educational contexts, as is the case with the Learning Designer, which was developed in order to stimulate teachers to make greater use of technology-enhanced learning, but can also be used to design face-to-face learning with little or no use of ICT.

The research team responsible for developing each tool has explored its usability and usefulness to the learning design community in a number of studies. The Learning Designer was evaluated iteratively in workshops and individual guided walkthroughs with lecturers in UK universities, who appreciated the way in which the tool simultaneously supported and challenged their current practice, particularly through the analysis of students’ learning experience (Masterman & Craft this issue). The usability of OpenGLM and its usefulness in creating representations of the design of courses were evaluated with a sample of 21 instructors, with favourable responses (Neumann & Oberhuemer 2009). CADMOS has been evaluated by non-expert teachers in an MSc programme (Katsamani & Retalis 2012), and it was found that the tool had a friendly interface and was simple to use. They also valued the guidance offered during the learning design process and the implementation of learning designs in Moodle. The functionality for modelling collaborative scenarios in Web Collage has been validated both by non-expert teachers and by CSCL experts, who created designs for their own practice; however, further work needed is to address the challenges posed by complex designs (Villasclaras-Fernández et al. in press). The usability of ScenEdit has been evaluated with groups of secondary school teachers in France (Emin et al. 2009).

To summarise, different tools and processes may be a better fit to different teachers, depending on their pedagogical aims, institutional and technological contexts, and so forth. The example “healthy eating” lesson, with its emphasis on inquiry-based learning that involved substantial face-to-face work in the classroom and at home, was not equally suited to every tool in our selection. This fact in itself provides an additional insight: face-to-face activities have not been adequately considered in research into the modelling of learning designs, and so we recommend this as an avenue for future work in the field.

Conclusion

We have thus reached the end of this second “Rashomon” exercise, in which we have compared different learning design authoring tools. Our purpose has not been to declare a winner; rather, it has been to highlight the different ways in which the tools conceptualise and represent learning, and to provide readers with a means to decide which one might best suit their needs. As Sobreira and Tchounikine (2012) comment, “languages and editors are trade-offs, design decisions relating to matters of concern, targeted users and/or expected utilisations”. We hope that we have indeed provided informative insights into these trade-offs.

By looking from different points of view, and through the lenses of the different tools, we have also highlighted a number of outstanding issues that are relevant to the wider learning design community: namely, the difficulties of using computers to model learning and teaching that happens independently of technology; the difficulties of designing learning at different scales of time and complexity; and the various stakeholders who use, and who may be influenced by, the activity of learning design and its mediating tools. Even if a universal solution to these matters may be unattainable, we should continue to seek viable smaller-scale solutions that address individual contexts. Indeed, this is reflected in the continuing increase in authoring tools available to teachers: descriptions and resources for more tools can be found on the LD-Grid website,11 in addition to the ones analysed in this article.

Acknowledgements

This work has been partially funded by the Learning Design Theme Team funded by the European Union through the STELLAR Network of Excellence for Technology Enhanced Learning (FP7-IST-231913). The Learning Designer project was funded by the EPSRC/ESRC Teaching and Learning Research Programme (Technology Enhanced Learning: RES-139-25-0406).

Notes

1. Please refer to Persico et al. (this issue) for a more detailed description of the learning scenario.

2. http://www.nquire.org.uk.

3. Available at: https://sites.google.com/a/lkl.ac.uk/ldse/.

4. We use the term “teacher-designer” in this article when we wish to foreground the teacher's role as designer of their students’ learning as distinct from their role in enacting the learning design with the students.

5. Available at: http://sourceforge.net/projects/openglm/.

6. Available at: http://cosy.ds.unipi.gr/cadmos/.

7. Available at: http://pandora.tel.uva.es/wic2/.

8. Available at: http://scenedit.imag.fr/.

9. Available at: http://coppercore.sourceforge.net/ and http://www.dotlrn.org,respectively.

10. Indeed, an XSL transformation has been recently developed to translate ScenEdit's designs to IMS-LD level A.

11. http://www.ld-grid.org/resources/tools.

References Alexander C Ishikawa S Silverstein M A Pattern Language: Towns, Buildings, Construction 1977 Oxford, UK Oxford University Press Anastopoulou S Creating personal meaning through technology-supported science inquiry learning across formal and informal settings International Journal of Science Education 2012 34 2 251 273 Boloudakis M Orchestrating learning activities with CADMOS: from the design to the enactment Paper presented at the workshop Classroom Orchestration: Moving Beyond Current Understanding of the Field 2012 International Conference of the Learning Sciences (ICLS), Sydney, Australia. [online] Available at: https://sites.google.com/site/iclsorchestration2012/schedule/papers-1 Botturi L Botturi L Stubbs T Comparing visual instructional design languages: a case study Handbook of Visual Languages in Instructional Design: Theories and Practices 2007 Hershey, PA Idea Group 315 343 Bower M Using the Learning Designer to develop a conceptual framework for linking learning design tools and systems Paper presented at the LAMS International Conference 2011 Sydney, Australia 61 71 Conole G An overview of design representations Proceedings of the 7th International Conference of Networked Learning (NLC2010) 2010 Denmark Aalborg 482 489 Dalziel J Implementing learning design: the learning activity management system Proceedings of the 20th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE) 2003 Adelaide, Australia 593 596 Derntl M ISURE: report on usage of and recommendations for instructional modelling specifications 2011 ICOPER project deliverable D3.2 [online] Available at: http://www.icoper.org/deliverables/ICOPER_D3.2.pdf Derntl M The conceptual structure of IMS learning design does not impede its use for authoring IEEE Transactions on Learning Technologies 2012 5 1 74 86 Derntl M Neumann S Oberhuemer P Community support for authoring, sharing, and reusing instructional models: the Open Graphical Learning Modeller (OpenGLM) Proceedings of 10th IEEE International Conference on Advanced Learning Technologies, ICALT 2011 2011 Athens, Georgia, USA 431 435 Emin V Pernin J.-P Aguirre J.-L ScenEdit: an intention-oriented authoring environment to design learning scenarios Proceedings of the European Conference on Technology-Enhanced Learning (EC-TEL 2010) 2010 Barcelona, Spain 626 631 Emin V Pernin J.-P Guéraud V Model and tool to clarify intentions and strategies in learning scenarios design Proceedings of the European Conference on Technology-Enhanced Learning (EC-TEL 2009) 2009 Nice, France 462 476 Hernández-Leo D Collage, a collaborative learning design editor based on patterns Educational Technology & Society 2006 9 1 58 71 IMS Global Learning Consortium Learning design specification 2003 [online] Available at: http://www.imsglobal.org/learningdesign/ Katsamani M Retalis S Making learning design in layers. The CADMOS approach Proceedings of the IADIS Multi Conference on Computer Science and Information Systems 2011 Rome, Italy 305 312 KatsamaniM Retalis S Designing a Moodle course with the CADMOS learning design tool Educational Media International 2012 49 4 317 331 Laurillard D Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology 2012 New York, NY Routledge Laurillard D A constructionist learning environment for teachers to model learning designs Journal of Computer Assisted Learning 2013 29 1 15 30 Masterman E Manton M Teachers’ perspectives on digital tools for pedagogic planning and design. Technology Pedagogy and Education 2011 20 2 227 246 Muñoz-Cristóbal J. A Lost in translation from abstract learning design to ICT implementation: a study using Moodle for CSCL Proceedings of the European Conference on Technology-Enhanced Learning (EC-TEL 2012) 2012 Saarbrucken, Germany 264 277 Neumann S Oberhuemer P User evaluation of a graphical modeling tool for IMS learning design Proceedings of the 8th International Conference on Advances in Web Based Learning (ICWL 2009) 2009 Aachen, Germany 287 296 Papasalouros A Retalis S Papaspyrou N Semantic description of educational adaptive hypermedia based on a conceptual model Educational Technology & Society 2004 7 4 129 142 Persico D ‘Learning design Rashomon I – supporting the design of one lesson through different approaches’ Research in Learning Technology 2013 vol. 21 20224 Special Supplement on the Art and Science of Learning Design Prieto L. P GLUE!-PS: a multi-language architecture and data model to deploy TEL designs to multiple learning environments Proceedings of the European Conference on Technology-Enhanced Learning (EC-TEL 2011) 2011 Palermo, Italy 285 298 Rossi G Web Engineering. Modelling and Implementing Web Applications 2008 London, UK Springer-Verlag Sobreira P Tchounikine P A model for flexibly editing CSCL scripts International Journal of Computer-Supported Collaborative Learning 2012 7 4 567 592 Vignollet L Martel C Burgos D Editorial: comparing educational modelling languages on the “planet game” case study Journal of Interactive Media in Education 2008 2008 2 [online] Available at: http://www-jime.open.ac.uk/jime/article/view/2008–16 Villasclaras-Fernández E. D Pozzi F Persico D Implementing computer-interpretable CSCL scripts with embedded assessment: a pattern based design approach Techniques for Fostering Collaboration in Online Learning Communities: Theoretical and Practical Perspectives 2011 Hershey, PA IGI Global 261 277 Villasclaras-Fernández E. D Web collage: an implementation of support for assessment design in CSCL macro-scripts Computers & Education 2013