Orchestrating learning activities using the CADMOS learning design tool

Maria Katsamani^* and Symeon Retalis

Department of Digital Systems, University of Piraeus, Piraeus, Greece

Abstract

(Received 7 March 2012; final version received 10 June 2013; Published 16 September 2013)

This paper gives an overview of CADMOS (CoursewAre Development Methodology for Open instructional Systems), a graphical IMS-LD Level A & B compliant learning design (LD) tool, which promotes the concept of “separation of concerns” during the design process, via the creation of two models: the conceptual model, which describes the learning activities and the corresponding learning resources, and the flow model, which describes the orchestration of these activities. According to the feedback from an evaluation case study with 36 participants, reported in this paper, CADMOS is a user-friendly tool that allows educational practitioners to design flows of learning activities using a layered approach.

Keywords: learning design; learning design tools; CADMOS; separation of concerns; orchestrations of learning activities

^*Corresponding author. Email: marykatsamani@gmail.com

Research in Learning Technology 2013. © 2013 M. Katsamani and S. Retalis. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) Licence (http://creativecommons.org/licenses/by/3.0/) permitting use, reuse, distribution and transmission, and reproduction in any medium, provided the original work is properly cited.

Citation: Research in Learning Technology Supplement 2013, 21: 18051 - http://dx.doi.org/10.3402/rlt.v21i0.18051

Introduction

“Design for learning” has been defined as the process of “designing, planning and orchestrating learning activities as part of a learning session or programme” (Koper 2005). A “learning design” (LD) is the outcome of this design process. LDs are created by a range of different people, with different levels of skills and expertise. Like many of the researchers involved in this area, we are particularly interested in supporting the work of educational practitioners (i.e. “teachers as designers”), who have only basic computer skills and who are not experts in LD specifications like IMS-LD. Such practitioners may be involved in creating a LD for a simple activity, for a course lasting a small number of hours, for a course lasting a few weeks or even months, or a curriculum for a teaching programme lasting a year (Britain 2004).

Experts in the LD field (Conole 2008; Hernández-Leo et al. 2007, 2011; Hilera et al., 2010; Wichmann, Engler, and Hoppe 2010) argue that a LD must be shared and reused among the community. However, teachers are used to creating LDs in paper either in a narrative format or by using graphical methods, which are not formal representations. Although the IMS-LD specification (IMS Global Consortium 2003) can formally describe any design of teaching learning processes for a wide range of pedagogical approaches (Koper 2001; Koper and Olivier 2004), creating the appropriate xml files, it is not an easy process for the teachers (Barchino et al. 2012). Therefore, visual LD languages and graphical LD tools were proposed in order to simplify the authoring process. On the one hand, the visual LD languages, e.g. E2ML (Botturi 2006), coUML (Derntl and Renate 2007), PCeL (Figl and Derntl 2006), and PoEML (Caeiro 2008), provide specific notations (symbols and rules) for creating a design, but they are not accompanied by a tool and they do not provide explicit support for practitioners’ design decision-making. On the other hand, the graphical LD tools, e.g. COMPENDIUM (Conole et al. 2008), MOT+ (Paquette et al. 2008), LAMS (Dalziel 2003), OPEN-GLM (Derntl, Neumann, and Oberhuemer 2011), and WEBCOLLAGE (Dimitriadis 2010), are based on specific design principles and philosophies and support the practitioner during the design process via a user-friendly visual design environment. Languages and tools have advantages and disadvantages that influence the manner and extent of their usage.

LD experts have reported different teachers’ requirements/needs in LD—both for visual instructional languages and design tools—but they all seem to agree that the most important of them are usability, guidance, formalization, pedagogical neutrality, design flexibility and the use of design patterns (Botturi et al. 2006, 2008; Figl and Derntl 2006; Griffiths et al. 2005; Koper 2006). In fact, there is no ideal visual LD language or tool that can fully meet all the aforementioned criteria. It is still an open research problem. Trying to solve this problem, we have developed a new graphical LD tool called CADMOS, which supports the design of courses (or “units of learning”) with duration of up to a few hours. One of the innovative aspects of this tool, which is addressed to teachers who are not experts in LD and have basic computer skills, is that it offers guidance by splitting the design process into two interrelated stages: (1) the creation of a conceptual learning activity model which contains the learning activities and the corresponding resources, and (2) the creation of a flow model which contains the orchestration of these learning activities.

The structure of the paper is as follows. In the next section, we compare five popular LD tools, according to teachers’ basic design requirements. In the third section, the CADMOS tool is briefly presented via an example. Then, the evaluation comments that were made about the tool by 36 practitioners (most of them in-service teachers) as well as the outcomes from the evaluation of the participants’ designs are presented. The paper ends with some concluding remarks and plans for future work.

About graphical LD tools

It is well documented that practitioners prefer tools that allow them to specify the learning activities and orchestrate them according to some rules/principles via a graphical user interface (Koper 2006). As mentioned before, LD experts agree that the most important requirements on designing are:

• Usability: A tool must be easy to learn and use.
• Guidance: A tool should guide practitioners through the design process. A tool should provide ready-to-use patterns for those who would like to embody easily a learning strategy in their design.
• Formalization: A tool must have a formal representation model, meaning a “closed set of concepts and rules for composition of concepts in order to describe designs” (Botturi et al. 2006). In that way, it could provide interoperability between different tools, e.g. a design that would be created in a specific environment to be opened and edited in another environment like IMS-LD.
• Pedagogical Neutrality: A tool should be able to represent any pedagogical theory and any teaching subject.
• Design Flexibility: A tool should describe different entities from different perspectives, so that a designer could edit an entity in the design without impacting another.

We compared five popular LD tools according to the aforementioned criteria and we present the results in Table 1.

From Table 1, we can see that the users of MOT+ must have specialized knowledge of the LD standards and they have to understand the complex underlying modeling approach (Paquette et al. 2006), while COMPENDIUM, WEBCOLLAGE, OPEN-GLM and LAMS are less demanding so that they can be easily used by any teacher with basic knowledge and skills in computers. WEBCOLLAGE offers guidance to a practitioner, who can initially decide which learning strategy to use (in the format of learning activity flow patterns) and then follow a set of steps on how to organize the course. OPEN-GLM and COMPENDIUM provide elementary guidance through pop-up windows or forms that ask designers to complete specific metadata and through ready-to-use design patterns. The rest of the tools offer no guidance at all.

COMPENDIUM, WEBCOLLAGE, MOT+ and OPEN-GLM offer pedagogical neutrality – WEBCOLLAGE only for collaborative scenarios. MOT+, WEBCOLLAGE and LAMS allow the user to export a lesson in the standard IMS-LD Level A, while OPEN-GLM exports a LD in IMS-LD level A & B and COMPENDIUM is not compatible to IMS-LD. Finally, only MOT+ fulfils the criterion of design flexibility, by giving the teacher the possibility to create different but interrelated design models.

From our review, it seems that there is no LD tool that satisfies all of the aforementioned criteria. This fact motivated us to develop a graphical LD tool, which should:

1. be appealing to novice designers who do not have specialized knowledge of the LD specification
2. guide practitioners through the design process
3. allow teachers to design any course using any instructional method
4. enable practitioners to design learning activities from different perspectives and in different layers
5. have a formal graphical notation and a meta-model for facilitating the export of designs in IMS-LD format as well as import of IMS-LD designs for further editing

The CADMOS tool explained via a short example

CADMOS supports the “separation of concerns” notion in the LD process. This concept stems from the principles of web engineering (Papasalouros, Retalis, and Papaspyrou 2004; Rossi et al. 2008) and argues that the designer should build the design in layers, creating two design sub-models: the conceptual and the flow model:

• The conceptual model defines the learning activities in which the different roles of the course will be engaged. Each learning activity may be simple or composite. A simple activity is a task that is assigned to one course-related role. A composite learning activity consists of two or more simple learning activities of the same role (e.g. student, teacher). In this model, the learning resources/services that correspond to each one of these activities are also defined.
• The flow model defines the navigational patterns (orchestration) of the learning activities. It describes the chronological order of the activities, any rules that affect their execution and the separation of the activities’ flow into phases.

CADMOS advocates that if a practitioner wants to add navigational rules to a set of learning activities, s/he can change the flow model, keeping the conceptual model intact. Thus, with the same set of learning activities, different practitioners can produce variations of learning activity flows according to their own instructional philosophy or learning context. For example, one practitioner might decide that students must study the syntax and semantics of a higher-level programming language, as well as accessing various problem-solving examples, before tackling the assignments. Or the student may not be allowed to take the final assignment before having performed all the suggested learning activities (e.g. study theory and examples and submit earlier assignments). It is obvious that a practitioner can change a learning object (resource), which is linked to a learning activity in the conceptual model, without changing anything in the flow model. On the other hand, if the practitioner adds or deletes a learning activity in the conceptual model, the flow model must embody these changes.

The CADMOS tool has been created for novice learning designers, i.e. practitioners with basic computer skills and basic knowledge of LD standards. The whole design process supported by the CADMOS tool is considered to be incremental: The practitioner first defines the learning activities and then moves to the definition of their orchestration. If s/he wants to add or remove an activity s/he can return to the conceptual model, make the changes and then revise the flow model accordingly. S/he can also edit just the flow model, i.e. add rules, conditions or phases in the navigation between the activities without making any changes to the learning activities or the learning resources that are linked to those learning activities.

Figures 1 and 2 show the outcome of a CADMOS LD process for a learning session named “Programming Fundamentals” – part of a programming course in the last grade of high school (aged 16–17 years). First, at the conceptual level, the practitioner needs to make an overall design of the learning activities and specify the learning objects/services that will be used by the students. This specific learning session consists of six main learning activities according to the IEEE/ACS computer science curriculum (Kornecki 2008); five of them are simple learning activities and one is a composite activity, comprising of two simple ones. In the first activity, each student studies different representations of algorithms. The second composite activity is performed by groups of two students and is decomposed into two simple activities: (1) the study of theory about the syntax and the semantics of a higher-level language, and (2) the step-by-step monitoring of an algorithmic problem-solving process using a higher-level language. Then each student solves some algorithmic problems and the teacher sends feedback. Finally, there is a self-assessment activity at the end of the session, with open and closed questions that each student must answer in order to get a final grade from the teacher.

Figure 1.  The LD conceptual model of the course “Programming Fundamentals.”

The first activity is a simple learning activity of the “theory” type, which is linked to a hypermedia learning object. The second activity is a composite activity which consists of (1) a “theory” type activity, which is linked to a hypermedia resource that presents the basic commands of algorithms, and (2) an “example” type activity, which is linked to a flash-based learning object. The third and the fifth activity type is “assessment”. They are linked to quiz services. Finally, the fourth and the last activity are “feedback” type activities, which are linked to “hypertext” resources. Figure 1 shows the LD conceptual model designed using the CADMOS tool.

The largest part of the screen is the white workspace/canvas, where the practitioner can create the LD. Above the workspace, the pressed button named “LD ConceptualModel” indicates that the conceptual model is being designed. On the left side of the workspace there is the toolbar, which consists of the schemata that defines a composite activity, a simple activity and a resource. Also, there is a “Links” part that consists of an arrow that connects the title of the course with composite and simple activities and composite with simple activities and a dashed line that connects a simple activity with a resource. When the practitioner presses the button of an activity or a resource, the corresponding icon appears on the workspace. Then s/he has to define the properties of each object, in order to complete the design (e.g. title, type, description). A practitioner can relate several specific learning objects or services to a learning activity.

After creating the conceptual model, the practitioner moves to the next layer, i.e. specification of the flow model, which concerns the orchestration of the learning activities. Pressing on the “LD FlowModel” button will cause the CADMOS tool to automatically create the flow of the learning activities, based on the idea of swim lanes. This idea can also be found in the CompendiumLD tool (Conole 2008). The tool creates a swim lane for every role of the course, by placing the corresponding activities one after the other in chronological order. Then the designer can modify the flow by changing the order of the activities, adding rules for the execution of the activities and defining any phases of the flow. Figure 2 shows the flow model for the learning scenario entitled “Programming Fundamentals.” As we can see, there are three different swim lanes, one for the student's activities, one for the group's activities and one for the teacher's activities.

Figure 2.  The LD flow model of the course “Programming Fundamentals” with rules.

The tool contains three different types of rules:

• The “User Choice” rule indicates that a specific activity will be completed when its actor wants to.
• The “Time Limit” rule shows that an activity should be completed by a specific time.
• The “Score Condition” (if–then–else) rule helps the designer define which activity will be done if the score of an activity is above a threshold or not.

Also, the teacher can divide the learning tasks into “phases,” which is a way to divide the flow into different sections. This is very useful especially when a learning scenario follows a specific strategy (e.g. Think–Pair–Share) or a learning flow pattern (Hernández-Leo et al. 2010).

Apart from the aforementioned rules, the designer may use a “comments” icon to add comments next to the learning tasks, if s/he wants to explain other rules. After creating the flow model, the LD can be saved in CADMOS format or exported as an IMS/LD level A & B xml manifest file, so as to be reused by other IMS/LD editors or players. Finally, the tool has the ability to import an existing IMS/LD level A manifest file and represent it in its own model.

User evaluation

Scope

The CADMOS tool was evaluated systematically in January 2011 by 36 out of 39 MSc students attending the course on “LifeLong Learning” of the MSc programme on e-learning technologies at the Department of Digital Systems, University of Piraeus. A total of 25 of them were teachers (20 being high school teachers and five elementary school teachers) and 30 of them had previously used LD tools like LAMS, MyUdutu and Dialog Plus.

The purpose of this evaluation case study was to see if:

1. CADMOS's underlying LD philosophy, which advocates the use of two LD views, is well accepted by the teachers–designers and can guide them when creating LDs
2. Teachers feel that the CADMOS tool provides them with all the elements needed to design their lesson in an easy and simple manner
3. Teachers are satisfied with the tool and think that the interface and its functions are simple enough for it to be used by teachers with basic computer knowledge

Process

The evaluation case study consisted of two phases:

• Phase 1: presentation of the CADMOS tool in a laboratory where the students experimented with the tool by creating a LD for a prescribed learning scenario suggested by the Greek Pedagogical Institute and which was given in a narrative descriptive form (3 hours)
• Phase 2: the students were asked to submit two LDs using CADMOS; one was a learning scenario given in a narrative form and another was a design of a learning unit of their choice (1 week).

During this evaluation case study, students had access via the Moodle platform to the CADMOS manual, informative material about the tasks as well as a web forum for posing questions and asking for technical assistance.

Evaluation toolkits

When teachers submitted their designs, they were asked to answer an online questionnaire comprised of 25 questions. More specifically, it contained 22 close-ended questions rated using a five-point Likert Scale (Totally Disagree, Disagree, Neutral, Agree, Totally Agree) and three open-ended questions. The first set of close-ended questions investigated whether the CADMOS-supported LD method enabled participants to model complete, pedagogically flexible, adaptable and reusable scenarios (Koper 2006). The second part was related to measurement of the usability of the CADMOS tool, and of user satisfaction, as well as getting input from the participants about further improvements to the tool. In the open-ended questions, the participants had to note positive or negative comments about the tool and any software bugs.

All submitted LDs were analysed using an assessment rubric that measured the following criteria:

1. Completeness in description/documentation of the LD (presence of well-written goals and prerequisites, defined roles, well described metadata in learning activities/resources, full correspondence between activities and resources)
2. Plot of the learning scenario as it is described via the conceptual model
3. Plot of the learning scenario as it is described via the flow model
4. Expressiveness of the LD representation (names of the learning activities, the learning resources, phases, rules)
5. Appropriateness of the proposed learning resources (objects and services) in relation to the activities and the learning goals
6. Visualization of the scenario
7. Creative LD that could promote collaboration, active learning and the quality of interaction.

Findings

The evaluation comments were positive. Table 2 shows a sample of the data collected from the questionnaire.

The majority of the participants (69%) were satisfied or very satisfied with the CADMOS method and the tool. 100% of them stated that the use of the tool was simple and they learned to use it easily, while 97% of them said that they managed to effectively complete the LD. 69% of the participants said that were satisfied with the guidance that the tool offered during the design and only 17% of them claimed that CADMOS did not have all the necessary functions for the process of designing. These students stated that they wanted to be able to assign more than one learning goal or prerequisite to a learning activity at the conceptual design model.

A very important finding was that everybody said that the design process, using the two different models (conceptual and flow) was very helpful. All of them declared that the creation of the conceptual model is simple and 97% of them stated that the editing of the flow model is simple and easy.

In the open-ended questions, teachers commented that the CADMOS tool could facilitate collaboration among various designers thanks to its visual metaphors. They reported as extremely useful the fact that they could export their LDs in IMS-LD level format and open them in any LD player. As expected, they mentioned some software bugs during the use of the tool, which have been fixed. Finally, they mentioned that it would be very useful to integrate ready-to-use design templates in the tool.

The LDs were evaluated according to the aforementioned rubric using a scale from 1 to 3 (1, low score; 2, medium score; 3, high score) (Allen and Tanner 2006; Arter and McTighe 2001). Table 3 shows the mean rates per evaluation criterion, for the submitted LDs:

Table 3 shows that the participants were able to create a better-structured conceptual model than they could a flow model. This demonstrates that the teachers can more easily create the concept map of the proposed activities, and the related learning resources/services, than their flow. In the future, the tool could contain learning flow design patterns in order to help the designers in the orchestration of the activities. Moreover, we note that the highest rate, is in “Visualisation of the scenario,” showing that CADMOS environment is easy to use and understand. Also, practitioners’ LDs have a high mean rate on the criterion of “Creative LD that could promote collaboration, active learning and the quality of interaction,” which means that they escaped from the traditional lecture-based teaching model and tried to offer challenging and stimulating collaborative learning activities. The low mean rate in the criterion of “Appropriateness of the proposed learning resources in relation to the activities and the learning goals” shows that teachers have difficulties in choosing the most appropriate learning resources and services for supporting the proposed learning activities.

Discussion and future work

This paper presented the CADMOS tool, a graphical LD editor that is intended to be used by learning designers with basic computer skills and without any technical knowledge of the IMS-LD. The case study showed that this tool allows the easy creation of a LD by specifying and structuring two separate but interrelated models, i.e. the conceptual model, which describes the activities of the course and the corresponding learning resources and the flow model, which describes the orchestration of these activities. This paper discussed CADMOS tool, version 1.6. The tool is being continuously upgraded and newer versions are being released. At the moment, we are working on adding into CADMOS the functionality of exporting its LDs in an appropriate format in order that they can be enacted as online Moodle courses. So, we are examining the mapping among the elements of CADMOS and Moodle. Also, our intention is to add more rules in the flow model, as well as ready-to-use design templates for different learning strategies, such as Think Pair Share (TPS), JIGSAW, PYRAMID and Predict Observe Explain (POE). Finally, our scope is to organize several case studies with teachers for further validating the way CADMOS supports the design process.

Acknowledgements

This work has been partially supported by the “PAKE Attikis & Stereas Elladas” project: “Teachers Professional Development on the use of educational technologies in classroom practice” funded by the Greek Ministry of Education. The tool can be found at: http://cosy.ds.unipi.gr/CADMOS

References

Allen, D. & Tanner, K. (2006) ‘Rubrics: tools for making learning goals and evaluation criteria explicit for both teachers and learners’, Life Sciences Education, vol. 5, pp. 197–203. PubMed Abstract | PubMed Central Full Text | Publisher Full Text

Arter, J. & McTighe, J. (2001) Scoring Rubrics in the Classroom: Using Performance Criteria for Assessing and Improving Student Performance, Corwin Press, Thousand Oaks, CA.

Barchino, R., et al., (2012) ‘Interoperability between visual UML design applications and authoring tools for learning design’, Information and Control, International Journal of Innovative Computing, vol. 8, no. 1(B), pp. 845–865.

Botturi, L. (2006) ‘E2ML: a visual language for the design of instruction’, Educational Technology Research & Development, vol. 54, no. 3, pp. 265–293.

Botturi, L., et al., (2008) ‘Comparing visual instructional design languages: a case study’, in Handbook of Visual Languages in Instructional Design: Theories and Practices, eds L. Botturi & T. Stubbs, Hershey, PA: Idea Group, pp. 315–343.

Botturi, L., et al., (2006) ‘A classification framework for educational modeling languages in instructional design’, Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies, (ICALT 2006), July 5–7, Kerkrade, The Netherlands, pp. 1216–1220

Britain, A. (2004) ‘A review of learning design: concept, specifications and tools’, A report for the JISC E-learning Pedagogy Programme, [Online] Available at: http://www.jisc.ac.uk/uploaded_documents/ACF83C.doc

Caeiro, M. (2008) PoEML: A Separation-of-Concerns Proposal to Instructional Design. Handbook of Visual Languages for Instructional Design: Theories and Practices. Information Science Reference, eds L. Botturi & T. Stubbs, Springer-Verlag, Berlin, pp. 183–207.

Conole, G. (2008) ‘Capturing practice: the role of mediating artefacts in learning design’, in Handbook of Research on Learning Design and Learning Objects: Issues, Applications and Technologies, eds L. Lockyer, S. Bennett, S. Agostinho & B. Harper, IGI Global, Hersey PA, pp. 187–207.

Conole, G., et al., (2008) ‘Visualising learning design methodology to foster and support creativity in design’, Journal of Educational Media International, vol. 45, no. 3, pp. 177–194. Publisher Full Text

Dalziel, J. R. (2003) ‘Implementing learning design: the learning activity management system (LAMS)’, in Interact, Integrate Impact: Proceedings of the 20th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, eds G. Crisp, D. Thiele, I. Scholten, S. Barker & J. Baron. Adelaide, pp. 25–29.

Derntl, M., Neumann, S. & Oberhuemer, P. (2011) ‘Community support for authoring, sharing, and reusing instructional models: the open graphical learning modeler (OpenGLM)’, Proceedings of IEEE ICALT 2011, 11th IEEE international conference on advanced learning technologies. Athens, GA, pp. 431–435.

Derntl, M. & Renate, M. (2007) ‘coUML–a visual language for modeling cooperative environments’, in Handbook of Visual Languages for Instructional Design: Theories and Practices, Information Science Reference, eds L. Botturi & T. Stubbs, Springer-Verlag, Berlin, pp. 155–184.

Dimitriadis, Y. A. (2010) ‘Supporting teachers in orchestrating CSCL classrooms’, Proceedings of the 7th Pan-Hellenic Conference with International Participation: Information and Communication Technologies in Education, Korinthos, Greece, pp. 71–82.

Figl, K. & Derntl, M. (2006) ‘A comparison of visual instructional design languages for blended learning’, in Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2006, eds E. Pearson & P. Bohman, AACE, Chesapeake, VA, pp. 941–948.

Griffiths, D., et al., (2005) ‘Learning design tools’, in Learning Design, a Handbook on Modelling and Delivering Networked Education and Training, eds R. Koper & C. Tattersall, Springer-Verlag, Berlin, pp. 109–135.

Hernández-Leo, D., et al., (2007) ‘A framework for the conceptualization of approaches to “Create-by-Reuse” of learning design solutions’, Journal of Universal Computer Science, vol. 13, no. 7, pp. 991–1001.

Hernández-Leo, D., et al., (2010) ‘A multicase study for the evaluation of a collaborative learning pattern-based visual design approach’, Journal of Visual Languages and Computing, vol. 21, no. 6, pp. 313–331. Publisher Full Text

Hernández-Leo, D., et al., (2011) ‘LdShake: learning design solutions sharing and co-edition’, Computers & Education, vol. 57, no. 4, pp. 2249–2260.

Hilera, J. R., et al., (2010) ‘Reusing and sharing learning designs in a virtual university: a real case’, Proceedings of ED-MEDIA, June 28–July 2, 2010, Toronto, Canada, pp. 460–465.

IMS Global Consortium. (2003). IMS Learning Design Specification, [Online] Available at: http://www.imsglobal.org/learningdesign/

Koper, R. (2001) Modeling Units of Study from a Pedagogical Perspective – The Pedagogical Metamodel Behind EML Open University of the Netherlands, Technical report, Heerlen, The Netherlands.

Koper, R. (2005) ‘An introduction to learning design’, in Learning Design. A Handbook on Modeling and Delivering Networked Education and Training, eds R. Koper & C. Tattersall, Heidelberg: Springer-Verlag, pp. 3–20.

Koper, R. (2006) ‘Current research in learning design’, Educational Technology & Society, vol. 9, no. 1, pp. 13–22.

Koper, R. & Olivier, B. (2004) ‘Representing the learning design of units of learning’, Educational Technology & Society, vol. 7, no. 3, pp. 97–111.

Kornecki, A. J. (2008) ‘Computing curricula for the 21st century’, IEEE Distributed Systems Online, vol. 9, no. 2, pp. 1–5. Publisher Full Text

Papasalouros, A., Retalis, S. & Papaspyrou, N. (2004) ‘Semantic description of Educational Adaptive Hypermedia based on a Conceptual Model’, Educational Technology & Society, vol. 7, no. 4, pp. 129–142.

Paquette, G., et al., (2008) ‘The MOT+ visual language for knowledge-based instructional design’, in Handbook of Visual Languages for Instructional Design. Theories and Practices, eds L. Botturi & T. Stubbs, IGI Global, Hersey PA, pp. 132–153.

Paquette, G., et al., (2006) ‘Learning design based on graphical knowledge modelling’, Educational Technology & Society, Special issue on Learning Design, vol. 9, no. 1, pp. 97–112.

Rossi, G., et al., eds. (2008) Web Engineering. Modeling and Implementing Web Applications, Springer-Verlag, London.

Wichmann, A., Engler, J. & Hoppe, H. U. (2010) ‘Sharing educational scenario designs in practitioner communities’, Proceedings of the 9th International Conference of the Learning Sciences, June 29–July 2, 2010, Chicago, IL, pp. 750–757.