Figure 1. Types of educational interactions. Diagram from Anderson (2004, p. 46). Used under the Creative Commons license.
Brenda Cecilia Padilla Rodrigueza* and Alejandro Armellinib
aInstitute of Learning Innovation, University of Leicester, Leicester, UK; bInstitute of Learning and Teaching in Higher Education, University of Northampton, Northampton, UK
Abstract
While learning is commonly conceptualised as a social, collaborative process in organisations, online courses often provide limited opportunities for communication between people. How do students engage with content-based courses? How do they find answers to their questions? How do they achieve the learning outcomes? This paper aims to answer these questions by focusing on students’ experiences in an online content-based course delivered in a large Mexican organisation. Sales supervisors (n=47) participated as students. Four main data sources were used to evaluate engagement with and learning from the course: surveys (n=40), think-aloud sessions (n=8), activity logs (n=47) and exams (n=43). Findings suggest that: (1) Students engage with a content-based course by following the guidance available and attempting to make the materials relevant to their own context. (2) Students are resourceful when trying to find support. If the materials do not provide the answers to their questions, they search for alternatives such as colleagues to talk to. (3) Content-based online learning designs may be engaging and effective. However, broadening the range of support options available to students may derive in more meaningful, contextualised and rewarding learning experiences.
Keywords: learning design; training; student engagement; content-based learning; workplace learning
*Corresponding author. Email: brenda.padilla@yahoo.com.mx
Published: 6 September 2013
RLT 2013. © 2013 B.C. Padilla Rodriguez and A. Armellini. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) Licence (http://creativecommons.org/licenses/by/3.0/) permitting use, reuse, distribution and transmission, and reproduction in any medium, provided the original work is properly cited.
Citation: Research in Learning Technology 2013, 21: 22106 - http://dx.doi.org/10.3402/rlt.v21i0.22106
Learning is commonly conceptualised as a social, collaborative process, in which people communicate and actively build knowledge. Students who work and share ideas with others are generally more motivated and display better academic performance than passive students (Beaudoin 2002; Swan 2002). Several authors have emphasized the importance of fostering interactions between people, or social interactions, in online educational contexts (Woo and Reeves 2007).
Social interactions are valuable as a means to improve student engagement (Zepke and Leach 2010), which is critical to the effectiveness of learning activities (Kuh 2009; Noe, Tews, and McConnell Dachner 2010). In this paper, student engagement refers to the way in which participants interact with course materials and activities to achieve learning outcomes. This term has also been defined as students’ involvement in their own learning process (Axelson and Flick 2010), or the time and effort students devote to learning activities (Kuh 2009).
Despite the acknowledged importance of social interactions for student engagement and learning, in organisations online courses often provide limited or no opportunities for communication between people (e.g., Padilla Rodriguez and Fernandez Cardenas 2012; Welsh et al. 2003). Sometimes just-in-time, just-for-me demands (e.g., a single person requiring training) make the delivery of online courses with social interactions unviable. In such contexts, content-based learning designs constitute an option.
This paper focuses on students’ experiences in an online content-based course delivered in a large Mexican organisation with a high geographical dispersion. Specifically, it addresses the following questions: How do students engage with content-based courses? How do they find answers to their questions? How do they achieve learning outcomes?
In this paper, content-based learning design refers to a way of organising a course that focuses on fostering learner–content interactions and includes no activities to enable communications between people. Moore (1989) describes learner–content interaction as an intellectual process that results in changes in learners’ perspective, understanding or cognitive structures. This implies processes such as analysing the material, relating it to previous knowledge or applying it to problem solving; in other words, using the content to perform activities that can enhance learning (Abrami et al. 2011).
Internet-enabled devices and tools make a variety of learner–content interactions possible. These include replaying sections of a podcast, searching information, following links to glossary entries, answering multiple-choice questions and checking automatic feedback (Anderson 2003; Caladine 2008). The capability to design and deliver these learner–content interaction opportunities has increased with the maturity of institutional virtual learning environments (often referred to as learning management systems), higher levels of connectivity and digital literacy.
Learner–content interactions can be designed to perform some of the functions traditionally carried out by teachers (Anderson 2003), such as suggesting a learning pathway. They can foster flexibility by enabling participants to work independently, at their own pace and in their own time. This type of interactions can contribute to the achievement of learning outcomes and course completion: the more learners interact with the content, the better grades they tend to achieve (Zimmerman 2012).
While this type of interaction has advantages, focusing only on learner–content interactions excludes the potential benefits of other types of educational interactions (see Figure 1). For example, exchanges between peers can create meaningful learning experiences (Anderson and Garrison 1998; Conole 2013; Salmon 2011), which can help relate new information to previous knowledge and facilitate problem solving (Mayer 2002). Online course participants tend to value opportunities to work and share ideas with others (Chang and Smith 2008; Su et al. 2005).
Figure 1.
Types of educational interactions. Diagram from Anderson (2004, p. 46). Used under the Creative Commons license.
Other issues may arise when no clear teaching presence is deployed on a course (Garrison, Cleveland-Innes, and Fung 2010). Students may be tempted to browse through the content at speed, avoiding the difficult areas and thus reducing potential educational benefits. Even if learners are given the option of consulting a tutor, few do, for fear of representing an inconvenience (Cotton and Gresty 2007). The lack of guidance is often evident in content-based courses, which can be an isolating experience for students: it could lead to confusion when there is no one available to answer questions (Padilla Rodriguez and Fernandez Cardenas 2012; Smedley 2011). This paper seeks to deepen our understanding of students’ experiences in an online content-based course.
This study took place at a large Mexican organisation (+6000 employees) with 30 distribution centres and offices in the country. As part of a Leadership Programme delivered via the e-learning platform Moodle, sales supervisors had to study a content-based course on Performance Feedback. This course aimed to improve the communication competence of employees in charge of managing retailers. Participants had one week to finish the course, with a commitment of approximately five study hours.
The design of the course incorporated six non-assessed activities that fostered interactions with the content and required explicit, observable responses from the students; for example, providing an answer to a question instead of reflecting internally on a topic. Table 1 presents the online tools included, their purpose and characteristics.
| Tool | Purpose | Characteristics |
| Hyperlinks | To link key terms to glossary definitions. | Available in the reading materials |
| Personal wikis | To provide an individual space for students to write their reflections. | Only accessible to the owner of the wiki and the administrators of the course Used as an alternative to blogs, which were blocked in the organisation's e-learning platform |
| Multiple-choice questions | To encourage students to practise and reflect on the course concepts. | Automated feedback provided for both correct and incorrect answers |
| Polls | To stimulate thinking on the topic and how it relates to others’. | Enabled students to see the general responses of the group |
| Podcasts | To make content more user-friendly by the use of the human voice (Nie et al. 2010). | Brief (less than a minute) Included text transcripts |
| Discussion forum | To offer a channel of general support. | Only built-in communication tool available Monitored by Education Department staff |
| Final exam | To assess learning. | Included only closed questions, which were automatically graded Mandatory for course accreditation |
Sales supervisors (n=47, 14 women and 33 men) studied a content-based course on Performance Feedback. Their age ranged from 25 to 57 with a mean of 38 years. Their average tenure was 5 years. The average time in their current job was 4 years. Nine months before the study, they had received a netbook computer. They had had weekly compulsory training to learn the basics of technology use. All participants had had online learning experiences in the organisation.
Four people dropped out of the course at different stages. Ten sales supervisors who did not participate in the course formed a control group.
Four main data sources were used, as shown in Table 2.
Two online surveys were used in this study to obtain an insight into individual perceptions and tendencies within groups (Baruch and Holtom 2008). One focused on diagnosis, gauging previous knowledge on the topic. It had an informed consent statement at the beginning. The second survey concentrated on course evaluation. It included closed questions about the study hours spent, perceived engagement with the activities, and students’ evaluation of course. Open questions explored learners’ perceived responsibility in relation to their own performance, and their suggestions to improve the course.
The think-aloud method consists of observing participants, while they verbally articulate their behaviours, feelings and thoughts as they engage with an activity. Throughout this process, the researcher's input is minimum, generally limited to prompts to keep talking when participants fall quiet. Data are audio recorded for further analysis (Young 2005).
The think-aloud method is recommended for the study of learner–content interactions (Anderson 2003) and for e-learning research (Cotton and Gresty 2007). Data are collected during the actual event of interest, providing reliable and accurate information. This method prevents problems associated with memory failure, which may occur when data are collected after the conclusion of the activity, and artificiality, which may happen if participants are asked to report on a hypothetical situation (Young 2005).
The Moodle log system provides interesting information about participants’ online behaviours and activities within a course (Estrada et al. 2011). Each log entry contains an action and an information field. These indicate that a click happened and specify what the user did. Course logs were checked and edited to only include the information of students.
A final exam with multiple-choice, matching and true/false questions evaluated knowledge acquisition.
At the beginning of the course, all participants received information about the study and answered a diagnostic survey. The researchers then used the think-aloud method to observe a convenience sample of eight students – located in two different cities – as they engaged with a content-based learning design. Data were audio recorded and transcribed.
The think-aloud transcripts were coded and analysed using NVivo software. Themes for categorization were based on students’ navigational decisions and potential evidence that learning was taking place.
At the end of the course, 43 students completed the exam and the evaluation survey. Central tendency measurements and percentages were obtained where applicable. Open questions were coded using emergent themes. Employees from the control group also sat the final exam.
Moodle log entries were checked and categorised as passive or active. Viewing a resource (e.g., a discussion forum, a wiki, a page with reading material, etc.) was considered passive. Views of the front (landing) page of the course were excluded. Active contributions included clicks that resulted in an observable response (e.g., editing a wiki, selecting a poll answer). Medians were obtained.
Finally, the information from the different data sources and methods was compared and contrasted. Figure 2 shows the procedure timeline.
Results were grouped according to the research questions: (1) how did students engage with a content-based course; (2) how did students find answers to their questions; and (3) how did students achieve learning outcomes.
The evaluation surveys indicate that participants generally followed the recommended structure and spent an average of four and a half hours on the course, out of the recommended five hours. Most students (35/40; 88%) reported being engaged or very engaged with the activities. Half claimed that there was nothing they could have done to further benefit from the course. Fifteen students said time had been an issue, but it was unclear whether they meant that senior management should give them more time to study, or that they should organise their time more efficiently.
Think-aloud data revealed that students used different strategies to make information more relevant or personalised. These included the following:
Some students read superficially, skimming through the text. However, activities seemed to encourage them to go back and spend time on deeper readings. One student explained it as follows:
Lots of times, […] we read once and think, “I've read”, and we answer; and then we read again and think, “If I had read twice, I would have answered correctly”. You won't gain anything by going too fast. It's better to take the necessary time to read better.
Activity logs provided evidence of students’ engagement with the content. Participants had a median of 86 clicks throughout the course, 29 related to active contributions and 55 to passive viewings. Less than half of the students (19/47) checked optional resources like podcasts and glossary entries.
During the think-aloud sessions, six of the eight participating students had questions that were not answered by the content of the course (e.g., Should I write it here? How do I do that?). Sometimes they would ask the researcher what to do. On other occasions, they would read their own notes and try to find answers there. According to the activity logs, all but one student (46/47) checked the automated feedback received in at least one activity. Incorrect answers seemed to encourage students to read again (e.g., I think I have an incorrect answer… I am going to read again and then I will answer [again]).
In the evaluation surveys, 29 out of 40 participants had no suggestions to improve the course, but three mentioned the importance of having embedded social interactions in the course. Students did not use the general discussion forum, which was available for questions and comments. Only six people viewed it during the duration of the course.
Although the course fostered no social interactions, the think-aloud method provided some evidence of potentially meaningful peer exchanges happening outside the virtual learning environment. During all of the sessions, either via phone calls or face-to-face interactions, work colleagues distracted students when they were navigating through the course. They interrupted to discuss job matters (e.g., retailers and sales), which were directly or indirectly related to the content of the course. Participants did not seem particularly bothered (or surprised) by these distractions, as colleagues also represented a source of support.
Students were asked whether they had use Moodle's private messaging system. Eighteen people answered. Fourteen had sent at least one private message to another participant. Ten had sent three or more messages.
The reading resources and activities were valuable for achieving learning outcomes. All survey respondents considered that the materials fostered their reflection on the course topics, and all but one (39/40) reported having learned ‘a lot’ or ‘very much’.
In the diagnostic survey, students’ average self-assessment of their own previous knowledge of the course topic was 7.6/10. This initial self-diagnosis is consistent with the control group's mean examination result (7.1/10). Students who completed the course performed better than the control group in the exam (9.5 versus 7.1).
The results of this study provide evidence of content-based learning designs as engaging, effective alternatives for online courses in corporate settings. Participants benefitted from self-pacing, that is, the flexibility to study whenever it suited them, without depending on others’ input to move forward. They engaged with their course following the structure, guidance and recommendations provided. Most students successfully completed the course and performed better than the control group.
Activities requiring explicit responses and automated feedback were useful as a means of ensuring comprehension and encouraging a return to earlier parts of the content when confusions arose. Learner–content interactions performed functions usually carried out by teachers (Anderson 2003) and provided a useful scaffold for students to meet the learning outcomes (Zimmerman 2012).
Some participants attempted to contextualise the materials, making it more relevant to them and their work. However, as in Cotton and Gresty's study (2007), other students skimmed through the resources. When participants had questions, there was seemingly no one to help (Padilla Rodriguez and Fernandez Cardenas 2012; Smedley 2011). The general support discussion forum was a dead space, rarely viewed and never used.
Students were resourceful when attempting to obtain extra help. They moved beyond what the course offered. They took notes they could look back to, sought communication from peers via Moodle's private messages, or turned to work colleagues available face-to-face to discuss ideas. This finding is consistent with the notion that learner–content interactions are limited in comparison to the more meaningful learning experiences that exchanges between people may create (Anderson and Garrison 1998; Chang and Smith 2008; Salmon 2011; Su et al. 2005). It also highlights the importance of informal learning activities, which were not planned, suggesting greater freedom for learners to choose sources of support and evidence the significance of interactions with people (Eraut 2004).
The value of informal learning activities has been highlighted in the past (Ozolins, Hall, and Peterson 2008; Zhang, Peterson and Ozolins 2011). This study adds to this research by providing further evidence of the importance of interactions beyond the embedded activities of an online course. While these interactions may be “invisible” to learning designers and teachers, they may have a significant influence on the learning outcomes and on knowledge transfer. Understanding these interactions should inform design and delivery decisions. Additional research is needed to evaluate the specific relationship between engagement in informal learning via different types of interactions and the achievement of learning outcomes.
The conclusions from this research can be mapped against three areas: student engagement, learner support and effectiveness of content-based learning designs.
Students engaged with a content-based online course offered by their organisation by following the guidance available and attempting to make the materials relevant to their own context. Structured learner–content interactions were designed into the course and provided standard opportunities for the acquisition of critical knowledge and skills. These processes did not depend on online facilitators or peers, and constituted a “safety net” for the achievement of the learning outcomes.
Students were resourceful in their search for support. If the materials did not provide answers to their questions, they looked for viable alternatives, such as reviewing their own notes and identifying colleagues to talk to, both online and face to face. These informal learning activities are valuable because of their potential impact on the achievement of learning outcomes and their application in the workplace.
Content-based courses can provide an engaging route to effective and efficient online learning in corporate settings: they help students achieve learning outcomes without the deployment of significant resources during delivery. However, excluding social interactions from online courses may result in course materials being the students’ only source of help. Some will find alternative ways of obtaining adequate support. Others might not. Broadening the range of support options available to students, that is, “humanising” support, may foster more meaningful, contextualised and rewarding learning experiences.
The findings of this study will inform learning and design and delivery decisions, and improve future versions of the course at the participating organisation. The short duration of the course and the relatively small sample prevent these results from being generalizable to all populations and settings. However, the findings presented in this article may be valuable to educators and trainers in similar contexts.
Abrami, P. C., et al. (2011) ‘Interaction in distance education and online learning: using evidence and theory to improve practice’, Journal of Computing in Higher Education, vol. 23, no. 2–3, pp. 82–103. Publisher Full Text
Anderson, T. (2003) ‘Modes of interaction in distance education: Recent developments and research questions’, in Handbook of Distance Education, eds M. Moore & G. Anderson, Erlbaum, Mahwah, NJ, pp. 129–144.
Anderson, T. (2004) ‘Toward a theory of online learning’, in Theory and Practice of Online Learning, eds T. Anderson & F. Elloumi, Athabasca University, Athabasca, Canada, pp. 33–60.
Anderson, T. & Garrison, D. R. (1998) ‘Learning in a networked world: new roles and responsibilities’, in Distance Learners in Higher Education, ed. C. Gibson, Atwood, Madison, WI, pp. 97–112.
Axelson, R. D. & Flick, A. (2010) ‘Defining student engagement’, Change: The Magazine of Higher Learning, vol. 43, no. 1, pp. 38–43. Publisher Full Text
Baruch, Y. & Holtom, B. C. (2008) ‘Survey response rate levels and trends in organizational research’, Human Relations, vol. 61, no. 8, pp. 1139–1160. Publisher Full Text
Beaudoin, M. F. (2002) ‘Learning or lurking? Tracking the ‘invisible’ online student’, The Internet and Higher Education, vol. 5, pp. 147–155. Publisher Full Text
Caladine, R. (2008) ‘Learning activities model’ in Encyclopedia of Information Technology Curriculum Integration. Volume II, ed. L. A. Tomei, Information Science Reference (IGI Global), London, UK, pp. 503–510.
Chang, S.-H. H. & Smith, R. A. (2008) ‘Effectiveness of personal interaction in a learner-centered paradigm distance education class based on student satisfaction’, Journal of Research on Technology in Education, vol. 40, no. 4, pp. 407–426.
Conole, G. (2013) Designing for Learning in an Open World, Springer, New York.
Cotton, D. R. E. & Gresty, K. A. (2007) ‘The rhetoric and reality of e-learning: using the think aloud method to evaluate an online resource’, Assessment & Evaluation in Higher Education, vol. 32, no. 5, pp. 583–600.
Eraut, M. (2004) ‘Informal learning in the workplace’, Studies in Continuing Education, vol. 26, no. 2, pp. 247–273. Publisher Full Text
Estrada, E., et al. (2011) ‘If the teacher works, does the student work too? Results of a 37 e-courses analysis’, in Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2011, eds T. Bastiaens & M. Ebner, AACE, Chesapeake, VA, pp. 3426–3430.
Garrison, D. R., Cleveland-Innes, M. & Fung, T. S. (2010) ‘Exploring causal relationships among teaching, cognitive and social presence: student perceptions of the community of inquiry framework’, The Internet and Higher Education, vol. 13, no. 1, pp. 31–36. Publisher Full Text
Kuh, G. D. (2009) ‘What student affairs professionals need to know about student engagement’, Journal of College Student Development, vol. 50, no. 6, pp. 683–706. Publisher Full Text
Mayer, R. E. (2002) ‘Rote versus meaningful learning’, Theory into Practice, vol. 41, no. 4, pp. 226–232. Publisher Full Text
Moore, M. G. (1989) ‘Three types of interaction’, The American Journal of Distance Education, vol. 3, no. 2, pp. 1–6. Publisher Full Text
Nie, M., et al. (2010) ‘The role of podcasting in effective curriculum renewal’, ALT-J, Research in Learning Technology, vol. 18, no. 2, pp. 105–118. Publisher Full Text
Noe, R. A., Tews, M. J. & McConnell Dachner, A. (2010) ‘Learner engagement: a new perspective for enhancing our understanding of learner motivation and workplace learning’, The Academy of Management Annals, vol. 4, no. 1, pp. 279–315. Publisher Full Text
Ozolins, I., Hall, H. & Peterson, R. (2008) ‘The student voice: recognising the hidden and informal curriculum in medicine’, Medical Teacher, vol. 30, pp. 606–611. PubMed Abstract | Publisher Full Text
Padilla Rodriguez, B. C. & Fernandez Cardenas, J. M. (2012) ‘Developing professional competence at a Mexican organization: legitimate peripheral participation and the role of technology’, Procedia – Social and Behavioral Sciences, vol. 69, no. 2012, pp. 8–13. Publisher Full Text
Salmon, G. (2011) E-Moderating: The Key to Online Teaching and Learning, Routledge, New York.
Smedley, A. (2011) ‘Review: issues with e-learning in nursing and health education in the UK: are new technologies being embraced in the teaching and learning environments?’, Journal of Research in Nursing, vol. 16, no. 1, pp. 91–92. Publisher Full Text
Su, B., et al. (2005) ‘The importance of interaction in web-based education: a program-level case study of online MBA courses’, Journal of Interactive Online Learning, vol. 4, no. 1, pp. 1–19.
Swan, K. (2002) ‘Building learning communities in online courses: the importance of interaction’, Education, Communication & Information, vol. 2, no. 1, pp. 23–49.
Welsh, E. T., et al. (2003) ‘E-learning: emerging uses, empirical results and future directions’, International Journal of Training and Development, vol. 7, no. 4, pp. 245–258. Publisher Full Text
Woo, Y. & Reeves, T. C. (2007) ‘Meaningful interaction in web-based learning: A social constructivist interpretation’, The Internet and Higher Education, vol. 10, no. 2007, pp. 15–25. Publisher Full Text
Young, K. A. (2005) ‘Direct from the source: the value of ‘think-aloud’ data in understanding learning’, Journal of Educational Enquiry, vol. 6, no. 1, pp. 19–33.
Zepke, N. & Leach, L. (2010) ‘Improving student engagement: ten proposals for action’, Active Learning in Higher Education, vol. 11, no. 3, pp. 167–177. Publisher Full Text
Zhang, J., Peterson, R. F. & Ozolins, I. Z. (2011) ‘Student approaches for learning in medicine: what does it tell us about the informal curriculum?’ BMC Medical Education, vol. 11, no. 87. Available at: http://www.biomedcentral.com/1472-6920/11/87 PubMed Central Full Text
Zimmerman, T. D. (2012) ‘Exploring learner to content interaction as a success factor in online courses’, The International Review of Research in Open and Distance Learning, vol. 13, no. 4, pp. 152–165.