Figure 1. Comparative usefulness of different learning activities.
ORIGINAL RESEARCH ARTICLE
Vicki H.M. Dalea* and Jeremy Singerb
aLearning Enhancement and Academic Development Service, Student and Academic Services, Southpark House, University of Glasgow, Glasgow, Scotland; bSchool of Computing Science, Sir Alwyn Williams Building, University of Glasgow, Glasgow, Scotland
(Received: 14 March 2019; Revised: 3 June 2019: Accepted: 3 June 2019; Published: 10 July 2019)
There is an increasing move in higher education to blend university courses to include a Massive Open Online Course (MOOC). This article reports on the learner experiences of such a course, which incorporated a purposely designed MOOC as part of the blend, to teach Haskell functional programming. A survey revealed that students most valued the programming exercises, quizzes and instructional videos, while the follow-up focus group highlighted the flexibility of the MOOC, usefulness of the videos, drop-in sessions and programming exercises. The overall mix of activities was regarded as particularly useful. While discussions were not rated as highly in the survey, students in the focus group commented on their value, particularly for getting support from external learners. The perceived lack of face-to-face contact was the biggest issue; however, this reflected a lack of awareness of lab sessions which could have been better signposted. There was perceived to be a gap between the MOOC and the rest of the course in terms of level of difficulty and authenticity of learning tasks. These issues were positively addressed in subsequent runs of the course. The outcomes of this study are relevant to educators seeking to incorporate MOOCs into blended courses.
Keywords: MOOC; learner experience research; blended learning; Haskell programming
*Corresponding author. Email: vicki.dale@glasgow.ac.uk
Research in Learning Technology 2019. © 2019 V.H.M. Dale and J. Singer. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2019, 27: 2248 - http://dx.doi.org/10.25304/rlt.v27.2248
The changing digital landscape is impacting on learning and teaching within higher education (Gardiner 2015). The sector is seeing increasing adoption of blended learning, in response to technological advancements, and the opportunity to support a more flexible learning experience that also enables students to take ownership of their learning (Gordon 2014). Blended learning may be defined as the optimal combination of face-to-face learning with online learning opportunities, with true blended learning affording the potential to reduce face-to-face contact time (Adekola, Dale, and Gardiner 2017), particularly when that time has traditionally been used for information transmission. Alongside the increasing adoption of blended learning more generally, Massive Open Online Courses (MOOCs) are seen to be having a disruptive effect on higher education (Conole 2013).
There are increasing instances where educators at tertiary level have formally adopted MOOCs as part of their blended provision. Koller (2012) first commented on this possibility, recognising developments at Antioch and Vanderbilt universities. Early examples of MOOCs being integrated into curricula use the ‘content licensing’ method of integration (Sandeen 2013); that is, adopting MOOCs produced from institutions outside the institution offering a hybrid or blended course for their students. Caulfield et al. (2013) call this the ‘distributed flip’ approach, in terms of a MOOC serving as the online preparation for face-to-face teaching, although other authors refer to ‘wrapping’ a campus-based course around a MOOC (Bruff et al. 2013). It is important not to confuse either of these approaches with blended MOOCs (bMOOCs), such as sending online learners physical ‘kit’ to use in their home environment (Atiq et al. 2016).
Several studies document learner experiences of blended learning involving a MOOC. Caulfield et al. (2013) examined the experiences of learners engaged on a Stanford MOOC as part of their distributed flip curriculum at the University of Puerto Rico. Students engaged with the video and interactive assessments, but not discussion forums, as they were able to converse face-to-face with their peers. A blended course that involved wrapping local teaching around a Stanford MOOC was delivered at Vanderbilt University by Bruff et al. (2013). Students appreciated the flexibility and accessibility of the videos, but preferred to interact with their local learning community, rather than other MOOC learners. They also expressed dissatisfaction with the ‘subject coupling’ (complementarity of content) of the Vanderbilt course material running alongside the Stanford-produced MOOC. Holotescu et al. (2014) described how students studying web programming at University Politehnica Timisoara, Romania, were asked to spend 10% of their study time engaging with a MOOC of their choice, enabling a flipped classroom approach. While students recognised the value of discussions and feedback in MOOC forums, they felt a lack of tutor engagement on the MOOC. In a study of postgraduate students in South Africa who voluntarily took a blended course including a MOOC to improve their study skills and graduate attributes, participants developed a sense of belonging with their peers as a result of the face-to-face contact, but the study did not examine their interaction with the wider MOOC community (Jaffer, Govender, and Brown 2017).
Some studies have investigated student performance in blended courses including a MOOC. Firmin et al. (2014) described the outcomes of a study comparing the performance of two groups in three mathematics and statistics courses; a student group from San Jose State University (SJSU) who needed the courses in order to progress, and a group of adult/community and student learners. The study concluded that student effort – in terms of engagement with videos and submission of problem sets – was the single biggest predictor of performance, which was shown to be higher in the student group. A study of pre- and post- performance in relation to an introductory physics MOOC at Massachusetts Institute of Technology (MIT) revealed that all MOOC cohorts demonstrated learning gain; however, there was no evidence that on-campus learners who took the MOOC as part of a blended course performed any better (Colvin et al. 2014). Griffiths et al. (2014) from Ithaca S+R and the University System of Maryland did several side-by-side comparisons of traditional courses versus blended ones that incorporated an external MOOC. Students on the blended courses performed as well as or slightly better than those on traditional courses, but were more dissatisfied in terms of how much they enjoyed the course, how much they thought they learned and how much the course had increased their interest in the subject. There were also issues of ‘content fit’ in relation to local curricula. The issue of content mismatch was also observed in a study of engineering students at SJSU, despite their teachers working with MIT to adapt their MOOC for SJSU students (Ghadiri et al. 2013). In that study, 91% of students passed the blended course, compared with 59% of the previous cohort who undertook the traditional course. SJSU MOOC learners valued the flexibility to watch lectures in their own time, and quizzes, though some learners requested fewer quizzes and more problem-solving demonstrations. In the Netherlands, Conijn, Van den Beemt, and Cuijpers (2018) showed that the frequency and order of activity access in a MOOC undertaken as part of a blended course were correlated with the final on-campus course exam grade. At the University of Zagreb, students studying mathematics as part of an engineering qualification were given the option of undertaking an external MOOC of their own choice instead of project work (Bralić and Divjak 2018). MOOC students outperformed project students, and learning diaries revealed that students valued the flexibility (the self-paced nature of the MOOC, the ability to pause videos) and more frequent opportunities for self-testing. Students also perceived linkage between the MOOC and their college course, though one student was unmotivated by the relative lack of face-to-face contact.
At the time this study was undertaken, there were few published studies of learner experience research on a blended course that incorporated a MOOC as core material, authored within the same institution, rather than as a supplement. A study of the integration of a MOOC produced by the University of Leeds with their first-year medical anatomy course (as an optional addition rather than core resource) had mixed results – while students appreciated videos and quizzes, they were less engaged with collaborative tasks such as discussion boards, and they actively rejected the idea of MOOCs replacing campus-based teaching (Swinnerton et al. 2016). More recently, Cornelius, Calder, and Mtika (2019) described the blending of an undergraduate course on sustainable development in Africa at the University of Aberdeen, incorporating a purposely designed MOOC to replace lectures, encourage critical thinking and facilitate learners to engage with a global community. A student engagement survey revealed that students who took the blended course scored higher on learning with others, and reflective skills, than non-MOOC cohorts. A focus group highlighted that the MOOC students appreciated a variety of media and benefited from discussion with the extended MOOC community. Another study described a proposed design and associated evaluation of a purposely designed MOOC as part of a blended provision for health care postgraduates and practitioners, but is still a work in progress (Meinert et al. 2018).
This article contributes to this growing field of study by investigating the learner experiences of students enrolled on an innovative blended course that included a purposely designed MOOC which was concurrently available to external learners. The study sought to understand the benefits and challenges associated with this new flexible mode of studying, and to assess the impact of integrating university learners with a wider learning community, and to identify support needs to help the institution move forward in its provision for blended learning incorporating MOOCs. The study was undertaken as part of a wider investigation into institutional, student and staff transitions to blended learning (Adekola, Dale, and Gardiner 2017; Adekola et al. 2017).
This study examines learner experiences of a new blended fourth-year honours undergraduate course ‘Haskell functional programming’ at the University of Glasgow (UofG), traditionally taught in the classroom. The university’s strategic investment in blended and online learning development (the BOLD project) provided an opportunity for academics to apply for funding to develop blended or online courses or programmes to enhance the student learning experience and quality of learning outcomes. The 10-week course was redesigned to include a 6-week MOOC, followed by 4 weeks of blended learning. The first 6 weeks constituted the FutureLearn ‘Functional programming in Haskell’ MOOC.1 The last 4 weeks were blended in the sense that materials were largely online in the institution’s virtual learning environment (VLE), Moodle, and optional face-to-face drop-in sessions were timetabled throughout the 10 weeks to allow participants to interact directly with their tutors and each other. The first run of the blended course, incorporating the MOOC, was offered in the 2016–17 session.
Cognisant of the need to provide personalised tuition and immediate feedback on students’ programming skills (Vihavainen, Luukkainen, and Kurhila 2012), the MOOC was designed to include an interactive programming interface (‘TryHaskell’). Using an adaptive learning approach, the MOOC included a number of programming exercises that became increasingly complex as students progressed through the MOOC (Singer and Archibald 2018). The MOOC also included readings, instructional videos, videos of interviews with experts, and quizzes.
We considered that interacting with a global community of learners was a required skill of programming graduates, who would – in practice – be interacting professionally on online networks to help source solutions to complex problems. In addition, we believed that because the MOOC would likely also include professional learners, they would be introduced to interesting examples of Haskell programming from industry.
Therefore, the research questions underpinning this study were:
a. What did they value most?
b. What did they find most challenging?
A sequential explanatory mixed methods approach was undertaken (Jones, Torres, and Arminio 2013), combining a survey with a follow-up focus group. All fourth-year students who were enrolled on the course were invited to take part in the survey, and a call among this group for volunteers was made to recruit focus group participants. The focus group was facilitated by author Dale with assistance from a Graduate Teaching Assistant tutor on the MOOC, who was able to clarify any subject-related details. Both facilitators were therefore independent of the course lecturers, to avoid biasing students’ responses. The focus group was audio-recorded and professionally transcribed by an external company.
The survey was hosted on the Bristol Online Survey platform. The survey included nominal items (demographics, prior experience and motivations), Likert-scale data (on a scale of 1 to 5 where 1 = most negative and 5 = most positive) and open text questions. Quantitative data were exported into Excel and SPSS for further data representation, summarisation and analysis.
Qualitative data from the survey and focus group were subject to thematic analysis (Braun and Clarke 2006), by author Dale. The open text survey responses and focus group transcript were first printed and read through, then annotated, to identify relevant codes. Codes were grouped into categories. Ideally, emerging ‘themes’ would then have been identified, although there were insufficient qualitative data to make the case for this. To establish reliability, the original anonymised data were shared with author Singer, and a face-to-face meeting was held to review and agree the coding structure. We have focused our discussion on what we feel are the most salient and strongly expressed points. To enhance the credibility aspect of trustworthiness (O’Brien et al. 2014), quotes from the survey and focus group have been included.
Ethical approval was granted by the College of Science and Engineering ethics committee (#300160028). Potential participants were contacted via email, and a plain language statement and consent form (the latter for the focus group only) were circulated in advance of participation. Participants were assured that their responses would be kept confidential, although the aggregated anonymised results would be shared with the course leads. Raw data were stored in accordance with the 1998 Data Protection Act, and participants were assured that they could withdraw from the study, and that their decision to participate or not would not influence their relationship with their teachers or the co-researcher. No incentive was made available for the survey; however, a catered lunch was used to incentivise focus group participants, who were also given a £10 Amazon voucher each as a reward for participating.
Thirty-six students participated in a survey (a response rate of 29%) and six students participated in the focus group. The question prompts for the focus group are listed in Appendix 1.
Thirty-four participants were undergraduate computing students and two were postgraduate students who had undertaken the course as an elective. The majority of learners (78%) had previously participated in a MOOC. All study participants engaged with the MOOC, with 87% completing it. Approximately half the students (49%) followed the 6-week recommended schedule, while nine (24%) studied at the own pace in advance of the schedule, eight (22%) studied at their own pace behind schedule and two (6%) answered ‘other’, being ahead for some parts of the MOOC and behind for others.
Approximately half of the survey participants had expected the course to include a MOOC (49%); the others were unaware that it would include a MOOC. Only one participant (3%) signed up for the course on the basis that it included a MOOC; another 11 were partly motivated by the inclusion of a MOOC (30%) but the majority’s (67%) decision to sign up for the course was not predicated on this fact. Once they knew the course would include a MOOC, the majority (59%) were either motivated or extremely motivated to undertake part of their course on a MOOC. Thirty-two per cent were somewhat motivated. Only two students (5%) were not very motivated, and only one was not motivated at all by the inclusion of the MOOC.
Figure 1 shows that the most valued types of activity in terms of their perceived usefulness (all median 4) were programming exercises (86% useful/extremely useful), quizzes (81%), instructional videos (73%), the programming interface (65%), the face-to-face drop in sessions (61%), discussions (56%) and expert interviews (51%). The least valued type of activity (median 3) was the readings (49% useful/extremely useful).
Figure 1. Comparative usefulness of different learning activities.
Figure 2 shows that learners felt most supported by their lecturers (70% well/extremely well supported, median 4). They felt less supported by other sources of help (all median 3): the TryHaskell programming interface (46% well/extremely well supported), graduate teaching assistants (45%), their University of Glasgow peers (43%) and other learners on the FutureLearn MOOC (42%).
Figure 2. Sources of support on the blended course.
In terms of the survey comments relating to what went well (n = 29), students appreciated most the flexibility afforded by the MOOC, allowing them to undertake learning at their own pace (n = 10):
“For people with no functional programming experience it is useful to have online training (you can re-read, google stuff), while if done in classroom might not be able to go at the pace of the lecture. This is particularly good at the beginning of a new study area, when you have no base and foundation for it already.”
The videos (n = 5), drop-in sessions/workshops (n = 5), programming exercises (n = 4) and the TryHaskell compiler (n = 3) were also valued:
“Videos were informative and well-paced.”
“Workshops to discuss MOOC content.”
“Exercises enhanced the work we were learning on.”
“The online exercises worked very well.”
In particular, it was the overall design of the MOOC, and the combination of activities, which strengthened the learning experience (n = 5):
“The quizzes, videos, and exercises formed a very strong combination. The reading provided greater detail on topics given in the lighter materials e.g. videos. The discussions that sat alongside the materials were extremely useful.”
“The course was great overall.”
Other miscellaneous survey comments on what worked well (n = 1 or 2) related to the quizzes, the social element of online discussions, the amount of support offered, the practical elements being better than a lecture for learning coding and the readings.
In the focus group, the main benefits to learning were identified as the flexibility associated with the MOOC:
“Yes, so basically it was nice that you could just go at your own pace. If you had a busy week or whatever you could do it beforehand or after. You don’t have to be like specifically in person there. You could just basically do it whenever you wanted to.”
Related benefits in the focus group included control over video speed, and the ability to revisit lectures. The quizzes were also considered useful but theoretical:
“The quizzes were pretty good but there could be more of them.”
“I mean the quizzes were all very technical as in you were going through the lecture and the quizzes were more like asking you some specific part of the lecture.”
From the survey comments (n = 26), the biggest frustration proved to be the lack of face-to-face contact with teachers and peers (n = 8), particularly the lack of labs (n = 5):
“Would have been useful to have a lab session, let’s say around week 5 or 6 with some exercises or challenges and active feedback.”
The lack of lectures affected a small minority of students (n = 2) and one student felt that it was easier to have concepts explained face-to-face:
“As mentioned before, non-mandatory face-to-face sessions made me feel less inclined to attend (and more likely to miss out on useful learning/understanding).”
“Explaining a difficult concept using a video or text is not as good as explaining it in class, where you can ask questions right away when it’s still fresh in mind and face-to-face explanations are clearer as well.”
Other miscellaneous survey comments on what did not work so well (n = 1 or 2) related to the expert interviews, reading that was too long or in-depth, the ‘big jump’ between weeks, lack of long form programming practice, lacking confidence in knowledge or a poor pairing of the MOOC and the Glasgow course (running sequentially rather than in tandem).
In the focus group, the biggest frustrations related to not feeling prepared for the assignment, due to the exercises being theoretical or disaggregated rather than allowing students to prepare for long-form programming:
“Well, it’s very difficult for the assessed exercise in the sense that…at least for me I was faced with new stuff there because otherwise you’d made them really simple examples, like we have some optional exercises and that’s all easy. When you go into assessed exercise you’re actually faced with I/O and everything. So basically you, kind of, need some sort of experience with that sort of stuff beforehand and I don’t think MOOC really prepared us in any way for that or not really properly at least.”
Despite being well received, the TryHaskell interactive programming tool was sometimes buggy and students felt restricted by the line-by-line coding approach, with students requesting longer working programme examples and more complex exercises, which would have helped them prepare for their higher level assignment:
“Definitely some more just bigger kind of things, for example, the exercises. The ones we had were like we were given a list of numbers, find me the biggest one or give me the sum or something which is obviously pretty basic. I would say give some more real life examples or something where you can actually have more I/O interactions and stuff like that or whatever you need further on … Or something, you know, I mean, if you have an assessed exercise at such a level you want to have some sort of progressive exercises to that level, don’t just throw me there.”
Not having labs was also raised in the focus group as an issue:
“No labs. I feel like there should be like all upper case, like NO LABS … there definitely should be labs.”
Student attitudes to studying within a massive cohort were diverse; 37% considered it not at all or not very useful, while 27% considered it somewhat useful, and 35% considered it useful or extremely useful.
In terms of what was gained studying as part of a massive cohort (n = 24 comments), the functionality in the MOOC for comments and discussion was identified as the biggest advantage. This included having more people to interact with and more support (n = 4) as well as having broader discussions (n = 4), interacting with people with the same problem (n = 3), receiving interesting comments on students’ posts (n = 3) and seeing how others approached tasks (n = 2):
“Most of my questions on MOOC were answered by those who are not Glasgow students.”
“There was more discussion on things that aren’t usually brought up in class. Also there was a wide range of backgrounds to pull from.”
“When I got stuck at some of the exercises I had a look through the comments, and it turned out a few other people had similar problems.”
In terms of what was lost studying as part of a massive cohort (out of 20 comments), eight students considered ‘nothing’ or ‘not much’, while others commented on the lack of face-to-face contact with teachers or peers in timetabled classes (n = 6), a reduced focus on Glasgow students (n = 2), reduced motivation (n = 2) or a lack of individualised teaching and feedback (n = 2):
“Personal interaction and a motivation to learn [was lost]. Experience of learning became less ‘real’.”
“Difficult to find answers that may be specific to assignments of Glasgow students.”
“Having the MOOC meant (to some extent) that I didn’t feel obliged to go to face-to-face sessions outside of the final lectures. Even though this clashes with one of my demonstration slots, I still feel having some impetus to go would have deepened my understanding more than the MOOC.”
Forty-two per cent of respondents felt either prepared or extremely well prepared for the rest of the course (the remaining 4 weeks of the semester, taught on campus), while 57% felt not very prepared or somewhat prepared. No participants considered themselves not at all prepared.
The majority of students’ expectation of the course were either partly met (44%) or fully met (47%) with two participants (6%) feeling that their expectations had been exceeded. Only one participant felt that their expectations had not been met at all (3%). When asked in the focus group why some participants’ expectations were not met, responses indicated that some students might have had unrealistic expectations, having never done a MOOC before, while others commented on the big ‘jump’ at week 4 into a more complex topic. There was also a sense that the MOOC component was very ‘hands off’. Over half (51%) of participants considered that more courses should be taught in this blended way incorporating a MOOC, while just under half (46%) were not sure and 3% (one student) stated that more courses should not be taught this way.
The major benefit to learners in this study was the flexibility offered by the blended format, in terms of the first part of the course being delivered online. The benefits of such a flexible approach have been advocated in relation to technology-enhanced learning generally (Gordon 2014) and in blended courses that incorporate a MOOC (Bralić and Divjak 2018; Bruff et al. 2013). However, this assumes that students have the study skills and self-directed learning readiness to take responsibility for their learning in this way (Malik 2008). Related to this, one disadvantage was the perceived lack of labs, which would have allowed students to interact face-to-face with their teachers and peers. Face-to-face interactions in blended and online courses are associated with a sense of belonging, essential to positive student learning outcomes (Tayebinik and Puteh 2012; Thomas, Herbert, and Teras 2014). While this perception was due to unclear communication and signposting about the availability of face-to-face interaction, it does highlight the importance of teacher presence including the need to provide clear signposting for students not used to managing their own learning in a blended curriculum (Greener 2008). A face-to-face induction is considered critically important by learners engaging with a blended curriculum that includes a MOOC (Cornelius, Calder, and Mtika 2019). Perhaps, this would suggest that the face-to-face component of a blended curriculum should take precedence over the MOOC in terms of signposting activities for learners (wrapping a course around a MOOC, rather than a distributed flip approach that requires the learners to independently engage with the MOOC up front).
The MOOC was designed to cater for a massive audience, and therefore used a combination of media and activities in line with FutureLearn’s (2018) recommendations. Students in this study particularly valued the mix of different types of learning activity. However, there was a paradox in terms of the perceived value of the discussions; these were among the least valued in terms of rating, and fellow MOOC learners were considered the source of least support overall. As identified by Israel (2015), this is consistent with other studies (Bruff et al. 2013; Caulfield et al. 2013), where students favoured videos and quizzes, and interacting with local students in preference to a massive cohort. Reluctance to engage in discussions was also observed in Conijn, Van den Beemt, and Cuijpers’s (2018) study. However, it is clear that a subset of our learners did particularly value the engagement with a massive cohort in order to find solutions to problems, consistent with Holotescu et al. (2014). This suggests that some learners are more comfortable engaging in discussion on MOOCs than others, which gives weight to the argument that there are different types of learners in MOOCs, reflecting their preference for different types of activity (Poellhuber, Roy, and Bouchoucha 2019).
One of the issues of incorporating a MOOC into a higher education course is the need to balance the needs of local learners with those of a massive cohort with varying experience of the subject. Most MOOCs are designed at ‘entry level’ into higher education. However, the second part of the course was developed independently for delivery on the VLE, specifically for honours level and masters students. This created a ‘gap’ between the two halves, leading to a disjointed experience in terms of the level of thinking, but also in terms of constructively aligning the entry-level programming exercises with the higher level end-of-course assessment. This was despite the fact that the programming exercises were considered the most useful aspect of the MOOC, perhaps unsurprising since this was most aligned with the expected outcomes of the course.
Limitations of this study include the fact that only 29% of learners responded; there may have been a response bias favouring those students with extreme views (Phillips, Reddy, and Durning 2016). In addition, this study sought only to explore learner experiences, rather than learning gains. Future research is needed to consider whether learners’ performance is enhanced by this blended course approach. The study was also of one cohort undertaking a single course within one institution. The findings are therefore not intended to be generalisable but offer insights for other institutions seeking to design and deliver a blended course that includes a MOOC.
Two further iterations of the course have taken place since this initial case study. The course has been adapted in three distinct ways, in response to student feedback.
Ideally, a longitudinal study would be carried out to assess the benefits of these changes; however, ethical approval was only sought and granted to survey UofG learners for the first run of the course, to assess learners’ first perceptions of what we believed at the time to be a brand new form of blended course design. However, ethical permission was subsequently granted to the authors to review the outcome of a separate evaluation of Haskell students’ use of ACJ (#400150118). Only four students completed the evaluation in 2018; however, their experiences were overwhelmingly positive in terms of the perceived helpfulness of seeing peers’ coding submissions (all students rated this 5/5), and comparing others’ submissions for their own learning (all students rated this 5/5). Following our initial ethics application (#300160028), permission was subsequently granted to examine routinely gathered end-of-course evaluation survey data in the subsequent two academic sessions. In relation to improvement 2, no students in the subsequent two cohorts commented negatively on the stability of the TryHaskell interface; however, a minority of students (n = 2 in each year) commented that key concepts were not well explained, which reflects the complexity of some topics in the course. In relation to improvement 3, only one student reported a perceived lack of sufficient lab time in the 2017–18 survey, and no students reported this as an issue in the 2018–19 survey. Ongoing evaluation data will be used to continually refine and enhance the course design.
This paper describes the outcomes of a learner experience research study on students undertaking a MOOC designed intentionally as part of a blended course. This type of blended learning is still relatively rare and understudied; therefore, the results are intended to guide other educators wishing to develop a blended course that includes a purposely designed MOOC. In particular, the study highlighted the importance of effective learning design, incorporating a range of different learning types, as well as the need for clear signposting about the face-to-face element of a blended course, associated with a strong teacher presence, to encourage learners to take responsibility for their learning.
We would like to thank QAA Scotland and the University of Glasgow for co-funding this study, and all study participants for their time and input. We also thank Cristian Urlea for co-facilitating the focus group, as well as Dr Wim Vanderbauwhede (co-course lead) and Niall Barr (systems developer), without whom this new blended course would not have been delivered. We would also like to thank Sarah Honeychurch for sharing the outcomes of her ACJ evaluation study.
Adekola, J., Dale, V. H. M. & Gardiner, K. (2017) ‘Development of an institutional framework to guide transitions into enhanced blended learning in higher education’, Research in Learning Technology, vol. 25, pp. 1–16. doi: 10.25304/rlt.v25.1973
Adekola, J., et al., (2017) ‘Student transitions to blended learning; an institutional case study’, Journal of Perspectives in Applied Academic Practice, vol. 5, no. 2, pp. 58–65. doi: 10.14297/jpaap.v5i2.273
Atiq, M. S. Z., et al., (2016) ‘Understanding student experiences in a blended-learning MOOC: a phenomenographic study’. Paper presented at the ASEE’s 123rd Annual Conference & Exposition, New Orleans, LA. doi: 10.18260/p.27102
Bralić, A. & Divjak, B. (2018) ‘Integrating MOOCs in traditionally taught courses: achieving learning outcomes with blended learning’, International Journal of Educational Technology in Higher Education, vol. 15, no. 1, p. 2. doi: 10.1186/s41239-017-0085-7
Braun, V. & Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative Research in Psychology, vol. 3, no. 2, pp. 77–101. doi: 10.1191/1478088706qp063oa
Bruff, D. O., et al., (2013) ‘Wrapping a MOOC: student perceptions of an experiment in blended learning’, Journal of Online Learning and Teaching, vol. 9, no. 2, p. 187.
Caulfield, M., Collier, A. & Halawa, S. (2013) ‘Rethinking online community in MOOCs used for blended learning’, [online] Available at: http://er.educause.edu/articles/2013/10/rethinking-online-community-in-moocs-used-for-blended-learning
Colvin, K. F., et al., (2014) ‘Learning in an introductory physics MOOC: all cohorts learn equally, including an on-campus class’, The International Review of Research in Open and Distributed Learning, vol. 15, no. 4. pp. 263–283. doi: 10.19173/irrodl.v15i4.1902
Conijn, R., Van den Beemt, A. & Cuijpers, P. (2018) ‘Predicting student performance in a blended MOOC’, Journal of Computer Assisted Learning. doi: 10.1111/jcal.12270
Conole, G. (2013) ‘MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs’, [online] Available at: http://eprints.rclis.org/19388/
Cornelius, S., Calder, C. & Mtika, P. (2019) ‘Understanding learner engagement on a blended course including a MOOC’, Research in Learning Technology, vol. 27. doi: 10.25304/rlt.v27.2097
Firmin, R., et al., (2014) ‘Case study: using MOOCs for conventional college coursework’, Distance Education, vol. 35, no. 2, pp. 178–201. doi: 10.1080/01587919.2014.917707
FutureLearn. (2018) ‘Pedagogy and learning design’, [online] Available at: https://partners.futurelearn.com/course-creation/pedagogy-and-learning-design/
Gardiner, K. (2015) ‘Reasons to be open – embracing the digital landscape’, Paper presented at the Association for Learning Technology Conference (ALT-C), University of Manchester, Manchester.
Ghadiri, K., et al., (2013) ‘The transformative potential of blended learning using MIT edX’s 6.002 x online MOOC content combined with student team-based learning in class’, Environment, vol. 8, p. 14.
Gordon, N. (2014) Flexible Pedagogies: Technology Enhanced Learning, [online] Available at: https://www.heacademy.ac.uk/flexible-pedagogies-technology-enhanced-learning
Greener, S. L. (2008) ‘Self-aware and self-directed: student conceptions of blended learning’, MERLOT Journal of Online Learning and Teaching, vol. 4, no. 2, pp. 243–253.
Griffiths, R., et al., (2014) ‘Interactive online learning on campus: testing MOOCs and other platforms in hybrid formats in the University System of Maryland’, [online] Available at: http://www.sr.ithaka.org/wp-content/mig/reports/S-R_Interactive_Online_Learning_Campus_20140710.pdf
Holotescu, C., et al., (2014) ‘Integrating MOOCs in blended courses’, Paper presented at the The International Scientific Conference eLearning and Software for Education.
Israel, M. J. (2015) ‘Effectiveness of integrating MOOCs in traditional classrooms for undergraduate students’, International Review of Research in Open and Distance Learning, vol. 16, no. 5, pp. 102–118. doi: 10.19173/irrodl.v16i5.2222
Jaffer, T., Govender, S. & Brown, C. (2017) ‘“The best part was the contact!”: understanding postgraduate students’ experiences of wrapped MOOCs’, Open Praxis, vol. 9, no 2, pp. 15. doi: 10.5944/openpraxis.9.2.565
Jones, S. R., Torres, V. & Arminio, J. (2006) ‘Choices and consequences of mixing methods in qualitative research’, in Negotiating the Complexities of Qualitative Research in Higher Education: Fundamental Elements and Issues, Routledge, London, pp. 135–151.
Koller, D. (2012) ‘How online courses can form a basis for on-campus teaching [blog post]’, [online] Available at: https://www.forbes.com/sites/coursera/2012/11/07/how-online-courses-can-form-a-basis-for-on-campus-teaching/#6bc5ac2f2887
Malik, S. (2008) ‘On campus, but out of class: an investigation into students’ experiences of learning technologies in their self-directed study AU – Deepwell, Frances’, ALT-J, vol. 16, no 1, pp. 5–14. doi: 10.1080/09687760701850166
Meinert, E., et al., (2018) ‘Real-world evidence for postgraduate students and professionals in healthcare: protocol for the design of a blended massive open online course’, BMJ Open, vol. 8, no. 9. doi: 10.1136/bmjopen-2018-025196
O’Brien, B. C., et al., (2014) ‘Standards for reporting qualitative research: a synthesis of recommendations’, Academic Medicine, vol. 89, no. 9, pp. 1245–1251. doi: 10.1097/ACM.0000000000000388
Phillips, A. W., Reddy, S. & Durning, S. J. (2016) ‘Improving response rates and evaluating nonresponse bias in surveys: AMEE Guide No. 102’, Medical Teacher, vol. 38, no. 3, pp. 217–228. doi: 10.3109/0142159X.2015.1105945
Poellhuber, B., Roy, N. & Bouchoucha, I. (2019) ‘Understanding participant’s behaviour in massively open online courses’, International Review of Research in Open and Distance Learning, vol. 20, no. 1. pp. 222–242.
Sandeen, C. (2013) ‘Integrating MOOCs into traditional higher education: the emerging “MOOC 3.0” era’, Change: The Magazine of Higher Learning, vol. 45, no. 6, pp. 34–39. doi: 10.1080/00091383.2013.842103
Singer, J. & Archibald, B. (2018) ‘Functional baby talk: analysis of code fragments from novice haskell programmers’, Electronic Proceedings in Theoretical Computer Science, vol. 270, pp. 37–51. doi: 10.4204/EPTCS.270.3
Singer, J., et al., (2019) ‘From a thousand learners to a thousand markers: Scaling peer feedback with Adaptive Comparative Judgement’, Paper presented at the 12th Annual University of Glasgow Learning and Teaching Conference, Glasgow, [online] Available at: http://eprints.gla.ac.uk/183564/1/183564.pdf
Swinnerton, B. J., et al., (2016) ‘The integration of an anatomy massive open online course (MOOC) into a medical anatomy curriculum’, Anatomical Sciences Education, vol. 10, no. 1, pp. 53–67. doi: 10.1002/ase.1625
Tayebinik, M. & Puteh, M. (2012) ‘Sense of community: how important is this quality in blended courses’, Paper presented at the The International Conference on Education and Management Innovation, Singapore, [online] Available from: https://arxiv.org/ftp/arxiv/papers/1504/1504.00249.pdf
Thomas, L., Herbert, J. & Teras, M. (2014) ‘A sense of belonging to enhance participation, success and retention in online programs’. The International Journal of the First Year in Higher Education, vol. 5, no. 2, pp. 69–80. doi: 10.5204/intjfyhe.v5i2.233
Vihavainen, A., Luukkainen, M. & Kurhila, J. (2012) ‘Multi-faceted support for MOOC in programming’, Paper presented at the Proceedings of the 13th Annual Conference on Information Technology Education.
1 https://www.futurelearn.com/courses/functional-programming-haskell