ORIGINAL RESEARCH ARTICLE
Have a question? Just ask it: using an anonymous mobile discussion platform for student engagement and peer interaction to support large group teaching
Elaine Tan*, Adrian Small and Paul Lewis
Newcastle University Business School, Newcastle University, Newcastle upon Tyne, United Kingdom
(Received: 13 August 2019; Revised: 31 October 2019; Accepted: 24 November 2019; Published: 22 January 2020)
Abstract
This article analyses the pilot of an anonymous question and answer mobile application with a large cohort of undergraduate students (460) enrolled on an Operations Strategy Management module. The mobile application allowed students to pose anonymous questions to both peers and staff, create replies and vote on questions posted by other users. The aim of the pilot was to evaluate how this application could be used to enhance communication, engagement and student learning both inside and outside of class time to overcome some of the challenges presented by large cohort teaching. An initial evaluation was undertaken exploring both the analytics attached to the platform and a thematic analysis of the posts. The initial findings of the pilot were positive, with the majority of students installing and regularly accessing the application with use increasing over time. The questions posed demonstrated engagement beyond surface-level memorisation of module content, and there were indications that the application could be beneficial in supporting student community awareness and interaction within large cohorts.
Keywords: mobile learning; technology-enhanced engagement; peer support; anonymous engagement; large cohort teaching
*Corresponding author. Email: Elaine.tan@ncl.ac.uk
Research in Learning Technology 2020. © 2020 E. Tan et al. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2020, 28: 2323 - http://dx.doi.org/10.25304/rlt.v28.2323
Introduction
Teaching a large number of students in a lecture format is argued to require the same skill set and commitment as small group teaching – in terms of being able to motivate students, organising the lecture content effectively and stimulating the learning environment (Exeter et al. 2010; French and Kennedy 2017). One approach for allowing students to engage further in lectures is for lecturers to pose questions that students answer, or for students to ask their own questions. Student engagement during a large lecture, however, has been argued to be primarily passive (Prince 2004), leading authors such as Kenney (2012) to describe the act of giving a lecture like talking into a void and lacking interactivity. This lack of engagement can be bound by rigorous social norms governing interaction (Ellison et al. 2016). For students interacting in a large group setting can be a nerve-wracking experience due to low self-efficacy (Bandura 1997), presenting the uninviting prospect of publicly exposing their lack of understanding to peers. If students continue to disengage over a period of time, it can result in a downward spiral in the quality of lectures given (French and Kennedy 2017).
If students are unwilling to ask questions during lectures, they may seek answers to questions through individual emails or by arranging a personal meeting with a lecturer. Where this occurs, it can result in an untenable burden for teaching staff – receiving multiple emails on the same topic, all requiring similar, but individual, responses. Students, therefore, may shy away from regularly sending emails for fear of being seen as being ‘difficult’ or ‘needy’, or receiving a negative reaction (van der Meij 1988). For many students, the perceived negatives such as self-consciousness, fear of negative evaluation, and lack of psychological safety (Roberts and Rajah-Kanagasabai 2013; van der Meij 1988) presented by using these channels dissuade students from seeking help. They instead struggle on alone and in the dark, or become reliant on the similar levels of understanding and sadly misunderstanding of their peers (Exeter et al. 2010).
Large cohort teaching through lectures
The use of lectures as the primary mode of delivery is now a standard format of large cohort teaching (French and Kennedy 2017). Seen as efficient, in terms of time, space and finance, students now regularly experience large cohorts of several hundred, where most interaction occurs in large groups (Tormey and Henchy 2008); lectures are set to be a fixed feature of university teaching (Saunders and Hutt 2015).
In such large group environments, a student is rarely seen as an individual and is disempowered from asking questions, engaging with peers or staff members (McKenzie et al. 2013), or in activities likely to promote active learning or deep engagement, that resolve misunderstandings (Exeter et al. 2010). To address these issues, several researchers have explored the role of technology in promoting interactivity, one area being that of interactive voting or response systems (d’Inverno 2003; Draper and Brown 2004; Graham et al. 2007).
Early incarnations of this technology restricted student engagement to responding to multiple-choice questions, using what were commonly referred to in the literature as ‘clickers’ (see Patterson, Kilpatrick, and Woebkenberg 2010; Stowell 2015). This later evolved to exploit the ubiquitous availability of mobile phones (Hatun Ataş and Delialioğlu 2017; Stowell 2015) and further to include web-based platforms (Chou 2003; Lahlafi and Rushton 2016) with dedicated applications that accommodated short answer text.
The findings of these studies, although overlooking, to some extent, the ‘wear out’ effect (Wang 2015), indicate that a significant impact can be obtained in terms of increasing student interactivity (d’Inverno 2003) and promoting to deeper learning when successfully combined with peer instruction (Mazur 1997). However, the primary function of these platforms in the studies conducted has been to gather information in the form of votes, a time-limited and teacher-driven activity. Whilst being a useful lecture tool, the activity and interaction created is one that is confined to the time and space of the lecture (Tormey and Henchy 2008). Nevertheless, the use of mobile devices as an accepted platform for student engagement (whilst debated initially) is now well established and widely accepted (Aagard, Bowen, and Olesova 2010; Ciampa 2014; Gikas and Grant 2013).
A second set of studies can be identified where the interaction occurs away from the lecture theatre (Gikas and Grant 2013; Martin and Ertzberger 2013). These studies remove time and space constraints, but the comments or questions made by each student are attributed and visible to all those enrolled on the module/course, which can enact similar fears and awkwardness that stops students engaging in large group settings. Other research has explored the potential of existing social media microblogging platforms such as Twitter (Camiel et al. 2014; Elavsky, Mislan, and Elavsky 2011; Junco, Heibergert, and Loken 2011; Lowe and Laffey 2011). These investigations, in particular those surrounding Twitter, have resulted in increased engagement and greater interpersonal interaction (Junco, Heibergert, and Loken 2011), but questions have been raised, for example, by Mercier, Rattray and Lavery (2015) regarding the depth of this engagement. Camiel et al. (2014) reported that whilst the Twitter platform did promote engagement, it discouraged effective note taking. A further drawback of the use of public platforms such as Twitter was student reluctance to post publicly citing privacy concerns (Lin, Hoffman, and Borengasser 2013), exposing their lack of understanding, or unnecessarily intruding on their personal lives (Tiernan 2014). This consideration was also raised by Lowe and Laffey (2011) who commented that the public nature of the channel could be followed by anyone.
Anonymity
For students lacking in self-efficacy, the prospect of posting publicly can be challenging and thus they may become ‘overwhelmed and fearful’ (Krüger-Ross et al. 2013, p. 130) at the prospect. The role of anonymity to support student questioning is an area of current exploration because technology provides a mechanism by which students can interact (Patterson, Kilpatrick, and Woebkenberg 2010) without exposing their identity. Results from previous explorations, whilst acknowledging the difficulties presented by anonymity, have also cautiously highlighted the positive applications of such platforms. Bergstrom, Harris and Karahalios (2011, p. 627) suggested that the use of anonymous applications can be useful due to the ‘few speaking opportunities’ in lectures. These applications remove ‘many social pressures’ as well as being ‘ostracized by their peers’ (Bergstrom, Harris, and Karahalios 2011, p. 627) in order to allow participation and encouraging freedom of expression, allowing students to partake in wider discussions without the requirement of friends or followers (Li and Literat 2017). Kang, Dabbish and Sutton (2016, p. 368) when discussing the motivations for using anonymous platforms concluded that their appeal lies in the ability to ‘gain social validation and social support from the community’. In an earlier work, Kang, Brown and Kiesler (2013, p. 2660) had also noted that anonymity was used by Internet users when accessing help as a way of ‘preserve[ing] their public or self-image’.
Roberts and Rajah-Kanagasabi (2013) explored the use of anonymous discussion boards and reported in their study that student willingness to be named in their posts and questions can be determined by a number of factors, including self-consciousness, fear of negative evaluation, lack of trust, perceived psychological safety and self-efficacy. When removing the need for named contributions in their study, they found the only remaining factor was self-efficacy. As such they suggested that the role of anonymous contributions was effective in promoting student engagement and providing a safe space for interaction. The development of anonymous social media applications in recent years has increased in popularity (Black, Mezzina, and Thompson 2016) with platforms such as ‘Yik Yak’ and ‘Whisper’ (both now defunct), and ‘Secret’ being developed perhaps indicating that the anonymity element is of particular importance to the student body (Heaslip, Donovan, and Cullen 2014).
The questions this study addresses are the following:
- To what extent would a voluntary system of anonymous questions be adopted by students outside of class time?
- What kind of questions were asked by students using this platform?
- How successful would such a platform be in promoting engagement and interaction within a large cohort setting?
Methodology and empirical setting
A case study approach was adopted for this study, which allowed a focus on the dynamics that were involved in using the mobile discussion platform (Eisenhardt 1989). The data drew on both qualitative (what was posted on the platform) and quantitative data (usage, post views and interaction time) (Yin 1981).
The anonymous mobile discussion platform itself resembles the concept of ‘Yik Yak’, a controversial microblogging technology accessed primarily on smart phones, that has raised concerns due to the potential for cyberbullying and the posting of other offensive material (Black, Mezzina, and Thompson 2016; Lee et al. 2017) – but one that proved to be hugely successful in generating student conversations and engagement and feedback (Robison and Connell 2017). This concept was retooled to create a platform with a positive educational focus. It is important to note that in the context of this investigation, the tool was developed by members of the student population and was personal, course based and importantly publicly anonymous. Students could only enrol and use the application if they provided their university email as their log in credentials. This meant students will only access content to modules relevant to their course of study. Figures 1 and 2 provide screenshots of the application and demonstrate the features available.
Figure 1. Interface of application.
Figure 2. Interface with lecturer comment highlight in red.
An element of community self-management was available in the application with ‘votes’ applied to questions to ensure the most relevant content was displayed. Question(s) with higher number of votes ‘floated’ to the top, whilst others receiving fewer votes dropped further down the list before disappearing entirely. As a measure to maintain a safe and inoffensive environment, where questions are deemed unsuitable, the population could ‘downvote’ posts. When a question received five downvotes, it was deleted. This system was similar to that described by Black, Mezzina and Thompson (2016) as akin to a ‘Neighbourhood watch’ within the Yik Yak application, who also noted that the postings of most users on Yik Yak were mostly ‘benign’ (from an analysis of 4000 posts). Others have confirmed the self-moderating nature of anonymous platforms with Savenski, Chou and Roy (2016) reporting a degree of self-censorship existing even in anonymised spaces. Questions could also be flagged with a report, and when there are five reports, they could be removed by the moderator. Any registered user was able to post a reply to a question including staff, although these replies were displayed as a different colour as shown in Figure 2.
Analytical approach
The anonymous question application was implemented for a period of 37 days on an Operations Strategy Management Module with a cohort of 460 students. During this period, the application was monitored by the two teaching staff to respond to submitted questions. Questions were then gathered and subjected to thematic analysis from which a series of categories were inductively created (Namey et al. 2008) as they emerged from the data. Engagement data made available by the developers were also examined.
Findings
Uptake
The technology gained traction with the students with a large number installing the platform on their device (n=294) approximately 63% of the cohort. Of these, a further 292 went on to become active users of the platform. From the statistics analysed there were 293 unique users, with 38.1% being recurrent returners.
Figures 3 and 4 outline the adoption of the platform over time. Initially, there was a fairly low uptake; however, in the lead up to final assessment there was a rapid uptake in the levels of access. The number of responses during these periods was fairly consistent (see Figure 3).
The number of questions being accessed increased, suggesting that students were spending more time within the application and accessing more questions as time progressed (see Figure 4).
The interactions of students also appeared to take place both inside and outside of class time, with 112 posted outside of core hours and 304 posted during core hours (see Figure 5).
Figure 5. Hour of interaction (reply or posting of questions).
Questions
During the pilot period, 164 questions were posted by 53 unique student users. Three questions were removed by the downvoting mechanism, one was removed by the moderator and 58 questions were deleted by student users. All questions submitted to the application were available for analysis and two duplicated questions were removed. The content of these questions was then inductively coded and categorised, from which the 10 categories were created (see Table 1).
Clarification
These comments were the most commonly asked, with students seeking clarification of points of misunderstanding, either from lectures, revision sessions or in past papers, and in some cases were directly addressed to the lecturer ‘Hi Adrian and Paul, can you…’ using the platform to pose questions. These questions tended to ask for further support or clarification on materials covered in lectures and were longer and much more detailed than any other categorisation. Other questions falling into this group sought to draw upon the support of peers in order to aid revision and learning. The address used in these questions was different from those distinctly addressed to the members of staff frequently using the system ‘Does anyone…’ to initiate their question. These questions sought reassurance or input into specific topics or questions. Frequently, very specific questions were asked or peer explanations were sought to confusing issues, for example, ‘Can anyone explain the diagram for parallel sourcing?’. Similarly, students used the large cohort available to access collective knowledge ‘Has anyone come across how to calculate the earliest start/finish time?’. This category also contained questions asking for speculation regarding examination content, ‘Has anyone thought of any potential C questions?’, which would not be answered by members of staff.
Strategic
These were questions to support strategic approaches to assessment, asking for further details as to what specifically would be included in the examination, such as ‘Do we need to know every single thing from every single topic on this examination to do well?’ and for hints as to where they should focus their attention and on what provided resource ‘I would like to know which textbook in the reading list is the best for revision’.
Duplicate
These were identical questions that had been posted twice in quick succession by the same user. This happened on 22 occasions.
Functional
These questions were low-level functional questions, regarding arrangements, ‘Will there be a formulae sheet?’, ‘How long is the examination?’ and ‘Where can I find past examination papers?’. These questions formed the second largest grouping, with several repeated questions from different users asking the same query.
Community
These comments form a sense of community among students. They hold no real question regarding the learning or assessment of the course, and instead provide an insight into the social relationships between students enrolled into the course. These comments range from questions such as ‘How did everyone find the operations examination this morning?’ to form a sense of community awareness, to comments promoting community interaction such as ‘Where are we going for bevs after the examination?’. Many are congratulatory messages to the group celebrating their achievements ‘Congratulations on finishing your exams people!’.
Included in this category are requests for resource sharing, and demonstrate the evolving and collaborative practices of students ‘Have you guys heard of Quizlet? It’s a flash card app and website where you can share questions with each other! I'm keen on sharing mine, would anyone like to contribute?’. The platform was also used by a small group of students to initiate a study support group, arranging a meeting in the library ‘If anyone is up for it we could book a room at the ****** library for a couple hours and share notes and discuss the exam. Everyone’s welcome!’
Students used the platform to anonymously confess their anxiety, and to seek reassurance from peers. In these comments, students express their own anxiety about the examination process, seeking reassurance from others that they too were finding the topics challenging ‘Anyone else struggling?...’ or to simply state ‘When is the resit? I’m booking my train ticket now’.
Off topic
These questions were those that had no relevance to the course, student experience or examination. Whilst extremely limited in number, these posts were voted down or removed by the moderator as containing inappropriate content.
Complaint
These comments were made by students who used the platform to anonymously air grievances. These mainly concerned the assessment (in the context of implementation, the examination format had been changed from previous years, and as such, the past papers were of limited value). ‘The lecturers for this module are shocking, they don't reply to emails. We students pay 9k a year and a reply is the least you could do!’ or the perceived difficulty of the examination ‘What’s the point in making it harder, your [sic] gonna get more resit students now’.
Shortcutting
These questions were asked by students who attempted to use the platform as a convenient means of answering low-level questions, or accessing the work of other students. These questions could be answered easily if attempted such as ‘What chapter is minimum order quantities in?’, or ‘How many working days are there in a year?’ In these instances, students were utilising the perceived convenience of the platform applying strategies commonly seen applied to search engines.
Within the category of shortcutting, the platform acted as a way for students to confess to their own shortcomings and to seek help from their colleagues. These questions provided a way for a student to admit to not having attended the lectures or seminars and request that resources were shared with them on a more private channel ‘anyone have any notes they wouldn’t mind sticking in the Facebook group chat?’ and ‘I missed the seminar, what’s the layout for B and C, what’s the essay title?’. The platform here provided them with a way of requesting information regarding the content of these missed sessions, to the wider cohort. It should be noted, however, that these responses were not well received by peers and generated a number of downvotes and negative responses.
Examination technique
Questions students asked in this area surrounded how best to approach the task of revising for the examination. Questions asked how peers addressed the challenge of covering all content necessary for revision ‘What’s your revision technique for this module? Would you go through the PP [Powerpoint] and take notes or do you solve PST [Past] exam papers?’, or to staff requesting access to further resources for revisions ‘Where would one find some extra exercises for break-even analysis’.
Named comment
This only occurred in one incidence, but the platform was used by a student to provide their name and to ask for help. Here they stated that they ‘had a rough time of things lately’, made their request and then signed off with their name. It may suggest that there are incidences where students are happy to be named, but it is interesting that they have chosen a public channel, rather than a personal communication platform such as email to make this request. This may be indicative of the convenience of the channel, or that its role is perhaps seen by a limited number of students as a replacement for email.
Interaction
As with the platform’s features, all questions were subject to interaction with students casting both up and down votes on each question. Whilst many of the questions posed did not receive votes, there were categories that elicited a greater amount of interaction than others. The 164 questions elicited 255 responses from students. Of the questions posted there were approximately 367 votes cast, with 57 questions received no votes, 83 receiving positive voting and 12 negative responses.
Positive votes
These questions tended to be strategic questions surrounding the examination. The highest ranking question by student response was ‘Do we need to know every single thing from every single topic on this examination to do well? Any tips on how on earth we are meant to revise/remember it all!’. Community-building comments such as ‘If anyone is up for it we could book a room at the **** library for a couple hours and share notes and discuss the exam. Everyone’s welcome’ were also highly rated. However, also included in this positive category were questions and requests which students found valuable ‘Question 2 I don’t understand as to why you multiply B by 2 and why C is 5+50?’.
Negative votes
Questions that received negative votes were those that expressed sentiments such as personal criticism of the lecturer, or questions seen by students to be shortcutting ‘Missed the last seminar, what was the essay title?’. There were very few questions that received negative votes, which suggests that students would reserve downvotes only for a limited range of questions.
Replies
The number of students who engaged in posting responses was, however, limited. Whilst there were a large number of replies generated (255), 157 of these responses were generated by staff. The remaining 58 were generated by students, with two students accounting for 47 of these responses. As such, only 51 of the overall replies posted were by 25 by student users. Occasionally students would respond to questions posed with humour ‘Stu-DYING!’, attempts to answer the questions posed, or an empathetic statement to show that the poster was not alone. Where replies were made by members of staff, in answer to a student question it did initiate a conversation with the original poster with the exchange of several messages. We can speculate that students made use of other channels in addition to the developed platform to undertake discussion not visible to staff, for example, the Facebook page mentioned by student in their comments.
Self-deleted questions
Of the data set available there were 58 deletions made by students. The deleted questions by student fell across all categories, but with clarification (14) and duplicate (12) questions being the most frequently occurring. The reason for this may be that they posted the same or a similar question as another student, and were undertaking these actions as an administrative function. Complaints also formed the third largest category in self-deleting questions (7). Interestingly, the complaints within self-deleted category (7) were much higher than those in the published posts (3). Reasons for this may be that the students used the platform to vent frustration but still felt an element of reluctance to air these publicly, and five of these posts that were published were deleted within 1 min of posting.
Discussion
The uptake and use of this voluntary platform appears to have been relatively successful. Of the questions posted, these were opened 4611 times, suggesting that even though the questions may be limited in number they were made use of by students. During the pilot the uptake of the platform increased over time, in contrast to the findings of Wang (2015), who raised concerns over this issue when exploring the use of interactive class-based systems. During the early part of the trial, students who emailed staff to ask a question often received a reply to repost the question on the platform. As this became unnecessary after a while, the increasing use of the platform suggests that this increase was student-driven. The system appears relatively successful in increasing interaction outside of class time, as proposed by Cavanagh (2011) and therefore encouraging active time on task with several interactions occurring outside of core hours (see Figure 5).
The second research question asked ‘what kind of questions were asked by students using this platform?’ The type of questions varied across the cohort and gave some indication of how the system was used to encourage interaction with staff. The use of the platform as a device for active disclosure of anxiety or misunderstanding is an interesting finding and confirms the results of Roberts and Rajah-Kanagasabai (2013), who suggested that students will make use of this facility if provided, and limited only by their self-efficacy. Krüger-Ross et al.’s (2013) discussion regarding overcoming the fear of publicly asking questions in class also appears to have been addressed, as well as Rothstein and Santana’s (2011) consideration, that students who asked questions adopted a greater ownership of their learning.
From the analysis of the self-deleted questions, there are uncertainties as to how anonymous students believed this platform to be, as several posts were made then quickly deleted. This may be an indication of the strength of the strict social norms governing behaviour in large cohorts as suggested by Ellison et al. (2016), or as found by Savenski, Chou and Roy (2016), that a degree of self and community censorship exists even within these anonymous spaces. Alternatively, it could also imply that as suggested by Roberts and Rajah-Kanagasabai (2013) when exploring the use of anonymous discussion boards, that ‘lack of trust’ is still an issue.
Community-based comments and questions provided a limited insight into an active voice of the cohort, with events organised and resources shared. Comments regarding shared experiences received positive votes and demonstrated the beginnings of community awareness (Draper and Brown 2004) and indications of social validation (Kang, Dabbish and Sutton 2016). This implies as Baron et al. (2016) stated that opening up this ‘digital backchannel’ has enacted additional engagement. It is possible that students already used Facebook as an alternative method of peer communication as indicated by one student comment, and reserved their questions in the anonymous platform for lecture engagement. However, the data would suggest that even if this were the case, the platform itself still proved a useful tool in promoting a sense of community in which the lecturer and those who did not have a Facebook account could participate without intrusion into personal spaces (Tiernan 2014).
The questions posed by students with the application indicated that clarification was the main concern of students as Exeter et al. (2010) suggest. The questions asked in the category of clarification, tended to be detailed and in-depth, and demonstrated an active engagement with the course content as students synthesised questions to support their developing understanding (Miyake and Norman 1979). This may provide a counterpoint to Mercier, Rattray and Lavery (2015) or Camiel et al. (2014), who have previously queried the depth of engagement provided by backchannels. We suggest that this indicates the importance of provision of a separate, private platform for out-of-class use when students have had sufficient time to adequately grasp the concepts detailed before being able to create questions.
Limitation
There are a number of limitations attached to this initial pilot. Firstly, the pilot was for a brief duration of 37 days at the end of term. During this time students are engaged in assessment activities and the results here may well not be representative of use during normal term time. Secondly, the application was still under development and was subject to a number of bugs, and what students described as ‘annoying’ interface problems, which may have discouraged student adoption. Further work with the developers is required to refine the system. This may explain the occurrence of several deleted posts, which were re-posted almost immediately. Thirdly, the pilot took place across one single cohort of students in the second year, even though large in number, and does not provide a wide sample.
Implications for practice and further research
We conclude that the application could be used to support student in their out-of-class learning activities. The platform was accessed by students throughout the day and night, suggesting potential to become an important tool in their learning activities and stimulate engagement and discussion. The visibility of the questions posed and the provided answers has potential to support both staff and students in raising awareness of the common questions and in provision of more efficient mechanism for staff in answering these on a group rather than on individual basis. Secondly, the use of such an anonymous platform has the potential to support community development by encouraging those less inclined to contribute to class discussions to comment. Further research could explore the dynamics of this study by more closely examining the relationship between staff and students within this platform – for instance, whether staff were to assume a more limited role as expert and what could be done to encourage greater peer response. Our results revealed that there were a large number of lurkers using the platform, and again further research could uncover their reservations towards contribution, even within an anonymous setting.
Acknowledgements
The authors would like to thank the students who developed the platform and allowed them to gain a better insight into how students could use the technology to enhance their learning.
References
Aagard, H., Bowen, K. & Olesova, L. (2010) ‘Hotseat: opening the backchannel in large lectures’, Educause Quarterly, vol. 33, p. 2.
Bandura, A. (1997) Self-Efficacy, Worth Publishers, Freeman, New York.
Baron, D., et al., (2016) ‘Investigating the effects of a backchannel on university classroom interactions: a mixed-method case study’, Computers & Education, vol. 94, pp. 61–76. doi: 10.1016/j.compedu.2015.11.007
Bergstrom, T., Harris, A. & Karahalios, K. (2011) ‘Encouraging initiative in the classroom with anonymous feedback’, in Human-Computer Interaction – INTERACT 2011: 13th IFIP TC 13 International Conference, eds P. Campos, et al., Lisbon, Portugal, September 5–9, 2011, Proceedings, Part I. Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 627–642.
Black, E.W., Mezzina, K. & Thompson, L.A. (2016) ‘Anonymous social media? Understanding the content and context of Yik Yak’, Computers in Human Behavior, vol. 57, pp. 17–22. doi: 10.1016/j.chb.2015.11.043
Camiel, L.D., et al., (2014) ‘Twitter as an in-class backchannel tool in a large required pharmacy course’, American Journal of Pharmaceutical Education, vol. 78, p. 67. doi: 10.5688/ajpe78367
Cavanagh, M. (2011) ‘Students’ experiences of active engagement through cooperative learning activities in lectures’, Active Learning in Higher Education, vol. 12, pp. 23–33. doi: 10.1177/1469787410387724
Chou, C. (2003) ‘Interactivity and interactive functions in web-based learning systems: a technical framework for designers’, British Journal of Educational Technology, vol. 34, pp. 265–279.
Ciampa, K. (2014) ‘Learning in a mobile age: an investigation of student motivation’, Journal of Computer Assisted Learning, vol. 30, pp. 82–96. doi: 10.1111/jcal.12036
d’Inverno, R. (2003) ‘Using a personal response system for promoting student interaction’, Teaching Mathematics and its Applications, vol. 22, pp. 163–169. doi: 10.1093/teamat/22.4.163
Draper, S.W. & Brown, M.I. (2004) ‘Increasing interactivity in lectures using an electronic voting system’, Journal of Computer Assisted Learning, vol. 20, pp. 81–94.
Eisenhardt, K.M. (1989) ‘Building theories from case study research’, Academy of Management Review, vol. 14, pp. 532–550. doi: 10.5465/AMR.1989.4308385
Elavsky, C.M., Mislan, C. & Elavsky, S. (2011) ‘When talking less is more: exploring outcomes of Twitter usage in the large-lecture hall’, Learning, Media and Technology, vol. 36, pp. 215–233. doi: 10.1080/17439884.2010.549828
Ellison, N.B., et al., (2016) ‘The question exists, but you don’t exist with it’: strategic anonymity in the social lives of adolescents’, Social Media + Society, vol. 2, pp. 1–13. doi: 10.1177/2056305116670673
Exeter, D.J., et al., (2010) ‘Student engagement in very large classes: the teachers’ perspective’,. Studies in Higher Education, vol. 35, pp. 761–775. doi: 10.1080/03075070903545058
French, S. & Kennedy, G. (2017) ‘Reassessing the value of university lectures’, Teaching in Higher Education, vol. 22, pp. 639–654. doi: 10.1080/13562517.2016.1273213
Gikas, J. & Grant, M.M. (2013) ‘Mobile computing devices in higher education: student perspectives on learning with cellphones, smartphones & social media’, The Internet and Higher Education, vol. 19, pp. 18–26. doi: 10.1016/j.iheduc.2013.06.002
Graham, C.R., et al., (2007) ‘Empowering or compelling reluctant participators using audience response systems’, Active Learning in Higher Education, vol. 8, pp. 233–258. doi: 10.1177/1469787407081885
Hatun Ataş, A. & Delialioğlu, Ö. (2017) ‘A question–answer system for mobile devices in lecture-based instruction: a qualitative analysis of student engagement and learning’, Interactive Learning Environments, vol. 26, no. 1, pp. 75–90. doi: 10.1080/10494820.2017.1283331
Heaslip, G., Donovan, P. & Cullen, J.G. (2014) ‘Student response systems and learner engagement in large classes’, Active Learning in Higher Education, vol. 15, pp. 11–24.
Junco, R., Heibergert, G. & Loken, E. (2011) ‘The effect of Twitter on college student engagement and grades’, Journal of Computer Assisted Learning, vol. 27, pp. 119–132. doi: 10.1111/j.1365-2729.2010.00387.x
Kang, R., Brown, S. & Kiesler, S. (2013) ‘Why do people seek anonymity on the internet?: Informing policy and design’, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM Press, pp. 2657–2666. doi: 10.1145/2470654.2481368
Kang, R., Dabbish, L.A. & Sutton, K. (2016) ‘Strangers on Your Phone: Why People Use Anonymous Communication Applications’, in Proceedings of the 19th ACM conference on computer-supported cooperative work & social computing, ACM Press, pp. 359–370. doi: 10.1145/2818048.2820081
Kenney, J.L. (2012) ‘Getting results: small changes, big cohorts and technology’, Higher Education Research & Development, vol. 31, pp. 873–889. doi: 10.1080/07294360.2012.672402
Krüger-Ross, M., Waters, R. & Farwell, T. (2013) ‘Everyone’s all a-Twitter about Twitter: three operational perspectives on using Twitter in the classroom’, In K. Kyeong-Ju Seo (Ed.), Using Social Media Effectively in the Classroom: Blogs, Wikis, Twitter, and More, Routledge, New York, NY, pp. 117–130.
Lahlafi, A. & Rushton, D. (2016) ‘Mobile phones: not a distraction in the classroom but a means of engagement?’, in Innovative Business Education Design for 21st Century Learning, eds P. Daly, et al. Springer International Publishing, Cham, pp. 7–23. doi: 10.1007/978-3-319-32622-1_2
Lee, J.-S., et al., (2017) ‘What people do on Yik Yak: analyzing anonymous microblogging user behaviors’, in Social Computing and Social Media. Applications and Analytics, ed G. Meiselwitz, Springer International Publishing, Cham, pp. 416–428. doi: 10.1007/978-3-319-58562-8_32
Li, Q. & Literat, I. (2017) ‘Misuse or misdesign? Yik Yak on college campuses and the moral dimensions of technology design’,. First Monday, vol. 22. doi: 10.5210/fm.v22i7.6947
Lin, M.-F.G., Hoffman, E.S. & Borengasser, C. (2013) ‘Is social media too social for class? A case study of Twitter use’, Tech Trends, vol. 57, pp. 39–45. doi: 10.1007/s11528-013-0644-2
Lowe, B. & Laffey, D. (2011) ‘Is Twitter for the birds?: using Twitter to enhance student learning in a marketing course’, Journal of Marketing Education, vol. 33, pp. 183–192. doi: 10.1177/0273475311410851
Martin, F. & Ertzberger, J. (2013) ‘Here and now mobile learning: an experimental study on the use of mobile technology’, Computers & Education, vol. 68, pp. 76–85. doi: 10.1016/j.compedu.2013.04.021
Mazur, E. (1997) ‘Peer Instruction: Getting Students to Think in Class’, in AIP Conference Proceedings, vol. 399, pp. 981–988. doi: 10.1063/1.53199
McKenzie, W.A., et al., (2013) ‘A blended learning lecture delivery model for large and diverse undergraduate cohorts’, Computers & Education, vol. 64, pp. 116–126. doi: 10.1016/j.compedu.2013.01.009
Mercier, E., Rattray, J. & Lavery, J. (2015) ‘Twitter in the collaborative classroom: micro-blogging for in-class collaborative discussions’, International Journal of Social Media and Interactive Learning Environments, vol. 3, p. 83–99. doi: 10.1504/IJSMILE.2015.070764
Miyake, N. & Norman, D.A. (1979) ‘To ask a question, one must know enough to know what is not known’, Journal of Verbal Learning and Verbal Behavior, vol. 18, pp. 357–364.
Namey, E., Guest, G., Thairu, L. & Johnson, L. (2008) ‘Data reduction techniques for large data sets’, in Handbook for team-based qualitative research, eds G. Guest & K.M. MaCQueen, Altamira Press, Lanham, MD, pp. 137–162.
Patterson, B., Kilpatrick, J. & Woebkenberg, E. (2010) ‘Evidence for teaching practice: the impact of clickers in a large classroom environment’, Nurse Education Today, vol. 30, pp. 603–607. doi: 10.1016/j.nedt.2009.12.008
Prince, M. (2004) ‘Does active learning work? A review of the research’, Journal of engineering education, vol. 93, no. 3, pp. 223–231.
Roberts, L. & Rajah-Kanagasabai, C. (2013) ‘I’d be so much more comfortable posting anonymously’: identified versus anonymous participation in student discussion boards’, Australasian Journal of Educational Technology, vol. 29, pp. 612–625.
Robison, M. & Connell, R.S. (2017) ‘Harnessing Yik Yak for good: a study of students? Anonymous library feedback’, Journal of Web Librarianship, vol. 11, pp. 35–55. doi: 10.1080/19322909.2016.1236001
Rothstein, D. & Santana, L. (2011) ‘Teaching students to ask their own questions’, Harvard Education Letter, vol. 27, pp. 1–3.
Saunders, F.C. & Hutt, I. (2015) ‘Enhancing large-class teaching: a systematic comparison of rich-media materials’, Higher Education Research & Development, vol. 34, no. 6, pp. 1233–1250.
Savenski, M., Chou, S. & Roy, D. (2016) ‘Tracking the Yak: an empirical study of Yik Yak’,. Proceedings of the Tenth International AAAI Conference on Web and Social Media (ICWSM 2016), Presented at the International Conference on Web and Social Media, Cologne, Germany, pp. 671–674.
Stowell, J.R. (2015) ‘Use of clickers vs. mobile devices for classroom polling’, Computers & Education, vol. 82, 329–334. doi: 10.1016/j.compedu.2014.12.008
Tiernan, P. (2014) ‘A study of the use of Twitter by students for lecture engagement and discussion’, Education and Information Technologies, vol. 19, pp. 673–690. doi: 10.1007/s10639-012-9246-4
Tormey, R. & Henchy, D. (2008) ‘Re-imagining the traditional lecture: an action research approach to teaching student teachers to “do” philosophy’, Teaching in Higher Education, vol. 13, pp. 303–314. doi: 10.1080/13562510802045337
van der Meij, H. (1988) ‘Constraints on question asking in classrooms’, Journal of Educational Psychology, vol. 80, 401–405. doi: 10.1037/0022-0663.80.3.401
Wang, A.I. (2015) ‘The wear out effect of a game-based student response system’, Computers & Education, vol. 82, pp. 217–227.
Yin, R.K. (1981) ‘The case study crisis: some answers’, Administrative Science Quarterly, vol. 26, pp. 58–65. doi: 10.2307/2392599