Worth it? Findings from a study of how academics assess students’ Web 2.0 activities

Kathleen Gray^a,*, Jenny Waycott^a, Rosemary Clerehan^b, Margaret Hamilton^c, Joan Richardson^d, Judithe Sheard^e and Celia Thompson^f

^aSchool of Medicine and Department of Information Systems, The University of Melbourne, Melbourne, Victoria, Australia; ^bFaculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia; ^cSchool of Computer Science and IT, RMIT University, Melbourne, Victoria, Australia; ^dSchool of Business Information Technology, RMIT University, Melbourne, Victoria, Australia; ^eFaculty of Information Technology, Monash University, Melbourne, Victoria, Australia; ^fSchool of Languages and Linguistics, The University of Melbourne, Melbourne, Victoria, Australia

(Received 15 March 2011; final version received 28 July 2011; Published: 3 February 2012)

Abstract

Educational commentators have offered many pedagogical rationales for using Web 2.0 to support learning in higher education, and academics are being encouraged to find ways for their students to use social web technologies. Questions arise as to the value of these activities compared to more conventional assignments, and whether implementing such changes to student assessment is worth the effort. We conducted a survey of academics’ assessment of students’ Web 2.0 activities in Australian universities and found that this form of assessment is being conducted by a small number of academics, in a range of fields of study, but mainly in Humanities and Social Sciences, with varying kinds of intended and actual learning outcomes. Blogging and wiki-writing predominate, low and medium-stakes assessment are most common, and different methods of marking and feedback are in use. Qualitative feedback from the survey and follow-up interviews gave further insights into benefits and challenges of Web 2.0 assessment in relation to pedagogy, policy and practice. It appears that academics’ conservative approaches to conducting assessment and their novice approaches to utilising social web technologies are factors which seriously limit realising the potential of Web 2.0 for medium or high-stakes assessment.

Keywords: assessment; assignments; higher education; social web; Web 2.0

*Corresponding author. Email: kgray@unimelb.edu.au

Citation: Research in Learning Technology 2012, 20: 16153 - DOI: 10.3402/rlt.v20i0.16153

RLT 2012. © 2012 K. Gray et al. This is an Open Access article distributed under the terms of the Creative Commons "Attribution 3.0 Unported (CC BY 3.0)" license (http://creativecommons.org/licenses/by/3.0/) permitting use, reuse, distribution and transmission, and reproduction in any medium, provided the original work is properly cited.

Introduction

When introducing a new assignment to university students, academics routinely encounter the question, ‘What's it worth?’ as students calculate how much of their overall result for the subject is at stake. As they gauge the benefits and risks of adopting a new method to improve teaching or assessment, academics themselves may also wonder whether it will be worth the effort.

This paper reports findings from a survey and interviews with Australian academics undertaking summative assessment of students’ Web 2.0 activities. These data provide insights into this aspect of Web 2.0 pedagogy, including the investment and benefit at various stages of the assessment process, from planning through to evaluation.

Drawing on these findings, we discuss academics’ experiences and decision-making regarding the educational worth, assessable value and relative merits of assessing student learning using Web 2.0. Our research raises important considerations for individual academics, and also has implications for educational developers and technologists and for institutional leaders and policy-makers.

Background

Web 2.0, or social web, technologies allow co-creation of web content by large numbers of people simultaneously and over extended time spans. Content is open for participants to create, manipulate or propagate by commenting, editing, rating and tagging. Communication may be facilitated via avatars, fans, friends, locating, profiling and syndication. Thus a group may ‘create a collective work whose value far exceeds that provided by any of the individual participants’ (O'Reilly and Battelle 2009, p. 2). Web 2.0 can leverage content and communication not only on the open web but also in closed environments, for instance in the corporate sector (Schneckenberg 2009) and in university Learning Management Systems (e.g. Conde et al. 2011).

The idea that Web 2.0 offers pedagogical affordances for collaborative learning, mobile learning and other aspects of learning and teaching is now established (e.g. Kuswara 2011) and some of these affordances may have been taken up in practice (Conole and Alevizou 2010). Web 2.0 innovations in higher education have appeared in recent years (for example, Franklin and van Harmelen 2007, Lee and McLoughlin 2010), however, it is still not clear to what extent students’ Web 2.0 activities are formally assessed, or what affordances Web 2.0 offers specifically for assessment. Most published accounts of Web 2.0 pedagogies in higher education have described only formative assessment (i.e. providing feedback on work in progress so that a student can improve it before it is marked or graded) or low stakes assessment (i.e. earning marks that make very little impact on the student's overall standing), according to Gray et al. (2010). Most of this literature has overlooked the issue of summative, high stakes assessment, or has described it theoretically – for example, ‘working with a set of circumstances rather than trying to control or alter them’ (Hughes 2009, p. 30); and ‘capturing the visible evidence of invisible learning’ (Bass and Eynon 2009, p. 4).

Reasons can be inferred to explain the paucity of published examples of Web 2.0 assessment practices that actually demonstrate how summative assessment is being done and how it can be done well. Academics may be hesitant to reveal that they have deployed such novel technology for summative assessment. Using Web 2.0 to assess student work can pose challenges for institutional values and policies (Johnson, Levine and Smith 2009, p. 5), including for the institutional reputation that hinges on assessment, and for the academic integrity of assessment. Using Web 2.0 for summative, high stakes assessment could have an impact for good or ill on the integrity and reputation of the field of study or the university where it is used (Ehlers 2009). Over and above the pressures on traditional forms of assessment (Sadler 2009, Boud 2000), Web 2.0 assessment can raise concerns about major accreditation frameworks, employers’ and other external stakeholders’ expectations, endorsement of learning resources and activities, and questions of intellectual property (Collis and Moonen 2008, pp. 100–101). The visible and open nature of content and communication in Web 2.0 is at odds with an academic culture that has traditionally supported the making of decisions about assessment alone and in private (Hattie 2009). As yet, the academic policies of the authors’ own universities, and possibly elsewhere, are silent on Web 2.0's assessment implications, such as the identification, ownership, safety, recording, privacy and preservation of student work.

Using Web 2.0 for assessment has significant implications for academic integrity, a term holding complex and contested meanings for academic authorship, textual ownership and the practices of acknowledging influential ideas and their media of expression (Thompson 2008). It challenges extant referencing and citation protocols to embrace a style of academic writing that captures the dynamic, dialogic and multi-authored nature of texts and artefacts generated in Web 2.0 (Gray et al. 2008).

The need to improve the practice of assessment in higher education is widely recognised (e.g. Boud 2000). Assessment's impact on student learning is well established (e.g. Rust 2002), including technology-based assessment (e.g. Kirkwood and Price 2008). If done well, assessment can support and reinforce the intended student learning; that is, assessment can work ‘for’ or ‘as’ learning (Earl 2003). Web 2.0 may well be able to offer affordances for every stage of assessment, from designing and implementing it, through supporting and grading it, to evaluating its effectiveness (Waycott et al. 2010a). Web 2.0 might offer ways to make assessment more authentic, personalised, engaging or problem-oriented (Elliott 2008).

There is a growing body of case-by-case accounts evaluating various approaches to assessing Web 2.0 activities (e.g. Farmer, Yue, and Brooks 2008, Ng and Hussain 2009, Lai and Ng 2010). Whitelock (2010) analyses eight case studies, and Waycott and Sheard (2011) present six more, illustrating Web 2.0 uses in assessment ‘for’ learning, assessment of learning processes, and overcoming discontinuity between formative and summative assessment. However, to date, there has been no systematic investigation into the aspects that academics find worthwhile – or conversely, not worth it – about assessment of student Web 2.0 activities.

The prospect of opportunities for improving assessment but equally significant challenges, gave rise to a national Australian study in 2009. This study aimed to paint a detailed picture of where, why and how academics were using Web 2.0 for summative, as well as formative, assessment of student learning. It aimed to fill gaps in our knowledge and provide a foundation for developing a more widely shared understanding of the pedagogical worth and assessable value of students’ Web 2.0 activities in higher education.

Method

Academics were invited to participate in an anonymous online survey and an optional semi-structured interview during the period August to October 2009. Human research ethics approval for this project was granted by each of the partner universities. We advertised the survey in national learning and teaching newsletters, inviting participation from any academic who had set assignments where their students used Web 2.0 technologies to do work for which they earned subject marks. We also issued invitations to academics who had recently published papers about their use of Web 2.0 tools in higher education. Web 2.0 technologies were defined as any of the following: blogging and microblogging, audio and video podcasting, social bookmarking, social networking, wiki writing or virtual worlds; using any tools, such as Twitter, Flickr, WordPress, Second Life, YouTube, Delicious, Facebook, or tools bundled in Learning Management Systems (e.g. Blackboard's wiki tool).

Survey respondents answered questions about one assignment: one graded task that had been required in one subject or unit of study. A total of 44 questions covered the way the assignment was designed and implemented, the marking process, student results, policy and procedure considerations, and background information about the subject. Multiple responses were possible to many of the questions. The survey included three open-ended questions about the assignment. A copy of the survey instrument and detailed reporting of these data are available at http://web2assessmentresources.wikispaces.com/4.+Further + Resources.

Semi-structured telephone interviews conducted by one researcher explored interviewees’ assessment practices and their perspectives on the use of Web 2.0 for assessment An interview protocol (Appendix 1) was used, although the precise questions varied from case to case, allowing conversation to develop organically.

Sixty-four academics responded to the survey, of whom 53 completed all or most questions, and 22 were interviewed. Most survey respondents (81%) identified their role as ‘subject coordinator’, for example, having overall responsibility for learning, teaching and assessment in the unit of study. Many respondents (37%) had first used the Web 2.0 assignment they described in 2009; 48% had begun the assignment between 2006 and 2008.

All open-ended survey responses and interviews were transcribed and analysed first separately and then together by four of the researchers, using open coding. Care was taken to account for both opportunities and challenges, i.e. positive and negative reports of experiences. Interview and survey findings were then categorised by two of the researchers using the organising principle of the different stages in the assessment cycle.

We could have categorised these data in many ways, following the literature; however, many of these categories would have failed to provide insights into summative assessment practices as well as into the assessment process as a whole. Franklin and van Harmelen (2007) have proposed recommendations for writing assessments involving Web 2.0; the majority of these are formative assessment, and Hatzipanagos and Warburton (2009) too make a case for exploring the links between formative assessment and social software. Bartlett-Bragg (2008) has proposed a framework for designing Web 2.0 assessment which highlights how much effort is needed in the initial planning stages, while Bower (2008), Bower, Hedberg, and Kuswara (2010), and Conole et al. (2004) have published methods for matching learning tasks and assignments with learning technologies which also emphasise the degree of preparatory effort required. To describe the experience of working with Web 2.0 assessment more fully than this, we chose to analyse the data using the organising principle of the different stages in the assessment cycle: not only designing the assignment, but also implementing and supporting the assignment with students, marking the assignment and giving summative feedback to students, and quality management of the assessment processes (Rust, O'Donovan, and Price 2005, Bloxham and Boyd 2007).

Findings

Descriptive statistics and comments from interviews and open-ended survey responses illustrate key benefits and challenges that academics weighed up at each stage of the assessment cycle.

Designing the assignment

Findings about the design of Web 2.0 assignments give a mixed picture of the extent to which academics are attributing value to such approaches. The 62 responses to a question about the type of Web 2.0 tool used showed that wiki writing (52%) and blogging (50%) were most commonly used. Uses of social networking (27%), audio or video podcasting (26%), virtual world activities (19%) and social bookmarking (18%) for student assignments were reported less often.

Web 2.0 assignments were designed for a range of settings, including all year levels in 35 undergraduate degrees and 16 postgraduate coursework degrees, and in both small and large classes. While 23% of respondents were working with classes of 30 students or fewer, 25% had classes of between 31 and 100 students, and 25% had more than 100 students (the largest class reported had over 500 students).

Differences were apparent in the value of these assignments for different fields of study. In almost half the 50 responses to the ‘field of study’ question, the assignment was part of a course in Humanities or Social Sciences (42%). Almost a third of assignments came from each of Information Technology (32%) and Education (30%). Mentioned less often were Health and Medicine (16%), Management and Commerce (12%), Creative Arts, and Natural and Physical Sciences (6% each), Architecture, Engineering, and Law (2% each).

Web 2.0 assignments appeared equally useful for assessing various types of learning outcomes: generic or graduate skills or attributes (55%), specialised knowledge or skills required in a discipline or profession (44%) and foundation knowledge or skills preparatory to a discipline or profession (45%). These assignments were valued not only for demonstrating outcomes but also for facilitating learning processes, for instance:

Interviewees cited a number of challenges in getting the assessment design right. Some of these were common to any assignment, for instance the challenge of making room for student creativity while also providing sufficient structure:

The bottom third of the class had difficulty thinking about what to post on [i.e. blog about] when it was left completely up to them … If you give too much guidance are you constraining the best students? If you don't give enough guidance you're leaving the weaker students to flounder. (Interviewee 11)

Other challenges arose from needing to learn by trial and error how a Web 2.0 assignment would work:

We didn't understand how the interaction would unfold, how the tool would behave, a lot of downtime and template restrictions. (Interviewee 18)

Implementing the assignment

Findings about how academics are introducing and supporting Web 2.0 assignments with their students suggest that it is not a simple matter to do this well. Compared to more conventional assignments, the extra efforts required to get students to work effectively with Web 2.0 may detract from the perceived worth of the activity.

Students commenced work on most assignments either at the start of the study period (55%) or in the middle of the study period (34%). The time they were expected to spend on the assignment was most often 11–20 hours (33%), followed by 1–10 hours (27%), 21–30 hours (17%) and 31–40 hours (11%). Students were allowed more than a month to complete most assignments (66%) or at least several weeks in a number of other cases (22%).

Students were expected to do the assignment using their own IT facilities (88%) or IT facilities provided by their university (78%). They were expected to work on it in various locations including:

Academics interviewed spoke at length about their need to develop new strategies to help students interact online: ‘I would give them an example at the beginning in the first week or second week on how this would work and the sorts of comments that I would be expecting’. (Interviewee 15)

Several interviewees described the work involved in encouraging students to draw on their personal experience to generate an online identity: ‘There's a process that goes into them finding their different voices, how to share appropriately, how to write with authority’. (Interviewee 8)

Academics reported coaching students in co-creation and collaboration: ‘I said it had to be [500 words] that you had to add and then you had to value-add to the content. So we talked about what we meant by that’. (Interviewee 22)

Providing adequate support for students posed new workload challenges for some staff:

[There is a lot of] work involved in setting up [the blog assignment] and making sure all the students know how to do it. If you ask them to write an essay they just go off and write it, you don't have to spend the first three weeks of the course teaching them about essays. (Interviewee 11)

It's hard yakka [work] for the teachers … because we often have to be on [online]. almost all hours at times … You tend to do that to give the support. (Interviewee 1)

Marking and giving feedback on the assignment

Findings about marking and giving feedback on Web 2.0 assignments indicate that at this stage of the assessment cycle most academics are not taking up the open and collaborative approaches to feedback and assessment that the social web affords, but are exercising their individual authority and discretion as assessors.

The value of Web 2.0 assignments as a proportion of a student's overall mark in the unit of study ranged from very little to 100%, with most assignments worth up to 50%, as shown in Table 1. Most students received marks plus other feedback, including confirmatory comments (67%), explanatory comments (64%), and corrective comments (61%).

The majority of Web 2.0 assignments were marked by one staff member (63%), with just over a quarter marked by more than one (27%). Academics gave various responses to a free-text question asking them to identify strategies they used to brief other markers, including students who had to mark each other's work, as shown in Table 2. Peer-assessment or self-assessment was factored into marking in only 14% and 11% of cases respectively.

Establishing a framework for evaluating the work that students produced was not straightforward. One interviewee found it difficult to set criteria that were appropriate to different levels of study, noting that the differences were ‘sort of formally in place with essays, you know, the number of words you write as a Masters student is different to a Grad Dip but when it comes to creating [Web 2.0 content], perhaps it's a bit less clear’ (Interviewee 7). In some assignments the diversity of content types that students could produce created another challenge for marking: ‘[Students] are producing very different things, so I suppose the criteria need to capture that in some way’ (Survey response). Another survey respondent noted that the different style of writing in a Web 2.0 assignment made marking difficult: ‘because the content does not have to be of an academic standard it can be hard (to) assess, e.g. marking sloppy or lazy prose’.

In some instances calculating a final mark for individuals’ work was complicated by the need to assess how it had been influenced by class members’ running online exchange of feedback and interaction. Twenty-three percent of respondents said that equal marks were allocated to everyone in a group project.

Interviewees reported that they had developed new marking techniques to respond to Web 2.0 affordances in a variety of ways:

We have a complete history of everything they do … So we can actually look in and see what this particular group has excelled at, which one gives us a wide variety of results so that we can separate the good students from the others. (Interviewee 13)

The assessor is not assessing a written document, they're assessing a page which … is a whole labyrinth of choices and connections, so they've got to actually work their way through … decision paths and decision tree and things like that. (Interviewee 10)

In particular, there were many instances where staff found merit in being able to combine formative and summative assessment, for example:

I leave comments on the blogs as we go through but usually engaging with the content, not sort of marking it, but then I can add those sorts of comments against the rubric when I assess them. (Interviewee 11)

Sometimes assessing process as well as product was problematic, for instance when marking co-created wiki content:

How do you mark assignments when students can change/overwrite each other's work! Many students who contributed early, found that their work was completely lost. How do you manage this process of overwriting and still contributing to the same content? (Survey response)

Some academics marked students’ work in stages (34%). However the multilateral peer-to-peer interactions and repeated student-to-staff interactions during a cumulative Web 2.0 assignment could put pressure on staff responsible for marking large numbers of students:

Last year I had them doing a blog every week … the marking was killing me because you've got to mark them and get them back in a week. And then you've got four or five classes, that's six or seven hours work a week in marking. And then you've got [other] assignments and exams. (Interviewee 12)

Managing the quality of assessment processes

In terms of overall assessment management, using Web 2.0 for student assignments holds clear value for some academics but a level of risk for others. The survey addressed academic administration and quality management by asking participants to respond to a series of statements about their assignment in relation to major assessment policy issues. These were distilled from a review of Australian universities’ publicly available assessment policy and procedure documents. Table 3 shows that most academics were confident that they were managing their assignment appropriately in most respects. However there were noteworthy levels of non-alignment or uncertainty in matters such as offering supplementary assessment, keeping students’ marked work on file, safeguarding students’ identity and privacy online, and recording students’ learning in ways that would allow it to be tendered as evidence of meeting prerequisites for further study (e.g. for recognition of prior learning).

Several interviewees described how Web 2.0 affordances supported the quality management of assessment in terms of monitoring progress, reinforcing academic integrity, and protecting students’ moral rights. For example:

Students start off being able to see their own posting. So they can't copy off one another. And then later on they see the other postings but then they can't edit their own. So it truly is their own work because that first bit is done in their creative sense. (Interviewee 12)

One of the reasons why I chose to go with the blogs and make it an informal writing exercise (was) because then I could get a sense of their voice, get a sense of consistency as well … so when it comes to marking the formal assessment tasks that's another plagiarism check. (Interviewee 21)

Academics also noted key challenges for quality management of assessment, such as ‘being able to guarantee ongoing access to materials hosted on sites outside the university’ (Survey response) and ‘all those issues of confidentiality and copyright’ (Interviewee 20). As Interviewee 13 stressed:

I tell the students over and over again that it is on the World Wide Web, it's not associated with the university, be careful what you put up there, make sure you are comfortable with this.

Discussion

These findings show that academics have found many features that are worthwhile, and many ways to see value in designing, conducting, marking and managing student Web 2.0 assignments. They report a range of challenges and a few real risks: some familiar to anyone changing their assessment practices but others unique to the assessment of student Web 2.0 activities.

It is noteworthy that a relatively small number of academics responded to the survey and many of them had used the Web 2.0 assignment for the first time that year. We may speculate that other academics did not respond either because their use of this form of assessment was less well developed, or because they did not use this form of assessment at all. Although the use of student Web 2.0 activity in university education is rising perceptibly, our findings show that low and medium-stakes assignments are most common. This is consistent with our previous research, which indicated that Web 2.0 assignments are offered chiefly for optional enrichment or as preparation for a more conventional written assignment or examination (Gray et al. 2010).

Academics in our study, the majority of whom were from Humanities and Social Sciences disciplines, were clearly finding value in the opportunities that Web 2.0 afforded for assessment of openly published and collaborative student work. As these academics would appear to represent the prime essay-writing disciplines, it could be proposed that written text remains the most assessable of the many Web 2.0 forms of activity. According with this, and consistent with the types of innovation most commonly reported in the literature, we found that Web 2.0 assignments most frequently took the form of blogging and wiki writing. The academics still identified challenges with these affordances of open publishing and collaborative writing. Even though they were willing to put effort into coaching students in the use of these new forms, they remained concerned, for instance, that students’ text-building efforts were not ‘academic’ enough. Academic staff seemed to be trying to deal with these issues as isolated practitioners, innovating through trial and error.

Interestingly, most respondents were subject coordinators, potentially responsible for teams of tutors and markers, yet they volunteered little evidence of open or collaborative marking practices, and there was little suggestion of novel methods used to evaluate the effectiveness of their assessment practices. There would seem to be substantial room for more creative uses of social media technologies in assessment to automate or streamline marking; to devise new forms of feedback; and to incorporate multilateral assessment, making use of peers or even extending Whitelock's (2010), p. 322) concept of a ‘community’ of assessors beyond the classroom. The use of Web 2.0 technologies may prove too disruptive for conventional assessment practices, shaking the foundations of two hitherto solid pillars of contemporary assessment thinking. The division between formative and summative assessment and the straightforward alignment of assessment tasks with objectives may ultimately be re-engineered by Web 2.0 assessment forms. In the Web 2.0 environment, formative and summative assessment can appear like a double helix – and perhaps become indistinguishable. As the process of content creation is transparent for the duration of the task and after, this makes it possible (and arguably raises student expectations) that summative assessment will attach value to the processes that students use to create their assessable products. It may be necessary to revisit constructive alignment between the learning activity required and the learning outcome intended (Collis and Moonen 2008).

Even when academics undertake solely summative assessment in these assignments, the affordances of Web 2.0 mean that some formative feedback is most likely inbuilt. Thus, the work of arriving at a student's final mark can – and arguably must – explain how the student's response to feedback has added value and accrued marks along the way to the later summative grade. A third pillar of assessment convention therefore may require re-engineering here: As well as the students’ content creation being made highly visible, the often very private evaluative work of giving formative and summative feedback and the detail of the actual feedback given can become more transparent than ever before, not only for a student cohort, but also for academic colleagues and others. This represents a shift from academics conducting assessment in a ‘black box’ (Hattie 2009).

Thus, in using Web 2.0 for assessing student learning, questions about which (and whose) ‘achievement’ is to be assessed may, at least in the short term, become more contested. Is it the student who has been prepared to take individual risks (which may not, in the end, have been totally successful), or the student who has collaborated wholeheartedly with their peers? Is it the student who has advanced in their progress towards a professional identity, or the student who has shown particular talent in using Web 2.0 tools? And what of the academics? Is it the one who has introduced a specific Web 2.0 assessment form for learning a particular technique or concept, or the one who has redesigned assignments to be more engaging for students? Is it the academic who has found a method to facilitate staged assessment, or the one who has devised a method for monitoring group work? An academic without a sound rationale for assessing students’ Web 2.0 activities will struggle to justify the added effort flowing from the assessment design – effort which may involve addressing critical questions about the allocation of extra attention and support for the duration of the task; about the calculation of results for students’ learning outcomes; and about the propriety of interactions between university students and others who are active in the wider social Web.

A limitation of this study is that it offers an overview of academics’ self-reported practices only. We cannot be certain whether there were others who chose not to respond and what their reasons might have been. Still needed are in-depth comparative studies of individual academics’ decisions in using Web 2.0 for assessment, as well as a fuller understanding of university students’ perspectives on these kinds of assessments. Accordingly, a follow-on study from the survey reported here involves semester-long observation and document analysis in 17 diverse subject learning and teaching case studies of Web 2.0 assessment, as well as interviews and focus groups with participating staff and students (Waycott et al. 2010b).

Conclusion

Most advocates of using student Web 2.0 activities in higher education offer little detailed advice on how to conduct assessment which comes to grips with the unique features of Web 2.0, its difference from previous forms of student writing and staff marking or its academic administration. This study raises important considerations for teaching academics, for educational developers and technologists and for learning and teaching leaders and policy-makers. For individuals or groups who have heard enthusiastic commentaries about the assessment potential of Web 2.0. our research offers new insights, derived from broadly based empirical data, into the chequered realities of current practice.

It may be inadvisable for intending innovators to follow the example of our study participants in proceeding conservatively and applying Web 2.0 affordances selectively to suit their pre-existing pedagogical context. Nor does is seem to be a good idea to work on these pedagogical innovations as a sole practitioner of Web 2.0 assessment in one's academic department or discipline. In terms of educational efficiency and effectiveness, incremental and individualistic uptake of Web 2.0 may not achieve sufficient scope and scale to deliver substantial improvement of academic assessment, and so may not be worth it in the face of workload and policy concerns.

Institutionally and collegially concerted shifts in this practice have more likelihood of adding the anticipated value to higher education. So far, although reform of student assessment in higher education is called for – ‘The revolution will encompass ‘feedback from assessment’ and the development of visible learning and visible teaching’ (Hattie 2009) – and despite the potential of Web 2.0 technologies to facilitate such change, Web 2.0 cannot be seen to be transforming university assessment. On the contrary, its affordances are being appropriated, in some cases, to perpetuate conventional, invisible university assessment practices (even to the extent of constraining it inside online Learning Management Systems). This trend is at odds with the view of some commentators that its very openness is what makes the social web most valuable. Staff and students who are ideologically inclined towards this openness are increasingly likely to seek to disrupt the status quo of assessment standards and policy with ‘challenges concerning collaboration, evaluation, and the general aim and status of the material produced by students’ (Dohn 2009).

For those whose role is to encourage and support innovation in higher education, it is clear that the majority of teaching staff and students too will need broadly-based, well-informed advice and support to make the most of the assessment potential of Web 2.0. There is much work to be done before we can be confident that using Web 2.0 for assessment is safe, fair, engaging and worthwhile for students and staff.

Acknowledgements

We are grateful to the 64 study participants from universities throughout Australia who were willing to share their experiences with the wider academic community; and to anonymous reviewers for constructive feedback on this paper. Support for this publication was provided by the Australian Learning and Teaching Council (ALTC) Ltd., an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this paper do not necessarily reflect the views of the ALTC.

References

Bartlett-Bragg, A. (2008) ‘Reframing assessment: using social software to collect and organise learning’, Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2008, Chesapeake, VA, USA. Chesapeake VA: AACE. pp. 4020–4024.

Bass, R. & Eynon, B. (2009) ‘Capturing the visible evidence of invisible learning (Introduction and synthesis of findings)’, Academic Commons, (January), pp. 4–29.

Bloxham, S. & Boyd, P. (2007) Developing Effective Assessment in Higher Education, Open University Press, Milton Keynes.

Boud, D. (2000) ‘Sustainable assessment: rethinking assessment for the learning society’, Studies in Continuing Education, vol. 22, pp. 151–167. [Crossref]

Bower, M. (2008) ‘Affordance analysis: matching learning tasks with learning technologies’, Educational Media International, vol. 45, pp. 3–15. [Crossref]

Bower, M., Hedberg, J. G. & Kuswara, A. (2010) ‘A framework for Web 2.0 learning design’, Educational Media International, vol. 47, pp. 177–198. [Crossref]

Collis, B. & Moonen, J. (2008) ‘Web 2.0 tools and processes in higher education: Quality perspectives’, Educational Media International, vol. 45, pp. 93–106.

Conde, M., et al. (2011) ‘Web services layer for Moodle 2.0: a new area of possibilities in web based learning’, International Journal of Technology Enhanced Learning, vol. 3, pp. 308–321. [Crossref]

Conole, G., et al. (2004) ‘Mapping pedagogy and tools for effective learning design’, Computers and Education, vol. 43, pp. 17–33. [Crossref]

Conole, G. & Alevizou, P. (2010) A Literature Review of the Use of Web 2.0 Tools in Higher Education. A Report Commissioned by the Higher Education Academy. Open University, Milton Keynes.

Dohn, N. (2009) ‘Web 2.0: inherent tensions and evident challenges for education’, International Journal of Computer-Supported Collaborative Learning, vol. 4, pp. 343–363. [Crossref]

Earl, L. (2003) Assessment as Learning: Using Classroom Assessment to Maximise Student Learning, Corwin Press, Thousand Oaks, CA, Corwin Press.

Ehlers, U. D. (2009) ‘Web 2.0 – e-learning 2.0 – quality 2.0? Quality for new learning cultures’, Quality Assurance in Education, vol. 17, pp. 296–314. [Crossref]

Elliott, B. (2008) ‘Assessment 2.0’, International Journal of Emerging Technologies in Learning, vol. 3, pp. 66–70.

Farmer, B., Yue, A. & Brooks, C. (2008) ‘Using blogging for higher order learning in large cohort university teaching: a case study’, Australasian Journal of Educational Technology, vol. 24, pp. 123–36.

Franklin, T. & van Harmelen, M. (2007) Web 2.0 for Content for Learning and Teaching in Higher Education. JISC, Bristol.

Gray, K., et al. (2008) ‘Web 2.0 authorship: issues of referencing and citation for academic integrity’, The Internet and Higher Education, vol. 11, pp. 112–118. [Crossref]

Gray, K., et al. (2010) ‘Students as Web 2.0 authors: implications for assessment design and conduct’, Australasian Journal of Educational Technology, vol. 26, pp. 105–122.

Hattie, J. (2009) ‘The black box of tertiary assessment: an impending revolution’, in Tertiary Assessment & Higher Education Student Outcomes: Policy, Practice & Research. Ako Aotearoa, Wellington, New Zealand, pp. 259–275.

Hatzipanagos, S. & Warburton, S. (2009) ‘Feedback as dialogue: exploring the links between formative assessment and social software in distance learning’, Learning, Media and Technology, vol. 34, pp. 45–59. [Crossref]

Hughes, A. (2009) Higher Education in a Web 2.0 World: Report of an Independent Committee of Inquiry into the Impact on Higher Education of Students. Widespread Use of Web 2.0 Technologies”, Becta/Department of Employment and Learning/Higher Education Academy and others, UK.

Johnson, L., Levine, A. & Smith, R. (2009) The 2009 Horizon Report. New Media Consortium, Austin, TX.

Kirkwood, A. & Price, L. (2008) ‘Assessment and student learning: a fundamental relationship and the role of information and communication technologies’, Open Learning: The Journal of Open and Distance Learning, vol. 23, pp. 5–16. [Crossref]

Kuswara, A. (2011) ‘Realising the potential of Web 2.0 for collaborative learning using affordances’, Journal of Universal Computer Science, vol. 17, pp. 311–331.

Lai, Y. C. & Ng, E. M. W. (2010) ‘A case study of infusing web 2.0 tools for blended learning: virtual presentations as an alternative means of assessment’, in Comparative Blended Learning Practices and Environments, ed. E. M. W. Ng, Information Science Reference, Hershey, PA.

Lee, M. & McLoughlin, C., eds. (2010) Web 2.0-Based-e-Learning: Applying Social Informatics for Tertiary Teaching, IGI Global, Hershey, PA.

Ng, H. Z. & Huzzain, R. M. R. (2009) ‘Empowering learners as the owners of feedback while Youtube-ing’, Interactive Technology and Smart Education, vol. 6, pp. 274–285. [Crossref]

O'Reilly, T. & Battelle, J. (2009) Web Squared: Web 2.0 Five Years On. Special Report [White Paper] for the Web 2.0 Summit, San Francisco, CA, [online] Available at: http://assets.en.oreilly.com/1/event/28/web2009_websquared-whitepaper.pdf

Rust, C. (2002) ‘The impact of assessment on student learning’, Active Learning in Higher Education, vol. 3, pp. 145–158. [Crossref]

Rust, C., O'Donovan, B. & Price, M. (2005) ‘A social constructivist assessment process model: how the research literature shows us this could be best practice’, Assessment and Evaluation in Higher Education, vol. 30, pp. 231–240. [Crossref]

Sadler, D. R. (2009) ‘Fidelity as a precondition for integrity in grading academic achievement’, Assessment and Evaluation in Higher Education, vol. 34, pp. 1–17. [Crossref]

Schneckenberg, D. (2009) ‘Web 2.0 and the empowerment of the knowledge worker’, Journal of Knowledge Management, vol. 13, pp. 509–520. [Crossref]

Thompson, C. (2008), ‘Dialogism and social computing: academic authorship in cyberspace’, in Changing University Learning and Teaching: Engaging and Mobilising Leadership, Quality and Technology, eds. J. Mcconachie et al., Post Pressed, Flaxton, Australia, pp. 359–374.

Waycott, J., et al. (2010a) ‘Implications for academic integrity of using web 2.0 for teaching, learning and assessment in higher education’, International Journal for Educational Integrity, vol. 6, pp. 8–18.

Waycott, J., et al., (2010b) ‘Transforming assessment in higher education: a participatory approach to the development of a good practice framework for assessing student learning through social web technologies’, in Curriculum, Technology & Transformation for an Unknown Future, eds. C. H. Steel, et al. Proceedings ascilite Sydney 2010, pp. 1040–1050.

Waycott, J. & Sheard, J. (2011) ‘Editorial 27(5): preface to the special issue – assessing students’ Web 2.0 activities in higher education’, Australasian Journal of Educational Technology, vol. 27, pp. iii–ix.

Whitelock, D. (2010). ‘Activating assessment for learning: are we on the way with web 2.0?’, in Web 2.0-Based-e-Learning: Applying Social Informatics for Tertiary Teaching. eds. M. Lee & C. McLoughlin, IGI Global, Hershey, PA, pp. 319–342.

Appendix 1. Interview protocol

HREC 0932125

Thank you very much for completing the online survey.