RLT Research in Learning Technology 2156-7077 Co-Action Publishing 21281 10.3402/rlt.v21.21281 Capturing the sociomateriality of digital literacy events Capturing the sociomateriality of digital literacy events Bhatt Ibrar a * de Roock Roberto b School of Education, University of Leeds, Leeds, UK Language, Reading, and Culture, University of Arizona, Tuscon, AZ, USA Corresponding author. Email: edib@leeds.ac.uk; ibrarbhatt@gmail.com 31 01 2014 2013 21 10.3402/rlt.v21.21281 30 04 2013 04 12 2013 © 2013 I. Bhatt and R. de Roock 2013

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This paper discusses a method of collecting and analysing multimodal data during classroom-based digital literacy research. Drawing on reflections from two studies, the authors discuss theoretical and methodological implications encountered in the collection, transcription and presentation of such data. Following an ethnomethodological framework that co-develops theory and methodology, the studies capture digital literacy activities as real-time screen recordings, with embedded video recordings of participants’ movements and vocalisations around the tasks during writing. The result is a multimodal rendition of digital literacy events on- and off-screen, allowing linguistic and multimodal transcriptions to capture the complexity of the data in a format amenable to analysis. Acquiring such data allowed for the development of detailed analyses of digital literacy events in the classroom, including interaction that would otherwise have escaped standard ethnography and video analysis, through sensibilities that approach social and material items without a priori hierarchies. This leads us to a ‘performative’ notion of digital literacies and an analytic methodology that is useful for researchers paying greater attention to the sociomaterial assemblages in which digital literacy events unfold.

digital literacies multimodality video analysis screen capture literacy studies actor-network theory
Introduction

Video analysis is a booming methodology within qualitative research arenas (Knoblauch 2012). It has been successfully employed in a number of areas alongside other methodologies (e.g. Deppermann 2013; Fields and Kafai 2009; Garcez 1995) and, as it begins to coalesce as a methodology (Knoblauch 2012), there remains a notable acceleration of its use and development throughout social research. Due in particular to its ability to capture elements increasingly salient in recent digital literacy theory, we suggest in this paper that video analysis also holds great promise for research in digital literacies and learning technology more generally. Its successful use, however, requires overcoming both technological and methodological issues, including the relationship between theory and methodology.

This paper discusses a method of collecting multimodal data in classroom-based digital literacy research. We draw from the insights and experiences of two research studies that examine interactions around, through and with computers during digital literacy events. Given our focus here on the data collection aspect of our research methodology, we touch on analyses and conclusions relatively lightly. The first study examined digital composition and web research in a college classroom. It addresses the following questions: What are the digital literacy practices of learners whilst working on assignments in classrooms? How do these practices relate to the digital demands of the programmes of study and digital literacy practices of the students’ social lives? The second examined learning practices around the integration of laptops into a 6th grade classroom, focusing on ways laptops were integrated into everyday classroom learning practices by both teachers and students. Specifically, it asks: What practices are evidenced in student learning through a combination of face-to-face and computer mediated collaboration when engaging with new digital media? Both studies capture on-screen composition and online activities as digital recordings alongside embedded video recordings of students’ real-time interactions. This provides a rich multimodal rendition of the activities on- and off-screen, which are then transcribed in ways that capture the complexity of the data in a format amenable to analysis. These data are augmented by observation notes, interviews and collected student work. Appropriate CAQDAS1 (Lewins and Silver 2007) tools are then used to manage, transcribe, code and analyse the data. The proposed combination of research techniques was adopted in order to explore what participants see, say and do in relation to what they write in the classroom. This provided us with insights through a composite picture of classroom interaction and a detailed, dynamic rendition of digital literacy activities.

New literacies, new methods

As textual practices are transformed by emerging technologies and reinterpreted by new understandings of literacy, research methodologies have also evolved. This evolution, however, has largely been parallel rather than dialectical. The digital revolution has helped transform the way we think about literacy; turns to the visual (Mitchell 1994), social (Gee 1996), multimodal (Jewitt 2009) and post-human (Hayles 1999) in studies of semiotic exchange have meant a drastic re-evaluation of ‘what counts’ in meaning-making to include consideration of an increasingly complex array of agents, modes and semiotic systems. However, how social interaction in general, and literacy in particular, is examined has not changed nearly as radically (Caperton 2010; Heath and Luff 2000). For example, drawing on Gee's theorisation of video-game based literacy practices, Caperton (2010) argues that researchers need to explore and refine innovative methodological approaches for observing and capturing the unique forms of learning and literacy in digital gaming environments. Echoing Gee and Caperton, in our own research we found the need to enhance traditional and even cutting edge methods of data collection and analysis with ‘bleeding edge’ (Woods and Dempster 2011) techniques that combine advances in digital technology with a robust methodological framing. We therefore argue that the complex and distributed nature of learning and literacy in digital environments requires tools and techniques for the opening up of interesting educational ‘black-boxes’ to scrutinise how student work, game-play and learning are done through a rich account of their practical action, so as to inform pedagogic practice.

Our research methodology is therefore guided by ethnomethodological (e.g. Garfinkel 1967) approaches that prioritise seeking an emic/insider perspective over an a priori commitment to specific methodological framings. In other words, the phenomena being examined, and their constitutive practices, guide theorising and data gathering. We focus on empirically observable events and interactions as they happen, including their shaping, trajectories, impasses, breakthroughs and enactment through ‘sociomaterial’ work. This paper focuses on the ways sociomateriality extends ethnomethodological approaches to research and the ramifications for video analysis and literacy research. These notions are developed in the below sections.

The studies

Both studies aim to conceptualise personal and non-curricular digital literacy practices in classroom spaces, their mobilisations as resources in curricular work, and how cyberspace is invoked as a conduit for doing so. Study 1 was part of a series of ethnographic case studies (cf. Bhatt 2012) exploring how student assignments are written in the UK's Further Education (FE) sector. The FE sector consists of young people undergoing second-chance school education, degree-level programmes and a plethora of vocational courses as part of British ‘post-compulsory’ education.

The research adopts a phased multi-method ethnographic approach involving screen recordings with embedded video recordings in a screen-in-screen format, in order to capture computer writing assignment activities in practice (Figure 1). Data were augmented by classroom observations noting organisational behaviour and embeddedness of digital media, and interviews to analyse the participants’ applications of digital literacy in daily life and how such practices interact with the demands of class work and projects.

Data capture setup for Bhatt (2012) used a screen capture recording with an embedded video recording and omni-directional microphone for students’ vocalisations.

Study 2 (de Roock forthcoming) was an eight-month video ethnography in a 6th grade (10–11 years) language classroom with primarily bilingual Mexican-American students of a public middle school in a Southwestern US city. In contrast to study 1, video recordings were focused on whole class and small group interactions. Multiple and simultaneous screen-in-screen recordings of individual students were paired with wide-angle recordings of the small group (Figure 3). Students (sometimes in different classrooms) were captured interacting online while interacting with their peers around them. As with study 1, interview data also focuses on out of school digital media and technology use. The ‘density’ of the data is high; each video contains the on-screen activity, mouse clicks, a keystroke log, a narrow angle webcam video and high-quality audio.

A screenshot from Transana (study 2) showing video from a tripod-mounted camera synced with a screen-in-screen video captured from each of the participants’ computer plus a conversation analysis style transcript (de Roock forthcoming).

The level of granularity we adopted, drawing on video analysis and ethnomethodology, serves to highlight and make observable to the reader the complex interplay of actors within assemblages that emerge in a given digital literacy event. Aspects of our analyses, therefore, are at times ‘fine-grained’, comparable to other focussed and video-based ethnographies such as those cited later in this paper. But as we aim to highlight the interactions and enactments that emerge in specific circumstances, such a level of granularity is not always necessary; supporting ethnographic data (e.g. documents, interviews, observations) as part of broader investigations may lead to less ‘fine-grained’ analytic choices when following actors and their associations. We hope to highlight the way research methodology is intimately tied to researcher epistemologies, is bounded within a broader network of actors, and emerges from both the context and phenomena under investigation.

Theoretical background Literacies and the ‘digital’

Scribner and Cole (1981, p. 236) discuss literacies as ‘socially organised practices [that] make use of a symbol system and a technology for producing and disseminating it’. Following Scribner and Cole, and drawing also from Street (1984), Barton and Hamilton (2000) define literacy as ‘general cultural ways of utilising written language which people draw upon in their lives’ (p. 7). ‘Literacy’ from these perspectives is relativised, ethnographically studied in the particular contexts of which they are an integral part (Street 2001) and conceptually expanded beyond a tendency to view it as a synonym for ‘competency’.

From the subsequent ‘social turn’ in literacy studies (c.f. Gee 2000), notable works focussing on a similarly expansive notion of the term ‘digital literacies’ (note the plural) (e.g. Bawden 2008; Gillen and Barton 2010; Lankshear and Knobel 2008; Martin and Grudziecki 2007) have sought to uncover the complex social, cultural and technical practices emerging through literacy in digital environments. These investigations and critiques take us beyond Gilster's (1997) early vision of ‘digital literacy’ (note the singular) as simply a set of information management skills and the competencies involved in operating digital media generally. Lankshear and Knobel's (2008) pluralisation of the term echoes earlier visions of an expanded notion of ‘literacies’ (c.f. Barton and Hamilton, Scribner and Cole, Street, etc.) that goes beyond the competencies of the individual. Their conceptualisation of ‘digital literacies’ therefore encompasses ‘digital codification’ and ‘enculturations’ (Lankshear and Knobel 2008, pp. 5–7) that relate to the myriad of meaning-making practices evoked across different settings, communities and identities in digital environments. Expanding and unresolved definitions and conceptualisations of digital literacy/literacies have led to what Chase and Laufenberg (2011) refer to as its ‘inherent squishiness’ (p. 535), with evolving and emerging technologies to shape both what digital literacies represents and how we go about examining them.

Digital literacy events

We began with cases of digital literacy events as observable units of analysis. A literacy event is an empirical occasion involving interaction and activities around a text. It is a key term in the study of literacy as social practice (Heath 1982, 1983), and as a construct has developed in a number of studies (cf. Baynham and Prinsloo 2009). What is central in the capture and analysis of literacy events is the ‘configuration of action, talk and text’ (Prinsloo and Baynham 2008, p. 4, emphasis added). Far from being isolated instances of reading and/or writing, literacy events form part of highly contextualised encounters, such as religious rituals (Besnier 1995) and bedtime stories (Heath 1982). As a unit of analysis they yield snapshots of the social and cultural order in which literacy activities are institutionally and organisationally mediated (Prinsloo and Baynham 2008), yet also remain intimately tied to material culture (Brandt and Clinton 2002).

Drawing from this notion of ‘event’ and Lankshear and Knobel's (2008) framing above, for us ‘digital literacy events’ are observable occasions in which digital text is central and where meanings are ‘mediated by texts that are produced, received, distributed, exchanged, etc., via digital codification’ (Lankshear and Knobel 2008, p. 5). Examples in today's digitally infused classroom spaces include such things as designing and delivering presentations, the writing of assignments and Web-based research for projects. For researchers, the significance in terms of elucidating digital literacy practices arises from the repeated observation of the digital literacy events, and the rituals and relationships involved.

Digital literacy events are also multimodal. They cannot be thought of as simply linguistic, especially as modes of communicating grow and intermix (Jewitt 2008; Kress 2003; Lemke 2002). Multimodality as a synthesising construct allows us to better understand how digital literacy is done with its focus on the inter-dependent layers of communicatory modes. Multimodal interactions, therefore, in as much as they are integral to digital literacy events, necessitate the generation and analysis of data from multiple modes and often requiring a range of analytic tools, including conversation analysis (Deppermann 2013).

Sociomateriality

Latour (2004, p. 227) argues against a priori divisions of ‘social’ and ‘material’ elements by asking readers to envisage a battlefield with soldiers:

[A]ccount for the dynamic of a battle by imagining, first, a group of soldiers and officers stark naked; second, a heap of paraphernalia—tanks, paperwork, uniforms—and then claim that ‘of course there exists some (dialectical) relation between the two’… [T]here exists no relation whatsoever between the material and the social world, because it is the division that is first of all a complete artefact. To abandon the division is not to “relate” the heap of naked soldiers with the heap of material stuff, it is to rethink the whole assemblage from top to bottom and from beginning to end. (Latour 2004, p. 227)

Books, computers, algorithms, smart phones and other intrusions of non-human agencies, as with Latour's ‘tanks, paperwork, uniforms’, are inescapable in our digital literacy practices, and best understood as acquiring their characteristics through their interpenetration with humans. Sociomateriality as a portmanteau construct takes account of this relational ontology in the social world; computers, for example, can be said to mediate interlocutors, have a voice and be interlocutors in themselves (Nicholls 2009). Material artefacts more generally are both part of our cultural inheritance and mediating factors in our cognition (Hayles 2012; Hutchins 1995). As Orlikowski (2007, p. 1437) contends, ‘there is no social that is not also material, and no material that is not also social’.

Understanding any actor, therefore, requires following its everyday activities and examining its interactions with its social and material environment. Similarly, to understand digital literacy events, the entirety of the interactional context – human and non-human, online and offline – needs to be examined.

In the course of research, we increasingly drew from the notion of a ‘sociomaterial assemblage’ (Fenwick, Edwards, and Sawchuk 2011; Johri 2011; Orlikowski 2010) to conceive of digital literacy events as constitutive entanglements of social and material agencies, in the way Latour contends. Assemblage theory originates from Gilles Deleuze and Felix Guattari, with the term itself as a translation of their original French term agencement (Deleuze and Guattari 1980). More abstract than its English counterpart, agencement connotes an entanglement of ‘tentative and hesitant unfolding’ (Law 2004, pp. 41–42). For the purposes of this paper, we use sociomaterial assemblage as an approximate synonym for actor-network 2 (cf. Law 2009) and use this theoretical construct to address the complex entanglement of social and material work that goes into classroom digital literacies.

This conceptual shift has been developed within the field of STS (Science, Technology and Society), and has shaped its various derivative projects including actor-network theory (Callon 1986; Latour 2005; Mol 2003) and feminist technoscience (Barad 2007; Haraway 1997). It holds that there are no independently existing entities except that they materialise and attain different qualities through their particular relations and configurations over different spatialities and temporalities. Realities therefore can be ‘collateral’ (Law 2012) and ‘multiple’ (Mol 2003), rather than singular or coherent, and inseparable from – and assembled by – the practices which emerge at a scene of interest. This relates to a ‘performativity of practice’ (Law 2012, p. 161) and means we have to carefully attend to the ecology of practices (and their contestations, impasses, breakthroughs, etc.) in a digital literacy event to see how sociomaterial relations are assembled and their realities (such as class work, assignments) are done. In other words, an assemblage of competing and disparate practices holds the cultural order of the digital literacy event together. This, we suggest, is a ‘performative’ conceptualisation of digital literacies; that digital literacy practices, their pre-existing realities and texts, perform the reality of the classroom, the assignment, the game-play, etc. Consistent with Gourlay and Oliver (2013) we adopt sociomaterial perspectives as an expansion to the view of literacy as an embodied ‘social practice’(c.f. Barton and Hamilton, Scribner and Cole, Street, etc.), and discuss theoretical and methodological implications to the collection, transcription and presentation of data.

Sociomateriality subsequently provides a theoretical lens and conceptual shift to frame our methods construction and analyses as we endeavour to take more kinds of evidence into account. In digital literacy events, learners do more than write on a screen at their desks. Their written work is undertaken alongside a host of interrelated behaviours, including talk around the task, and interactions with actors (human and non-human) across different spatialities and temporalities. The intersections and interactions of these practices, and their contribution to student work/play, form the basis of how we seek to use this approach to explore digital literacy events. We thus began to search for additional methodological tools to better understand what occurs when students interact with computers in classrooms.

Methodology

There is nothing new about the individual methods we utilise for capturing data. Video-based research, for example, has been around for decades (e.g. Heath, Hindmarsh, and Luff 2010; Goodwin 1994, 2000; Leeuwen and Jewitt 2000) as have predominantly audio-based methods for conversation analysis (Psathas 1995), and think-aloud techniques (Ericsson and Simon 1993). The combination of them in the way we propose, however, is relatively new in educational research. New elements include the combination of methods that already exist, their use in novel ways to permit multimodal analyses and utilising the latest advances in qualitative research software tools.

Video recorded data

Video data offer greater flexibility over solely handwritten or typed observation notes and audio recordings. They also provide context that would otherwise be uncaptured in audio recordings (Edwards and Westgate 1994), and also allow for retrospective coding and analysis. The relative ease with which video file data can be manipulated (segmented, slowed down, etc.) also allows for new levels and combinations of qualitative and quantitative analyses (Jacobs, Kawanaka, and Stigler 1999; McLarty and Gibson 2000). In this respect, ethnomethodologically inspired studies of workplace interaction have been influential in incorporating the rigour of linguistic transcription with multimodal sensibilities on the detail of how technologies feature in day-to-day conduct and interactions (e.g. Heath, Knoblauch, and Luff 2000; Luff, Hindmarsh, and Heath 2000; Suchman 1987). Among them are studies of doctor–patient interaction (Greatbatch et al. 1995; Heath 1986; Pasquandrea 2011), journalists’ writing strategies (Van Hout, Pander Maat, and De Preter 2011) and the role of computers in co-operative work (Heath et al. 2002).

Ethnomethodology and conversation analysis, as analytic orientations, have been highly influential on studies of workplace interaction and have led to a large body of empirical work illuminating the complex character of sociomaterial interaction, how tools and technologies feature in mundane organisational activities. Therefore, like Heath, Hindmarsh, and Luff (2010, p. 1), our choice of methodological approach emerges from a framework that ‘prioritises the situated and interactional accomplishments of practical action’. Our understanding of digital literacy events and classroom composition as similarly complex and sociomaterial calls for a level of detail and temporality that video analysis can begin to provide, but when used alone is insufficient. Combining multiple views (e.g. screen capture) of participants’ interaction with technologies and each other was therefore necessary to address the relational and mutually constitutive work between human and material elements involved in classroom writing.

Screen recording with video

Screen recording software has been employed by some researchers as a tool to record the iterative processes of on-screen composition of texts (Asselin and Moayeri 2010; Geisler and Slattery 2007). Predominantly used by writing-process researchers, screen recordings mitigate against the potential distortions of both think-aloud protocol data for writing research (Hayes and Flower 1980), and keystroke logging techniques (Miller, Lindgren, and Sullivan 2008). We are not the first to try to capture writers’ complete interactions on- and off-screen. Notably, nearly 30 years ago, Bigum and Gilding's (1985) attempt at synchronously capturing students’ writing, movements, and talk around a task required two monitors, a video mixer, a video tape recorder, and a means of splitting the computer video signal (see Figure 2).

The data capture setup from Bigum and Gilding (1985, p. 96) had the camera positioned so that it photographed the students from one side and the front.

Since then studies adopting similar approaches to incorporate screen activity with video analysis in co-ordinated work are few in number. Among them are Garcez's (1995) microanalyses of students’ interactions with a multimedia programme, Hindmarsh et al.'s (2000) exploration of collaborative work online (without using dedicated screen capture software) and Woulds's (forthcoming) multimodal analysis of interaction in a second language classroom. More linguistically oriented studies include explorations of journalistic writing strategies (Van Hout and Macgilchrist 2010; Van Hout, Pander Maat, and De Preter 2011) and collected screen recording data of Facebook chats (Meredith and Stokoe forthcoming). More recently, Garcez, Bulla, and Loder (forthcoming) provide a more up-to-date review of this type of research procedure, its implications and what it affords applied linguistics researchers adopting micro-ethnographic approaches.

Data management

In the process of data management and analysis, decisions were made that reflect theoretical orientations, issues of practicality and the unique ‘demands’ of these types of data (Luff and Heath 2012). For example, the screen capture software used (Blueberry Flashback Recorder) allows for the screen-in-screen format as shown in Figure 1 and 3, conflating the screen recording with the webcam and audio recording along with additional data (keystrokes, mouse clicks, etc.). Important methodological decisions included the placement of the webcam image, quality of the video and audio chosen, and privileging of modes. These decisions surrounding our apparatus are outlined in the section below.

‘Reassembling’ digital literacies

Having argued that digital literacy events are entangled within assemblages of human and non-human agencies, we are then required to acknowledge the agencies of the very apparatus we use to capture them. The management of our data, its packaging and formulation for your reading of it here together assemble, as Law (2012) makes clear, a ‘putative reality’ (p. 156) of the digital literacy event.

According to Barad (2007) ‘[a] performative understanding […] takes account of the fact that knowing does not come from standing at a distance and representing but rather from a direct material engagement with the world’ (p. 49; emphasis in the original). What becomes important for research is ‘the nature of these enactments. […] And humans are not the only ones engaged in performative enactments’ (p. 49).

Consistent with the performative conception, the so-called ‘subjects’ and ‘objects’ of our research are always and already entangled with the very apparatuses (cameras, recordings, software, etc.) we use to record, or rather re-present, the phenomena in question. Barad (2003) further argues, drawing on Bohr's quantum physics and feminist theory, that ‘apparatuses are not mere static arrangements in the world, but rather … are dynamic (re)configurings of the world’ (p. 816 emphasis in original). Rather than assume their neutrality, apparatuses therefore ‘are themselves phenomena’ (p. 816). In this section we discuss how we as researchers, our tools, our interests and our intervention, are subsequently entangled in these assemblages and sociomaterial events, and our involvement in them is as important as the students’.

We laid out the data in the software based on research-related decisions, but decisions were also made for us by the apparatus we used, namely through the rules and conditions of the technological artefacts of data capture. The apparatus provides a symbolic output, which transforms pieces of matter into useable documents (Latour and Woolgar 1986, p. 51). These transformations, and their representations, can be characteristically classified as ‘inscriptions’ (Latour 1987) from which can emerge a multitude – or ‘cascade’ – of more inscriptions (p. 241) depending on how they are used and in what context by readers and users of our research. According to Latour (1999), inscriptions refer to the ‘types of transformations through which an entity becomes materialised into a sign, an archive, a document, a piece of paper, a trace’ (p. 306). For us, in each of the below cases a relationship exists between an inscription and the researched phenomena, and indicates representational work taking place (Table 1).

Data capture decisions surrounding our apparatus.

What we did Transformation conducted

Setting up of video recording Based on hunches and judgements of interest we placed and angled webcams and tripod-mounted cameras to capture the students’ movements and talk around the computers. The cameras, for their part, stay obediently mounted; the webcams show us one tight angle of this view continuously while the camera allowed for more flexibility alongside mounted and wireless audio recording.
Choosing and implementing screen recording software (audio, webcam video and screen recording) Blueberry Flashback (created for screencasts) is powerful and affordable for installing on multiple computers. Set to record by the researcher on multiple computers at a time then exported to USB sticks quickly at the end of class periods. Naming of files involved date, period, participant and then later some code for content (e.g. ‘MSP’ for Movie Star Planet). This process cast a wide net to over-gather data for later focus and yielded a comprehensive view of ‘what happened’ which we wanted to later view and scrutinise with ease.
File conversion Exported Flashback files to make accessible in preparation for management and analysis. In the process decisions were made and the data were altered into just video/audio, losing original timestamp and keystrokes. Audio was prioritised and video quality was reduced (for economy of conversion time and storage space) and the screen-in-screen image was docked in a particular place on the screen based on a cursory viewing of data and judgement of least important space on screen.
File management For ease of access on researchers’ multiple devices, video files were password protected and stored on Web servers (study 1) and an external hard-drive (study 2). File names determined organisation, based on the on-the-fly development of their naming system.
Data preparation Data were prepared for analysis in CAQDAS (ELAN, Transana) for transcription and manipulability (slowing down, segmentation, etc.). Data files managed within Transana, which automatically creates transcript and waveforms, since biased towards CA. Files are organised by day in ‘series’, so a folder would contain all videos from a given day to more easily triangulate with field notes, audio interviews and still photos.
Gisting CAQDAS allow viewing audio and video within their interfaces. Video files were given keywords according to phenomena of interest within and assembled into ‘collections’ of related videos based on keywords. Screen capture video emerged as the most useful. Starting with files tagged on-site, we scrutinised the recordings for noteworthy and discernible moments to address our research questions and insights of interest to research communities generally. In associated transcription files, we then created timestamp associated cursory summaries and open coding of salient videos. Since Transana is biased towards CA studies and not well designed for large data sets, this system was developed as a work-around.
Pairing and syncing (study 2) When specific video segments were selected, they were paired with 1–2 concurrently recorded co-present screen recordings along with the tripod-mounted camera recording. They were carefully synced so they advanced from the same point in time and provided multiple data streams of the same time period across multiple computers.
Coding and logging Based on open coding, selective coding was conducted on salient video segments to advance theoretical sampling and find specific moments for transcription. For study 1, selecting clips to analyse was achieved through first creating a descriptive video log. Avoiding the risk of a type of data saturation brought about by a potentially overwhelming amount of data, in line with Heath et al. (2010), we decided to focus on fragments of a descriptive video log of the events, of approximately 10 seconds, for selection and repeated viewing. Selection can be based on a range of concerns influenced by the research questions, observational notes, and other relevant and supporting data (e.g. interviews).
Transcription Data transcribed in CAQDAS tools were influenced by the theoretical orientation of the argument and epistemologies of software designers. Transana enables for vertical, CA style transcriptions while transcriptions in ELAN adopted multimodal conventions (Bezemer and Mavers 2011) to account for the host of interrelated behaviours, including gaze, talk around the task, and interactions with non-human agencies (e.g. Google searches). The subsequent representational system depicted in Figure 4 integrates, yet separately identifies, the combinative modes of sociomaterial activity and reflects our analytic attention.
Analysis Generating conceptual understandings from the recordings, in order for us to make arguments in oral and written presentations, is shaped by a range of concerns including research questions, notes and supporting data (e.g. interviews). Influenced by ethnomethodological and multimodal analysis methods, we examined the transcriptions of clips to develop theoretical perspectives and illustrations of phenomena. While preliminary understandings of the context glossed from fieldwork and field notes directed the inquiry, shaping the selection of data along the way, close analyses of clips provide essentially an additional layer of ethnographic inquiry that serve to extend, complicate and/or contradict preliminary findings.
Screenshots for presentations Talking stills of the recordings helps us to then create a tellable story of the data, with our descriptions alongside, for written and oral presentation purposes. It is a format which is accessible and (re)useable for all, and helps us to illustrate the concepts and arguments explored above.

Through managing the research process, it becomes clear how each transformation that was conducted – for example, turning talk into audio files and then into transcribed digital text – generates a chain of inscriptions each with their own unique properties designed to scale, normalise and communicate the phenomena for a particular audience. In other words we make the observed phenomena conform to the practices of our communities. Inscriptions therefore contain semiotic properties and are ontologically independent of what they ostensibly represent. Each (re)presentation inevitably involves some form of simplification, and the exclusion and addition of elements in the enacting of its new realities, in order for our data to be calculable and transportable for research presentation purposes and theorisation.

As researchers, we remain entangled in the researched phenomena in concert with the wider sociomaterial assemblage of persons, texts, software, hardware, etc. A salient example is discussed in Latour's (1999) account of research carried out in a site within the Amazon forest by a botanist, a pedologist and a geographer. The standardisation of the researchers’ step-by-step inscriptions of field data, the forest's soil, tree and flower samples, etc., form accessible charts, tables and graphs. These are then stored in cabinets and, subsequently, turned into ‘a concept’ which then transmutes into ‘an institution’ (p. 36).

From the outset (driving to the site and setting up) to the present (publishing screenshots of recordings) of our investigations, we engaged in a number of activities that transformed what we came to call the ‘digital literacy event’ into something else. In our data collection, management, analyses and subsequent (re)presentation, we made decisions which ‘reassembled’ the classroom activities to take into account (1) elements of classroom digital literacies which do not cohere with the dominant ontology of the ‘classroom’, and (2) how these contribute to the messiness and non-linearity of what is interpreted as student ‘work’ and ‘play’.

Analyses Study 1

With study 1 (participant: ‘Sara’), in observing how the classroom assignment took its shape, initial analysis reveals an ad hoc use of material artefacts and interactions with actors not always in situ; a sociomaterial assemblage extended into multiple spatial and temporal realms constituting:

A college policy which spells out and limits the purpose and scope for using digital media at the college, with ‘social’ and ‘non-educational chat’ considered as an ‘unacceptable’ activity.

Interactions with online resources: school and government websites, personal social networking ties and Google search suggestions.

Capricious violations of the college policy, including contact with friends through social networking, some of which was nonetheless assignment-related.

Utilisation of previous work as a basis upon which to begin her assignment.

All of these things, and more, were part of Sara's endeavour to garner information for her writing, and thereby transmute the assignment construction into a sociomaterial assemblage of multiple relations with ‘collateral realities’ (Law 2012). Such realities, according to Law (2012, p. 156) are ‘incidental’, sometimes ‘unintentional’, and ‘done in [the] practices’ at a scene of interest. Therefore, rather than there being an immediate reality inside the classroom, with an external reality of less significance, Sara assembles a collateral network of realities as part of her writing tactics.

A salient example is in Figure 4, where the transcript excerpt from ELAN is a clip of data showing Sara's behaviour whilst Web searching. Upon repeated viewing of the data clip, slowed down and expanded by the transcription of gesture and bodily movement, long pauses occur between turns during Web searching and the ensuing discussion about class work.

A transcript screenshot using ELAN from study 1.

The pauses correspond with Sara's exchanges with non-human actors (websites, search engines, etc.) during writing. Such apparent moments of ‘silence’, in a conventional transcript, appear as key moments of ‘interaction’ in the representational system proposed here; the materiality, in a sense, has a ‘voice’, in addition to the other interactions and ‘voices’ which are represented in a traditional linguistic transcription.

In the transcript screenshot, Sara is interacting with Google's algorithmic auto-search function. The algorithm is organised by software engineers far removed from the classroom, yet able to readily intervene to suggest search terms that it ‘thought’ she was searching for. The algorithm's interruptions and suggestions lead her far away from her original task, and the teacher was forced to get her on task again. Thus, the programming behind an algorithm is a crucial part of the assemblage as it interacts with the multiple written and oral realities of the other parts of the event's assemblage: the teacher's instructions, the friend's advice, the acceptable use policy, etc. The apparently simple division between human actors and passive material artefacts is thus made much more complex.

Study 2

After beginning with Latour's (2005) urging to ‘follow the actors’, this study came to focus on a particular network of interest (Fenwick and Edwards 2010) that became visible and analysable only through the methodology outlined above. In the process of data gathering, the surreptitious use of a fashion themed MMOG (massively multiplayer online game) by a group of girls emerged as salient. The game, Movie Star Planet, was appropriated by the girls to engage in literacy and identity work that was otherwise denied to them in the classroom space. The girls engaged in peer interaction, content creation and various expressions of identity through a variety of sociomaterial practices and semiotic modes. This assemblage disrupts traditional notions of classroom context; while it emerged from within a school space using sanctioned technologies, it brought together the digital activities and face-to-face collaboration of the girls in new ways. The emergent assemblage connected the girls and hundreds of other individuals in both synchronous and asynchronous interactions while reorganising their own offline interactions.

As others carrying out connective ethnographies acknowledge (Fields and Kafai 2009; Leander 2008), online ‘lives’ are not distinct from offline worlds. Through the process of creating avatars (through textual, image, video manipulation) the girls were able to try-on and try-out identities both literally, by creating and dressing up a character, and by adopting various ‘stances’ (Bucholtz 2009) in the process of digital creation, transformation and interaction. For example, both character names and status updates utilised words like ‘sexy’ and ‘love’, reflecting their age group's entrance into the heterosexual marketplace (Eckert 1994) and directly reflecting the discourse evidenced in their daily in-person interactions. However, such identity work was evoked by a virtual world consisting of millions of individuals influenced by popular culture discourses operating within software built by specific, likely male, designers and programmers with financial motivations (Gee and Hayes 2010). The avatars reveal the complex distributed nature of identities, enacted not only through technologies but also by them. Such practices address broader digital literacies and participation skills, indicating the importance of understanding and drawing on the non-curricular digital media practices of students.

Concluding remarks

Our discussion of the utility and affordances of this methodology for studying digital literacy events emerges from the notion of sociomateriality in education (Fenwick, Edwards, and Sawchuk 2011) and the interpenetration of humans and technical artefacts in digital literacies. Literacy practices and the technologies used for them are each given purpose as a result of, and via, the other. A framework to explore the nature of, theorise, and talk about one must therefore encompass the other as research explicates the relationships between digital technologies and literacy events.

Sociomateriality as a theoretical sensibility, places ‘equal and mutual emphasis of both social and material considerations’ (Johri 2011, p. 214). Our approach has revealed that a variety of agencies interfere when digital tools are used in traditional classrooms, transforming or displacing quotidian institutional practices. Search engines and virtual world environments, for example, exert force: they limit or expand the scope of a writer's choices. Sara, in study 1, was heavily influenced by choices thrown up at her by Google's search engine algorithms, which are based on a continuously developing artificial intelligence system. Her written work was subsequently shaped by these interactions. In study 2, the girls’ enactment of literacy and identity was transformed when laptops facilitated their peer network's entanglement with a pre-existing assemblage of other adolescents and software developers. Our analyses are preliminary and offered as specific examples of the kind of broader investigations that could derive from such connective methodologies. Echoing Gourlay and Oliver's (2013) call for a sociomaterial approach, our findings undermine a monolithic and taxonomic conception of digital literacies, where they are made up of discrete and measurable skills or even individual practices. The implications for digital literacy research are that literacy events should be treated as assemblages, requiring disentanglement and reassembly using appropriate methodological tools and techniques.

In our efforts to examine digital literacy events, we required a methodology to account for the sociomaterial complexity of classroom writing in real time. In our endeavour to do this, we developed a transdisciplinary approach that can be used to explore other forms of work in digital environments. An understanding of materiality as passive elements did not help us explain digital literacy events sufficiently. The multilayered and messy entanglement of social and material actors, and their contesting realities as student ‘work’ or ‘play’ unfolds, is well captured through a connective methodology which brings together networks that are disparate conceptually but not in practice.

As we are set to face an ever more digitised educational future (Johnson et al. 2013), the methodology proposed here is useful for researchers paying greater attention to the sociomaterial assemblages in which activity, including learning and literacy, occurs (Gourlay and Oliver 2013; Johri 2011; Sørensen 2010; Waltz 2006). We are eager to see how this methodology will be applied across a range of research sites and disciplines, by researchers committed to advancing the fields of learning technology and literacy studies empirically and methodologically, and how it will contribute to the continuing discussion (e.g. Luff and Heath 2012) around the unique challenges of using video-based data in qualitative research.

Acknowledgements

We are grateful to the reviewers for their comments on an earlier draft of this paper, Chris Bigum for his advice, the students and teachers who allowed us access to their classrooms, and members of the actor-network theory Facebook group for their discussions.

Study 1 discussed in this paper is funded through an ESRC WR DTC studentship.

Notes

Computer Assisted Qualitative Data Analysis Software.

According to Law (2009, p. 146), there is little difference between the term assemblage (i.e. agencement) and actor-network.

References Asselin M Moayeri M New tools for new literacies research: an exploration of usability testing software International Journal of Research & Method in Education 2010 33 1 41 53 Barad K Posthumanist performativity: toward an understanding of how matter comes to matter Signs 2003 28 3 801 831 Barad K. M Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning 2007 Durham, NC Duke University Press Barton D Hamilton M Barton D Hamilton M Ivanic R Literacy practicesSituated literacies: reading and writing in context 2000 London Routledge 7 15 Bawden D Lankshear C Knobel M Origins and concepts of digital literacy Digital Literacies: Concepts, Policies and Practices 2008 Oxford Peter Lang 17 32 Baynham M Prinsloo M Baynham M Prinsloo M Introduction The Future of Literacy Studies 2009 Basingstoke Palgrave Macmillan 1 20 Besnier N Literacy, Emotion, and Authority: Reading and Writing on a Polynesian Atoll 1995 Cambridge Cambridge University Press Bezemer J Mavers D Multimodal transcription as academic practice: a social semiotic perspective International Journal of Social Research Methodology 2011 14 3 191 206 Bhatt I Digital literacy practices and their layered multiplicity Educational Media International 2012 49 4 289 301 Bigum C. J Gilding A A video monitoring technique for investigating computer-based learning programs Computers & Education 1985 9 2 95 99 Brandt D Clinton K Limits of the local: expanding perspectives on literacy as a social practice Journal of Literacy Research 2002 34 3 337 356 Bucholtz M Jaffe A. M From stance to style: gender, interaction, and indexicality in Mexican immigrant youth slang Stance: Sociolinguistic Perspectives 2009 Oxford Oxford University Press 146 170 Callon M Callon M Law J Rip A The sociology of an actor-network Mapping the Dynamics of Science and Technology: Sociology of Science in the Real World 1986 Basingstoke Macmillan 19 34 Caperton I. H Toward a theory of game-media literacy: playing and building as reading and writing International Journal of Gaming and Computer-Mediated Simulations (IJGCMS) 2010 2 1 1 16 Chase Z Laufenberg D Embracing the squishiness of digital literacy Journal of Adolescent & Adult Literacy 2011 54 7 535 537 Deleuze G Guattari F Mille plateaux: capitalisme et schizophrenie 1980 Paris Les Editions de Minuit Deppermann A Multimodal interaction from a conversation analytic perspective Journal of Pragmatics 2013 46 1 1 7 de Roock R Digital Literacies as Interactional Achievements: A Multimodal Approach to Understanding Learning with New Digital Media Unpublished Doctoral Dissertation, Tucson, AZ. forthcoming Eckert P Entering the Heterosexual Marketplace: Identities of Subordination as a Developmental Imperative 1994 Working Papers on Learning and Identity, No 2, Institute for Research on Learning, Palo Alto, CA. Edwards A Westgate D Investigating Classroom Talk 1994 London Falmer Press Ericsson K. A Simon H. A Protocol Analysis: Verbal Reports as Data 1993 Cambridge, MA MIT Press Fenwick T. J Edwards R Sawchuk P Emerging Approaches to Educational Research: Tracing the Sociomaterial 2011 London Routledge Fenwick T. J Edwards R. J Actor-Network Theory in Education 2010 London Routledge Fields D Kafai Y A connective ethnography of peer knowledge sharing and diffusion in a tween virtual world International Journal of Computer-Supported Collaborative Learning 2009 4 1 47 68 Garcez P. M Helping Philippe: constructions of a computer-assisted language learning environment Working Papers in Educational Linguistics 1995 11 2 39 66 Garcez P. M Bulla G. S Loder L. L Cavalcanti M. C Zanotto M. S Práticas de pesquisa microetnográfica: geração, segmentação e transcrição de dados audiovisuais como procedimentos analíticos plenos Trajetórias de pesquisa em Linguística Aplicada Editora da UFMG, Belo Horizonte forthcoming Garfinkel H Studies in Ethnomethodology 1967 Englewood Cliffs, NJ Prentice-Hall Gee J. P Social Linguistics and Literacies: Ideology in Discourses 1996 London Taylor & Francis Gee J. P Barton D Hamilton M Ivanic R The new literacy studies: from “socially situated” to the work of the social Situated Literacies: Reading and Writing in Context 2000 London Routledge 180 196 Gee J. P Hayes E Women and Gaming: The Sims and 21st Century Learning 2010 New York Palgrave Macmillan Geisler C Slattery S McKee H. A DeVoss D. N. N Capturing the activity of digital writing: using, analyzing, and supplementing video screen captureDigital Writing Research: Technologies, Methodologies, and Ethical Issues 2007 Hampton Press, Cresskill, NJ 185 200 Gillen J Barton D Digital Literacies, A Research Briefing by the Technology Enhanced Learning Phase of the Teaching and Learning Research Programme (TLRP) 2010 JISC, London London Knowledge Lab Gilster P Digital Literacy 1997 Chichester John Wiley Goodwin C Professional vision American Anthropologist 1994 96 3 606 633 Goodwin C Leeuwen T. V Jewitt C Practices of seeing, visual analysis: an ethnomethodological approachThe Handbook of Visual Analysis 2000 London Sage 157 182 Gourlay L Oliver M Goodfellow R Lea M ‘Beyond “the social”: digital literacies as sociomaterial practice’ Literacy in the Digital University: Critical Perspectives on Learning, Scholarship & Technology 2013 London Routledge 79 94 Greatbatch D How do desk-top computers affect the doctor–patient interaction Family Practice 1995 12 1 32 36 Haraway D. J Modest-Witness@Second-Millennium. FemaleMan-Meets-OncoMouse: Feminism and Technoscience 1997 New York Routledge Hayes J. R Flower L. S Gregg L. W ‘Identifying the organization of writing processes’ Cognitive Processes in Writing 1980 Hillsdale, NJ Lawrence Erlbaum 3 30 Hayles K How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics 1999 Chicago, IL University of Chicago Press Hayles K How we Think: Digital Media and Contemporary Technogenesis 2012 The University of Chicago Press, London Heath C Body Movement and Speech in Medical Interaction 1986 Cambridge Cambridge University Press Heath C Hindmarsh J Luff P Video in Qualitative Research: Analysing Social Interaction in Everyday Life 2010 CA Sage, Los Angeles Heath C Knoblauch H Luff P Technology and social interaction: the emergence of “workplace studies” The British Journal of Sociology 2000 51 2 299 320 Heath C Luff P Technology in Action 2000 Cambridge Cambridge University Press Heath C Configuring awareness Computer Supported Cooperative Work (CSCW) 2002 11 3–4 317 347 Heath S. B What no bedtime story means: narrative skills at home and school Language in Society 1982 11 1 49 76 Heath S. B Ways with Words: Language, Life, and Work in Communities and Classrooms 1983 Cambridge Cambridge University Press Hindmarsh J Object-focused interaction in collaborative virtual environments ACM Transactions on Computer–Human Interaction (TOCHI) 2000 7 4 477 509 Hutchins E Cognition in the Wild 1995 Cambridge, MA MIT Press Jacobs J. K Kawanaka T Stigler J. W Integrating qualitative and quantitative approaches to the analysis of video data on classroom teaching International Journal of Educational Research 1999 31 8 717 724 Jewitt C Multimodality and literacy in school classrooms Review of Research in Education 2008 32 1 241 267 Jewitt C The Routledge Handbook of Multimodal Analysis 2009 London Routledge Johnson L Adams Becker S Cummins M Estrada V Freeman A Ludgate H The NMC Horizon Report: 2013 Higher Education Edition 2013 Austin, Texas The New Media Consortium Johri A The socio-materiality of learning practices and implications for the field of learning technology Research in Learning Technology 2011 19 3 207 217 Knoblauch H Introduction to the special issue of qualitative research: video-analysis and videography Qualitative Research 2012 12 3 251 254 Kress G. R Literacy in the New Media Age 2003 London Routledge Lankshear C Knobel M Digital Literacies: Concepts, Policies and Practices 2008 Oxford Peter Lang Latour B Science in Action: How to Follow Scientists and Engineers Through Society 1987 Milton Keynes Open University Press Latour B Pandora's Hope: Essays on the Reality of Science Studies 1999 London Harvard University Press Latour B Harrison S Pile S Thrift N. J Nonhumans Patterned Ground: Entanglements of Nature and Culture 2004 London Reaktion 224 227 Latour B Reassembling the Social: An Introduction to Actor-Network-Theory 2005 Oxford Oxford University Press Latour B Woolgar S Laboratory Life: The Construction of Scientific Facts 1986 Princeton, NJ Princeton University Press Law J After Method: Mess in Social Science Research 2004 London Routledge Law J Turner B. S Actor network theory and material semioticsThe New Blackwell Companion to Social Theory 2009 Oxford Wiley-Blackwell 141 158 Law J Rubio F. D Baert P Collateral realitiesThe Politics of Knowledge 2012 London Routledge 156 178 Leander K Coiro J Knobel M Lankshear C Leu D Toward a connective ethnography of online/offline literacy networks Handbook of Research on New Literacies 2008 New York Lawrence Erlbaum Associates/Taylor & Francis Group 33 65 Leeuwen T. V Jewitt C The Handbook of Visual Analysis 2000 London Sage Lemke J. L Discursive technologies and the social organization of meaning Folia Linguistica 2002 35 1–2 79 96 Lewins A Silver C Using Software in Qualitative Research: A Step-by-Step Guide 2007 London Sage Luff P Heath C Some “technical challenges” of video analysis: social actions, objects, material realities and the problems of perspective Qualitative Research 2012 12 3 255 279 Luff P Hindmarsh J Heath C Workplace Studies: Recovering Work Practice and Informing System Design 2000 Cambridge Cambridge University Press Martin A Grudziecki J DigEuLit: concepts and tools for digital literacy development Italics 2007 5 4 249 267 McLarty M. M Gibson J. W Using video technology in emancipatory research European Journal of Special Needs Education 2000 15 2 138 148 Meredith J Stokoe E Repair: comparing Facebook “chat” with spoken interaction Paper Submitted to Discourse & Communication vol. no forthcoming Miller K. S Lindgren E Sullivan K. P. H The psycholinguistic dimension in second language writing: opportunities for research and pedagogy using computer keystroke logging TESOL Quarterly 2008 42 3 433 454 Mitchell W. J. T Picture Theory: Essays on Verbal and Visual Representation 1994 Chicago, IL University of Chicago Press Mol A The Body Multiple: Ontology in Medical Practice 2003 Durham, NC Duke University Press Nicholls A. V The Social Life of the Computer in Ramingining 2009 Charles Darwin University PhD Thesis Orlikowski W. J Sociomaterial practices: exploring technology at work Organization Studies 2007 28 9 1435 1448 Orlikowski W. J The sociomateriality of organisational life: considering technology in management research Cambridge Journal of Economics 2010 34 1 125 141 Pasquandrea S Managing multiple actions through multimodality: doctors’ involvement in interpreter-mediated interactions Language in Society 2011 40 04 455 481 Prinsloo M Baynham M Prinsloo M Baynham M Introduction: renewing literacy studiesLiteracies, Global and Local 2008 Amsterdam John Benjamins 1 16 Psathas G Conversation Analysis: The Study of Talk-in-Interaction 1995 Thousand Oaks, CA Sage Scribner S Cole M The Psychology of Literacy 1981 Cambridge, MA Harvard University Press Sørensen E The Materiality of Learning: Technology and Knowledge in Educational Practice 2010 Cambridge Cambridge University Press Street B. V Literacy in Theory and Practice 1984 Cambridge Cambridge University Press Street B. V Literacy and Development: Ethnographic Perspectives 2001 London Routledge Suchman L. A Plans and Situated Actions: The Problem of Human-Machine Communication 1987 Cambridge Cambridge University Press Van Hout T Macgilchrist F Framing the news: an ethnographic view of financial newswriting Text & Talk 2010 30 2 147 169 Van Hout T Pander Maat H De Preter W Writing from news sources: the case of Apple TV Journal of Pragmatics 2011 43 7 1876 1889 Waltz S. B Nonhumans unbound: actor-network theory and the reconsideration of “things” in educational foundations Educational Foundations 2006 20 3 51 68 Woods D. KDempster P. GTales From the Bleeding Edge: The Qualitative Analysis of Complex Video Data Using Transana2011ForumQualitative Social Researchvol. 12, no. 1, Available at: http://www.qualitative-research.net/index.php/fqs/article/view/1516/3120 [Accessed 15 November 2013] Woulds SLanguage, Literacy and Technology: An Analysis of Multimodal Interaction in an ESOL ICT ClassroomPhD Thesis (draft), University of Leedsforthcoming