Figure 1. The Our Story main user interface.
ORIGINAL RESEARCH ARTICLE
Natalia Kucirkovaa* and Garry Falloonb
aInstitute of Education, University College London, London, UK; bFaculty of Human Sciences, Department of Educational Studies, Macquarie University, Sydney, NSW, Australia
(Received: 2 November 2017; final version received: 21 May 2018; Published: 19 June 2018)
This article suggests new ways of working with visual data collected with or via iPads. Using the example of two iPad apps that we co-created, we argue that multimedia and display recorder apps can generate highly authentic data, capable of providing unique insights into the activities and experiences of young children that more conventional data methods cannot achieve. We discuss and illustrate how the use of the apps addresses some empirical and ethical challenges concerning the positioning of the child and researcher in observational research, notably in relation to observer effects and researcher subjectivity. We outline some principles and strategies for researchers interested in using iPad apps and address some challenges and use considerations of these innovative methods.
Keywords: iPads; visual methods; data quality; observer effects; researcher subjectivity
*Corresponding author. Email: n.kucirkova@ucl.ac.uk
Research in Learning Technology 2018. @ 2018 N. Kucirkova and G. Falloon. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2018, 26: 2029 - http://dx.doi.org/10.25304/rlt.v26.2029
This article details our experience of undertaking empirical research in which iPad-generated visual data formed a principal source of evidence for children’s engagement in classroom activities and of their learning processes. Increasing numbers of young children are exposed to iPads and comparable tablets on a daily basis. Observational studies (e.g. Hutchison, Beschorner, and Schmidt-Crawford 2012) and experimental studies (e.g. Outhwaite, Gulliford, and Pitchford 2017) document children’s engagement with these tools, particularly focusing on the potential of tablets for nurturing children’s literacy learning and scaffolding conceptual and procedural knowledge in science and computational thinking. What has been less explored is the potential of iPads to serve as a data collection tool, which can render visible children’s views and feelings and reveal information about strategies they use to solve learning problems.
We outline the affordances of two apps: Our Story used in a literacy context and Shou Recorder used in a science context. Using these apps, we collected children’s photographs and videos that they took with iPads, videos and photographs of children using iPads, as well as video display captures of their interactions with apps, and integrated them into coherent research stories. The integration of data and its interrogation yielded a rich account of children’s views, experiences and capabilities beyond the level of comparable visual methods. We suggest that specific iPad apps can significantly contribute to the authenticity of data in a research project but point out significant empirical and ethical challenges in achieving this quality.
We detail two case studies illustrating the use of apps for capturing highly authentic data that reveal unique insights into young children’s experiences and learning processes in literacy and science. The first study explored the app ‘Our Story’ in a project evaluating the Helicopter technique in UK classrooms (Cremin 1 et al. 2013; Cremin, Flewitt, Mardell & Swann, 2016). The second study combined the display recorder app and the virtual manipulative (VM) science app, ‘Electronics for Kids’, in an investigation of children’s learning of simple electrical circuit concepts. We argue that the combination of media within the ‘Our Story’ app and the possibility of personalising narratives are major assets for gathering young children’s accounts. We also suggest that using a display recorder in the science study captured young children’s manipulations, revealing detailed information about the efficacy of VMs for building basic science concepts.
We chose these two examples to discuss ways in which the visual recording features of iPad apps can contribute to data authenticity. The illustrations demonstrate how the capabilities of the device offer considerable flexibility in determining what data can be gathered, who can gather it, and when and where it can be gathered. We begin our argument with a definition of data authenticity and the key parameters of high-quality data. We then consider iPads as specific data-collection tools and their key functionality for gathering authentic data. The features of the two apps are described in detail and in relation to their capabilities for maximising the collection of authentic data. We illustrate what authentic data ‘looked like’ and discuss challenges involved in collecting these within our studies.
Recent moves towards more flexible learning environments supported by an increasing array of mobile technologies – both in formal and informal learning contexts – present significant challenges for researchers wishing to examine m-Learning, particularly the processes that learners engage in when using mobile devices. Crompton, Burke, and Gregory (2013) defined m-Learning as ‘learning across multiple contexts, through social and content interactions, using personal electronic devices’ (p. 4). Their recent 2017 synthesis of 113 studies on m-Learning in pre-K–12 contexts, revealed a predominance of pre- and post-test, questionnaire and quasi-experimental methods used in studies designed to evaluate the ‘impact’ or ‘effect’ of a specific, mobile device–supported intervention. While the majority of reviewed studies revealed positive outcomes (62%) or were not designed to evaluate these (34%), they commented that new research methods are needed to ‘provide a more refined look at intervening variables that better explain how positive outcomes occurred’ (p. 56). The reliance on self-report data, and limitations to the extent to which pre- and post-test and quasi-experimental methods provide robust information valuable to informing teaching and learning practices with mobile devices (as opposed to evaluating the ‘performance’ of the device), were viewed as limitations to the current literature in the field. Interestingly, of the 113 studies reviewed, only two used visual data from single or multipoint video.
The need for less subjective and more robust methods in m-Learning research was also expressed by Garcia-Cabot, de-Marcos, and Garcia-Lopez (2015), who recommended using mobile, not fixed, technologies to capture contextual data and information about learners. While many researchers might identify with these themes, they face significant research difficulties in gathering data of high quality and authenticity. The variable nature of m-Learning environments, where learners move and interact with learning materials and each other using their devices from anywhere at any time, challenges conventional, more static methods of data gathering. In school environments, this is amplified by the current trend towards large, open and flexible learning environments, where students can opt to work in different spaces and collaborate with multiple others as they complete learning tasks (Author 2 2015). Rapidly changing dynamics in these environments necessitate the use of methods that can capture high quality data about students’ learning interactions in multiple scenarios. These considerations bring us to the basic question of what constitutes good data.
Data quality needs to be defined in relation to the three processes involved in data handling: data collection, data analysis and data interpretation. In this article we focus on data collection, as that is the first point of contact between researchers and participants. In educational contexts, different data-gathering techniques require different quality controls. Observational research facilitated by technologies can be either structured and systematic or less structured and informal (Foster 1996). To achieve high-quality data in observational educational research, researchers need to minimise subjectivity and take steps to reduce researcher bias. Typically, this is achieved by following conceptualisations that are grounded in sound theoretical arguments and empirical knowledge. However, the practical steps taken by researchers in the pursuit of this goal can vary from study to study. For example, in their studies of children’s use of technology at home and in preschools, Plowman, Stephen, and McPake (2010) collected 16 hours of video data in eight different settings. The authors coded the videos with broad categories relevant for their framework of guided interaction (e.g. presence of adult, interaction between adult and child) and used selected video vignettes to prompt reflection among the participants. In the context of adult–child reading sessions at home, Nicholas (2016) avoided subjectivity with multiple video recordings supplemented with a range of complementary data sources.
In our work, we were keen to ensure that we would reflect as accurately as possible the nature of children’s experiences and capabilities in classrooms. The goal of this endeavour was conceptualised in terms of data authenticity.
A well-known computer scientist, Clifford Lynch, defines authenticity as ‘verifying claims that are associated with an object – in effect, verifying that an object is indeed what it claims to be, or what it is claimed to be’ (2000, p. 37). Lynch’s definition highlights the importance of trustworthiness, in ensuring that what is recorded and used as data accurately represents the nature of activities in any given research context. Although we focus on technology-mediated data collection processes, we use authenticity in terms of originality, integrity and trustworthiness of data. Our research participants are young children, so the question of trustworthiness of data from observations needs to be considered from the children’s and the researchers’ perspectives. From the researcher’s perspective, we consider two empirical biases: observer effects and researcher’s subjectivity. From the children’s perspective, we consider authenticity in terms of the ethical imperative for researchers to represent as accurately as possible children’s capabilities and experiences. For both perspectives, the specific features of technology used for visual data collection are important. These features need to be considered in terms of the device (technology) as well as its specific program (application) attributes. We first consider the attributes of the iPad, followed by the attributes of two specific iPad applications.
iPads are portable, and children’s engagement with them can be highly mobile, especially if it involves apps that require external input (e.g. taking a picture with the camera), collaboration (e.g. use of the same app by different people, or sharing knowledge about using an app with others) and physical movement (e.g. prompts for interactivity or moving with the device to work in different spaces).
iPads are compact in that they integrate several technologies into one, including an audio recorder, high-quality photo and video cameras, a keyboard, access to the Internet and a variety of specially designed software programs (apps). The ubiquity of iPads in households (e.g. Kabali et al. 2015) and in school settings (e.g. Culén and Gasparini 2011) makes them ideal candidates for gathering visual data. iPads can be used instead of conventional cameras, or in addition to these, and they can be used to record data from ‘outside’ or ‘inside’ the device.
To collect visual data from the outside, researchers can use the native video recording app that comes with the iPad. This allows them to have the camera either inward- or outward- facing, capturing visuals of the environment or of the user. The iPad’s still and video cameras offer a convenient means for both children and researchers to capture noteworthy learning events valuable for research purposes. In addition, researchers can collect video data from within the device, using a display capture app, which can provide useful information about children’s interaction with different app content and features.
In this article, we focus on two specific apps that record visual data both from inside and outside. We led the development of the two apps at our separate institutions, and in the section ‘App considerations’ we describe these two apps in detail, along with the rationale for their development.
Since the late 1930s when the concept of reactivity or the ‘observer effect’ emerged in the work of Jersild and Meigs (1939), research has been concerned about the influence of direct observation on the behaviours of research participants, particularly in the field of education (e.g. Ary, Jacobs, and Sorenson 2010; La Donna et al. 2017; Maisling and Stern 1969). While live video recording in classrooms has been shown to be very effective for collecting data that ‘helps avoid subjective judgement and increases feedback sources’ (Liang 2015), some still question its authenticity, claiming that those being observed may be ‘staging a performance’ (La Donna et al. 2017, p. 498) that does not reflect accurately their normal behaviours.
Earlier research carried out by Author 2 indicated this to be the case in his work with very young children (5-year-olds), investigating their use of different apps for numeracy and literacy development and learning collaboration (Author 2 2013, 2014). Conventional ‘over the shoulder’ video recordings revealed strong evidence that the children’s behaviours were substantially influenced by researcher presence, with recorded audio suggesting children tailored these according to what they thought the observer desired or was expecting. The earlier studies also revealed challenges to gathering whole class data when conventional video recording techniques were used. It was difficult to record more than one pair or group at a time – and the children needed to remain in the same place. Working with 5-year-olds meant achieving the latter was virtually impossible, and trying to follow them with a camera was equally problematic. A data-gathering solution that could be installed on each device was needed to address these issues.
The recording app used in this work evolved from early trials of a shareware app called Display Recorder and was a pragmatic response to the need for a mobile data capture tool able to operate on iPads, which were quickly becoming the ‘device of choice’ for schools in New Zealand. Constraints imposed by the locked design of Apple’s iOS meant that no off-the-shelf apps were available, necessitating a purpose-designed solution.
Author 2 (2013, 2015) described in detail use of the Shou Recorder app, which records the user’s interactions while they are engaged with other apps on the device. The recorder app ‘runs in the background’ and, according to how it is set up, can capture user finger placements, a video of the display and front-facing information from the device’s facecam, and microphone or device audio. All of these are merged into single MOV or MP4 files that are stored on each device and can be exported for later analysis. The advantages of using a device-based system as opposed to other recording methods, such as apps that mirror to a laptop or require remote server addressing, is that no Internet connection is required, and recordings can be made on multiple devices at the same time. Countering this are limitations in the internal storage capacity of iPads and the data-intensive nature of video recordings. However, using high compression, the display recorder app is able to capture about an hour of HD video per 800 MB of available storage capacity. This can be further increased by reducing video quality or size.
The Our Story app was developed at The Open University, UK to enable the authorship and sharing of children’s stories on smartphones and tablets. The app has two modes: Create and Share. The Create mode comprises two parts: a gallery of pictures and a storyboard. For each picture in the gallery, users can add text involving words, sentences or whole paragraphs, and/or add their own recorded sound. The storyboard (filmstrip at the bottom of the gallery of pictures) enables users to put their pictures together in sequence, making up a story with a beginning and an end. The app requires no previous proficiency in picture editing or screen writing; it is intuitive to use and was especially developed for children. With a single finger tap, users can add their own pictures or recordings, or modify the text as they wish, and as such create a unique story, adjustable to the competence, knowledge and interest levels of each child. Figure 1 shows the app’s child-friendly user interface.
Figure 1. The Our Story main user interface.
The key asset of Our Story relevant for data authenticity is its open-ended design, where users can add their own multimedia content. There are no templates or scripts restricting users’ input, allowing for highly original, personal or personalised content. Users’ stories can be in text, visual or audio format or a combination of all three, arranged in a sequence (narrative). These assets have been found useful by several researchers who have used the app in their own research projects (e.g. Canning et al. 2017; Sung and Siraj-Blatchford 2015).
In a case study with 16 preschoolers and their childminders and day nursery practitioners, Canning et al. (2017) used Our Story for documenting children’s curiosity and imagination. Canning et al. (2017) argued that the app’s unique affordances helped children with ‘re-storying’ (i.e. reviewing and editing stories), ‘connecting their various experiences from home, popular culture and other learning spaces’ and ‘anchoring experiences across domains’ (ibid, p. 305). The researchers concluded that the combination of multimedia woven into a narrative facilitated these outcomes: ‘The findings show that photographs or video on their own may not have immediate obvious significance, but when woven into a narrative, accompanied by text or speech, children’s interests, their imaginative capability and endeavours become apparent’ (ibid, p. 305). In this article, we consider how Our Story’s features contribute to the authenticity of data.
The first study details and discusses data taken from the Helicopter Stories project, in which the Our Story app was used to get an insight into children’s views and feelings of a story-acting/storytelling intervention. The intervention was delivered by the theatre and education charity MakeBelieve Arts, and its impact was documented in several academic publications (e.g. Author 1 et al. 2016, 2017). Data collection occurred over eight weeks in four different settings and six classrooms in an inner London borough and the south of England. There were three visits to each classroom where data were collected from three case study children, selected by their teachers. Our Story was used with these children at the last (end-of-programme) visit, at which we aimed to gather their views and reflections on the storytelling and story acting (using Helicopter Stories) delivered by the MakeBelieve Arts charity. The study was approved at The Open University, UK and followed the ethical guidelines of the British Educational Research Association (BERA 2014).
Except for one case study child who was absent at our last visit, all case study children were approached at the ‘review visit’ and were asked about their stories and experiences of the Helicopter technique. The researcher showed children a selection of pictures from the story-acting session on the iPad and prompted the conversations with an informal question along the lines of: ‘I have a few pictures of the story you told today but I forgot what it was about; would you like to tell me?’ The conversations were video recorded either by fellow researcher or by a stand-alone camera. The children were free to use the app in any way they wanted.
The children chose to reconstruct and represent their individual stories in different ways, and the app supported a variety of approaches. While some children were keen on re-constructing the story they told the teacher in the morning, other children used the retelling session as an opportunity to create a new multimodal story. For both processes, children realised the potential of the app to different extents. All children took delight in exploring and choosing the pictures that represented their own stories. In addition to the pictures preselected by the researcher, the children were keen on selecting different pictures from the app’s photo folder. The children were particularly keen on finding pictures that displayed their stories.
Some children relied on the pictures to retell their stories, some children didn’t need any pictorial cues to re-produce the story they had told the teacher, and some children made new stories during the reminiscing with the researcher. So, some children took great effort to re-construct their original stories and some children preferred to make new ones, using the iPad camera to take new pictures and insert them in their stories. In this respect, the children’s choice to create a new story (rather than re-construct their original) suggests that they made different investments in their original storytelling. It also showed that the app’s intuitive and independent way of manipulation allowed children to take ownership and control of the reminiscing session and to unlock their creativity. Consequently, there was great variability in how children structured their stories and recollections. Some seemed preoccupied by ‘getting the story right’ and spent considerable time finding the right pictures and putting them in the order of their previously narrated story. Here, the filmstrip at the bottom of the app’s main menu assisted them to position their experiences in chronological order. Other children were motivated to take new pictures with the camera, and the app’s open-endedness and possibility for customisation may have stimulated actions and behaviours that went beyond the expectations of a traditional review activity.
During picture viewing, we also gained some insights into children’s views on aspects of story acting. For example, when looking at a picture from acting out her story, one case study child (CS1) commented that she was unsure of why case study child 2 (CS2) performed his story in a particular way:
Researcher: What’s [CS2] doing? [CS2 was featured in the picture]
CS1: Stretching.
Researcher (surprised): Is that what happened in your story?
CS1: Yeah, I didn’t know what he was doing at all.
In contrast, CS3 accompanied a picture from her story-acting session with a big smile and a comment: ‘those are all my little babies’. When prompted to describe what a particular boy was doing in another picture from her story, CS3 said:
CS3: Trying to catch all the babies.
Researcher: Why did he have his arms open?
CS3: He’s trying to fly like [a] bird.
Together, the children’s choice of pictures and comments showed that they had different perceptions of their friends’ roles during story acting. Further glimpses into the children’s perceptions of roles and abstract thinking was provided by their use of the app’s audio and text-annotation function. Most children used the text feature to describe the characters represented in the pictures. This was in most cases reduced to children’s own names and their friends’ names, but there were also instances of children willing to write simple story segments (e.g. ‘mummy feeded the cat’). Children were mostly skilled at finding and typing individual letters but asked for the researcher’s help when it came to more complicated words and letters they were unfamiliar with. One child pretended to write the story on her own and ‘read’ the finished story to the researcher. This child seemed to enjoy the unrestricted text box, accommodating long strings of ‘non-words’.
The audio recording feature of the app was also popular and was exploited to different degrees in children’s story retellings. While some children used the recorder to record parts or entire stories they had told the teacher, other children used it to enrich the story they told in the morning. For example, CS4 found a picture of a ‘big giraffe’, which was part of his original story, and accompanied this with a recording of the giraffe’s munching sound. Another child, CS5, was eager to add a scary animal sound to his pictures.
Interestingly, when children chose to audio record their entire stories, they followed two different strategies. On one hand, some children (CS1 and CS5) recorded parts of their stories, matching individual pictures showing particular segments of their stories. Other children (CS2 and CS3) recorded their entire stories in one go and in relation to only one picture. Surprisingly, one child (CS6) identified the recording feature as a means of creating a message for an unknown audience. This girl recorded a piece we hadn’t observed her telling before. Her recording started with, ‘Hello, my sister is coming to my house today. And I can’t wait. Today I really am excited. And it was my sister’s birthday’.
Importantly, we also found that two children (CS1 and CS2) found the audio recording feature intimidating, or perhaps daunting, and took a long time to bring themselves to record their voices. However, once they had recorded and listened to their voices together with the researcher, they were eager to record more sounds and independently used the microphone and audio play buttons. The possibility to record sounds without restrictions meant that children with limited vocabularies could re-construct, capture and act out their original stories in one easy process. For CS5, whose entire story was about a dinosaur who makes an arrrh sound, recording the arrh sound and playing it back to the researcher was a powerful moment of story sharing. In sum, the Our Story app accommodated children’s various communication and storytelling strategies and enabled us to gain insight into the diversity and richness of their stories.
This second study details and discusses data from an investigation involving 44 five-year-olds’ use of three VM apps and explores whether the apps could assist the children to learn simple electrical circuit-building techniques and concepts. The study was undertaken in a Year 1–6 primary school located in a provincial town near Hamilton, New Zealand. The new entrant (Year 1) class was selected following past successful studies involving the lead teacher from 2012 to 2016. The children had been at school for less than 3 months at the commencement of the research. Standard informed parental consent and student assent processes were followed, and research ethics approval was gained from the researcher’s university. This discussion will concentrate on the use of one circuit-building app, ‘Electronics for Kids’, as illustrative of how the display recorder enabled authentic data to be collected that provided unique insights into the children’s emerging knowledge of circuit components, construction procedures and concepts of electrical current.
Electronics for Kids is a VM-based app developed by Koto Games. It comprises a range of templated challenges where children select and ‘drag and drop’ components to build circuits, starting with a single-bulb uncontrolled design, working through to uncontrolled and controlled series and parallel circuits (Figure 2). Some circuits also include variable and fixed resistors.
Figure 2. The home screen of ‘Electronics for Kids’.
The children worked in pairs, each pair using an iPad Air that had installed on it the display and audio recorder app. A few pairs whose consent had been obtained also activated the facecam recorder (Figure 3). The classes were divided into two groups – each of approximately 22 children – with data being recorded during two sessions per group (each of about 35 min). The pairs remained constant across both recording sessions, and a total of just over 12 h of display data were gathered from them. Data were subsequently analysed using StudioCode video analysis software Vosaic Software, Lincoln, NE, USA. for any evidence of conceptual knowledge development, using a framework generated from much earlier research using physical manipulatives (Osborne 1983; Shipstone 1984). Reflecting on this procedure, using display capture yielded data of high trustworthiness and authenticity, as it not only mitigated the observer effects noted in earlier studies but also supported data collection from a large cohort of children working under the same conditions, at the same time. In this sense, data collected using this method held high internal validity.
Figure 3. Sample screen capture showing facecam.
The analysed recordings suggested the children either forgot the recorder was operating, or if they did remember it made no difference to their behaviours or performance. No visual or verbal evidence was found indicating children ‘staged a performance’ as in the earlier studies. Considering these children had only been at school a few months and this was the first time they had been involved in research or with the researcher, the high level of trust established in past studies through the researcher spending considerable time in the children’s classrooms had yet to be established. In past studies, the trusting relationship established between the children and the researcher ‘naturalised’ the research environment, arguably enhancing data authenticity. However, in this study it was not the case, yet data were of comparable authenticity, suggesting the data method played a significant role. Although some children who activated the facecam would have had a visible reminder of the recorder in operation (Figure 3), apart from occasionally moving the small image to get it away from their work, no evidence was found of the system interfering or in any way affecting their behaviours.
In the next section, we reflect on our empirical and design examples in relation to two key benefits of obtaining authentic data: minimising observer effects and researcher subjectivity.
In terms of display data authentically representing children’s interactions with each other and the apps in the research context, the ability to capture information from all children at the same time, irrespective of their location in the classroom and the stage they were at in their work, provided a holistic account of children’s interactions with the apps. The detail and accuracy of data gathered using display capture is arguably superior to that enabled by conventional means and includes information on finger placement and menu, tool and option selection, supported by clear audio of conversations and facial images. All together, these data points provide rich and authentic multimodal data, unaffected by audio difficulties associated with videoing from a distance.
Data yielded by the display recorder lessens the likelihood of misinterpreting events, improving the accuracy of coding decisions and enhancing inter-rater reliability. iPad recording apps can facilitate authentic data collection with minimal observer effects at a relative low cost. Recorded data can provide useful illustrations of students’ learning task engagement and the knowledge and strategies they use and develop, which can be easily shared with teachers and the children’s parents. The system also reduces self-report bias and facilitates rapid data collection of individual or pairs of users on a large scale, no matter where they are physically located. This is especially useful in environments where the devices are deployed across several classrooms, or where children are free to move to different learning spaces, including outside the classroom. These affordances are noteworthy for the study of several learning processes, including reading activities (Miller and Warschauer 2014) and situations where children are engaged in activities at learning stations or in groups (Author 2 2017). Such methods are essential for minimising observer effects in contemporary, digitally supported scenarios where opportunities to learn are becoming more mobile, and there is a dual emphasis on collecting ‘real time’ data from both personalised and networked learning across different environments (Savin-Baden and Tombs 2017).
Attribution errors are a widely researched phenomenon in psychology research where they relate to cognitive errors humans make when they misinterpret the true cause of a behaviour or event. Children’s views are often excluded from evaluation research because of the belief that their views are incongruent and reduced in terms of factual information, or that unequal power relationships between researchers and children impact on children’s responses (see Punch 2002). In research studies where young children’s views have been included, researchers have typically obtained these through interviews or the use of feeling/face cards or questions illustrated through cartoons. In some studies, children’s views are gleaned from accounts of adults closely working with them (parents, teachers). There is also evidence of other, more sensitive measures used with specific groups of children – for example, the Talking Mats, which facilitate interviews with young people who have significant learning difficulties (Mitchell 2010). Sometimes, fictional characters are introduced as a less threatening figure for children to relate to when sharing their feelings. For instance, in Willow and Hyder’s research (1998), a storybook character called ‘Splodge’, who came from another planet and needed to know more about our planet, was used to gather children’s views. In line with other colleagues’ work, we adopt an appreciative orientation towards children’s work and views (Clark and Moss 2005). Our theoretical orientation grounds us in a view of children as capable of recounting their views and experiences, as long as responsive support and resources are provided. We also believe that children’s active involvement in a piece of research can be a framework for realising several educational goals (see e.g. Mordock and Krasny 2010).
In the Helicopter project, the use of Our Story allowed us to gain a deeper insight into what children genuinely felt and thought about the intervention. The use of audio recording and playback could be seen as an indication of children’s emerging audience awareness. Their implicit understanding of appropriate length and pace of their recordings was evidenced in their ability to stop and re-record their sounds. Some made conscious decisions about the quality of their recordings, as illustrated by CS1, who verbally indicated to stop the recording when the story got too long. Additionally, the opportunity to play back children’s in situ recordings gave rise to shared enthusiasm, as both the children and the researcher enjoyed re-listening to their impromptu recorded sounds. Children smiled and laughed as they listened to their voices and appreciated the opportunity to verify the accuracy of their recording. Consistent with literature regarding research rigour and transparency of findings, Author 1 et al. considered researcher reflexivity as a key process in minimising researcher subjectivity (Strauss and Corbin 1998). Notably in qualitative research methods such as action research, for example, reflection is a fundamental element of the approach, as it serves as a connecting point between action and research (Coughlan and Coghlan 2002).
It is important that researchers are cognisant of their own role in data collection and data interpretation and take steps to minimise their own influence on the observed situation and participants’ behaviour. Researcher subjectivity can lead to attribution errors, that is, ‘the fallacious attribution of characteristics to an event or an individual’ (Frensch 2007, p. 21). Arguably, the use of multimedia story-making apps, such as Our Story, to document children’s experiences, minimises researcher subjectivity in interpreting the data.
If a researcher collects data overtly, there is the possibility of seeking ongoing ethical assent. It could be argued that this possibility is removed by data being collected directly by a technology-based system, such as in these two studies. In the following section, we discuss the ethical challenges related to the use of display capture methods, especially with young children.
The display recording app physically removes researchers from the data-collection process, minimising observer effects and maximising the completeness and authenticity of data sets. An issue, however, is that researchers collect all data, including ‘grey data’ that are not directly relevant for the study. Despite children usually self-activating the recording app, past research has revealed many of them quickly forget it is operating, as there is no visible recording indicator (e.g. Author2 2015). This was evidenced by the nature of recorded conversations, which at times contained personal information, such as what was happening at home, what they intended to do or where they were going after school, or details of disagreements they were having with other children. The ability to move with a device to a private location while still being recorded may have contributed to children’s perceptions of privacy and security, possibly making it more likely that personal information was disclosed.
Although over the past five years of studies in which the recorder has been used no information of concern was captured, the potential still exists for this to happen, and researchers using this system must put in place measures for dealing with any recorded data that reveals concerns about children’s safety and well-being. Ethical concerns also exist around children’s consent and ongoing assent processes with screen capture, and these vary considerably between countries and institutions. Particularly with young children, parents should be given the opportunity to see the recorder in action, so they are in a better position to make an informed decision about its use with their child.
This issue is partly countered by the advantage of not losing important data related to children’s interaction with the device, especially if movement is involved. We therefore recommend that researchers using systems like these put in place robust ethical measures for dealing with the grey data, particularly any of a personal nature, through clear referral procedures negotiated with principals or head teachers of the institutions within which the research is carried out. It is also insufficient to justify display capture’s continued use based on a single parental consent, sign off or agreement. Obtaining regular student assent is essential for any reflexive researcher (Pillow 2003) and strongly applies to research using display capture methods.
When using both apps children are data producers and data collectors, and this role needs to be made explicit to them. With the display recorder app, the children may see their face on-screen, and with Our Story children create their own stories with their own pictures and sounds. Children might decide to keep these stories saved on the iPad, or they might decide to share their stories with other children or adults. For online sharing, it is important to consider the distribution settings of the app to ensure that each story is sent to the right recipient. Our Story uses the Dropbox sharing mechanism, whereby each sender and recipient needs to be authenticated using an email address and password. Adults who give consent for data sharing on children’s behalf need to be aware that there is no complete guarantee that stories shared online are fully protected from hacking or non-deliberate modification and deleting. At all times it is important to ensure that children’s stories contain only appropriate images and videos.
The Our Story and Shou apps are potentially valuable assets for teachers interested in assembling rich multimedia evidence of children’s progress and work practices, as well as for researchers interested in collecting samples of children’s learning and development. The availability of multimedia not only enhances the observations but also provides an opportunity for dialogue: the front camera allows children to see their own image, and teachers can use this to discuss their work with them, encouraging meta-thinking and awareness of self. Photo- and video-based profiles can also provide a dialogue opportunity with children’s parents, who appreciate seeing an authentic and dynamic record of their child’s activities, in situ. Display capture video can also provide teachers with valuable information about children’s learning processes and how they interact with others and the apps to solve problems and complete tasks. This information is of considerable value for helping teachers improve their pedagogy and learning task design, and it offers insights into the nature of skills and capabilities children use while completing these.
Our Story can also be used to document practice and children’s achievements. The use of photographic evidence is commonplace in many Western preschools, and Our Story allows for integration of pictures with text or audio annotations. The possibility of recording children’s progress in a story format might be of use to practitioners who seek to share children’s progress with various audiences.
Our direct involvement in the conceptualisation, design, use and evaluation of two data collection apps has taught us lessons concerning data authenticity and the empirical and ethical challenges concerning the positioning of the child and researcher in observational research. We posit that observer effects and researcher subjectivity can be significantly reduced if we use apps specifically designed for display recording and creation of open-ended multimedia stories. Researchers can authentically capture children’s capabilities and experiences as long as they combine the use of innovative methods with well-established ethical principles.
Natalia Kucirkova would like to thank Professors Cremin and Swann, Drs Flewitt and Faulkner and MakeBelieve Arts for their permission to use data from the Helicopter project in this article. Both authors would like to thank Dr Rosie Flewitt for her critique of an earlier draft of this article. Natalia’s research is supported by the Economic and Social Research Council [grant number ES/N01779X/1].
Ary, D., Jacobs, L. & Sorensen, C. (2010) Introduction to Research in Education, Cengage, Belmont.
British Educational Research Association. (2004) Ethical Guidelines. British Educational Research Association. Available at: https://www.bera.ac.uk/researchers-resources/publications/ethical-guidelines-for-educational-research-2011
Canning, N., et al. (2017) ‘An innovative methodology for capturing young children’s curiosity, imagination and voices using a free app: our story’, International Journal of Early Years Education, vol. 25, no. 3, pp. 292–307.
Clark, A. & Moss, P. (2011) Listening to young children: The mosaic approach, Jessica Kingsley Publishers, London.
Cremin, T., Swann, J., Flewitt, R., Faulkner, D. & Kucirkova, N. (2013) Evaluation report of make believe arts helicopter technique of storytelling and storyacting, The Open University, Milton Keynes, UK.
Cremin, T., Flewitt, R., Swann, J., Faulkner, D., & Kucirkova, N. (2017). Storytelling and story-acting: Co-construction in action. Journal of Early Childhood Research, 1476718X17750205.
Coughlan, P. & Coghlan, D. (2002) ‘Action research for operations management’, International Journal of Operations & Production Management, vol. 22, no. 2, pp. 220–240.
Cremin, T., Flewitt, R., Mardell, B., & Swann, J. (Eds.). (2016). Storytelling in early childhood: Enriching language, literacy and classroom culture. Taylor & Francis.
Crompton, H., Burke, D. & Gregory, K. (2017) ‘The use of mobile learning in PK-12 education: a systematic review’, Computers & Education, vol. 110, pp. 51–63.
Culén, A. L., & Gasparini, A. (2011). iPad: a new classroom technology? A report from two pilot studies. INFuture Proceedings, 3(2), 199–20
Falloon, G. W. (2015) ‘What’s the difference: Learning collaboratively using iPads in conventional classrooms’, Computers & Education, vol. 84, pp. 62–77.
Falloon, G.W. & Khoo, E. (2014). Exploring young students’ talk in iPad supported collaborative learning environments. Computers & Education, 77, 13–28.
Falloon, G.W. (2013). Young students using iPads: App design and content influences on their learning pathways. Computers & Education, 68, 505–521.
Foster, P. (1996) ‘Observational research’, in Data Collection and Analysis, eds R. Sapsford & V. Jupp, SAGE, New York, pp. 57–93.
Frensch, F. (2007) The Social Side of Mergers and Acquisitions: Cooperation relationships after mergers and acquisitions, Springer Science & Business Media, Berlin.
Garcia-Cabot, A., de-Marcos, L. & Garcia-Lopez, E. (2015) ‘An empirical study on m-learning adaptation: learning performance and learning contexts’, Computers & Education, vol. 82, pp. 450–459.
Hutchison, A., Beschorner, B. & Schmidt-Crawford, D. (2012) ‘Exploring the use of the iPad for literacy learning’, The Reading Teacher, vol. 66, no. 1, pp. 15–23.
Jersild, A. & Meigs, M. (1939) ‘Direct observation as a research method’, Review of Educational Research, vol. 9, no. 5, pp. 472–482.
Kabali, H., et al. (2015) ‘Exposure and use of mobile media devices by young children’, Pediatrics, vol. 136, no.6, pp. 1044–1050.
La Donna, K., et al. (2017) ‘Staging a performance: learners’ perceptions about direct observation during residency’, Medical Education, vol. 51, no. 5, pp. 498–510.
Liang, J. (2015) ‘Live video classroom observation: an effective approach to reducing reactivity in collecting observational information for teacher professional development’, Journal of Education for Teaching, vol. 41, no. 3, pp. 235–253.
Lynch, C. (2000) ‘Authenticity and Integrity in the digital environment: a exploratory analysis of the central role of trust’, Authenticity in a Digital Environment, ed. Ross Parry, Council on Library and Information Resources, Washington, DC. pp. 314–332.
Maisling, J. & Stern, G. (1969) ‘Effect of the observer in the classroom’, Journal of Educational Psychology, vol. 60, no. 5, pp. 351–354.
Miller, E. B. & Warschauer, M. (2014) ‘Young children and e-reading: research to date and questions for the future’, Learning, Media and Technology, vol. 39, no. 3, pp. 283–305.
Mitchell, W. (2010) ‘I know how I feel’ listening to young people with life-limiting conditions who have learning and communication impairments’, Qualitative Social Work, vol. 9, no. 2, pp. 185–203.
Mordock, K. & Krasny, M. (2001) ‘Participatory action research: a theoretical and practical framework for EE’, The Journal of Environmental Education, vol. 32, no. 3, pp. 15–20.
Nicholas, M. (2018). Affordances of using Multiple Videoed Events to construct a rich understanding of adult–child book readings. International Journal of Research & Method in Education, 41(2), 125–141.
Osborne, R. (1983) ‘Towards modifying children’s ideas about electric current’, Research in Science and Technological Education, vol. 1, no. 1, pp. 73–82.
Outhwaite, L., Gulliford, A. & Pitchford, N. (2017) ‘Closing the gap: efficacy of a tablet intervention to support the development of early mathematical skills in UK primary school children’, Computers & Education, vol. 108, pp. 43–58.
Pillow, W. (2003) ‘Confession, catharsis, or cure? Rethinking the uses of reflexivity as methodological power in qualitative research’, International Journal of Qualitative Studies in Education, vol.16, no. 2, pp.175–196.
Plowman, L., Stephen, C. & McPake, J. (2010) ‘Supporting young children’s learning with technology at home and in preschool’, Research Papers in Education, vol. 25, no. 1, pp. 93–113.
Punch, S. (2002) ‘Research with children: the same or different from research with adults?’ Childhood, vol. 9, no. 3, pp. 321–341.
Savin-Baden, M. & Tombs, G. (2017) Research methods for the digital age, Bloomsbury Academic, Oxford.
Shipstone, D.M. (1984) ‘A study of children’s understanding of electricity in simple DC circuits’, European Journal of Science Education, vol. 6, no. 2, pp. 185–198.
Strauss, A. & Corbin, J. (1998) Basics of Qualitative Research Techniques, SAGE, New York.
Sung, H.Y. & Siraj-Blatchford, J. (2015) ‘Supporting family learning and interaction through information and communications technology in public libraries in Taiwan’, Journal of Librarianship and Information Science, vol. 47, no. 4, pp. 294–302.
Willow, C. & Hyder, T. (1998) It Hurts You Inside: Children Talking About Smacking, Jessica Kingsley Publishers, London.