ORIGINAL RESEARCH ARTICLE
A participatory framework proposal for guiding researchers through an educational mobile app development
Rita Tavaresa*, Rui Marques Vieiraa and Luís Pedrob
aCIDTFF, Department of Education and Psychology, University of Aveiro, Aveiro, Portugal;
bDigiMedia, Department of Communication and Arts, University of Aveiro, Aveiro, Portugal
(Received: 25 November 2019; Revised: 1 June 2020; Accepted: 27 November 2020; Published: 31 December 2020)
Abstract
The research work reported in this article is part of a wider study aimed at developing a mobile application (app) for Science Education in primary school. For that, we designed a participatory framework proposal nested within the larger framework of Educational Design Research. This framework emerged from the authors’ need to organise the different phases of the mobile apps development and to arrange the expected products that arise from them. Our framework suggests a grounded, participatory and user-centred approach, relating literature contributions with data collected among future end-users. In this study, we exemplify the implementation of the proposed framework by presenting the outcomes of a specific moment of the preliminary research: students’ stories and drawing analysis, to define the mobile app concept. For that, we (1) present and describe the participatory framework proposal, (2) identify and characterise the research method adopted to define the mobile app concept, (3) reveal and analyse data collected from the implementation of a creative writing and drawing activity performed by fourth grade primary-school students, and (4) describe the implications of data analysis in the mobile app concept definition. Our intention is to share with other educational researchers an approach that can be used to develop educational mobile apps grounded on future end-users’ perceptions and ideas. We also aim to contribute to deepen the Educational Design Research apps by proposing and exemplifying the implementation of a framework that brings together researchers, students, teachers and experts.
Keywords: participatory framework; mobile application; educational software development; educational design research; mixed-methods; participatory design; user-centred design
*Corresponding author. Email: ritaveigatavares@ua.pt
Research in Learning Technology 2020. © 2020 R. Tavares et al. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2020, 28: 2370 - http://dx.doi.org/10.25304/rlt.v28.2370
Introduction
Educational software development is highly complex, as it comprises didactic, graphical, functional and technical specifications developed according to different interactive and iterative stages. Over the years, its complexity has resulted in several development approaches, some based on software development methods, others based on educational design methods (usually referred to as Instructional Design – Cf. Seel et al. 2017) and some based on both (Allen 2007; Brown and Green 2016; Sommerville 2011).
Although software development and educational design methods are often intertwined or indistinctly used in the literature and websites related to educational software development, it is important to distinguish them. Educational design methods are systematic approaches to develop educational solutions (e.g. Massive open online courses), sustained by (1) learning theories/approaches; (2) students’ learning needs analysis; (3) learning contents and/or environmental design, selection and/or development; and (4) implementation and evaluation of the educational solutions to measure their impact on and adequacy to the learning goals (University of Michigan 2003 referred to by Brown and Green 2016; Tchounikine 2011).
Software development methods predict educational design methods if we intend to develop educational software; however, their main focus includes the following technical procedures: graphical design, user experience definition, and programming and testing processes (Sommerville 2011). There are several software development methods: some more sequential, others more iterative, some predicting higher levels of future end-users’ participation and others that do not predict any kind of collaboration at all (Cf. Beck 2000; Ben-Zahia and Jaluta 2014; Boehm and Turner 2003; Matković and Tumbas 2010; Munassar and Govardhan 2010; Office of Information Services 2008; Ruparelia and Hewlett-Packard Enterprise Services 2010; Sommerville 2011). Sommerville (2011) underlines that regardless of the adopted method, any software development process will always have to include four key stages: (1) Specification – definition of the software features and restrictions, (2) Design and development – production of the software according to the specifications, (3) Validation – software testing in order to validate its suitability to the users’ needs and (4) Evolution – adjustments, corrections and/or changes to the software according to the users’ needs.
The educational design methods are also diverse, some are more linear and others more iterative (Boyle 2002; Brown and Green 2016; Costa 2013; Hinostroza and Mellar 2001; Johnson and Henderson 2002; Tchounikine 2011). According to Chen (2011), there are currently more than 100 educational design methods, almost all based on the Analysis, Design, Development, Implementation and Evaluation (ADDIE) model. Brown and Green (2016) suggested that regardless of their specificities, all educational design methods include three key stages: (1) Analysis – needs’ survey, determining the development of the educational solution and the design/definition of its requirements; (2) Production – design and development of the educational solution; and (3) Evaluation – implementation of the educational solution and evaluation of its suitability, according to the goals and/or its impact on the learning process.
Our proposal relates the four key stages of software development processes with the three key stages of educational design methods (see the Participatory framework proposal section). Furthermore, our proposal suggests a grounded approach, relating literature contributions with data collected from among experts and future end-users due to the choice of following a comprehensive approach (Plomp 2013). Regarding the future end-users’ collaboration, our intention was to adopt a participatory and user-centred approach, since we as researchers ‘know less about how the users interact with the solution and about how it effect their everyday practice, only the users themselves can provide this input’ (Ørngreen 2015, p. 28). Thus, our proposal aims to contribute to a renewed understanding of how educational researchers can benefit from the involvement, among others, of the educational solutions’ future end-users, namely, contributing to their ideas.
The literature analysis revealed that there are few frameworks to support educational mobile applications (apps) development (Cf. Kucirkova 2017; Shing and Yuan 2016; Zydney and Warner 2016). According to Bodily, Leary and West (2019, p. 65), ‘Tthere appeared to be a lack of recent scholarship on learning theories and design frameworks’. The authors also underline that studies are ‘very technology-centric, and our field could benefit from a greater balance of papers studying theory and design frameworks’ (Bodily, Leary, and West 2019, p. 78). In line with this, we also believe that Educational (Design) Research could benefit from studies relating to the theory and the design approach(es) that supports those frameworks. Our framework can represent a contribution, as it entails both approaches: educational design and software development.
Our contribution lies in the fact that the development of educational solutions should engage the end-users as much as possible. As User Experience designers have been doing all over the years (Hartson and Pyla 2019; Mirri, Roccetti, and Salomoni 2018; Norman 2013), we support the idea that educational researchers should find ways to actively involve the educational solutions’ end-users (students, teachers, parents ...), instead of grounding educational solutions’ approaches only in literature reviews or their research rationales. Arising from this, we also support the idea that educational researchers, developers and end-users must work together from the solution design concept to its deployment. This could represent an opportunity to enhance the solutions’ development based on systematic processes of co-reflection and adjustments, according to the end-users’ needs, expectations and ideas (Hartson and Pyla 2019; Mirri, Roccetti, and Salomoni 2018; Norman 2013). Furthermore, we support the idea that this co-design approach could represent an opportunity to develop and share new and grounded scientific knowledge.
Trying to contribute to this shift, our participatory framework proposal (although in a flexible way) helps to guide those who adopt it through the necessary phases and products for the development of an educational mobile app, contributing to extend the range of outputs of Educational (Design) Research. In the following sections, we (1) present and describe the participatory framework proposal, (2) identify and characterise the research method adopted to define the mobile app concept, (3) reveal and analyse data collected from the implementation of a creative writing and drawing activity performed by fourth grade primary-school students, (4) describe the implications of the data analysis in the mobile app concept definition and (5) present some final considerations.
Participatory framework proposal
As mentioned earlier, our participatory framework proposal aims to help other educational researchers to develop educational mobile apps grounded on future end-users’ perceptions, ideas and needs (International Organization for Standardization 2010; Ritter, Baxter, and Churchill 2014). Because it is quite flexible and comprehensive, we believe that our framework proposal can also be used in the development of other technological educational solutions.
The proposal relates the four key stages of the software development process (Sommerville 2011) with the three key stages of the educational design methods (Brown and Green 2016). In order to better understand their relationship, we matched both (see Table 1). With that, we were able to overview the implications of each phase in our mobile app development and, by doing so, were able to define our educational design (research) approach (see also the Methods section).
To develop our mobile app, we adopted a participatory approach (also mentioned in the literature as co-design) (Hartson and Pyla 2019). For that, we defined several interactions and iterations with different participants, namely, students, teachers and experts from different knowledge fields. Nowadays, end-users’ (and other stakeholders’) participation within technological solutions design is broadly accepted and implemented (Hamzah 2018; Hartson and Pyla 2019; Mirri, Roccetti, and Salomoni 2018; Robertson and Simonsen 2012). As Simonsen and Hertzum (2012, p. 10) suggested, participatory design ‘has a lot to offer’, for instance, by helping educational researchers and developers to clarify needs and goals, define suitable learning approaches and design learning environments tailored to the students’ needs. In our study, future end-users’ participation was crucial, namely, in the mobile app concept definition through the students’ stories and drawing analysis (see Results and discussion).
As our participatory framework proposal is nested within the Educational Design Research, we briefly clarify it. Educational Design Research is a comprehensive approach that considers all phases and processes of the development of an educational solution as a whole, combining different research techniques and participants (Plomp 2013). According to Barab and Squire (2004, p. 1), this approach is focused on the development of ‘technological tools, curriculum, and especially theory that can be used to understand and support learning’, such as ‘design principles or guidelines derived empirically and richly described, which can be implemented by others interested in studying similar settings and concerns’ (Amiel and Reeves 2008, p. 35).
By focusing on the design and the development of the educational solution for future end-users’ needs, Educational Design Research can be a user-centred design approach (Keates, Trewin, and Elliott 2006; Mirri, Roccetti, and Salomoni 2018). By ‘letting the users actively contribute to the design and to the content development process’ (Mirri, Roccetti, and Salomoni 2018, p. 2), Educational Design Research can also be a participatory approach. In our study, we adopt both approaches: user-centred and participatory design. The first one by developing the mobile app according to the future end-users’ needs – in our study, the students as primary end-users using the mobile app and the teachers as secondary end-users by supporting its usage (see Results and discussion). The participatory design approach is reflected by predicting the future end-users’ participation in different phases of the mobile app development.
According to Majgaard, Misfeldt, and Nielsen (2011), students and teachers’ participation in our study aims to improve (1) the innovative nature of the proposed educational solution, and (2) the usability, effectiveness and relevance of our mobile app. This attempt aims to ground our research in reliable and transferable principles of educational design, thus, contributing to bringing educational research closer to the classroom practices, as well as to influence ‘policy making and strategic school decisions’ (Reimann 2011, p. 46). For that, we adopted the four main processes of user-centred software development, relating them with the three main phases of Educational Design Research. This relationship is presented in Table 2. Besides this relationship, Table 2 illustrates the systematic characteristics of Educational Design Research, overlapping it with a well-known educational design method: the ADDIE model (Anderson and Shattuck 2012; de Villiers and Harpur 2013; McKenney and Reeves 2013; Plomp 2013; Reimann 2011; The Design-Based Research Collective 2003; Wang and Hannafin 2005).
| Educational Design Research main phases (Plomp, 2013) | User-Centered Software Development main processes (International Organization for Standardization, 2010) |
| Preliminary research | |
| Needs and context analysis, alongside the design of the conceptual framework(s) based on the literature review | → Understand, analyse and specify the context of use |
| → Specify user requirements (e.g., target audience, characteristics, needs) | |
| Prototyping phase | |
| Guidelines definition for the design of the educational solution and development and improvement of prototypes through iterative and interactive processes of design, formative evaluation and revision | → Design and develop (e.g., guidelines, frameworks, prototypes) |
| Assessment phase | |
| Implement and evaluate the impact of the educational solution next to the target audience, if they really want to use it, and if the educational solution meets the proposed | → Evaluate the design and development according to the proposed and to the users’ perspective |
de Villiers and Harpur (2013, p. 256) propose a ‘Generic Model of Design-Based Research Process within a Context’, representing:
- the starting point for the approach – the problem and the need for innovation;
- the phases and the processes that arise from the problem, according to the ADDIE model and influenced by the collaboration of the different participants and the context itself;
- the research products, according to the Educational Design Research approach – practical solutions and theoretical contributions;
- the importance of synergies (which should result) between practice and theory, and between educational design and research.
The integration of the ADDIE model in the Educational Design Research approach is related to the fact that the researcher repeats as many times as necessary each phase and/or process to reach the ‘ideal’ educational solution (Plomp 2013). Despite its classification, ADDIE is not a specific model, but rather an illustration of the common conceptual components shared among many methods of educational design: Analysis, Design, Development, Implementation, Evaluation and, more recently, Revision (Bichelmeyer 2005; Branch and Kopcha 2014; Brown and Green 2016; Molenda 2003). The last two components aim at a continuous, formative and integrated evaluation and revision of the developments carried out, allowing the researcher to go back to each stage whenever desirable and/or necessary in order to correct, adjust, improve and/or implement new integrations.
Focusing on the development of an educational mobile app, in the Analysis phase, among other aspects, one proposes that the researcher (1) defines the problem and the need for innovation (e.g. low availability of Science Education mobile apps), (2) analyses and specifies user requirements (e.g. target audience) and (3) defines the mobile app concept (e.g. the app characters and the graphic environments).
In the Design phase, the researcher deepens and defines aspects, such as (1) the topic to be approached in the educational mobile app (e.g. human body), (2) the learning goal(s) (e.g. identify some common human body parts), (3) the digital educational contents to be integrated and developed (e.g. animations, games and simulations) and (4) the assessment tools and strategies (e.g. quizzes). These aspects are related to the educational app didactic specifications. It is also in the Design phase that the aspects related to the graphical, functional and technical specifications of an educational mobile app are defined.
In the Development phase, one proposes the production and validation of the aspects defined in the Design phase (e.g. prototype development), identifying possible changes and/or improvements to the development of the final mobile app version. Thus, in the Implementation phase, the prototype of the educational app is tested and, finally, the final version is made available to the end-users.
Transversely and iteratively, in the Evaluation phase, we propose that the researcher, using different criteria and strategies, evaluates the mobile app design, development and implementation processes (e.g. questionnaire and focus group). Regarding the Revision phase, one proposes a continuous analysis of the adequacy of the issues defined and/or developed, allowing the researcher to improve or redefine them (e.g. the mobile app concept, its specifications and the developed prototype). The revisions can occur in a sequential or non-sequential way. For instance, in the Development phase, the researcher can identify new aspects related to the specifications defined in the Design phase, returning to this phase to redefine those aspects and then advance with the Development again (McKenney and Reeves 2012, 2015; Plomp 2013).
From the exposed, regarding the integration of the ADDIE model in the Educational Design Research approach, one proposes that in the Preliminary research, the researcher integrates Analysis and Design components aiming at the Mobile App Delimitation (see Figure 1). In this phase, we suggest two kinds of deliverables: guidelines and frameworks, to support the mobile app design. As guidelines, we propose the development of the Preliminary draft (Costa 2013), organising and detailing the following aspects: (1) the mobile app target, (2) the topic to be approached in the mobile app, (3) the mobile app learning approach, (4) the digital educational contents that the mobile app must integrate, (5) the learning management components that the mobile app must predict and (6) the mobile app concept.
Figure 1. Participatory framework proposal for guiding researchers through an educational mobile app development.
Frameworks can be part of the Preliminary draft or related to it. For instance, in our project, one of the developed frameworks in this phase was the mobile app learning approach, as we proposed an authorship framework relating to the Universal Design for Learning principles, the Enquiry-Based Science Education and the BSCS 5E’s (cf. Tavares, Vieira, and Pedro 2017). However, the researcher can adopt an existing one (e.g. game-based learning and project-based learning). Using again our project as an example, another framework developed related to the Preliminary draft was the Relational structure of the conceptual Educational Data Mining framework for Science Education, a particular framework of our project that derives from the integration of the learning management components in the mobile app and that defines how these components influence or promote the students’ scientific competencies development and self-regulated learning by exploring the app (Tavares, Vieira, and Pedro 2017).
According to the participatory and user-centred approaches adopted in the Preliminary research, one proposes students, teachers and experts’ participation in the mobile app delimitation. For instance, in our project, students’ participation occurred through the implementation of a creative writing and drawing activity, which allowed us to define the mobile app concept (the specific moment of the Preliminary Research detailed in this article – see Results and discussion). Teachers’ participation emerged from the implementation of a questionnaire, helping us to define, among others, the target audience, the topic to be approached in the mobile app and to validate the learning approach proposal. Experts’ participation was requested to validate, for instance, the creative writing and drawing activity, the questionnaire and the Relational structure of the conceptual Educational Data Mining framework for Science Education.
In the Prototyping phase, we suggest that the researcher integrates Design and Development components, aiming at the Mobile App specification (see Figure 1). In this phase, we also suggest guidelines and frameworks as deliverables to support the mobile app development. As guidelines, we propose the development of the Authoring guide (Costa 2013), detailing the mobile app’s didactic, graphical, functional and technical specifications. As a result of the didactic specifications, one proposes the development of the digital educational content, Scripts and Storyboards. If the researcher does not have the skills to define the technical specifications, these can be defined by a developer.
Frameworks can also be part of the guidelines or related to it. As frameworks, in this phase, one proposes the development of the mobile app’s Educational Data Mining conceptual framework for Science Education, another particular framework of our project that derives from the Relational structure of the conceptual Educational Data Mining framework for Science Education. In this phase, we also suggest the development of the mobile app Wireflows (Laubheimer 2016) based on graphical and functional specifications, and the mobile app prototype.
For instance, in our project, from a focus group session implementation with User Experience experts, we developed the mobile app Wireflows based on the experts’ Sketches. In this regard, in the Prototyping phase, we propose the experts’ participation, although we believe that this phase can also benefit from the contribution of students and teachers, according to the researcher’s options and/or needs. In the Prototyping phase, we also requested the experts’ participation to validate the mobile app didactic specifications, the Scripts and the Storyboards. We also requested the experts’ participation to validate the graphical and functional specifications, and the Educational Data Mining conceptual framework for Science Education, detailed through 11 graphical algorithms (e.g. Graphic algorithm 1: General app algorithm; Graphic algorithm 9: Data mining algorithm for the development of scientific competencies).
In the Assessment phase, one proposes that the researcher integrates the Implementation component in order to make the prototype available to the future end-users and its validation (see Figure 1). In this phase, as deliverable, we suggest the mobile app’s final version. In this phase, we propose the end-user’s participation, for instance, to test the mobile app prototype and to evaluate it through a questionnaire related to the task, in order to validate its suitability, according to the proposed and to the end-user’s perspective.
Transversely and iteratively to all phases, we propose the integration of the Revision and Evaluation components, aiming at (1) a continuous analysis of the adequacy of the defined and/or developed in each phase, (2) the improvement or redefinition of the app’s specifications and/or developments, and (3) the evaluation of the mobile app design, development and implementation processes.
The purpose of our participatory framework proposal is to help educational researchers to organise the definition, management and implementation of the different phases and tasks of the complex of designing and developing an educational mobile app. It also aims to help educational researchers to develop an educational mobile app grounded on future end-users’ perceptions, ideas and needs, contributing to deepen the Educational Design Research app, in particular by proposing a framework that brings together researchers, students, teachers and experts. This collaborative development of educational mobile apps could become a new means for improving the range of outputs of Educational (Design) Research, as well as for facilitating the development of future educational mobile apps in both academia and industrial contexts.
In the following sections, we exemplify the implementation of our framework by revealing, analysing and discussing data collected from the implementation of the creative writing and drawing activity performed by the students. For that, we present the research method adopted and describe the implications of the data analysis in the mobile app concept definition.
Method
As mentioned above, in order to develop our project we adopted the Educational Design Research approach, integrating the ADDIE model components in the three phases (Anderson and Shattuck 2012; de Villiers and Harpur 2013; McKenney and Reeves 2013; Plomp 2013; Reimann 2011; The Design-Based Research Collective 2003; Wang and Hannafin 2005). In this article, we detail a specific moment of the Preliminary Research: the implementation of the creative writing and drawing activity performed by fourth grade primary-school students.
Deriving from the proposed framework, participatory and user-centred, by asking the end-users’ participation we were able to define the mobile app concept according to their perceptions and ideas. For that, we implemented the creative writing and drawing activity performed by students according to the following correlated moments:
- Design and validation: The first author of this study has professional experience as a primary-school teacher, having taught from 2005 to 2010 in several public Portuguese schools. Given her experience in the area, she designed the activity. In order to ensure the scientific and pedagogical accuracy of the proposed activity, it was analysed and validated by two experts.
- Implementation and data collection: The activity was carried out in a school setting, with students being enrolled in a fourth grade class (n = 25). All students performed the creative writing and drawing activity that is proposed, and all students’ stories and drawings were considered valid for our study.
- Data analysis: We performed content analysis on the students’ stories and drawings.
Participants
The University of Aveiro, namely, the Department of Education and Psychology through the Research Centre on Didactics and Technology in the Education of Trainers, is one of the most prestigious Portuguese institutions in (ongoing) teacher training activities, working in this area since the academic year 1977–1978 (University of Aveiro 2019). This University has a long tradition in the establishment of partnerships and implementation of research projects in primary and secondary schools located in the Aveiro region.
In order to implement the creative writing and drawing activity, the authors requested the collaboration of one of the partner schools, that is, the collaboration of the fourth grade class (n = 25). The participation of this particular grade students was related to the delimitation of the mobile app’s target audience, defined in the previous moment of the Preliminary Research through the implementation of a questionnaire to primary-school teachers (see the Participatory framework proposal). Among others, from the data analysis of the teachers’ answers, we could deduce the target audience (fourth grade primary-school students – 8-9 years of age) and the topic to be approached in the mobile app (Human Body).
Data collection
The activity included two correlated tasks: firstly, students should write an original story (maximum 20 lines) and, in the second task, illustrate that story. The activity was performed in a school setting and was implemented as a regular classroom activity, with no previous enquiry, without orientation related to the tasks to be performed and without the researchers’ presence. This implied that the students would carry out the proposed tasks autonomously and completely free of any pre-conceptions.
For the first task, the authors defined a set of words related to the Human Body topic. As mentioned above, this thematic area was defined in the previous moment of the Preliminary Research from data analysis of the teachers’ answers to a questionnaire (n = 118): Primary-school teachers’ conceptions about their knowledge and their educational practices in Science Education using digital educational resources (Tavares, Vieira, and Pedro 2019b). The instrument was implemented using the University of Aveiro Questionnaires platform. To assure the internal consistency and reliability of the questionnaire, we applied a pilot version of the instrument to 17 primary-school teachers, which allowed us to validate the instrument (α = 0.86, r = 0.63, p < 0.01). The final version of the instrument (α = 0.87, r = 0.71, p < 0.01) was implemented in 228 primary-school teachers, of which we considered 118 answers complete and valid for our study (Anastasi 1976; Guilford and Fruchter 1973; Nunnaly and Bernstein 1994). The instrument comprised nine questions (Q) of which, to define the mobile app’s thematic area, we analysed two:
- – In Q7, from the presented list, we gather one or more topics that teachers frequently explore with their students. Select one or more options (Fluid fluctuation; Dissolution in liquids; Seeds, germination and growth; Light, shadows and images; Electrical circuits, batteries and lamps; Changes of state; Sustainability on Earth; and Human Body);
- – In Q8, from the presented list, we gather the 2 topics that teachers considered most easily approached using digital educational resources (Fluid fluctuation; Dissolution in liquids; Seeds, germination and growth; Light, shadows and images; Electrical circuits, batteries and lamps; Changes of state; Sustainability on Earth; and Human Body).
These questions emerged from the adaptation of the Experimental Science Education Collection contents (Martins et al. 2007) in animations, games, simulations, quizzes and information areas (the digital educational contents integrated in the mobile app – (Cf. Tavares, Vieira, and Pedro 2017). This collection has eight manuscripts related to the topics presented in Q7 and Q8.
The set of words was also related to the topics, Healthy Eating and Physical Activity, intervention areas that constitute two of the 11 ongoing ‘Priority Health Programs’ articulated with the Portuguese ‘National Health Plan’: ‘The Portuguese National Programme for the Promotion of Healthy Eating’ and ‘The Portuguese Program for the Promotion of Physical Activity’ (Directorate-General of Health of Portugal 2017). The authors chose to work on these two topics as it is estimated that 29% of the boys and 32% of the girls between the ages of 6 and 9 years are overweight, and 12% and 11%, respectively, are obese (World Health Organization 2018). According to Rito, Silva, and Breda (2016, p. 46), this problem ‘is mainly attributed to unbalanced diets associated to lack of physical activity’. For these reasons, we chose to relate the Healthy Eating and Physical Activity topics to the Human Body thematic area, defining the following set of words for the activity: human body, agriculture, biological, sustainable, eat, cook, food, balanced, healthy, rules, physical activity and physical exercise.
In the second task, the authors proposed to the students to illustrate their original stories. To avoid bias in data collection, we chose not to ask them to draw any specific element, such as the story character(s), the environment where it occurred or both.
All the 25 class students performed the creative writing and drawing activity that is proposed, and all data collected were considered valid to our study (25 written productions and 25 graphic productions), helping us to define the mobile app concept.
All the study phases and moments were implemented in Portugal, where the authors developed their research. Therefore, the original data are only available in Portuguese (Tavares, Vieira, and Pedro 2019a).
Results and discussion
Content analysis was performed to deduce and validate the following information related to the mobile app concept and to its digital educational contents:
- the possible approaches to the Human Body topic;
- the subtopics to focus on the digital educational contents (animations, games, simulations, quizzes and information areas);
- the main characters;
- the possible graphic environments.
For this, we adopted an inductive conceptual analysis approach, determining the existence and frequency of concepts related to the above information in the students’ stories and drawings (Creswell and Creswell 2018; Mayring 2000; Richardson 2003). The analysis of the written and graphic productions was conducted without the adoption of any theoretical frameworks. As a result, the content analysis categories and subcategories were defined as the data were analysed (Denzin and Lincoln 2003; Richardson 2003). In this view, we ‘remain open to new ways of seeing and understanding’ data, and thus, finding new approaches and expanding our ideas on how to define the mobile app concept (Thorne 2014, p. 109).
Content analysis was performed using webQDA®, a qualitative computer data analysis programme. Firstly, we transcribed and anonymised the written productions assigning an identity (ID) to each one, generating individual portable document format files (.pdf). Secondly, we scanned the graphic productions and anonymised them by assigning the written productions corresponding ID, generating individual portable network graphics files (.png). These files were uploaded to the adopted software and assembled in a single project.
In order to organise the data, the sources were coded according to the classifications: written Productions and graphic Productions. Then, we proceeded with a first read of the written productions (Creswell and Creswell 2018). From this initial reading, we defined the Topic approach attribute (related to the possible approaches to the Human Body topic) and the following five values: Everyday life, Right/Wrong, Before/After, Challenge and Physical Activity. Each written production was coded according to one value – see Table 3. With this first sources classification, we could deduce (1) the possible approaches to the Human Body topic: descriptions related to daily situations (Everyday life – 11 references) and describing right and wrong attitudes related to the human body (Right/Wrong – 7 references).
To validate and define (2) the subtopics to focus on the mobile app’s digital educational contents, (3) the main characters and (4) the possible graphical environments, we analysed the written and graphic productions. This analysis resulted in the system of categories presented in Table 4. Our units of analysis included words, sentences and graphic elements, performing data frequency analysis (Leavy 2017; Prior 2014).
As each ID corresponds to two sources, to avoid duplication, the references related to the Graphic analogy were matched, identifying common elements between the written and graphic productions (grey references in Table 4). For the ID where the same references were identified, we only considered one. For instance, ID1 refers the word ‘house’ in the written production, and in the graphic production a house was drawn. In this case, we only considered one reference to the element ‘house’. Only the references registered twice or more and common to both the types of production were considered.
The original references were translated into English regarding the present text coherence. Although some of the translated sentences had been subjected to small adjustments, semantic, idiomatic, cultural and conceptual equivalence were preserved.
In order to assure the adequacy of the inductive system of categories and the coded ID, we submitted clone versions of our project into a validation process, asking two experts to code 10% of our coded ID selected randomly. Then, we calculated the reliability (r) of the designed system of categories, according to the following equation (Amado 2014): , where Ta represents the total of agreements and Td the total of disagreements between our coded ID and the expert ones. By applying this validation approach, we verified the reliability of the designed system of categories (r=0.91). As a result of the sources exploratory analysis, we defined two main categories:
- – Didactic Analogy: elements related to the Human Body subtopics. This category was applied only to the analysis of the written productions.
- – Graphic Analogy: elements related to the main characters and the possible graphic environments of the app and of the mobile app’s digital educational contents. This category was applied to both the analysis of the written and graphic productions.
In the Didactic Analogy, we defined the subcategory Subtopic: Healthy Eating, Physical Activity, Organic Agriculture/Eating, Sustainable Agriculture/Eating, Obesity and Diabetes. In the Graphic Analogy, we defined the subcategories: Graphic environments and Characters. Analysing the sources, we could validate (2) the subtopics to focus on the digital educational contents: Healthy Eating (25 references), Physical Activity (20 references) and Organic Agriculture/Eating (19 references). The analysis also allowed us to deduce the following:
- – (3) The main characters: a boy (14 references) and a girl (5 references). As a secondary character, we will consider the mother (8 references).
- – (4) The possible graphic environments: the characters house (5 references) and the kitchen garden (4 references).
Finally, we analysed the most frequent words used in the written productions – see Table 5. For that, we excluded discursive connectors (e.g. ‘and’, ‘or’, ‘that’, ‘if’, ‘to’, ‘ie’ and ‘this is’) and only considered words and short sentences with at least 10 references.
The analysis allowed us to deduce that the most frequent words and short sentences used by the students in the written productions were eat (39 references), healthy (38 references), food (36 references) and eating (35 references).
Data analysis allowed us to deduce the following mobile app concept: The mobile app digital educational contents will be a set of stories related to daily situations and describing right and wrong attitudes related to the Human Body topic and the Healthy Eating, Physical Activity and Organic Agriculture/Eating subtopics. The digital educational contents, especially the animations, games and simulations, will be designed around aspects, such as eat, healthy, food and eating. The mobile app and the digital educational content stories will have as main characters a boy and a girl and will take place mainly at the house and the kitchen garden of the characters.
Conclusion
From this study, we intended to share with other educational researchers an approach that can be used to support the development of educational mobile apps grounded on future end-users’ perceptions, ideas and needs. By sharing some examples of how the proposed framework was applied in our project and it impacts, we aimed to demonstrate its operationalisation and the real impact of its usage.
Our focus was on (1) the research products, namely, the technological educational solution (the mobile app), and (2) the theory that derives from its development, which can be used to understand and support learning (Cf. Tavares, Vieira, and Pedro 2017); the study aims to contribute to a renewed understanding of how to apply the Educational Design Research approach and to underline the importance of this approach in the educational field.
Data analysis and its impact description aimed to share the foundations of our research, revealing how the end-user’s participation was crucial to improving the innovative nature of the mobile app and its effectiveness and relevance, relating to our intention to promote students’ positive attitudes towards the human body, healthy eating and physical activity through the mobile app concept.
Students’ participation also allowed us to ground our research in reliable design principles and to bring educational research closer to the classroom practices. By adopting a participatory and user-centred design, we were able to simultaneously confront and ground our initial idea on the mobile app concept. For this reason, we support the idea that our framework can represent an opportunity to a new means of improving the range of outputs of Educational (Design) Research.
Our framework can also represent an opportunity for (educational) researchers and developers, in both the academia and industrial contexts, to develop educational mobile apps in a collaborative way, benefitting from the inputs of different stakeholders, namely, the educational solutions’ end-users. Furthermore, this framework can contribute to deepen participatory and user-centred approaches in educational research. Because our framework is quite flexible and comprehensive, we believe that it can be used in the development of other technological educational solutions.
Statements on open data, ethics and conflict of interest
The anonymised written and graphic productions are available open access on FigShare® (Tavares, Vieira, and Pedro 2019a). This study meets the necessary ethical requirements and does not include activities or results that pose safety problems for the participants. All data were treated confidentially, and the participants were treated anonymously in this study. The authors declare that there is no conflict of interest in the reported research.
References
Allen, M. (2007) Designing Successful e-Learning: Forget What You Know About Instructional Design and Do Something Interesting, John Wiley & Sons, Inc., Hoboken, NJ.
Amado, J. (2014) Manual de investigação qualitativa em educação (Handbook of Qualitative Research in Education), 2nd edn, Imprensa da Universidade de Coimbra, Coimbra. doi: 10.14195/978-989-26-0879-2
Amiel, T. & Reeves, T. (2008) ‘Design-based research and educational technology: rethinking technology and the research agenda’, Educational Technology & Society, vol. 11, no. 4, pp. 29–40, [online] Available at: www.jstor.org/stable/jeductechsoci.11.4.29
Anastasi, A. (1976) Psychological Testing, 4th edn, MacMillan, New York, NY.
Anderson, T. & Shattuck, J. (2012) ‘Design-based research: a decade of progress in education research?’, Educational Researcher, vol. 41, no. 1, pp. 16–25. doi: 10.3102/0013189X11428813
Barab, S. & Squire, K. (2004) ‘Design-based research: putting a stake in the ground’, The Journal of the Learning Sciences, vol. 13, no. 1, pp. 1–14. doi: 10.1207/s15327809jls1301_1
Beck, K. (2000) Extreme Programming Explained: Embrace Change, Addison-Wesley Longman Publishing Co., Boston, MA.
Ben-Zahia, M. & Jaluta, I. (2014) ‘Criteria for selecting Software development models’, in Global Summit on Computer & Information Technology, Institute of Electrical and Electronics Engineers (IEEE), Sousse, pp. 1–6. doi: 10.1109/GSCIT.2014.6970099
Bichelmeyer, B. (2005) ‘“The ADDIE model” – a metaphor for the lack of clarity in the field of IDT’, in AECT 2004 IDT Futures Group Presentations, eds E. Boling, IDT Record, Indiana, pp. 1–7, [online] Available at: http://www.unco.edu/CETL/sir/clt/documents/IDTf_Bic.pdf
Bodily, R., Leary, H. & West, R. E. (2019) ‘Research trends in instructional design and technology journals’, British Journal of Educational Technology, vol. 50, no. 1, pp. 64–79. doi: 10.1111/bjet.12712
Boehm, B. & Turner, R. (2003) ‘Observations on balancing discipline and agility’, Proceedings of the Agile Development Conference, 2003, Institute of Electrical and Electronics Engineers (IEEE), Salt Lake City, UT, pp. 32–39. doi: 10.1109/ADC.2003.1231450
Boyle, T. (2002) ‘Towards a theoretical base for educational multimedia design’, Journal of Interactive Media in Education, vol. 2, pp. 1–16, [online] Available at: www-jime.open.ac.uk/2002/2. doi: 10.5334/2002-2
Branch, R. & Kopcha, T. (2014) ‘Instructional design models’, in Handbook of Research on Educational Communications and Technology, 4th edn, eds J. Spector, et al., Springer Science+Business Media, New York, NY, pp. 77–97. doi: 10.1007/978-1-4614-3185-5
Brown, A. & Green, T. (2016) The Essentials of Instructional Design: Connecting Fundamental Principles with Process and Practice, 3rd edn, Taylor & Francis, New York, NY.
Chen, I. (2011) ‘Instructional design methodologies’, in Instructional Design: Concepts, Methodologies, Tools, and Applications, eds Information Resources Management Association, IGI Global, London, pp. 80–94.
Costa, F. (2013) ‘Designing educational multimedia resources’, in Media in Education, eds A. Moreira, O. Benavides & A. J. Mendes, Springer, New York, NY, pp. 29–40. doi: 10.1007/978-1-4614-3175-6_3
Creswell, J. W. & Creswell, J. D. (2018) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 5th edn, Sage Publications, Los Angeles, CA.
de Villiers, R. & Harpur, P. (2013) ‘Design-based research – the educational technology variant of design research: illustrated by the design of an m-learning environment’, Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference on – SAICSIT ’13, Association for Computing Machinery (ACM), East London, South Africa, pp. 252–261. doi: 10.1145/2513456.2513471
Denzin, N. & Lincoln, Y. (2003) Collecting and Interpreting Qualitative Materials, Sage Publications, Thousand Oaks, CA.
DGS (Directorate-General of Health of Portugal). (2017) Programas de Saúde Prioritários (Priority Health Programs), Portuguese Government, Portuguese National Health Service & Directorate-General of Health of Portugal, Lisbon, [online] Available at: https://www.sns.gov.pt/wp-content/uploads/2017/07/DGS_PP_MetasSaude2020.pdf
Guilford, J. & Fruchter, B. (1973) Fundamental Statistics in Psychology and Education, 5th edn, McGraw-Hill, New York, NY.
Hamzah, A. (2018) ‘The role of participatory design in mobile application development’, IOP Conference Series: Materials Science and Engineering 325 012011, IOP Publishing Ltd, Yogyakarta, pp. 1–5. doi: 10.1088/1757-899X/325/1/012011
Hartson, R. & Pyla, P. (2019) The UX Book: Agile UX Design for a Quality User Experience, 2nd edn, Morgan Kaufmann, Cambridge.
Hinostroza, J. & Mellar, H. (2001) ‘Pedagogy embedded in educational software design: report of a case study’, Computers & Education, vol. 37, pp. 27–40, [online] Available at: http://www.sciencedirect.com/science/article/pii/S036013150100032X. doi: 10.1016/S0360-1315(01)00032-X
International Organization for Standardization. (2010) Ergonomics of Human – System Interaction – Part 210: Human-Centred Design for Interactive Systems, ISO, Geneva.
Johnson, J. & Henderson, A. (2002) ‘Conceptual Models: begin by designing what to design’, Interactions, 9(1), pp. 25–32, [online] Available at: http://dl.acm.org/citation.cfm?id=503366. doi: 10.1145/503355.503366
Keates, S., Trewin, S. & Elliott, J. (2006) ‘Including users with motor impairments in Design’, in Encyclopedia of Human Computer Interaction, ed C. Ghaoui, Idea Group, Hershey, PA, pp. 317–323.
Kucirkova, N. (2017) ‘iRPD – a framework for guiding design-based research for iPad apps’, British Journal of Educational Technology, vol. 48, no. 2, pp. 598–610. doi: 10.1111/bjet.12389
Laubheimer, P. (2016) Wireflows: A UX Deliverable for Workflows and Apps, Nielsen Norman Group, [online] Available at: https://www.nngroup.com/articles/wireflows/
Leavy, P. (2017) ‘Chapter 5: qualitative research design’, in Research Design: Quantitative, Qualitative, Mixed Methods, Arts-Based, and Community-Based Participatory Research Approaches, ed P. Leavy, The Guilford Press, New York, NY, pp. 124–163.
Majgaard, G., Misfeldt, M. & Nielsen, J. (2011) ‘How design-based research and action research contribute to the development of a new design for learning’, Designs for Learning, vol. 4, no. 2, pp. 8–27. doi: 10.16993/dfl.38
Martins, I., et al. (2007) Colecção Ensino Experimental das Ciências: Educação em Ciências e Ensino Experimental – Formação de Professores (Experimental Science Education Collection: Science Education and Experimental Teaching – Teacher Training), Ministério da Educação e Ciência – Direcção-Geral de Inovação e de Desenvolvimento Curricular, Lisboa, [online] Available at: http://www.dge.mec.pt/guioes-didaticos-eb
Matković, P. & Tumbas, P. (2010) ‘A comparative overview of the evolution of software development models’, International Journal of Industrial Engineering and Management, vol. 1, no. 4, pp. 163–172, [online] Available at: http://www.iim.ftn.uns.ac.rs/casopis/volume1/ijiem_vol1_no4_4.pdf
Mayring, P. (2000) ‘Qualitative content analysis’, Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, vol. 1, no. 2, [online] Available at: http://nbn-resolving.de/urn:nbn:de:0114-fqs0002204
McKenney, S. & Reeves, T. (2012) Conducting Educational Design Research, Routledge, New York, NY.
McKenney, S. & Reeves, T. (2013) ‘Systematic review of design-based research progress: is a little knowledge a dangerous thing?’, Educational Researcher, vol. 42, no. 2, pp. 97–100. doi: 10.3102/0013189X12463781
McKenney, S. & Reeves, T. (2015) ‘Educational design and construction: processes and technologies’, in The Future of Ubiquitous Learning, eds B. Gros & M. M. Kinshuk, Springer, Berlin, pp. 131–151. doi: 10.1007/978-3-662-47724-3_8
Mirri, S., Roccetti, M. & Salomoni, P. (2018) ‘Collaborative design of software applications: the role of users’, Human-Centric Computing and Information Sciences, vol. 8, no. 6, pp. 1–20. doi: 10.1186/s13673-018-0129-6
Molenda, M. (2003) ‘In search of the elusive ADDIE model’, Performance Improvement, vol. 42, no. 5, pp. 34–36. doi: 10.1002/pfi.4930420508
Munassar, N. & Govardhan, A. (2010) ‘A comparison between five models of software engineering’, IJCSI International Journal of Computer Science Issues, vol. 7, no. 5, pp. 94–101, [online] Available at: http://www.ijcsi.org/articles/A-Comparison-Between-Five-Models-Of-Software-Engineering.php
Norman, D. A. (2013) The Design of Everyday Things, Revised an., Basic Books, New York, NY.
Nunnaly, J. & Bernstein, I. (1994) Psichometric Theory, 3rd edn, McGraw-Hill, New York, NY.
Office of Information Services. (2008) ‘Selecting a development approach’, [online] Available at: https://www.cms.gov/research-statistics-data-and-systems/cms-information-technology/xlc/downloads/selectingdevelopmentapproach.pdf
Ørngreen, R. (2015) ‘Reflections on design-based research: in online educational and competence development projects’, in Human Work Interaction Design, Work Analysis and Interaction Design Methods for Pervasive and Smart Workplaces, eds J. Nocera, et al., Springer International Publishing, Switzerland, pp. 20–38. doi: 10.1007/978-3-319-27048-7_2
Plomp, T. (2013) ‘Educational design research: an introduction’, in Educational Design Research, eds T. Plomp & N. Nieveen, SLO – Netherlands Institute for Curriculum Development, Enschede, 2nd edn, pp. 10–51.
Prior, L. (2014) ‘Content analysis’, in The Oxford Handbook of Qualitative Research, ed P. Leavy, Oxford University Press, Oxford, pp. 359–379.
Reimann, P. (2011) ‘Design-based research’, in Methodological Choice and Design: Scholarship, Policy and Practice in Social and Educational Research, eds L. Markauskaite, P. Freebody & J. Irwin, Springer Science+Business Media B.V., London, pp. 37–50. doi: 10.1007/978-90-481-8933-5
Richardson, L. (2003) ‘Writing: a methodology of inquiry’, in Collecting and Interpreting Qualitative Materials, eds N. Denzin & Y. Lincoln, Sage Publications, Thousand Oaks, CA, pp. 499–541.
Rito, A., Silva, A. & Breda, J. (2016) ‘MUNSI PROGRAMME – Integrated municipal programme for monitoring and addressing overweight and obesity in primary school children’, in Health by Numbers Portugal – Directorate-General of Health Scientific Journal Proceedings of the 1st Portuguese National Public Health Congress, eds P. J. Nogueira, et al., Directorate-General of Health, Lisboa, p. 46, [online] Available at: https://www.dgs.pt/documentos-e-publicacoes/portugal-health-by-numbers-special-issue-december-2016.aspx
Ritter, F., Baxter, G. & Churchill, E. (2014) Foundations for Designing User-Centered Systems, Springer-Verlag, London.
Robertson, T. & Simonsen, J. (2012) ‘Challenges and opportunities in contemporary participatory design’, Design Issues, vol. 28, no. 3, pp. 3–9. doi: 10.1162/DESI_a_00157
Ruparelia, N. & Hewlett-Packard Enterprise Services. (2010) ‘Software development lifecycle models’, ACM SIGSOFT Software Engineering Notes, vol. 35, no. 3, pp. 8–13. doi: 10.1145/1764810.1764814
Seel, N. M., et al. (2017) ‘Chapter 1: what is instructional design?’, in Instructional Design for Learning: Theoretical Foundations, eds N. Seel, T. Lehmann, P. Blumschein & O. Podolskiy, Sense Publishers, Rotterdam, pp. 1–17.
Shing, S. & Yuan, B. (2016) ‘Apps developed by academics’, Journal of Education and Practice, vol. 7, no. 33, pp. 1–9, [online] Available at: https://eric.ed.gov/?q=%22Mobile+apps+development%22&id=EJ1122634
Simonsen, J. & Hertzum, M. (2012) ‘Sustained participatory design: extending the iterative approach’, Design Issues, vol. 28, no. 3, pp. 10–21. doi: 10.1162/DESI_a_00158
Sommerville, I. (2011) Software Engineering, 9th edn, Pearson Education, Boston, MA.
Tavares, R., Vieira, R. & Pedro, L. (2017) ‘Preliminary proposal of a conceptual Educational Data Mining framework for Science Education: scientific competences development and self-regulated learning’, 2017 International Symposium on Computers in Education (SIIE), Institute of Electrical and Electronics Engineers (IEEE), Lisboa, NJ, pp. 1–6. doi: 10.1109/SIIE.2017.8259644
Tavares, R., Vieira, R. & Pedro, L. (2019a) ‘Creative writing and drawing activity performed by 4th grade primary-school students: figshare’, Figshare, Romania, London and Cambridge, doi: 10.6084/m9.figshare.8938052.v1.
Tavares, R., Vieira, R. & Pedro, L. (2019b) ‘Questionnaire: primary-school teachers’ conceptions about their knowledge and their educational practices in Science Education using digital educational resources’, Figshare, Romania, London and Cambridge, doi: 10.6084/m9.figshare.10013015.v1
Tchounikine, P. (2011) Computer Science and Educational Software Design: A Resource for Multidisciplinary Work in Technology Enhanced Learning, Springer Heidelberg Dordrecht, New York, NY. doi: 10.1007/978-3-642-20003-8
The Design-Based Research Collective. (2003) ‘Design-based research: an emerging paradigm for educational inquiry’, Educational Researcher, vol. 2, no. 1, pp. 5–8, [online] Available at: http://journals.sagepub.com/doi/abs/10.3102/0013189X032001005
Thorne, S. E. (2014) ‘Applied interpretive approaches’, in The Oxford Handbook of Qualitative Research, ed P. Leavy, Oxford University Press, Oxford, pp. 99–115.
University of Aveiro. (2019) History of the University of Aveiro, [online] Available at: https://www.ua.pt/#/en/historia
Wang, F. & Hannafin, M. (2005) ‘Design-based research and technology-enhanced learning environments’, Educational Technology Research and Development, vol. 53, no. 4, pp. 5–23. doi: 10.1007/BF02504682
World Health Organization. (2018) COSI Factsheet: Childhood Obesity Surveillance Initiative – Highlights 2015–17, [online] Available at: http://www.euro.who.int/__data/assets/pdf_file/0006/372426/WH14_COSI_factsheets_v2.pdf?ua=1
Zydney, J. & Warner, Z. (2016) ‘Mobile apps for science learning: review of research’, Computers & Education, vol. 94, no. 2016, pp. 1–17. doi: 10.1016/j.compedu.2015.11.001