ORIGINAL RESEARCH ARTICLE

Adaptive pedagogical strategies responding to emergency remote teaching – immediate responses of Hungarian primary school teachers

László Horváth* symbol

Institute of Education, ELTE Eötvös Loránd University, Budapest, Hungary

(Received: 15 December 2022; Revised: 8 March 2023; Accepted: 15 March 2023; Published: 11 April 2023)

Digital disruption is not a new phenomenon in education; however, it has become more prominent during the COVID-19 pandemic due to school closures and the related emergency remote teaching (ERT) period. Our study aims to explore the different pedagogical strategies that primary school teachers adopted during this period and determine how successful these strategies were in involving and engaging students. Altogether, 4028 teachers from 343 primary schools answered our online survey from all the regions of Hungary. The sample adequately represents the Hungarian primary school teacher population in terms of gender and age. We used cluster analysis and identified four clusters of pedagogical strategies; then, we used analysis of variance to explore how teachers’ digital competence and their ability to involve students in online learning varied across different clusters. Our analysis grasps the complexity of the issue, as it shows that two rather distinct strategies were both successful in involving students, and thus, there is no single solution best suited to digital learning. Overall, digitally competent teachers loosened the originally strict structure of education and provided more feedback, which proved to be an important element in successfully involving students in digital learning during ERT. The framework validated in our research can be used by policymakers and school administrators in different national and educational contexts, enabling them to understand the complexity of online teaching and learning. Furthermore, our results can offer some practical pointers for school teachers on how to combine different pedagogical strategies.

Keywords: COVID-19; ICT integration in education; teachers’ digital competence; cluster analysis.

*Corresponding author. Email: horvath.laszlo@ppk.elte.hu

Research in Learning Technology 2023. © 2023 L. Horváth. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Citation: Research in Learning Technology 2023, 31: 2978 - http://dx.doi.org/10.25304/rlt.v31.2978

Introduction

The education sector globally has experienced increased exposure to digital disruption in recent years, particularly during the emergency remote teaching (ERT) period that occurred as a response to COVID-19 and the related school closures. The central question of this paper is how the education sector can adapt and react to the changes and challenges of digital disruption. More specifically, we would like to examine how teachers adapt their teaching strategies by considering technology integration. This paper chose to analyse one country, examining empirical data from Hungary, focusing on the ERT period starting in March 2020.

Digital disruption is not a new phenomenon in education; however, it has increased during the ERT period and related school closures. Digital disruption can be defined as ‘the rapidly unfolding processes through which digital innovation comes to fundamentally alter historically sustainable logics for value creation and capture by unbundling and recombining linkages among resources or generating new ones’ (Skog, Wimelius, and Sandberg 2018, p. 432). It is especially interesting to examine the effects of digital disruption on the field of education because digital disruption has a two-fold impact. It not only affects the process of teaching and learning and the operation of schools, but it also affects the wider socio-economic contexts of students newly entering education and the world of work, as education must equip students with the necessary working skills.

In our article, we first describe the context of our study: the ERT period (in general) and the Hungarian education system (in particular, related to issues of digital disruption). The theoretical background of our article builds on the intersection of technology integration and innovation theory; therefore, we then introduce this related context. After establishing the theoretical background of our article, we present the research methodology. We employed a quantitative research strategy to explore how teachers adapt their teaching strategies to the challenges of digital disruption. We conducted an online survey among Hungarian primary school teachers (N = 4028) to gather their perceptions regarding changes in their pedagogical practice and digital competence. We first present the descriptive data; then, we try to establish typical pedagogical strategies via cluster analysis using the introduced variables. Finally, we present the details of the established clusters in relation to teachers’ digital competence (TDC) and their ability to involve students in the teaching and learning processes. To conclude, we discuss the relevance of our research, practical implications, possibilities to generalise the research, and limitations and further research opportunities.

The context of this study: ERT

There is empirical evidence from all around the world about how teachers have reacted to ERT. A narrative study by Nilsberth et al. (2021) emphasised how assessment in teaching shifted from a summative to a formative approach. Another study by Beattie, Wilson, and Hendry (2022) referred to the challenges of providing personalised learning experiences during ERT, while others focused on the realisation of professional development needs and the lack of teacher competencies (Castañeda-Trujillo et al. 2021; Yong et al. 2021).

The term ERT refers to a sudden change in the teaching context whereby stakeholders do not have enough time to plan for this transition or the necessary infrastructure and competencies (Barbour et al. 2020). ERT provides a unique context to explore how teachers integrate technology into their teaching and learning practices.

It is especially important to explore how teachers reacted to ERT in the case of Hungary. The 2016 Hungarian Digital Education strategy concluded that teachers rarely used digital tools: less than 20% of teachers used digital solutions in more than a quarter of their classes (Digital Success Programme 2016). In 2019, the European Commission (2019) found that Hungary is one of the lowest-ranking EU member states in terms of the percentage of classes supported by ICT tools. Suddenly, in March 2020, the situation changed from 1 week to the next, as schoolteachers were required to ensure the continuity of teaching via remote tools. This global phenomenon allows researchers to gauge teachers’ initial considerations and strategies related to technology integration. The circumstances for technology integration were not ideal; thus, these responses should be considered sudden responses in an emergency and not well-planned strategic approaches. Nevertheless, we believe that our data can provide an in-depth understanding of technology integration processes in education.

First, we will review existing empirical research on technology integration. After discussing the relevant dimensions of the phenomenon, we will focus on TDC as an important factor determining successful technology integration.

Technology integration through the lens of innovation theory

According to the National Center for Educational Statistics (2005), technology integration can be defined as ‘the incorporation of technology and technology-based practices into the daily routines, work, and management of an organization’. Our study focuses on daily routines and work rather than management. Daily routines and work are the core business of schools: the processes and organisation of teaching and learning. To encompass both the top-down and bottom-up approaches, we interpret technology integration through the lens of educational innovation theory. Using innovation theory also aligns with the broader theme of digital disruption, as it can be understood from the perspectives of the Schumpeterian notion of creative destruction, putting the dynamics of disruptive innovation (Christensen 1997) into focus.

The OECD (2019) defines innovation using the Oslo Manual’s definition (OECD 2018): ‘a new or improved product or process (or a combination thereof) that differs significantly from the unit’s previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)’. The broad definition of innovation encompasses various types of innovation. Furthermore, the innovation theory perspective distinguishes between the output, product, process, and nature of the phenomenon, where researchers’ focus could shift between the specifics of the given innovation and the characteristics of the agent responsible for creating/adapting the given innovation. The Oslo Manual (OECD 2018) defines the former as an ‘object’ and the latter as a ‘subject’ method for measuring innovation. Our analysis adopts a subject method as we do not focus on the specific digital innovations, technologies, and solutions adopted but on the agent (teachers) who are adopting the innovations and changing their practices and processes.

Innovation theory (which is predominant in public sector innovation studies; Arundel, Bloch, and Ferguson 2019) could shed light on the usually invisible (Fuglsang 2010) and hidden (NESTA 2007) everyday innovation processes (Lippke and Wegener 2014) driven by employees –in this case, teachers (Darsø and Høyrup 2012). Innovation processes can also be described using the notion of bricolage or tinkering; that is, those with lived experience of the process can create new structures through intrinsic actions that allow new ways of working (Fuglsang 2010). This latter perspective is especially relevant for analysing emergent strategies in ERT as teachers had little to no time to plan and prepare for this situation; therefore, they had to rely on their existing experiences to try to solve the challenge.

In the next section, we will review recent research results regarding technology integration in education to provide a theoretical base for our analysis of emergent strategies. We will examine technology integration through the lens of innovation theory, which was presented in this section.

Technology integration in education

The previously cited definition of technology integration does not encompass the complex nature of the phenomenon. Amiel, Kubota, and Wives (2016) also draw attention to the importance of context in the emergence, diffusion, and adoption of (digital) innovations in an educational setting: technology integration does not happen in isolation; it must be understood from a systemic perspective. Following an ecosystem approach, Amiel, Kubota, and Wives (2016) emphasise (after Nardi and O’Day 1999) that the focus should not be on the technology itself but on the activities executed with the help of the technology and the context of implementation.

It is important to investigate issues related to technology integration from a critical perspective as this integration does not always have positive results and can even lead to the reproduction of traditional methods (Chand, Deshmukh, and Shukla 2020). Drumm (2019) emphasises that teachers often rationalise their digital teaching with folk pedagogies (personalised mental models of teaching) and pseudo-theories of learning, which Drumm partly considers a hindrance but also sees as a starting point for teachers. Considering that teachers’ beliefs play an important role in successful technology integration, a failed or unsuccessful attempt at integration could reinforce traditional beliefs about the importance of ‘knowing content’ and about inequalities in learning (Chand, Deshmukh, and Shukla 2020). Technology integration without the necessary pedagogical considerations could easily facilitate behaviourist approaches to learning through automation and interactivity (Drumm 2019). Based on TALIS data, Gil-Flores, Rodríguez-Santero, and Torres-Gordillo (2017) highlight that high ICT use is not only linked with constructivist beliefs about learning but also a higher level of collaboration among teachers. Besides individual factors (digital competence, beliefs), group and organisational level factors also play an important role in explaining technology integration. School leaders play an important role in supporting teachers in technology integration by improving the ICT infrastructure and creating a positive learning culture (Vermeulen et al. 2017).

There are many models of technology integration, which change depending on the different elements they highlight. Among the most prominent ones are the technology acceptance model (TAM) and its updated version, the unified theory of acceptance and use of technology (UTAUT; Davis 1989; Davis, Bagozzi, and Warshaw 1989; Venkatesh et al. 2003). These models are based on different theories, including the theory of reasoned action, the theory of planned behaviour, the innovation diffusion theory, and the social cognitive theory. The models predict technology use by considering factors such as performance and effort expectancy, social factors, facilitating conditions, and moderating variables such as gender, age, and experience. There are also other models that are more pedagogically focused, such as the Substitution – Augmentation – Modification – Redefinition (SAMR) model (Puentedura 2003) or the Passive-Interactive-Creative – Replace-Amplify-Transform (PIC-RAT) model (Kimmons, Graham, and West 2020). However, these models do not have widespread, validated tools that would enable researchers to use them as valid and reliable instruments in explaining technology integration. Although these are prominent models widely used in the literature and educational practice, they do not address an important factor explaining technology integration in education: TDC.

Teachers’ digital competence

Falloon’s (2020) article summarises the different frameworks for understanding TDC: the Technological, Pedagogical, and Content Knowledge (TPACK) framework, the Distributed Thinking and Knowing, Engagement, Communication and Community, Knowledge Building (DECK) framework, the Critical Digital Literacy Framework (CDL), the Teacher Education Information Literacy (TEIL) framework, the International Society for Technology in Education (ISTE) standards, the Digital Competence for Educators (DigCompEdu) framework and the TDC framework. The majority of studies on TDC use one or a combination of these frameworks (see, for example, Akturk and Ozturk 2019; Cerratto Pargman, Nouri, and Milrad 2018; Chou et al. 2020).

This long list of different frameworks shows that there is ambiguity surrounding TDC in the scientific discourse. For this study, we selected a recently developed framework, the DigCompEdu, which has been, so far, validated in a few countries (Benali, Kaddouri, and Azzimani 2018; Cabero-Almenara et al. 2020; Ghomi and Redecker 2019). The framework describes six areas of TDC (using 22 indicators), considering both teachers’ professional and pedagogical competencies and learners’ competencies. The six areas are (1) professional engagement, (2) digital resources, (3) teaching and learning, (4) assessment, (5) empowering learners, and (6) facilitating learners’ digital competence (Redecker 2017).

Research methodology

This study uses the results of an online survey of Hungarian primary school teachers to explore the response of the school system to the digitally disruptive phenomenon caused by COVID-19 in the form of ERT. The aim of our study is to explore the different pedagogical strategies that primary school teachers used during this period. By analysing what strategies digitally competent teachers used and how successful these strategies were in involving and engaging students, we can discuss the effectiveness of these pedagogical strategies.

As our study is explorative, we used a quantitative research strategy to answer our research questions. We created a database using our online survey and analysed the data with Jamovi in alignment with our research questions. The following table (Table 1) describes the main aim of this study with its sub-questions, along with the data analysis method used to answer the research questions.

Table 1. Research questions and data analysis methods.
Aim: Explore how teachers adapt their teaching strategies considering technology integration.
RQ1: What are the different pedagogical strategies that primary school teachers adopted during the early days of ERT? Cluster analysis and a description of clusters along the main variables.
RQ2: What pedagogical strategies did digitally competent teachers adopt in their response to the sudden shift to online learning? Analysis of variance: Comparing the previously identified clusters (pedagogical strategies) with teachers’ digital competence.
RQ3: How successful was each pedagogical strategy in terms of student involvement in online learning? Analysis of variance: Comparing the previously identified clusters (pedagogical strategies) with the reported percentage of involved students.
ERT, emergency remote teaching.

The next subsections detail the sample and the instrument used in this study to answer our research questions, and then the study presents the results.

Sample

The survey was sent out to 450 primary schools in Hungary that participated in the EFOP-3.1.7 project (focusing on supporting primary schools’ capacity to combat early school leaving) in May 2020 (just a few months after the sudden shift to ERT). The final sample consists of 4028 teachers from 343 primary schools from all the regions in Hungary. In each responding school, nearly half (47%) of the teachers completed our survey, which is overall a high response rate (76% of schools participated).

The sample has the following demographic characteristics. The majority of the respondents are female (85.2%), which aligns with the proportion of women in the teacher population as a whole (85%; Hajdu et al. 2022). Hungarian public education is characterised by an aging teacher workforce: 47.7% of teachers are above 50 years old (Hajdu et al. 2022). In our sample, 43.7% of teachers are above 51 years old; therefore, we consider our sample as representative in terms of age. Respondents were also asked about their position and disciplinary focus (Table 2). Overall, our sample adequately characterises the Hungarian primary school teacher population in terms of gender and age.

Table 2. Background information on the sample.
Background variables N % of valid responses
Gender Female 3364 85.2
Male 584 14.8
Age 25 years old or younger 46 1.2
25–30 years old 161 4.1
31–40 years old 739 18.8
41–50 years old 1268 32.3
51–60 years old 1467 37.3
61–65 years old 235 6
65 years old or older 15 0.4
Position Teacher with managerial roles (e.g. principal, vice-principal, head of committee) 1271 31.7
Teachers without managerial roles 2551 63.6
Other (e.g. school psychologist, pedagogical assistant) 190 4.7
Disciplinary area based on the National Core Curriculum (multiple choice) Physical education and sports 1092 27.9
Lifestyle and practice 915 23.3
Informatics 319 8.1
Arts 1217 31
Our earth and our environment 740 18.9
Human and nature 819 20.9
Human and society 513 13.1
Mathematics 1484 37.9
Ethnic language and literature 71 1.8
Foreign languages 453 11.6
Hungarian language and literature 1626 41.5

Instrument

To assess the changes in the pedagogical practices and the organisation of education, we created a framework of items where respondents could indicate their chosen approach in their previous routine. Respondents could then indicate the direction of the change on a scale of 0–100. A point ranging from 0 to 49 meant a shift towards the element indicated on the left side of the scale, while a point between 51 and 100 indicated a shift towards the element on the right side of the scale. The following items were used:

Several pedagogical strategies exist depending on the combination of these items. Therefore, this study aimed to explore the different pedagogical strategies that primary school teachers in Hungary followed during the early days of ERT.

Additionally, it is important to understand how teachers’ digital competencies influenced their choice of pedagogical strategy during the ERT period. Another aim of our study was to explore what pedagogical strategies digitally competent teachers followed in response to the sudden shift to online learning. To assess TDC, we used DigCompEdu. The instrument measures the level of TDC by summing a set of items. We used the instrument by offering a 5-point Likert scale where respondents could assess each item and clarify how often they use it in their everyday practices. The instrument was translated into Hungarian by Horváth et al. (2020) for a teacher-educator sample; their translation was reworded in this study for a primary education sample.

Finally, student involvement in online learning during ERT was measured by teachers’ approximation of the percentage of students they were able to actively involve in the teaching and learning process. Thus, we wanted to explore how successful each pedagogical strategy was in terms of student involvement in online learning. This was measured separately for grades 1 to 4 (ISCED 1) and grades 5 to 8 (ISCED 2) as we felt that younger students could be considered a different target group as they are in the process of learning how to read, which could prove an important factor in using online learning tools and connecting to online lessons.

Results

Descriptive statistics

To provide an overall picture of Hungarian primary schools’ responses to ERT, we present the main descriptive statistics, the level of TDC, and the percentage of students involved in digital learning as reported by teachers.

First, the results regarding the changes in pedagogical practice in terms of the examined dimensions show (Figure 1) that the main strategy was to have a looser structure of classes with teachers providing more control than in face-to-face settings. Teachers provided feedback more frequently and reported using a summative type of assessment while shifting towards asynchronous solutions in the organisation of learning.

Fig 1
Figure 1. The mean and standard deviation of the scales (0–100) showing the shift in pedagogical strategies of Hungarian primary education teachers during the emergency remote teaching (ERT) period. Points lower than 50 mean a shift towards the left side of the scale, and points higher than 50 mean a shift towards the right side of the scale.

To measure TDC, the 22 items of the DigCompEdu (assessed on a 5-point Likert scale) were summed. The reliability statistic according to McDonald’s ω is high (0.930); therefore, there is internal consistency. We also checked the validity of the one-factor model with Confirmatory Factor Analysis (CFA) using a Diagonally Weighted Least Squares (DWLS) method and robust standard errors (considering the ordered nature of the items). The CFA yielded acceptable model fit measures (χ2 (203) = 6367; p < 0.001; CFI = 0.978; TLI = 0.975; RMSEA = 0.092; 95% CI [0.090; 0.094]; SRMR = 0.079) according to the thresholds proposed (CLI and TLI > 0.95; RMSEA < 0.10, and SRMR < 0.08) by Boateng et al. (2018). The summed-up TDC variable ranges from 22 to 110, with a mean of 88 and a standard deviation of 14.8. By calculating the levels of TDC according to the original instrument (similar to the levels of the Common European Framework of Reference for Languages), we can see that primary school teachers reported a high level of digital competency (45% [N = 1699] C1 – leader; 24.6% [N = 929] B2 – expert).

Finally, the teachers believed they were able to involve around 75%–80% of their students in digital learning in the ERT period. They reported higher success with involving younger children (ISCED 1; M = 79.5%; SD = 23%) as compared to ISCED2 students (M = 74.6%; SD = 20.6%).

Identifying typical pedagogical strategies responding to ERT (RQ1)

The main aim of our study is to explore pedagogical strategies that primary school teachers used immediately after shifting to ERT. In accordance with our first research question (RQ1: What are the different pedagogical strategies that primary school teachers followed during the early days of ERT?), we conducted a cluster analysis on the variables used to describe changes in pedagogical practices and the organisation of education. The following variables were included in the cluster analysis:

A K-means cluster analysis method was used with the Hartigan-Wong algorithm using Jamovi. By looking at the gap statistic (k) using the method developed by Kassambara and Mundt (2020), we established that the optimal number of clusters was four.

The cluster membership variable was saved to the database, providing opportunities to analyse and compare the different groups. The following diagram (Figure 2) shows the descriptive statistics of the cluster variables for each of the four clusters with the number of respondents for each cluster.

Fig 2
Figure 2. Means and standard deviation of the scales (0–100) for each cluster describing the shift in Hungarian primary education teachers’ pedagogical strategies during the emergency remote teaching (ERT) period. Points lower than 50 mean a shift towards the left side of the scale, and points higher than 50 mean a shift towards the right side of the scale.

Teachers in Cluster 1 loosened up the structure of learning, deviating from the usual schedule of 45-minute classes and opting for a more flexible arrangement. Respondents in this group reported that they acted in more supportive ways and provided more formative feedback than before. They also opted for an asynchronous type of teaching. Teachers in Cluster 2 adopted the opposite strategy of Cluster 1 in some regards. They tightened the structure, strictly sticking to the previous schedule, and were more controlling (as in providing more structure). While they did not change the organisation of learning nor the quantity of feedback provided, they still noted a shift towards summative assessments. Teachers in Cluster 3 were similar to those in Cluster 1 in that they opted for a loosened structure, but they accompanied this with stricter control in learning. They opted for a synchronous approach with a summative type of feedback and a stronger controlling role. Finally, Cluster 4 teachers provided a tighter structure of learning and emphasised asynchronous learning opportunities; they also noted they took on a more supportive teacher role with less, although formative, feedback.

Table 3 describes and highlights the key characteristics of each cluster to provide a comprehensive overview of each pedagogical strategy that we identified through our cluster analysis.

Table 3. Basic characteristics of pedagogical strategies employed by Hungarian primary school teachers during ERT.
Cluster no. Cluster 1 (N = 377) Cluster 2 (N = 546) Cluster 3 (N = 486) Cluster 4 (N = 453)
Structure of learning (tight – loose) tight tight tight tight
no change no change no change no change
loose loose loose loose
Role of teacher (support – control) support support support support
no change no change no change no change
control control control control
Quantity of feedback (less – more) less less less less
no change no change no change no change
more more more more
Organisation of learning (asynchronous – synchronous) asynchronous asynchronous asynchronous asynchronous
no change no change no change no change
synchronous synchronous synchronous synchronous
Type of assessment (formative – summative) formative formative formative formative
no change no change no change no change
summative summative summative summative
ERT, emergency remote teaching.

Once we identify the characteristics of the pedagogical strategies of Hungarian primary school teachers, it is worth it to explore the demographic composition of these clusters. Table 4 indicates the gender, age, and disciplinary composition of the clusters. Although the contingency table analysis shows there are no significant associations between variables (except for gender, which has a weak association), looking at the descriptive percentages provides insights regarding the composition of the clusters. According to the results, most male teachers are in Cluster 4, while Cluster 3 has the most senior teachers. Regarding disciplinary composition, Cluster 2 has the most PE teachers, while Cluster 1 has humanistic-oriented teachers, and Cluster 4 mostly hosts natural science-oriented teachers.

Table 4. Demographic composition of clusters.
Descriptive indicators Cluster1 Cluster2 Cluster3 Cluster4 Contingency table analysis
Gender (% of male teachers within clusters) 13.3 17.5 14.0 20.1 χ2 (3) = 9.46; p = 0.024
Cramer’s V = 0.0717
Age (% of teachers above 50 within clusters) 43.6 43.5 49.2 43.8 χ2 (18) = 14.6; p = 0.690
Cramer’s V = 0.0515
Disciplinary field (% of teachers teaching […] within clusters) Physical education and sports 24.20 29.80 25.10 28.20 χ2 (3) = 4.90; p = 0.179
Cramer’s V = 0.0517
Lifestyle and practice 23.60 21.10 20.50 21.20 χ2 (3) = 1.38; p = 0.711
Cramer’s V = 0.0274
Informatics 10.30 8 10.20 8.60 χ2 (3) = 2.36; p = 0.500
Cramer’s V = 0.0359
Arts 34.80 25.70 30.70 26.60 χ2 (3) = 10.6; p = 0.014
Cramer’s V = 0.0761
Our earth and our environment 16.80 18 18.60 19.90 χ2 (3) = 1.30; p = 0.728
Cramer’s V = 0.0267
Human and nature 23.60 17.60 21.70 19.40 χ2 (3) = 5–78; p = 0.123
Cramer’s V = 0.0562
Human and society 15.50 13.10 11.30 12.40 χ2 (3) = 3.43; p = 0.330
Cramer’s V = 0.0433
Mathematics 35.60 37 34.40 39.10 χ2 (3) = 2.31; p = 0.511
Cramer’s V = 0.0355
Ethnic language and literature 1.60 1.50 1.50 2.50 χ2 (3) = 1.88; p = 0.599
Cramer’s V = 0.0320
Foreign languages 14.10 11.50 12.50 12.40 χ2 (3) = 1.41; p = 0.704
Cramer’s V = 0.0277
Hungarian language and literature 42.10 39.60 40.30 38.40 χ2 (3) = 1.22; p = 0.747
Cramer’s V = 0.0259
Note: Largest percentages within clusters are highlighted in bold.

Our analysis successfully identified and described the pedagogical strategies that Hungarian primary school teachers followed in the early days of ERT. This enables us to answer our first research question (RQ1: What are the different pedagogical strategies that primary school teachers followed during the early days of ERT?), identifying and describing four pedagogical strategies based on different characteristics.

In the next section, these clusters represent groups of teachers following distinctive pedagogical strategies. According to our research aims, we explored what characterises digitally competent (based on the DigCompEdu) teachers’ behaviour and how effective these strategies were in involving and engaging students.

Assessing TDC and teachers’ ability to involve students in ERT (RQ2 and RQ3)

To answer our second research question (RQ2: What pedagogical strategies did digitally competent teachers adopt in their response to the sudden shift to online learning?), we investigated how digitally competent teachers acted during ERT and what kind of pedagogical strategies they followed. As discussed, we summed up the indicators of the DigCompEdu scale to get an overall indicator of TDC.

The sub-dimensions of the DigCompEdu framework show that Hungarian primary school teachers are advanced in selecting, creating, and managing digital resources (M = 4.35; SD = 0.68) and in professional engagement (M = 4.21; SD = 0.66), focusing on organisational communication, professional collaboration, reflective practice, and digital continuous professional development. There are areas that need development, such as teaching and learning (M = 3.89; SD = 0.79), where teachers reported lower levels of competence in teaching, guidance, collaborative learning, and self-regulated learning with digital solutions. Another area that needs improvement is facilitating learners’ digital competence (M = 3.77, SD = 0.90), meaning developing learners’ information and media literacy, communication, content creation, responsible use, and problem-solving.

The summed-up variable of DigCompEdu measuring TDC failed to demonstrate normality (Shapiro-Wilk test: W = 0.969; p < 0.001). Therefore, we used a non-parametric one-way analysis of variance (Kruskal-Wallis) to explore differences between the clusters of pedagogical strategies. We used the Dwass-Steel-Critchlow-Fligner method to compare the different groups pairwise. Figure 3 shows the results (descriptive data and statistically significant comparisons). According to the results of the Kruskall-Wallis test (χ2 (3) = 107.2; p < 0.001), there are significant and intermediate (ε2 = 0.058) differences between the clusters. Furthermore, all pairwise comparisons are significant. Teachers in Cluster 3 (M = 93.5; SD = 12.2) and Cluster 1 (M = 91.1; SD = 12.8) had higher levels of digital competence compared to those in Cluster 2 (M = 88.1; SD = 13.5) and Cluster 4 (M = 84.2; SD = 15.4).

Fig 3
Figure 3. Means and standard deviations of teachers’ digital competence (TDC) (measured by the sum of DigCompEdu scores) for each cluster, indicating statistically significant differences between groups according to the results of the Dwass-Steel-Critchlow-Fligner pairwise comparisons after the non-parametric ANOVA (Kruskal-Wallis).
*p < 0.05; **p < 0.01; ***p < 0.001.

Thus, in answer to our second research question (RQ2: What pedagogical strategies did digitally competent teachers adopt in their response to the sudden shift to online learning?), the results show that teachers who were more digitally competent reacted to ERT by loosening the structure but providing stricter control over the teaching and learning process and using summative assessments and synchronous learning opportunities. As all the DigCompEdu means for the clusters are between 82.5 and 100 points (C1 – leader domain), we must treat this conclusion with caution as it can mean, in reality teachers are basically on the same level of expertise. It is important to examine the effectiveness of teachers in each cluster in terms of their ability to involve and engage students during ERT periods, which is our third research question (RQ3: How successful can each pedagogical strategy be considered in terms of student involvement in online learning?).

As stated, primary school teachers reported that they could involve, on average, 79.5% (SD = 23%) of students on the ISCED1 level and, on average, 74.6% (SD = 20.6%) of students on the ISCED2 level. However, as before, both variables failed the prerequisite of normality (Shapiro-Wilk test: ISCED1: W = 0.835; p < 0.001; ISCED2: W = 0.917; p < 0.001); therefore, the non-parametric ANOVA was used. Figure 4 shows the descriptive statistics and the results of pairwise comparisons.

Fig 4
Figure 4. Means and standard deviations of the percentage of involved students (ISCED 1 and 2) in emergency remote teaching (ERT) (based on self-declaration of teachers) for each cluster, indicating statistically significant differences between groups according to the results of the Dwass-Steel-Critchlow-Fligner pairwise comparisons after the non-parametric ANOVA (Kruskal-Wallis).
*p < 0.05; **p < 0.01; ***p < 0.001.

The results and comparisons show that the pedagogical strategies used by teachers in Cluster 3 (ISCED1 M = 83.3%; SD = 20.3%; ISCED 2 M = 80.4%; SD = 18.9%) and Cluster 1 (ISCED 1 M = 85.2%; SD = 19.8%; ISCED2 M = 78.8%; SD = 18.5%) are successful. There are no significant differences between the percentage of involved students for Cluster 3 and Cluster 1; however, both percentages are significantly different from Cluster 2 (ISCED1 M = 77%; SD = 24%; ISCED2 M = 73%; SD = 20.1%) and Cluster 4 (ISCED1 M = 73.7%; SD = 25.6%; ISCED2 M = 68.7%; SD = 22.5%). The same pattern can be seen for both ISCED1 and ISCED2 students. However, on the descriptive level, it seems that the strategies applied by Cluster 1 teachers are slightly more successful for ISCED1 students, while strategies used by Cluster 3 teachers are slightly more successful for ISCED2 students.

Thus, in answer to our third research question (RQ3: How successful can each pedagogical strategy be considered in terms of student involvement in online learning?), it seems that the most successful approach for ISCED1 students is a loosened structure with more supportive teacher behaviour, more formative-type feedback, and more asynchronous learning opportunities. Instead, more control, summative feedback, and synchronous learning opportunities worked better for ISCED2 students.

Discussion and conclusion

The main aim of our study was to explore how the education sector adapted and reacted to the changes and challenges of digital disruption. We used the example of the COVID-19-induced ERT period to gauge teachers’ emergency responses. To identify the different pedagogical strategies that primary school teachers followed during the early days of ERT (RQ1), we clustered respondents based on their answers regarding the change in practice in the structure of education, the role of the teacher, quantity of feedback, organisation of learning and type of assessment. At the end of our analysis, four distinctive groups emerged.

To better understand the different strategies primary school teachers used, we examined what strategies digitally competent teachers followed (RQ2) and which strategy led to the highest percentage of students being involved in digital learning (RQ3). According to our results, the successful strategies involved teachers loosening the strict frameworks, teaching structure, and learning processes and providing more feedback than usual (common elements of Cluster 3 and 1). Digitally competent teachers (RQ2) exercised more control over the teaching and learning processes than usual and relied on synchronous teaching solutions and more summative assessments during digital learning (Cluster 3). The same strategies (more control, synchronous teaching solutions, and summative assessments) were also best at successfully involving students in the learning process (RQ3; Cluster 3 and 1). The study found that a pedagogical strategy that is more supportive and provides more formative assessment with dominantly asynchronous teaching methods can also be successful. It seems this latter strategy was more successful for ISCED1 students, while the former strategy (Cluster 3) was more successful for ISCED2 students.

Our results show that both synchronous and asynchronous strategies can be successful. In Cluster 1, teachers paired asynchronous learning with more support, while in Cluster 3, teachers paired the synchronous approach with more control. Our results emphasise that online teaching and learning is a complex issue that cannot be described using one dimension (e.g. asynchronous or synchronous) but requires different characteristics to be considered together. This finding is in line with Fernandez, Ramesh, and Manivannan (2022), who emphasised that synchronous learning was more stressful for students, but asynchronous learning put a greater burden on them. Therefore, our findings indicate that the complex pedagogical strategy linking teachers’ role (support/control) to their learning organisation (synchronous/asynchronous) can have important practical implications: teachers’ control behaviours can help manage and structure workload, while their supportive behaviours can help manage stress.

Our cluster analysis approach can be generalised and should be tested in other educational and national contexts as it seems to grasp the complexity of the issue. Adding to this complexity, the results’ implications are that two distinct strategies can both be successful, emphasising the importance of personalisation (reiterating the claims of Beattie, Wilson, and Hendry 2022), and that there is no one-size-fits-all approach to digital learning. The literature suggests that technology integration can lead to the reproduction of traditional methods (such as in the case of Cluster 3), as confirmed by Chand, Deshmukh, and Shukla (2020), but it can also link to constructivist beliefs (such as in the case of Cluster 1) as concluded by Gil-Flores, Rodríguez-Santero, and Torres-Gordillo (2017).

Overall, loosening the originally strict structure of education and providing more feedback (either summative or formative – Nilsberth et al. 2021, supported the effectiveness of the latter form) was important for digitally competent teachers. It also proved to be an important element in successfully involving students in digital learning during ERT.

Finally, this study has some limitations. Although our sample is representative of Hungarian primary school teachers, it is unclear how it can be generalised to other national and educational contexts. Although the international literature discussed in this paper provides some support for the applicability of our claims to other contexts, the results should be further explored. Additionally, as the study employed a quantitative approach using a self-reporting questionnaire, further data could ensure the validity of our results by conducting qualitative studies to understand the different pedagogical strategies in practice.

Overall, our research sheds light on the complexity of online teaching and learning during emergencies (in this case, the COVID-19 pandemic). Although teachers’ strategies were immediate responses to an external phenomenon, it seems there were deliberate pedagogical considerations behind these choices. Our research adds to the discourse on online teaching and learning by introducing a framework to identify different pedagogical strategies for online teaching and learning. Policymakers and school administrators can use the framework and underlying results to better understand their approach to online teaching and learning and introduce supporting policies that consider the complexity of the model. Finally, the results can be useful for school teachers as they can help them see the complex interrelations and different elements that a pedagogical strategy can contain, highlighting possible pitfalls and opportunities.

Funding

Project no. 134206 has been implemented with the support provided from the National Research, Development and Innovation Fund of Hungary, financed under the OTKA Postdoctoral Excellence funding scheme.

References

Akturk, A. O. & Ozturk, H. S. (2019) ‘Teachers’ TPACK levels and students’ self-efficacy as predictors of students’ academic achievement’, International Journal of Research in Education and Science, vol. 5, no. 1, pp. 283–294.
Amiel, T., Kubota, L. C. & Wives, W. W. (2016) ‘A systemic model for differentiating school technology integration’, Research in Learning Technology, vol. 24, p. 31856. doi: 10.3402/rlt.v24.31856
Arundel, A., Bloch, C. & Ferguson, B. (2019) ‘Advancing innovation in the public sector: aligning innovation measurement with policy goals’, Research Policy, vol. 48, no. 3, pp. 789–798. doi: 10.1016/j.respol.2018.12.001
Barbour, M., et al., (2020) Understanding Pandemic Pedagogy: Differences between Emergency Remote, Remote, and Online Teaching, Canadian ELearning Network, Halfmoon Bay. doi: 10.13140/RG.2.2.31848.70401
Beattie, M., Wilson, C. & Hendry, G. (2022) ‘Learning from lockdown: examining Scottish primary teachers’ experiences of emergency remote teaching’, British Journal of Educational Studies, vol. 70, no. 2, pp. 217–234. doi: 10.1080/00071005.2021.1915958
Benali, M., Kaddouri, M. & Azzimani, T. (2018) ‘Digital competence of Moroccan teachers of English’, International Journal of Education and Development using Information and Communication Technology, vol. 14, no. 2, pp. 99–120.
Boateng, G. O., et al., (2018) ‘A novel household water insecurity scale: procedures and psychometric analysis among postpartum women in western Kenya’, PLoS One, vol. 13, no. 6, p. e0198591. doi: 10.1371/journal.pone.0198591
Cabero-Almenara, J., et al., (2020) ‘Development of the teacher digital competence validation of DigCompEdu check-in questionnaire in the University Context of Andalusia (Spain)’, Sustainability, vol. 12, no. 5, p. 6094. doi: 10.3390/su12156094
Castañeda-Trujillo, J. E., et al., (2021) ‘Pedagogical strategies used by English teacher educators to overcome the challenges posed by emergency remote teaching during the COVID-19 pandemic’, Íkala, Revista de Lenguaje y Cultura, vol. 26, no. 3, pp. 697–713. doi: 10.17533/udea.ikala/v26n3a12
Cerratto Pargman, T., Nouri, J. & Milrad, M. (2018) ‘Taking an instrumental genesis lens: new insights into collaborative mobile learning’, British Journal of Educational Technology, vol. 49, no. 2, pp. 219–234. doi: 10.1111/bjet.12585
Chand, V. S., Deshmukh, K. S. & Shukla, A. (2020) ‘Why does technology integration fail? Teacher beliefs and content developer assumptions in an Indian initiative’, Educational Technology Research and Development, vol. 68, no. 5, pp. 2753–2774. doi: 10.1007/s11423-020-09760-x
Chou, C., et al., (2020) ‘Developing and validating a scale for measuring teachers’ readiness for flipped classrooms in junior high schools’, British Journal of Educational Technology, vol. 51, no. 4, pp. 1420–1435. doi: 10.1111/bjet.12895
Christensen, C. M. (1997) Innovator’s Dilemma, Harvard Business Review Press, Boston, MA.
Darsø, L. & Høyrup, S. (2012) ‘Developing a framework for innovation and learning in the workplace’, in Practice-Based Innovation: Insights, Applications and Policy Implications, eds H. Melkas & V. Harmaakorpi, Springer-Verlag, Berlin, pp. 135–154.
Davis, F. D. (1989) ‘Perceived usefulness, perceived ease of use, and user acceptance of information technology’, MIS Quarterly, vol. 13, no. 3, pp. 319–340. doi: 10.2307/249008
Davis, F. D., Bagozzi, R. P. & Warshaw, P. R. (1989) ‘User acceptance of computer technology: a comparison of two theoretical models’, Management Science, vol. 35, no. 8, pp. 982–1003. doi: 10.1287/mnsc.35.8.982
Digital Success Programme (2016) Digital Education Strategy of Hungary, Digital Success Programme, [online] Available at: https://digitalisjoletprogram.hu/files/d4/6b/d46bf17fdef3c9b5c1d38bd6db64c2a7.pdf
Drumm, L. (2019) ‘Folk pedagogies and pseudo-theories: how lecturers rationalise their digital teaching’, Research in Learning Technology, vol. 27. doi: 10.25304/rlt.v27.2094
European Commission (2019) 2nd Survey of Schools: ICT in Education: Hungary Country Report, European Commission, Luxembourg. doi: 10.2759/76901
Falloon, G. (2020) ‘From digital literacy to digital competence: the teacher digital competency (TDC) framework’, Educational Technology Research and Development, vol. 68, no. 5, pp. 2449–2472. doi: 10.1007/s11423-020-09767-4
Fernandez, C. J., Ramesh, R. & Manivannan, A. S. R. (2022) ‘Synchronous learning and asynchronous learning during COVID-19 pandemic: a case study in India’, Asian Association of Open Universities Journal, vol. 17, no. 1, pp. 1–14. doi: 10.1108/AAOUJ-02-2021-0027
Fuglsang, L. (2010) ‘Bricolage and invisible innovation in public service innovation’, Journal of Innovation Economics & Management, vol. 5, no. 1, pp. 67–87. doi: 10.3917/jie.005.0067
Ghomi, M. & Redecker, C. (2019) ‘Digital competence of educators (DigCompEdu): development and evaluation of a self-assessment instrument for teachers’ digital competence’, Proceedings of the 11th International Conference on Computer Supported Education, SCITEPRESS – Science and Technology Publications, Heraklion, pp. 541–548. doi: 10.5220/0007679005410548
Gil-Flores, J., Rodríguez-Santero, J. & Torres-Gordillo, J.-J. (2017) ‘Factors that explain the use of ICT in secondary-education classrooms: the role of teacher characteristics and school infrastructure’, Computers in Human Behavior, vol. 68, pp. 441–449. doi: 10.1016/j.chb.2016.11.057
Hajdu, T., et al., (2022) A közoktatás indikátorrendszere 2021 [Indicator System of the Hungarian Public Education 2021], Eötvös Loránd Kutatási Hálózat – Közgazdaság- és Regionális Tudományi Kutatóközpont, Közgazdaság-tudományi Intézet, Budapest.
Horváth, L., et al., (2020) ‘Tanárképzők digitális kompetenciájának mérése – a DigCompEdu adaptálása a hazai felsőoktatási környezetre [Measuring the digital competence of teacher educators – adapting the DigCompEdu in Hungarian higher education]’, Neveléstudomány, vol. 2020, no. 2, pp. 5–25. doi: 10.21549/NTNY.29.2020.2.1
Kassambara, A. & Mundt, F. (2020) factoextra: Extract and Visualize the Results of Multivariate Data Analyses, [online] Available at: https://CRAN.R-project.org/package=factoextra
Kimmons, R., Graham, C. R. & West, R. E. (2020) ‘The PICRAT model for technology integration in teacher preparation’, Contemporary Issues in Technology and Teacher Education, vol. 20, no. 1, [online] Available at: https://citejournal.org/volume-20/issue-1-20/general/the-picrat-model-for-technology-integration-in-teacher-preparation
Lippke, L. & Wegener, C. (2014) ‘Everyday innovation – pushing boundaries while maintaining stability’, Journal of Workplace Learning, vol. 26, no. 6/7, pp. 376–391. doi: 10.1108/JWL-10-2013-0086
Nardi, B. A. & O’Day, V. (1999) Information Ecologies: Using Technology with Heart, MIT Press, Cambridge, MA.
National Center for Educational Statistics (2005) Forum Unified Education Technology Suite, National Center for Educational Statistics, [online] Available at: https://nces.ed.gov/pubs2005/tech_suite/index.asp
NESTA (2007) Hidden Innovation. How Innovation Happens in Six ‘Low Innovation’ Sectors, London, [online] Available at: https://media.nesta.org.uk/documents/hidden_innovation.pdf
Nilsberth, M., et al., (2021) ‘Digital teaching as the new normal? Swedish upper secondary teachers’ experiences of emergency remote teaching during the COVID-19 crisis’, European Educational Research Journal, vol. 20, no. 4, pp. 442–462. doi: 10.1177/14749041211022480
OECD (2018) The Measurement of Scientific, Technological and Innovation Activities, Oslo Manual 2018, 4th edn, OECD, Paris. doi: 10.1787/24132764
OECD (2019) Measuring Innovation in Education 2019 What Has Changed in the Classroom? OECD, Paris. doi: 10.1787/9789264311671-en
Puentedura, R. R. (2003) ‘A matrix model for designing and assessing network-enhanced courses’, Hippasus [Preprint], [online] Available at: http://www.hippasus.com/resources/matrixmodel/
Redecker, C. (2017) Digital Competence of Educators, ed Y. Punie, Publications Office of the European Union, Luxembourg.
Skog, D. A., Wimelius, H. & Sandberg, J. (2018) ‘Digital disruption’, Business & Information Systems Engineering, vol. 60, no. 5, pp. 431–437. doi: 10.1007/s12599-018-0550-4
Venkatesh, V., et al., (2003) ‘User acceptance of information technology: toward a unified view’, MIS Quarterly, vol. 27, no. 3, p. 425. doi: 10.2307/30036540
Vermeulen, M., et al., (2017) ‘The role of transformative leadership, ICT-infrastructure and learning climate in teachers’ use of digital learning materials during their classes: teachers’ use of digital learning materials’, British Journal of Educational Technology, vol. 48, no. 6, pp. 1427–1440. doi: 10.1111/bjet.12478
Yong, K. L., et al., (2021) ‘Challenges in emergency remote teaching among Malaysian public elementary school teachers’, International Journal of Emerging Technologies in Learning (iJET), vol. 16, no. 24, pp. 74–90. doi: 10.3991/ijet.v16i24.27453