Identifying online communities of inquiry in higher education using social network analysis

Research in Learning Technology 2018. @ 2018 Shazia K. Jan. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.


Introduction
Since its inception, the community of inquiry (CoI) framework (Garrison, Anderson, and Archer 2000) has been applied extensively for practice and research in online and blended learning (Garrison and Arbaugh 2007;Kineshanko 2016).Reporting on research between 2000 and 2011, Halverson et al. (2013) state, 'the Community of Inquiry framework seems to be one of the most utilized theories for blended learning… ' (p. 24).Prior to the development of the CoI survey (Arbaugh et al. 2008), research using the CoI framework relied exclusively on extensive and time-consuming qualitative analysis of online discourse transcripts between participants (lecturers/ tutors and students).Since the introduction of the CoI survey, a mix of qualitative and quantitative analysis has been used for investigating CoIs.More recently, social network analysis (SNA) has also been applied as an analytical tool in conjunction with other methodologies for exploring CoIs; however, application of SNA has been limited and lacks appropriate pedagogical grounding (Jan and Vlachopoulos 2018).The importance of qualitative content analysis in CoI research cannot be overstated; however, 'there is a need to refine research methodologies for effective assessment of things within a CoI such as group cohesion, inquiry progress and direction' (Garrison 2017, p. 165).Assessing a CoI over time through qualitative content analysis is difficult and challenging (Jokismovic et al. 2014); however, such an evaluation is required for formative diagnostics, timely intervention (Garrison 2017) and response to emergent conditions during a learning activity (Bower 2017).
Cognisant of this need, and recognising the untapped potential of SNA, the integrated methodological framework (IMF) (Jan and Vlachopoulos 2018) was developed to allow for structural exploration and identification of communities of learning in higher education online learning.The IMF came about from findings of a systematic literature review (Jan, Vlachopoulos, and Parsell 2018) conducted in search of studies that integrate SNA with community-based pedagogical frameworks, namely the CoI and CoP frameworks.The IMF embeds SNA in structural components of a CoI and CoP and allows for identification of communities of learning at the whole-network (macro) and individual (micro) level, thereby achieving the dual purpose of theoretically grounding SNA constructs and providing a means to qualitatively assess a CoI and CoP.The IMF has been applied in (Jan and Vlachopoulos 2018) and (Jan and Vlachopoulos in press) and further testing is ongoing.
The goal of this article is to demonstrate the effectiveness of the IMF in identifying an online CoI at different points in time during a course designed with the intention of engaging students in a CoI.The case study consists of two offerings of the same online course over successive semesters with slight variations in course design.By comparing findings from each offering, the case study aims to validate the capacity of the IMF to capture the impact of the different learning designs on the formation of a CoI, thereby proving to be an effective framework with practical applications for research, assessment, diagnostics and intervention.The article begins by providing an overview of the CoI framework, research methodologies commonly used in investigating a CoI and an explanation of the structural characteristics of a CoI.An outline of the IMF and research questions follow this.Finally, the case study is presented and findings discussed.

Theoretical framework
The CoI framework is based on the collaborative constructivist view of teaching and learning which situates learning in the interplay between social and individual production of knowledge.With its roots in Dewey's (1859Dewey's ( -1932) ) ideas on critical thinking, collaborative learning and practical inquiry, and Vygotsky's (1978) view of learning as a transaction between individuals and society, the CoI framework was developed as a guide for online pedagogical practices and research (Garrison 2017).The CoI framework is a learning-centred, process-based model driven by the continuous interactions between three intersecting presences: social presence (SP), teaching presence (TP) and cognitive presence (CP).SP is defined as 'the ability of participants in a community of inquiry to project themselves socially and emotionally as "real" people…' (Garrison, Anderson, and Archer 2000, p. 94).CP is 'the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse' (Garrison, Anderson, and Archer 2000, p. 89), and TP is described as a presence that 'manages the environment and focuses and facilitates learning experiences' (Garrison and Kanuka 2004, p. 98).Each presence comprises of a sequence of interdependent stages that interact progressively and create the learning experience.In the seminal article 3 (page number not for citation purpose) introducing the CoI framework, Garrison, Anderson and Archer (2000) present a coding template with indicators for identifying SP, CP and TP in text-based communication.Along with this, three other supporting publications (Anderson et al. 2001;Garrison, Anderson, and Archer 2001;Rourke et al. 1999) form the crux of the CoI framework.
Prior to the development of the CoI survey, research based on the CoI framework was predominantly qualitative in nature and focused on individual presences rather than the entire framework (Arbaugh et al. 2008).Much of the earlier research on the framework centred on defining the structure of the three presences.This later shifted to understanding the relationships between the presences and then to investigating intra-presence dynamics (Garrison 2017).In a thematic synthesis of CoIbased empirical studies published between 1999 and 2014, Kineshanko (2016) found that the largest percentage (39%) of the 329 artefacts examined in detail were on one or two specific presences.The inter-relationships between SP, TP and CP are complicated, not yet fully understood, and are the subject of ongoing research.Garrison (2017) consolidates the current state of knowledge regarding the dynamics between the presences.Summarising, research confirms that SP is an integral precursor to CP which includes collaboration and critical discourse, and CP is enhanced and sustained when SP is established.SP is also the foundation that sustains the community after it has been formed by functionalities that lie within TP, and TP is necessary to sustain participation.A CoI must involve full and open communication as over time, high levels of SP are replaced by TP and CP as participants assume different roles and responsibilities.Each participant in a CoI embodies each presence, and the presences evolve mutually.
In terms of the structural characteristics of a CoI, group cohesion or degree of interactions between participants is a component of SP which is always present in a CoI (Garrison 2017); therefore, SP is the backbone of the community.Hence, an ongoing assessment of a CoI must include examination of engagement between participants in the network of online interactions.If group cohesion is a component of SP, it can be reasonably assumed that the overall density of the interactional network represents the level of SP in a CoI.Recent studies (Shea and Bidjerano 2010;Tirado, Hernando, and Aguaded 2015) have validated this assumption.Therefore, as a starting point in the assessment and identification of a CoI, the overall configuration of the interactional network is taken to be representative of the degree and distribution of SP which is the underlying presence of CP and TP as well.

Analytical framework
Figure 1 shows the integrated methodological framework (IMF) for structural exploration and identification of communities of learning in higher education online learning using SNA as the key methodology.The framework provides pedagogical grounding to SNA by embedding SNA constructs within the structural components of the CoI and CoP frameworks.Additionally, the IMF includes the support of selective qualitative analysis which may or may not be required depending on the context and depth of investigation.Specific SNA constructs have been selected for inclusion in the IMF based on their appropriateness with parallel structural components of a CoI and CoP.Note that the IMF is flexible and adaptable to different contexts (discussion forums, blogs, wikis, etc.) and should be interpreted accordingly.For further details including definitions of SNA constructs, development and application of the IMF, please see (Jan and Vlachopoulos 2018).
As shown in Figure 1, network cohesion, centralisation, core-periphery structure, number and size of components and cliques, reciprocity and transitivity measures are used to structurally identify a CoI.In a CoI, one would expect to see dense networks (signifying SP) throughout with relatively equal distribution of ties and key nodes across the network; therefore, the centralisation would be low.There would be no clear core-periphery structure and participants would ideally be connected within one large component.The number of isolates would be low representing full communication.There would be large number of cliques representing closely-knit groups with high mutual exchange (reciprocity) and high transitivity signifying an open, nonrestrictive network in terms of information flow.The key assumption underlying this interpretation is that since SP is the foundation for CP and TP, and the configuration of connections in a network represent SP, the balanced distribution of SP is critical for the existence of a CoI.

Research questions
The key objective of this article is to demonstrate the effectiveness of the IMF in identifying an online CoI.By applying the IMF and comparing findings from two successive offerings of the same online course with slight variations in design, the case study aims to validate the IMF as a valuable methodological framework for structurally exploring and identifying a CoI.For each offering of the course, the case study uses the IMF to determine the following: Can a CoI be structurally identified at different points in time during the course?Does the design of the course lead to the formation of a CoI at the end of the course?What practical pedagogical implications can we draw from the findings?The study has been approved by the University's Ethics Review Committee, reference number 5201600892.

Context of the study
The study was conducted on an online course in the Faculty of Arts over semester 2 (S2) and semester 3 (S3) at a large metropolitan university in Australia in 2017-2018.Moodle was used as the LMS for the course.The course curriculum, content, learning tools and activities were identical in S2 and S3.To inculcate a sense of community amongst the students, the lecturer chose weekly discussion forums as the preferred tool for online interaction of the students (Andresen 2009).To encourage online engagement and generate extrinsic motivation (An, Shin, and Lim 2009;Rovai 2007), participation in the discussion forums was allocated 20% of the final grade.The remaining 80% was divided between a short paper, quizzes and a final essay.The weekly discussion activity comprised of 5-6 guided and facilitated discussion forums.Students were asked to choose 1 forum each week and comment briefly on the question posed by 11:00 pm on Friday.Students could only see posts by other students after posting their own comment.Students were instructed to discuss points raised by fellow students and were also provided with a detailed rubric for participation, a good design practice for promoting engagement (Garrison 2017).Although the content, learning outcomes, assessments and learning tools used were identical in S2 and S3, the two courses differed in four aspects discussed below and summarised in Table 1.
• Cohort: In S2, a total of 138 students saw the course to completion.Of the 138, 90 students were enrolled undergraduates at the University, whereas 48 students enrolled via Open Universities Australia (OUA).In S3, of the 106 students that initially enrolled, 99 students saw it to completion.All students were enrolled undergraduates at the University.• Duration: S2 ran over a period of 13 weeks, whereas S3 ran over a 6-week period.
• Participation rubric: The participation rubric in S3 included response to posts as a key criterion, whereas the one in S2 did not.• Facilitation: Prior to commencement of the course, the tutor facilitating the discussion forums in S3 was instructed to reach out to students and be proactive, whereas the tutor facilitating the discussion forums in S2 was not provided any instructions.

Methodology
Application of the IMF is a multi-stage process comprising of four successive stages (see Jan and Vlachopoulos 2018).This study includes stages 1 to 3 of the IMF.Stage 4, which involves selective qualitative analysis, is not applied as it is not required to achieve the objective of the study.The following sub-sections describe the three stages of application of the IMF within the context of the case study.

Stage 1: preparation of data
Interaction data from the discussion activities in S2 and S3 was extracted from the LMS at the end of each week and coded into matrices in UCINET 6.0 (Borgatti, Everett, and Freeman 2002) to generate directed and weighted networks.The size of the networks is determined by the number of nodes, that is, the number of students and tutor/lecturer.The direction of a tie indicates the initiator and receiver, and the weight represents the number of interactions between each node (Wasserman and Faust 1994).For each week, specific SNA measures were calculated in UCINET 6.0, and radial network diagrams were generated in Social Network Visualizer 2.3 (Socnetv 2017).The radial diagrams are based on degree centralities of the nodes.Degree centrality is the number of ties to other nodes in the network (Wasserman and Faust 1994).Weights of the edges are taken into account when computing distances between the nodes; therefore, the closer the nodes are to each other in the network diagrams (Table 2 and Table 3 below), the shorter the distance between them (Socnetv 2017).The thickness of the lines connecting nodes in the diagrams represents the weight of the tie, that is, the number of interactions between two nodes.

Stage 2: static and temporal analysis
Static analysis refers to the analysis of cross-sectional networks.Cross-sectional networks are networks generated at a certain point in time, for instance, at the end of week 1.Temporal analysis refers to structural comparison of successive crosssectional networks.In stage 2, preliminary conclusions are made from overall examination of the network diagrams generated in stage 1.Detailed examination of the corresponding SNA measures then confirms these conclusions.For instance, if a CoP is suspected, successive cross-sectional networks are compared to ascertain the presence of the process of legitimate peripheral participation (LPP) which is the cornerstone of a CoP.Simply stated, LLP signifies learning as students successively move from the periphery towards the centre of the community as experts (tutor or lecturer) move outwards.A network that resembles a CoP would present with a clear and dynamic core-periphery structure.On the contrary, if a CoI is suspected, one would not expect to see a clear core-periphery structure and evidence of LPP and instead would see a more equally distributed network.Therefore, to confirm preliminary conclusions, further in-depth analysis was undertaken.

Stage 3: aggregate analysis
Aggregate analysis refers to the examination of the cumulative network of all the interactions over the entire duration of the course.The aggregate analysis includes cumulative SNA measures as well as cumulative radial network diagrams from S2 and S3.The cumulative networks as a stand-alone do not reveal temporal structural dynamics of community formation and evolution; however, they present an overall snapshot of the structure of the community formed, if any, under the influence of the learning design.As such, analysis of the cumulative networks was used for confirmation or rejection of the conclusions drawn in stage 2.

Findings
As stated earlier, the IMF needs to be adapted and interpreted based on the context of the investigation.Therefore, before presenting findings from the study, it is imperative to explain the adaptation required to the context.In this study, the design of the discussion activities in both S2 and S3 was such that one would not expect to see deeply nested threads within the weekly discussion networks as students were to select only 1 out of 4-6 discussion forums each week.So, two very active students who choose two different forums in a week might not be connected directly or indirectly to one another unless another student or the tutor engages across two different forums within that week.Therefore, it would not be a surprise if the weekly networks appear clunky or disconnected and show low reciprocity (mutual exchange) and transitivity (e.g. if A->B, B->C then C->A).Based on the view that connections formed amongst participants are non-transient paths that represent potential for information flow, for the sake of brevity, cross-sectional networks comprising of a number of weeks of discussion activity are used in the investigation.Also, since the goal here is to identify the type of community formed at certain points in S2 and S3 and at the end of the semesters, the analysis is restricted to successive cross-sectional networks as opposed to successive cumulative cross-sectional networks.If the objective was to explore community evolution over time, successive cumulative networks would need to be examined.Having set the stage for the forthcoming analysis, detailed examination of the interactional data obtained from the discussion activities in S2 and S3 follows.

Session 2 (S2)
Table 2 shows successive cross-sectional networks from S2 over a 9-week period, that is, from week 5 to week 13 (stage 1).The networks consist of 139 nodes (138 students,1 tutor).All weeks included a discussion activity except for week 6; therefore, there were a total of 8 discussion activities (4-6 discussion forums within each).Note that discussion activities from week 1 to week 4 were not included in the analysis because of inconsistency in the size of the networks since the end of week 4 was the cut-off for dropping out of the course.Since there were a number of drop-outs, to avoid irregularity, discussion networks prior to week 4 were excluded.In the network diagrams in Table 2, students who did not engage in the discussion activities, that is, students who did not post to the discussion forums at all and students who did not either receive a response to their post or respond to another post appear as isolates on the extreme periphery.The tutor is represented by the node in green.
Overall examination of the network diagrams in Table 2 (stage 2) shows that a small percentage of students engaged in the discussion activities.This is indicated by the large number of isolates on the periphery of the weeks 5-9 and weeks 10-13 networks.In weeks 5-9, the tutor is placed between the centre and periphery of the network indicating his or her higher level of activity as compared to majority of the students; however, in weeks 10-13, the tutor moves out to the periphery indicating a lack of engagement.Both networks visually appear low density and with an unequal distribution of ties.Judging by the network diagrams alone, based on the IMF, the networks do not show a resemblance to a CoI.Therefore, visual inspection of the two successive cross-sectional networks does not indicate the formation of a CoI between weeks 5-9 and weeks 10-13.
Further analysis using the corresponding SNA measures in Table 2 confirms the preliminary conclusion of an unidentified community in both cross-sectional networks.In weeks 5-9, 109 (79%) of the 138 students posted to the discussion forums; however, only 31 (22%) engaged in the discussion forums within 4 components.In weeks 10-13, 95 (69%) posted to the forums and 32 (23%) engaged in the discussion forums within 3 components.Therefore, a small percentage of students participated in the discussion activities due to which both networks have a significantly low average degree of less than 1.Both networks have only 3 cliques which indicates close connections (repeated interactions) between a small number nodes (students and/or tutor) while the remaining nodes are loosely connected.In both networks, there are only three prominent nodes, the core nodes.The networks do not depict a core-periphery structure which is re-affirmed by the low centralisation of both networks.The relatively high reciprocity (19.4%) in weeks 5-9 is indicative of the mutual exchange between the core nodes.The reciprocity drops to 5.4% in weeks 10-13, possibly due to reduced engagement of the tutor.The low transitivity of both networks re-affirms the weak connections within the networks.The low density, unequal distribution of ties, small number of cliques, a low degree of mutual exchange and low transitivity lead to the conclusion that neither network resembles a CoI.Therefore, the static and temporal analysis of the cross-sectional networks confirms the earlier conclusion of unidentified communities.Finally, the aggregated network (stage 3) and corresponding SNA measure shows that from weeks 5-13, only 45 (33%) of the students engaged in the discussion forums within 3 components.The tutor is placed inwards due to relatively high activity in weeks 5-9.The average density, centralisation and number of cliques remain low.The core still contains only 3 nodes of which one is the tutor.The overall low reciprocity and transitivity again testify to the weak connections in the network.In conclusion, the structure of the aggregate network also does not bear resemblance to the structural characteristics of a CoI; therefore, the overall community formed cannot be classified as a CoI.

Session 3 (S3)
Table 3 shows successive cross-sectional networks over a 6-week period in S3 (stage 1).The weeks 1-2 network consists of 107 nodes (106 students, 1 tutor).The weeks 3-5 network comprises of 100 nodes (99 students, 1 tutor) since in week 3, 9 students dropped out of the course while 2 new students joined the course.Each week consisted of 2 discussion activities (4-6 discussion forums within each) with the exception of week 3 which had 1 discussion activity only and the final week, that is, week 6 in which there was no discussion activity.Therefore, there were a total of 9 discussion activities: weeks 1-2 (4 discussion activities) and weeks 3-5 (5 discussion activities).Again, the tutor is represented by the green node in the network diagrams.Same as in the networks from S2, the nodes on the extreme periphery represent isolates, that is, students who either did not post to the discussion forums at all or did not receive or post a response to others.Overall examination of the network diagrams of the successive weeks 1-3 and weeks 3-5 networks in Table 3 (stage 2) shows engaged and active networks even though there a sizeable number of isolates on the peripheries.The tutor appears to be active in both networks; however, he or she is placed towards the periphery which implies repeated interactions with certain students only.Within the active nodes, the networks appear dense with a reasonably equal distribution of connections.Furthermore, the networks show low degree of centralisation, and there is no visible core-periphery structure.Therefore, judging by the network diagrams alone, according to the IMF, both successive networks structurally resemble a CoI.
analysis using the corresponding SNA measures in Table 3 confirms the preliminary conclusion of a CoI in both cross-sectional networks.In weeks 1-2, 89 (84%) of the 106 students posted to the discussion forums; however, as indicated by the size of the component, only 46 (43%) engaged in the discussions.In weeks 3-5, 81 (82%) of the 99 students posted to the discussion forums, and 52 (53%) engaged in the discussions.Regardless of the proportion of engaged students, the average degree of both networks is greater than 1 indicating a sizeable level of interactions.Even though the centralisation of the networks increases from 1.80% to 4.01%, it remains low.There are only 2 nodes in the core in weeks 1-2, one of which is the tutor.The weeks 3-5 network has a large core consisting of 13 nodes including the tutor again.Both networks have a considerable number of cliques, which indicates tightly-knit groups maximally connected to one another.The number of cliques increases from 18 to 25 between the two networks indicating stronger ties over time.The weeks 1-2 network has very low reciprocity (2.1%) meaning students and tutor are not responding to one another.The reciprocity increases to (8.7%) in weeks 3-5 indicating greater mutual exchange; however, it is still low.In line with the number of cliques, the transitivity increases from one network to the other but remains on the lower side meaning the networks are restrictive in terms of information flow.The low reciprocity and transitivity are a feature of the design of the discussion activity.Considering the reasonable average degree, low centralisation, equal distribution of connections, high number of cliques and evidence of mutual exchange, the static and cross-sectional analysis structurally identifies both cross-sectional networks as CoIs.
Finally, the aggregate network (stage 3) comprises of 109 nodes (108 students, 1 tutor) since it includes all students from the beginning to the end of the course.Overall, a total of 90 (83%) students posted to the discussion forums of which 61 (56%) engaged in the discussion activities as per the size of the one large component.The network has low centralisation and a large core indicating fairly equal distribution of connections.There is no evidence of a core-periphery structure.The large number of cliques represents tightly-knit groups.As expected, the reciprocity and transitivity are low.Considering the high average degree, equal distribution of connections, low centralisation, large number of cliques and presence of some mutual exchange, based on parameters in the IMF, the aggregate analysis confirms the structural presence of an overall CoI at the end of the course.

Discussion
The key objective of this article was to assess the effectiveness of the IMF in structurally exploring the formation of and identifying online CoIs.The IMF was applied to the discussion activities of two successive offerings of an online course with slight variations in the design in each offering.It was found that in S2, participants 11 (page number not for citation purpose) did not come together to form the structure of a CoI either during or at the end of the course.However, in S3, a CoI was structurally identified during as well as at the end of the course.The disparity in findings from S2 and S3 can presumably but not exclusively be attributed to design differences in the two offerings.The key differences in the design of the S2 and S3 course included the cohort, facilitation technique, rubric and duration.Although the impact of each of these differences cannot be isolated, the case study corroborates findings from previous research reporting the impact of time (Akyol, Vaughan, andGarrison 2011), rubric (Swan et al. 2007) and facilitation (Garrison 2017) on the development of a CoI.An additional factor contributing to the differences could be that the cohort in S2 included OUA students, whereas the cohort in S3 included university students only.The assumption being that the OUA students might not be as invested in engaging online as university students enrolled in a programme.At this point, further investigation needs to be conducted to verify this assumption.Regardless, the case study validates the IMF as an effective and valuable methodological framework for structurally exploring and identifying a CoI without needing extensive qualitative analysis.The findings verify the IMF's capability to capture and reflect variations in learning design, thereby allowing for ongoing evaluation of a CoI for assessment, diagnostics and intervention purposes.
Given the potential of the IMF, the framework promises useful practical applications for online learning designers, researchers and practitioners.The IMF can be used for retrospective as well as ongoing evaluation of a course of study.For instance, if a lecturer intends to engage students in a CoI, examination of a cross-sectional network during the course using the IMF would indicate if, in-fact, a CoI is being formed.If a CoI structure is not observed, intervention can be planned to alter the structure by, for example, adjusting the facilitation technique.The IMF can also be used to identify key participants or groups in large networks for selective qualitative analysis.Additionally, the radial network diagrams provide an effective visual illustration of participation and engagement which can be shared with students and tutors for feedback.
Coding of interactional data into matrices to create networks can be a fairly time-consuming task, especially for large networks that can be seen as a limitation.However, automation of the IMF would take care of this limitation and is under consideration.Also, although the case study highlights that different learning designs lead to different network structures which might or might not resemble a community, it does not consider other critical factors that might be contributing to online engagement and consequently community formation, for instance, individual student attributes like goal orientation and self-efficacy.Furthermore, considering the scope of the study, selective qualitative analysis of the discussion transcripts was not undertaken.To further validate the IMF, qualitative analysis of the communication between key participants is required to verify the presence of CP and TP along with the underlying SP.To date, the IMF has been effectively used to assess community formation in Jan and Vlachopoulos (in press), Jan andVlachopoulos (2018), andVlachopoulos, Matos, andKoutsogiannis (2018).Further validation is underway in ongoing research.
In conclusion, the IMF proves to be effective in assessing and evaluating learning designs intended to engage students in communities of learning.As such, the framework is considered as an advancement in social learning analytics techniques and methodologies.Researchers of online learning are encouraged to use the IMF and contribute towards its validation and refinement going forward.

Table 1 .
Differences in the learning design of the course in S2 and S3.