Skip to content

Advertisement

  • Research article
  • Open Access

Big course small talk: twitter and MOOCs — a systematic review of research designs 2011–2017

International Journal of Educational Technology in Higher Education201815:44

https://doi.org/10.1186/s41239-018-0127-9

  • Received: 25 June 2018
  • Accepted: 12 October 2018
  • Published:

Abstract

Although research on the use of Twitter in support of learning and teaching has become an established field of study the role of Twitter in the context of Massive Open Online Courses (MOOCs) has not yet been adequately considered and specifically in the literature. Accordingly, this paper addresses a number of gaps in the scholarly interface between Twitter and MOOCs by undertaking a comprehensive mapping of the current literature. In so doing the paper examines research design through: data collection and analysis techniques; scope and scale of existing studies; and theoretical approaches and underpinnings in the empirical research published between 2011 and 2017. Findings serve to demonstrate the diversity of this line of research, particularly in scale and scope of studies and in the approaches taken. By mapping the research using a systematic review methodology it is shown that there is a lack of qualitative data on how Twitter is used by learners and teachers in MOOCs. Moreover, a number of methodological gaps exist in published quantitative survey research at the interface between Twitter and MOOCs, including issues in the trustworthy reporting of results and full consideration of tweet and tweet meta-data collection. At the same time the paper highlights areas of methodological “best practice” in the research around these issues and in other important areas such as large-scale hashtag analyses of the use of Twitter in MOOCs. In reviewing the literature the findings aim to strengthen the methodological foundation of future work and help shape a stronger research agenda in this emerging area.

Keywords

  • MOOC
  • Twitter
  • Systematic review
  • Methodology

Introduction

The extent of the research literature on Massive Open Online Courses (MOOCs) has mirrored the explosion in the number of such courses and learner enrolments. Systematic literature reviews that attempt to classify and synthesize this large new literature have been valuable to scholars in the field. For example, at the time of this writing, two of the early published reviews had accrued over 650 citations, according to Google Scholar (Liyanagunawardena et al. 2013; Veletsianos and Shepherdson 2016). The publication of major review articles on MOOCs continues to be an active line of research, as illustrated by recent studies providing a content analysis of trends and patterns in the literature (Bozkurt et al. 2017; Zawacki-Richter et al. 2018) and a systematic review of the research methods and topics investigated (Zhu et al. 2018).

The research landscape concerned with the application and implications of the microblogging platform Twitter for teaching and learning is similar. Here, systematic literature reviews have made a major contribution to scholarship, as evidenced by Gao et al. (2012) and Williams et al. (2013). However, no review on the intersection of these two research areas has been conducted to date based on the results of a systematic search of the major publication databases. The question arises as to what is the small talk about big courses and hence the current study was undertaken. The next section provides a brief overview of research literature that pertains to both Twitter and MOOCs in the context of learning and teaching to highlight the research gap(s) that this study seeks to address.

Microblogs, and Twitter in particular, promise that they can enable participatory, collaborative, and even playful forms of online (and mobile) learning (Dunlap and Lowenthal 2009). Given that use of the Twitter platform is free, and that most of its content is published openly online, it is not surprising that MOOC-using teachers have successfully adopted it as an alternative site of learning to augment their courses (Salmon et al. 2015; van Treeck and Ebner 2013). In some cases teachers have even taken to Twitter as the primary platform to engage learners and deliver the MOOC itself (Bozkurt et al. 2016). To-date studies have investigated a variety of topics, such as learner experience (Kop 2011), social capital acquisition Joksimović et al. (2015a, b) and Twitter’s mirror of the course unfolding inside the MOOC platform Joksimović et al. (2015a, b). It is noteworthy that the literature on the use of Twitter in the context of MOOCs is characterised by a range of research approaches but there is no systematic review of this to date.

Through the analysis of Twitter hashtags, researchers have conducted studies that extend beyond individual courses to explore large Twitter datasets comprising aggregates of many MOOC learners in multiple courses. For example, these studies have analysed MOOC learners according to the following foci: sentiment towards courses (Shen and Kuo 2015); temporality and learning Zhang et al. (2015); levels and types of discussion (Veletsianos 2017); and the relative influence of participants in conversations about MOOCs, including discussions on the phenomenon itself (Costello et al. 2017).

Given the scope and diversity of this research, a comprehensive and systematic review is required to build a deeper understanding of Twitter for MOOC learners and teachers and to help shape a research agenda in this emerging area. Therefore the current study explores this interface by systematically reviewing and synthesising the extant research literature that concerns both MOOCs and Twitter as an important research topic in its own right.

Research questions

This study centres on the following broad research objective: To investigate the types of research design being conducted and published on the use of Twitter for teaching and learning in the contexts of MOOCs. To achieve this objective the study was framed around the following Research Questions (RQs):
  • RQ 1: How can the published literature be classified according to the methodological approaches followed by researchers?

  • RQ 2: What are the tools and methods researchers have reported using to collect and analyse twitter data?

  • RQ 3: What is the scope of the published research in terms of sample size, number of tweets and Twitter users, and study time frames?

  • RQ 4: What deeper theoretical assumptions define or underpin the adopted research methodologies?

  • RQ 5: To what extent have researchers studied Twitter in concert with one or more other social networks?

Methodology

This study employed a systematic literature review methodology (Petticrew and Roberts 2009; Okoli 2015; Kitchenham 2004) using the most common and reputable online databases/indices i.e. Web of Knowledge, EBSCO, Google Scholar, Scopus and IEEE Explore. Systematic literature reviews are typically utilised to help categorise and better understand large bodies of information. They aid this understanding by mapping out major themes, areas of uncertainty and by identifying gaps in research that can indicate where further work is needed (Petticrew and Roberts 2009). A systematic literature review provides a clear protocol for a comprehensive search strategy, has explicit inclusion and exclusion criteria for studies, and a method of evaluating or synthesising findings. While there may be a risk of an overly narrow mechanistic approach if the analysis is not framed by an explicit theoretical lens, the major advantage and defining feature of the systematic literature review is that it clearly utilises a systematic and replicable methodology (Petticrew and Roberts 2009; Kitchenham 2004).

The inclusion criteria for this study comprised the rules the authors used to build the literature corpus: in other words, the first casting of the net. For inclusion, studies were required to meet five criteria: (1) examined the use of Twitter (or Sina Weibo) as a site for research on a MOOC or MOOCs, (2) empirical in their scope, (3) written in English, (4) published in peer reviewed journals or conference proceedings, and (5) published or available online between January 2011 and July 2017. For the basis of this study empirical research was defined according to a number of sub-criteria. Firstly, studies were required to have gathered primary data, i.e. conceptual, purely theoretical and opinion pieces were excluded. Data could be gathered from Twitter directly in the form of tweet content and/or tweet metadata, such as the network structures and characteristics of a Twitter community that discusses MOOCs, or indirectly, such as via interviews or surveys of perceptions of people who use Twitter for MOOC teaching, learning or research. The data collection instrument of the study needed to include a direct reference to Twitter or a tweet, i.e., a survey had to mention Twitter or tweets specifically. Thus, studies that asked participants about social media more generally were excluded.

Following Liyanagunawardena et al. (2013) and Veletsianos and Shepherdson (2016), we used the following search terms for Twitter, MOOCs and their academic synonyms (which also capture studies of Sina Weibo, the Chinese equivalent of Twitter):
  • (micro-blogging OR micro-blog OR microblogging OR Microblog OR twitter OR Tweet) AND (“Massive Open Online Course” OR “Massively Open Online Course” OR MOOC).

As indicated above the following databases/indices were searched to identify the published literature: Web of Knowledge, EBSCO, Google Scholar, Scopus and IEEE Explore. Of these sources, Google Scholar is known to index by far the largest amount of content, including non-peer reviewed work and other grey literature Haddaway et al. (2015). Therefore, following Gao et al. (2012), it was decided to search the full text of Google Scholar articles, sorting the search results by relevance and analysing the first two hundred results, at which point, we felt confident to have identified the most relevant results. The lead researcher conducted the search and the analysis of all the returned articles. There was of course considerable overlap between the databases. Most articles could be excluded from the study based on a reading of the abstract .e.g. whether it was an empirical study or just a thought piece. 41 articles need to be downloaded and read in full to determine if they met the criteria. This resulted in 34 eligible studies. We only include peer-reviewed studies which excludes “grey literature” such as dissertations, theses, working papers, unpublished papers blogs etc. Although the established practice for systematic literature reviews this is a prospective limitation especially if we consider that peer reviewed studies may be more likely to publish favourable outcomes.

Findings

Following the systematic search strategy outlined above and the application of the inclusion criteria, a dataset of 34 eligible articles was identified. Figure 1 below shows how many studies were published annually for each year for which full year data is available.
Fig. 1
Fig. 1

Number of studies published for which data available for complete calendar year (n = 30)

The results of a detailed analysis of these articles according to our research questions are outlined in the next section.

At the top level we first classified the studies by dividing them into those that gathered perceptions data (n = 16) through survey i.e. self-reports via questionnaires or interviews and those that gathered primary Twitter data (n = 33). We further classified Twitter data studies as those that analysed tweets (n = 23) and those that analysed tweet metadata (n = 19) such as networks of Twitter users. Studies that analysed tweets included those in which a human evaluator qualitatively analysed tweets. The evaluator in such studies typically used a defined research instrument derived from a theory or other study, e.g., (Kop 2011) used social presence indicators. Our analysis showed that the evaluation was primarily based on content or socially orientated, e.g., whether a tweet refers to a course topic or an interaction with a learner/teacher. Other analyses of tweets (n = 22) used machine-learning approaches to analyse content. Machine learning approaches comprised those that detected themes via topic modelling and keyword mining Joksimović et al. (2015b) and/or sentiment towards particular topics i.e. sentiment analysis (Abeywardena 2014; Shen and Kuo 2015; Costello et al. 2016). Other machine-learning or computational approaches analysed the metadata of tweets, such as information on time periods of twitter activity, quantity of twitter activity and changes in twitter activity over time Zhang et al. (2015). A special category of tweet metadata was found to have analysed the connections between twitter users, such as their activity or whom they follow, known broadly as social network analysis (SNA). As per Table 1 below 12 studies conducted SNA.
Table 1

Full Mapping of Research Methods Used

 

Interviews

Surveys / Questionnaires

Researcher analysis of social media content

Machine analysis of social media content

Social network analysis

Other metadata analysis

Abeywardena 2014

   

x

  

Alario-Hoyos et al. 2014

     

x

Alario-Hoyos et al. 2013

 

x

   

x

Bell et al. 2016

 

x

  

x

x

Bozkurt et al. 2016

   

x

x

 

Chen et al. 2016

     

x

Costello et al. 2017

    

x

 

Costello et al. 2016.

   

x

  

Cruz-Benito et al. 2015

 

x

x

x

 

x

Cruz-Benito et al. 2017

 

x

x

x

 

x

de Keijser and van der Vlist 2014

    

x

x

de Waard et al. 2011

 

x

    

Enriquez-Gibson 2014a

   

x

  

Enriquez-Gibson 2014b

   

x

 

x

Fournier et al. 2014

x

x

x

x

x

x

García-Peñalvo et al. 2015

   

x

 

x

Jiang and Kotzias 2016

    

x

 

Joksimović et al. 2015a, b

   

x

x

 

Joksimović et al. 2015a, b

  

x

x

 

x

Knox 2014

 

x

x

x

 

x

Kop 2011

 

x

x

x

 

x

Koutropoulos et al. 2014

   

x

  

Kravvaris et al. 2016

   

x

 

x

Liu et al. 2016

 

x

x

  

x

Saadatmand and Kumpulainen 2014

x

x

   

x

Salmon et al. 2015

x

x

 

x

  

Shen and Kuo 2015

   

x

x

x

Skrypnyk et al. 2015

    

x

 

Spilker et al. 2015

 

x

 

x

 

x

Tu 2014

  

x

x

x

 

van Treeck and Ebner 2013

   

x

 

x

Veletsianos 2017

  

x

x

 

x

Yeager et al. 2013

  

x

 

x

x

Zhang et al. 2015

     

x

Totals

3

14

11

22

12

22

It should be noted that researchers typically collected data from more than one data source e.g. all but one of the 16 studies on learner perception also combined this with analysis of tweet data in some way with a notable exception being the interesting study of Saadatmand and Kumpulainen (2014) which took a phenomenological approach purely based on participant testimony.

While a detailed enumeration of the studies comprising each category is provided in Table 1 above a typology of research approaches according to method is depicted in Fig. 2 below. 
Fig. 2
Fig. 2

A typology of research on Twitter and MOOCs

In terms of the predominant methods being employed only a minority of studies (n = 2) utilised specific instruments, such as pre-existing coding schemes for qualitative analyses of either tweets or learner responses to surveys or interviews (van Treeck and Ebner 2013; Veletsianos 2017). However, the majority of surveys simply asked questions about whether participants found Twitter easy to use or useful for learning without using an explicit theoretical reference point or an instrument derived from extant literature. These types of questions have some relation to constructs from the Technology Acceptance Model (TAM) literature; however, only one study specifically mentioned a model of this type (Koutropoulos et al. 2014). Lastly, one study (Salmon et al. 2015) stated that the researchers were not part of the MOOC teaching team, highlighting that few of the studies clarified this point. One study, Knox (2014), stated that the researcher was also a teacher. However, many studies implied that an overlap in the teaching and research teams, or at least their affiliations, existed.

How tweets were collected and analysed

RQ2 concerned the data collection methods employed in the research studies analysed and the prevalence of the various analysis methods employed.

A total of 16 studies reported using 11 different aggregators or methods for collecting tweets and tweet metadata, as listed in Table 2 below. Of these tools and methods, four were used in more than one study, namely, Crawler, NodexxL, Twinonmy and gRSShopper.
Table 2

Tweet Data and Metadata Collection Methods/Tools

Collection method/tool

Number of studies

gRSShopper

3

Crawler

2

NodexxL

2

Twinonomy

2

Twitter API

1

Crowdmap

1

Digital Methods Initiative Twitter Capture and Analysis Toolset

1

Search box on Twitter website

1

TagsExplorer

1

GNIP API

1

TwitterSTAT

1

As shown in Table 3 (below), 20 analytical and computational tools were reportedly employed for data storage, processing and analysis in studies that analysed tweets and/or tweet metadata. The tools mentioned in more than one study were R, Gephi, Excel, NVivo, NodexxL and Microsoft Translation API.
Table 3

Software Tools Used to Analyse Tweets

Analytical tool

Number of studies

Excel

3

Gephi

3

R

3

NVivo

2

NodexxL

2

Microsoft Translation API

2

t-SNE’s scikit-learn implementation

1

Digital Methods Initiative Twitter Capture and Analysis Toolset (DMI-TCAT)

1

Wekka

1

Netlytic

1

OpinionFinder

1

TAGsExplorer

1

SurveyGizmo

1

Pajek64 3.15

1

Big Query

1

Dedose

1

PHP

1

SQL

1

TagMe Semantic Annotation tool

1

Linguistic Inquiry and Word Count analysis software

1

Scope of the studies

RQ3 examined the amount of primary data considered by researchers, the length of time the data pertained to and the numbers of tweets and twitter users. Seven of the studies either interviewed or surveyed the participants. The number of participants ranged from a minimum of 12 to a maximum of 3362, as outlined in Table 4 below.
Table 4

Number of participants interviewed/surveyed

Study

Number interviewed

Number surveyed

Response rate

Alario-Hoyos et al. (2013) Cohort A

0

3362

Not stated

Alario-Hoyos et al. (2013) Cohort B

0

Not stated

Not stated

Alario-Hoyos et al. (2013) Cohort C

0

Not stated

Not stated

Cruz-Benito et al. (2017)

0

212

27%

De Waard et al. (2011)

0

40

0.53%

Fournier et al. (2014) Cohort A

0

32

Not stated

Fournier et al. (2014) Cohort B

0

63

Not stated

Fournier et al. (2014) Cohort C

0

74

Not stated

Liu et al. (2016)

0

361

Not stated

Saadatmand and Kumpulainen (2014) Cohort A

12

0

Not stated

Saadatmand and Kumpulainen (2014) Cohort B

0

20

Not stated

Salmon et al. (2015) Cohort A

29

0

Not stated

Salmon et al. (2015) Cohort B

0

155

Not stated

Total

41

4319

 
A total of 13 studies reported the number of Twitter users that were included in the research where tweets or twitter metadata was analysed. The range in the size of the studies analysed was considerable, with the smallest study including 173 unique Twitter users and the largest including 278,685 users (see Table 5). The median value across the studies where this information was provided was 2431 users.
Table 5

Number of Twitter Users

Study

Number of individual Twitter users

de Keijser and van der Vlist (2014)

278,685

Zhang et al. (2015)

62,074

Chen et al. (2016)

25,620

Costello et al. (2016)

14,890

Veletsianos (2017)

4931

van Treeck and Ebner (2013) Cohort B

4085

van Treeck and Ebner (2013) Cohort A

2431

Joksimović et al. (2015a, b)

835

Skrypnyk et al. (2015)

800

Alario-Hoyos et al. (2014)

569

Bozkurt et al. (2016)

431

Cruz-Benito et al. (2015)

256

Alario-Hoyos et al. (2013)

173

Table 6 below illustrates the magnitude of the range of tweets sampled in each of the studies reviewed. The smallest number was 131, while the largest was 12,314,067, and the median value was 2486.
Table 6

Number of Tweets Analysed

Study

Number of tweets analysed

Chen et al. (2016)

12,314,067

Shen and Kuo (2015)

402,812

de Keijser and van der Vlist (2014)

106,316

Zhang et al. (2015)

95,015

Costello et al. (2016)

32,309

Costello et al. (2017)

32,309

Bozkurt et al. (2016)

20,000

Knox (2014)

18,745

Veletsianos (2017)

16,423

Bell et al. (2016)

6603

Fournier et al. (2014)

3104

Kop (2011)

3022

Enriquez-Gibson (2014a)

3000

Abeywardena (2014)

2853

Joksimović et al. (2015a, b) Cohort B

2486

Skrypnyk et al. (2015)

2483

Joksimović et al. (2015a, b)

2483

Joksimović et al. (2015a, b) Cohort A

2433

Tu (2014)

1386

De Waard et al. (2011)

1123

Salmon et al. (2015)

664

Alario-Hoyos et al. (2013)

659

van Treeck and Ebner (2013) Cohort B

393

van Treeck and Ebner (2013) Cohort A

367

Kravvaris et al. (2016)

362

Alario-Hoyos et al. (2014)

173

García-Peñalvo et al. (2015)

167

Spilker et al. (2015)

150

Cruz-Benito et al. (2015)

131

In terms of study timeframes some systematic literature reviews of Twitter in teaching and learning MOOC contexts have examined the length of intervention or, more generally, the timeframe in which data were collected (Gao et al. 2012;Williams et al. 2013). This timeframe may have been based on an individual MOOC or set according to a calendar year, for example. The shortest time frame studied was 2 weeks, while the maximum was that examined by Zhang et al. (2015), who collected 260 weeks’ worth of data. The median time frame was 10.5 weeks. The study time frames are outlined in Table 7 below.
Table 7

Study Time Frames

Study

Study time frame (weeks)

Zhang et al. (2015) course B

260

Zhang et al. (2015) course A

104

Shen and Kuo (2015)

52

Enriquez-Gibson (2014b)

43

Abeywardena (2014)

26

de Keijser and van der Vlist (2014)

26

Knox (2014)

18

Saadatmand and Kumpulainen (2014) course C

13

Joksimović et al. (2015a, b) course A

12

Joksimović et al. (2015a, b) course B

12

Joksimović et al. (2015a, b)

12

Saadatmand and Kumpulainen (2014) course B

12

van Treeck and Ebner (2013) course B

11

Fournier et al. (2014)

10

Kop (2011)

10

Saadatmand and Kumpulainen (2014) course A

10

Alario-Hoyos et al. (2014)

9

Spilker et al. (2015)

8

van Treeck & Ebner (2013) course A

8

Bozkurt et al. (2016)

6

Alario-Hoyos et al. (2013)

6

De Waard et al. (2011)

6

Koutropoulos et al. (2014)

6

Cruz-Benito et al. (2015)

4

García-Peñalvo et al. (2015)

4

Tu (2014)

2

In terms of the number of courses, the majority of the studies examined a single MOOC. Table 8 below, however, lists the studies that reported analysing data pertaining to more than one MOOC. The term “multiple” indicates that the researchers studied multiple MOOCs but did not specify an exact number (such as in studies of the #MOOC hashtag), which is somewhat unhelpful in judging the methodological trustworthiness of the research.
Table 8

Studies of More than One MOOC

Study

Number of courses

Abeywardena (2014)

Multiple (did not report number)

Costello et al. (2017)

Costello et al. (2016)

Enriquez-Gibson (2014a)

Enriquez-Gibson (2014b)

Shen and Kuo (2015)

Tu (2014)

Zhang et al. (2015)

Kravvaris et al. (2016)

320

Veletsianos (2017)

116

Chen et al. (2016)

18

Saadatmand and Kumpulainen (2014)

3

Joksimović et al. (2015a, b)

2

Kop (2011)

2

van Treeck and Ebner (2013)

2

Theoretical assumptions

To answer RQ 4 an analysis was undertaken of the deeper theoretical assumptions or perspectives that researchers used within and across the 34 studies. The findings revealed a very diverse picture with little overall pattern. The theoretical assumptions or at times relatively light conceptual touchstones to indicate particular theoretical lens were not always explicit but when noted they included: Rhizomatic learning (Saadatmand and Kumpulainen 2014; Bell et al. 2016), Connectivism (Saadatmand and Kumpulainen 2014, Cruz-Benito et al. 2015), and social presence/Community of Inquiry (CoI) (Kop 2011; Enriquez-Gibson 2014a; Spilker et al. 2015; Bozkurt et al. 2016), with the latter cited most frequently (n = 5).

Social networks beyond twitter

Lastly, in terms of other social networks as per RQ5, 14 of the studies did not focus on Twitter or a MOOC in isolation but examined them in concert with one or more other social networks. Of the social networks other than Twitter encountered in this review, Facebook was the most prevalent, with Google+ also featuring prominently. Other social networks appeared less frequently, often with only one mention each. The social networks other than Twitter examined by the studies are shown in Fig. 3 below.
Fig. 3
Fig. 3

Other Social Networks Considered Beyond Twitter

Discussion

The amount of published research on the use of Twitter in MOOCs has increased, as shown in Fig. 1. This point thereby underscores the value of better understanding the scope and nature of this research body as a whole. As this is clearly a growing area of interest, this paper contributes to this branch of research by mapping out the current state of the field by collating, interpreting and summarizing the methods employed by extant published studies. Future researchers can use this mapping to situate their research interests or to identify gaps or under-researched areas. The following discussion section elaborates on some of these areas and generally reflects on the findings in relation to the research questions.

Firstly, the systematic literature reviews reveals a relative dearth of research utilising qualitative methods. For instance, only four of the 34 studies conducted interviews. In addition, the review highlights the breadth of this research area and the diversity of approaches taken–for better and worse. Our analysis of the theoretical underpinnings as per RQ4 suggests that researchers should be cognizant of existing theories and theoretical constructs, particularly when surveying MOOC Twitter learners where use of survey instruments derived from prior studies or established theory appear lacking. What this may highlight is that the field is in need of more theorizing to properly advance. It may be that the lure of this new, available and abundant site of research data has caused studies to be undertaken in a haste that did not allow for proper research design that would incorporate theoretical underpinnings. Our recommendation here is that authors pay heed to this in future and situate their work more explicitly with reference to relevant theory.

Secondly, in addressing RQ2, we analysed the data collection tools and methods employed in the research studies and the prevalence of the various analysis methods reported. As presented above many studies reported the tweet data and metadata collection methods they employed; however, four studies did not report the collection methods. Furthermore, many tweet collection methods employed screen-scraping or manual searching, which have methodolnogical implications for the reliability of the data, reproducibility of the studies and rights of the Twitter users (Driscoll and Walker 2014). Official Twitter APIs for example will remove deleted tweets from their datasets, respecting this right of users, or remove tweets from deceased people. Five of the studies included some critical analysis of tweet data collection, such as limitations of particular collection methods. One paper claimed to have a “complete corpus” of tweets (Bozkurt et al. 2016), while others contained discussions on how tweets can be harvested from Twitter and the relative limitations of such techniques, including the fact that only a sample can be retrieved (Koutropoulos et al. 2014; de Keijser and van der Vlist 2014). One study calculated that its tweet sample represented 80% of the underlying data (Veletsianos 2017). The key point is that a greater consideration and acknowledgement of the complex nature of tweet collection could be made in future studies in this area, and there is also scope for more studies using Twitter Streaming APIs and Big Data infrastructure.

On a related point Excel, Gephi, R, NVivo and NodexxL were the most commonly used analysis tools, as shown in Table 3, although a long list of other tools was employed suggesting a broad range of tools are being adopted. Most of these tools were used for SNA, but NVivo was also used for qualitative analysis. In the future the field would benefit from more explicit discussion around the advantages and disadvantages of particular analysis tools such as around their ease of use, sophistication and cost or availability. For example Gephi is a specialised tool for network analysis that can be used to readily create network statistics and visualisations. R by contrast arguably requires greater expertise to use but is a general purpose statistical platform which can conduct not only network analysis but multiple statistical and machine learning analyses and on potentially very large data sets via big data and cloud techniques. R and Gephi are open source software so free to download and use. The NodexL plugin is open source but the software it relies on, Microsoft Excel, is proprietary paid for software as is NVivo.

Thirdly, there also appears to be a gap in the research when we consider the scale of the survey research, as only one study reported a large (given the potential pool of participants) number of respondents, i.e., over three thousand respondents Alario-Hoyos et al. (2013). This issue is not helped, however, by the large proportion of studies (85%) that did not report explicit response rates, which suggests potential lack of rigor in at least the reporting of these studies. For instance, Bell et al. (2016) found greater levels of discussion of a MOOC on Facebook than on Twitter.

Another related point is that the size of scope of the studies varied widely. Indeed, an alternative mapping of this literature might examine study size. Nine of the studies analysed over 10,000 tweets. Manual qualitative evaluation at this scale becomes difficult in a practical sense for all but relatively small samples (Veletsianos 2017) i.e. it would be prohibitive for a researcher to manually read and classify thousands of tweets. However, the relationship between data scale and practical study methods is not symmetrical, i.e., machine-learning techniques can be used in small-scale studies; thus, we employed the study mapping approach based on method type. Another small but interesting subcategory of the research studies consists of those that considered multiple MOOCs. A total of 15 studies were conducted on more than one MOOC, but several analyses were performed on over 100 MOOCs (Shen and Kuo 2015; Tu 2014; Zhang et al., 2015; Kravvaris et al. 2016; Veletsianos 2017; Costello et al. 2016; Costello et al. 2017). The variance of the dataset sizes in these studies has implications for the comparability of findings. Future studies may need a stronger justification for the use of particular analytic approaches taken.

Fourthly, the study timeframes over which research was conducted generally mapped to course lengths. However, considering a window that stretches beyond the course could be valuable for researchers and course designers, such as the approach used by Bozkurt et al. (2016), which showed Twitter activity up to 3 weeks after the course. The findings on study length also mapped the trend in shorter MOOC durations as course lengths have been shown to inversely correlate with completion rates (McIntyre 2016). The work of Zhang et al. (2015) is notable for its examination of a large body of MOOC learners (and other stakeholders) with regard to the temporal dimension of their Twitter activity related to MOOCs. They noted, for instance, peaks of activity during particular times of the year and week. This is an underexplored aspect of research and highlights the significant potential for studies of large datasets of the Twitter activity related to multiple MOOCs. Indeed, a few studies belonged to a special category that analysed the hashtag #MOOC itself (Abeywardena 2014; Shen and Kuo 2015; Zhang et al. 2015; Costello et al. 2016; Costello et al. 2017). This hashtag can be used to create large datasets that contain not only learners and teachers but also researchers, platform providers and other MOOC stakeholders.

Fifthly we found that Facebook was the second most common social network researched after Twitter. It should be born in mind however, some studies were excluded from our analysis at the dataset creation stage because they did not employ separate questions for Facebook and Twitter use even though it could be argued that they are quite different media. Hence we recommend that researchers are clear in the framing of their questions and reporting of their results where possible and appropriately disaggregate data derived from different social networks.

Finally, some further interesting possible analyses were beyond the scope of our original research questions. For instance we did not examine the issue of ethics, data protection and ethical approval. This could be usefully examined in a future study. Another interesting analysis that was beyond the scope of the current study would be to examine in detail the specific research questions that the studies used and how these related to the topics under analysis. For future researchers, we also recommend that they try to include as much details of their approach, their data and analyses as possible to help facilitate reviews of this research. Moreover in the five points above we include examples where we believe best practice has been shown.

Conclusion

The media hype surrounding MOOCs may have somewhat abated but interest from learners continues to grow. While the digital footprints left by these learners and their teachers is usually analyzed within the big course learning environment of MOOCs the small talk of learners spreads outwards via Twitter in a myriad of ways. As demonstrated in this systematic literature review the variance in the scope and scale of studies exploring the interface between Twitter and MOOCs suggest that future researchers will do well to carefully justify their approaches and consider some of using qualitative methods to analyse appropriate research problems. Finally, by critically synthesising and developing a typology of the literature in this area, this study has hopefully provided an agenda for this growing area of research and contributed signposts and jumping-off points for future work.

Declarations

Acknowledgements

The authors would also like to thank Beijing Normal University Big Data Centre for Technology-mediated Education and the National Institute for Digital Learning and in Dublin City University. Ms. Virginia Thomas is acknowledged for her role in preparation of the manuscript.

Funding

This work was supported by the Chinese National Education Science Foundation [Project No. CCA120110].

Availability of data and materials

The data that support the findings of this study comprise academic papers are available from the respective publishers. We include full references to all of the papers in this article text.

Declarations

The study involved no human participants. All information, comprising the articles and papers analyzed, has not been used for purposes other than originally intended i.e. to contribute to scholarly research. There is no conflict of interest. The authors have included some of their own work in the analysis according to the criteria outlined in the article.

Authors’ contributions

The first author conducted the primary analysis which was validated by the second and third authors. All authors contributed to the literate review and read and approved the final manuscript.

Competing interests

The study involved no human participants. All information, comprising the articles and papers analyzed, has not been used for purposes other than originally intended i.e. to contribute to scholarly research. There is no conflict of interest. The authors have included some of their own work in the analysis according to the criteria outlined in the article.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Dublin City University, Dublin, Ireland
(2)
Beijing Normal University, Beijing, China

References

  1. Abeywardena, I. S. (2014). Public opinion on OER and MOOC: A sentiment analysis of twitter data. International Conference on Open and Flexible Education, 296–307.Google Scholar
  2. Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Munoz-Organero, M., Hugo, A., & Parada, G. (2014). Delving into participants’ profiles and use of social tools in MOOCs. IEEE Transactions on Learning Technologies, 7(3), 260–266. https://doi.org/10.1109/TLT.2014.2311807.View ArticleGoogle Scholar
  3. Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Parada, G. H. A., Muñoz-Organero, M., & Rodríguez-de-las-Heras, A. (2013). Analysing the impact of built-in and external social tools in a MOOC on educational technologies. Scaling Up Learning for Sustained Impact: 8th European Conference, on Technology Enhanced Learning, 8095, 5–18. https://doi.org/10.1007/978-3-642-40814-4.View ArticleGoogle Scholar
  4. Bell, F., Mackness, J., & Funes, M. (2016). Participant association and emergent curriculum in a MOOC: Can the community be the curriculum? Research in Learning Technology, 24(1). https://doi.org/10.3402/rlt.v24.29927.View ArticleGoogle Scholar
  5. Bozkurt, A., Akgun-Ozbek, E., & Zawacki-Richter, O. (2017). Trends and patterns in Massive Open Online Courses: Review and content analysis of research on MOOCs (2008–2015). International Review of Research in Open and Distributed Learning, 18(5), 118–147.View ArticleGoogle Scholar
  6. Bozkurt, A., Honeychurch, S., Caines, A., Bali, M., Koutropoulos, A., & Cormier, D. (2016). Community tracking in a cMooc and nomadic learner behavior identification on a connectivist rhizomatic learning network. Turkish Online Journal of Distance Education, 17(4), 4–30. https://doi.org/10.17718/tojde.09231.View ArticleGoogle Scholar
  7. Chen, G., Davis, D., Lin, J., Hauff, C., & Houben, G. (2016). Beyond the MOOC platform: Gaining insights about learners from the social web. 8th ACM Conference on Web Science, 15–24. https://doi.org/10.1145/2908131.2908145.
  8. Costello, E., Brown, M., Nair, B., Mhichíl, M. N. G., Zhang, J., & Lynn, T. (2017). # MOOC Friends and Followers: An Analysis of Twitter Hashtag Networks. In European Conference on Massive Open Online Courses, (pp. 170–175). Cham: Springer.Google Scholar
  9. Costello, E., Nair, B., Brown, M., Zhang, J., Nic Giolla Mhichíl, M., Donlon, E., & Lynn, T. (2016). Social media #MOOC mentions: Lessons for MOOC research from analysis of twitter data.  In S. Barker, S. Dawson, A. Pardo, & C. Colvin (Eds.), Show Me The Learning. Proceedings ASCILITE 2016 Adelaide, 157-162.Google Scholar
  10. Cruz-Benito, J., Borras-Gene, O., Garcia-Penalvo, F., Blanco, A., & Theron, R. (2017). Learning communities in social networks and their relationship with the MOOCs. IEEE Revista Iberoamericana De Tecnologias Del Aprendizaje, 12(1), 24–36. https://doi.org/10.1109/RITA.2017.2655218.View ArticleGoogle Scholar
  11. Cruz-Benito, J., Borrás-Gené, O., García-Peñalvo, F. J., Blanco, Á. F., & Therón, R. (2015). Detection of non-formal and informal learning in learning communities supported by social networks in the context of a cooperative MOOC, In  International Symposium on Computers in Education (SIIE) (pp. 195–200). IEEEGoogle Scholar
  12. de Keijser, T. van der Vlist, F.N. (2014). Following UvAMOOC on Twitter – A network analysis of a massive socio technical and cross platform online learning environment. Online, Retrieved from https://www.fernandovandervlist.nl/papers/following-uvamooc-on-twitter.htmlGoogle Scholar
  13. de Waard, I., Abajian, S., Gallagher, M. S., Hogue, R., Keskin, N., Koutropoulos, A., & Rodriguez, O. C. (2011). Using mLearning and MOOCs to understand chaos, emergence, and complexity in education. The International Review of Research in Open and Distributed Learning, 12(7), 94–115. https://doi.org/10.19173/irrodl.v12i7.1046.View ArticleGoogle Scholar
  14. Driscoll, K., & Walker, S. (2014). Working within a black box: Transparency in the collection and production of big twitter data. International Journal of Communication, 8, 1745–1764.Google Scholar
  15. Dunlap, J. C., & Lowenthal, P. R. (2009). Tweeting the night away: Using twitter to enhance social presence. Journal of Information Systems Education, 20(2), 129 Google Scholar
  16. Enriquez-Gibson, J. (2014a). Tracing digital traces of phatic communion in #mooc, In Proceedings of the 9th International Conference on e-Learning (pp. 45–49).Google Scholar
  17. Enriquez-Gibson, J. (2014b). Following hushtag (#)MOOC: Mobility of online courses on twitter, In Proceedings of the 9th International Conference on Networked Learning (pp. 111–120).Google Scholar
  18. Fournier, H., Kop, R., Durand, G. (2014). Challenges to research in MOOCs. MERLOT Journal of Online Learning and Teaching, 10(1), 1–15. Retrieved from http://nparc.cisti-icist.nrc-cnrc.gc.ca/npsi/ctrl?action=shwart&index=an&req=21275387&lang=en
  19. Gao, F., Luo, T., & Zhang, K. (2012). Tweeting for learning: A critical analysis of research on microblogging in education published in 2008–2011. British Journal of Educational Technology, 43(5), 783–801. https://doi.org/10.1111/j.1467-8535.2012.01357.x.View ArticleGoogle Scholar
  20. García-Peñalvo, F. J., Cruz-Benito, J., Borrás-Gené, O., & Blanco, Á. F. (2015). Evolution of the conversation and knowledge acquisition in social networks related to a MOOC course, (pp. 470–481). https://doi.org/10.1007/978-3-319-20609-7_44.View ArticleGoogle Scholar
  21. Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of google scholar in evidence reviews and its applicability to grey literature searching. PloS One, 10(9), e0138237. https://doi.org/10.1371/journal.pone.0138237.View ArticleGoogle Scholar
  22. Jiang, S., Kotzias, D. (2016). Assessing the use of social media in massive open online courses. Retrieved from http://arxiv.org/abs/1608.05668 Google Scholar
  23. Joksimović, S., Dowell, N., Skrypnyk, O., Kovanović, V., Gašević, D., Dawson, S., & Graesser, A. C. (2015a). How do you connect?: Analysis of social capital accumulation in connectivist MOOCs, (pp. 64–68). https://doi.org/10.1145/2723576.2723604.View ArticleGoogle Scholar
  24. Joksimović, S., Zouaq, A., Kovanović, V., Gašević, D., Jovanović, J., & Hatala, M. (2015b). What do cMOOC participants talk about in social media? (pp. 156–165). https://doi.org/10.1145/2723576.2723609.View ArticleGoogle Scholar
  25. Kitchenham, B. (2004). Procedures for performing systematic reviews. Staffordshire: Keele University.Google Scholar
  26. Knox, J. (2014). Digital culture clash: “Massive” education in the E-learning and digital cultures MOOC. Distance Education, 35(2), 164–177. https://doi.org/10.1080/01587919.2014.917704.View ArticleGoogle Scholar
  27. Kop, R. (2011). The challenges to connectivist learning on open online networks: Learning experiences during a massive open online course. The International Review of Research in Open and Distance Learning, 12(3), 19–37. doi:https://doi.org/10.19173/irrodl.v12i3.882 View ArticleGoogle Scholar
  28. Koutropoulos, A., Abajian, S. C., deWaard, I., Hogue, R., Keskin, N. Ö., & Rodriguez, C. O. (2014). What tweets tell us about MOOC participation. International Journal of Emerging Technologies in Learning (iJET), 9(1), 8–21. https://doi.org/10.3991/ijet.v9i1.3316.View ArticleGoogle Scholar
  29. Kravvaris, D., Kermanidis, K., & Ntanis, G. (2016). How MOOCs link with social media. Journal of the Knowledge Economy, 7(2), 461–487. https://doi.org/10.1007/s13132-014-0219-2.View ArticleGoogle Scholar
  30. Liu, M., McKelroy, E., Kang, J., Harron, J., & Liu, S. (2016). Examining the use of facebook and twitter as an additional social space in a MOOC. American Journal of Distance Education, 30(1), 13–25. https://doi.org/10.1080/08923647.2016.1120584.View ArticleGoogle Scholar
  31. Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distance Learning, 14(3), 202–227. https://doi.org/10.19173/irrodl.v14i3.1455.View ArticleGoogle Scholar
  32. McIntyre, C. (2016). UK MOOC Report 2016: An insight into MOOCs provided by UK institutions. MOOCLab.Club. Retrieved from https://www.mooclab.club/Reports/UK%20MOOC%20Report%202016.pdf
  33. Okoli, C. (2015). A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems, 37(1), 879–910 Retrieved from https://hal.archives-ouvertes.fr/hal-01574600.Google Scholar
  34. Petticrew, M., & Roberts, H. (2009). Systematic reviews in the social sciences. Malden, MA: Blackwell.Google Scholar
  35. Saadatmand, M., Kumpulainen, K. (2014). Participants’ perceptions of learning and networking in connectivist MOOCs. Journal of Online Learning and Teaching, 10(1), 16–30. Retrieved from http://jolt.merlot.org/vol10no1/saadatmand_0314.pdf
  36. Salmon, G., Ross, B., Pechenkina, E., & Chase, A. (2015). The space for social media in structured online learning. Research in Learning Technology, 23(1), 28507–28514. https://doi.org/10.3402/rlt.v23.28507.View ArticleGoogle Scholar
  37. Shen, C., & Kuo, C. (2015). Learning in massive open online courses: Evidence from social media mining. Computers in Human Behavior, 51, 568–577. https://doi.org/10.1016/j.chb.2015.02.066.View ArticleGoogle Scholar
  38. Skrypnyk, O., Joksimović, S., Kovanović, V., Gašević, D., & Dawson, S. (2015). Roles of course facilitators, learners, and technology in the flow of information of a cMOOC. The International Review of Research in Open and Distributed Learning, 16(3), 188–217. https://doi.org/10.19173/irrodl.v16i3.2170.View ArticleGoogle Scholar
  39. Spilker, M. J., Silva, M. P., Leal, M. J., & Morgado, L. (2015). Social media and microblogging: Expanding boundaries in MOOCs, conferences and content curation. 2nd European Conference on Social Media, 421–428.Google Scholar
  40. Tu, H. (2014). Exploring the pattern of coursera network on twitter: How far are MOOCs reaching out? Communication Annual, 113–129.Google Scholar
  41. van Treeck, T., & Ebner, M. (2013). How useful is twitter for learning in massive communities? an analysis of two MOOCs. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter & society, Peter Lang.(pp. 411–424). Google Scholar
  42. Veletsianos, G. (2017). Toward a generalizable understanding of twitter and social media use across MOOCs: Who participates on MOOC hashtags and in what ways? Journal of Computing in Higher Education, 29(1), 65–80. https://doi.org/10.1007/s12528-017-9131-7.View ArticleGoogle Scholar
  43. Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. International Review of Research in Open and Distributed Learning, 17(2), 198–221 Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2448/3655
  44. Williams, S. A., Terras, M. M., & Warwick, C. (2013). What people study when they study twitter? classifying twitter related academic papers. Journal of Documentation, 69(3), 384–410. https://doi.org/10.1108/JD-03-2012-0027.View ArticleGoogle Scholar
  45. Yeager, C., Hurley-Dasgupta, B., & Bliss, C. A. (2013). cMOOCs and global learning: An authentic alternative. Journal of Asynchronous Learning Networks, 17(2), 133–147. https://doi.org/10.24059/olj.v17i2.347.View ArticleGoogle Scholar
  46. Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018). What research says about MOOCs – An explorative content analysis. International Review of Research in Open and Distributed Learning, 19(1), 242–259.View ArticleGoogle Scholar
  47. Zhang, J., Perris, K., Zheng, Q., & Chen, L. (2015). Public response to “the MOOC movement” in china: Examining the time series of microblogging. The International Review of Research in Open and Distributed Learning, 16 (5), 144–159.Google Scholar
  48. Zhu, M., Sari, A., & Miyoung Lee, M. (2018). A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). The Internet and Higher Education, 37, 31–39.View ArticleGoogle Scholar

Copyright

© The Author(s) 2018

Advertisement