Examining learning engagement in MOOCs: a self-determination theoretical perspective using mixed method

Abstract

MOOCs as a learning approach are gaining popularity, and helping learners and instructors understand how learning engagement is constructed in a MOOC context is of increasing importance. Although previous research has undoubtedly enriched our knowledge of MOOCs, our understanding of student engagement in the MOOC context is still limited. This study adopts a sequential explanatory mixed-methods approach to examine student engagement in MOOCs from the self-determination theory (SDT) perspective. A total of 693 valid responses to a MOOC Engagement-Motivation scale were collected and 82 MOOC participants were interviewed. The results showed significant differences between the MOOC completers and non-completers with respect to the rank of motivators for enrolment and the rank of learning activities for participation. The association between perceived competence and emotional engagement was significantly higher in the MOOC completion group. The results of a multiple regression analysis indicated that the SDT model can significantly predict student engagement. Perceived competence registered the largest positive impact, and perceived relatedness had a slight negative impact on engagement. The three components of engagement can also predict learners’ perceived learning. Emotional engagement showed the largest positive impact. However, logistic regression analysis indicated that these components of engagement poorly predicted MOOC learners’ completion. Qualitative analyses of student interview data revealed three main factors that can promote learners’ SDT needs: active learning, course resources, and instructor accessibility. Implications of the findings can help MOOC designers and educators to better engage their participants.

Introduction

By the end of 2018, 101 million learners had registered in one or more of the 11,400 Massive Open Online Courses (MOOCs) offered by over 900 institutes worldwide (Shah, 2018), indicating that MOOCs continued to capture the attention of many universities and the public worldwide since “the year of the MOOC” in 2012 (Pappano, 2012). MOOCs are essentially online courses offered by institutes in partnership with providers such as Coursera, edX, Udacity, and FutureLearn via the Internet to large number of people usually without any charge (Hew & Cheung, 2014). Global enthusiasm for MOOCs has generated a bubble of hype and a desire to embrace the idea among many universities. This interest has also led to numerous research studies, which can be grouped into several main categories:

• i. Impact of MOOCs on institutions: Courtney (2013), Gore (2014), and Wu (2013) examined the challenges and issues that MOOCs bring to academic libraries, such as the role of librarians and access to materials. Fox (2013) and Lombardi (2013) discussed how MOOCs affect higher education institutions in terms of faculty identity, workload, roles and responsibilities, and institutional policy.

• ii. Student motives, attitudes, and challenges of learning in MOOCs: (Hew & Cheung, 2014) reported several main reasons why students sign up for MOOCs, including the desire to learn a new topic or to further their current knowledge, and to collect as many completion certificates as possible. The main learning challenges in a MOOC are having insufficient prior content knowledge and failing to understand the content with no-one to ask for help (Belanger & Thornton, 2013).

• iii. Learners’ behaviour profiles in MOOCs: Several recent studies examined students’ behaviour profiles in MOOCs platform (Kahan, Soffer, & Nachmias, 2017; Khalil & Ebner, 2017). For example, Khalil and Ebner (2017) employed k-means clustering to group participants according to their MOOC learning behaviour. Four different profiles were identified: “dropout”, “perfect students”, “gaming the system”, and “social”. Kahan et al. (2017) also examined the different types of participant behaviour in a MOOC. Seven types of behaviour were identified: “Tasters”, “Downloaders”, “Disengagers”, “Offline Engagers”, “Online Engagers”, “Moderately Social Engagers”, and “Social Engagers”. Poellhuber, Roy, and Bouchoucha (2019) likewise examined MOOC learners’ behaviour and classified them into five different profiles: “Browser”, “Self-Assessor”, “Serious Reader”, “Active-Independent”, and “Active-Social”.

• iv. Instructor motives and challenges of teaching MOOCs: Instructors’ motives for teaching MOOCs include a desire to gain personal rewards, such as increasing their personal reputation within their discipline and with the general public, to help get tenure (Kolowich, 2013) to increase student access to higher education worldwide (Kolowich, 2013), or to experience teaching a large and diverse audience throughout the world, which no residential course can offer (Mackness, Mak, & Williams, 2010; Roth, 2013). Challenges of teaching MOOCs include having a sense of speaking into a vacuum when recording video lectures due to the absence of student immediate feedback, being burdened by the heavy demands of time and money, and encountering a lack of student participation in online forums (Hew & Cheung, 2014).

• v. Click-stream analysis of log data: The types of activities students carry out in MOOCs have been examined, such as the number of students viewing videos, submitting assignments, and commenting in course forums (Anderson, Huttenlocher, Kleinberg, & Leskovec, 2014; Coetzee, Fox, Hearst, & Hartmann, 2014; Coffrin, Corrin, de Barba, & Kennedy, 2014). More students watch videos than work on course assignments (Coffrin et al., 2014). A consistent decline in the number of students participating in the courses weekly was found. Factors that appear to predict student grades include the number of MOOC forum posts, and students’ marks in the first 2 weeks (Coetzee et al., 2014; Coffrin et al., 2014).

Given the popularity of MOOCs, helping students and instructors understand how learner engagement is constructed in a MOOC context is of increasing importance. While previous research has undoubtedly enriched the comprehension of MOOCs, our understanding of student engagement in the MOOC context is still limited. Although the concepts of engagement has been used for investigation, most previous studies actually focused primarily on examining learners’ behavioral engagement (Li & Baker, 2016) such as learner participation in academic activities. For instance, using clustering analysis, Kizilcec, Piech, and Schneider (2013) identified four prototypical engagement trajectories (i.e., completing, auditing, disengaging, sampling) based on how often learners watched the video lectures and did the assessments. Applying a similar method, Ferguson and Clow (2015) identified seven distinct patterns of engagement (i.e., samplers, strong starters, returners, mid-way dropouts, nearly there, late completers and keen completers). Based on self-reported surveys and interviews from the participants in a cMOOC, Milligan, Littlejohn, and Margaryan (2013) recognized three types of participation (i.e., active participation, passive participation, and lurking). In terms of the interaction with other people in the online forums, Baek and Shore (2016) indicated that the greater interaction reduced students’ dropout rate and increased student performance. Meanwhile, very few previous studies have investigated the phenomenon of student MOOC engagement from an integrative perspective, such as the behavioral, emotional, and cognitive engagement components (Hew, 2016; Floratos, Guasch, & Espasa, 2015). Moreover, many previous studies fell short of explicating the factors that may promote participants’ self-determination theory (SDT) components (sense of autonomy, relatedness, and competence) (Chen & Jang, 2010).

This study provides a new contribution by grounding the investigation of the three engagement components on the self-determination theory (SDT) perspective. As Schunk and Mullen (2012) defined, engagement is “the manifestation of students’ motivation” (p. 220). The notion of student engagement may be further explained using the construct of motivation. Motivation is the force that energizes students to be actively engaged in learning tasks (Darr, 2012). SDT is a frequently used theory to explain human motivation (Losier, Perreault, Koestner, & Vallerand, 2001; Niemiec et al., 2006). It has been applied in various settings but overlooked in large-scale open online learning research (Chen & Jang, 2010). Most studies on student engagement and SDT have been conducted in the conventional face-to-face classrooms (e.g., Chen & Jang, 2010; Rotgans & Schmidt, 2011; Shen, McCaughtry, Martin, Fahlman, & Garn, 2012; Skinner, Furrer, Marchand, & Kindermann, 2008). In the conventional settings, learners usually share the same academic goals, are familiar with one another, and are supervised closely by their teachers (Chiu & Hew, 2018). By contrast, MOOCs learners do not know most of their peers (Gillani & Eynon, 2014), are not supervised by the teacher, and are under no expectation to complete the course (Chiu & Hew, 2018). The use of SDT can help us gain an in-depth understanding of the possible relationships between MOOC participants’ psychological needs and their perceived engagement when undertaking a MOOC. The present study is guided by the following research questions:

• i. Compared to the MOOC completers, what motivated the MOOC non-completers to participate in a MOOC? What kinds of learning activity did the non-completers frequently participate in?

• ii. Compared to the MOOC completers, how did the three psychological needs (as espoused by SDT) relate to the three types of engagement for the non-completers?

• iii. How well did the components of self-determination theory (autonomy, relatedness, competency) predict the components of engagement (behavioral, emotional, cognitive)?

• iv. How well did the components of engagement (behaviour, emotion, cognition) predict perceived learning and completion?

• v. What specific factors might affect MOOC participants’ sense of autonomy, relatedness, and competence?

Theoretical framework

(Hew, Lan, Tang, Jia & Lo, 2019) criticized that explicit engagement with theory of educational technology-relevant studies was absent. Al-Rahmi, Aldraiweesh, Yahaya, and Kamin (2018) indicated that it was important for the MOOC studies to apply considerable use of theoretical framework to examine the factors that promote learning. Previous systematic review studies suggested that engagement and motivation were two dimensions that can be further explored under the MOOC learning context (Deng, Benckendorff, & Gannaway, 2017; Joksimović et al., 2018). The theoretical framework in this study consists of engagement components based on the constructs of engagement theory and motivational components from the perspective of self-determination theory.

Engagement components

In this study, engagement refers to the manifestation of learning motivation, which means that learners put energy and effort into the learning process to achieve desired learning goals (Reschly & Christenson, 2012; Schunk & Mullen, 2012). Student engagement can be operationalized as three interlinked elements: behavioral, emotional, and cognitive engagement. Behavioral engagement refers to students participating in a learning activity such as asking questions and completing an assignment (Fredricks, Blumenfeld, & Paris, 2004). Emotional engagement refers to students’ feelings or emotions toward teachers, peers, or learning (Fredricks et al., 2004). These affective reactions include interest, boredom, happiness, sadness, and anxiety in taking a course (Connell & Wellborn, 1991; Skinner & Belmont, 1993). The core idea of cognitive engagement concerns the level of investment in learning (Fredricks et al., 2004). Lamborn, Newmann, and Wehlage (1992) and Wehlage, Rutter, Smith, Lesko, and Fernandez (1989) highlight that cognitive engagement is a psychological investment in mastering the knowledge and skills, instead of “simply doing the work” (Fredricks et al., 2004, p. 64). Therefore, this type of engagement can be conceptualized as students’ understanding of the subject content being taught (Rotgans & Schmidt, 2011).

Self-determination theory (SDT)

The SDT of motivation posits that all individuals possess three fundamental psychological needs that move them to act or not to act – the needs for autonomy, relatedness, and competence (Deci & Ryan, 1985). Autonomy refers to the need for freedom or perceived choice over one’s actions (Deci & Ryan, 1985). Online students come to a course with complex blends of motivational reasons (Hew & Cheung, 2014), therefore it is important to consider supporting student autonomy with regard to facilitating effective learning (Artino, 2008). Deci and Ryan (2000) suggest that learning engagement in a self-determined learning context requires the satisfaction arising from a sense of autonomy. In MOOCs, students can determine which particular topics they wish to study and how they study, which according to the SDT should lead to a feeling of autonomy. Without the freedom and choice, learners will not be successful in self-directed learning (Loyens, Magda, & Rikers, 2008). The psychological need for autonomy provides a motivational basis for students’ behavioral engagement in a course (Skinner et al., 2008). Participants with high levels of autonomy are also likely to enjoy their lessons (i.e., having a higher level of emotional engagement) (Skinner et al., 2008) and feel the sense of ownership and responsibility for the work they perform (Lee, Pate, & Cozart, 2015). Autonomy is also assumed to lead to more extensive cognitive engagement as it gives learners more time and space to determine and evaluate their goals and decisions, but this assumption has not been extensively examined (Rotgans & Schmidt, 2011).

Relatedness refers to the sense or feeling of being connected to other people (Deci & Ryan, 1985). Students are found to perceive a sense of relatedness in their learning when the instructor supports their needs (Deci & Ryan, 2000; Ryan, La Guardia, Solky-Butzel, Chirkov, & Kim, 2005). The more the learners felt connected with peer learners in an online learning environment, the more eager they were to share knowledge (Kim, Glassman, & Williams, 2015). Several studies have demonstrated that students’ sense of relatedness can predict their behavioral and emotional engagement (Furrer & Skinner, 2003). A sense of relatedness can encourage positive student feeling toward a course (i.e., foster affective engagement), and motivate students to participate in the course activities (i.e., encourage behavioral engagement) (Furrer & Skinner, 2003).

Competence refers to the need to master one’s pursuits or learning (Helme & Clark, 1998); hence it may be considered a critical motivation factor for students’ cognitive engagement (Skinner et al., 2008). Competence may also provide a motivational basis for behavioral and emotional engagement; a sense of mastery about the topic being studied may encourage learners to further participate in the course activities, and foster positive learner feelings about the course.

Method

Research design

This study adopted a sequential explanatory mixed-methods approach involving two phases: quantitative data collection, and qualitative data collection (Creswell, Plano Clark, Gutmann, & Hanson, 2003). In this approach, we first collected and analyzed the quantitative data (questionnaire data). The qualitative data (interview data) were collected and analyzed second in the sequence to help explain the quantitative results obtained earlier.

The rationale for choosing the approach was that neither quantitative nor qualitative methods are sufficient alone to capture the details of a situation. But when used in combination, quantitative and qualitative methods can complement each other and provide a deeper understanding of the research problem (Ivankova, Creswell, & Stick, 2006).

The quantitative data helped the researchers to compare the differences of learning preference, motivations, models of associations among SDT components and Engagement components between those MOOC completers and non-completers. Meanwhile, using the quantitative data, we can predict which component(s) of motivational need could predict the engagement and completion. However, recognizing the differences was not enough to improve MOOC learning and teaching. The qualitative interview data helped the researchers to clarify the course design features, which helped us to promote a better learning and teaching context and in turn to improve learning effectiveness and performance.

Data collection and participants

The data consisted of self-reported questionnaire and interviews. Ethical consent to conduct the study was approved by the authors’ University IRB. Each questionnaire participant was given an Amazon e-gift card (of 10 US dollar) as compensation and each interviewee participant was given an additional Amazon e-gift card (of 20 US dollar). The surveys were conducted via the Internet from May 2017 to Apr 2018. We invited MOOC instructors to send relevant information of this study by email to their students. We also put up posters advertising the study on the University campus to reach more potential participants. A total of 693 learners answered the questionnaire based on one MOOC they had participated (See Table 1). The MOOCs that they reported to have participated included disciplines such as Mathematics, Medicine, Social Science, and Business. Of the 693 learners, 82 gave their consent to be interviewed through email (See Table 1). Majority of these interviewees were from Mainland China (N = 47) and Hong Kong (N = 18), while remaining were from India (N = 4), Italy (N = 4), Australia (N = 2), and other six countries (e.g. United Kingdom, United States, and Singapore).

Table 1 Background information of participants

Questionnaire

The questionnaire collected learners’ demographic information (e.g., age, gender), the hours per week they spent studying, whether they had completed the MOOC, the rank of activity participation, and the rank of motivators for participation. For the question concerning activity participation, the participants were asked to rank the various activities they participated in beginning from the most frequent (1) to the least frequent (7) – see Fig. 1.

Fig. 1

Rank of learning activity participation

For the rank of motivators for participation question, the participants were asked to rank the various factors that motivated them to participate in the learning activities beginning from the most important (1) to the least important (10) – see Fig. 2.

Fig. 2

Rank of motivators for participation

Instrument

Learner engagement and motivation were measured with the validated MOOC Engagement-Motivation (MEM) scale developed by (Lan & Hew, 2018). The MEM scale was adapted from the existing validated instruments and all items were answered on a 5-point scale ranging from strongly disagree (1) to strongly agree (5):

The motivation component consists of three psychological needs: (a) four items for the perceived autonomy (Standage, Duda, & Ntoumanis, 2005), for example, “In this MOOC, I can decide which activities I want to complete”, (b) four items for the sense of relatedness (Furrer & Skinner, 2003), for example,“In this MOOC, when I interact with peers I feel accepted”, and (c) three items for the perceived competence (McAuley, Duncan, & Tammen, 1989), for example, “After attending this course for a while, I felt competent in applying the concepts or theories”. The internal consistency coefficient (Coronbach’s alpha) was 0.77, 0.84, 0.72 for the three factors respectively.

The engagement component consists of three elements: (a) four items for the behavioral engagement (Skinner et al., 2008), for example, “When I am in the course, I pay attention in the course”, (b) four items for the emotional engagement (Skinner et al., 2008), for example, “When I am in the course, I feel good”, and (c) three items for the cognitive engagement (Rotgans & Schmidt, 2011), for example, “When I am in the course, I was engaged with the topic at hand”. The internal consistency coefficient (Coronbach’s alpha) was 0.77, 0.73, 0.70 for the three factors respectively.

The total internal consistency coefficient was 0.84 for the MEM scale, indicating the reliability of this instrument was good based on the criteria (above 0.70) from Hair, Black, Babin, Anderson, and Tatham (1998). In addition, according to the four indices of the model fit (Gatignon, 2010; Hooper, Coughlan, & Mullen, 2008; Kline, 2005), the results of the confirmatory factor analysis (CFA) showed that the MEM structure had good model fit (normed chi-square (NC) = 2.606 < 3, comparative fit index (CFI) = .961 > .95, and root mean square error of approximation (RMSEA) = .048 < .05).

In addition, participants’ perceived learning was measured using the 6-item learning and learning interest subscales of Alavi’s (1994) Learning Instrument, for example, “I have increased my understanding of basic concepts”.

Interview protocol

An interview invitation was included in the survey. An email interview with 15 questions (Fig. 3) was conducted when a participant accepted the invitation. According to Meho (2006), e-mail interview can be a viable alternative to telephone interview, since records of the interaction can be easily kept and managed. In addition, the English interview script was translated into Chinese by the researchers for Chinese interviewees.

Fig. 3

Interview questions

Based on the participants’ answers, following questions were asked to clarify some details that the interviewer did not understand. The records of those interviewees who did not finish the interviews were removed (e.g., did not provide further explanations that the interviewer asked). Finally, completed interview data from 82 participants were analyzed and reported in this study.

Data analysis

Quantitative data were analyzed using IBM SPSS Statistics (version 23) and qualitative data were analyzed using NVivo (version 11). The participants were separated into the completion group (CG) and the non-completion group (NCG) based on their self-reported information about their MOOC completion.

To answer RQ (1), Mann-Whitney U Test was conducted to examine if significant differences existed on the rank of the reasons that motivated the MOOC learners and the learning activity participation between CG and NCG. The effect size statistic r was calculated: r = z value / square root of N, where the z value was reported in the Mann-Whitney U Test output in SPSS and N was the total number of cases, i.e. 693.

To answer RQ (2), grouped correlational analysis was conducted to explore the relationships among the three psychological needs and the three types of engagement between the CG and the NCG. To determine whether the relationships among the three psychological needs and the three types of engagement were significantly different between CG and NCG, r values from the correlational analysis results in the SPSS were converted into z values. According to Pallant (2010), the association coefficients were statistically significantly different if the z value is less than or equal to − 1.96 or the z value is greater than or equal to 1.96. In addition, Mann-Whitney U Test was conducted to examine if significant differences existed on the six factors between the two groups.

To answer RQ (3) and RQ (4), multiple regression was conducted to investigate the predictive power of SDT components on each component of engagement, and these components of engagement on perceived learning. Logistic regression was conducted to investigate the predictive power of these components of engagement on MOOC completion. All issues concerning the sample size, multicollinearity and singularity, outliers, normality, and linearity were checked. All the three components of SDT were treated as independent variables to predict each type of engagement (i.e. dependent variable) one by one. According to Tabachnick, Fidell, and Ullman (2007), the sample size should be above 50 + 8 m (where m = the number of independent variables, it was 3 in this study). Therefore, the current sample size of 693 used indicated that there was no sample size concern in this study. In terms of multicollinearity and singularity, according to Pallant (2003) and Gorard (2001), the correlations among these variables should be above .30 but below .80. In addition, tolerance value and variance inflation factor were used to determine the presence of multicollinearity. A commonly used cut-off point was tolerance value of less than .10 or a VIF value above 10. The outliers can be checked in terms of the Mahal. Distance value. Linearity was inspected through the Normed P-P Plot of the regression standardised residual and the Scatterplot.

To answer RQ (5), a qualitative analysis of thematic synthesis (Thomas & Harden, 2008) was conducted on the interview data to investigate the factors that might affect the participants’ sense of autonomy, relatedness, and competence. The thematic analysis was conducted in six steps: (1) going through the email interview records (i.e., familiarization); (2) coding the records based on three major themes – active learning, instructor, and course resources (Hew, 2016) (i.e., coding); (3) generating more concrete themes under each major theme (i.e., generating themes). Emerging categories were allowed to emerge based on the constant comparison method (Glaser, 1965) during the coding process; (4) going through these themes again (i.e., reviewing themes); (5) describing and naming the themes (i.e., defining themes); (6) summarizing. The first author coded the interview data to generate relevant factors that might affect the participants’ sense of autonomy, relatedness, and competence. The second author independently examined the data and resolved any disagreement through mutual discussion.

Results

RQ (i): compared to the MOOC completers, what motivated the MOOC non-completers to participate in a MOOC? What kinds of learning activity did the non-completers frequently participate in?

The most important motivator for participating in the course activities reported by both completers (M = 2.82, Mdn = 2.00) and non-completers (M = 2.49, Mdn = 2.00) was the desire to increase knowledge. However, the MOOC completers significantly reported earning the course certificate (M = 4.05, Mdn = 3.00) and enrolling in a social community (M = 5.08, Mdn = 5.00) as more important motivators than the non-completers (M = 4.96, Mdn = 5.00; and M = 6.15, Mdn = 6.00 respectively). The non-completers (M = 6.35, Mdn = 6.00), on the other hand, reported preparation for advanced placement exam preparation as a more important motivator than the completers (M = 6.73, Mdn = 7.00). No other significant difference was found concerning other motivators between the completers and non-completers (See Table 2).

Table 2 Descriptive statistics on the rank of learning motivation

Both the completers and non-completers ranked watching video lectures as the top learning activity they participated in, and doing peer assessments as the least frequently participated activity (See Table 3). The Mann-Whitney tests revealed a significant difference in video watching frequency between the completers (M = 1.99, Mdn = 1.00) and non-completers (M = 1.56, Mdn = 1.00, U = 32,996, z = − 3.823, p < .000, r = .006); a significant difference in posting forum messages frequency (completers: M = 4.21, Mdn = 4.00 and non-completers: M = 4.86, Mdn = 5.00, U = 31,932.5, z = − 3.846, p = .000, r = .006); and a significant difference in doing peer assessments (completers: M = 5.70, Mdn = 7.00 and non-completers: M = 6.38, Mdn = 7.00, U = 32,825, z = − 3.697, p = .000, r = .006). No significant difference was found regarding the frequency of reading forum messages, doing course readings, and submitting quizzes.

Table 3 Descriptive statistics on the rank of activity participation

RQ(ii): compared to the MOOC completers, how did the three psychological needs relate to the three types of engagement for the non-completers?

Almost all the factors in both groups were significantly correlated with each other (See Table 4), except the relationship between the sense of relatedness and perceived competence in the non-completion group (r = −.115, p = .145). Compared to the participants in the CG, the participants in the NCG showed lower association coefficient between perceived competence with emotional engagement (r = .417^** < .708^** in CG). In addition, the Mann-Whitney tests indicated that two groups have differences on the factors of sense of relatedness (CG: mean = 2.20, median = 2.00; NCG: mean = 2.55, median = 2.50), perceived competence (CG: mean = 3.83, median = 4.00; NCG: mean = 3.60, median = 3.67), and cognitive engagement (CG: mean = 3.94, median = 4.00; NCG: mean = 3.83, median = 4.00).

Table 4 Relationships between components of SDT and components of engagement

RQ(iii): how well did the components of self-determination theory (autonomy, relatedness, competency) predict the components of engagement (behavioral, emotional, cognitive)?

Multiple regression was used to assess the ability of the three components of SDT (perceived autonomy, sense of relatedness, and perceived competence) to predict each type of engagement. Preliminary analysis was conducted to ensure no violation of the assumptions of normality, linearity, multicollinearity and singularity. Specifically, all the association coefficients were above .30 and below .80, all the tolerance value were above .10 and all the VIF values were less than 10. One response from a participant might be identified as outlier, but we kept the case for further regression analysis because this case did not influence the outcomes. According to the Normed P-P Plot of the regression standardised residual and the Scatterplot, the linear relationship was explicit.

The results of multiple regression analysis included the adjusted R² (.443, .547, .484 for BE, EE, and CE respectively), ANOVA (p < .001 for all) and standardized coefficients beta of each independent variables (See Table 5). We can see that the influence of all the components of SDT to the three types of engagement were on the significant level (p < .001 or p < .05). Perceived competence had the largest positive impact on all the three types of engagement (β = .423, .452, .501 for BE, EE, CE respectively), followed by the positive impact from perceived autonomy to BE, EE and CE respectively (β = .312, .379, .250). The sense of relatedness component had a slight negative impact on the three types of engagement (β = −.093, −.084, −.104 for BE, EE, CE respectively).

Table 5 Multiple regression analysis results of SDT components on engagement

RQ(iv): how well did the components of engagement (behaviour, emotion, cognition) predict perceived learning and MOOC completion?

Multiple regression was used to assess the ability of the three components of engagement (behaviour, emotion, cognition) to predict students’ perceived learning. Preliminary analysis was conducted to ensure no violation of the assumptions of normality, linearity, multicollinearity and singularity. Specifically, all the association coefficients were above .30 and below .80, except for the association between emotional engagement and perceived learning (.846). All the tolerance values were above .10 and all the VIF values were less than 10. One response from a participant might be identified as outlier, but we kept the case for further regression analysis because this case did not influence the outcomes. According to the Normed P-P Plot of the regression standardised residual and the Scatterplot, the linear relationship was explicit.

The results of multiple regression analysis included the adjusted R² (.78), ANOVA (p < .001) and standardized coefficients beta of each independent variables (See Table 6). We can see that the influence of all the components of engagement were on the significant level (p < .001). Emotional engagement had the largest positive impact on perceived learning (β = .482), followed by the positive impact from behavioural engagement to perceived learning (β = .286) and from cognitive engagement to perceived learning (β = .199). In addition, there was no mediation or moderation effect between the motivational needs of SDT and perceived learning in terms of three components of engagement.

Table 6 Multiple regression analysis results of engagement components on perceived learning

Direct logistic regression was performed to assess the impact of three components of engagement (behavioural engagement, emotional engagement, and cognitive engagement) on the likelihood of a MOOC completion. The full model containing the three predictors was statistically significant, χ² (3, N = 654) = 14.797, p < .01, indicating that the model was able to distinguish between the MOOC completers and non-completers. However, the model as a whole only explained between 2.2% (Cox and Snell R square) and 3.3% (Nagelkerke R squared) of the variance in a MOOC completion, and correctly classified 75.1% of cases. As shown in Table 7, only emotional engagement and cognitive engagement made a unique statistically significant contribution to the model. The strongest predictor of a MOOC completion was cognitive engagement, recording an odds ratio of 1.202. This indicated that those MOOC participants who cognitively engaged in a MOOC were over 1.2 times more likely to complete the MOOC than those who did not cognitively engaged in the learning, controlling for all other factors in the model.

Table 7 Logistic regression analysis - variables in the equation

RQ(v): what specific factors may promote MOOC participants’ sense of autonomy, relatedness, and competence?

Three main factors that can promote MOOC participants’ sense of autonomy, relatedness, and competence were found: active learning, course resources, and instructor accessibility (See Table 8). Active learning strategies consisted of connectivity and interactivity; course resources involved flexibility, conciseness, and visualization; instructor accessibility consisted of instructor availability (i.e., willingness, behaviors, and approaches) and instructor attributes (i.e., professionality, personality, and teaching style). Definitions of the various factors are given in Table 8:

Table 8 Course design features promote learning engagement

Based on the number of interviewees mentioned, the flexibility of course resources was the dominant factor affecting participant perceived autonomy (N = 72, 87.8%). Instructor availability (i.e., willingness (N = 18, 21.95%), behaviors (N = 16, 19.51%), approach (N = 16, 19.51%)) were more likely to affect the participants’ sense of relatedness with tutors. Active learning connectivity (N = 23, 28.04%) were more likely to affect the sense of relatedness with peers. The connectivity of active learning (N = 43, 52.43%) was the main factor affecting participant perception of competence.

Discussion

Overall, the MOOC completers watched more videos, posted more forum messages, and did more peer interactive activities than the non-completers. The completers were more motivated to earn the course certificate, join the social community, and prepare for advanced placement exams.

In general, the components of SDT (autonomy, relatedness, competence) and the components of engagement (behavioural, emotional, cognitive) were all significantly correlated with one another. Compared to the completers, the association of perceived competence and emotional engagement was much lower for the non-completers on a significant level. In addition, MOOC completers reported significantly higher perceived sense of competence and cognitive engagement levels than the non-completers, but they reported significantly lower on the sense of relatedness than the non-completers.

According the results of multiple regression analysis, the SDT model accounted for a large variance of each type of engagement (44.3% for BE, 54.7% for EE, and 48.4% for CE). The component of perceived competence had the largest positive impact on engagement, while the component of sense of relatedness had a slight negative impact on engagement. The components of engagement accounted for 78% variance of perceived learning. Thereinto, the emotional engagement had the largest positive impact on the perceived learning. Furthermore, the three-component engagement model only accounted for 2.2–3.3% variance of MOOC completion.

In terms of the factors promoting participants’ sense of autonomy, relatedness, and competency, the flexibility of course resources promoted both the perception of autonomy and competence, instructor availability promoted the sense of relatedness with tutors, the connectivity of active learning both promoted the sense of relatedness with peers, and the connectivity of active learning also promoted the participants’ perception on competence.

Perceived autonomy and engagement

According to Jang, Reeve, and Deci (2010), when students engage in face-to-face classroom learning, there is very often some aspect of the instructional method that plays a role in the initiation and regulation of the engagement. Under the autonomous learning context such as a MOOC, King et al. (2015) suggested that content access should not simply be considered from physical or technical aspect, but also require tailoring the design for individual’s learning purpose and personal learning context.

In the present study on MOOCs, we found that the flexibility of course resources (i.e. the learning materials are available for repeated access without the limitations on time and place) strongly promoted the participants’ perception of learner autonomy. Learners can choose to watch or read the various learning resources at a time and place that best suit them. Learners can also decide what they would learn in a MOOC according to their needs. Most MOOC students are working adult learners who have at least an undergraduate degree (Dillahunt, Wang, & Teasley, 2014). Working adult learners have many competing commitments on their time schedule. Hence, self-paced MOOCs which allow participants to engage with the course materials at a time of their choice, rather than at a time stipulated by the course instructor, will tend to satisfy adult learners’ need for autonomy.

Participants’ sense of autonomy is further promoted when the course resources in MOOCs provide different challenges or activities, or additional materials (e.g., readings, videos, websites) for people who are interested to study more about the topics. The provision of different activities or additional materials offers extra flexibility for participants to decide what they choose to focus in the MOOC according to their needs and desires. There are also no negative learning consequences from activities participation (e.g., missing an assignment or failing an assessment) in a MOOC. The participants can learn at their own pace based on their interests and can skip the learning materials they considered less important. We found positive correlations between the participants’ perceived autonomy and their reported behavioural, emotional, and cognitive engagement. These findings corroborate with those reported in traditional classroom settings, where learners with heightened sense of autonomy are more likely to participate in the course activities (behavioural engagement) (Jang et al., 2010), are more likely to enjoy the course (emotional engagement) (Skinner et al., 2008), and are more likely to expend greater effort in understanding the materials (cognitive engagement) (Mahatmya, Lohman, Matjasko, & Farb, 2012; Rotgans & Schmidt, 2011).

Sense of relatedness and engagement

Analysing the participants’ interactions and how it influences learning is one of the major research areas in the discussion forum of MOOCs (Almatrafi & Johri, 2018). Social value of learning was promised to be a positive approach to promote learning in a MOOC learning context (Freitas & Paredes, 2018). In our recent study, we found negative associations between the participants’ perceived relatedness toward the tutors and peers, and their reported behavioural, emotional, and cognitive engagement for both the completers and non-completers. These findings in the context of large open courses are not consistent with the results reported in conventional learning settings. Previous studies indicated that the more the learners feel connected with peer learners in a conventional online learning environment, the more eager they are to share their knowledge to help each other understand the materials better (i.e., facilitate affective engagement) (Kim et al., 2015). However, in the MOOC context, our findings (see Table 3) suggest the majority of participants appear to engage more with the self-paced activities such as watch video lectures, read the course materials, or complete the self-paced quizzes. MOOC participants are less inclined to engage in interactive activities with other people such as posting forum messages, and doing peer assessments. As a result, even though many MOOCs technically had peer interactive activities (e.g., discussion forums), they were not often used by learners (Kizilcec, Pérez-Sanagustín, & Maldonado, 2017) – thus, making it difficult for learners to develop a sense of relatedness with each other. The observed negative association between sense of relatedness and engagement could be a sign that learners’ sense of relatedness with their peers was not adequately met in the MOOC learning environment.

In the present study on MOOCs, we found that instructor availability was the most frequently reported factor that can promote the participants’ sense of relatedness to tutor (i.e., willingness to make themselves accessible to students), rather than instructor professionality (i.e., depth of specialised knowledge), instructor teaching style (e.g., speaking empathetically), or instructor personality (e.g., humorous). In other words, willingness to help can promote learner engagement when participants feel that instructors who make themselves available to respond to students show strong commitment to help learners. However, to respond to every student is an impossible feat for any instructor teaching a large-scale online course. From our interview data, we found several strategies reported by the participants through which the MOOC instructors attempted to make themselves more available, including providing weekly feedback (instead of daily) to common problems raised by learners, and setting aside regular online hours per week to interact synchronously with learners. Instructors may also offer a fee-paying consultation scheme for students who wish to receive personalized feedback. Such a scheme may be found in the Epidemics – the Dynamics of Infectious Diseases MOOC where students can opt for the “Free Plan” or the “ExperTA Plan”. In the “Free Plan” option, students do not need to pay any money should they wish to communicate with the teaching staff. Students’ questions will be referred to any available teaching assistant selected at random. In the “ExperTA Plan”, students are charged a base fee of $4.99 plus $0.39 for every minute after the 10th minute. Students will be connected to teaching assistants who have received high ratings from other learners.

Perceived competence and engagement

We found positive associations between the participants’ sense of competence, and their reported behavioural, emotional, and cognitive engagement for both the MOOC completers and non-completers. Competence refers to the need for a person to master one’s pursuits or learning (Helme & Clark, 1998). A heightened sense of mastery about the topic being studied is likely to increase students’ cognitive engagement, as well as encourage student participation in the course activities (behavioral engagement) and student interest the course (affective engagement) (Skinner et al., 2008). The most frequently reported factor that can promote learners’ sense of competence was the connectivity feature of active learning. To recall, the connectivity of active learning refers mainly to the instructional strategy of relating the course content to relevant daily (real-world) examples or cases. Many participants reported that the use of real-world examples help them better understand how the course content can be applied in practice. Most MOOC learners are adult learners with a bachelor’s degree or above (Dillahunt et al., 2014). Adult learners are more engaged in learning when course content that is presented is applicable to real-life situations (Knowles, Holton, & Swanson, 2011). This finding implies that instructors should emphasize application of content to real-world practices over mere transmission of information that does not explain how the information can be applied in real-life. Interestingly, the interactivity feature of active learning was hardly mentioned by the participants as a factor that can promote their sense of competence. The interactivity of active learning refers to participants using social interactivity tools (e.g., forum) to interact with their peers. In traditional classroom settings, peer discussion, where students interact with one another to exchange opinions, is commonly viewed as one of the activities that students found very beneficial for them to understand the course content (Ertmer et al., 2007). This, however, does not seem to be the case in the context of MOOCs. From the interview data, we found that MOOC participants do not seem to attach much importance with respect to the need for peer interaction in the large-scale open online courses. It is likely that the anonymous nature of MOOCs, along with job or family responsibilities reduces student expectations of course interaction with their peers (Chiu & Hew, 2018).

Engagement, perceived learning and completion

In a MOOC learning context, learning engagement (behavioural engagement, emotional engagement, and cognitive engagement) can predict participants’ perceived learning, but could not predict participants’ MOOC completion. One plausible reason for this is that many MOOC learners have no intention of finishing the course but want to use it to fulfil other needs such as refreshing their memory of particular topics (Davis, Jivet, Kizilcec, Chen, Hauff, & Houben, 2017). Many MOOC participants engage only with certain topics within a MOOC to gain new information or knowledge about a MOOC – they are not interested in the rest of the course materials (Wang & Baker, 2014). In the context of MOOCs, not completing a course is not necessary a breach of expectation, but the natural result of a free and open registration process where students get as much as they wish out of a course (Reich & Ho, 2014). Swan (2001) indicated that the factors of clarity of design, interaction with instructor, and active discussion among course participants significantly influenced students’ perceived learning. Consistent with Swan’s findings, our findings indicated more specific factors (See Table 8) that promoted the MOOC participants’ learning engagement.

Conclusion

MOOCs as a learning approach are gaining popularity, and helping learners and instructors understand how learning is constructed in a MOOC context is of increasing importance. In this study, based on the questionnaires and interviews, we explored the motivators that prompted the MOOC completers and non-completers to participate in the course activities, and the types of activities they frequently participated in. We examined the possible relationships between the components of SDT and components of engagement. We conducted the multiple regression on each type of engagement using the components from the SDT model. Using the three types of engagement, we conducted the logistic regression to investigate their predictive power on the completion of a MOOC. We also investigated the factors that might promote participants’ sense of autonomy, relatedness, and competence.

This study showed the differences between the MOOC completers versus those who did not complete with regard to their behaviour engagement (e.g. higher frequency of posting forum messages), motivation (e.g. more eager to gain a certificate and being involved in the social community), psychological needs (e.g. perception of competence) and cognitive engagement. In terms of the relationships between the psychological needs and the three types of engagement, compared to the completers, only the association of perceived competence and emotional engagement were higher for those non-completers. The interviews findings indicated that the flexibility of course resources was the dominant factor affecting the perception of autonomy. Instructor availability (especially their willingness to help) and active learning (both connectivity and interactivity) were the main factors affecting the sense of relatedness with tutors and with peers respectively. The connectivity of active learning most enhanced the participants’ perception of competence.

This study implies that satisfying MOOC participants’ psychological needs on perceived autonomy and perceived competence through MOOC design could improve all aspects of engagement. These aspects of engagement in turn affect the participants’ perceived learning. Several recommendations based on the interview data to support students’ autonomy (e.g., providing additional course resources, offering optional challenges or specialism so that learners can choose the specific activities with which they wish to engage rather than have mandatory tasks), and competence (e.g., emphasizing how the content can be applied to real-world practices) have been discussed in the preceding Discussion section. Here we briefly provide other additional suggestions to support student autonomy and competence in the context of MOOCs.

Student autonomy in a MOOC can also be supported if the course was designed with no imposition of weekly deadlines on the course activities (Martin, Kelly, & Terry, 2018). This gives participants sufficient time to do the activities within a specific time period best suitable to them since the imposition of deadlines can create unnecessary external pressure, thereby undermining learner intrinsic motivation and increasing the risk of withdrawal if they fall behind (Martin et al., 2018). Student autonomy in MOOCs can also be supported if learners are able to effectively regulate their own learning; however, many students cannot do it well (Jansen, van Leeuwen, Janssen, Conijn, & Kester, 2020). Compared to students in traditional face-to-face classes, MOOC students have to self-regulate their own learning to a greater extent due to the greater measure of autonomy in a MOOC, and the absence of instructor guidance (Jansen et al., 2020; Kizilcec et al., 2017). One strategy that has been found helpful in improving MOOC students’ self-regulated learning (SRL) is the use of short instruction videos (3–4 min) that contains information about SRL phases, and suggestions on how to improve SRL in each phase, followed by reflection question asking students to indicate how they could improve their learning in each SRL phase (Jansen et al., 2020).

Additional strategies to support student competence in a MOOC include the use of optimal challenge (e.g., offering opportunities to extend learning via increasing levels of challenge as the course progressed), providing students with a sense of progression to build a sense of competence through mechanisms such as a progress bar, and providing positive and constructive feedback (Martin et al., 2018). Evaluation of these strategies showed evidence of positive student sense of competency in a MOOC (Martin et al., 2018).

However, the current MOOC design appears to inadequately meet the MOOC participants’ sense of relatedness. Recall that the sense of relatedness refers to the feeling of being connected to other people. One possible avenue to help promote participants’ sense of relatedness to other people is to motivate individuals to interact with each other in the MOOC discussion forums. To provide incentives for learners to participate in the forums, several prior MOOC studies have found the use of reputation system and badges to be effective. Coetzee et al. (2014), for example, found that reputation systems, in which participants receive extrinsic rewards such as “reputation points” when their contributions are rated as useful by others, produce faster response times and larger numbers of responses per post. The use of a badge system also showed an increase in learners’ participation in the forums as it gave students goals or targets to shoot for (Anderson et al., 2014).

There were two limitations in this study. First, this study is limited to about 700 MOOC participants. Additional research could recruit more participants. Future researchers could conduct longitudinal studies to examine participants’ self-determination and engagement perspectives over time as MOOCs continue to evolve. Another limitation is that the data gathered from this study could not be directly extracted from users’ actions in the MOOC due to the lack of access permission from the MOOC platforms. Therefore, a self-reported questionnaire was employed. We recommend that future work be conducted to analyze the possible relationship between learners’ answers to the MEM questionnaire and their actual performance in a MOOC, provided the researchers could obtain the relevant permission from MOOC providers.