Can emotional design really evoke emotion in multimedia learning?

Abstract

The aim of this study was to investigate the effects of emotional design on learners’ performance and emotions. If emotional design has any effects on learners’ performance and emotions, then we attempted to explore dimensions that are accounted for such effects. 316 Chinese university students participated in this study. The students were divided into two groups: NED group - students in this group learned material based on neutral design, and PED group - students in this group learned material based on positive design. We explored the differences between two groups in student learning performance as well as emotion. Prior knowledge, retention, and transfer tests were also carried out and outcomes were compared across two groups. In addition, student emotion was measured using a positive affect scale and biofeedback instruments. The results showed that PED group outperformed NED group on retention and transfer tests. However, insignificant difference was found between two groups on student emotion. Our results suggest that, although the effect of emotional design was insignificant on student emotion, it was significant on student learning performance (i.e. on retention and transfer tests). Based on our results, several implications were drawn and suggestions were made for educators and researchers.

Introduction

Emotions can influence cognitive processes and learning (Pekrun, 2006; Saarelainen & Ruokamo, 2007; Yang, Yang, & Isen, 2013) and so research on multimedia learning (Mayer, 2001) has also begun to consider emotions during learning process and its influence on learning outcomes (Um, Plass, Hayward, & Homer, 2012; Mayer & Estrella, 2014). For example, Cognitive Affective Theory of Learning with Media (CATLM) was proposed (Moreno, 2005, 2006) to highlight importance of motivational and affective aspects in learning; especially, how they mediate learning by increasing or decreasing cognitive engagement.

Emotion and learning

Emotions are defined as individuals’ judgments about the world that are evoked as a reaction to and an interaction with certain stimuli (Desmet, 2002; Plass, Heidig, Hayward, Homer, & Um, 2014; Saarelainen & Ruokamo, 2007; Um et al., 2012). Emotions play an important role in judgment, perception, learning and other cognitive functions, allowing humans to be more creative and flexible in solving problems. Cognitive neuroscience proved the interaction between emotion and learning (Fassbender, Richards, Bilgin, Thompson, & Heiden, 2012; Goetz, Frenzel, Pekrun, Hall, & Ludtke, 2007). According to the information processing theory, learning is a process that involves taking in information, organizing and storing it to be retrieved at a later time (Siegler, 1998). That is, first, a learner pays attention to information and brings it in; after that, information is actively manipulated in working memory and passively held in long-term memory (Slate & Charlesworth Jr, 1988). How emotion influences learning process can be explained by two hypothesizes, i.e. emotions-as-suppressor-of-learning and emotions-as-facilitator-of-learning. The emotions-as-suppressor-of-learning hypothesis postulates that emotions impair information processing. It is possible that emotion can impact all steps of information processing. For example, emotion can take attention away from information to be learned (Blanchette & Leese, 2010; Uzun & Yıldırım, 2018). Information does not enter into the information processing system if a student does not attend to it; as a result, learning does not occur (Slate & Charlesworth, 1988). Our working memory has limited capacity, and if a learner has some task-irrelevant thoughts because of emotion, then processing task-irrelevant information places additional demands on working memory, and therefore, interrupts the processing of the essential information (Sweller, Ayres, & Kalyuga, 2011).

The emotions-as-facilitator-of-learning hypothesis assumes that emotion fosters information processing (Knörzer, Brünken, & Park, 2016; Um et al., 2012). Studies have suggested that positive emotions have a positive effect on information processing (Isen & Baron, 1991; Erez & Isen, 2002). Chung, Cheon, and Lee (2015) reported that emotion can help broaden the scope of attention. Um et al. (2012) argued that emotion facilitates working memory processes, e.g. memory encoding. Isen, Thomas, Clark, and Karp (1978) found that a positive mood state serves as an effective facilitator of retrieval of information. That is, positive emotion can serve as a retrieval cue for learning content from long-term memory. As a result, learners learn better in positive state of mind (Rader & Hughes, 2005).

Emotional design was the main focus of this present study. Design was defined as “to create, fashion, execute, or construct according to plan” (The Merriam-Webster Dictionary, 2020). Therefore, Smith and Ragan (2005) suggested that the term design can be referred to as a planning and execution process to develop something in order to solve a specific problem. In the context of this present study, we use the term emotional design to describe planning and implementation of visual elements in learning material that affect learners’ emotions and foster learning (Um et al., 2012). That is, we planned and embedded visual elements into learning material in order to facilitate positive emotions in students and foster their learning performance. Emotional design included positive as well as neutral visual elements in learning material. Therefore, the purpose of this paper was to explore whether emotional design can influence emotion and learning performance of students. If emotional design has any effects on learning outcomes, then which emotion component can be accounted for such effects?

Emotional design in the multimedia learning

In the context of this present study, multimedia learning is referred to situations in which students learn from multiple media such as words, pictures, and animations. Um et al. (2012) developed the multimedia learning material, “How immunization works,” based on two different designs: the neutral emotional design (NE) and the positive emotional design (PD). In NE, elements of learning material were in achromatic color (gray-scale) and with neutral shapes whereas in PD, the color and shape effects were applied to learning material elements. This was one of the first studies on emotional design and the scholars attempted to induce learners’ positive emotions by making the essential elements in graphics of the learning material more appealing (e.g. rendering essential elements with human-like features and with distinct, appealing colors). For example, the main characters in illustrations of the learning material (such as T-Cell, B-cell, and antigens) were converted from geometric shapes in grayscale color into little faces with expressive eyes rendered in appealing colors (i.e. purple, blue, and green). In addition, the neutral background color was changed to warm one (e.g. orange or yellow). This approach applied established effects to induce positive emotions without changing the learning content. The results showed that students who learned material containing positive emotional design features maintained higher positive emotions and got better scores both in the comprehension and transfer tests.

Later, Plass et al. (2014) replicated the study of Um et al. (2012) by using the same variation in design. Plass and his colleagues carried out two experiments in which they compared comprehension and transfer tests results of students in two groups. They found that emotional design had positive effects on learning outcomes of the comprehension test only. This finding was inconsistent with that obtained by Um et al. (2012).

Mayer and Estrella (2014) implemented emotional design elements in learning material which explained how a virus causes a cold. They redesigned the essential elements, such as demonstrating expressive eyes of the host cell that looked worried when the virus approached, sick look when the host cell was under attack, and aggressive eyes of the virus. In addition, the scholars rendered each character in appealing colors that easily catch attention, such as blue for the virus (with small green circles for the genetic material at the end of each tentacle), red for the host cell, and yellow for the sac in which new blue viruses were created within the host cell. With such design, the scholars aimed to determine whether using emotional design in learning material has a positive effect on student learning outcomes. The results showed that students who learned material with the emotional design performed better than students in the control condition on a subsequent learning test.

The above-mentioned studies showed evidence of the effectiveness of the emotional design in learning material on the learning performance (Mayer & Estrella, 2014; Plass et al., 2014; Um et al., 2012). However, the relationship between the emotional design and learners’ emotion during learning was not explored in related studies. For example, Heidig, Müller, and Reichelt (2015) re-designed the web-based learning material in terms of its aesthetics and usability to identify the influence of emotional design on learning. The findings showed that objective differences in aesthetics or usability did not affect learners’ emotional states. However, the perceived aesthetics and usability positively affected the emotional states of the learners. Learners’ emotional states had a minor impact on learning outcomes but a larger impact on intrinsic learning motivation.

Park, Knörzer, Plass, and Brünken (2015) examined the influence of emotional design on learning outcomes using eye-tracking tools. Participants emotions (positive vs. neutral) were induced experimentally first and then they learned with learning material which was varied in its design (with or without anthropomorphisms). The researchers aimed to explore how their intervention induces positive emotions and facilitate learning. Learners who were in a positive emotional state before learning process had better learning outcomes in comprehension and transfer tests and showed longer fixation durations on the textual information. However, anthropomorphisms in the learning environment did not induce positive emotions. That is, the positive emotion before learning and the emotional design had a positive effect on learners’ academic performance. Nevertheless, the emotional design did not affect emotional states.

In summary, the emotional design in the multimedia learning can affect the learning performance, but it is still unclear whether the emotional design can facilitate learning processes by evoking positive emotions during learning. So, in this present study, we aimed to investigate whether the emotional design can induce the positive emotions of learners and facilitate their learning.

In this present study, we applied the emotional design to learning material based on general recommendations of Um et al. (2012). Multimedia learning material, “The Formation of Lightning,” was adopted from previous studies on various principles of multimedia learning (Mayer & Moreno, 1998, 2002; Cheon, Crooks, & Chung, 2014; Chung et al., 2015). The learning material calls “multimedia” because it explains the concept of formation of lightning through multiple media, i.e. pictures, animations and narration. We re-designed this multimedia learning material by embedding emotional design features, e.g. we changed sky color from white to blue, cloud color from grey to white, or tree color from white to green.

Multimedia learning theories

The theories about multimedia learning, such as cognitive theory of multimedia learning (CTML) (Mayer, 2005) or cognitive load theory (CLT) (Sweller et al., 2011), explain cognitive processes during learning with multiple media. For example, CTML states that when a learner uses multimedia learning material, dual-channel information processing is taking place in his/her brain (Mayer & Moreno, 1998). According to CLT, learning performance can be negatively affected when cognitive load exceeds the limit of cognitive capacity of our working memory (Sweller et al., 2011). However, these important theories do not consider the impact of emotion on learning.

Therefore, scholars extended CTML with Cognitive Affective Theory of Learning with Media (CATLM) (Moreno, 2005, 2006, 2007, 2009; Rey, 2010). That is, scholars focused on the cognitive and emotional processes in multimedia learning (Moreno, 2006). A bridge was built between the emotional and cognitive processes in multimedia learning by adding motivational and emotional factors. In addition, the importance of emotion in learning was emphasized (Moreno, 2006). CATLM describes three relevant cognitive processes for successful information processing: selecting, organizing, and integrating. CATLM has three assumptions to describe the impact of motivational and self-regulatory skills as well as the influence of individual learner characteristics on learning. According to Moreno (2006), CATLM also includes affective mediation. That is, motivational factors mediate learning by increasing or decreasing the cognitive engagement.

Incorporating affection within CATLM suggests the emotional design hypothesis, which holds that making essential elements visually appealing (e.g. rendering them with face-like shapes and bright, warm colors) initiates and guides cognitive processing during learning (Mayer & Estrella, 2014). That is, emotional design of essential elements in multimedia learning material can prime and sustain improved cognitive processing during learning, which in turn leads to improved learning outcomes. For example, Mayer and Estrella (2014) suggested that emotional design directs attention of a learner towards essential learning material as well as encourages him/her to learn it better.

Dimensions of emotion

Different dimensions of emotions are discussed in the literature. Scholars mainly used them to define emotions. Wundt and Judd (1897) suggested that emotions can be described by three dimensions such as pleasurable - unpleasurable, arousing - subduing, and strain - relaxation. Later, Schlosberg (1954) proposed three dimensions of emotion such as pleasantness - unpleasantness, attention – rejection, and level of activation. Pratchak (1970) also believed that emotions have three dimensions, namely intensity, similarity, and polarity. Different dimensions of emotion are based on different perspectives of emotion. In this present study, Schlosberg’s three-dimensional emotional partition is adopted. That is, emotions are divided into pleasantness, attention, and activation.

Emotion measurement

Most studies on emotions do measure emotions. Self-reporting and physiological measures are the most commonly used methods. Self-reporting is a method which uses various kinds of emotion rating scales reported by participants. For example, the previous studies used the Positive Affect and Negative Affect Scale (PANAS) (Watson, Clark, & Carey, 1988) to measure emotion. Physiological measures of emotions are based on the bodily reactions such as pupil dilation (e.g. measured by eye tracking), skin conductance (e.g. measured by galvanic skin response), brain activity (e.g. measured by electroencephalogram), heart rate (e.g. measured by electrocardiogram), and facial expressions (e.g. measured by facial electromyography).

In this present study, we used galvanic skin response (GSR) and electroencephalogram (EEG) instruments to measure emotions objectively. GSR also measures emotional activation, e.g. GSR is known as a reliable instrument to measure pure sympathetic activation. When a human feels any emotion his/her body reacts according to it and starts excreting sweat. This brings more conduction in body and so variation of resistance can be measured (Sharma & Kapoor, 2014). GSR can reflect the psychological state of the learners through the change of the conductivity of the skin, which reflects the level of arousal in the learning process. Shi, Ruiz, Taib, Choi, and Chen (2007) used GSR to evaluate users’ stress and activation levels while using unimodal and multimodal versions of the same interface.

EEG can reflect brain activities in its different regions during various cognitive processes. A higher dimensional complexity of EEG activity over frontal cortical regions can be found during emotional imagery. According to different frequency bands, EEG can be divided into delta (0.5~3HZ), theta (4~7HZ), alpha (8~13HZ), beta (14~30HZ) and gamma (30~45HZ). The five frequency bands are closely related to the physiological and psychological activities (Aftanas et al., 1998). Among them, theta/beta negatively correlated with attention, the higher the theta/beta value, the lower the concentration of attention.

So on the basis of the previous studies, this present research adopted the three-dimensional emotional partition (i.e. pleasantness, attention, and activation) proposed by Schlosberg (1954). We collected and analyzed the data collected from the positive affect scale, GSR and EEG in order to explore whether emotional design can induce emotion and then facilitate the learning performance. To this end, the following three research questions were addressed in this study:

• (1) What are effects of emotional design on learners’ academic performance in multimedia learning?

• (2) What are effects of emotional design on learners’ emotions in multimedia learning?

• (3) If emotional design has any effects on learners’ emotions then what emotional dimension (i.e. pleasantness, activation, and attention) can be accounted for such effects?

Method

The ethics issues relevant to the research and approvals under which the data was collected and reported have been considered in this research. First, this study was performed following the Institutional Ethical Guidelines of Nanjing Normal University (China), and second, there was no any potential conflict of interest in this work.

Experimental design and participants

This study was based on a quantitative research method with an experimental research design. The data was collected using a pre- and post-tests, questionnaire, and physiological measuring instruments. Two experiments were carried out to explore the effects of emotional design (i.e. positive vs neutral) of essential elements in multimedia learning material on learning outcomes (Creswell, 2014).

316 participants were recruited from three different universities in Jiangsu Province, China (Fig. 1). Among them, 203 were freshmen, 35 were junior, and 78 were senior students. Their average age was 19 (range = 16 to 25). The first experiment took place in real classroom with 281 participants. The second experiment took place in a laboratory with 35 participants. Thus, we had two different measurements; the first data set was collected in real classroom and the second one was collected in a laboratory. Both, classroom and laboratory, relate to physical environment; however, the former was a learning space (i.e. a room in which usually students learn) and the latter was a researching space (i.e. a room in which usually scientific experiments or research are carried out). Real classroom could accommodate the large number of students and lacked physiological measuring instruments. These instruments are very expensive and could be used to collect data from one student at a time. Therefore, learner emotion was difficult to measure in classroom using such instruments and so we used a questionnaire only. We employed a between-subjects design with 281 participants divided (non-randomly) into the positive emotional design group (PED) (n = 181) and the neutral emotional design group (NED) (n = 100). In contrast to classroom, laboratory was equipped with instruments for collecting physiological data and could not accommodate the large number of students. Therefore, learner emotion was measured in laboratory by such instruments and a questionnaire. We invited 35 junior university students to join the physiological measurement. The participants were randomly divided into PED group (n = 18) and NED group (n = 17). Both, in the classroom and in the laboratory, the participants completed the tests related to learning material.

Fig. 1

Student distribution

It is possible that emotion and learning performance of the participants could be affected by many factors during the learning process, e.g. design elements of learning material, prior knowledge on meteorology, age, gender, year in college, etc. In this present study, we controlled all of these factors very strictly. For example, we divided students into two groups in such way that, based on analyses of variance (ANOVA), the groups did not differ significantly in terms of their prior knowledge, experience, age, year in college, or proportion of men and women in the groups. The only difference between two groups was emotional design.

Material and measures

Material

It is advised that researchers replicate or re-analyze the findings of other scholars in educational research (Towne & Shavelson, 2002). Learning material created by Um et al. (2012) or Mayer and Estrella (2014) based on emotional design was often used by other scholars in emotional design studies. In this present research, we used the multimedia learning material developed by Mayer and Moreno (1998). We aimed to explore the role of emotional design in learning. The multimedia learning material titled “The Formation of Lightning” was designed for the computer- assisted instruction in Mayer and Moreno (1998). The content was about the principle of lightning and developed for university students. It depicts air moving from the ocean to the land, water vapor condensing to form a cloud, the rising of the cloud beyond the freezing level, the formation of ice crystals in the cloud, the movement of updrafts and downdrafts, the building of electrical charges within the cloud, the division of positive and negative charges, the traveling of a negative stepped leader from the cloud to the ground, the traveling of a positive stepped leader from the ground to the cloud, the negative charges following the path to the ground, the meeting of the negative leader with the positive leader, and the positive charges following the path towards the cloud. “The Formation of Lightning” learning material was developed as four-minute flash animation and presented to the participants using PowerPoint® software.

Learning material was developed based on emotional design and included neutral or positive visual elements. Learning material with the neutral emotional design included elements in monochromatic grayscale colors such as black, white, and gray and contained no sound media. Learning material developed based on the positive emotional design included the same elements but in a vivid color, such as blue, green, etc., and with the anthropomorphism features such as baby-face bias to the ice crystals and water drops (e.g. blue water drops with black eyes, and the aggressive ice crystals), and the sound of lightning. Therefore, there was no difference in learning content provided to two groups that would confound the results. The relevant verbal information was given by narration. Additionally, the visual part of the material incorporated structuring text elements (in Chinese), pictures, and animations. The pacing of the material was the same for all learners and could not be replayed. A screen shot of the emotional design for two groups is shown in Fig. 2.

Fig. 2

Emotional design for NED and PED groups

Prior knowledge, retention and transfer tests

Knowledge of meteorology prior to this study was measured in order to control individual differences such as prior knowledge. The eight question items for measuring the prior knowledge were adopted from Mayer and Moreno (1998). There was a 7-item checklist (e.g., “I know what a cold front is”, “I regularly read the weather maps in the newspaper” with “Yes” answer scored as “1” and “No” answer scored as “0.” The last item was a self-rating question in which the participants were asked to report the degree of their meteorology knowledge on a 1-item 5-point scale with “1” equal to “very little” and “5” equal to “very much.”

Student learning performance was represented by scores of the retention and transfer tests. Therefore, the characteristics of successful learning performance was high scores on the retention and transfer tests. The retention test consisted of the following question: “Please write down an explanation of how lightning works” (Mayer & Moreno, 1998).

The transfer test consisted of the following four questions: “What could you do to decrease the intensity of lightning?” “Suppose you see clouds in the sky, but no lightning. Why not?” “What does air temperature have to do with lightning?” and “What causes lightning?” (Mayer & Moreno, 1998).

The positive affect scale (PAS)

The Positive Affect Scale (PAS) was a revision of PANAS adapted for Chinese learners (Qiu, Zheng, & Wang, 2008; Watson, Clark & Tellegen, 1988). This instrument was validated through thorough scrutiny of experts in the field. In PAS, respondents were asked to answer the emotional reactions which they experienced to different emotional design. There were nine items, i.e. active, passionate, happy, elated, excited, proud, delighted, energetic, grateful, on a 5-point Likert-type scale ranging from 1 (very slightly or not at all) to 5 (very much). Coefficient alpha was calculated and reached 0.85 and the internal consistency reliability of the scale in this experiment was 0.92. In this present study, PAS was used twice; before and after the learning process.

Physiological measurement

Following general recommendations of Shen et al. (2018), the Spirit-16 system was used to collect the physiological data (i.e. GSR and EEG) in real time. In addition, BioTrace+, the software of the Spirit-16 system, decomposed the EEG data into required theta/beta.

Procedure

The tests were completed using computers and questionnaires using paper and pencil. Each subject was tested either in a classroom or laboratory and the total test time was about 45 min.

Experiment in classroom

281 participants took part in the classroom experiment. Each participant was seated at an individual computer and tested in an individual cubical, so they could not communicate with each other. Participants were asked to sign an informed consent form and filled the positive affect scale (PAS1 – pre-test). Then, participants were given verbal instructions from the experimenter explaining that they would see two presentations of a slideshow on the formation of lightning and would later be answering a series of questions about what they learned. Participants viewed the slideshow presentation. After that, participants answered the questionnaires. After all participants had completed the questionnaires, they completed the 5-min retention test. Then they answered the transfer test questions, for which they were given 15 min. Finally, participants filled PAS again (PAS2 – post-test).

Experiment in laboratory

35 participants attended the experiment in the laboratory. None of the participants reported any history of medical illness or psychotropic medication. All participants denied use of any medication that would affect the cardiovascular, respiratory, or central nervous system. A written consent was obtained from the participants at the beginning of the study when they were introduced to the experiment protocols.

First, the participants were informed about the experimental requirements. After that, the participants were asked to wear headsets and adjust volume levels (if needed) while watching the test video. Next, they were asked to complete the positive affect scale (PAS1 – pre-test). Then the electrode plates were installed. The participants listened to the light music and then watched the 4-min lightning flash animation. They completed the 5-min retention question. After that, they answered the transfer test questions for which 15 min were given. Finally, participants filled PAS again (PAS2 – post-test).

The recorded data was divided into 5 parts: 1 (baseline: subjects listened the light music), 2 (experiment: 1 min), 3 (experiment: 2 min), 4 (experiment: 3 min), 5 (experiment: 4 min).

The collected data was analyzed in SPSS20.0. We set a priori alpha-level at 0.05 since an alpha level of less than 0.05 is accepted in most educational research as statistically significant.

Data analysis

The data in which some answers were missing was considered as invalid. The data was analyzed as follows:

• 1. Scoring student answers to the items of the positive affect scale prior to the experiment (PAS1).

• 2. Evaluating and scoring student answers to the items of the prior knowledge test. We evaluated and scored student answers according to general guidelines of previous studies (Mayer & Moreno, 2002; Cheon et al., 2014; Chung et al., 2015). Each sentence of student answers as “0” if it did not contain any key concept and as “1” if it contained one key concept.

• 3. Collecting and filtering the physiological measurement data. In learning process, physiological signal was not often stable because of poor contact or student movements (e.g. head). Such noise in the data can be filtered out by the range of frequency, like theta wave frequency of 4 ~ 8 Hz. In order to explore the difference between the different subjects’ GSR and EEG, it is necessary to normalize the GSR and EEG, so that human body dependency is neglected (Sharma & Kapoor, 2014). Normalized data was obtained using the following equation:

$$ \mathrm{normalized}\_\mathrm{GSR}=\mathrm{experimental}\_\mathrm{GSR}-\mathrm{baseline}\_\mathrm{GSR} $$

• 4. Evaluating and scoring student answers to the items of the retention and transfer tests. We evaluated and scored student answers in the same way as we did for the prior knowledge test. In order to ensure the objectivity of the scoring of the pre-test, retention and transfer tests, there were two teachers who scored student responses to the tests. Inter-rater reliability of average score of the two teachers’ marking was very high (Creswell, 2014). Average score of the two teachers was given as the final score.

• 5. Scoring student answers to the items of the positive affect scale after the experiment (PAS2).

We employed three statistical tests in this present study. An independent t-test was used to assess if the differences in learning performance and emotion between two groups were statistically significant. Repeated measures ANOVA was administered to test the changes of student emotion. Mann-Whitney non-parametric test was carried out to compare the data obtained from GSR and EEG during two different test periods (i.e. baseline and experimental). According to Creswell (2014), an independent t-test is used to test for differences between two independent groups, repeated measures ANOVA is used to test the effects of a continuous dependent variable measured several times, and the Mann-Whitney non-parametric test is used to test whether two independent samples arise from populations with equivalent means.

Results

Does emotional design affect retention and transfer scores?

The results of retention and transfer tests for two groups are presented in Table 1. Based on t-test analysis, retention test scores in the PED group (n = 199, M = 3.74; SD = 2.52) were significantly higher than those in the NED group (n = 117, M = 3.21; SD = 2.24), t(314) = 2.042, p = 0.042 < 0.05, d = 0.24.

Table 1 Mean and standard deviation values of retention and transfer scores for two groups

The results also showed that the PED group (n = 199, M = 1.4; SD = 0.81) performed significantly better than the NED group (n = 117, M = 0.85; SD = 0.99) on the transfer test, t(314) = 5.454, p = 0.000 < 0.05, d = 0.64).

These results suggest that the positive emotional design had an influence on student learning performance on the retention and transfer tests.

Does emotional design affect student emotion?

The mean scores and standard deviation values for two groups obtained by PAS are shown in Table 2. The t-test analysis results showed that there was not a significant difference between the two groups in terms of their emotion.

Table 2 Mean score and standard deviation on PAS scores for two groups

For PAS1, the scores of the PED group (n = 199, M = 24.68; SD = 7.29) were slightly lower than those of the NED group (n = 117, M = 24.77; SD = 8.44), so there was not a significant difference between the two groups, t(314) = − 0.101, p = 0.920 > 0.05, d = − 0.01.

For PAS2, the scores of the PED group (n = 199, M = 20.73; SD = 7.33) were slightly higher than those of the NED group (n = 117, M = 20.69; SD = 10.02); still there was not a significant difference between the two groups, t(314) = 0.042, p = 0.960 > 0.05, d = 0.01.

To further explore the changes of student emotion, the repeated measures ANOVA was carried out. The two measurement points of the PAS were as repeated measures and between-subject factors.

It was found that there was a main effect for the measurement point of the PAS within-subjects comparisons (F(1, 314) = 44.913, p < 0.05, η2 = 0.13). However, according to the results, there was no main effect between-subjects comparisons (F(1, 314) = 0.001, p = 0.973 > 0.05, η2 = 0.92), no interaction effect between measurement points of the PAS and the emotional design (F(1, 314) = 0.012, p = 0.912 > 0.05, η2 = 0.99). Therefore, the change of learner’s emotion was not significant.

Our results suggest that the emotional design had no influence on student emotion.

Is there any changes in three emotional dimensions?

Evidence from GSR and EEG data

Mean and standard deviation values of physiological measurements such as GSR and theta/beta and the results of the Mann–Whitney non-parametric tests are reported in Table 3. According to the results, in the baseline period, there was not a significant difference in GSR between the PED group (n = 18, M = 4.45, SD = 0.15) and NED group (n = 15, M = 3.12, SD = 0.06), U = 82.500, Z = -1.898, p = 0.058. Also, insignificant difference was found in theta/beta between the PED group (n = 18, M = 9.25, SD = 3.87) and NED group (n = 15, M = 10.46, SD = 3.05), U = 116.000, Z = -0.687, p = 0.492.

Table 3 Means and standard deviations of GSR and theta/beta for the two groups

In terms of the change in GSR from baseline to experimental period, there was also not a significant difference between the PED group (n = 18, M = -0.33, SD = 0.14) and NED group (n = 15, M = -0.35, SD = 0.12), U = 119.000, Z = -0.578, p = 0.563. In addition, no significant difference in theta/beta was found between the PED group (n = 18, M = 1.6, SD = 1.05) and NED group (n = 15, M = 4.49, SD = 0.63), U = 103.000, Z = -1.157, p = 0.247.

The change trend of GSR and theta/beta for two groups

As mentioned earlier, the of GSR and theta/beta data was recorded during 5 different time periods: 1 (baseline), 2 (experiment: 1 min), 3 (experiment: 2 min), 4 (experiment: 3 min), 5 (experiment: 4 min). The change trend in five periods for two groups is shown in Fig. 3.

Fig. 3

Change trend graph of GSR and Theta/Beta for two groups

The change trend of GSR was on the decline and consistent for the PED and NED groups. The GSR of the PED group was higher than that of the NED group, and the GSR was positively correlated with the activation of emotion, that is, those who learned with positive emotional design had more obvious activation of emotion.

The theta/beta values of the PED and NED groups had increased in stages 2 and 3, but decreased gradually in the stages 4 and 5. In addition, the theta/beta values negatively correlated with attention. That is, attention of the participants in two groups decreased significantly at the beginning of learning, and then their attention began increasing after they learned for about 2 min.

However, the theta/beta value of the PED group was lower than that of the NED group, and the theta/beta value of the PED group rose slightly slower and smaller (amplitude) at the beginning of the experiment, which indicated that those who learned with emotional design paid more attention on learning material.

Discussion and conclusion

Emotional design has a positive effect on learners’ retention and transfer performance

Compared to the NED group, the PED group had significantly higher scores on the retention and transfer tests. The emotional design facilitated learning outcomes with respect to retention as well as transfer test scores. Earlier related studies suggested that positive emotion can facilitate learning (Fassbender et al., 2012; Goetz et al., 2007). The benefits of emotion on learning performance was explained through information processing theory (Siegler, 1998; Slate & Charlesworth Jr, 1988) as well as the emotions-as-suppressor-of-learning and emotions-as-facilitator-of-learning hypotheses (Blanchette & Leese, 2010; Knörzer et al., 2016; Um et al., 2012; Uzun & Yıldırım, 2018). However, our results showed no relationship between emotional design and student emotion. According to our results, student emotion had decreasing trend line, and therefore, it could not be accounted for any effects on learning performance. Other scholars, in line with CATLM, explained that incorporating visually appealing emotional elements (e.g. objects colored in bright and warm colors) fosters learning (Mayer & Estrella, 2014). For example, the emotional design can be useful for students when they are engaged in selecting, organizing, or integrating essential learning information more efficiently (Moreno, 2009). As a result, students can remember learning information, retrieve or apply it to different context better (Mayer, 2005). Perhaps, this notion can explain the effects of the emotional design on learning performance of the participants in this present study.

The emotional design of this present study involved modification of the key elements of the learning material on the formation of the lightning. Namely, we changed colors, shapes, and anthropomorphism features of key elements in learning material. Therefore, the educators can also re-design key elements in their learning material with the emotional design in order to facilitate learning performance of students in multimedia learning.

Our findings provide an important replication and extension of earlier published studies (Mayer & Estrella, 2014; Plass et al., 2014; Um et al., 2012) showing the benefits of the emotional design on learning performance in multimedia learning. Thus, the major contribution of this work is to show the effects of the emotional design on learning performance based on the evidence collected in different learning environments (i.e. classroom and laboratory), using different research instruments (i.e. objective and subjective) and learning material, and with participating students from different demographics.

Emotional design has insignificant effect on emotion of learners

There was not a significant difference in PAS, GSR, and theta/beta between two groups. In addition, with respect to three dimensions of emotion (i.e. pleasantness, attention, and activation), the PED and NED groups had equal scores. Therefore, we may conclude that there were not any interaction effects between the emotional design and student emotions. This finding shows that the emotional design does not induce student positive emotions in multimedia learning. This result contradicts those obtained in the previous studies by Um et al. (2012) and Plass et al. (2014).

The reason for this phenomenon may relate to the emotional design. That is, in this present study, we only used such elements of the emotional design as colors and shapes of objects in learning material. In other related studies (Um et al., 2012; Plass et al., 2014), scholars attempted to induce positive emotions by using colors and shapes not only of objects but also of the narration text. The effects of variations of elements of the emotional design could better induce positive emotions. Therefore, we may conclude that redesigning colors and shapes of objects in learning material as in the present study may not be enough to induce learners’ emotions.

Another possible reason is that learning material was shown to the participants for too short time or learning material contained too much knowledge to be learned in limited time so that there was not enough time for their emotion induction. Furthermore, learning material presentation was controlled by the instructor and the participants could not stop or replay it; this could also possibly prevent inducing positive emotions. According to the control-value theory of academic emotions (Pekrun, 2006), when learners lose control or think that success is unattainable, their positive emotions cannot be induced and they experience disappointment, boredom, and anger.

Our results related to inducing emotion with the emotional design contradicts previous findings and no explanation can be derived from the obtained data of the present study. Hence, further systematic investigations on this issue are necessary. Xue, Qian, Wang, and Zhou (2015) also found that the emotional design did not induce positive emotion, but it was useful in improving intrinsic motivation. As the emotions-as-facilitator-of learning hypothesis assumes, emotion can facilitate the cognitive processes by enhancing motivation, interest or other factors. This aspect can be explored in future studies.

According to the data collected from PAS1 and PAS2 as well as the GSR, positive emotions of the participants in two groups declined from the beginning to the end of the experiment. This finding echoes those obtained from the previous studies. Scholars suggested that the GSR rises instantaneously when learners feel excited because they start working on a task. After a short time, the GSR usually drops gradually which indicates that learners calmed down. The GSR increases again only when a new event occurs (Lawler, 1980; Dawson, Schell, & Filion, 2007. Andreassi and Whalen (1967) found that there were decreases in GSR and this results was associated with student overwhelming; there were increases in GSR again in learning and they were associated with new overwhelming. Therefore, we may conclude that the GSR level raised right at the beginning of the experiment because the participants were overwhelmed and it gradually decreased because they calmed down.

In this study, we could not get any proof of the emotional design induced positive emotions. However, the emotional design improved the outcomes from the retention and transfer tests. This result proves the predictions of the Cognitive Affective Theory of Learning with Media (CATLM) and extends the theory of the Cognitive Theory of Multimedia Learning (CTML), which advocates that students are more encouraged to engage in appropriate cognitive processing during learning when the essential graphics in a lesson are re-designed to be more appealing. Therefore, in order to facilitate learning performance, the emotional design needs to be considered in multimedia learning.

Implications

The results showed that emotional design has a positive effect on learning outcomes which is consistent with the notion of Cognitive Affective Theory of Learning with Media (CATLM) (Moreno, 2005, 2006; Mayer & Estrella, 2014). That is, emotional design was beneficial for retention and transfer test scores. As Mayer (2014) said, emotional design can prime and sustain improved cognitive processing during learning (i.e., selecting, organizing, and integrating), which in turn leads to improved learning outcomes. Emotional design can direct the learner’s attention towards essential material and encourage learners to get to know the essential material better (Mayer & Estrella, 2014).

In multimedia learning, educators and researchers should make full use of the emotional design to arouse students’ interest and improve their academic performance. The emotional design provides a new direction for designing multimedia learning material. Such material design does not only focus on usage of pictures, texts or animations, but also begins considering emotion and pays attention to inducing emotions of students in learning.

Future research directions

According to our research results, the emotional design has insignificant effect on emotion during learning. The findings are inconsistent with those obtained by Um et al. (2012). The emotional design is a complex concept. The mechanism of the emotional design to affect learning outcomes is not very clear. We infer that the emotional design may affect learning interest, learning attitude, learning motivation, and memory. Based on CATLM, motivational factors mediate learning by increasing or decreasing cognitive engagement and metacognitive factors mediate learning by regulating cognitive processing and affect (Moreno, 2005). Therefore, in the future, the mechanism of the emotional design should be studied deeper. The relationship between emotion and learner interest, motivation, and memory should be studied further. In particular, the relationship between emotion and learners’ motivation and metacognition is worth further exploration. Addressing the issue of age in the learners when emotional design is applied is another future research direction. It will be useful to investigate and show evidence of whether younger learners are more prone to be influenced emotionally and cognitively by positive emotional design.

Limitations

There are still some limitations in this study, which can be improved in the future. This experiment is only a one-time experiment and so follow-up long term studies with larger population should be carried out to validate our results. The material of this study showed too much knowledge in limited time, so the learners had little time for processing and integrating the knowledge. In addition, learners could not control the material which may prevent from inducing their positive emotions. Furthermore, learning material in this study was about lightning and so it is not clear whether similar results can be obtained when learning content is related to other domain knowledge. All these issues should be addressed in the future. Another issue to consider is that all participants in this study were from China and therefore, our results should be treated with caution. Perhaps, different findings can be obtained with participants from different socio-cultural backgrounds.