Examining the key influencing factors on college students’ higher-order thinking skills in the smart classroom environment

Abstract

To understand the development of students’ higher-order thinking skills (HOTS) in the smart classroom environment, a structural equation modeling analysis was used to examine the relationships between key factors that influence students’ learning and their HOTS within a smart classroom environment. A sample of 217 first-year Chinese college students, who studied in a smart classroom environment for one semester, completed a survey that measures their smart classroom preferences, learning motivation, learning strategy, peer interaction, and HOTS. The results indicated that peer interaction and learning motivation had a direct impact on students’ HOTS. Furthermore, indirect effects were found between students’ learning strategy and HOTS through the mediator peer interaction, and between smart classroom preferences and HOTS through the following: learning motivation, the combination of learning strategy and peer interaction, and the combination of learning motivation, learning strategy and peer interaction. Based on these findings, this study recommends that instructors teaching in a smart learning environment should focus on improving peer interaction and learning motivation, as well as smart classroom preferences and learning strategy, to hone students’ HOTS.

Introduction

Smart classrooms have gained the attention of scholars and educators worldwide. In 2017, the EDUCAUSE Center for Analysis and Research identified technology-enabled learning environments as a strategic investment for colleges and universities (Brooks 2017; Lee et al. 2019). It also predicted that the smart classroom would become widely utilized by 2022.

The term ‘smart classroom’ refers to a physical classroom that integrates advanced forms of educational technology. Such an environment provides opportunities for student learning and participation in formal educational learning experiences that exceed what traditional classrooms can offer (Macleod et al. 2018). Li et al. (2015) identify four features of the smart classroom. First, the smart classroom is a technology-rich learning environment that combines physical and virtual spaces. Second, the smart classroom provides information and communication technology tools, learning resources, and interaction support for various teaching and learning activities, including personalized learning, group learning, inquiry-based learning, collaborative learning, mobile learning, and virtual learning. Third, the smart classroom is capable of storing, collecting, computing, and analyzing learners’ data in order to make optimized pedagogical decisions. Fourth, the smart classroom is an open environment that brings the learners to an authentic learning context. Previous studies have also indicated that the smart classroom environment can stimulate students’ learning motivation, promote active learning, and improve academic performance (Jena 2013; Liu et al. 2011). However, the impact of the smart classroom environment on students’ higher-order thinking skills (HOTS) is less clear.

The importance of HOTS has been emphasized by policymakers, educators, researchers, and the general public (Abosalem 2016; Elfeky 2019; Lu et al. in press). Upon conducting an analysis of previous studies, Hwang et al. (2017) identified three HOTS: problem-solving, critical thinking, and creativity. Problem-solving refers to the ability to identify a problem, collect and analyze relevant information, select and implement a relevant solution. Critical thinking refers to the ability to analyze information objectively, think clearly and rationally, and make a reasoned judgment. Creativity refers to being able to create new objects and develop innovative ideas and methods by elaborating upon, refining, analyzing, and evaluating existing ones.

Researchers argue that HOTS fall under the umbrella of twenty-first-century skills, which comprise the essential skills that youth need to prepare for the future (Ananiadou and Claro 2009; Collins 2014). Therefore, it is important for educators to identify and use learning environments that stimulate the development of students’ HOTS.

Purpose of this research

To date, few studies have been done from the students’ perspective on the relationships between the key factors influencing students’ learning and students’ HOTS, when they are instructed in a smart classroom context. Only one recent study by Wu et al. (2019) has attempted to investigate the effects of students’ learning motivation, learning style, and internet attitude on their HOTS in the smart classroom. Their study examined data from 784 students in primary schools and found that students’ learning style and internet attitude had a direct impact on the students’ HOTS, but learning motivation did not have a significant impact on their HOTS. Furthermore, the influence of other key learning factors on HOTS in the smart classroom has remained unclear. Particularly, very few studies have explored the relationship between the key factors influencing college students’ learning and their HOTS within the smart classroom environment. Understanding the factors that influence HOTS can help educators and curriculum designers develop more rigorous learning opportunities and assessment tools.

Thus, this study aims to fill the existing research gap by investigating the following research question:

What are the relationships of the key factors influencing student learning and college students’ HOTS in a smart classroom environment?

Research framework

Previous studies have explored the various factors that are associated with student achievements in terms of skills and knowledge in other learning environments. In general, the key factors that influence student learning include classroom preference (Moore 1989; Tsai 2008), learning motivation (Pintrich1999), learning strategy (Garcia and Pintrich 1992), and peer interaction (Hwang et al. 2017; Osman et al. 2011).

Learning environment preferences

Learning environment preferences refer to students’ perceptions of a specific learning environment (Fraser 1998). Accordingly, students’ smart classroom preferences (SCP) are about their perceptions of the smart classroom (MacLeod et al. 2018). Students’ preferences toward a certain learning environment have increasingly drawn attention from educators; it is believed that if educators know about their students’ perspectives of their learning environment, they can make the necessary adjustments (Chuang and Tsai 2005).

Previous studies indicate that classroom preferences affect students’ learning in different educational environments. For example, Chang et al. (2010) found that students’ learning outcomes were in alignment with their environment preferences in a classroom setting where student-centered and teacher-centered instructional approaches coexisted. Furthermore, Hwang et al. (2017) found that student preferences toward the mobile learning environment were related to HOTS.

Learning motivation

Learning motivation (LM) prompts individuals to take actions that will help them achieve a goal, or fulfill a need or expectation in the learning process (Gopalan et al. 2017). Although there is no consensus on the matter, a prominent study conducted by Pintrich et al. (1991), which identifies the three general motivational constructs: value, expectancy, and affect.

Previous studies demonstrated that students’ LM is a fundamental link between student performance and achievement in various learning environments. For example, Roberts and Dyer (2005) found that students’ learning motivation could be associated with their critical thinking, an aspect of HOTS, in an online learning environment. Similarly, Gong et al. (2020) stated that students’ learning motivation had a direct impact on their computational thinking skills, which includes creativity, algorithmic thinking, cooperation, critical thinking, and problem-solving in a flipped classroom setting. Conversely, Wu et al. (2019) reported that students’ learning motivation did not affect HOTS in the smart classroom environment.

Learning strategy

Learning strategy (LS) refers to “a set of processes or steps that can facilitate the acquisition, storage, and/or utilization of information” (Dansereau 1985). A key study of LS by Pintrich et al. (1991) identifies cognitive, metacognitive, and resource management strategies as the main components of LS.

Researchers have determined that LS positively influences student skills and knowledge development in learning environments. Mayer (1998) argued that the cognitive and metacognitive components of LS had an important influence on successful problem-solving in traditional academic settings. This view has been supported by Gong et al. (2020) who reported that students’ learning strategy had a direct impact on certain HOTS such as creativity, critical thinking, and problem-solving in a flipped classroom environment. Moreover, Wilgis and Mcconnell (2008) and Soltis et al. (2015) found that specific learning strategies such as process-oriented, guided inquiry and concept mapping could significantly improve students’ HOTS in a blended learning environment.

Peer interaction

Peer interaction (PI) is “a form of cooperative learning that enhances the value of student-to-student interaction and results in different advantages of learning outcomes” (Christudason n. d.). Hwang et al. (2017) have indicated that PI includes collaboration and communication skills. Specifically, collaboration is the ability of two or more people to work together and share their perspectives and ideas with respect to achieving learning goals or completing learning tasks. Communication refers to the ability to ‘‘articulate thoughts and ideas effectively by using oral, written and nonverbal communication skills in a variety of forms and contexts’’ (Frazier and Reynolds 2012).

Previous studies point out that PI is an important factor influencing students’ learning outcomes. For example, Tsai et al. (2011) found that students’ PI could be a predictor of HOTS in a constructivist context-aware ubiquitous learning environment. Hwang et al. (2017) also verified that collaboration and communication were positively related to HOTS in a mobile learning environment.

The relationships between learning environment preferences, LM, LS, and PI

Previous studies have explored the relationship between learning environment preferences, LM, LS, and PI in various educational contexts (Al-Khaldi and Al-Jabri 1998; Houle 1996; Hwang et al. 2017; Tsai 2008). For example, Houle (1996), as well as Al-Khaldi and Al-Jabri (1998), argued that students’ classroom preferences could affect students’ learning motivation in the technology-supported classroom. Moreover, Tsai (2008) found that students’ preferences toward the constructivist Internet-based learning environment were related to their learning strategies and outcomes.

In addition, existing research shows that both learning motivation and learning strategy have positive influences on peer interaction (King 1991; Yang and Chang 2011). For instance, Yang and Chang (2011) found that learning motivation was positively related to students’ peer interactions in an interactive blogging learning environment. Besides, Tsuei (2011) found that a peer-assisted learning strategy was positively related to peer interaction in a computer-supported collaborative learning environment. Furthermore, existing research has shown that motivation has a critical effect on strategy choices (Ellis 1994; Gong et al. 2020). For example, Gong et al. (2020) reported that there was a positive relationship between LM and LS in the flipped classroom instruction environment.

The relational model and hypotheses

Based on our review of related studies, we believe that students’ learning environment preferences, LM, LS, and PI can influence student achievements related to skills and knowledge in various learning environments (Ananiadou and Claro 2009; Pintrich et al. 1991; Tsai et al. 2011). Therefore, as shown in Fig. 1, we assume that SCP, LM, LS, and PI may influence students’ HOTS when they are taught in a smart classroom. Our hypotheses are as follows:

Fig. 1

Proposed research model and hypotheses

Hypothesis 1 (H1):

The level of SCP will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 2 (H2):

The level of LM will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 3 (H3):

The level of LS will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 4 (H4):

The level of PI will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 5 (H5):

The level of SCP will be positively related to the degree of college students’ LM within a smart classroom environment.

Hypothesis 6 (H6):

The level of SCP will be positively related to the degree of college students’ LS within a smart classroom environment.

Hypothesis 7 (H7):

The level of LM will be positively related to the degree of college students’ LS within a smart classroom environment.

Hypothesis 8 (H8):

The level of SCP will be positively related to the degree of college students’ PI within a smart classroom environment.

Hypothesis 9 (H9):

The level of LM will be positively related to the degree of college students’ PI within a smart classroom environment.

Hypothesis 10 (H10):

The level of LS will be positively related to the degree of college students’ PI within a smart classroom environment.

Methods

Participants

To investigate the research question, this study used a total number of 217 students enrolled in the Ideological and Moral Cultivation and Legal Basis (IMCLB) course at a university in central China. Both the course and the university were purposely selected for two reasons. Firstly, the course is a compulsory general course for all first-year students at the university. As such, the number of students taking the course allowed us to collect a sufficient number of participants from different disciplines. Secondly, the university attaches great importance to information technology and has built several smart classrooms. All university instructors are provided with training opportunities to learn how to use smart classroom technologies, and were encouraged to conduct their instructional practices in the smart classroom. Particularly, most instructors of the IMCLB course at this university have taught its content in the smart classroom for 3 years.

At this university, the IMCLM course is a semester (12 weeks) long. Instructors and students meet one or two times per week. All classes refer to the same learning materials and facilities in the smart classrooms. Participating students were organized into groups for learning activities. Each group had 4–5 students, organized in a cluster seating arrangement, allowing them to interact with one another and easily work together.

Instruments

The survey adopted elements of the Collaboration, Communication, Critical Thinking, Problem-solving and Creativity Awareness questionnaire (4C1PA), the Preference Instrument of Smart Classroom Learning Environments (PI-SCLE), and the Motivated Strategies for Learning Questionnaire (MSLQ) to measure students’ HOTS, PI, SCP, LM, and LS.

The 4C1PA was developed by Hwang et al. (2017), and consists of five dimensions capturing students’ HOTS and their tendency to engage in PI. The HOTS tendency subscale (alpha = 0.888) consists of a three-dimensional construct: problem-solving, critical thinking, and creativity. Each dimension has three items. One representative item of this scale is: “I like to observe something I haven’t seen before and understand it in detail.” The Peer Interaction subscale (alpha = 0.858) consists of a two-dimensional construct: collaboration and communication. Each dimension has three items. One representative item of this scale is: “I try to provide useful and sufficient information when I conduct collaborative learning.” All items of 4C1PA were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).

The PI-SCLE was developed by MacLeod et al. (2018), and includes eight distinct dimensions: student negotiation, inquiry-based learning, reflective thinking, functional design, connectedness, ease of use, perceived usefulness, and multiple sources (alpha = 0.951). Each dimension has three items. All items were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). ‘‘In the smart classroom learning environment, I prefer that I can get the chance to talk to other students’’ is one representative item of the student negotiation dimension.

The MSLQ was developed by Pintrich et al. (1991), and includes LM (alpha = 0.833) and LS (alpha = 0.863). LM has three dimensions: value component (8 items), expectancy component (5 items), and affective component (3 items). LS has two dimensions: cognitive and metacognitive strategy (14 items) and resource management strategy (11 items). All items were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). One representative item of LM is “In a class like this, I prefer course material that really challenges me so I can learn new things.”

Data collection and analysis

Data were collected at the end of the semester. Before the survey was administered, permission was granted by the university to conduct the research. All 217 participants were introduced to the purpose of the research by a researcher of this study during their instructor’s absence. Participants were informed that their information would only be used for educational research and that their survey results would not affect their grades in the course. All responses were both anonymous and given voluntarily. The survey was issued during a mid-class break, then imported into SPSS 22.0 and SmartPLS 3.2.8 for data analysis. A structural equation modeling analysis was conducted to analyze the relationships between the key influencing factors and students’ HOTS.

Results and discussion

The partial least square (PLS) method was used to verify the proposed research model. PLS was appropriate for the sample size of this study (Chin 1998; Gefen et al. 2000) and well-suited for testing theories in the early stages of development (Fornell and Bookstein 1982). Hair et al. (2014) introduced the Standardized Root Mean Square Residual (SRMR) as a goodness of fit measure for PLS-SEM that can be used to avoid model misspecification. In general, a value less than 0.08 is considered a good fit (Hu and Bentler 1998). The value of SRMR of the model in this study was 0.06, thus, the goodness of fit for the proposed model was verified as acceptable.

Confirming the measurement model

The measurement model was assessed by the reliability of measures, convergent validity, and discriminant validity. As shown in Table 1, the average variance extracted (AVE) values for all factors were over 0.6, which suggested adequate convergent validity (Fornell and Larcker 1981). The reliability of the measurement model was examined using the composite reliability and Cronbach’s alpha.

Table 1 Results of the measurement model

Findings indicated that the composite reliability (CR) coefficients were over 0.8, which demonstrated satisfactory reliability (Nunnally and Bernstein 1994). Cronbach’s alpha was over 0.8 and within acceptable limits (Helmstadter 1964). Furthermore, to evaluate the discriminant validity, the square roots of AVE were compared to correlations among latent variables (Fornell and Larcker 1981), in which all latent correlations were less than the corresponding AVE square roots. Table 1 shows the results of the measurement model. In sum, the adequacy of the measurement model indicates that all the items were reliable indicators of the hypotheses they were purposed to measure.

Structural equation modeling analysis

A structural model was used to test the hypotheses using path coefficients (β value), R² value, and t-value bootstrapping (500 resamples) (Cohen 1988). The PLS path modeling estimation for this study is shown in Fig. 2. Path coefficients along with the associated t-values are provided and the variance given is explained.

Fig. 2

The structural model for HOTS

Findings reflect that H2, H4, H5, H6, H7 and H10 are supported, while H1, H3, H8, and H9 are rejected. LM (β = 0.244, p < 0.001) and PI (β = 0.665, p < 0.001) were positively related to HOTS, collectively accounting for 58.3% of R². In addition, SCP (β = 0.286, p < 0.001) and LM (β = 0.505, p < 0.001) positively impacted LS, accounting for 48.4% of R². SCP (β = 0.510, p < 0.001) had a significant positive effect on LM as well, accounting for 26% of R². Furthermore, LS (β = 0.298, p < 0.01) had a significantly positive effect on peer interaction, accounting for 16.9% of R².

Analysis of indirect and total effects among key factors

Further, the direct and indirect effects of factors in each hypothesis were examined (Ullman and Bentler 2003). As shown in Fig. 2 and Table 2, SCP had both direct and indirect influences on LS, and LM mediated the indirect influence. Furthermore, although SCP had no direct influence on HOTS, there were three indirect paths leading from SCP towards HOTS, where LM, LS, and PI acted as partial mediators. In addition, LM had both direct and indirect influence on HOTS. An indirect effect is reflected in the path from LM to HOTS through LS and PI, which suggests that the combination of LS and PI can also mediate the relationships between LM and HOTS.

Table 2 Analysis of indirect and total effects between key factors

Discussion of results

This study revealed that PI and LM were directly related to students’ HOTS in the smart classroom environment. This result may be explained by the fact that the smart classroom is a student-centered learning environment. Unlike the traditional teacher-centered classroom, a student-centered classroom is a place where the students are actively involved in the learning process (Utecht 2003). In a student-centered class, students no longer only rely on their instructor to give them instructions. Instead, students actively communicate, collaborate, and learn from each other, as well as apply and improve their critical thinking, problem-solving, and creativity skills (Jones 2007). It can thus be confirmed that PI and LM are two primary factors to students’ HOTS in a smart classroom. This finding suggests that instructors should endeavor to enhance students’ PI and LM, in order to promote students’ HOTS in the smart classroom environment. For instance, instructors should provide the opportunity for students to engage in self-directed learning, explore high-interest topics and ideas, work collaboratively on projects, and share in decision-making during the learning process (Jones 2007; Yang 2001; Yang et al. 2000).

It is interesting to note that although students’ SCP and LS had no direct effect on HOTS, both did have a significant and positive indirect effect on HOTS. First, the finding that SCP had no direct effect on HOTS was consistent with the previous study (Hwang et al. 2017), which found that students’ preferences toward the mobile learning environment had an indirect effect on HOTS via students’ interaction. This finding may be explained by the fact that HOTS were used to describe students’ learning outcomes, which are directly related to cognitive activities. Additionally, learning environment preferences were used to describe students’ perceptions of their learning environment. Second, different from a previous study (Gong et al. 2020), this study found that LS had no direct effect on HOTS. This result could be explained by the fact that this study was conducted in the smart classroom environment, which is different from previously used learning environments. In the smart classroom, students were more engaged in the self-directed learning activities. Students’ LS may directly reflect on PI, thus, LS had indirect effect on HOTS via PI.

This study found that SCP positively impacted LM and LS, and LS had a significantly positive effect on PI. Furthermore, the association between SCP and HOTS was mediated by learning motivation, the combination of learning motivation, learning strategy and peer interaction, and also by the combination of learning strategy and peer interaction. Meanwhile, LS had a significant and positive indirect effect on HOTS via the mediating factor peer interaction. This finding suggests that, in order to develop students’ HOTS in the smart classroom environment, instructors and instructional designers should also take SCP and LS into account. For instance, to best meet students’ learning needs, interests, strategies, and abilities, instructors and instructional designers should better incorporate the constructs of the smart classroom environment and technology into the learning process. These constructs include student negotiation, inquiry learning, reflective thinking, ease of use, perceived usefulness, multiple sources, connectedness, and functional design (MacLeod et al. 2018).

Conclusions and future research

Given the importance of HOTS and the prevalence of smart classrooms in higher education, it is critical to understand the relationships between students’ HOTS and the key influencing factors, when learning in a smart classroom environment. This study proposed a research model and used a survey to collect data from 217 college students who had learning experience within a smart classroom environment. A structural equation modeling analysis method was used to explore the relationships between the four key factors influencing student learning (SCP, LM, LS, and PI) and students’ HOTS. The results of this study expand our knowledge of these key factors affecting students’ problem-solving, critical thinking, and creativity skills. These results can be used to inform educational processes and pedagogy, which will improve students’ HOTS in the smart classroom environment. The most significant findings of this study indicate that students’ PI and LM directly impact students’ HOTS in the smart classroom. In contrast, SCP and LS do not directly impact HOTS. This study supports the work of other studies, suggesting that peer interaction and learning motivation positively affects students’ ability to learn knowledge and skills in their learning environments (Gong et al. 2020; Hwang et al. 2017; Roberts and Dyer 2005; Tsai et al. 2011). In summary, the results of this study indicate that, in order to develop students’ HOTS, instructors should consider students’ learning motivation, peer interaction, learning strategy, and preferences toward the smart classroom when analyzing, designing, developing, implementing, and evaluating learning activities in a smart classroom environment.

While the present research has important implications, it still has limitations. It should be noted that we have only examined four important factors that influence student learning via a structural equation modeling analysis method. Moreover, the context was limited to one subject area conducted in the smart classroom environment. Future research is encouraged to involve more subject areas and more related factors, such as students’ learning styles and approaches to studying, and teaching methods and strategies. Particularly, future studies should extend to different subject areas with other related factors and employ a mixed-methods approach, like adding follow-up interviews or qualitative answers to capture the opinion of the students, to support the triangulation of quantitative results.