Student use and perception of technology enhanced learning in a mass lecture knowledge-rich domain first year undergraduate module

Abstract

The purpose of this study was to examine student use and perceptions of technology enhanced learning tools (TELTs), including their value for learning. Students enrolled onto a 12-week undergraduate science module had access to four TELTs each week, (i) a recording of the lecture (Panopto™), (ii) an animated mini review of the lecture (ShowMe), (iii) a multiple choice quiz hosted on Blackboard® (MCQ), and (iv) a module Twitter feed. Ninety-five students completed a survey at the end of the module, which included quantitative and qualitative questions, to examine whether they perceived the TELT to be useful for their learning. Analysis of the quantitative data suggest that Twitter was used significantly less than the other three TELTs (p < 0.001) with less people agreeing that it helped their learning (p < 0.001), whilst ShowMe and MCQ had a greater occurrence of an ‘agree’ rating compared to Twitter and Panopto (p ≤ 0.002). A thematic analysis of the qualitative responses identified assessment as a common theme across all four TELTs, being a positive factor for Panopto, ShowMe and MCQ, but negative for Twitter. Students highlighted ShowMe as being particularly useful for simplifying information. Based on this study TELTs similar to ShowMe (i.e. animations) are most recommended as this was one of the two highly rated TELTs (alongside MCQ), but may have more potential for crossover with other subjects, and students found it useful for more than just assessment.

Introduction

Technology enhanced learning (TEL) has the potential to enhance the student experience by facilitating self-paced learning, lowering inhibition thresholds for asking questions, and allowing access to learning on an as-and-when basis (Kamath, 2015), all of which are factors that may contribute to informal and incidental learning outside of the formal learning space (Peart et al., 2014). There is a body of research examining the role of the teacher in facilitating TEL, and a reservation or anxiety from staff has been described with respect to the technology acceptance model (Blackwell et al., 2014; Gong et al., 2004; Louw, 2015; Teo et al., 2008). Such problems may present a barrier to the promotion of TEL to support student learning. However, an aspect that has been less researched is the contribution of students to the impact of TEL, and in particular their perspectives on TEL tools (TELT). There is a wealth of information on student perceptions of a selection of ‘learning objects’ (LO) and ‘web based learning tools’ (WBLT) (Cochrane, 2005; R. Kay, 2011; R. H. Kay & Knaack, 2009; Nurmi & Jaakkola, 2006; Vargo et al., 2003). Such studies can provide a useful insight for LO and WBLT developers, but the implication from a teaching perspective is somewhat limited unless the teacher is likely to use that very specific tool in their teaching. For example Cochrane (2005) evaluated two audio engineering LOs (interactive mixing desk and microphone chooser) and Nurmi and Jaakkola (2006) evaluated three LOs to teach fractions, the Finnish language and electrical DC circuits. Such tools are typically designed to be used in the classroom, and as a result do not promote extra-curricular and informal/incidental learning.

It may be more beneficial from a teaching perspective to understand the benefits of more generic TELTs, which staff can have more control over, implemented over a longer period of time. However current examples are limited, and are focused primarily upon an institution virtual learning environment (VLE) or social networking. Šumak et al. (2011) collated Electrical Engineering and Computer Science students’ perceptions of using a general virtual VLE (Moodle) and reported that perceived usefulness was a strong predictor of intention to use the VLE. This again has limited scope today as the use of a VLE is now commonplace within Higher Education. However, the identification of perceived usefulness as a predictor of acceptance is important as it can direct research to now determine what characteristics students consider useful, or indeed how students define ‘useful’. Junco et al. (2011) incorporated Twitter into a semester long module for pre-health professional majors, where students were encouraged to continue class discussions, organise study groups, and connect with each other and with staff. They concluded that the use of Twitter successfully increased student engagement, which may present a promising development for a wide range of teachers as the uses could be incorporated into any subject of study. However, the authors did not provide any data on the student perception of the use of Twitter, therefore it is unclear why the students engaged with Twitter, and without knowing this it is difficult to predict what other TELTs may be well accepted.

Lecture capture is another example of a generic TELT available to teaching staff. A narrative review identified that students use lecture capture to review content, but there is mixed evidence as to its effect on student grades and attendance (Karnad, 2013). This practice has been examined specifically in higher education science students, namely in the areas of veterinary medicine (Danielson et al., 2014) and pharmacy (Marchand et al., 2014). Danielson et al. (2014) reported that students perceived lecture capture to be most useful for learning in content driven lecture sessions compared to group work sessions, particularly for reviewing segments flagged in their notes, recapping a fast lecture, studying for examinations, and reviewing content missed due to absence. Staff agreed lecture capture to be beneficial for students to review lecture content, but identified reduced attendance as a risk. Marchand et al. (2014) also identified reduced attendance as a concern for staff, however neither study reported actual attendance data so it is unknown if the staff concerns were realised. In a letter to the editor Lach and McCarthy (2015) challenged the findings of Marchand et al. (2014) stating that attendance should not be a concern, as attendance is not a learning outcome and does not guarantee that learning will occur. They argue that staff focusing on the possible negative effect upon attendance may shadow the opportunities afforded by technology.

A form of TELT currently more researched in younger ages is the use of electronic whiteboards to simplify and share information (Castek & Beach, 2013; Maher, 2013). The potential for these to support learning in higher education has received little attention in the literature, other than being described as a useful tool for in the moment teaching (Archibald et al., 2014). Using such a tool to deliver a short summary video of the lecture may act to reinforce learning from the lecture, rather than replace it. Simplifying the lecture content in such a way may also help students direct their learning, and has been proposed as a way to help students overcome troublesome knowledge, as they present scientific mechanisms in the absence of other barriers such new terminology (Peart et al., 2014).

An understanding of student perceptions of different TELTs could be of benefit for a number of reasons; (i) identification of shared characteristics between well perceived TELTs may help predict what tools are likely to work in future, (ii) targeting particular types of TELTs may act to reduce staff anxiety by reducing choice and preventing over saturation with TELTs, (iii) contribute to the planning of departmental and institutional TEL strategies. The objective of this study was to integrate four different TELTs that the teaching staff could control into a semester long undergraduate sport and exercise science module, with the aim of examining student use and perceptions of the TELTs. Of particular interest was their views on accessibility, use and value for learning.

Method

Participants

The study focused on a 12-week Level 4 (first year Undergraduate) module called Energetics of Exercise, which included 210 students from BSc Applied Sport and Exercise Science, BSc Sport, Exercise and Nutrition and BSc Psychology with Sport Sciences. The summative assessment for the module consisted of three multiple-choice examinations throughout the semester. All procedures were approved by the institution’s ethics committee, and all participants were provided with verbal and written information to ensure informed consent.

Experimental approach

The module was delivered primarily in a traditional manner with a one-hour lecture each week accompanied by 2 hours of additional contact time in a seminar, laboratory, or computer class. Alongside this traditional approach all of the students, irrespective of whether they chose to take part in the study, also had voluntary access to four different TELTs to support their learning each week. The TELTs were chosen with the intention of each serving a different function, to allow an insight into for what purpose students accessed the TELTs, as opposed to which TELT they preferred for a given purpose. It was also deemed important that the student could use each in his or her own time without the reliance of another person.

• 1. A Panopto™ recording of the lecture which allowed the students to download a video file (Fig. 1a). This was chosen as a simple TELT that would require no technological knowledge or extra-curricular effort from the staff, with the purpose of allowing students to recap the lecture content.

• 2. An animated mini review of the lecture using an interactive whiteboard mobile application (ShowMe), which consisted of a < 5-min video developed by the lecturer, focusing on what the lecturer deemed to be the essential part of the lecture (Fig. 1b). It was anticipated that the students would use these videos to recap on the main points as a start of, or alternative to, further reading.

• 3. A multiple choice quiz (MCQ) hosted on the VLE (Blackboard®) (Fig. 1c). Devised by the lecturer, these quizzes mimicked the summative assessment for the module to provide the students an opportunity for formative assessment.

• 4. A module Twitter feed sharing relevant information (Fig. 1d). Previous studies have identified Twitter as a tool to facilitate student engagement in a course, and encourage student discussion (Gikas and Grant, 2013; Junco et al., 2011).

Fig. 1

Screenshots of Panopto (a), ShowMe (b), MCQ (c) and Twitter (d)

Survey design and analysis

Student perceptions of the four TELTs were collected via an online survey in week twelve of the module (95 respondents). The survey used was adapted from the technology acceptance model (TAM) survey (Šumak et al., 2011) and a similar learning technology survey from Rossing et al. (2012) to include quantitative (7-point likert scale) and qualitative responses (Table 1). Questions that were deemed the most appropriate to the current study, with a focus on their use were selected from each of these inventories, and as the inventories were adapted both internal reliability and construct validity were checked. Both measurements were found to be satisfactory, determined by a Cronbach Alpha score > 0.7, and the average variance extracted >0.5. Given this, the median response for the six questions was taken as the overall score. This overall score was then classified as ‘did not use’, ‘disagree’ (1–2), ‘unsure’ (3–5), and ‘agree’ (6–7). A contingency table analysis with post-hoc tests on the adjusted residuals was used to compare the ratings of each TELT. To account for multiple comparisons the threshold for significance was p < 0.003. All statistical analysis was performed using PASW Statistics 22.0 for Windows (SPSS, Inc., Chicago, IL, USA). The qualitative data was analysed via a thematic analysis, with raw quotes from the survey added to first order themes that sat under the common second order themes of ‘positives’ and ‘improvements’.

Table 1 A representation of the questions included in the online questionnaire

Results

Data describing the use of each TELT is displayed in Table 2. Significant cells in the contingency table were ‘Twitter – did not use’ (p < 0.001), ‘Twitter – agree’ (p < 0.001), ‘ShowMe – agree’ (p = 0.002) and ‘MCQ – agree’ (p < 0.001) (Table 2). These results suggest that Twitter was used significantly less than the other three TELTs with less people agreeing that it helped, whilst ShowMe and MCQ had a greater occurrence of an ‘agree’ rating compared to Twitter and Panopto.

Table 2 Frequency of student responses on whether each TELT was useful to support their learning. Agree = median score 6–7; unsure = median score 3–5; Disagree = median score 1–2

A thematic analysis has been presented in Table 4. One main finding was that a first order theme shared across all TELTs was that of assessment preparation. This theme was identified as a positive for Panopto (‘helped prepare for the exam’), ShowMe (‘can pick out main points for exam revision’) and MCQ (‘helped to know what to expect on the exam’). Specific reasons for these TELTs being perceived as useful for assessment could have been to help check understanding (MCQ; ‘helps you find out what you actually know’), tailor and personalise the delivery (Panopto; ‘can jump to specific points’) and provide information in a different format (ShowMe; ‘easier to understand than the lectures’). Conversely the theme of assessment was identified as an area for improvement with Twitter (‘not clearly linked to the exams’). In fact, the only positive theme for Twitter was convenience (‘easy to access links’).

Discussion

The purpose of this study was to implement different TELTs into the delivery of a first year undergraduate science module, and collect student perceptions of their use and perceived value for learning. It was found that approximately 80% of the students surveyed accessed Panopto, ShowMe and MCQ to support their learning. However less than half of the cohort accessed Twitter, significantly less than the other three methods. Furthermore, only 4% of students agreed that it helped them with their studies, which is again in stark contrast to the other three methods (Table 2). This is despite students identifying it as a convenient way to get information (Table 4). Previous authors have advocated the use of Twitter to support Higher Education students, including Junco et al. (2011) who observed greater levels of engagement in students assigned to a Twitter group as opposed to a control. Gikas and Grant (2013) also reported generally positive student perceptions of using Twitter, including being able to embed learning within their normal lives and the ability to have discussions with classmates. Interestingly interaction was a theme under suggested improvements in the current study, with students commenting that ‘everyone should follow each other’ and that they should ‘share content’. A possible reason for the lack of interaction was a lack of student awareness, with one student commenting an improvement would be to ‘make students more aware’. However, the way in which staff made students aware of Twitter was no different to the other TELTs. Furthermore staff regularly updated the Twitter feed with both course specific information and relevant sources for further learning e.g. recent articles linked to that week’s course content, but no students posted their own tweets. The fact that Twitter was accessed more on mobile devices may suggest that it was used more for keeping up to date rather than active engagement. Of note is that the intervention implemented by Junco and colleagues used Twitter as a sole source of information (e.g. discussion board, announcements page, reading lists etc.), with no competing TELTs being used simultaneously. It may be that the use of Twitter in the current study was influenced by the fact that the traditional VLE was also being used for the roles described by Junco et al. (2011), or that three other TELTs were available at the same time so students directed their attention elsewhere. These factors may have had more of an influence if students were unfamiliar with the workings of Twitter.

A possible reason for the other three TELTs being used by a greater proportion of students is the perceived link to assessment, with the thematic analysis identifying this as a positive theme for ShowMe, MCQ and Panopto, but an area for improvement for Twitter (Table 4). Furthermore, ShowMe and MCQ had a greater proportion of students ‘agree’ compared to Panopto (Table 2), which may again be linked to assessment as these were the two TELTs used predominantly to help prepare for the examination (Table 3). A reason for ShowMe being rated higher than Panopto may be due to the delivery of information. Whilst students liked being able to alter the delivery and pace of the lecture using Panopto, the ShowMe videos offered an alternative delivery that helped to simplify content (Table 4). Moreover, the delivery on ShowMe was suggested to be convenient and time efficient for the student. The use of the ShowMe App has been described in past work as a way for students to create notes and share ideas at primary and secondary level (Castek & Beach, 2013; Maher, 2013). It has also been suggested to be popular for clinical teaching within medical residences, specifically for annotating notes and producing quick diagrams for ‘in the moment’ teaching (Archibald et al., 2014). This latter point is the most related to the use of the App in this study i.e. to simplify information. Research has highlighted the use of animations in other formats as a positive way to support science teaching, with authors reporting enhanced self-efficacy and enjoyment in elementary schools (Barak et al., 2011), and improved retention of information in university students (Lin & Atkinson, 2011). Of note is that the animations used by Barak et al. (2011) and Lin and Atkinson (2011) were specific LOs used in a formal learning environment, whereas the current study is novel in that the animations were produced by the teaching staff and could be accessed outside of the formal learning environment. It is advised that those looking to implement a similar strategy to the current study consider their clarity of speech when producing the videos, as this was commented on by some students (Table 4).

Table 3 A summary of student access to the TELTs

Table 4 Thematic analysis of qualitative responses for each TELT

It is perhaps clear why the MCQ was rated highly by the students, as it mimicked the summative assessment of the module. Some of the comments in the ‘confirming understanding’ theme suggest it may have also been of benefit for formative feedback (‘can track my learning’) (Table 4), however what is unclear is whether this TELT would have been perceived as useful if the mode of summative assessment had been different i.e. an essay or report. Another factor that may explain the positive perception of the MCQs is that this was the only TELT that required the student to take an active part. However no students alluded to this in the questionnaires.

As students appeared to be extrinsically motivated by assessment, TELTs were not used a lot for general interest (Table 3). All of the TELTs had the capacity to be used on mobile devices, however they were used primarily on PCs/laptops (Table 3). This may suggest that the potential for incidental or informal learning was not enhanced by using the TELTs. Each TELT was accessed mostly at home, therefore perhaps promoting extra-curricular study as it was hoped that the ShowMe videos would, but still in a formal and structured fashion. However, these points are speculative as we have no comparison to their learning habits prior to the study. Furthermore ‘general interest’ is quite vague and open to individual interpretation. Perhaps this would have been better termed ‘further study’ or ‘further understanding’.

In summary the current study has identified shared characteristics between the TELTs that students engage with and perceive to be useful. Such characteristics include links to the summative assessment and offering an alternative method of content delivery to the traditional lecture. Previous research has shown that time is a barrier for staff to incorporate TELTs into their teaching (Reed, 2014), so an appropriate practical message may be to start with one type of TELT. Based on this study TELTs similar to the ShowMe App are most recommended as this was one of the two highly rated TELTs (alongside MCQ), but has more scope for crossover with other subjects as the students found it useful for more than just assessment. Moreover the lecturer has full control over the content, so can make each animation specific to the current topic of study. Further work should look at the potential transfer of animations for learning in other subject disciplines, and investigate if there are any alternative TELTs for meeting the same purpose of simplifying information. It should be considered that this paper describes only those students that completed the survey (45% response rate), and the perceptions of the other students enrolled on the module are unknown. Furthermore, we cannot discount that using the TELTs in combination may have influenced the perception of each TELT, and we relied upon student self-reported usage for each TELT. Future work should monitor the actual use of the TELTs using tracking statistics, and look to examine predictive validity of student perceptions of TELTs to gain more insight into what student perception means for learning and academic performance, and what the implications are for teaching staff.