Beyond emergency remote teaching: did the pandemic lead to lasting change in university courses?

Abstract

The COVID-19 pandemic significantly disrupted traditional methods of teaching and learning within higher education. But what remained when the pandemic passed? While the majority of the literature explores the shifts during the pandemic, with much speculation about post-pandemic futures, a clear understanding of lasting implications remains elusive. To illuminate this knowledge gap, our study contrasts pedagogical practices in matched courses from the pre-pandemic year (2019) to the post-pandemic phase (2022/2023). We also investigate the factors influencing these changes and the perceptions of academics on these shifts. Data were gathered from academics in a large comprehensive Australian university of varying disciplines through a mixed-methods approach, collecting 67 survey responses and conducting 21 interviews. Findings indicate a notable increase in online learning activities, authentic and scaffolded assessments, and online unsupervised exams post-pandemic. These changes were primarily driven by university-guided adaptations, time and workload pressures, continued COVID-19 challenges, local leadership, an individual desire to innovate, and concerns about academic integrity. While most changes were seen as favourable by academics, perceptions were less positive concerning online examinations. These findings illuminate the enduring effects of the pandemic on higher education, suggesting longer-term implications than previous studies conducted during the acute phase of the pandemic.

Introduction

In 2020 and 2021, higher education institutions globally had to modify curricula and pedagogy due to the COVID-19 pandemic (UNESCO, n.d.). This rapid shift became commonly known as ‘emergency remote teaching’ (Hodges et al., 2020). Emergency remote teaching (ERT) involves unplanned, quick adaptation, often using existing technology and resources, and with emphasis on preserving instruction rather than enhancing learning quality (Watermeyer et al., 2021). This type of teaching is distinct from online learning, which is a thought-out approach designed for online delivery and is considerate of learners’ needs and preferences (Hodges et al., 2020). During this emergency phase, face-to-face classes were stopped or transferred online to lessen COVID-19 risks (Crawford et al., 2020; Johnson et al., 2020). Class assessments moved online, activities requiring specific locations or equipment were disrupted, and students had to work more independently, regardless of their self-regulation skills (Bartolic et al., 2022a; Slade et al., 2022). Many academics felt ill-prepared for the changes that transpired (Sum & Oancea, 2022) and reported concerns that teaching quality suffered during this time (Weidlich & Kalz, 2021).

COVID changed teaching and learning practices in a profound manner. For example, a consortium comprising nine institutions from around the world investigated changes to teaching and learning during the early stages of the pandemic, collecting data from 4243 students, 281 instructors, 15 senior administrators, and 43 instructional designers (see Bartolic et al., 2022a, 2022b; Guppy et al., 2022a, 2022b). This body of work showed challenges faced in ERT (Guppy et al., 2022a), including the modifications in assessment approaches corresponding to the digital shift (Bartolic et al., 2022a) and student vulnerabilities and confidence in an online learning environment (Bartolic et al., 2022b). However, are these findings a question of a momentary disruption and a return to the previous status quo? Or does the pandemic represent the kind of external shock that fundamentally changes the landscape? Funding bodies report substantial challenges for teaching and learning innovations to have long lasting impacts (Kottmann et al., 2020). What is interesting about the pandemic is that it forced change across all levels of the university all at once, and this may prove to be a useful lesson for understanding how educational change itself can unfold in different circumstances. Therefore, it is important to ask what, if anything, has been retained and why.

In a systematic review from 2023, Imran et al. analysed 68 studies on blended and online teaching modes, aiming to identify emerging themes in learning modes from the post-pandemic era. Notably, of the studies they examined, only a handful were conducted in the recent post-pandemic years of 2022 and 2023. Among the few that were, none compared pre-pandemic conditions to the post-pandemic environment nor centred their analysis on data highlighting shifts in teaching practices from the viewpoint of educators. Instead, a significant portion of these studies offered mere speculations about the future in the aftermath of the pandemic. This led the authors to emphasise a noteworthy gap in the literature, concluding that “future research should focus on the long-term effects of COVID-19 on teaching modes and the resulting changes in curriculum development” (p. 8). Echoing this sentiment, Kerres and Buchner (2022) noted that much of the current research predominantly centres around the pre- and mid-pandemic phases, with scant attention paid to post-pandemic impacts. They argue that despite the plethora of available research, “it is still difficult to grasp a clear picture of the effects of the pandemic on education in the various sectors of education worldwide” (p. 6). This ambiguity primarily stems from the dearth of data concerning educational transformations in the post-pandemic world.

The current research intends to delve deeper into the post-pandemic aftermath than previous studies. We explore the elements from the pre-pandemic era that were retained and what adjustments made during the pandemic persisted, if any. Further, using ecological systems theory as a framework, we explore which factors at the individual, faculty/discipline, university and outside the university contributed to retain changes. This will enhance understanding into how educational change occurs and may allow universities, faculties, and academics to tackle the challenging problem of sustaining change to teaching and learning practices.

Emergency remote teaching

While there is little literature on the post-pandemic state of curriculum changes compared to the pre-pandemic state, much research documents the changes made during the pandemic. The COVID-19 pandemic necessitated the adoption of various digital tools and technologies in higher education, such as digital classrooms, learning management systems, and online collaboration platforms (Bartolic et al., 2022b; Guppy et al., 2022a, 2022b; Koh & Daniel, 2022). Academics deployed a range of online strategies, such as synchronous and asynchronous activities, online polls, uploading teaching materials online, online simulations and virtual office hours (Bartolic et al., 2022b; Slade et al., 2022; Sum & Oancea, 2022). Classrooms were transformed with varying success by attempting to replicate face-to-face lectures in virtual spaces, pre-recording content to be watched by students at their own pace, and through the provision of live skills demonstrations and online versions of the flipped classroom (Hew et al., 2020; Koh & Daniel, 2022; Slade et al., 2022).

In tandem, institutions were required to rethink assessment strategies that required campus attendance. Consequently, alternative assessment methods were developed, including online exams and virtual presentations (Bartolic et al., 2022b; Selwyn et al., 2021) and types of authentic assessments such as fieldwork, group work and portfolios were reduced (Bartolic et al., 2022b; Slade et al., 2022). While Mottiar et al. (2022) found that the pandemic-induced disruption provided a unique opportunity for academics to experiment with new assessment techniques, a study by Slade et al. (2022) found that most academics just translated their original assessment to an online format. This finding is supported by Mottiar et al. (2022), who revealed that 95% of the 192 surveyed academics made modifications to their assessments, primarily introducing online quizzes (53%) and open-book exams (49%). However, when questioned if these changes would continue once the pandemic was over, many academics said they preferred to return to paper-based assessments, with only 10% reluctant to return to paper-based methods.

Despite the lack of pre- and post-pandemic data, there has been widespread speculation about possible educational practices to be retained from the pandemic. Guppy et al. (2022b) found that a large majority of 281 academics (86.1%) expected a rise in online, blended, and hybrid teaching, as well as an increase in fully online courses (52%) in the next 2–3 years. Echoing these findings, Watermeyer et al. (2021) suggested an “unstoppable shift” (p.625) towards online learning that was “quickened” by the pandemic (p.638)—a change so rapid that Anderson (2020) compared it to a decade’s worth of digital growth happening in a few months. Concerns about the potential downsides of this swift digital shift, such as potential impacts on student experience, recruitment efforts, and academic workload, have also been raised by academics (Watermeyer et al., 2021). Now, two years later, we can consider these speculations.

Factors likely to impact change

While there has been much discussion of the influences on pedagogical change during the pandemic, there is a dearth of information on whether these hold in a post-pandemic environment. To frame our study, we adopted Bronfenbrenner’s (1977, 1979) ecological systems theory. This theory elaborates on how the inherent characteristics of individuals and the interplay between their immediate and larger environments contribute to their development and transformation. The first four levels seem most relevant to this context: the individual level (concerning the personal characteristics of academics), the microsystem level (associated with the specific discipline and faculty), the mesosystem level (pertaining to the broader institutional context), and the exosystem (related to external factors).

The individual level explores the role of personal experiences and reflections instigated by pandemic-driven changes. There has been much discussion in the pandemic literature about experimentation with pedagogy, concerns for academic integrity, and the desire for innovation, among other things. Academics forced to experiment with novel teaching and evaluation methods, as suggested by Mottiar et al. (2022), have potentially gained insights into improved practice, sparking motivation for future innovative teaching. Conversely, many academics perceived a dip in their teaching quality (Weidlich & Kalz, 2021), a sentiment that could dampen enthusiasm to sustain and further develop these practices beyond the pandemic. There have been mixed reports about concerns for academic integrity. While the pandemic prompted changes in assessment practices due to academic integrity concerns (Koh & Daniel, 2022), Slade et al. (2022) suggest that this was not a significant change driver. Yet, the flourishing of the remote proctoring industry during this period (Selwyn et al., 2021), where proctored assessments are primarily used to uphold academic integrity (Dawson, 2023), provides a counter view.

The microsystem level emphasises the role played by discipline, faculty, local leadership, student factors and peer support in executing pedagogical transformation. Systematic reviews conducted by Liu et al. (2020) and Sum and Oancea (2022) of 131 and 32 studies, respectively, highlighted the role of supportive local leadership and collegial advocacy as key accelerants for embracing technological shifts. Possibly, in the throes of change, effective leadership becomes a beacon of guidance, while affirmation from colleagues can foster a sense of shared responsibility. Weidlich and Kalz’s (2021) research amplifies this point, finding that an environment of support—both technical and cultural—played a pivotal role in the process of adaptation. Although, it should be noted that the authors found that technical support did not mitigate all the individual challenges experienced during the pandemic. These insights underscore the idea that change is more likely to gain momentum and be sustained when facilitated within disciplines themselves rather than being dictated by centralised mechanisms.

The mesosystem level examines institutional influences on transformation processes, such as university directives and workload. Several reports underscore a heightened workload during the pandemic (Guangul et al., 2020; Watermeyer et al., 2021), a factor that may curb an academic’s capacity for change, with predictions of lasting post-pandemic workload burdens (Watermeyer et al., 2021). Lee and Jung (2021) speculate that institutional support and directives can shape the direction of change, regardless of whether the change is mandated. Contrarily, observations by Slade et al. (2022) provide a cautionary note. They highlight the potential confusion that can arise from sudden or unanticipated alterations to institutional policies and procedures. Such changes threaten to slow the transformation process. Moreover, the review of 32 studies by Sum and Oancea (2022) reveals another layer of complexity—the potential for institutional policies to curb decision-making autonomy. Their findings suggest a potential tension between the need for coherent, coordinated institutional response and the creative autonomy required at the individual and microsystem levels. Thus, while institutional resources and support are undeniably crucial in facilitating change, the way they are employed can significantly impact the trajectory and effectiveness of transformation.

Lastly, the mesosystem includes influences external to the university. For this study, we acknowledge the previous and continued impact of COVID-19 on higher education, and thus include it as a factor. Accrediting bodies do wield influence over course design and delivery, and so have also been included.

The current study

The pandemic forced a rapid and unplanned shift to ERT worldwide, drastically altering the traditional model of higher education. This unprecedented change presents a unique opportunity to study the long-term effects and the potential for enduring transformations in pedagogical practices. While numerous studies have examined the immediate impact of the pandemic on higher education, less attention has been paid to its longer-term effects post-pandemic (Kerres & Buchner, 2022) and to the authors’ knowledge, no study to date has compared the pre- and post-pandemic teaching practices while examining the influences of those changes in a university setting.

The current study seeks to address this gap by investigating what elements of these emergency adaptations have endured beyond the immediate crisis within the context of a single institution. We compare pedagogical practices from the pre-pandemic era (2019) and identify practices that have emerged (and been retained) in the post-pandemic landscape (2022/2023). We employed a mixed-methods approach to achieve these aims, focusing on three research questions:

• 1. To what extent did academics change their pedagogical practices from pre- to post-pandemic in terms of:

  • a. learning activities

  • b. assessment tasks and

  • c. examinations.

• 2. Using ecological systems theory as a framework, how did factors at the individual, faculty/discipline, university and outside the university contribute to the changes?

• 3. Lastly, what were academics’ perceptions of these changes?

Methods

Participants

Participants came from a large comprehensive Australian University and were required to have taught the same undergraduate unit (subject of study, sometimes called courses) before (2019) and after the pandemic (2022/2023). Due to local restrictions from the pandemic in 2020 and 2021, academic teaching staff had two years of teaching mostly online only (including online only examinations). In 2022 and 2023, as the pandemic situation lessened, academics and students were able to return to campus. Academics could maintain their changes or return to all or some of their pre-COVID teaching practices. Recruitment occurred via email and word-of-mouth. As the study was advertised broadly across the university, we were not able to collect or access data on how many academics saw the study invitation. Thus, the participant response rate for this study could not be determined. The Deakin University ethics board approved the study.

Sixty-seven academics completed the survey, and a subset of 21 academics (31%) participated in a post-survey interview. Participants were most likely to be 40 years or older (n = 48; 72%) and female (n = 32; 48%; male n = 32; 45%). The majority (n = 53; 80%) were Level B (Lecturer) or Level C (Senior Lecturer) academics, and most respondents had more than ten years of tertiary teaching experience (n = 48; 72%). The average teaching allocation in their workload was 52% (range = 15–100%; SD = 19%). Participants came from across the University (Faculty of Education and Arts = n = 13; 19%; Faculty of Health = n = 23; 34%; Faculty of Science, Engineering, and Built Environment = n = 20; 30%; Faculty of Business and Law = n = 11; 16%).

Survey

The survey took approximately 30 min, and participants could win one of two $250 gift certificates. After providing demographics (e.g., age, gender identity, academic level, teaching workload allocation, and the faculty in which they were employed), participants completed the following survey sections.

Changes from pre-pandemic to post-pandemic

Participants were asked to delineate the specific changes in their units (subject of study, sometimes called courses) between 2019 (pre-pandemic) and 2022–2023 (post-pandemic). A series of questions prompted them to compare the types of learning activities (e.g., seminars, lectures, etc.,) focus of assessment excluding examinations (e.g., authentic tasks, scaffolded, inclusive, etc.) and modes of examinations (e.g., face-to-face, online, proctored, unsupervised, etc.,) implemented in their units during the aforementioned periods.

Factors that influenced change

Participants were also questioned about the factors that motivated changes in their learning activities, assessments (excluding examinations), and examinations. The following factors were rated on a 5-point scale ranging from ‘not at all influential’ to ‘extremely influential'. For analyses, percentages represent a combination of very influential and extremely influential responses. Bronfenbrenner’s (1977, 1979) ecological systems theory was used to categorise factors. Representing the individual level, academics were asked to rate factors such as their reflective practice, a desire to innovate, scholarly activities (e.g., attended workshops, conferences, professional development, sharing practice, engagement with the literature), and concerns about academic integrity and cheating. Microsystem-level factors included student-related considerations such as enrolment preferences (e.g., online vs. on-campus), student satisfaction (e.g., the University student rating system), and grade distributions from the prior trimester. It also included local cultural factors like local leadership (e.g., Course Directors, Associated Head / Director of School (T&L) and Head of School, support teams (Faculty level teaching and learning support teams), and colleagues. Mesosystem-level factors encapsulated organisational instructions, resources, and workload. Lastly, accreditation requirements and lingering issues related to the COVID-19 pandemic make up the exosystem level factors. For the full list of factors, see Additional file 1.

Perception of changes in 2022–2023

On a scale of 0–10 (with 10 indicating higher agreement), participants were asked to rate their satisfaction with change, the sustainability of the change, how positivity of their change experience was, how valuable the change was and the magnitude of change.

Workload

To gauge perceived workload, participants were asked to compare their current workload in 2022–2023 with that of 2019 using a 5-point Likert scale ranging from “much less” (1) to “much more” (5).

Interview

Participants who consented to an interview met with one of the authors for a 30–60-min interview. The semi-structured interview comprised open-ended questions that were used as prompts during the interviews. Not all participants received all prompts; responses from the interview were dependant on what they discussed. These questions related to the changes they had implemented, their decision-making processes concerning the maintenance of pandemic-related changes or a return to pre-COVID practices, their evaluations of what went well or poorly, and the lessons they learned from this experience (see Additional file 1 for full list of interview prompts). Participants who were interviewed were given a $30 gift certificate for their time.

Analysis was informed by thematic framework analysis (Ritchie & Spencer, 1994) a systematic approach to coding. After familiarisation with the data, two researchers read a sample of the interview transcripts to develop a coding framework that included deductive and inductive codes focusing on influences on staff decisions. The research assistant (KD—see acknowledgement) then coded all the transcripts using this framework. Data were then discussed among the team against the codes and ordered based on the categories of learning activity, assessment and examination. These codes and categories were then interpreted in relation to survey data and to generate themes. Illustrative quotes have been used for this article to highlight key themes.

Results

Unit information

From the survey data, academics were responsible for mostly first-year units (n = 30 units; 45%) followed by 2nd year (n = 16 units; 24%) and 3rd year (n = 16 units; 24%). The majority (n = 46 units; 69%) were core units in their respective courses. The average enrolment of the units was medium to large, with slightly fewer students in 2022/2023 (M = 487 students) compared to 2019 (M = 498 students). Ten of the units (15%) were Work Integrated Learning (WIL) units.

Compared to 2019, 95% of academics had changed their units. The majority of academics changed learning activities and assessments and examinations in their unit (n = 25; 37%), followed by learning activities only (n = 12; 18%), learning activities and assessment (n = 10; 15%), learning activities and examination (n = 8; 12%), assessments and examinations (n = 4; 6%), assessments only (n = 3; 4%) and examinations only (n = 1; 1%). Four academics (6%) reverted to 2019 practices after the pandemic.

Learning activity changes

Fifty-five academics (82%) indicated that, in 2022/2023, they had retained at least one change in learning activities that differed from their 2019 teaching practices (see Fig. 1). The following section discusses what increased, decreased and what remained the same.

Fig. 1

Change in learning activities from 2019 to 2022/2023

Academics noted significant increases in online learning activities post-pandemic, including the utilisation of pre-recorded chunked videos (n = 33), online interaction (n = 32), online self-paced content (n = 30), online Q&A / discussion sessions (n = 29), online self-paced activities (n = 28) and online recording of on-campus lecture /seminars (n = 23). This shift signifies academics’ embrace of more flexible approaches post-pandemic. They noted that continuing these learning activities offered during the pandemic promoted convenience by giving students various choices to engage with learning content that aligns with their needs and circumstances. One strong theme was that these teaching approaches were retained for student wellbeing and equity, particularly a desire to “reduce the gap between advantages and disadvantages between the two modes [on-campus and online]” (Interview 14) as well as personalise the learning experience so that “synchronous and the asynchronous students are acknowledged and have activities explicitly designed for them” (Interview 9). One academic raised the concern that the “level of inclusivity was a level of access that might have been actually restrictive pre-COVID. It might have been to the point that we weren’t letting people into education based on their ability to get into a room” (Interview 10).

Several interviewees realised that they did not understand online students well before the pandemic, with one stating that they had “this assumption that [online students] students, like a cloud, just wanted to float in and out. They didn’t want to be anchored to anything, and they were quite happy to do things on their own terms, and there’s certainly a cohort that loves that. But I think the pandemic forced us to say, ‘Okay, let’s structure their time more specifically’, and it’s been a huge success” (Interview 13). While the pandemic forced the creation of many of these resources, many saw this as positive. For some, “the pandemic was a shakeup. It gave me fresh eyes that we were no longer doing things in the old way anymore” (Interview 13). In contrast, others had ideas before the pandemic that they never had time to implement, “so it just [went] on the back burner, we’ll get to it one day, and that one day came in 2020” (Interview 17). Others found the pandemic created opportunities that they previously thought were impossible, such as a virtual internship, and continued the practice post-pandemic “I was sceptical initially that they wouldn’t be of the same quality, but they turned out to be actually really good” (Interview 22).

On-campus only lectures decreased in most units (n = 22). This reflects a move away from running the same lecture multiple times on different campuses towards other forms of delivering content, with one academic remarking, “Actually, now, we just do the online [lectures], … the students seem to feel much more comfortable about asking questions and talking” (Interview 21). The remaining 15 learning activities (shown in Fig. 1) had no change in most units. No change may indicate that the focus had not changed for these learning activities (e.g., the activity was/wasn’t a focus in 2019 and remained a similar focus (or not) post-pandemic).

The overall finding of learning activities reflects an enhanced level of flexibility provided by academics, which has not removed more traditional options of learning but, instead, added more options (with on-campus lectures as the exception).

Assessment changes

Forty-two academics (63%) reported retaining changes in assessment post-pandemic that differed from 2019 (see Fig. 2).

Fig. 2

Change in assessment from 2019 to 2022/2023

When comparing what they were doing in 2019 to what they are now doing post-pandemic, there were three areas that increased the most: authentic assessment, scaffolded assessment, and online delivery of assessment. There was a notable trend towards utilising authentic assessment tasks (n = 23), aiming to enhance the applicability of assessments in real-world contexts and to make them more meaningful, relevant, and valuable for students with the pandemic presenting an opportunity for change “I still need a way of assessing the students. Now, it’s an opportunity to start making it more authentic” (Interview 4). Additionally, there has been a shift towards employing scaffolded and sequential assessment tasks to alleviate the stress associated with high-stakes assessments for students (n = 18). This approach assists students by tailoring activities to develop their knowledge progressively and offering them opportunities to gauge their understanding throughout the trimester. One survey respondent noted, “Student motivation was very mixed … scaffolded [tasks] ensure those with high motivation can connect at a more intense level while those with low motivation are still able to learn and achieve solid results”.

The COVID-19 pandemic prompted a transition from traditional face-to-face assessments, such as presentations and in-class tests, to online formats (n = 18). Academics have reported that this change yielded several benefits, leading them to maintain online assessments even after the pandemic. These advantages included a reduction in absenteeism and equal treatment for online and on-campus students in terms of the assessment process and submission procedures, with one academic commenting, “A lot of our decisions came down to: is this fair for everyone? Are we actually disadvantaging people by having this assessment? Our answer to the [assessment] was we absolutely were, and we had qualitative and quantitative feedback to support that” (Interview 14).

As shown in Fig. 1, there were no areas that most units focused less on. There were eight areas where assessments stayed about the same in most units between 2019 and 2022/2023.

Examinations

Thirty-six academics (54%) said they had an examination in their unit. In 2019, most examinations reported were in the form of invigilated on-campus exams, accounting for 97% of units (n = 35). However, post-pandemic, only one clinical assessment remained on-campus and invigilated (3%). Seventeen per cent were still invigilated but were moved online. The remaining 80% of exams were either unsupervised online (44%) or removed entirely from the unit (33%). This data mirrors institutional data and public-facing data (Johnson et al., 2023). In 2019, 686 units had an invigilated on-campus exam. In 2022, ~ 45% were unsupervised online (n = 489), ~ 10% were invigilated online (n = 112), and there were a negligible number of on-campus examinations (such as OSCEs not timetabled through the normal exam system). The remaining units did not have an online exam. Overall, there was a near complete removal of on-campus face-to-face examinations and an 84% decrease in the number of invigilated exams.

The shift towards online exams encouraged academics to reflect on the nature of the knowledge they were assessing towards questions that prompted “higher order thinking”: “Most of them [questions] are now real-world rather than, “What’s the most efficacious treatment for substance use disorder?” They are, “Here’s a person, here are all their symptoms, and tell us what order of treatments you would consider” (Interview 14).

While 97% of exams in 2019 were at a set time (e.g., 12–2 pm), this was reduced to 39% in 2022/2023, with most units (62%) allowing students to complete the exam anytime within a specific period (e.g., 24 h). Again, equity was seen as the main positive of moving to a more open format “students with limited time to go to do the exam or those who had difficulty finding the time to do the exam could actually pick a time that’s suitable for them during that 24-h period” (Interview 6) and that unsupervised online exams were “by design, more accessible” (Interview 9).

There were notable changes in exam weightings (percentage of the unit grade attributed to the examination component), from an average of 42% to an average of 29% in 2022/2023, an average decrease of 14% compared to 2019. The decline in exam weightings suggests a shift in the assessment structure of units. This change likely reflects the adoption of alternative assessment methods and a re-evaluation of the relative importance of exams in the overall grading scheme. Some academics were concerned with academic integrity which may explain the drop in weighting and reduced number of exams. However, not all were concerned with academic integrity, putting the onus back on the student “… If you cheat, then ultimately, it depends on how much [you will] learn” (Interview 1).

The university’s change in policy to move away from on-campus supervised exams had a big impact on decision making “I feel the pressure [of the university] on not wanting to return to exams, so I definitely have that thought that the policy is to withdraw from on-campus exams” (Interview 10). While some academics were happy with this move and saw it as an opportunity, others wanted the assessment policy to return to what it was in 2019. These findings highlight the substantial shift in examination methods prompted by the pandemic and encouraged and supported by the university. There was a clear move away from traditional, on-campus, supervised exams as directed by the university.

Factors that influenced change

Figure 3 shows the percentage of influence for each factor. Each factor was ranked from most influential (rank 1) to least influential (rank 20) across learning activities, assessments, and examinations. Additionally, an overall ranking was derived by aggregating the data from these categories. This dual approach allowed us to obtain both a holistic understanding of the factors' influence, irrespective of the specific teaching practice, and a detailed perspective on their impact on learning activities, assessments, and examinations. Using the consolidated ranking, we applied General Linear Modelling to discern whether the global rank order of each factor exhibited significant variations across learning activities, assessments, and examinations. Table 1 presents the rank order of factors for learning activities, assessments, examinations based on the percentage of combined very influential and extremely influential responses. These findings suggest that the ongoing changes in university teaching, assessment methods and examinations post-pandemic are mostly influenced by internal systemic pressures (such as time/workload and organisational instructions), external factors (COVID-19 issues), and individual factors (desire to innovate) and discipline factors (local leadership). General Linear Modelling revealed that the combination of these factors differs depending on learning activities, assessments, or examinations. For example, while organisational instructions were important for all three areas, they were significantly more influential for examinations than learning activities and assessments. Local leadership, on the other hand, while not the top-ranked factor, was equally important across all changes in teaching practices.

Fig. 3

The percentage of influence for learning activities, assessments, and examinations. Percentages represent a combination of very influential and extremely influential responses

Table 1 Rank order of factors based on percentage of combined very influential and extremely influential responses

At the individual level, a desire to innovate, integrity concerns and reflective practice all appeared to have had a notable influence. Innovation drove learning activities and assessment; integrity was the second biggest influence on exams after organisational instructions and reflective practice was important when designing learning activities. This suggests that the drive to improve teaching methods, safeguard against academic integrity and engaging in reflective practice have led to changes that individuals want to continue even when conditions returned to ‘normal’. Many interviewees commented on seeking out other innovative practices “My mindset’s always been trying to innovate and improve on my own practices….. Seeing what my colleagues do at other universities and learning what to do and what not to do, I suppose” (Interview 16). And the opportunity to innovate brought about by the pandemic “… It actually gave us that freedom of experimenting with things that we might not have gotten away with if it were not for COVID” (Interview 8).

At the microsystem level, local leadership played a critical role indicating that discipline- and Faculty-level decisions are important in facilitating staff change. Academics were grateful for local leadership “I think if it wasn’t for [local leadership] we would’ve drowned, to be honest” (Interview 3), and course directors “[The] course director has always been fantastic at saying, “Yes, that’s a good idea. Give it a go. If it doesn’t work, it doesn’t work, but let’s give it a go and try it” (Interview 4).

The mesosystem level factors—time, workload pressures, and organisational instructions—ranked highest suggesting that the most influential factors maintaining and driving change are those from the University level. It shows that the university’s strategies and workload allocation significantly impacted academic work. Aptly put by one academic, “Well, the biggest change that happened as a consequence of the pandemic was the agility of the organisation. I think finally, they discovered that you didn’t need a year and a half of approvals…” (Interview 15). This was a common sentiment, that the disruption to the organisation had been useful. Although, it should be noted that not all academics saw University interference as positive “you tell me, do I have liberty for these things? Can I choose? No, I can’t choose. I’m a hostage of the system” (Interview 11).

The comments provided by the academics about workload indicate two contrasting situations: being constrained due to lack of time and being able to innovate when given extra workload hours for development. Workload pressure was a high influence on both learning activities and assessment. One academic wanted more regular workload allocation for improvement. “Things like not having [workload] allocations that support continual improvement. You get some hours once every five years to improve a unit …It can be a significant amount of investment that you put into trying new practices” (Interview 19). Other staff were not concerned about trying to stick to the allocated workload for the task reflecting the hard choice some academics have to make to invest in their teaching; otherwise, they have to compromise on interactive pedagogies “Look, if you’re smart, you would design your teaching to fit into the [workload allocation] just so that you get maximum recognition for activity. Unfortunately, for me, it produces… transmission teaching [which] is not very interactive….. I find that not very engaging and not fun to do. Students don’t like it, but it fits beautifully into the [workload allocation]” (Interview 22).

At the exosystem level, COVID-related issues, such as staff shortages and illnesses that impact staff and student’s ability to get on campus, were significant and paramount in maintaining changes and to re-focus on learning and teaching design. The lower ranking of professional accrediting bodies likely reflects that only specific, accredited courses had to make adjustments that needed to be maintained post-pandemic. As stated by one academic, “We are accredited, our course, and so there are also limitations in terms of what our accrediting body will accept” (Interview 2).

Academics’ perceptions of these alterations

Table 2 shows staff perceptions of their changes to learning activities, examinations, and assessments. In the aftermath of the pandemic, academics were fairly satisfied with the changes they had made to learning activities, examinations, and assessments post-pandemic, generally finding that the changes were sustainable and of value to students. A repeated measures ANOVA showed that these were generally similar across teaching practices with two significant differences. The magnitude of changes across the board was relatively large but significantly larger for exams. And although not unsatisfied with the exam changes, academics were significantly less satisfied with this than with changes to other assessments and learning activities. In relation to data related to ‘value’, while the changes made to exams appear to be rated less valuable than the assessments and learning activities, there was found to be no significant difference from either one. Few academics saw a reduced workload due to changes, particularly with exams, in which one person reported less work. Academics reported that workload was “somewhat” or “much more” in 2022 for changes to learning activities (60%), exams (64.7%) and assessments (48.6%) compared to 2019.

Table 2 Staff perceptions of the changes they made to learning activities, examinations, and assessments

Discussion

We answered calls from Imran et al. (2023) and Kerres and Buchner (2022), among others, for future research to explore the impact of COVID-19 on long-term higher education teaching practices. Our study provides insights into the pre/post-pandemic changes at a single institution from the perspectives of its academics. We found that, post-pandemic, there was an increase from pre-pandemic teaching (2019) in online learning activities, authentic and scaffolded assessments, and online unsupervised exams. This was primarily driven (in order of most influence) by mesosystem changes driven by university directives, time and workload pressures, ongoing exosystem COVID-19 challenges, microsystem local leadership, and an individual desire to innovate and concerns about academic integrity. There was a notable synchronicity across individual, microsystem mesosystem and exosystem: all levels came into play to support these on-going changes. Overall, despite increased workload, most changes were seen as favourable by academics, albeit to a lesser extent for examinations.

Learning activities

Our data highlight the increase in online modes of learning activities, assessments, and examinations since 2019. This corroborates the broader transition towards digital education (Liu et al., 2020), predictions of a greater online emphasis post-pandemic (Guppy et al., 2022b), and reflects Fuller’s et al. (2020) speculation of prudent future-proofing. The interviews revealed that differences from pre-pandemic times reflected an intensified commitment to inclusivity and student-centred learning. The increased use of online learning activities, online assessments, and online, unsupervised, flexible exams aligns with calls for accessible, equitable online learning environments (Butler-Henderson & Crawford, 2020) and accommodation of diverse student needs and life circumstances (Tai et al., 2023b). This was most apparent in learning activities, where, except for on-campus lectures, which decreased, online forms of learning supplemented rather than replaced traditional modes, thus catering to diverse student needs. Flexibility is highly valued in an assessment regime that promotes inclusion (Tai et al., 2023a). Importantly, we believe this signals the start of a change, as discussed by Peter Goodyear (Rapanta et al., 2021), where flexibility is key, and learning is designed to fit in with students’ lives rather than students fitting in with university schedules.

Examinations

There has been a long-standing tradition of on-campus invigilated exams, both internationally (Dawson et al., 2023) and at our institution (Johnson et al., 2023). Resisting the temptation to revert to pre-established practices, our institution persisted with most unsupervised online exams after the pandemic. Our data indicate this university directive was instrumental in promoting change, as academics reassess the purpose of their examinations. Academic integrity was the main factor influencing academics’ decisions around exams, and this may be influenced by both internal institutional factors as well as shifts in the academic integrity landscape. During the pandemic, evidence accumulated that remote proctoring’s ability to detect cheating may not be as great as claimed by its vendors (Dawson, 2023), that face-to-face exams are more prone to cheating than previously thought (Harper, et al., 2021), and that unsupervised online exams are challenging from an integrity perspective (Dawson et al., 2023). A decrease in the weighting of examination grades, with a redistribution to other assessments, suggests a reassessment of exams’ role in evaluating student learning, reflecting a growing recognition of the need to address learning outcomes not well judged by exams and the importance of holistic assessment methods such as authentic and scaffolded assessment (Boud, 2000), concerns about online exams’ academic integrity (Butler-Henderson & Crawford, 2020), or a combination of both, as inferred from the data.

Assessments

As speculated by Mottiar et al. (2022), COVID-19 likely provided an opportunity to experiment with different assessment methods, a view supported by our data, which shows a desire to innovate and to use the pandemic as an opportunity to do things differently. As academics moved away from examinations, the rise in reported authentic and scaffolded assessments underscores a holistic, supportive evaluation approach by the academics involved in our study. It highlights a concern for assessment that mirrors real word application and potential for enhanced employability skills (Villarroel et al., 2018). This was coupled with an increased awareness of the stress associated with high-stakes assessments and the necessity to provide sequential tasks with actionable feedback (Ajjawi et al., 2022; Boud & Molloy, 2013). Mesosystem factors such as workload and resourcing had the most considerable influence on assessment decisions. Interviewees raised concerns regarding the potential trade-off between workload pressures and the quality of teaching. This is consistent with existing literature highlighting increased workloads for faculty transitioning to ERT (Guangul et al., 2020; Watermeyer et al., 2021).

Academic perceptions

Interestingly, the academics in our study generally reported satisfaction with the changes in teaching approach, particularly those related to learning activities and assessments. However, their reception of exam changes was less optimistic, potentially due to challenges associated with online, unsupervised exams or doubts about their effectiveness. These findings echo those of Watermeyer et al. (2021), who reported academics’ misgivings about the integrity of online assessment methods. Notably, despite general satisfaction with their accomplishments, an overall increase in workload that continues post-pandemic signals that this transition may have exerted additional strain on academics. Our findings also denote a substantial magnitude of changes across all domains, with exams experiencing the most significant shift, underscoring the pandemic’s disruptive role in university teaching.

Limitations

These findings should be considered in the context of several limitations. First, the sample was drawn from a single institution. While this allowed us to conduct an in-depth investigation into the changes occurring within one context, it limits the generalisability of our findings to other institutions. Notably, the institution under consideration has a significant history of online and distance education. This potentially rendered it better equipped to transition to online learning during the pandemic and post-pandemic period, a situation that might not mirror other universities without such a foundation. Second, the academics who chose to participate in our study might inherently be more open to changes and have a more favourable view of the shift to online learning, assessments, and examinations. While some interviewed were resistant to change, particularly for exams, many participants were teaching and learning champions. Thus, those who participated may not represent the broader academic community, including those who might be more resistant to change or face more challenges. Third, due to the small sample size, our study could not provide a more nuanced understanding of post-pandemic changes on different faculties or departments, ages, teaching experiences etc., within the university. For instance, some disciplines might be able to adapt more easily to online teaching and assessments than others, experiencing different levels and changes. Lastly, our study focused on elements of academic practices that had increased or changed substantially during and post-pandemic. We did not delve into the most prevalent practices or those that might have remained stable during this period. Therefore, our study offers a partial view, emphasising the changes without comparing those dominant or unchanged practices.

Conclusion and implications

To the authors’ knowledge, this is the first study to provide insights into the pre/post-pandemic changes and the factors that influenced change from the perspectives of academics. Our study counters pandemic-driven teaching as purely emergency remote teaching (Hodges, 2020) or a desperate alternative (Fuller, 2020). At least within the confines of one institution, emergency remote teaching served as a catalyst, prompting renewed considerations of learning activities, assessments, and examinations, which have since been sustained in the post-pandemic environment. Key influencers in these changes were:

• University leadership directives.

• Workload considerations.

• Guidance from local teaching and learning leaders

• Academic reflection, attention to integrity, and a drive towards innovation

A marked drive towards enhanced inclusivity and student-centric pedagogy emerged as a pivotal theme.

While these changes have been sustained in the interim, the longevity of their impact remains an open question. Notably, the absence of a single dominant influencing factor underscores the intricate web of synergies and interactions requisite for genuine educational transformation. This suggests that when universities confront significant shifts—whether from pandemics or impending innovations like the introduction of GenAI—a multifaceted approach is essential. Universities must craft a clear strategic vision complemented by resource allocation and pragmatic implementation strategies. Concurrently, local leadership require the latitude to customise changes befitting their disciplines, and front-line educators should be trusted with the autonomy to reflect, innovate, and actualise these shifts. By understanding and leveraging these multifaceted influences, universities can navigate not only pandemic-induced shifts but also other monumental changes on the horizon.