New advances in artificial intelligence applications in higher education?

Introduction

There has been growing interest in the educational potential of Artificial Intelligence (AI) applications in higher education (HE) for the past decade. Despite the recent peak of excitement towards advanced features and techniques of AI-driven large language models (LLMs) such as OpenAI’s ChatGPT, their actual impact on HE institutions and participants has been largely unknown. Thus, many discussions in the field have involved overstated hype and untested hypotheses, whether optimistic or pessimistic, about the impact of AI in education (AIEd).

In the not-so-distant past, the editors of the ETHE article collection “Can artificial intelligence transform higher education?” concluded that the reality is disappointing and little has been achieved (Bates et al., 2020). In that special issue, Zawacki-Richter et al. (2019) published a systematic literature review of primary research articles on AIEd that identified four key areas of AI applications for teaching and learning (profiling and prediction, assessment and evaluation, intelligent tutoring systems, and adaptive systems and personalization). That review found low participation of educators and educational scientists in the included studies. Furthermore, the articles pointing to newly developed applications often focused on technological and methodological issues while neglecting pedagogical and ethical aspects of application in educational contexts.

Much has transpired in the development of artificial intelligence since then. Following the 2019 systematic review, the team of authors (with the additions of John Y. H. Bai, Berrin Cefa, and Frank Loglo) was commissioned by the Office of Technology Assessment at the German Parliament (Bundestag) to carry out an updated review. The report has not yet been published, but it can already be revealed that in the five years following the review in 2019 (with 146 included studies), eight times as many studies (N = 1,167) on AIEd were published as in the decade before. The field is growing rapidly and is much broader than the current discussion about LLMs would suggest.

In view of the rapid development of AIEd, the aim of this special issue was to revisit the realities of AIEd in the realm of higher education in the post-Covid-19 era. The call for papers was intended to invite contributions that address the recent development of AI in higher education and to provide more comprehensive and collective answers to the following questions:

• What is the actual impact of AI on different aspects of HE institutions (e.g., student support systems, administration, professional development, and infrastructure)?

• What is the actual impact of AI on different aspects of learning and teaching in HE (e.g., assessment, data literacy, design of learning activities)?

• What is the actual impact of AI on different stakeholders in HE (e.g., students, teachers, administrators, causal workers)?

We were particularly interested in making sense of the impact of AI on educational accessibility and (in-)equity regarding the cost, quality, and access in HE.

However, the responses to our call for papers were mixed. Although we received a high number of submissions (125– whereas Bates et al., 2020, only received 23 articles for review), we had hoped for more precisely directed submissions showing the various impacts of AI on higher education. The six articles represented here were selected for their scholarly address of the call while meeting the high standards of this journal.

Brief overview of accepted articles

Readers are recommended to start with Bond et al.’s meta-review of articles synthesizing research on AIEd published between 2018 and July 2023. Following a rigorous search strategy, they retrieved 66 publications from seven databases that met their inclusion criteria, which again highlights the rapid growth of the AIEd research field. Two-thirds of the included articles or conference papers were systematic reviews. A very interesting analysis of the meta-review is an overview of the benefits and challenges mentioned in the reviews of AIEd. Opportunities for personalized learning are stated as the top benefit, while the greatest challenges are ethical concerns, curriculum development issues, infrastructural problems, a lack of teacher technical knowledge, and a potential shift in authority moving to AI systems. Bond et al. also distilled the research gaps identified in the reviews. These include ethical issues, and more research focusing on educational applications with a wider range of stakeholders, requiring more interdisciplinary approaches and longitudinal studies. In this way, the meta-review provides an excellent basis for further work on key topics in AIEd research.

Given the emergent awareness and interest in LLMs, it is not surprising that they, and ChatGPT in particular, feature dominantly in this issue. LLMs are mentioned in all six articles in this collection and many of these articles demonstrate efforts to assess the implications of, and develop appropriate responses to, the widespread availability of these tools.

Walter’s article addresses the influence of generative AI on institutional policies and presents the institutional responses to these new developments based on a case study of a university of applied sciences in Switzerland. A central issue addressed here is the competencies and skills of teachers and students to understand and deal with AI systems in order to establish a new “culture of AI” for academic integrity. Furthermore, the article offers a detailed analysis of strategies for embedding skills such as “prompt engineering” within curricula and pedagogical practices. Walter’s discussion is rounded out with a narrative literature review that summarizes some key challenges and practical suggestions for implementing AI in education.

From the students’ perspective, Sun et al. considered the impact of ChatGPT on students’ programming behaviors, performances, and perceptions. In an empirical mixed-methods study with 82 students enrolled at a Chinese university, they compared a class that employed ChatGPT facilitated programming with another class without ChatGPT support (i.e., self-directed programming). Using a combination of log data, computer screen recordings, questionnaires, and semi-structured interviews, Sun et al. elaborated on the activities for which the students use the tool for programming, how useful the tool was perceived to be, and whether its use impacted students’ programming performance in their final assignment. While no significant difference was found between the two classes in terms of their academic performance, the data revealed distinct learning strategies across the two groups. Furthermore, the qualitative data gathered from the post-course interviews provide insight on both positive and negative aspects of the user experience from the perspective of the students themselves.

Also centering student perceptions, Escalante et al.’s study compared the impact of human- and ChatGPT-generated feedback on students’ writing during two, six-week courses of English as a new language (ENL). Study 1 found no difference in academic performance between groups, and weekly surveys in Study 2 showed that perceptions of the two types of feedback were almost balanced. About half of the ENL students preferred the feedback from the human tutor, while the other half were more satisfied with the ChatGPT-generated feedback. Those who appreciated the dialogue with the tutor stated that they were more engaged and improved their communication skills. The other group found the AI feedback more readable and detailed.

Abbas et al. examined whether university students’ self-reported use of generative AI is harmful or helpful for their learning. The general conclusion of this study on intensive ChatGPT use is rather negative. Among other findings, the authors noted that students who self-reported frequently using ChatGPT tended to have poor CGPAs and were more likely to procrastinate; self-reported levels of workload and time pressure were also positively related with students’ reported use of ChatGPT, whereas students who were more sensitive to rewards (i.e., feared lower grades) were less likely to use the tool. The authors concluded by taking an institutional perspective and emphasizing the importance of providing support, especially for efficient time management and workload distribution. Along with other papers in this issue, Abbas et al. suggested that it is essential to develop skills in dealing with generative AI in order to empower students to use these tools critically and creatively.

These considerations also inform the last article in our collection by Delcker et al., which deals with the AI competence of first-year students as a predictor for the use of AI tools for learning in higher education. They found that attitudes toward AI as well as perceived benefits significantly predict the intended use of AI tools. The authors emphasized the need to facilitate students’ AI competencies and integrate AI applications in the instructional design to create safe learning environments that foster possibilities to actively engage with AI.

Conclusion

The articles that are represented here are strongly influenced by the current development around generative AI tools with a view to teaching and learning processes at the micro level. While these efforts are indeed necessary to evaluate the capabilities of these tools and their potential impacts, our aims of focusing in particular on the system, organizational, and administrative levels; and exploring the effects and implications for different stakeholders in higher education could not be achieved.

We must now be careful about making premature conclusions. Perhaps we have simply not reached the relevant research groups with our call. However, as the aforementioned systematic reviews and the meta-review by Bond et al. in this volume show, AIEd research continues to focus primarily on students as research objects at the micro level and the research is often not primarily pedagogical in nature.

Even four years after the last special issue on AIEd in this journal, educational research still seems to be lagging behind the dynamic technical and methodological development of AI algorithms and applications. There is still a lot to be done from an educational perspective in this rapidly growing field of educational technology.