Exploring the impact of Artificial Intelligence and robots on higher education through literature-based design fictions

Abstract

Artificial Intelligence (AI) and robotics are likely to have a significant long-term impact on higher education (HE). The scope of this impact is hard to grasp partly because the literature is siloed, as well as the changing meaning of the concepts themselves. But developments are surrounded by controversies in terms of what is technically possible, what is practical to implement and what is desirable, pedagogically or for the good of society. Design fictions that vividly imagine future scenarios of AI or robotics in use offer a means both to explain and query the technological possibilities. The paper describes the use of a wide-ranging narrative literature review to develop eight such design fictions that capture the range of potential use of AI and robots in learning, administration and research. They prompt wider discussion by instantiating such issues as how they might enable teaching of high order skills or change staff roles, as well as exploring the impact on human agency and the nature of datafication.

Introduction

The potential of Artificial Intelligence (AI) and robots to reshape our future has attracted vast interest among the public, government and academia in the last few years. As in every other sector of life, higher education (HE) will be affected, perhaps in a profound way (Bates et al., 2020; DeMartini and Benussi, 2017). HE will have to adapt to educate people to operate in a new economy and potentially for a different way of life. AI and robotics are also likely to change how education itself works, altering what learning is like, the role of teachers and researchers, and how universities work as institutions.

However, the potential changes in HE are hard to grasp for a number of reasons. One reason is that impact is, as Clay (2018) puts it, “wide and deep” yet the research literature discussing it is siloed. AI and robotics for education are separate literatures, for example. AI for education, learning analytics (LA) and educational data mining also remain somewhat separate fields. Applications to HE research as opposed to learning, such as the robot scientist concept or text and data mining (TDM), are also usually discussed separately. Thus if we wish to grasp the potential impact of AI and robots on HE holistically we need to extend our vision across the breadth of these diverse literatures.

A further reason why the potential implications of AI and robots for HE are quite hard to grasp is because rather than a single technology, something like AI is an idea or aspiration for how computers could participate in human decision making. Faith in how to do this has shifted across different technologies over time; as have concepts of learning (Roll and Wylie, 2016). Also, because AI and robotics are ideas that have been pursued over many decades there are some quite mature applications: impacts have already happened. Equally there are potential applications that are being developed and many only just beginning to be imagined. So, confusingly from a temporal perspective, uses of AI and robots in HE are past, present and future.

Although hard to fully grasp, it is important that a wider understanding and debate is achieved, because AI and robotics pose a range of pedagogic, practical, ethical and social justice challenges. A large body of educational literature explores the challenges of implementing new technologies in the classroom as a change management issue (e.g. as synthesised by Reid, 2014). Introducing AI and robots will not be a smooth process without its challenges and ironies. There is also a strong tradition in the educational literature of critical responses to technology in HE. These typically focus on issues such as the potential of technology to dehumanise the learning experience. They are often driven by fear of commercialisation or neo-liberal ideologies wrapped up in technology. Similar arguments are developing around AI and robotics. There is a particularly strong concentration of critique around the datafication of HE. Thus the questions around the use of AI and robots are as much about what we should do as what is possible (Selwyn, 2019a). Yet according to a recent literature review most current research about AI in learning is from computer science and seems to neglect both pedagogy and ethics (Zawacki-Richter et al., 2019). Research on AIEd has also been recognised to have a WEIRD (western, educated, industrialized, rich and democratic) bias for some time (Blanchard, 2015).

One device to make the use of AI and robots more graspable is fiction, with its ability to help us imagine alternative worlds. Science fiction has already had a powerful influence on creating collective imaginaries of technology and so in shaping the future (Dourish and Bell, 2014). Science fiction has had a fascination with AI and robots, presumably because they enhance or replace defining human attributes: the mind and the body. To harness the power of fiction for the critical imagination, a growing body of work within Human Computer Interaction (HCI) studies adopts the use of speculative or critical narratives to destabilise assumptions through “design fictions” (Blythe 2017): “a conflation of design, science fact, and science fiction” (Bleecker, 2009: 6). They can be used to pose critical questions about the impact of technology on society and to actively engage wider publics in how technology is designed. This is a promising route for making the impact of AI and robotics on HE easier to grasp. In this context, the purpose of this paper is to describe the development of a collection of design fictions to widen the debate about the potential impact of AI and robots on HE, based on a wide-ranging narrative literature review. First, the paper will explain more fully the design fiction method.

Method: design fictions

There are many types of fictions that are used for our thinking about the future. In strategic planning and in future studies, scenarios—essentially fictional narratives—are used to encapsulate contrasting possible futures (Amer et al., 2013; Inayatullah, 2008). These are then used collaboratively by stakeholders to make choices about preferred directions. On a more practical level, in designing information systems traditional design scenarios are short narratives that picture use of a planned system and that are employed to explain how it could be used to solve existing problems. As Carroll (1999) argues, such scenarios are also essentially stories or fictions and this reflects the fact that system design is inherently a creative process (Blythe, 2017). They are often used to involve stakeholders in systems design. The benefit is that the fictional scenario prompts reflection outside the constraints of trying to produce something that simply works (Carroll, 1999). But they tend to represent a system being used entirely as intended (Nathan et al., 2007). They typically only include immediate stakeholders and immediate contexts of use, rather than thinking about the wider societal impacts of pervasive use of the technology. A growing body of work in the study of HCI refashions these narratives:

Design fiction is about creative provocation, raising questions, innovation, and exploration. (Bleecker, 2009: 7).

Design fictions create a speculative space in which to raise questions about whether a particular technology is desirable, the socio-cultural assumptions built into technologies, the potential for different technologies to make different worlds, our relation to technology in general, and indeed our role in making the future happen.

Design fictions exist on a spectrum between speculative and critical. Speculative fictions are exploratory. More radical, critical fictions ask fundamental questions about the organisation of society and are rooted in traditions of critical design (Dunne and Raby, 2001). By definition they challenge technical solutionism: the way that technologies seem to be built to solve a problem that does not necessarily exist or ignore the contextual issues that might impact its success (Blythe et al., 2016).

Design fictions can be used in research in a number of ways, where:

• 1. Fictions are the output themselves, as in this paper.

• 2. Fictions (or an artefact such as a video based on them) are used to elicit research data, e.g. through interviews or focus groups Lyckvi et al. (2018).

• 3. Fictions are co-created with the public as part of a process of raising awareness (e.g. Tsekleves et al. 2017).

For a study of the potential impact of AI and robots on HE, design fictions are a particularly suitable method. They are already used by some authors working in the field such as Pinkwart (2016), Luckin and Holmes (2017) and Selwyn et al. (2020). As a research tool, design fictions can encapsulate key issues in a short, accessible form. Critically, they have the potential to change the scope of the debate, by shifting attention away from the existing literature and its focus on developing and testing specific AI applications (Zawacki-Richter et al., 2019) to weighing up more or less desirable directions of travel for society. They can be used to pose critical questions that are not being asked by developers because of the WEIRD bias in the research community itself (Blanchard, 2015), to shift focus onto ethical and social justice issues, and also raise doubts based on practical obstacles to their widespread adoption. Fictions engage readers imaginatively and on an affective level. Furthermore, because they are explicitly fictions readers can challenge their assumptions, even get involved in actively rewriting them.

Design fictions are often individual texts. But collections of fictions create potential for reading against each other, further prompting thoughts about alternative futures. In a similar way, in future studies, scenarios are often generated around four or more alternatives, each premised on different assumptions (Inayatullah, 2008). This avoids the tendency towards a utopian/ dystopian dualism found in some use of fiction (Rummel et al., 2016; Pinkwart 2016). Thus in this study the aim was to produce a collection of contrasting fictions that surface the range of debates revolving around the application of AI and robotics to HE.

The process of producing fictions is not easy to render transparent.

In this study the foundation for the fictions was a wide-ranging narrative review of the literature (Templier and Paré, 2015). The purpose of the review was to generate a picture of the pedagogic, social, ethical and implementation issues raised by the latest trends in the application of AI and robots to teaching, research and administrative functions in HE, as a foundation for narratives which could instantiate the issues in a fictional form. We know from previous systematic reviews that these type of issue are neglected at least in the literature on AIEds (Zawacki-Richter et al., 2019). So the chief novelty of the review lay in (a) focusing on social, ethical, pedagogic and management implications (b) encompassing both AI and robotics as related aspects of automation and (c) seeking to be inclusive across the full range of functions of HE, including impacts on learning, but also on research and scholarly communications, as well as administrative functions, and estates management (smart campus).

In order to gather references for the review, systematic searches on the ERIC database for relevant terms such as “AI or Artificial Intelligence”; “conversational agent”, “AIED” were conducted. Selection was made for items which either primarily addressed non-technical issues or which themselves contained substantial literature reviews that could be used to gain a picture of the most recent applications. This systematic search was combined with snowballing (also known as pearl growing techniques) using references by and to highly relevant matches to find other relevant material. While typically underreported in systematic reviews this method has been shown to be highly effective in retrieving more relevant items (Badampudi et al. 2015). Some grey literature was included because there are a large number of reports by governmental organisations summarizing the social implications of AI and robots. Because many issues relating to datafication are foreshadowed in the literature on learning analytics, this topic was also included. In addition, some general literature on AI and robots, while not directly referencing education, was deemed to be relevant, particularly as it was recognised that education might be a late adopter and so impacts would be felt through wider social changes rather than directly through educational applications. Literature reviews which suggested trends in current technologies were included but items which were detailed reports of the development of technologies were excluded. Items prior to 2016 tended also to be excluded, because the concern was with the latest wave of AI and robots. As a result of these searches in the order of 500 items were consulted, with around 200 items deemed to be of high relevance. As such there is no claim that this was an “exhaustive” review, rather it should be seen as complimenting existing systematic reviews by serving a different purpose. The review also successfully identified a number of existing fictions in the literature that could then be rewritten to fit the needs of the study, such as to apply to HE, to make them more concise or add new elements (fictions 1, 3, 4).

As an imaginative act, writing fictions is not reducible to a completely transparent method, although some aspects can be described (Lyckvi et al., 2018). Some techniques to create effective critical designs are suggested by Auger (2013) such as placing something uncanny or unexpected against the backdrop of mundane normality and a sense of verisimilitude (perhaps achieved through mixing fact and fiction). Fiction 6, for example, exploits the mundane feel of committee meeting minutes to help us imagine the debates that would occur among university leaders implementing AI. A common strategy is to take the implications of a central counterfactual premise to its logical conclusion: asking: “what if?” For example, fiction 7 extends existing strategies of gathering data and using chatbots to act on them to its logical extension as a comprehensive system of data surveillance. Another technique used here was to exploit certain genres of writing such as in fiction 6 where using a style of writing from marketing and PR remind us of the role of EdTech companies in producing AI and robots.

Table 1 offers a summary of the eight fictions produced through this process. The fictions explore the potential of AI and robots in different areas of university activity, in learning, administration and research (Table 1 column 5). They seek to represent some different types of technology (column 2). Some are rather futuristic, most seem feasible today, or in the very near future (column 3). The full text of the fictions and supporting material can be downloaded from the University of Sheffield data repository, ORDA, and used under a cc-by-sa licence (https://doi.org/10.35542/osf.io/s2jc8). The following sections describe each fiction in turn, showing how it relates to the literature and surfaces relevant issues. Table 2 below will summarise the issues raised.

Table 1 Summary of the design fictions

Table 2 Issues raised in the fictions

In the following sections each of the eight fictions is described, set in the context of the literature review material that shaped their construction.

AI and robots in learning: Fiction 1, “AIDan, the teaching assistant”

Much of the literature around AI in learning focuses on tools that directly teach students (Baker and Smith, 2019; Holmes et al., 2019; Zawacki-Richter et al., 2019). This includes classes of systems such as:

• Intelligent tutoring systems (ITS) which teach course content step by step, taking an approach personalised to the individual. Holmes et al. (2019) differentiate different types of Intelligent Tutoring Systems, based on whether they adopt a linear, dialogic or more exploratory model.

  • o One emerging area of adaptivity is using sensors to detect the emotional and physical state of the learner, recognising the embodied and affective aspects of learning (Luckin, et al., 2016); a further link is being made to how virtual and augmented reality can be used to make the experience more engaging and authentic (Holmes et al., 2019).

• Automatic writing evaluation (AWE) which are tools to assess and offer feedback on writing style (rather than content) such as learnandwrite, Grammarly and Turnitin’s Revision Assistant (Strobl, et al. 2019; Hussein et al., 2019; Hockly, 2019).

• Conversational agents (also known as Chatbots or virtual assistants) which are AI tools designed to converse with humans (Winkler and Sӧllner, 2018).

• The adaptive pedagogical agent, which is an “anthropomorphic virtual character used in an online learning environment to serve instructional purposes” (Martha and Santoso, 2017).

Many of these technologies are rather mature, such as AWE and ITS. However, there are also a wide range of different type of systems within each category, e.g. conversational agents can be designed for short or long term interaction, and could act as tutors, engage in language practice, answer questions, promote reflection or act as co-learners. They could be based on text or verbal interaction (Følstad et al., 2019; Wellnhammer et al., 2020).

Much of such literature reflects the development of AI technologies and their evaluation compared to other forms of teaching. However, according to a recent review it is primarily written by computer scientists mostly from a technical point of view with relatively little connection to pedagogy or ethics (Zawacki-Richter et al., 2019). In contrast some authors such as Luckin and Holmes, seek to move beyond the rather narrow development of tools and their evaluation, to envisioning how AI can address the grand challenges of learning in the twenty-first century (Luckin, et al. 2016; Holmes et al., 2019; Woolf et al., 2013). According to this vision many of the inefficiencies and injustices of the current global education system can be addressed by applying AI.

To surface such discussion around what is possible fiction 1 is based loosely on a narrative published by Luckin and Holmes (2017) themselves. In their paper, they imagine a school classroom ten years into the future from the time of writing, where a teacher is working with an AI teaching assistant. Built into their fiction are the key features of their vision of AI (Luckin et al. 2016), thus emphasis is given to:

• 1. AI designed to support teachers rather than replacing them;

• 2. Personalisation of learning experiences through adaptivity;

• 3. Replacement of one-off assessment by continuous monitoring of performance (Luckin, 2017);

• 4. The monitoring of haptic data to adjust learning material to students’ emotional and physical state in real time;

• 5. The potential of AI to support learning twenty-first century skills, such as collaborative skills;

• 6. Teachers developing skills in data analysis as part of their role;

• 7. Students (and parents) as well as teachers having access to data about their learning.

While Luckin and Holmes (2017) acknowledge that the vision of AI sounds a “bit big brother” it is, as one would expect, essentially an optimistic piece in which all the key technologies they envisage are brought together to improve learning in a broad sense. The fiction developed here retains most of these elements, but reimagined for an HE context, and with a number of other changes:

• 1. Reference is also made to rooting teaching in learning science, one of the arguments for AI Luckin makes in a number of places (e.g. Luckin et al. 2016).

• 2. Students developing a long term relationship with the AI. It is often seen as a desirable aspect of providing AI as a lifelong learning partner (Woolf, et al. 2013).

Of course, the more sceptical reader may be troubled by some aspects of this vision, including the potential effects of continuously monitoring performance as a form of surveillance. The emphasis on personalization of learning through AI has been increasingly questioned (Selwyn, 2019a).

The following excerpt gives a flavour of the fiction:

Actually, I partly picked this Uni because I knew they had AI like AIDan which teach you on principles based in learning science.

And exams are a thing of the past! AIDan continuously updates my profile and uses this to measure what I have learned.

I have set tutorials with AIDan to analyse data on my performance. Jane often talks me through my learning data as well.

I work with him planning things like my module choices too.

Some of my data goes to people in the department (like my personal tutor) to student and campus services and the library to help personalise their services.

Social robots in learning: Fiction 2, “Footbotball”

Luckin and Holmes (2017) see AI as instantiated by sensors and cameras built into the classroom furniture. Their AI does not seem to have a physical form, though it does have a human name. But there is also a literature around educational robots: a type of social robot for learning.

a physical robot, in the same space as the student. It has an intelligence that can support learning tasks and students learn by interacting with it through suitable semiotic systems (Catlin et al., 2018).

There is some evidence that learning is better when the learner interacts with a physical entity rather than purely virtual agent and certainly there might be beneficial where what is learned involves embodiment (Belpaeme et al., 2018). Fiction 2 offers an imaginative account of what learning alongside robots might be like, in the context of university sport rather than within the curriculum. The protagonist describes how he is benefiting from using university facilities to participate in an imaginary sport, footbotball.

Maybe it’s a bit weird to say, but it’s about developing mutual understanding and… respect. Like the bots can sense your feelings too and chip in with a word just to pick you up if you make a mistake. And you have to develop an awareness of their needs too. Know when is the right time to say something to them to influence them in the right direction. When you watch the best teams they are always like talking to each other. But also just moving together, keeping eyes on and moving as a unit.

The protagonist in fiction 2 describes the high level and employability skills he is learning from a sporting application of robotics. This also reminds us of how the widespread use of AI and robots in wider social contexts may be a key driver for adoption in HE.

Fiction 3: “Criticalbot in conversation”

Both fictions 1 and 2 are glossy science fictions, with a strongly futuristic feel and, as in traditional design scenarios the technology seems to be used as intended by the designer. In contrast, the third fiction is inspired by Bayne’s (2015) description of Teacherbot, a chatbot developed to support a MOOC on elearning and digital cultures. Teacherbot does not masquerade as human. Students on the course are told what it is and its interactions are clumsy enough to reveal this anyway. Yet Bayne’s (2015) argument is that it offers positive non-human affordances. Students seem to learn from it, partly by reflecting on its place in their learning. Thus fiction 3, echoing the sample transcripts between Teacherbot and real students quoted in Bayne (2015), is a fictional dialogue between a conversational agent and a group of students working on an assignment (itself on the topic of bias in AI). Criticalbot, as its name suggests, is being imagined used to teach the students to be more critical, e.g. by prompting them to think harder about how to read an academic text, raising questions about the authority of authors, and prompting discussion around quotes from a key text.

Student 1: @CriticalBot we have been set the task to research bias in AIED. What do you know about bias in AIED?

CriticalBot: Blanchard is recommended in the reading list.

Student 2: I think he means rtfm.

Student 1: Blanchard?

CriticalBot: Sorry. I should have typed: Blanchard, E. G. (2015). Socio-cultural imbalances in AIED research: Investigations, implications and opportunities. International Journal of Artificial Intelligence in Education, 25(2), 204–228. No one’s perfect.

Student 1: @CriticalBot What is the main point he makes?

CriticalBot: Nice try. Try reading the abstract. Abstracts summarise key points. It may also be useful to read the conclusion section to grasp the author’s main argument before deciding on whether to undertake a closer reading.

As the quotation from the fiction illustrates, echoing Bayne (2015), the conversation in Fiction 2 is not necessarily smooth; misunderstandings and conflicts occur. The fiction brings into view the less compliant vision of the student who might wish to game the system, a potential problem with AI which is apparent in the literature of AWE (Hussein et al. 2019). This fiction encapsulates an important alternative potential imaginary of AI, as a simple, low-tech intervention. At the same time in being designed to promote critical thinking it can also be seen as teaching a key, high-level skill. This challenges us to ask if an AI can truly do that and how.

The intelligent campus: Fiction 4, “The intelligent campus app”

The AIED literature with its emphasis on the direct application of AI to learning accounts for a big block of the literature about AI in Higher Education, but not all of it. Another rather separate literature exists around the smart or intelligent campus (e.g. JISC 2018; Min-Allah and Alrashed, 2020; Dong et al., 2020). This is the application of Internet of Things and increasingly AI to the management of the campus environment. This is often oriented towards estates management, such as monitoring room usage and controlling lighting and heating. But it does also encompass support of wayfinding, attendance monitoring, and ultimately of student experience, so presents an interesting contrast to the AIEd literature.

The fourth fiction is adapted from a report each section of which is introduced by quotes from an imaginary day in the life of a student, Leda, who reflects on the benefits of the intelligent/smart campus technologies to her learning experience (JISC, 2018). The emphasis in the report is on:

• 1. Data driven support of wayfinding and time management;

• 2. Integration of smart campus with smart city features (e.g. bus and traffic news);

• 3. Attendance monitoring and delivery of learning resources;

The student also muses about the ethics of the AI. She is presented as a little ambivalent about the monitoring technologies, and as in Luckin and Holmes (2017), it is referred to in her own words as potentially “a bit big brother” (JISC 2018: 9). But ultimately she concludes that the smart campus improves her experience as a student. In this narrative, unlike in the Luckin and Holmes (2017) fiction, the AI is much more in the background and lacks a strong personality. It is a different sort of optimistic vision geared towards convenience rather than excellence. There is much less of a futuristic feel, indeed one could say that not only does the technology exist to deliver many of the services described, they are already available and in use—though perhaps not integrated within one application.

Sitting on the bus I look at the plan for the day suggested in the University app. A couple of timetabled classes; a group work meeting; and there is a reminder about that R205 essay I have been putting off. There is quite a big slot this morning when the App suggests I could be in the library planning the essay – as well as doing the prep work for one of the classes it has reminded me about.

It is predicting that the library is going to be very busy after 11AM anyway, so I decide to go straight there.

The fiction seeks to bring out more about the idea of “nudging” to change behaviours a concept often linked to AI and the ethics of which are queried by Selwyn (2019a). The issue of how AI and robots might impact the agency of the learner recurs across the first four fictions.

AI and robotics in research: Fiction 5, “The Research Management Suite TM”

So far in this paper most of the focus has been on the application of AI and robotics to learning. AI also has applications in university research, but it is an area far less commonly considered than learning and teaching. Only 1% of CIOs responding to a survey of HEIs by Gartner had deployed AI for research, compared to 27% for institutional analytics and 10% for adaptive learning (Lowendahl and Williams, 2018). Some AI could be used directly in research, not just to perform analytical tasks, but to generate hypotheses to be tested (Jones et al., 2019). The “robot scientist” being tireless and able to work in a precise way could carry through many experiments and increase reproducibility (King, et al., 2009; Sparkes et al., 2010). It might have the potential to make significant discoveries independently, perhaps by simply exploiting its tirelessness to test every possible hypothesis rather than use intuition to select promising ones (Kitano, 2016).

Another direct application of AI to research is text and data mining (TDM). Given the vast rate of academic publishing there is growing need to mine published literature to offer summaries to researchers or even to develop and test hypotheses (McDonald and Kelly, 2012). Advances in translation also offer potential to make the literature in other languages more accessible, with important benefits.

Developments in publishing give us a further insight into how AI might be applied in the research domain. Publishers are investing heavily in AI (Gabriel, 2019). One probable landmark was that in 2019, Springer published the first “machine generated research book” (Schoenenberger, 2019: v): a literature review of research on Lithium-Ion batteries, written entirely automatically. This does not suggest the end of the academic author, Springer suggest, but does imply changing roles (Schoenenberger, 2019). AI is being applied to many aspects of the publication process: to identify peer reviewers (Price and Flach, 2017), to assist review by checking statistics, to summarise open peer reviews, to check for plagiarism or for the fabrication of data (Heaven, 2018), to assist copy editing, to suggest keywords and to summarise and translate text. Other tools claim to predict the future citation of articles (Thelwall, 2019). Data about academics, their patterns of collaboration and citation through scientometrics are currently based primarily on structured bibliographic data. The cutting edge is the application of text mining techniques to further analyse research methods, collaboration patterns, and so forth (Atanassova et al., 2019). This implies a potential revolution in the management and evaluation of research. It will be relevant to ask what responsible research metrics are in this context (Wilsdon, 2015).

Instantiating these developments, the sixth fiction revolves around a university licensing “Research Management Suite TM “a set of imaginary proprietary tools to offer institutional level support to its researchers to increase and perhaps measure their productivity. A flavour of the fiction can be gleaned from this except:

Academic Mentor ™ is our premium meta analysis service. Drawing on historic career data from across the disciplines, it identifies potential career pathways to inform your choices in your research strategy. By identifying structural holes in research fields it enables you to position your own research within emerging research activity, so maximising your visibility and contribution. Mining data from funder strategy, the latest publications, preprints and news sources it identifies emergent interdisciplinary fields, matching your research skills and interests to the complex dynamics of the changing research landscape.

This fiction prompts questions about the nature of the researcher’s role and ultimately about what research is. At what point does the AI become a co-author, because it is making a substantive intellectual contribution to writing a research output, making a creative leap or even securing funding? Given the centrality of research to academic identity this indeed may feel even more challenging than the teaching related scenarios. This fiction also recognised the important role of EdTech companies in how AI reaches HE, partly because of the high cost of AI development. The reader is also prompted to wonder how the technology might disrupt the HE landscape if those investing in these technologies were ambitious newer institutions keen to rise in university league tables.

Tackling pragmatic barriers: Fiction 6, “Verbatim minutes of University AI project steering committee: AI implementation phase 3”

A very large literature around technologies in HE in general focuses on the challenges of implementing them as a change management problem. Reid (2014), for example, seeks to develop a model of the differing factors that block the smooth implementation of learning technologies in the classroom, such as problems with access to the technology, project management challenges, as well as issues around teacher identity. Echoing these arguments, Tsai et al.’s (2017, 2019) work captures why for all the hype around it, Learning Analytics have not yet found extensive practical application in HE. Given that AI requires intensive use of data, by extension we can argue that the same barriers will probably apply to AI. Specifically Tsai et al. (2017, 2019) identify barriers in terms of technical, financial and other resource demands, ethics and privacy issues, failures of leadership, a failure to involve all stakeholders (students in particular) in development, a focus on technical issues and neglect of pedagogy, insufficient staff training and a lack of evidence demonstrating the impact on learning. There are hints of similar types of challenge around the implementation of administration focussed applications (Nurshatayeva, et al., 2020) and TDM (FutureTDM, 2016).

Reflecting these thoughts, the fifth fiction is an extract from an imaginary committee meeting, in which senior university managers discuss the challenges they are facing in implementing AI. It seeks to surface issues around teacher identity, disciplinary differences and resource pressures that might shape the extensive implementation of AI in practice.

Faculty of Humanities Director: But I think there is a pedagogic issue here. With the greatest of respect to Engineering, this approach to teaching, simply does not fit our subject. You cannot debate a poem or a philosophical treatise with a machine.

Faculty of Engineering Director: The pilot project also showed improved student satisfaction. Data also showed better student performance. Less drop outs.

Faculty of Humanities Director: Maybe that’s because…

Vice Chancellor: All areas where Faculty of Humanities has historically had a strategic issue.

Faculty of Engineering Director: The impact on employability has also been fantastic, in terms of employers starting to recognise the value of our degrees now fluency with automation is part of our graduate attributes statement.

Faculty of Humanities Director: I see the benefits, I really do. But you have to remember you are taking on deep seated assumptions within the disciplinary culture of Humanities at this university. Staff are already under pressure with student numbers not to mention in terms of producing world class research! I am not sure how far this can be pushed. I wouldn’t want to see more industrial action.

Learning analytics and datafication: Fiction 7, “Dashboards”

Given the strong relation between “big data” and AI, the claimed benefits and the controversies that already exist around LA are relevant to AI too (Selwyn, 2019a). The main argument for LA is that they give teachers and learners themselves information to improve learning processes. Advocates talk of an obligation to act. LA can also be used for the administration of admissions decisions and ensuring retention. Chatbots are now being used to assist applicants through complex admissions processes or to maintain contact to ensure retention and appear to offer a cheap and effective alternative (Page and Gehlbach, 2017; Nurshatayeva et al., 2020). Gathering more data about HE also promotes public accountability.

However, data use in AI does raise many issues. The greater the dependence on data or data driven AI the greater the security issues associated with the technology. Another inevitable concern is with legality and the need to abide by appropriate privacy legislation, such as GDPR in Europe. Linked to this are clearly privacy issues, implying consent, the right to control over the use of one’s data and the right to withdraw (Fjeld et al., 2020). Yet a recent study by Jones (2020) found students knew little of how LA were being used in their institution or remembered consenting to allowing their data to be used. These would all be recognised as issues by most AI projects.

However, increasingly critiques of AI in learning centre around the datafication of education (Jarke and Breiter, 2019; Williamson and Eynon, 2020; Selwyn, 2019a; Kwet and Prinsloo, 2020). A data driven educational system has the potential to be used or experienced as a surveillance system. “What can be accomplished with data is usually a euphemism for what can be accomplished with surveillance” (Kwet and Prinsloo, 2020: 512). Not only might individual freedoms be threatened by institutions or commercial providers undertaking surveillance of student and teaching staff behaviour, there is also a chilling effect just through the fear of being watched (Kwet and Prinsloo, 2020). Students become mere data points, as surveillance becomes intensified and normalised (Manolev et al. 2019). While access to their own learning data could be empowering for students, techniques such as nudging intended to influence people without their knowledge undermine human agency (Selwyn, 2019b). Loss of human agency is one of the fears revolving around AI and robots.

Further, a key issue with AI is that although predictions can be accurate or useful it is quite unclear how these were produced. Because AI “learns” from data, even the designers do not fully understand how the results were arrived at so they are certainly hard to explain to the public. The result is a lack of transparency, and so of accountability, leading to deresponsibilisation.

Much of the current debate around big data and AI revolves around bias, created by using training data that does not represent the whole population, reinforced by the lack of diversity among designers of the systems. If data is based on existing behaviour, this is likely to reproduce existing patterns of disadvantage in society, unless AI design takes into account social context—but datafication is driven by standardisation. Focussing on technology diverts attention from the real causes of achievement gaps in social structures, it could be argued (Macgilchrist, 2019). While often promoted as a means of empowering learners and their teachers, mass personalisation of education redistributes power away from local decision making (Jarke and Breiter, 2019; Zeide, 2017). In the context of AIEd there is potential for assumptions about what should be taught to show very strong cultural bias, in the same way that critics have already argued that plagiarism detection systems impose culturally specific notions of authorship and are marketed in a way to reinforce crude ethnic stereotypes (Canzonetta and Kannan, 2016).

Datafication also produces performativity: the tendency of institutions (and teachers and students) to shift their behaviour towards doing what scores well against the metric, in a league table mentality. Yet what is measured is often a proxy of learning or reductive of what learning in its full sense is, critics argue (Selwyn, 2019b). The potential impact is to turn HE further into a marketplace (Williamson, 2019). It is evident that AI developments are often partly a marketing exercise (Lacity, 2017). Edtech companies play a dominant role in developing AI (Williamson and Eynon, 2020). Selwyn (2019a) worries that those running education will be seduced by glittering promises of techno-solutionism, when the technology does not really work. The UK government has invested heavily in gathering more data about HE in order to promote the reform of HE in the direction of marketisation and student choice (Williamson and Eynon, 2020). Learning data could also increasingly itself become a commodity, further reinforcing the commercialisation of HE.

Thus fiction 6 explores the potential to gather data about learning on a huge scale, make predictions based on it and take actions via conveying information to humans or through chatbots. In the fiction the protagonist explains an imaginary institutional level system that is making data driven decisions about applicants and current students.

Then here we monitor live progress of current students within their courses. We can dip down into attendance, learning environment use, library use, and of course module level performance and satisfaction plus the extra-curricula data. Really low-level stuff some of it. It’s pretty much all there, monitored in real time. We are really hot on transition detection and monitoring. The chatbots are used just to check in on students, see they are ok, nudge things along, gather more data. Sometimes you just stop and look at it ticking away and think “wow!”. That all gets crunched by the system. All the time we feed the predictives down into departmental dashboards, where they pick up the intervention work. Individual teaching staff have access via smart speaker. Meanwhile, we monitor the trend lines up here.

In the fiction the benefits in terms of being able to monitor and address attainment gaps is emphasised. The protagonist’s description of projects that are being worked on suggests competing drivers behind such developments including meeting government targets, cost saving and the potential to make money by reselling educational data.

Infrastructure: Fiction 8, “Minnie—the AI admin assistant”

A further dimension to the controversy around AI is to consider its environmental cost and the societal impact of the wider infrastructures needed to support AI. Brevini (2020) points out that a common AI training model in linguistics can create the equivalent of five times the lifetime emissions of an average US car. This foregrounds the often unremarked environmental impact of big data and AI. It also prompts us to ask questions about the infrastructure required for AI. Crawford and Joler’s (2018) brilliant Anatomy of an AI system reveals that making possible the functioning of a physically rather unassuming AI like Amazon echo, is a vast global infrastructure based on mass human labour, complex logistic chains and polluting industry.

The first part of fiction 8 describes a personal assistant based on voice recognition, like Siri, which answers all sorts of administrative questions.The protagonist expresses some unease with how the system works, reflecting the points made by Rummel et al. (2016) about the failure of systems if despite their potential sophistication they lack nuance and flexibility in their application. There is also a sense of alienation (Griffiths, 2015). The second part of the fiction extends this sense of unease to a wider perspective on the usually invisible, but very material infrastructure which AI requires, as captured in Crawford and Joler (2018). In addition, imagery is drawn from Maughan’s (2016) work where he travels backwards up the supply chain for consumer electronics from the surreal landscape of hi-tech docks then visiting different types of factories and ending up visiting a huge polluted lake created by mining operations for rare earth elements in China. This perspective queries all the other fictions with their focus on using technologies or even campus infrastructure by widening the vision to encompass the global infrastructures that are required to make AI possible.

The vast effort of global logistics to bring together countless components to build the devices through which we interact with AI. Lorries queuing at the container port as another ship comes in to dock.

Workers making computer components in hi-tech factories in East Asia. All dressed in the same blue overalls and facemasks, two hundred workers queue patiently waiting to be scan searched as they leave work at the end of the shift.

Exploitative mining extracting non-renewable, scarce minerals for computer components, polluting the environment and (it is suspected) reducing the life expectancy of local people. Pipes churn out a clayey sludge into a vast lake.

Conclusion: using the fictions together

As we have seen each of the fictions seeks to open up different positive visions or dimensions of debate around AI (summarised in Table 2 below). All implicitly ask questions about the nature of human agency in relationship to AI systems and robots, be that through empowerment through access to learning data (fiction 1), their power to play against the system (Fiction 3) or the hidden effects of nudging (Fiction 4) and the reinforcements of social inequalities. Many raise questions about the changing role of staff or the skills required to operate in this environment. They are written in a way seeking to avoid taking sides, e.g. not to always undercut a utopian view or simply present a dark dystopia. Each contains elements that might be inspirational or a cause of controversy. Specifically, they can be read together to suggest tensions between different possible futures. In particular fictions 7 and 8 and the commercial aspects implied by the presentation of fiction 5, reveal aspects of AI largely invisible in the glossy strongly positive images in fictions 1 and 2, or the deceptive mundanity of fiction 3. It is also anticipated that the fictions will be read “against the grain” by readers wishing to question what the future is likely to be or should be like. This is one of the affordances of them being fictions.

The most important contribution of the paper was the wide-ranging narrative literature review emphasising the social, ethical, pedagogic and management issues of automation through AI and robots on HE as a whole. On the basis of the understanding gained from the literature review a secondary contribution was the development of a collection of eight accessible, repurposable design fictions that prompt debate about the potential role of AI and robots in HE. This prompts us to notice common challenges, such as around commodification and the changing role of data. It encompasses work written by developers, by those with more visionary views, those who see the challenges as primarily pragmatic and those coming from much more critical perspectives.

The fictions are intended to be used to explore staff and student responses through data collection using the fictions to elicit views. The fictions could also be used in teaching to prompt debate among students, perhaps setting them the task to write new fictions (Rapp, 2020). Students of education could use them to explore the potential impact of AI on educational institutions and to discuss the role of technologies in educational change more generally. The fictions could be used in teaching students of computer science, data science, HCI and information systems in courses about computer ethics, social responsibility and sustainable computing—as well as those directly dealing with AI. They could also be used in Media Studies and Communications, e.g. to compare them with other future imaginaries in science fiction or to design multimedia creations inspired by such fictions. They might also be used for management studies as a case study of strategizing around AI in a particular industry.

While there is an advantage in seeking to encompass the issues within a small collection of engaging fictions that in total run to less than 5000 words, it must be acknowledged that not every issue is reflected. For example, what is not included is the different ways that AI and robots might be used in teaching different disciplines, such as languages, computer science or history. The many ways that robots might be used in background functions or to play the role themselves of learner also requires further exploration. Most of the fictions were located in a fairly near future, but there is also potential to develop much more futuristic fictions. These gaps leave room for the development of more fictions.

The paper has explained the rationale and process of writing design fictions. To the growing literature around design fictions, the paper seeks to make a contribution by emphasising the use of design fictions as collections, exploiting different narratives and styles and genre of writing to set up intertextual reflections that help us ask questions about technologies in the widest sense.