Learning analytics to develop future competences in higher education: a case study

Abstract

Higher education institutions are challenged to develop innovative educational solutions to meet the competence development requirements set by the emerging future. This qualitative case study aims to identify the future competences considered important for higher education students to acquire during their studies and how the development of these competences can be supported with learning analytics. Reflection on these issues is based on three dimensions (subject development, object, and social environment) of future competences. A special emphasis is placed on the views of 19 teaching professionals gathered from group interviews and analyzed through a qualitative content analysis. The findings indicate that subject development-related future competences, such as reflective competence, self-awareness and self-management, learning literacy, and personal agency and self-efficacy were strongly identified as necessary future competences. The potential of learning analytics to support their development was also widely recognized as it provides means to reflect on learning and competence development and increase one’s self-awareness of strengths and weaknesses. In addition, learning analytics was considered to promote goal-orientation, metacognition and learning to learn, active engagement as well as learning confidence. To deal with complex topics and tasks, students should also acquire object-related competences, such as changeability and digital competence. In addition, they need cooperation and communication competence as well as a developmental mindset to operate successfully in social environments. The use of learning analytics to support most of these object and social environment-related competences was considered promising as it enables the wide exploitation of digital tools and systems, the analysis and visualization of social interactions, and the formation of purposeful learning groups and communal development practices. However, concrete ways of applying learning analytics were largely unacknowledged. This study provides useful insights on the relationship of important future competences and learning analytics while expanding on previous research and conceptual modelling. The findings support professionals working at higher education institutions in facilitating successful conditions for the development of future competences and in advancing purposeful use of learning analytics.

Introduction

Nowadays, many educational institutions around the world are renewing educational practices to provide their students with comprehensive competences that can be utilized in complex, uncertain future contexts (Buckingham Shum & Deakin Crick, 2016). The necessity for acquiring professional, subject-specific competences is obvious, but there is an increasing need to master more generic competences applicable in different disciplines and emerging working-life contexts (Barrie, 2012; OECD, 2019). Although these generic abilities needed in the future have gained widespread attention also in higher education (HE), the ways in which they are acquired or facilitated still remain somewhat unclear (Barrie, 2003; Hershkovitz et al., 2016; Virtanen & Tynjälä, 2019).

Learning analytics (LA) can answer this challenge through its role in identifying and validating some of the significant measures of overall learning phenomena and fostering evidence-based practices and interventions which have often been impractical until recent years (Buckingham Shum & Deakin Crick, 2016; Hershkovitz et al., 2016; Mangaroska & Giannakos, 2018). LA refers to “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs” (Conole et al., 2011, para. 4). It is developed to harness massive amounts of data captured by the use of technologies as a part of educational practices (Gašević et al., 2017) for the needs of different stakeholders (Drachsler & Greller, 2012). The aim is to provide valuable insights into educational processes and means to improve teaching, learning and education (Siemens & Gašević, 2012).

Although LA seems to offer many promising approaches to measure and leverage the development of generic competences (Buckingham Shum & Deakin Crick, 2016; Gašević, 2019), further research and development work are still needed to better understand not only its possible benefits but also limitations as a part of learning and comprehensive competence development in HE. The study presented in this paper attempts to address this need and contribute to research by providing insights into what are the future competences considered important for HE students to acquire and how the use of LA could support their development even further. As working-life is constantly evolving and setting new priorities for competence development, identification of relevant future competences is needed to ensure that these desired learning outcomes are intentionally promoted in practice. In addition, an enhanced understanding of future competences serves as a foundation for aligning emerging technology-enhanced approaches, such as LA, to boost their development.

To consider these issues, a case study approach (see Cohen et al., 2005; Denscombe, 2011) is used in this study. A particular focus is placed on certain higher education institutions (HEIs) and the views of their teaching professionals as they are considered to hold a central role in promoting the development of future competences (e.g., Voogt & Pareja Roblin, 2012) and utilization of LA (e.g., Rienties et al., 2018) in practice. This study builds and expands on the outcomes of previous research and conceptual modelling. In particular, it aims to complement the Future Skills Triple Helix-Model developed by Ehlers (2020), described in more detail later in this paper. The tripartite structure of the model with its dimensional references (subject development, object, social environment) is employed to explore the importance of future competences and the potential of LA from versatile perspectives and to provide useful lenses for their alignment in HE.

The remainder of this paper is structured as follows. First, the theoretical framework of the study is introduced, including a definition of future competences and a discussion of how they are outlined in policy papers and current educational research. The focus then turns to the dimensions of future competences which also function as points of reference for this study. Next, the theoretical framework outlines what is meant by learning analytics and how it connects to the future competences. This is followed by sections in which the specific research questions are presented, and the context, study participants and methods used for data collection and analysis are outlined. After that, the main study findings are presented and conceptualized on the basis of the dimensions of future competences. They are also discussed in light of recent research and related work. In the final sections, the limitations of the study are identified and addressed and directions for future research are suggested.

Theoretical framework

Considerations for future competences in higher education

Generic, interdisciplinary competences have gained increased attention around the world. They have been referred to using several terms such as key competences (Council of the European Union, 2018), future competences (Marope et al., 2019) and transformative competencies (OECD, 2019). In some contexts, they have been called twenty-first century skills (Binkley et al., 2012) and competences (Ananiadou & Claro, 2009), generic employability skills (Curtis & McKenzie, 2002), soft skills (Robles, 2012), graduate attributes (Barrie, 2012; Hager & Holland, 2006) and generic capabilities (Bowden et al., 2000). These interchangeable terms, often used as synonyms, have remained rather loose and ill-defined while allowing space for competing and divergent conceptualizations (Rigby et al., 2009; Sin & Reid, 2005).

In this study, the term ‘future competences’ is used to refer to those generic competences that are relevant for HE students and which can prepare them for operating in emerging future contexts. Future competences enable students to solve problems and act successfully in a self-organized manner in uncertain settings and various contexts (Ehlers, 2020). They are seen to entail not only the mobilization of knowledge and skills but also personality traits such as attitudes and values (Binkley et al., 2012; OECD, 2019; Rigby et al., 2009). Thus, future competences are much more than just a set of general skills or pieces of knowledge. They are generic and interdisciplinary in the sense that they are acquired and supported across the boundaries of various disciplines and domains of knowledge (Barrie, 2012; OECD, 2019). This does not imply that they are necessarily independent of discipline-related knowledge, skills and dispositions but rather closely intertwined with them (Barrie, 2012; Hyytinen et al., 2019).

Future competences have been approached from various perspectives. A large body of literature has been published on this topic along with a movement towards a globalized educational reform (Wang et al., 2019). Despite the growing interest, there seems to be no common agreement on what constitutes these generic abilities (Treleaven & Voola, 2008).

Future competences are often outlined and defined by their relevance in terms of employability and subsequent success on the labor market, as noted by Rigby et al. (2009) as well as Treleaven and Voola (2008). Research has shown that employers place high importance on these generic competences when hiring graduate students (e.g., Finch et al., 2013). Research on how to prepare students for future employment has been done in close cooperation with academics and working-life representatives. One example of this is the research project Assessing and Teaching of 21st Century Skills (ATC21S), implemented between 2009 and 2012 and sponsored by Microsoft, Cisco and Intel. Based on an extensive international analysis of twenty-first century skills frameworks, the project resulted in a model for describing and assessing the skills needed in the twenty-first century (Binkley et al., 2012).

Increasingly, the competences relevant for the future are being perceived and described in a broad, universal sense. They are targeted to facilitate individual development not only for professional purposes but also for societal needs; future competences are needed to be active, committed members of society (Rigby et al., 2009). These wider perspectives have also been addressed in a number of projects and frameworks initiated by larger international forums, such as the Organisation for Economic Co-operation and Development (OECD), the European Union (EU) and The United Nations Educational, Scientific and Cultural Organization (UNESCO). In addition, several countries and national institutions have launched their own policies with similar emphases (Wang et al., 2019).

Many frameworks and studies address abilities such as critical thinking, problem solving, collaboration and communication, ICT literacy, creativity and learning literacy (Van Laar et al., 2017; Voogt & Pareja Roblin, 2012; Wang et al., 2019). In addition, qualities related to social and cultural awareness (Van Laar et al., 2017; Voogt & Pareja Roblin, 2012) as well as ethical awareness (Van Laar et al., 2017) are regarded as important. The competences selected for the frameworks often vary in terms of their definition, amount or subset (Van Laar et al., 2020). There are also differences influenced by cultural context (Wang et al., 2019). Many of the frameworks suggest that educational institutions should place a stronger focus on the integration of future competences across curriculums and to ensure sufficient space for them, as they are complex and cross-disciplinary in their nature (Voogt & Pareja Roblin, 2012).

Dimensions of future competences

Future competences have also been a target of recent research interest by Ehlers (2020) and Ehlers and Kellermann (2019). Based on extensive international research data, they have outlined a new type of model with a tripartite structure for identifying and explaining future competences. In this model, future competences are considered to express a relation to someone or something, and they are categorized into three dimensions in terms of their relation to: subject development (individual to him/herself), object (individual to a certain object or task), or social environment (individual to another person, group, or organization in the world).

Ehlers (2020) has recently developed and refined the model further, referring to it as a Triple Helix-Model of Future Skills. The model contains a total of 17 competence profiles which are clustered in terms of their relation within these dimensions. The main term used in the model is ‘Future Skills’ and this concept is considered to hold the same meaning as the term ‘future competences’ utilized in this study. An overview of the Triple Helix-Model is presented in Fig. 1.

Fig. 1

An overview of the Triple Helix-Model. This figure demonstrates the dimensions of future competences presented in the Triple Helix-Model. It is adapted from Future Skills—Future Learning and Future Higher Education (p. 42) by U-D. Ehlers (2020). Copyright 2020 by U-D. Ehlers. Adapted with permission (CC BY-NC-SA 4.0)

As presented in the model, the future competences attached to the subject development dimension are: learning literacy, self-efficacy, self-determination, self-competence, reflective competence, decision competence, initiative and performance competence, ambiguity competence and ethical competence. The object-related dimension consists of the following competences: design-thinking competence, innovation competence, systems competence and digital literacy. Finally, the competences related to the social environment dimension are: sense-making, future and design competence, cooperation competence and communication competence.

The conceptual background and dimensional structure of the Triple Helix-Model are the foundation for this study. The model requires researchers and practitioners to engage with the sets of future competences and their relations from multiple perspectives. Being rooted in internationally conducted research and developed for HE purposes the model provides a purposeful scope for identifying future competences relevant for HE students and ways to support them through educational approaches such as LA.

Learning analytics as a way to support the development of future competences

Several studies have indicated that traditional ways of teaching and learning are not effective to promote the development of future competences (Binkley et al., 2012; Ehlers, 2020; Ehlers & Eigbrecht, 2020; Virtanen & Tynjälä, 2019; Voogt & Pareja Roblin, 2012). Instead, educational activities need to be ingenious and creative and in line with wider educational goals and assessment practices (Ehlers, 2020).

The potential of LA has been a focus of growing interest in the search for new ways to support future competence orientation (Buckingham Shum & Deakin Crick, 2016; Joksimović et al., 2020). In particular, LA is considered promising for measuring future competences as it harnesses the potential of large datasets (Gašević, 2019) to track learners’ behavior, visualize patterns and provide appropriate feedback (Buckingham Shum & Deakin Crick, 2016). It draws on various data sources that can cover a variety of technologies as well as institutional environments, such as student information systems (SISs) and learning management systems (LMSs) (Gašević et al., 2014).

However, using LA to measure the development and learning progression of future competences is not without challenges (Hershkovitz et al., 2016). As pointed out by Buckingham Shum and Deakin Crick (2016), researchers and practitioners are forced to answer questions such as: how to strengthen the development of these competences without simplifying them into meaningless statistics? How to capture and make sense of data related to complex learning processes and outcomes, comprising values and attitudes as well as dispositions, narratives and identities? How to establish significant links “from clicks to constructs” in a responsible and ethical manner?

It has become clear that the assessment and development of future competences through LA is undoubtedly a very complex issue requiring thoughtful and deliberate exploration (Joksimović et al., 2020). It seems that the use of LA for these kinds of purposes is still in the early stages of development in HE and the most promising approaches relate to the use of LA “for assessment for learning, rather than assessment of learning” (Gašević, 2019, p. 49). However, the number of studies exploring the relation between future competences and LA from different perspectives is continuously growing, contributing to the development of this topical field of education (Joksimović et al., 2020).

A recent study conducted by Karaoglan Yılmaz (2021) indicated that university students’ academic self-efficacy and problem-solving skills can be improved by sending students feedback about LA results and providing them appropriate advice and guidance based on these results. Furthermore, LA-based feedback approaches appear to contribute to university students’ community of inquiry, that is, their perceptions of cognitive, social and teaching presence, as well as their reflective thinking skills (Yılmaz, 2020).

A study conducted by Silva et al. (2018) demonstrated that LA can be used to promote self-regulated learning in flipped classrooms and to assist in identifying strategies that can promote students’ academic performance. Kovanović et al. (2018) studied the use of LA for assessing student reflection. They examined what kinds of linguistic indicators of self-reflection can be captured from student writings and how they can be used to create an automated system for assessing students’ self-reflection. The results of this approach seemed promising.

In recent years, collaboration analytics (see Schneider et al., 2021) has also been a target of research interest for many. These studies have, for example, explored how LA tools can support learning collaboration literacy (Worsley, 2021), and facilitate collaborative learning and teamwork through a conceptual model of collaboration analytics (Martinez-Maldonado et al., 2021). An emphasis has also been placed on examining the potential of LA in terms of collaborative problem-solving (CPS). Studies have evidenced that LA can be utilized to analyze relationships between learner roles, CPS skills and outcomes (Dowell et al., 2020), to model the process of CPS (Cukurova et al., 2020) and to detect the factors that can affect learning performance and CPS awareness (Chen et al., 2019). In addition, LA can be used as a supportive tool for evaluating and monitoring individual progress within teamwork (Fidalgo-Blanco et al., 2015).

The above studies are only a snapshot of the diverse research conducted in this field, reflecting some of the topics of interest to scholars in recent years. While there is a large body of research showing promising results of utilizing LA in developing future competences, some concerns have also been raised. Studies have indicated that there is only little evidence of improvement in learning outcomes (Viberg et al., 2018) or active engagement with students (Tsai et al., 2020). In addition, utilizing LA to develop self-regulated learning skills is often overlooked (Tsai et al., 2020). The perceived potential of LA seems to be greater than its actual application (Viberg et al., 2018). In addition, there are concerns related to LA’s weak theoretical and pedagogical rationale for learning (Tsai et al., 2020). LA holds the promise of renewing education in profound ways but to reach its full potential it must be connected to learning sciences and grounded in pedagogical reasonings, as pointed out by Gašević et al. (2017), Jivet et al., (2017, 2018), Nunn et al. (2016) and Tsai et al. (2020).

Research questions

This qualitative study aims to provide novel perspectives and insights into the importance of future competences and the ways these competences could be supported through LA. It places a special emphasis on the views of HE teaching professionals which are explored on the basis of the following dimensions of future competences, that is: subject development (individual to him/herself), object (individual to a certain object or task), or social environment (individual to another person, a group, or an organization in the world). The specific research questions are as follows:

• What are the future competencies considered important for HE students to acquire during HE studies from the perspectives of the subject development, object and social environment dimensions?

• How could LA support the development of important future competences from the perspectives of subject development, object and social environment dimensions?

Methods

Research context

This study was conducted in four Finnish universities of applied sciences (UASs). The Finnish HE system comprises two kinds of HEIs: universities that are research driven and UASs that are professionally oriented, operating closely with working-life and advancing regional development. Both of them offer bachelor’s and master's degrees as well as professional specialization programmes, open studies, continuing professional education and teacher education. Research-intensive universities also provide third-cycle postgraduate degrees.

The study utilized a case-based approach (see Cohen et al., 2005; Denscombe, 2011) which aims to explore existing research phenomena in-depth and through certain instances, and not through a wide spectrum (Denscombe, 2011). The four selected UASs were members of a specific project network which aimed to promote the adoption of LA in HEIs and to provide recommendations for the effective utilization of LA.

Participants and LA pilots

A total of 19 study participants, six males and 13 females, were involved in the study (N = 19). They were teaching professionals (teachers, educational developers, pedagogical experts) working at the selected UASs under different job titles and in a variety of disciplines. The study participants operated at different degree levels, programmes and units in the UASs (bachelor’s, master’s, professional teacher education). Some of them were also involved with guidance and tutoring practices and some of them participated in research, development and innovation activities. This diversity of positions and tasks was considered to strengthen the validity of the study and to bring a variety of views to the discussion.

All the study participants were involved in the project network described above and thus familiar with LA to some extent. They were engaged in designing and developing LA pilots in the selected UASs with the aim of increasing understanding and gaining experiences of the use of LA. However, these pilots were at different phases of design, implementation and evaluation during the interviews.

Some study participants utilized LA data generated from a single course whereas others focused more on data obtained from wider study entities. Thus, the focus of development was on micro-level LA approaches leveraging data primarily for individual (teachers, tutors, students) or group needs rather than for institutional (meso-level) or national (macro-level) purposes (see Buckingham Shum, 2012). The use of LA was incorporated into the learning designs and aligned with the pedagogical approaches.

Many of the study participants utilized descriptive, built-in reporting and visualization tools provided by LMSs and/or integrated systems. These tools, considered here as LA applications, were used to provide insights into students’ study behavior and progress by harnessing user activity and interaction data stored in the systems, such as visits to the online course platform, contents accessed, tools used, posts made and sent in discussion forums, time spent on activities, completion of assignments and grades achieved (Macfadyen & Dawson, 2010). In addition, they indicated student attendance in classes as well as performance levels, such as achievement of learning objectives. This type of tracking data was commonly aggregated into reports and visualized through dashboards, which according to Schwendimann et al. (2017), refer to displays that accumulate information about learner(s), their learning process(es) and/or learning context(s).

The specific ways of utilizing these LA applications and data by the study participants was based on the goals of each pilot but in general, the overall purpose was to monitor student learning and study progress in one or multiple courses. Some participants aimed to identify especially students who were struggling or at risk of dropping out but also students who were performing well, and to provide them with timely support for learning. Furthermore, some desired to increase their knowledge about students, their characteristics and competences to inform their teaching and guidance as well as learning designs and to enable individualized learning. Not only were the teachers informed about student performance and development in detail, but the students themselves were also provided with relevant LA data and dashboards to increase their knowledge about their learning and study progress.

LA was also harnessed to generate automated student feedback based on data capturing student activity and interaction in online learning platforms. To promote study engagement, the students were sent automatic, customized feedback, for example, on their performance on course assignments or study progress. In addition, automated feedback was used to encourage students to engage in their studies if they had not visited the online learning environment actively or to inquire if further support was needed. In some cases, LA was also used to provide tools for assessing and grading students by analyzing log data on their activity and performance on online learning platforms.

Some attempts to combine SIS and LMS data into visual displays were also made to provide insights into student performance but these were aggregated manually. To enrich the information achieved through LMS tracking data, some study participants utilized students’ self-report surveys (e.g., self-/peer assessments) and appropriate visualizations to make learning dispositions and self-perceptions of one’s competence development more visible. Although these different types of data, LMS, SIS and self-report data, were not yet automatically combined, analyzed or displayed together, their potential should be further explored. This type of dispositional LA (see Buckingham Shum & Deakin Crick, 2012) seems to provide many new opportunities for understanding student learning and predicting performance (Tempelaar et al., 2017).

These pilot examples show that most of the LA applications used by the study participants were relatively simple and the potential of more advanced and sophisticated LA applications and techniques was not yet deployed. Despite this, the design and implementation of LA pilots offered the teaching professionals and students a chance to take the first steps in leveraging LA as part of teaching and learning in HE. It inspired many to take advantage of LA more widely in the future.

Group interviews

The qualitative data for this study was collected from group interviews carried out between November 2019 and March 2020. Group interviews aim to reach both the individual's own interpretations and the collectively shared meanings of the themes under reflection (Denscombe, 2011; Flick, 2014; Puusa, 2020a). The interview groups comprised 4–6 persons per UAS. One of the authors and researchers of this study also had an interviewee role in one group interview, sharing personal experiences of piloting LA as a part of course activities, considered useful in terms of reflecting on the research topic.

Preliminary interview themes and questions along with a summary of Ehlers and Kellermann’s (2019) model of future competences and its dimensional approaches were sent to the study participants by e-mail prior to the interviews. The purpose was not to limit reflection and discussion to the competences mentioned in this model but rather to orientate the study participants to the research topic and promote fruitful, reflective discussion. The most recent version of this model, the Triple Helix-Model, was not available at the time of the interviews.

The interviews were semi-structured and the predefined questions were applied in a flexible way. As acknowledged in the literature (Cohen et al., 2005; Denscombe, 2011; Flick, 2014; Hyvärinen, 2017; Puusa, 2020a), this left space for issues raised by the study participants and current circumstances. The first group interview functioned as a trial interview with an introduction to the topic. After that, the researchers further clarified the interview structure and the interview questions thus increasing the validity of the study.

Three group interviews were conducted mainly on campus, at the premises of the specific UAS. One of them, however, was carried out online, synchronously through Microsoft Teams. Online tools were also utilized in connection with on-campus interviews as one interviewer participated through Microsoft Skype. One group interview was later supplemented by a phone interview as a study participant was unable to participate in the group interview for its entire duration.

The length of the group interviews varied from 91 to 124 min and the supplementary individual interview took 45 min. The interviews were recorded and converted to a verbatim transcription. A total of 119 pages of transcribed interview data (single spacing, 12-point font) were accumulated. Relevant parts were utilized and analyzed to provide answers to the research questions addressed in this paper.

Qualitative content analysis

Qualitative content analysis (see Puusa, 2020b; Schreier, 2012) was employed in analyzing and interpreting the interview data. First, the appropriate unit of analysis was chosen based on the research questions. The researchers repeatedly read and reflected on the interview data with the aim of outlining the entity and identifying relevant constituent elements, as suggested by Cohen et al. (2005) and Puusa (2020b). The interview data was thematized and the units of analysis placed into the respective categories according to the dimensions of future competences, that is, subject development, object and social environment. This phase was followed by supplementing the analysis with narratives describing the interview content within these categories, as advocated in literature (Cohen et al., 2005). The researchers collaborated closely during the research process, compared and reflected on the findings to reach mutual understanding. According to Denscombe (2011), this strengthens the consistency of analysis and interpretation.

As a result, the future competences considered important for HE students to acquire as well as the ways LA were thought to support their development were identified and specified within the abovementioned dimensional categorization. Competences were grouped together when close connections were found between them. The iterative analysis and interpretation of the data also continued during the writing process.

The study findings were supplemented with the authentic voices of the interviewees to make the basis of interpretations more visible and to highlight the richness of data, as suggested by Denscombe (2011) and Puusa (2020b). The excerpts selected to illustrate the findings were translated from Finnish to English. When necessary, the authors clarified excerpts with insertions inside brackets []. Aliases and UAS codes (A, B, C, D) replaced the real names of the study participants and the UASs they represented.

The study results are reported and the research questions answered simultaneously in more detail in the following section by using the tripartite structure of the future competence dimensions (sub-sections A, B and C). The main results are gathered and summarized at the end of the Results section (sub-section D).

Results

Subject development-related competences and using LA to support their development

The process of subject development, that is, the emergence of a relationship to oneself, places a focus on individual, personal abilities to learn, develop and react to circumstances concerning oneself (Ehlers, 2020; Ehlers & Kellermann, 2019). The study data revealed four sets of future competences considered important and associated with the subject development dimension: reflective competence, self-awareness and self-management, learning literacy, and personal agency and self-efficacy. They were closely related and overlapped with each other. The study participants also identified multiple ways in which LA could be used to support the development of these sets of competence.

Reflective competence

The study participants identified the ability to reflect continuously and actively as one of the most significant future competences, having a close relation to the other competences as well. This is in line with the views expressed by Moon (2006). It was considered essential that HE students analyze their actions and experiences for the purpose of continuous learning and further development. Reflection, as an intellectual and affective activity, engages individuals in exploration of their experiences to reach a new understanding and appreciation (Boud et al., 1985). It has an important role in contributing to high-quality learning as well as appropriate learning behavior (Moon, 2006).

LA was considered an efficient way through which students can be encouraged to strengthen their reflective competence. While LA can be harnessed to compile, analyze and visualize massive amounts of data impractical to handle manually (Ferguson et al., 2016), it can also provide visible insights into the students’ cumulative learning processes and competence development as well as ignite purposeful reflection. Similar arguments have also been made by Chatti et al. (2012). Thus, the data in itself does not lead to learning but requires active, intentional reflection on it for learning to occur.

Some study participants saw the potential of LA relating especially to the assessment of and reflection on generic future competences, which according to literature (e.g., Muukkonen et al., 2020), are still rather imperceptible and subsidiary areas in HE teaching and learning. LA could strengthen their role and make them a more visible part of HEIs’ educational activities. Promoting students’ reflective competence through LA was considered to necessitate its constructive alignment (see Biggs, 2014) with learning objectives, pieced into purposeful parts, as well as learning activities and formative assessment practices.

However, in terms of reflection, the study participants also had some concerns which related especially to information overload and users’ insufficient abilities to interpret LA data by themselves. Also, aspects regarding the depth of reflection (Moon, 2006) were raised. Students were not viewed as a homogenous group needing the same amount of data or the same type of reflection as a basis for their learning and development but rather, the basis depended on an individual student’s needs and preferences. Thus, in line with the views of Chatti et al. (2012), it is important to create individualized approaches for presenting LA data that do not cause information overload but build upon students’ existing knowledge and practices.

In tackling the challenges of using LA as a reflection trigger, the value of collective approaches was emphasized. Joint reflection, according to Boud et al. (1985), refers to continuous discussion, interaction and communication with others. Educational staff and experts in various HEI fields could act as relevant co-reflectors with students by considering different background aspects and emotions influencing data analyses, helping to make ethically sustainable interpretations and providing appropriate support when necessary. This was how one study participant reflected on this matter: “…there is a need for a gatekeeper, the right person, who can really do these multi-perspective analyses of a situation…” (Alice, UAS D). In principle, it was emphasized that students should not be left alone with the data as it may increase the risk of misunderstanding and misinterpretation. This must be kept in mind especially when dealing with multidimensional and sensitive data which often calls for careful consideration and deeper reflection. As stated by Bulpitt and Martin (2005), the process of reflection is often much more than just rational thinking; it also involves emotions, feelings and instincts.

Self-awareness and self-management

According to the study participants, HE students must develop profound self-awareness of their existing competence levels for meeting future requirements. Not only do they need abilities to direct their competence development in a goal and future-oriented manner, but also qualities related to responsibility, time management and an entrepreneurial work approach, grouped here into the concept of self-management. The importance of creating a profound understanding of one’s own competence areas as well as managing oneself purposefully has also been recognized in many of the frameworks of future competences (e.g., Binkley et al., 2012; Ehlers, 2020).

By making the process of learning and competence development more visible, LA was considered to promote students’ self-awareness of their strengths and weaknesses, and to facilitate the taking of actions to meet possible development needs. However, this does not happen without reflection and understanding, as one of the study participants pointed out:

“Yeah, probably the level of awareness, that you are able to do something, [requires that] you have to first understand that perhaps I have something to improve here, or [that] this may be my weaker point. Or, [if] this is something I have to familiarize myself with, I must understand it first. And [only] then is it possible to act upon it. And then, of course, it may take some time before it can be done, but either way, it is the awareness that analytics could and does promote.” (Peter, UAS B)

Jivet et al. (2017) rightly argue that although utilization of LA data is often designed to foster awareness, being aware does not guarantee that necessary actions are taken to facilitate learning and intended outcomes are achieved. As they suggest, students should be encouraged to take subsequent steps, such as setting goals and tracking one’s own progress. That is, “awareness is not enough” and LA should be integrated into pedagogics which catalyzes the development of competences at a more comprehensive level (Jivet et al., 2017, p. 82).

At its best, LA could bring novel means for promoting goal-orientation. Students need new ways of directing their future actions according to individual needs and goals but also according to the learning objectives set in the study program. LA was seen to support students to recognize what kind of future professional they want (or are expected) to become, and how they can accomplish this. In addition, LA can provide tools for prioritizing the focuses of development between different competences. As pointed out by Sedrakyan et al. (2020), a student may want to spend more time on practicing a topic in which he/she lacks prior knowledge and spend less time on focusing on areas in which he/she has previous knowhow.

However, there is very little evidence of emerging LA approaches that would support goal setting and planning in educational settings as current practice primarily focuses on triggering reflection and supporting awareness (Jivet et al., 2017). As emphasized by Jivet et al. (2017), an increasing emphasis should be placed on designing LA to guide students with different performance levels, needs and motivating factors.

The study participants highlighted especially the role of feedback as a driving force for self-management and goal-orientation. At its best, it is “an interactive process in which the output or effect of an action is returned (fed back) to modify the next action toward reaching a goal” (Sedrakyan et al., 2020). As one study participant pointed out, “so that you can develop, you need feedback” (Mary, UAS C). While the utilization of LA was seen to widen the possibilities of giving and receiving feedback, the process calls for careful, ethical consideration and integrity. As stated by one study participant, “it’s a very sensitive issue of what kind of automatic message you send to a student if he/she isn’t within the average there, there in the stream” (Clare, UAS D). The feedback should not be too controlling or harsh, but rather transparent, well-timed and bring forth development and self-directed learning. To reach its fullest potential, feedback connected to LA data was thought to always require a student’s own consent and wish to receive it.

These are important considerations as a potential risk of LA is the disempowerment of students by increasing their dependence on institutional, continuous feedback (Buckingham Shum & Ferguson, 2012). The feedback connected with LA should be formative in nature (Buckingham Shum & Ferguson, 2012) and originate primarily in educational research on the learning process, founded on the regulatory mechanisms underlying the process, as well as on an awareness of the students’ learning goals to provide cognitive support and guide the learning process purposefully (Sedrakyan et al., 2020).

Learning literacy

Closely intertwined with the above competences related to reflection, self-awareness and self-management, the third set of competence highlighted as a prerequisite for coping with future requirements was students’ ability to learn, monitor and control their own learning process and competence development, and to take responsibility as learners. This entity could be referred to as a certain kind of learning literacy, a concept also used by Ehlers (2020). It has close connections to self-directed learning (SDL), that is, learners controlling their learning process by themselves (Durall & Gros, 2014), as well as to self-regulated learning (SRL) which highlights the learners’ role in initiating and directing their efforts to obtain knowledge and skills instead of relying just on teachers or other instructors (Zimmerman, 1989).

The utilization of LA offers means to monitor learning processes and promote the development of metacognitive skills, as also acknowledged by Durall and Gros (2014). In particular, it was seen to illustrate the connections between learning actions and learning outcomes. LA was thought to provide insights into whether the decisions and contributions made to learning have been productive and in line with the goals set by different stakeholders (students, teachers, management, policy leaders). In addition, it could indicate whether moderations are needed. These views are in line with the notions of Charleer et al. (2016) which state that empowering students to reason the process from effort to intended learning outcome with the help of LA is beneficial in terms of metacognition.

The study participants also mentioned that LA data could be harnessed to give insights into individual learning preferences. In addition, it enables monitoring study behavior, such as time spent on different learning activities. While these time-on-task measures have been widely studied and used for building predictive models of student learning, their actual value is still questionable (Kovanović et al., 2015). At the very least, they need further investigation, and caution and the integration of additional measures in their use (Kovanović et al., 2015).

In terms of learning literacy, the study participants emphasized its ever-developing nature. It is a competence in which they are never ready but are rather continuous learners. One participant pointed out that LA should act as “a measure of lifelong learning”; its potential could be exploited to support continuous competence development. It is essential to practice learning literacy and promote it through diverse pedagogical practices and technological tools. However, aligning LA with learning theories such as SRL is only just emerging and further research in revealing how it could benefit the process of learning in the best possible way is urgently needed (Marzouk et al., 2016).

Personal agency and self-efficacy

Some study participants raised the issue of the increasing need of young people today for encouragement and activation as well as for building faith in one’s abilities and strengths. The fast changes in HE and working-life settings were thought to require competences relying on a student’s personal agency and self-efficacy. Personal agency in an educational setting refers to an individual’s ability to influence one’s own studies, work actively and make use of the personal, relational and participatory resources available in learning environments (Jääskelä et al., 2017). Self-efficacy, in turn, relates to an individual’s belief and confidence in one’s own abilities to successfully perform assigned tasks and take responsibility for one’s own decisions (Ehlers, 2020; see Bandura, 1982).

In recent years, especially the connections between LA and agency have been the target of growing interest with diverse focuses and approaches. Some researchers (e.g., Jääskelä et al., 2020) have focused on developing analytic tools for a deeper examination of the phenomenon of agency itself whereas others (e.g., Prinsloo & Slade, 2016) have studied how LA should be developed to empower students as active agents and users of LA data, that is, not just be objects of quantified data.

In this study, LA was generally thought to promote students’ active engagement in their studies. In particular, it was seen as a useful tool to better identify, reach and support students who are not active in their studies and/or may feel disengaged: “…Perhaps, we should reach out to those [students] who are not active… Learning analytics could give us means to go a little deeper into what a student experiencing that kind of disengagement really needs…” (Sarah, UAS C). LA as an integrated element of the entire study path was thought to provide opportunities to examine the dynamic nature of agency and to monitor its development at different stages of studies.

In addition, LA was seen to support the development of students’ self-efficacy by better highlighting learning progress and competence development, for example in situations where students may feel unable to learn. It could provide more realistic, evidence-based information about the learning processes for students, rather than just relying on subjective perceptions. This was how one study participant described this:

“…One's own perception of one’s actions may differ from reality. If one could get evidence of that reality and be guided to assess it, like hey, how does it look now, is this correct or not, or how would you evaluate it yourself…” (Christina, UAS C)

In addition, gamification elements in which learners move from one level to another, were seen to promote student engagement and a positive drive towards studying and learning. Yamada et al. (2017) suggest that blending LA with educational psychology methods enables various viewpoints to be placed on educational evaluation.

Object-related competences and using LA to support their development

Object-related competences refer to an individual’s ability to act in unknown future environments, in relation to objects, work tasks, themes, subject matter and/or problems (Ehlers, 2020; Ehlers & Kellermann, 2019). The two sets of competences mentioned by the study participants from this perspective were: changeability and innovation competence, and digital competence. The potential of LA in facilitating their development was, however, perceived more clearly in the latter set only.

Changeability and innovation competence

According to the study participants, HEIs should challenge their students to work efficiently in ever-changing environments and with unforeseen tasks. Students should be prepared to tolerate uncertainty. One participant described it as follows: “…Not all situations can be prepared for here at school, you have to be creative and find those solutions and put up with the pressure…” (Peter, UAS B). Students were considered to need a certain kind of changeability, an ability to embrace change, facing and adapting to it, but also participating actively in shaping the future and its work. They must be agile in fast-changing settings. In addition, students should have a positive attitude and belief in the future and act according to the values considered essential (e.g., sustainability). To seize these changes, it was considered crucial for students to develop abilities related to innovation, creativity and problem-solving, grouped here as innovation competence. Although the study participants placed a strong emphasis on supporting students’ ability to embrace change and foster innovations already during their studies, LA as a concrete way to support this kind of competence orientation was not particularly referred to in the discussions.

Digital competence

The study participants stressed that the emerging future calls for experts who have sufficient digital competence in various domains. In recent years, there have been several attempts to identify and define what it means to be digitally competent (see Ilomäki et al., 2016; Kampylis et al., 2015; Spante et al., 2018). Ilomäki et al. (2016) state that it is much more than just technical competence but also an ability to utilize different digital technologies meaningfully for working, studying and living as well as an ability to assess them critically.

The participants in this study raised similar views and highlighted the value of digital problem-solving, management of technical systems, digital literacy and selection and segregation of information. A wider understanding of digital phenomena was emphasized which, according to Ilomäki et al. (2016), necessitates making sense of issues related to ethical aspects as well as limitations, challenges and critical use of technologies among other things. This was how one study participant described what is required of a HE student:

“You need to be able to solve problems in a digital environment. You need to be able to solve technical problems but also, if some visualization looks incorrect, you need to understand it… In my opinion, it is a necessity to master digital problem solving and understand digital systems.” (Mary, UAS C)

Thus, digital competence is utilizing the various opportunities offered by the digitalized world and finding one’s own way to connect to it. Ilomäki et al. (2016) state that digital competence is expected to develop in meaningful, long-term settings which highlight problem-orientation and the utilization of various technological tools in integrative ways. In these contexts, the potential of LA could be widely exploited, as a supportive tool for learning but also as a particular topic of learning. The study participants stressed that it is becoming increasingly important for HE students to utilize data collected from various sources, make sense of it and also use it proactively. This was considered even vital to meet the competence requirements of the future and its role as a part of comprehensive digital competence is thus to be highlighted. LA should increasingly be seen as a part of the comprehensive services and learning experiences that HEIs offer their students.

Social environment-related competences and using LA to support their development

Some views presented by the study participants had a clear relation to the social environment dimension of future competences which refers to an individual’s self-organized abilities to act in relation to his/her social settings and organizational contexts (Ehlers, 2020; Ehlers & Kellermann, 2019). From this perspective, the following two sets of competences stood out from the study data: cooperation and communication competence, and developmental mindset. However, the study participants’ references to the potential of LA in these competences was limited to a general level.

Cooperation and communication competence

The study participants mentioned that HE students are increasingly expected to involve themselves in communal knowledge construction and multidisciplinary project work during their studies, as these represent common ways of working in emerging working-life. It was considered important that students develop abilities which enable them to cooperate and communicate in national and international contexts as well as in multidisciplinary interactions. Finding ways to exploit a wide range of digital opportunities and networks as resources of social and organization-related competence development was also highlighted, in addition to investing in language proficiency. One study participant also mentioned the ability to mentalize as an important quality in the future, that is, an ability to be aware of not only one’s own thoughts and feelings but also another’s mind and well-being. It is about taking care of each other.

According to De Laat and Prinsen (2014), learning nowadays is strongly focused around social engagement and interaction and the potential of LA should be increasingly harnessed to monitor, analyze and visualize the students’ social learning behaviors and patterns. Similar views were raised in this study with participants suggesting that this type of analytics, also referred to in the literature as social learning analytics (SLA; Buckingham Shum & Ferguson, 2012) or collaboration analytics (e.g., Anaya et al., 2016; Martinez-Maldonado et al., 2021) could be developed for formulating purposeful groups with a specific focus. At its best, it could support students to make informed choices about where, when and with whom to participate (De Laat & Prinsen, 2014).

In line with De Laat and Prinsen (2014), the study participants emphasized that LA data harnessed to analyze and visualize social relations should increasingly be placed under the eyes of learners themselves to increase their awareness of the social aspects of learning.

“…When I look at mydata, it shows me, for example, my social network, with whom I have been most active last week, who is in my immediate zone, who is in the next zone, who is in the third zone. So, I think learning analytics, in order to support these meta skills, should be able to visualize different things, it should be able to show me my social network, for example, [which] no longer takes place within one single learning platform…” (Mary, UAS C)

However, as a relatively new phenomenon, utilizing LA for these kinds of purposes also raises some concerns (De Laat & Prinsen, 2014). One of the challenges identified by the study participants relates to the ethical utilization of data, an issue also addressed by Slade and Prinsloo (2013). The interpretation and sense-making of data should not be left only to machines but be supported with people promoting flexible intelligence and considering wider social and emotional aspects. Promoting collaboration and communication competence through LA challenges HEIs to consider its use from multiple perspectives in dialogue with different stakeholders and to ground it profoundly in social learning theories.

Developmental mindset

The study participants emphasized that not only should students develop their own individual abilities but increasingly also the joint competence of the community in which they work and/or study, referred to here as developmental mindset. This kind of communal development approach could be facilitated through novel formats of support such as coaching.

“…We should have a kind of coaching culture to this teaching, that the student is being coached [and] is able to follow the development and get feedback from it. The coach [could] give tips on what you have to do to in order to succeed in that matter.” (John, UAS B)

LA was considered promising in the promotion of this type of coaching relationship and communal development process between the student and the teacher. At its best, it could strengthen the student’s engagement in the process of competence development.

As stated by Ehlers (2020), the development focus in the field of HE is shifting to forms of learning and support which are characterized by active accompaniment rather than implementing classical instruction. He argues that future competences cannot be acquired or developed cognitively only. The process requires a more holistic approach, such as strengthening professionalism and enhancing personality development. Incorporating LA into these types of working-life imitating practices and networked environments would create a solid foundation for both individual and communal learning experiences.

Summary of the study results

The future competences identified as important from the dimensional perspectives as well as their relations with the potential of learning analytics are summarized in Fig. 2 below.

Fig. 2

Supporting the development of future competences through learning analytics

The majority of competences highlighted in this study were strongly related to the subject development dimension of future competences. It seems that the foundation for comprehensive competence development is strongly built upon an individual’s subjective abilities to learn and develop when facing future challenges, as also noted by Ehlers (2020) as well as Ehlers and Eigbrecht (2020). Similarly, the greatest perceived potential of LA for supporting the development of future competences appears to lie in this particular dimension. Reflection, in particular, seems to be an overlapping element integrating widely into the competences of this dimension as well as to the potential uses of LA. As stated by Moon (2006), reflection contributes to the process which targets good quality learning.

However, there was clearly less emphasis placed on the object-related competences as well as the possibilities LA has to offer in promoting their development. An individual’s abilities related to objects and/or tasks were referred to at some level but not highlighted to a great extent. Furthermore, the potential of LA within this dimension remained largely marginal and was perceived only as a part of wider digital competence, but not as a facilitator of changeability and innovation competence. The possible uses of LA when working with a specific task or when embracing change are still rather unexplored, with unidentified areas waiting to be addressed in more detail in the future. It would be worthwhile to consider how the use of LA data could be harnessed during HE studies to gain better insights into ways of working, into different scenarios and possible outcomes, or into the basis of decision making. At its best, LA data could be tailored for different settings, such as formal education, informal learning and workplace training (Ferguson et al., 2016).

Correspondingly, the social aspects related to future competences were recognized by the study participants at a rather generic level. While considered important, they were not analyzed or specified in much detail. Some preliminary ideas on how the tools of LA could be harnessed to capture social interactions were presented, but largely without more precise concreteness. This is likely due to the fact that using LA for social and communal development purposes is just beginning to emerge. SLA tools that build on social mobility on a large scale and support students to be better aware of productive social connectivity are still largely non-existent (De Laat & Prinsen, 2014).

The future competences addressed in this study were closely interrelated and interdependent while interacting with one another within a specific dimension and also between dimensions. This was also acknowledged by Ehlers (2020, p. 54) who described them as constantly evolving—not in “a binary state of either-or”. Developed as a coherent and comprehensive whole, these competences form a meaningful basis for informing student action in emerging authentic settings while also opening up new possibilities for exploiting the potential of LA to its fullest. At its best, LA benefits and evolves with its efforts to advance future competences, and vice versa.

Discussion

The competences considered important in this study were very much in line with many of the current frameworks and models for future competences, especially the Triple Helix-Model. In this study, the future competences were grouped, categorized and named slightly differently to the abovementioned model to better describe the views of the study participants, but they reflected rather similar aspects.

Several competences were presented similarly in both: reflective competence, learning literacy, self-efficacy, innovation competence, cooperation competence and communication competence. Digital competence in this study was referred to as digital literacy in the Triple Helix-Model.

Self-awareness and self-management in this study had a close relationship to the Triple Helix Model’s self-determination, self-competence, decision competence as well as initiative and performance competence. Exercising personal agency was included in the Model, but in this study, it was a discrete competence. The ability to embrace change, termed changeability in this study, converged with the Triple Helix-Model’s ambiguity competence, design-thinking competence, and future and design competence. A developmental mindset found to be important in this study had similarities with the future and design competence in the Triple Helix-Model, but in this study, it was addressed especially from communal development (social) perspectives.

As the results of this study and several other studies presented in this paper indicate, LA holds a promise to contribute to the advancement of these competences in various ways. Although the potential of LA has not been empirically demonstrated in this study but rather lies in the expectations and aspirations attached to it by the study participants, there are a growing number of studies in which it has been empirically researched (see Dowell et al., 2020; Karaoglan Yılmaz, 2021; Silva et al., 2018; Yılmaz, 2020). Many of the studies explore the use of LA to analyze future competence orientation from the perspective of some particular competence(s) in relatively limited contexts, such as course environments. However, there is a growing need to bring together research and knowledge in this field and build frameworks that integrate the use of LA in competence development as comprehensively as possible. Utilizing LA as a facilitator of future competences should not be limited to a single course but rather be integrated into the entire HE study path.

Despite its perceived benefits, the use of LA does not automatically or as such, lead to more profound competence development or better learning outcomes but instead calls for careful consideration and close dialogical cooperation between different stakeholders. LA can be harnessed to renew HE but only when the emphasis in its development is on aspects of learning, and not on the use of novel, emerging approaches or appealing technology. As pointed out by Ferguson et al., (2016, p. 38), “successful analytics do not begin with a set of data; they begin with an understanding of how people learn”. It is crucial for the HE teaching community to involve themselves not only in the process of working with LA tools but also to construct a rich knowledge base for shaping their use (Wise & Vytasek, 2017). As emphasized by Nunn et al. (2016), the value of LA in promoting future competences is strongly connected to the pedagogy and assessment regime emphasized in educational environments. They argue that if future competences are valued in pedagogical designs and assessment practices, and not discouraged, the tools of LA are also expected to better assist in their promotion and development.

Also, the technological aspects and conditions of data collection need attention. Producing idealized datasets is a considerable challenge for LA since the data is still rather often erroneous or incomplete and affected by the context of data collection and processing (Greller & Draschler, 2012). This is even a greater concern when dealing with data that aims to address the progress of future competences. Technologies should be used and developed in ways that enable collecting and combining rich datasets to increase the value of LA for learners and teachers (Ferguson et al., 2016). In the future, LA is expected to become more multimodal (Hershkovitz et al., 2016) as sophisticated techniques provide insights into authentic learning settings, both digital and physical, by employing various sources of data, such as video, audio, text, logs, biosensors and gestures (Blikstein & Worlsey, 2016).

Limitations

Due to the qualitative and case-based nature of this study, no wider generalizations can or should be drawn on the basis of the study findings. Instead, the results give insights on how the research phenomena was perceived in this specific context and thus, hopefully stimulate thinking among others interested in the same topics. These notions are in line with arguments made by Corbin and Strauss (2015) as well as Denscombe (2011).

From the perspective of validity, it is important to consider possible bias in data (Cohen et al., 2005). Since the focus was especially on teaching professionals who already had some previous knowledge and experience in this field, they may have perceived the potential of LA in supporting the development of future competences higher than those without any or only little experience. For this reason, it must also be noted that the study results cannot represent the views of a wider community of teaching professionals, either. Involving people with different backgrounds and views on the research phenomenon in the group interviews could have facilitated an even more diverse and multi-perspective discussion, as pointed out by Valtonen and Viitanen (2020). In addition, following the views of Denscombe (2011), utilizing method triangulation, that is, supplementing the interview data with other information sources, such as documents and questionnaires could have strengthened the validity of the study.

Conclusions

The emerging future calls for new generic competences. It is essential that teaching professionals, who are largely responsible for creating educational settings and learning conditions for their development, strive for mutual understanding on the kind of future competences needed and how the potential of new educational approaches such as LA can be harnessed to support them.

This qualitative case study adds to the current research by bringing these topical issues under reflection and by exploring their relation in a novel way, from multiple perspectives. The study findings can be used as a basis for strengthening the position of future competences in HE and the utilization of LA in educational processes, such as curriculum development and pedagogical design. The findings hold value also when grounding the use of LA into the aspects of learning science.

It would seem that similar research efforts with the same kind of focuses have not been conducted, at least not on a large scale. In the future, it would be interesting to continue this study by elaborating and grounding the findings in actual HE settings. What kind of pedagogy and technology are needed to capture the relevant LA data in terms of future competences? What kind of data should be collected and from where for further analysis and support? What kind of guidance interventions based on LA data would optimize future competence development?

In future research, it would be worthwhile to involve HE students’ reflections on these issues as this group needs these new abilities and operates in fast-changing settings. They are also the central beneficiaries and users of LA. The voices of teaching professionals and students are important to hear as they have the power to shape the future of HE.