Open Access

The perceptions of the meaning and value of analytics in New Zealand higher education institutions

International Journal of Educational Technology in Higher Education201714:35

https://doi.org/10.1186/s41239-017-0073-y

Received: 11 April 2017

Accepted: 12 September 2017

Published: 27 October 2017

Abstract

This article presents the current perceptions on the value of analytics and their possible contribution to the higher education sector in New Zealand. Seven out of eight research-intensive public universities in New Zealand took part in the study. Participants included senior management and those who have some role associated with decision-making within higher education (N = 82). The study found inconsistent understanding of the meaning of analytics across participants. In particular, three forms of perceptions of analytics were identified: structural; functional and structural-functional. It was evident that some participants viewed analytics in its structural elements such as statistics, metrics, trends, numbers, graph, and any relevant information/data to enhance better decision-making, whereas other participants perceived the notion of analytics in terms of functional aspect; as means to an end, a process to use the data to gain insights and taking action on complex problems, yet a third group viewed analytics from both structural-functional perspectives. These kinds of perceptions have to a larger extent influenced participants’ views on the value of analytics in shaping policy and practice. Also, literature has addressed a number of possible challenges associated with the large-scale institutional implementation of analytics. These challenges were: difficulties in extracting data from multiple databases, maintaining data quality, ethical and privacy issues, and lack of professional development opportunities. This article aims to broadly contribute to a better understanding of current perception and value of analytics in higher education, and in particular within the New Zealand context.

Keywords

AnalyticsHigher educationPerceived valuesChallenges

Introduction and literature

Institutions of higher education are increasingly accumulating a significant amount of data on students, teachers, facilities and curricula (Bichsel, 2012; Siemens & Long, 2011). The increasing deployment of technologies that support learning, teaching and administration contribute to the generation of the massive amount of data (Gantz & Reinsel, 2012). As more data on student activities are increasingly becoming available, institutions of higher education can seize this opportunity to utilize this data to better support student learning. Some institutions are employing analytics to help them identify at-risk students and develop the necessary student support (Scholes, 2016). In the last decade, analytics has become an important area of research within the educational technology area. However, there is still a mixed understanding, with regards to its meaning and contribution for better decision-making in higher education. Analytics is a broad term used to describe mechanisms by which large and diverse quantity of data can be transformed into valuable information. Norris, Baer, and Offerman (2009) describe analytics as a set of processes involving data analysis, measurement, improvement and comparision of individuals, programs, departments, institutions or enterprises, groups of organizations, and the entire industries. This view on analytics emphasizes the functional aspects of analytics, as a set of processes and procedures to accomplish certain outcomes. The utility of analytics requires reducing the complex data into meaningful information (Johnson, Levine, Smith, & Stone, 2010). INFORMS,1 a professional society for operation research and management, defines analytics as ‘a scientific process of transforming data into insights.' This definition is purposely broad to encompass many aspects, including data, processes, and tools. Over the years, the term ‘analytics’ has come to be associated with Data Science, and Big Data. Johnson et al. (2010) stated that analytics could include any form of data (quantitative, qualitative, and mixed). The literature has also mentioned the use of the term on functional grounds (descriptive analytics, predictive analytics, and prescriptive analytics) or even the object of analysis (Twitter analytics, Facebook analytics, and Google Analytics) (Arnold & Pistilli, 2012; Van Barneveld, Arnold, & Campbell, 2012).

In the discussion of Big Data within the context of higher education, Daniel and Butson (2013) present a multidimensional perspective of analytics and offer a conceptual framework with four elements; Institutional Analytics (IA), Information Technology Analytics (ITA), Learning Analytics (LA) and Academic Analytics (AA). They define IA as systems and approaches needed to analyze operational data within an institution with the fundamental purpose of improving the quality of decisions. They view ITA as collection and analysis of data associated with the use of technology, including system analysis (e.g. data warehouse, data standards, tools, processes, organizational synergies, and policies). Moreover, AA are concerned with gaining insights needed to understand information about a particular academic programme (e.g. Early detection systems) (Charlton, Mavrikis, & Katsifli, 2013; Daniel, 2015; Dawson, Bakharia, & Heathcote, 2010; Siemens, 2013; Tulasi, 2013; West, 2012). Campbell and Oblinger (2007) noted that most academic programs are engaging in proactive strategies to customize learning paths of students who are facing academic difficulties. Other researchers view AA as a way of using statistical techniques to help to teach staff to better support student learning (Campbell & Oblinger, 2007). LA is concerned with the measurement, collection, analysis, and reporting of data about learners and their contexts for understanding and optimizing learning and the environments in which it occurs (Jones, 2012; Siemens & Long, 2011). It involves the use of methods to interpret a broad range of data produced by and gathered on behalf of students to model and predict student progress and performance (Johnson et al., 2010). Daniel et al. (2013) characterisation of analytics along four components assumes structural-functional aspects of analytics, with each component a sub-area of the Big Data paradigm in the context of higher education. Arguably, the higher education sector is relatively slow in adopting analytics to improve practice, and as a scholarly area of inquiry, partly because the area is new, but mostly because various stakeholders have a different understanding of what constitutes analytics and the role analytics play in addressing challenges facing higher education.

Perceived value and use of analytics in higher education

Recent work reveals that institutions of higher education are exploring different ways of using analytics to garner useful insights from data to guide planning, interventions, and better decision-making (Nunn, Avella, Kanai, & Kebritchi, 2016). Analytics helps educators to understand how students engage with technology (DiCerbo, 2014; Hung & Zhang, 2012). Analytics can help institutions to assess the quality of teaching and identify support needed to enrich student learning (Badur & Mardikyan, 2011). Further, the use of analytics can help institutions to determine trends in student enrollment and profile shared interests in various disciplines and better predict graduate numbers for long-term planning of recruitment (Althubaiti & Alkhazim, 2014).

Bichsel (2012) reported examples of successful use-case scenarios for analytics across a diverse range of institutions. The first scenario is where analytics can be used to guide the development of proactive and efficient interventions to support students at-risk of failing their program of study. This scenario is the key driver of research and development in the early intervention system deployed to identify students at-risk (Arnold & Pistilli, 2012). For instance, Course Signal as one of the first generations of learning analytics systems provides students with real-time feedback using an intervention system that employs some communication mechanisms—personalized email, a stoplight, and a particular color to indicate student progress (Dietz-Uhler & Hurn, 2013). The second use-case scenario describes the utilization of analytics to improve student success, retention and graduation rates (Arnold & Pistilli, 2012). The third large-scale use of analytics in higher education is to support the development of the curriculum, enhanced student learning outcomes and behavior, personalized learning, improved instructor performance, post-educational employment opportunities, and improved research in the context of higher education institutions (Nunn, Avella, Kanai, & Kebritchi, 2016). Further, the acquisition of data generated from teachers use of technology can inform our understanding of online behaviors and identify areas in need of improvement to facilitate enhanced instructor-student behavior and support the development of research tools in online libraries (Xu & Recker, 2012), and guide curriculum development (Armayor & Leonard, 2010). The fourth use-case scenario but perhaps less explored is in informing professional development opportunities. Based on Sharda, Asamoah, and Ponna (2013), growing evidence in the literature suggests that analytics represents a critical component of the ability of scholars to generate knowledge, as well as to progress in research disciplines but lacks adequate expertise (Chen, Chiang, & Storey, 2012; Picciano, 2012). Lastly, institutional analytics can be used to identify post-education employment opportunities for graduates. For instance, institutions can use analytics to target education that more closely aligns with labor market needs and can be used to predict graduate employment, unemployment, or undetermined situations about job opportunities (Jantawan & Tsai, 2013; Kostoglou, Vassilakopoulos, & Koilias, 2013).

Challenges of harnessing analytics in higher education

The literature reports many possible problems relating to the deployment of analytics in higher education. Bottles, Begoli, and Worley (2014) for example stated that collecting and aggregating data from multiple databases to perform useful analytics is a challenging undertaking. Further, there is an enormous cost associated with the development of specific algorithms and tools to harvest, extract and analyze data (Daniel, 2015). Since the institutional use of analytics requires seamless movement and to some extent development of data from one system to another, issues around data standards and systems interoperability can further complicate matters. Since many institutions are highly siloed, and having a diverse data stewardship, issues of data governance and the capability to have the right access to relevant data are still regarded as a primary concern.

Further, the quality of information generated from large datasets is dependent on the quality of data collected and the instruments used. Lack of standardized measures and indicators makes inter (national) comparison difficult (Daniel, 2015). As learners expand the boundaries of the learning management system into open or blended learning settings, researchers are faced with new challenges of determining success from the students’ perspectives (Ferguson, 2012). This free movement of students within and between learning environments is likely to generate a significant amount of data.

Ethical and privacy are additional major issues associated with the legal discourse of analytics (Pardo & Siemens, 2014). Because of the rapid changes in technology, as well as how users store data and applications in cloud-based systems, the challenge of privacy and control mechanism, continue to affect adoption and deployment of analytics (Johnson et al., 2010; Sclater, 2014).

Additional concerns raised with increasing use of analytics include data interpretation, data ownership, data preservation, sharing data with parties outside of the institution environment, and proper training of staff members regarding the data (Sclater & Bailey, 2015). Moreover, the question of who owns aggregated data, because having an infrastructure with the capacity to house large amounts of information becomes a daunting task (West, 2012). Because of these different issues, institutions must achieve a balanced approach to safeguarding data while also assuring benefits to the educational process because the acceptance of analytics in higher education will depend on how institutions address these issues.

This study presents the current perception of analytics and their possible contribution to understanding the challenges facing the higher education sector in New Zealand. Seven out of eight research-intensive public universities in New Zealand took part in the study. A survey design involving administration of an online questionnaire was used because it provides a broader view of the current understanding of analytics across higher institutions in New Zealand.

Methods and procedures

Research reported in this study employed a survey design (see Additional file 1). The online survey was used to examine the perception and value of deploying analytics in seven research-intensive public universities in New Zealand. A questionnaire derived from (Goldstein & Katz, 2005) was utilized as the main instrument for collecting data about the items associated with the perception of analytics, how institutions are engaging with analytics, and the outcomes out of analytics use within the institutions of higher education. According to Fowler (2009), the principal idea of the survey method is the sequence of gathering data from a sample of a population, followed by describing the target population of interest and then producing statistical estimates about aspects of the sample of people. Surveys commonly use a structured set of questions or statements to measure attitudes, beliefs, values or tendencies to act (Goodwin, 2005). The items were measured on a 5-point Likert scale (1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, and 5 = strongly disagree). The questionnaire was tested for reliability, revealing a high Cronbach’s alpha of 0.90. Participants were also provided with open—ended questions at the end of some Likert scale items, to elaborate on their responses.

Analysis

The closed-ended questions were analyzed using Statistical Package for the Social Sciences (SPSS) version 23. Descriptive statistics including percentages and frequencies were used to summarize the results. The open-ended questions were analyzed thematically based on the associated constructed measured in the Likert scale questions. Key quotations were extracted to highlight findings. Further, themes derived from the open responses were further summarized using textual analysis using information retrieval techniques (see Figs. 1, 2 and 3).
Fig. 1

Summary of analytics term frequencies

Fig. 2

Analytics similarity of words and frequencies

Fig. 3

Extracted words associated with analytics

Participants demographic

Since the ultimate goal of the research was to develop a wider understanding the role of analytics in respect to improving decision-making, participants were predominantly senior management and those who have some role associated with decision-making within higher education (N = 82) (see Table 1). Specifically, those who responded to the survey included senior executive (vice chancellors, deputy vice chancellors, pro-vice-chancellors, heads of departments, directors, and managers. See Table 1).
Table 1

Demographics of participants

Demography item

N (%)

Role

 Vice Chancellor

0 (0)

 Deputy Vice Chancellor

1 (1)

 Pro-Vice Chancellor

3 (3)

 Dean

1 (1)

 Head of Department

20 (23)

 Director

9 (10)

 Manager

20 (23)

 Coordinator

13 (15)

 Other

21 (24)

Length of time in the role

 Less than a year

9 (10)

 1–5

48 (54)

 5–10

24 (27)

 10–15

7 (8)

 15–20

0 (0)

 More than 20

1 (1)

Ethnicity

 European

71 (82)

 Māori

0 (0)

 Asian

5 (6)

 Pacific

0 (0)

 Middle Eastern

2 (2)

 Latin American

1 (1)

 African

1 (1)

 Indigenous

0 (0)

 Other

5 (6)

Education

 

 Bachelor

10 (11)

 Master

25 (28)

 PhD

44 (50)

 Other

9 (10)

Findings

Key findings suggest that analytics has become an important phenomenon in higher education in New Zealand. However, the perception of analytics differed along three dimensions (structural, functional and combination of structural and functional). The structural perception of analytics views analytics as different interrelated components or elements that make up analytics as a knowledge system, with clearly discoverable and useful relationships. Examples of analytics viewed along the structural dimensions included: data, numbers, and statistics. Functional analytics relates to the view of analytics as a means to an end, knowledge structures, data driven decision-making approach or management tools that help institutions to accomplish specific goals. Participants viewed analytics as a set of processes and tools for extraction of data and utilizing this data to develop predictive models to address critical issues in higher education. In the structural-functional aspects, participants perceived analytics as the combination of both dimensions. Furthermore, some participants saw analytics as the use of different forms of data and information, which requires new computational tools and approaches for harvesting, analyzing and utilizing data to influence the quality of decision within higher education.

Perceptions of analytics: Textual analysis

Participants were asked, "what comes to your mind when you hear the word ‘analytics'?" Responses were compiled into a text file and loaded into NVivo. A basic list of terms together with their frequencies was obtained, using an information retrieval techniques, with the help of the following formula: for a term i in document j: (\( {\mathrm{W}}_{\mathrm{i},\mathrm{j}}={\mathrm{tf}}_{i,j}\ast \mathrm{Log}\frac{\mathrm{N}}{\mathrm{dfi}} \)); where W is the weight, tf i,j  = total number of occurrences of term i in a document j; df i  = number of documents containing i. And N = the total number of documents with terms of interest (Manning, Raghavan, and Schutze, 2008). Terms that do not make sense were removed, and a final list of words together with associated phrases are presented (see Table 2).
Table 2

Perceptions of analytics, basic list of terms and their frequencies

Word

Length

Count

Weighted (%)

Similar Words

data

4

58

13.2

data, inform, information, informative

analysis

8

15

3.4

analysis

learning

8

15

2.7

know, knowledge, learning, reading, see, take, teaching

student

7

12

2.7

student, students

numbers

7

12

2.7

come, comes, numbers

using

5

11

2.4

purposes, role, usage, use, using, utilised

statistics

10

10

2.3

statistic, statistical, statistics

course

6

9

1.8

course, courses, forming, line, tracking, trends

research

8

8

1.8

research

decision

8

8

1.8

conclusions, decision, decisions

design

6

7

1.3

design, designers, destinations’, indicators, intent, invented, purposes

analyzing

9

7

1.6

analyze, analyzing

measurement

11

7

1.5

bar, evaluation, measurement, measures, metrics

understanding

13

7

1.1

discern, reading, see, translation, understand, understanding

big

3

6

1.4

big, large

word

4

5

1.1

word, wrangling

graphs

6

5

1.1

charts, graphs

collected

9

5

1.1

collected, collecting, collection

achievement

11

5

1.1

achievement, management, succeed, success

prediction

10

4

0.9

prediction

Google

6

4

0.9

Google

complete

8

4

0.9

complete, completing, completion

extraction

10

4

0.8

education, extraction

describe

8

4

0.8

describe, key, line, reporting

systems

7

3

0.7

systemized, systems

support

7

3

0.7

help, support

processing

10

3

0.7

actionable, processing

mining

6

3

0.7

mining

better

6

3

0.7

best, better

based

5

3

0.7

based, basis

existing

8

3

0.6

existing, university, world

evidence

8

3

0.5

discern, evidence

tool

4

2

0.5

tool, tools

strategic

9

2

0.5

strategic

retention

9

2

0.5

retention

quantitative

12

2

0.5

quantitative

performance

11

2

0.5

performance

monitor

7

2

0.5

monitor, monitoring

environment

11

2

0.5

environment, environments

enrolment

9

2

0.5

enrolment, recruitment

buzz

4

2

0.5

buzz

blank

5

2

0.5

blank, space

applications

12

2

0.5

applications

publications

12

2

0.3

publications, world

engaged

7

2

0.3

engaged, take

trendy

6

1

0.2

trendy

KPIs

4

1

0.2

KPIs

intervention

12

1

0.2

intervention

interaction

11

1

0.2

interaction

insights

8

1

0.2

insights

higher

6

1

0.2

higher

hidden

6

1

0.2

hidden

generated

9

1

0.2

generated

enhance

7

1

0.2

enhance

EFTS

4

1

0.2

EFTS

DSS

3

1

0.2

DSS

driven

6

1

0.2

driven

discovery

9

1

0.2

discovery

decision-making

14

1

0.2

decision-making

As shown in Fig. 1, the list of words together with their frequency (t ≥ 3) is further summarized into key themes. The word ‘data’ was the most counted word, weighted at 13.2%, followed by the word ‘learning,’ ‘number,’ and ‘statistics.’ In line with the three forms of perceptions presented earlier (structural, functional, and structure-functional), general, this would suggest that the majority of participants thought of analytics notion as its structural elements of data, learning, numbers, and statistics. Term frequency (tf) were then plotted with similar words (see Fig. 2). Figure 3 presents fifteen phrases associated with analytics that was extracted from the responses, representing the total number of times of occurrence for each phrase. In the analysis of phrases, the term ‘Big Data’ and ‘Data Analysis’ were the most frequent ones, repeated in the participants’ responses. Drawing from the textual analysis, the notion of analytics as perceived by participants can be reconceptualized into predominantly two main dimensions of structural and functional, as well as the combination of both (see Fig. 4).
Fig. 4

Perceptions of analytics among participants

Structural

Over a third of participants (30, 37%) perceived analytics regarding its fundamental elements (e.g. data, statistics, numbers, KPIs, graphs, etc.). The majority of participants who viewed analytics in the form of structures identified as managers, and head of departments.

“Information or data relating to higher education. I think it perhaps could be information similar to market data.”

“Data collected for a range of purposes.”

Functional

The functional perception of analytics was observed in over half of the participants (40, 50%) in the study. Participants who viewed analytics in functional grounds considered analytics as a means to an end. They understood analytics as the collection of a process (algorithms, models, and techniques for data processing, and visualization) and tools (data warehouse, databases, Hadoop) used to answer the difficult questions, techniques for extracting data and useful discerning information to describe and predict performance outcomes. Some participants understood analytics as a systematic process of data collection, analysis, and use of information to support evidence-based decision-making. Participants who viewed analytics in their functional capacity equated analytics in higher education to Google Analytics and Big Data. They stated that analytics could be used to make a better judgment, evaluate, monitor, and enhance performance. Similarly, the functional view of analytics was common among managers and head of departments.

“Analysis of metrics such as research outputs (publications, etc.), student completions and external revenue.”

“Data analysis of students such as achievement rates, both regarding course and qualification completion rates.”

Participants who identified as directors viewed analytics as the process of data mining, knowledge discovery, and extraction of information. In particular, they saw analytics as a process that involves the use of data to address complex queries for better student performance.

“Use of data to answer difficult and continuing questions - as a discipline to teach and as a research tool, unique for student achievement.”

“Systemized analysis or sorting of data with the intent of producing meaningful information.”

Structural-functional

A small number of participants (8, 10%) who identified themselves as senior professors, coordinators, and managers, conceptualized analytics in the form of both structural and functional dimensions.

“Analysis of the statistical data collected on students in different courses, for example: how many students attend a course, how many completed the program, demographics, etc.”

“Big Data is using already existing data to make judgments.”

“Statistics. Data that allows me to make decisions based on the information.”

Among some participants (2, 3%) in the group, analytics was viewed as a trendy new buzzword. Overall results revealed an inconsistent understanding of the analytics and its utilization within higher education. Three alternative views of analytics were evident in the data (structural, functional and structural-functional). The majority of participants who identified as managers and heads of departments saw analytics distinctively regarding either metrics, numbers, KPI or the role these play in influencing decision-making in higher education. However, other participants saw analytics both as structural and functional. In the discourse of analytics, these participants also mentioned tools and processes often associated with the analysis and presentations of large-scale data (Big Data). The three-dimensional perception of analytics identified within the setting of higher education provides a critical understanding of the value, and the role analytics are more likely to play within the institutions of higher learning.

The value of analytics in higher education

Participants mentioned some potentials in the use of analytics in higher education. In particular, they stressed the role of analytics in strengthening knowledge of student behavior and demographics, optimization of resources, and improving faculty performance. They stated that the intentional use of student data and the learning environment is more likely to equip institutions to identify more efficient and proactive strategies to support students. Further, they reported the use of analytics could provide valuable insight into the factors that influence learners’ success. Participants reported several areas in which the use of analytics is useful, including student learning, resource optimization, for monitoring and improving faculty performance, and resource appraisal and allocation.

Student support

Participants consistently reported that student support among other is the most critical aspect of the value of analytics in higher education institutions. In particular, ‘helping students learn more effectively’ (37, 55%) was found to be more frequently stated value of analytics compared to ‘recruiting students’ (29, 44%) area. Three-quarters of the participants (50, 75%) indicated ‘understanding student demographics and behavior’ as the core value of analytics use within the institutions. Managers and head of departments mostly reported that institutions could benefit more from the use of analytics to examine student related issues, stating that in their institution’s student related use of analytics can enable right intervention and identification of students who may need adequate support. The particular use of analytics in higher education is in line with work on early detection systems (Arnold & Pistilli, 2012).

“Better tracking of student performance over time will allow for identification of both students who excel and could be targeted/nurtured for further education (postgraduate), as well as early detection and intervention for students who may be struggling before they fail their course.”

Resource optimisation

Similar to the student area, three-quarters of the participants (50, 75%) also identified ‘optimized use of resources’ as another important area which institutions can utilize analytics. Participants who viewed analytics as a way to maximize resources were predominantly coordinators and head of departments.

“Especially resources use in an institution can be well-optimized using particular sort of analysis.”

Faculty performance

Participants also reported ‘improving faculty performance’ as another important area for the use of analytics, a large number of this cohort (49, 73%) identified as the head of departments and managers.

“Improvement of faculty and research performance by extracting the right data and getting insightful ('meaningful') data.”

Administrative services

A large number of participants (43, 64%) regarded ‘improving administrative services’ as an important area where analytics could be deployed to address. Participants identified other benefits of analytics some of which are summarized in Table 3.
Table 3

Institutional benefits of analytics

Possible institutional benefits of using analytics

N (%)

Understanding student demographics and behaviors

53 (61)

Improving faculty performance

52 (60)

Optimizing use of resources

51 (59)

Improving administrative services

44 (51)

Creating data transparency/sharing/federation

41 (47)

Helping students learn more efficiently/graduate

40 (46)

Reducing administrative costs

32 (37)

Recruiting students

28 (32)

Containing/lowering costs of education

27 (32)

Other

11 (13)

Demonstrating higher education’s effectiveness/efficiency

0

From this finding, it was apparent that the possible benefits of analytics will be largely in the areas of student support, resource optimization, faculty performance, and administrative services (but not extending to cost reduction and student recruitment). However, these may be the areas in which institutions can find initial success stories in analytics (see Table 3, the table contains the data from the close-ended question no. 9 in the questionnaire Additional file 1). Some other participants (7, 12%) suggested other values of analytics including improvements in knowledge, security, quality assurance/enhancement, student learning and teaching, student enrollments, student retention, and student engagement.

“From my perspective, the greatest benefit is information on what is happening in the wider market and how can we use this to increase sales/enrollments.”

What these findings suggest is that analytics can be used in many areas, especially when institutions need to map their strategic initiatives, to develop predictive models to enable them to address current and future challenges. For instance, the University of Wollongong established Library Cube, a tailored database which joins library usage data with student data such as demographic characteristics and students’ academic performance) to predict the students’ grades (Jantti & Cox, 2013). It has also enabled the university to identify whether a correlation exists between usage of library resources and academic performance (e.g. student grades).

Along with providing information that could increase student success, this innovative use of analytics has the additional benefits of demonstrating the added value that a library provides to the institutions (Cox & Jantti, 2012). Many study participants provided instances of how analytics programs can improve processes such as comparing institutions’ progress and assisting with strategic planning in the central areas while increasing performance. Also, the use of analytics can enable faculty, staff, and administrators to have access to the culture of evidence-based decisions.

“Universities are large businesses with high levels of accountability, and they need the best information and evidence available to support their decision-making.”

“Good decisions are based on good data.”

Furthermore, analytics can help the academic community to focus on questions that are of strategic significance.

In the analysis, a significant number of participants (71, 88%) were aware that their institutions including departments, divisions, and faculties were currently using analytics to some extent. However, only (10, 12%) of people were not aware of whether if their institutions were using analytics or not. The variation of the way analytics is used in different institutions suggests analytics will continue to develop as a field of higher education due to growing availability of data (see Table 4, the table contains the data from the close-ended question no. 8 in the questionnaire Additional file 1).
Table 4

Use of analytics in institutions

Current institutional use of analytics

N (%)

Student enrollment management

62 (70)

Student progress

58 (65)

Finance and budgeting

58 (65)

Library

43 (48)

Central IT

41 (46)

Faculty teaching performance

41 (46)

Student learning

39 (44)

Faculty research performance

38 (43)

Human resources

35 (39)

Progress of strategic plan

34 (38)

Research administration

33 (37)

Instructional management

23 (26)

Procurement

22 (25)

Facilities

19 (21)

Cost to complete degree

17 (19)

Alumni/advancement

15 (17)

Faculty promotion and tenure

14 (16)

Other

6 (7)

Discussion

In this study, we observed that there are various understanding of analytics among stakeholders, Though participants at large pointed out the potential value of analytics in addressing challenges in higher education, the diversity in understanding what constitutes analytics, is likely to presents challenges in future adoption of analytics in higher education. Findings from this research are consistent with Offerman and Norris (2009) perception of analytics as a process involved with data analysis to measure, improve, and compare the performance of individuals, programs, departments, institutions or enterprises, groups of organizations, and entire industries. Results also corroborate Clow (2013) work that suggested that analytics is an interdisciplinary field of inquiry that involves the process of data assessment and analysis, to enable educational institutions to measure, improve, and compare the individual performance (Clow, 2013; Norris et al., 2009). Furthermore, results from this research suggest that the participants acknowledged some potentials in the use of analytics which contributes to a number of areas within the higher education. Participants discussed the value of analytics in supporting student learning, resource optimization, for monitoring and improving faculty performance, and way to appraise and allocate administrative services.

Moreover, participants reported that analytics offer interventions based on the predicted models grounded in the large datasets in educational institutions (Campbell & Oblinger, 2007; Clow, 2013). Participants pointed out that a systematic approach to the use of analytics will provide a better understanding of the patterns of learner behavior, networks, and interactions. However, because of the variation in the understanding of what constitutes analytics in the literature, there are some possible challenges in adoption (Mahroeian & Daniel, 2016; Daniel, 2015). These challenges include privacy, security, and ethical matters related to data collection, use, and distribution. In the light of this research and consistent with the literature, there is a need for exploration of mechanisms to provide appropriate transparency, data controls by students, information security, and accountability safeguards (Pardo & Siemens, 2014).

Conclusion, limitation, and future research

This study has provided a general overview of research on analytics and their perception in higher education institutions in New Zealand. The present study suggests that various stakeholders have a different interpretation of the notion of analytics. To some stakeholders, analytics understood along its structural elements (such as statistics, metrics, trends, numbers, graph, and any relevant information/data to enhance decision-making better). Whereas others think of analytics on functional dimensions, as a set of processes and tools for achieving particular goals. They help decision-makers gain insights and taking action on complex problems. Further, some stakeholders viewed analytics from both structural-functional perspectives. These three forms of perceptions of analytics identified in the study contribute to the theoretical understanding of the meaning and value of analytics in higher education sector. However, because of the variation in perception of meaning and use, it is more likely that institutions will face challenges in implementation. As analytics grows in importance within higher education, future work is needed to develop a consistent language around the meaning and value of analytics in higher education to help advance the theory and practice.

Though the outcome of this research provides a broader understanding of current perceptions and value of analytics, there are some limitations associated with the study. The first limitation is that the research is perception-based that utilizes a small sample. Further, those who participated are those who already have some role associated with analytics in higher education, and so results only reflect distinct perspectives of those individuals as compared to others who might be affected by the outcome of decisions (e.g., students). Because we are interested in the use of analytics to influence the decision in higher education, the study does not explore perspectives of individuals involved in the development of analytics and tools, the outcome of which would have enhanced results. However, such perspective will be pursued in future research.

Furthermore, benchmark data would have been useful to assess the impact of analytics in higher education across research-intensive public universities in New Zealand. Future research needs to focus on the extent to which institutions are engaged in the implementation of analytics support student learning, and the quality of teaching, and exploring possible ways of addressing challenges that can slow implementation of analytics in higher education. Also, there is a need to provide professional development opportunities and engage management in the culture of evidence-based informed decision-making through the utilization and support of analytics.

Declarations

Authors’ contributions

Authors contributed equally to the planning, design, analysis, and writing of this manuscript. All authors read and approved the final manuscript.

Authors’ information

Hamidreza Mahroeian is a Ph.D. candidate in higher education/educational technology at the University of Otago—New Zealand. He is exploring the role of analytics and Big Data in higher education institutions. Dr. Ben Daniel is a coordinator and senior lecturer in higher education/educational technology at the University of Otago—New Zealand. His research interests include the value of Big Data and learning analytics in higher education, theory and praxis of research methodology, and social media technologies in learning and teaching. Russell Butson is a senior lecturer in higher education/educational technology at the University of Otago—New Zealand. His research is focused on the learning that takes place within the university setting.

Ethics approval and consent to participate

The data for this research can be accessed by emailing the corresponding author. Ethical approval was obtained from ethics committee at the University of Otago. Furthermore, the participants gave informed consent and data has been anonymized.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Higher Education Development Centre, University of Otago

References

  1. Althubaiti, A., & Alkhazim, M. (2014). Medical colleges in Saudi Arabia: Can we predict graduate numbers? Higher Education Studies, 4(3), 1–8.View ArticleGoogle Scholar
  2. Armayor, G. M., & Leonard, S. T. (2010). Graphic strategies for analyzing and interpreting curricular mapping data. American Journal of Pharmaceutical Education, 74(5), 81–91.View ArticleGoogle Scholar
  3. Arnold, K. E., & Pistilli, M. D. (2012). Course signals at Purdue: Using learning analytics to increase student success. In Paper presented at the proceedings of the 2nd international conference on learning analytics and knowledge. Vancouver, British Columbia: Canada Retrieved from http://dl.acm.org/citation.cfm?id=2330666 Google Scholar
  4. Bichsel, J. (2012). Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations (research report). Louisville, CO: EDUCAUSE Center for Applied Research https://library.educause.edu/resources/2012/6/2012-ecar-study-of-analytics-in-higher-education
  5. Bottles, K., Begoli, E., & Worley, B. (2014). Understanding the pros and cons of big data analytics. Physician Executive, 40(4), 6–12.Google Scholar
  6. Campbell, J. P., & Oblinger, D. G. (2007). Academic analytics. In EDUCAUSE Retrieved 15 October 2007, from https://library.educause.edu/resources/2007/10/academic-analytics Google Scholar
  7. Charlton, P., Mavrikis, M., & Katsifli, D. (2013). The potential of learning analytics and big data. In Ariadne Retrieved from http://www.ariadne.ac.uk/issue71/charlton-et-al Google Scholar
  8. Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly: Management Information Systems, 36(4), 1165–1188.Google Scholar
  9. Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695. doi10.1080/13562517.2013.827653.View ArticleGoogle Scholar
  10. Cox, B. L., & Jantti, M. H. (2012). Discovering the impact of library use and student performance. EDUCAUSE review. no. July 18, pp. 1-9. Retrieved from http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1507&context=asdpapers
  11. Daniel, B. K. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920.View ArticleGoogle Scholar
  12. Daniel B. K., & Butson, R. (2013). Technology Enhanced Analytics (TEA) in Higher Education, Proceedings of the International Conference on Educational Technologies, 29 November–1 December, 2013, Kuala Lumpur, Malaysia,. 89–96.Google Scholar
  13. Dawson, S., Bakharia, A., & Heathcote, E. (2010). SNAPP: Realising the affordances of real-time SNA within networked learning environments. Proceedings of the 7th international conference on networked learning. 125-133.Google Scholar
  14. Dietz-Uhler, B, & Hurn, J. E. (2013). Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning, 12(1), 17–26.Google Scholar
  15. DiCerbo, K. E. (2014). Game-based assessment of persistence. Journal of Educational Technology & Society, 17(1).Google Scholar
  16. Ferguson, R. (2012). Learning analytics: Drivers, developments, and challenges. Int J Technol Enhanced Learning, 4(5–6), 304–317.View ArticleGoogle Scholar
  17. Fowler, F. (2009). Survey research methods (4th ed.). London, UK: Sage Publications.Google Scholar
  18. Gantz, J, & Reinsel, D. (2012). The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyse the future. Retrieved from http://www.emc.com/leadership/digital-universe/index.htm
  19. Goldstein, P. J. & Katz, R. N. (2005). Academic Analytics: The Uses of Management Information and Technology in Higher Education, ECAR Research Study Volume 8. Retrieved March 24, 2014, from http://www.educause.edu/ir/library/pdf/ers0508/rs/ers0508w.pdf.
  20. Hung, J. L. & Zhang, K. (2012). Examining mobile learning trends 2003–2008: A categorical meta-trend analysis using text mining techniques. Journal of Computing in Higher education, 24(1), 1–17.Google Scholar
  21. Jantawan, B., & Tsai, C.-F. (2013). The application of data mining to build a classification model for predicting graduate employment. Int J Comput Sci and Inf Security, 11(10), 1–7.Google Scholar
  22. Jantti, M., & Cox, B. (2013). Measuring the value of library resources and student academic performance through relational datasets. Evidence-Based Library and Information Practice, 8(2), 163–171.View ArticleGoogle Scholar
  23. Johnson, L., Levine, A., Smith, R., & Stone, S. (2010). The 2010 horizon report. Austin, Texas: The New Media Consortium.Google Scholar
  24. Jones, S. J. (2012). Technology review: The possibilities of learning analytics to improve learner-centred decision-making. The Community College Enterprise, 18(1), 89–92.MathSciNetGoogle Scholar
  25. Kostoglou, V., Vassilakopoulos, M., & Koilias, C. (2013). Higher technological education specialties and graduates’ vocational status and prospects. Education+ Training, 55(6), 520–537. doi:10.1108/et-03-2012-0026.View ArticleGoogle Scholar
  26. Mahroeian, H., & Daniel, B. K. (2016). The dynamic landscape of higher education: The role of big data and analytics. In EdMedia: World Conference on Educational Media and Technology (pp. 1320-1325). Association for the Advancement of computing in education (AACE).Google Scholar
  27. Manning, C. D.; Raghavan, P.; and Schutze, H. 2008. Introduction to information retrieval. Cambridge University press, 1st edition.Google Scholar
  28. Mardikyan, S., & Badur, B. (2011). Analyzing teaching performance of instructors using data mining techniques. Informatics in Education, 10(2), 245.Google Scholar
  29. Norris, D., Baer, L., & Offerman, M. (2009, September). A national agenda for action analytics. In National Symposium on Action Analytics (pp. 21-23).Google Scholar
  30. Nunn, S., Avella, J., Kanai, T., & Kebritchi, M. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning, 20(2). Retrieved from https://olj.onlinelearningconsortium.org/index.php/olj/article/view/790
  31. Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450.View ArticleGoogle Scholar
  32. Picciano, A. G. (2012). The evolution of big data and learning analytics in American higher education. Journal of Asynchronous Learning Networks, 16(3), 9–20.Google Scholar
  33. Scholes, V. (2016). The ethics of using learning analytics to categorize students on risk. Educational Technology Research and Development, 64(5), 939–955.View ArticleGoogle Scholar
  34. Sclater, N. (2014). Code of practice essential for learning analytics. JISC, London. Retrieved from https://analytics.jiscinvolve.org/wp/2014/09/18/code-of-practice-essential-for-learning-analytics/
  35. Sclater, N., & Bailey, P. (2015). Code of practice for learning analytics. London: JISC Retrieved from http://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics Google Scholar
  36. Sharda, R., Asamoah, D. A., & Ponna, N. (2013). Business analytics: Research and teaching perspectives. In Information Technology Interfaces (ITI), Proceedings of the ITI 2013 35th International Conference on (pp. 19-27). IEEE.Google Scholar
  37. Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.View ArticleGoogle Scholar
  38. Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 30–40.Google Scholar
  39. Tulasi, B. (2013). Significance of big data and analytics in higher education. International Journal of Computer Applications, 68(14), 23–25.View ArticleGoogle Scholar
  40. Van Barneveld, A., Arnold, K. E., & Campbell, J. P. (2012). Analytics in higher education: Establishing a common language. EDUCAUSE Learning Initiative, 1, 1–11.Google Scholar
  41. West, D. M. (2012). Big data for education: Data mining, data analytics, and web dashboards. Governance Studies at Brookings, 1-10. Retrieved from https://www.brookings.edu/research/big-data-for-education-data-mining-data-analytics-and-web-dashboards/
  42. Xu, B., & Recker, M. (2012). Teaching analytics: A clustering and triangulation study of digital library user data. Journal of Educational Technology & Society, 15(3), 103–115.Google Scholar

Copyright

© The Author(s) 2017