Assessing digital competence and its relationship with the socioeconomic level of Chilean university students

Digital competence (DC) is one of the key aspects in citizen development in the digital age. The DC is particularly important in forming university students and future teachers. This article presents the main results of a study to evaluate DC and its relationship with the socioeconomic level of first-year students of pedagogy in three Chilean public universities, located in the north, center, and south of the country. A quantitative research methodology was used, with a sample of 817 students, the data were collected through the DIGCOMP-PED evaluation instrument, which evaluates DC development using the DIGCOMP framework. The results were analyzed at the general and socioeconomic level on the variables of the educational establishment where they attended high school and the territorial area of the university they attended. The main results indicate that the level of DC achievement is intermediate, the areas with the highest levels of achievement were “network security” and “online communication and collaboration.” On the other hand, the lowest levels of achievement were reached in the areas “information and digital literacy,” “digital content creation,” and “problem solving.” The level of DC is higher among students of private establishments and those who attend universities located in the central area.

students (Durán et al., 2016).Diagnosing DC is a cornerstone for generating training plans aimed at developing these competencies in students.Future teachers need to develop DC as a basis for developing digital teaching competence (Silva et al., 2019).A digitally competent teacher is key to embedding DT in education (Engen, 2019).
According to several authors, the socio-economic level of families is relevant in the development of pupils' digital competence (Zhong 2011;Claro et al., 2012).In this sense, some studies highlight the economic influence (Román & Murillo, 2013), where it is proven that having a computer at home increases pupils' digital competence (Gómez-Pablos et al., 2020;Kuhlmeier & Hemker, 2007;Livingstone & Helsper, 2007).There is also evidence that students' socio-economic background influences their ability to use new technologies through the resources available to them outside school (Fernández-Mellizo and Manzano (2018).
The socioeconomic level (SES) in Chile is directly related to the type of establishments in which students enroll (Barrientos-Oradini & Araya Castillo, 2018).This has an impact on their scores in university admission tests and, therefore, on the university they have access to (Araneda-Guirrimán et al., 2018).Municipal establishments receive students from low SES, private subsidized establishments receive students from middle SES, and paid establishments receive students from high SES.Top-ranking universities are accessed by students achieving the highest scores on the selection tests (Brunner, 2012).In Chile, there are marked differences in the type of institution that young people access according to their socioeconomic level; admission to universities is related to the score in the selection, which limits the possibilities of young people with fewer resources, as they systematically obtain lower scores on this test (Catalán and Santelices, 2014).At a general level, SES is one of the variables that most influence students' DC (Hatlevik et al., 2018).This aspect is no different in Chile, where studies show that SES is directly related to students' DC level (Claro et al., 2015;Jara, et al., 2015).For this reason, this study seeks to determine the DC level of first-year students of pedagogy, crossed with two socioeconomic variables: the type of educational establishment where they attended secondary school and the territorial area of the university they attend.

Theoretical framework
Digital competence (DC), involves "the confident, critical and responsible use of, and engagement with, digital technologies for learning, at work, and for participation in society." (European Commission, 2018, p.4). DC is understood as the sum of skills, knowledge, and attitudes in technological, informational, multimedia, and communicative aspects, which give rise to a complex multiple literacy (Ferrari, 2012).
Several entities around the world have developed guiding frameworks to define dimensions and indicators for DC, of which the most well-known are: iSkills (Pérez-Escoda et al., 2019), International Society for Technology in Education (Fuller, 2020), DigiLit Leicester (Fraser et al., 2013), ICILS by the OECD (Punter et al., 2017), and finally, DIG-COMP by the European Economic Community (Redecker & Punie, 2017) and its latest version, DIGCOMP 2.1 (Carretero et al., 2017).The latter framework considers a globalizing concept of DC, which includes knowledge, skills, and attitudes, and covers the areas of: information, communication, content creation, security, and problem solving.It is currently being used in several studies to assess DC at the university level in general (López-Meneses et al., 2020;Vázquez-Cano et al. 2017) and in the fields of pedagogy (González-Calatayud et al., 2018;Gutiérrez & Serrano, 2016).
DC plays an important role in the personal and professional development of university students.For Gros (2015), students need to develop important skills to guide their educational processes, including digital skills.Developing digital skills is considered crucial for university students (Aguaded & Cabero, 2013).However, students do not demonstrate high proficiency in the use of DT for learning (Prendes-Espinosa and Román-García, 2017).Students reach university with a basic level of digital competence (Sánchez-Caballé et al., 2020).The university student body is part of a digital generation; however, they do not learn better with the use of DT; we need to work with them to develop DC. (Gutiérrez et al, 2018).This means that students' DC needs to be assessed to design extracurricular training plans and incorporate activities that encourage DC development into the curriculum.
Using the DIGCOMP framework we found numerous studies.López-Meneses et al. ( 2020) evaluated the competence of university students from three European universities in three areas: "information and data literacy, " "communication and collaboration, " and "digital content creation." The results showed that these future graduates had a highintermediate level of competence in "information and digital literacy" and "communication and collaboration, " but a low-intermediate level in "digital content creation." A study was carried out by (González-Calatayud et al., 2018) with the main purpose of improving DC of second year pedagogy students through tasks focused on working each of the areas of DIGCOMP.Students generally showed a medium level of digital competence in all areas, in the areas of "problem solving, " "information and data literacy, " and "digital content creation, " the lowest average values were found, and "communication and collaboration" and "network security" showed a higher average.Gutiérrez and Serrano (2016) analyzed DC in first year students of primary education pedagogy.In accordance with the DIGCOMP framework, the results indicate that participants consider themselves competent in the most basic aspects of digital competence.
Research on the socio-economic level of students has indicated that factors related to the family history of students influence their ICT literacy outcome (Siddiq et al., 2017).Students from low SES households express less self-efficacy in ICT (Vekiri, 2010).The ACARA study (2012) notes that children of parents with low educational levels showed poorer ICT literacy competency than children of parents with higher education.Hatlevik et al. (2018), in a study conducted in 15 countries, socio-economic status appears to be the most important predictor of computer competence and computing in all countries.
In Chile, the national digital skills test that evaluates proficiency levels and ICT skills for learning (SIMCE TIC) through simulated environments applied to high school students in 2013, shows that 46.9% of the students reached the initial level, 51.3% are at an intermediate level, while the advanced level is achieved by only 1.8%.In addition, it is observed that students in private schools show higher levels of ICT skills and students in municipal schools show the lowest levels (Rodríguez-Garcés & Muñoz-Soto, 2018).Analyzing the results of this ICT skills test, Jara et al. (2015) note that students who have a computer at home score better than those who do not, and students from families with a higher SES and more cultural possessions scored higher.On the other hand, Claro et al. (2015) found that the education level of parents was the most relevant factor in explaining the scores of students.
There are several tools to assess DC in undergraduate education at the level of selfperception or diagnosis, including: INCOTIC 2.0 (González et al., 2018), ACUTIC (Mirete et al., 2015), CODIEU (Casillas et al. 2018), REATIC (Moya-Martínez et al., 2011), INCODIES (Guillén-Gámez & Mayorga-Fernández, 2020).The latter follows the structure of the European DIGCOMP framework.This model may serve as a structure and basis for the development of specific DC evaluation by Petterson (2017).Measuring DC is a critical challenge to better understand its development, so further evaluation tools need to be developed for this measurement (HE & Zhu, 2017).
Objective assessment tools are increasingly being required, which are not based only on the perception of the user but measure the level of DCT by solving situations or problems in line with the indicators to be evaluated (Villa-Sánchez & Poblete-Ruiz, 2011: 150).It is also important to emphasize that there are differences between university students' perception of their own digital competence and the skills they demonstrate (Gabarda-Méndez et al., 2017).One way to combine this type of instrument is to mix evaluation tests with self-assessment tests following the former (Rosman, 2015).Therefore, the challenge is to use an objective, reliable, and valid DCT evaluation test, which allows to assess the knowledge of university students validly and reliably.

General and specific objectives
General Objective: To determine the level of development of Digital Competence (DC) of first year pedagogy students in Chilean public universities, and its relationship with socioeconomic level, through the variables: educational establishment where they attended high school and the university they attend.
Specific Objectives: • Objective 1: Evaluate the level of digital competence of a sample of students from three Chilean public universities.• Objective 2: Study the relationship between the level of achievement of digital competence and the educational establishment where they attended secondary education.• Objective 3: Study the relationship between the level of achievement of digital competence and the university in which they enroll.

Participants
The sample of this study was made up of 817 first year students of higher education who belonged to pedagogy programs of three public universities in northern, central, and southern Chile.This study was conducted during the 2020 academic year.The characteristics of the participants in this research are reported in Table 1.
65.1% of the students are female and 33.8% male.52.6% attend private establishments (co-financed) and 39.2%, municipal establishments.Regarding the programs in which students are enrolled, 53.5% study secondary education pedagogy, and 29.4%, primary education pedagogy.Regarding the university they attend, 54.8% are enrolled in a university in the center of the country; 33.9%, in the north; and 11.3%, in the south.

Instrument
In order to determine the DC level in students of pedagogy, the DIGCOMP-PED evaluation instrument was used, which presents a variety of situations that first year university students face in their daily life in terms of different uses of digital technology.The instrument was built considering the DIGCOMP framework (Redecker & Punie, 2017) specifically, its latest version, DIGCOMP 2.1 (Carretero et al., 2017).The instrument evaluates 21 indicators grouped into five areas of competence (Table 2).
After analyzing the dimensions and indicators considered by DIGCOMP 2.1, we generated the test-type evaluation instrument, composed of closed and multiple-choice items.The initial version had four choices for each of the 21 indicators, and a total of 84 items.To ensure the validity of content of the instrument, the 84 initial questions were validated through expert judgment.These included 5 experts in the field of higher education linked to the area of technologies and education, related to initial teacher training, four from Chile and one from Spain.Validation matrices were used, where each expert evaluated the validity conditions of each item with a Yes (1) or a No (0).Based on the scores assigned by the experts, the overall quality of the items could be established, with variations from 73 to 100%; the questions were selected with over 80% of the assessment.
The final version of the instrument was composed of the three highest rated items for each of the 21 indicators, so the final instrument was made up of 63 items, three items for each of the 21 indicators, distributed in the five areas: "digital information and literacy, " 9 items; "online communication and collaboration, " 18 items; "digital content creation, " 12 items; "network security, " 12 items; and "problem solving" 12 items.Indicator 1 was made up of the first three items; indicator 2, of the next three items; and so on.The items were marked as "correct" 1 point or "incorrect" 0. The range of scores for the indicators is from 0 to 3, and the overall instrument score ranges from 0 to 63.
Reliability analysis of the instrument was evaluated using the Kuder-Richardson-21 indicator (McGahee & Ball, 2009), indicating that the consistency of the responses obtained at the total level was acceptable (KR-21 = 0.60).Cronbach's alpha (α = 0,702).Difficulty level of the test was adequate (DL = 55.06%) and the minimum acceptable performance score (MAP) was 60%.Some examples are shown below (Figs. 1 and 2).

Procedure
This research, which involves human participants, was approved by the ethics committee of the University of Santiago de Chile Nº 410/2019.Participation in the research was voluntary and was not mediated by the provision of any incentive or reward.The teams responsible for the study considered safeguarding anonymity and compliance with data transfer, requesting informed consent from the participants prior to the application.The instrument was responded to digitally, using the link provided.It was applied at the three universities, the beginning of the academic year 2020 during application of the mandatory diagnostic tests applied by the Chilean Ministry of Education, the process that lasted one month.This instrument is not part of mandatory diagnosis, it is an initiative of the three participating universities.The answers given by the students were downloaded and saved in an Excel spreadsheet, and then exported to the SSPS statistical programs.

Statistical tests
Results analysis of the assessment instrument application to answer the research questions first considered a descriptive analysis of the data from the DC assessment instrument at the level of dimensions and indicators.Then, independent samples t-tests were performed to evaluate the mean differences in the scores obtained in the indicators and in the areas according to the variables studied.On the other hand, one-way ANOVA tests were performed to evaluate the differences in the scores obtained according to the variables: educational establishment where they attended high school and university.Tukey post-hoc tests were performed to identify the pairs of variables among which the statistically significant differences detected by the ANOVA test in SPSS (IBM Corp., 2016).

Level of achievement in areas and indicators
Regarding the areas of digital competence (Fig. 3), the level of achievement reached 55.1%.The areas of "information and digital literacy, " 47.7%; "problem solving, " 47.3%; and "digital content creation, " 45.5%" are the areas of lowest achievement, reaching percentages below 50%.Meanwhile, "network security, " 73.2% and "online communication and collaboration, " 58.2%, were the areas of digital competence that obtained the highest percentage of achievement.
As shown in Table 3, the area of highest achievement Online security presents four indicators, the highest mean is obtained by the Device protection indicator (M = 0.813, SD = 0.232) and the lowest mean is reached by the Health and wellbeing protection indicator (M = 0.670, SD = 0.241).

Digital competence by establishment of origin
The results of the level of digital competence by students' school of origin (Fig. 4) show that for the municipal sector the average is 53.5%; private subsidized, 55.6%; and private 58.9%.The dimension with the highest achievement for different educational centers is "network security, " with achievement values above 70%.The dimension with the lowest achievement for the rest of the schools is "digital content creation, " with an achievement level below 47%.
Differences In general, students from private paid tuition schools tend to have significantly higher scores than those who come from municipal, private-subsidized.

Digital competence by university admittance
The results of the level of digital competence by university of admittance (Fig. 5) show that for the central zone the average is 57.0%; north zone, 50%; and south zone, 53.3%.Highest achievement for the three institutions was "network security, " with levels above 71.0%.Lowest achievement for the three institutions was "digital content creation, " with achievement levels below 47%.
The indicators also reported other differences: Digital data, information and contents management (F ( 3  Environmental protection (F (3, 813) = 4.490, p < 0.05), Technical problem solving (F (3, 813) = 3.334, p < 0.05), Identifying technological needs and responses (F (3, 813) = 2.469, p = 0.085), Identifying gaps in digital skills (F (3, 813) = 11.359,p < 0.05).In all the above areas and indicators, students coming from the central zone university tend to have significantly higher scores than those from the northern zone university and then from the southern zone university.This trend does not occur with the indicator evaluate data, information, and digital content (F (3, 813) = 3.285, p < 0.05), where the university of the north zone has higher scores than those of the center zone.

Discussion and conclusions
The general objective of this study was to determine the level of Digital Competence (DC) development of first year pedagogy students in Chilean public universities, and its relationship with socioeconomic level through the variables: educational institution where they attended high school and university where they are pursuing their higher studies.The results found in the previous section will be discussed according to the specific objectives.The first objective was to evaluate the level of digital competence of a sample of students from three Chilean public universities.The results show that students have an intermediate level of achievement in the five areas of DC of the DIGCOMP framework (55.1%).These results agree with Rodríguez-Garcés & Muñoz-Soto (2018) who point out that for the ICT skills assessment test, 51.3% is an intermediate level.They differ from the study of self-perception of the level of digital skills according to DIGCOMP by Segrera-Arellana (2020) where most university students consider themselves at the advanced level.They also differ slightly from the study by Gutiérrez and Serrano (2016), where students in the first year of primary education pedagogy consider themselves competent in the most basic DC aspects of the DIGCOMP framework.The data agree with Gonzalez et al. ( 2018), who, using the same framework, found that university students generally show an average level of DC.This differs from other studies showing that university students have a basic level of DC (Liesa-Orús et al., 2016;Sánchez-Caballé et al., 2020).
The areas of highest achievement were "network security" and "online communication and collaboration, " whereas "problem solving, " "information and digital literacy, " and "content creation" recorded the lowest values.These results agree with those reported by González-Calatayud et al. (2018), who conducted a study with second year pedagogy students, using the DIGCOMP framework.The university students have a good perception of computer security knowledge and the awareness of performing backups more frequently (Roque Hernández and Juárez Ibarra, 2018).Future teachers, also present good attitudes towards security, but less knowledge, skills and practices related to the safe and responsible use of the Internet (Gallego-Arrufat, et al., 2019).The area of "network security, " coincides with the study by (Gallego-Arrufat et al., 2019), conducted on future undergraduate teachers from Spain and Portugal, where the area of digital security was the best evaluated, because students have good attitudes towards safety, but less knowledge, skills and practices related to safe and responsible use of the Internet.Data for similar studies that used the DIGCOMP framework in pedagogy students are discussed below.The level of achievement of "online communication and collaboration" agrees with the results found by López-Meneses et al. (2020) and Gutiérrez and Serrano (2016), which was high intermediate level.
Within the areas of lower achievement, "information and digital literacy" obtained a score below 50%, which differs from the high intermediate level obtained by students in the study by López-Meneses et al. (2020).Similar results have been reported by Gutiérrez and Serrano (2016) and in the study by Napal-Frailen et al. (2018), where this area records the highest achievement.The results in "content creation" coincide with the studies of López-Meneses et al. (2020) and Gutiérrez and Serrano (2016), who report low levels for this area as does our study.In addition, the study by Napal-Frailen et al. (2018) shows, like our study, that "problem solving" and "content creation" are among the areas of lowest achievement.
The second and third objectives were to study the relationship between socioeconomic level and achievement of DC through the variables of the school where they attended high school and the university they entered.In both, we sought to study possible differences between levels of DC development.
The results show a higher level of achievement in students from private schools (58.9%) and a lower achievement in the municipal sector (53.5%).The area with the highest levels of achievement for the three types of establishments was "network security" and the lowest level was "digital content creation" for private and subsidized private schools, and "problem solving" for municipal schools.Statistically significant differences were found in the scale in the following areas: "information and digital literacy, " "online communication and collaboration, " "network security, " and "problem solving, " with students coming from private establishments having significantly higher scores than those coming from municipal and private-subsidized establishments.
At the University level, the highest level of achievement in students who entered university is found in the central zone of the country (57%) and the lowest, in the university of the north of the country (50.7%).This finding is related to the fact that students with a higher SES obtain higher scores in the university selection tests come from educational establishments with a higher SES and access universities with the best ranking, which are in the central zone of the country.The areas with the highest levels of achievement for the three universities were "network security" and "online communication and collaboration;" the lowest level of achievement for universities in the central and northern zones was "content creation" and for the one in the southern zone, "problem solving." Statistically significant differences were found in the five areas of DIGCOMP, with students from the central zone university having significantly higher scores than students from the northern and southern zone universities.
The results obtained in this study confirm the relationship between the achievement level of ICT competencies and socioeconomic level of Chilean students.SES is directly related to the students' DC level (Claro et al., 2015;Jara, et al., 2015) and is one of the variables that most influence students' DC (Hatlevik et al., 2018).The use of ICT outside school is closely linked to SES, as its use in socioeconomic contexts is more limited (Hollingworth et al., 2011), and having a computer has an impact on the level of DC (Jara et al., 2015).In a study with Iranian university students (Nami & Vaezi, 2018) it was found that students with a personal computer demonstrated higher levels of technological literacy.The low levels of DC in the Chilean student body would be conditioned by the SES, with students from families of low SES displaying lower achievements than those with high SES (Rodríguez & Muñoz, 2018).

Conclusion
The findings of this study show that first year pedagogy students, according to the DIG-COMP framework, have not adequately developed DC.This ratifies the need to diagnose the level of DC through evaluation instruments and develop formative instances to improve their development, integrating DC as part of the curriculum of university programs, especially those of pedagogy, so that students use technology for their academic and personal development.Domingo-Coscolla et al., (2020) indicate that communication and collaboration should be prioritized during the teaching and learning process using digital technologies that favor them.
It is necessary to promote innovative actions in education that consider the use of technologies and developing DC in students (European Commission, 2018).For Guzmán-Simón et al. (2017), university teaching should incorporate DC as part of the academic training of students, especially in students who are preparing to become teachers.Developing DC in the early years is fundamental to achieve DC in the later years (Silva et al., 2019).González-Calatayud et al. (2018) show that after a training process the deficit areas of DC increase considerably.Students with basic DC levels perceive that formative work developed within an ICT subject helps them to improve DC (Gutiérrez & Serrano, 2016).
Measurement of DC is a critical challenge to better understand its development in practice (He & Zhu, 2017).Self-perception tests deliver higher ratings of DC than assessment tests (Gabarda-Méndez et al., 2017), one possibility is to combine them (Rosman, 2015).Therefore, it is appropriate to move forward in the construction of instruments to assess the level of DC, based on existing publications and existing DC frameworks.In this context, it is interesting to use a proven framework to identify and describe DC, such as DIGCOMP.The DIGCOMP-PED instrument used in this study is a good starting point for assessing DC in university students, as it consists of a set of questions that confront the student with concrete situations where the use of DC is necessary for its solution.
Among the corrective measures to improve the level of digital competence of students regardless of their socioeconomic level, is the training, which considers the areas of digital competencies according to DIGCOMP, each area can be a module or all a course.There is the experience of the development of digital competencies of university teachers according to the DigCompEdu Framework through a MOOC (Cabero-Almenara, et al., 2021).MOOCs contribute to the 2030 agenda for sustainable development because they can guarantee inclusion, as they are massive, open, free and accessible, in addition they develop the competences of autonomous work.
The main limitations of the study are related to the instrument in relation to the number of items, it would be advisable to consider a larger base of items and their validation with a greater number of experts.Another aspect to consider is the item alternatives, which only took into account one correct answer.The instrument was answered voluntarily online, therefore, there is not an equal representation of the different areas of teacher education in each university.
As proposals for the future, we consider interesting: • To address improvements to the instrument, expanding the questions for each indicator in future research.• Apply the instrument to university students from other areas of knowledge, such as engineering, medicine, and law, among others.• To carry out comparative studies between public and private, Latin American, and European universities in the same country since this subject is of growing interest.

Fig. 1
Fig. 1 Example of an item

Fig. 3
Fig. 3 Level of achievement in the areas of digital competence

Table 1
Sample characterization.Source: Prepared by the autor

Table 2
Area and Indicator DIGCOMP.Source: Prepared by the author The area of lowest achievement Digital content creation presents 4 indicators, the highest mean is presented by the Content

Table 3
Descriptive statistics for DigComp-PED areas and indicators.Source: Prepared by the author

Table 4
Digital Competence level by type of establishment.Source: Prepared by the author

Table 5
DC level by university admittance.Source: Prepared by the author