Meta-analysis of research in e-learning published in Spanish journals

The data on the year of publications show that similar values were found in the years analyzed (2011–2015), although there was a greater number of articles in 2012 (22.2 %) and a lesser number in 2011 (17.6 %), while the quantity was the same in the years 2013 and 2014 (19.4 %), with the articles published in 2015 contributing 21.3 % of the total.

The journals’ impact factor data shows that only one of the journals was indexed in JCR (f = 1, 0.9 %), the same as in JCR and SCOPUS combined (f = 1, 0.9 %), while most of the journals had the FECYT Seal (f = 42, 38.9 %), and only 24.1 % (f = 26) of the articles were found in journals that were both indexed by SCOPUS and had the FECYT Seal.

When classifying the articles by the numbers of authors, the articles with three authors totaled 33.3 %, followed by two (29.6 %) and more than 3 (19.4 %), with the least being the articles written by only one author (17.6 %).

The innovation and investigation experiments found in the articles were directed towards university surroundings (87.0 %); with a lesser number found for the institution-administration pair (0.9 %); those that had variable subjects of study (3.7 %), while those that were impossible to identify added up to 2.8 %.

Most of the studies were grouped under the label “various” (44.4 %), the descriptive studies added up to 27.8 % and the experimental studies to 23.1 %, with the case studies being the less addressed (4.6 %).

With respect to the research methodology employed, those that used quantitative methods were 51.9 %, only 6.6 % employed a mixed method of analysis; likewise, 21.7 % described projects, and 19.8 % used a qualitative methodology.

As for topics studied within the different subjects, we found that from greater to lesser frequency, the results found were: academic performance (f = 218; 22.45 %); learning, teaching and cognitive styles (f = 212; 21.83 %); interaction, communication (f = 165; 16.99 %); others (f = 1140; 14.42 %); degree of satisfaction (f = 57; 5.87 %); development of competencies (f = 55; 5.66 %); assessment (f = 48; 4.94 %); curricular integration into diverse areas (f = 41; 4.22 %); and attitudes/perception (f = 35; 3.60 %).

The topic studied results within the subject of “Design of materials and/or digital objects”, showed that studies that focused on the sub-category “academic performance” predominated (34.3 %), followed by “learning styles” (22.2 %). The least studied were those referring to “assessment” (1.9 %) (Fig. 1).

On the subject of “E-learning environments, distance learning platforms”, the topics related to “academic performance (20.4 %) and “learning styles” (25.0 %) were emphasized, with the topic of “assessment” being the least studied (3.7 %) (Fig. 2).

On the subject of “Tutorships and advice used in e-learning activities”, the topics “interaction and communication (39.3 %) predominated, with the “curricular integration into diverse areas” (1.9 %) being the least taken on (Fig. 3).

As for the subject “Didactic strategies and methodologies used in e-learning activities”, the topics “academic performance” (21.3 %) and “learning styles” (33.3 %) stood out (Fig. 4).

Within the subject “Use of synchronous and asynchronous communication tools in e-learning activities”, the effects of “interaction and communication” (28.7 %) were predominant, with those referring to “curricular integration into diverse areas” (1.9 %) being the less studied (Fig. 5).

On the subject “Use of assessment techniques and strategies in e-learning activities”, the studies on the topics of “academic performance” and “assessment” predominated (both 21.3 %), with those referring to “curricular integration to diverse areas” being the less attitudes and perceptions (1.9 %) (Fig. 6).

On the subject of “Organizational and institutional aspects related to e-learning activities”, research on “academic performance” (30.6 %) was highlighted, with those referring to “assessment” being less common (2.8 %) (Fig. 7).

For the subject “Collaborative and cooperative activities used in e-learning activities”, the most investigated topics were “interaction and communication” (27.8 %) followed by “academic performance (22.2 %), while the least researched was “attitudes and perceptions” (0.9 %) (Fig. 8).

Lastly, the results on the articles referring to “Studies on accessibility and usability”, indicated that “academic performance (22.2 %) and “learning styles” (24.1 %) predominated, while “attitudes and perceptions” (2.8 %) was the least researched (Fig. 9).

The comparison between the journals’ indexation with the type of study the article investigated were found to be related, with a Chi-square = 44.056 (p < 0.001). As the contingency table for the statistical analysis was 6x4, other association statistics were needed, as pointed by Rodríguez and Mora (2001), in order to re-enforce results. Values such as Phi (0.639, p < 0.001), Cramer’s V (0.369 and p < 0.001), with the Coefficient of contingency (0.538 and p < 0.001) were calculated, and they all showed that there was a relationship between both variables.

To understand the proportional reduction in the error, when using the values of the independent variable (journal indexing) to forecast the values of the dependent variable (type of study), the nominal statistic Goodman-Kruskal Tau was applied, obtaining a value of 0.821, p < 0.001 (dependent), and a value for the Coefficient of uncertainty (Cu) of 0.812 (p = 0.021). This means that knowing one variable reduces the error 80 % when forecasting the values of the other.

As can be observed in Fig. 10, the experimental studies were found in those journals that had all the impact factors (44 %), while the descriptive studies were gathered primarily in the journals that had been awarded the FECYT Seal (30 %), just as those labeled various (56.3 %). On the other hand, the ones containing case studies were found in SCOPUS and FECYT journals combined (40 %).

To unify and homogenize the criteria found in the articles selected, a comparison between the subject “Design of materials and/or digital objects used in e-learning activities”, with the type of method used in the article was performed, taking into account the existence of a relationship between both aspects. The Chi-square value was 58.714 (p < 0.001), with the Phi (0.744, p < 0.001), Cramer’s V (0.430, p < 0.001) and a Coefficient of contingency (0.597, p < 0.001), with these values indicating that there is a relationship among all the variables.

The Tau value was 0.831 (p < 0.001) for the dependent variable, with a Cu value of 0.702 (p < 0.001). This signifies that the knowledge of a variable reduces 70 % of the error when forecasting the values of the other.

The results of this comparison are shown in Fig. 11, indicating that the sub-category of academic performance studied within the subject “Design of materials and/or digital objects used in e-learning activities” and assessment were mainly studied a using quantitative methodology (50.9, and 3.6 %, respectively); attitudes and perceptions and the degree of satisfaction used mixed typologies (both 28.6 %). The interaction and communication used description of projects (21.7 %), in the same manner as that relative to curricular integration into diverse areas (8.7 %) and development of competencies (17.4 %) and others (17.4 %), while the styles of learning used qualitative methodologies (42.9 %).

Likewise, when addressing the subject “E-learning environments, distance learning platforms” with the type of method used in the article, we observed the existence of a relationship between both aspects, with a Chi-square of 42.590, p = 0.011, with values of Phi (0.634, p = 0.011), Cramer’s V (0.456, p = 0.011) and a Coefficient of contingency (0.535, p = 0.011), signifying that there was a relationship between both variables.

The Tau value was 0.731 (p = 0.005) for the dependent variable; however, the symmetric “Cu” obtained was not significant (p = 0.186). Therefore, knowledge of one variable did not reduced the error when forecasting the value of the other.

Data gathered on this comparison are shown in Fig. 12, indicating that the sub-category “academic performance” (29.1 %) and “assessment” (7.3 %) within the subject “E-learning environments and distance learning platforms” were elaborated mostly using quantitative methods; the “attitudes and perceptions” (14.3 %), “degree of satisfaction” (28.6 %), and the “development of competencies” (28.6 %) utilied mixed typologies; the “interaction and communication” used description of projects (21.7 %), likewise the methods used relative to the “curricular integration into diverse areas” (13 %) and “others” (17.4 %); while “learning styles” used qualitative methodologies (47.6 %).

With respect to the subject “Tutorships and advice used in e-learning activities” with the comparison of the type of method used in the article, we observed that there was a relationship between both aspects, Chi-square = 47.774 (p = 0.003), with values of Phi (0.675, p = 0.003), Cramer’s V (0.423 and p = 0.003), with the “Coefficient of contingency (0.559, p = 0.003), meaning that there was a relationship between both variables.

The Tau values were 0.631 (p < 0.001) for the dependent variable; however, those (Fig. 13) obtained for the symmetric “Cu” values were not significant (p = 0.230), therefore the knowledge of one variable did not reduce the error when forecasting the values of the other.

The data showed that in the articles selected, for the subject “Tutorships and advice used in e-learning activities”, quantitative methodologies were mostly used in the sub-categories “academic performance” (20.0 %), “attitudes and perceptions” (1.8 %), “assessment” (5.5 %) and the “curricular integration into diverse areas” (3.6 %). On the other hand, those that had a more qualitative approach were observed in “learning styles” (47.6 %), as was already found with the rest of the subjects. The results related to “interaction and communication” (63.6 %) used description of projects, just as the development of competencies (13.6 %) and others (18.2 %) did. And lastly, the “degree of satisfaction” used mixed typologies (28.6 %).

However, when analyzing the subject “Didactic strategies and methodologies used in e-learning activities” with a comparison of the type of methodology used in the article, we observed that there was no relation between them, as the Chi-square values were 31.087, p = 0.151, which were not significant.

Likewise, when looking at the subject “Use of synchronous and asynchronous communication tools in e-learning activities” compared to the type of methodology utilized in the article, we observed that there was a relationship between both aspects, with a Chi-square equal to 62.806 (p < 0.001) with values of Phi (0.770, p < 0.001), Cramer’s V (0.444, p < 0.001) and a Coefficient of contingency (0.610, p < 0.001), which implied that both variables were related.

The Tau value obtained was 0.831 (p = 0.025) for the dependent variable; however, the values obtained for the symmetrical “Cu”, were not significant (p = 0.201). Therefore the knowledge of a variable did not reduce the error when forecasting the values of the other.

Figure 14 shows the data obtained in the comparison, which demonstrate that the sub-category “academic performance” within the subject “Use of synchronous and asynchronous communication tools in e-learning activities” mostly used quantitative methodologies (25.5 %), just as “assessment” (7.3 %). The “curricular integration into diverse areas” (3.6 %), “attitudes and perceptions” (28.6 %), “degree of satisfaction” (28.6 %) and the “development of competencies” (14.3 %) used mixed typologies; while “interaction and communication” utilized description of projects (60.9 %), while at the same time, “others” and “learning styles” used qualitative methods (17.4 and 42.9 %, respectively).

In relation to the subject “Use of assessment techniques and strategies in e-learning activities” compared to the type of methodology used in the article, we observed that a relationship between both aspects existed, with values of Chi-square = 41.716, p = 0.014, Phi =0.627, p = 0.014), Cramer’s V (0.401, p = 0.014) and Coefficient of contingency (0.572, p = 0.014), indicating that there was a relationship between both variables.

The Tau value was 0.712 (p = 0.008) for the dependent variable, and the symmetrical “Cu” was significant at p < 0.001 with a value of 0.902. Therefore the knowledge of one variable reduced the error 90 % when forecasting the value of the other variable.

The data showed that in the articles selected for the subject “Use of assessment techniques and strategies in e-learning activities”, none of the topics studied greatly a quantitative methodology. However, the qualitative methodology was used in the sub-categories “academic performance” (33.3 %), “learning styles” (33.3 %). The “attitudes and perceptions” (14.3 %), the “degree of satisfaction” (28.6 %), the “curricular integration into diverse areas” (14.3 %) and “others” (14.3 %) mostly used mixed methodologies (Fig. 15).

The subject “Organizational and institutional aspects related to e-learning activities”, along with the type of methods used in the articles, were found to be related, with a Chi-square = 48.850 (p = 0.002), with values of Phi (0.679 and p = 0.002), Cramer’s V (0.424, p = 0.002) and Coefficient of contingency (0.569, p = 0.002), all of which point to a relationship between both variables.

The Tau value was 0.724 (p < 0.001) for the dependent variable, with a symmetric “Cu” which was significant, as shown by p = 0.038, and a value of 0.623. Therefore the knowledge of a variable reduces the error 62 % when forecasting the values of the other.

The results of the comparison of the subject “Organizational and institutional aspects related to e-learning activities” indicated that the articles did not use a quantitative methodology to a great extent. For qualitative methodologies, on the other hand, we found the “academic performance” (42.9 %) and “learning styles” (28.6 %). “Attitudes and perceptions” (14.3 %), “degree of satisfaction” (28.6 %), “curricular integration into diverse areas” (28.6 %) and “development of competencies” (14.3 %) used mixed methodologies (Fig. 16).

As for the comparison results of the subject “Collaborative and cooperative actions used in the e-learning activities” with type of methods used in the articles, we observed that there was a relationship between both aspects with Chi-square =46.327 (p = 0.004), values of Phi (0.661, p = 0.004), Cramer’s V (0.452, p = 0.004), and a Coefficient of contingency (0.543, p = 0.004), indicating that there was a relationship between both variables.

The Tau value reached was 0.653 (p < 0.001) for the dependent variable, with a symmetric “Cu” value of 0.836 (p = 0.048), a value that implied that the knowledge of one variable reduced 83 % of the error when forecasting the values of the other.

The data showed that on the articles selected under the subject “Collaborative and cooperative actions used in e-learning activities”, the the sub-categories of “attitudes and perceptions” (1.8 %) and “assessment” (3.6 %) were found using quantitative methods, while for the qualitative methods, we found it to be used in the sub-category “academic performance” (38.1 %) and “learning styles” (28.6 %). The degree of satisfaction (28.6 %) used mixed methodologies.

Lastly, the subject “Specific studies on accessibility, usability and their effects on e-learning activities” with the comparison of the type of methodology used in the articles, we observed the existence of a relationship between both aspects as shown by a Chi-square of 43.734 (p = 0.008), with values of Phi (0.645, p = 0.008), Cramer’s V (0.413, p = 0.008) and a Coefficient of contingency (0.542, p = 0.008), with a relationship therefore present (Fig. 17).

The Tau value was 0.639 (p = 0.001) for the dependent variable; however, the results obtained in the symmetric “Cu” were not significant (p = 0.127), therefore the knowledge of one variable did not reduce the error when forecasting the values of the other.

The data showed that in the articles selected for the subject “Specific studies on accessibility, usability and their effects on e-learning activities”, the sub-category of “assessment” (3.6 %), were observed using quantitative methods. While qualitative methods were used on “academic performance” (33.3 %) and “learning styles” (33.3 %). The description of projects were used mainly on the topics of “interaction and communication” (30.4 %), “curricular integration into diverse areas” (4.3 %), and “others” (17.4 %). Lastly, the topics “attitudes and perceptions” (42.9 %), “degree of satisfaction” (28.6 %) and “development of competencies” (14.3 %) used mixed methodologies (Fig. 18).