Skip to main content

Indicators of pedagogical quality for the design of a Massive Open Online Course for teacher training

Indicadores de calidad pedagógica para el diseño de un curso en línea masivo y abierto de actualización docente

Abstract

Massive Open Online Courses (MOOCs) have generated high expectations and revolutionized some educational practices by providing open educational resources for reference, usage and adaptation; therefore, their pedagogical quality is often questioned. The objective of this study is to identify indicators related to pedagogical, functional, technological and time factors in order to assess the quality of the MOOC entitled “Liderazgo en gestión educativa estratégica a través del uso de la tecnología” (Leadership in strategic educational management through the use of technology), offered as a teacher training program through Coursera to 10.161 participants. Via the Delphi method, a group of 55 experts agreed that time is a key factor to be considered in the design of learning activities. It was concluded that without measuring results, the success of a MOOC could not be evaluated; thus, institutions and consortia must establish evaluation indicators to focus their efforts on the enhancement of pedagogical quality. By providing relevant information, the learning potential of educational resources based on connectivism principles can be evaluated, and so can the quality of MOOCs. The goal is to contribute to a vision of a future in which everyone has access to a world-class education.

Resumen

Los cursos en línea, masivos y abiertos (MOOC) han generado importantes expectativas y han revolucionado algunas prácticas educativas, al ofrecer recursos educativos abiertos para su consulta, uso y adaptación; sin embargo, con frecuencia se cuestiona su calidad pedagógica. El objetivo de este estudio es identificar indicadores relacionados con factores pedagógicos, funcionales, tecnológicos y de tiempo, para evaluar la calidad del MOOC Liderazgo en gestión educativa estratégica a través del uso de la tecnología, ofrecido como recurso de actualización docente en Coursera a 10.161 participantes. Mediante el método Delphi, un grupo de 55 expertos acordó que el tiempo es un factor clave a considerarse en el diseño de las actividades de aprendizaje. Se concluye que sin medición de los resultados no se puede valorar el éxito de un MOOC, por ello instituciones y consorcios deben establecer indicadores de evaluación para enfocar sus esfuerzos para la mejora de su calidad pedagógica. Si se proporciona información relevante se podrá evaluar el potential de aprendizaje que poseen los recursos educativos basados en principios conectivistas y reconocer la calidad pedagógica de los MOOC, con el objetivo de coadyuvar a la visión de un futuro en el que todos tengan acceso a una educación de clase mundial.

References

  1. Aceto, S., Borotis, S., Devine, J., & Fischer T. (2014). Mapping and Analysing Prospective Technologies for Learning. Seville, Spain: Joint Research Centre, Institute for Prospective Technological Studies.

    Google Scholar 

  2. Arias, J. (2007). Evaluación de la calidad de Cursos Virtuales: Indicadores de Calidad y construcción de un cuestionario de medida (Doctoral dissertation). University of Extremadura, Extremadura, Spain.

    Google Scholar 

  3. Astigarraga, E. (2003). El método delphi. San Sebastián, Spain: Universidad de Deusto.

    Google Scholar 

  4. Barbera, E., Gros, B., & Kirschner, P. (2012). Temporal issues in e-learning research: A literature review. British Journal of Educational Technology, 43(2), 53–55. doi: 10.1111/j.1467-8535.2011.01255.x

    Article  Google Scholar 

  5. Bell, M. (2012). Massive open online courses moving ahead with MOOCs. Internet@Schools, 19(5). Retrieved from http://www.internetatschools.com/Articles/Column/Belltones/BELLTONES-Massive-Open-Online-Courses—Moving-Ahead-With-MOOCs-85936.aspx

  6. Bernal, Y., Molina, M., & Pérez, M. (2013). La Calidad de la Educación a Distancia: El caso de los MOOC. Revista Iberoamericana para la Investigación y el Desarrollo Educativo, 3(10), 1–13. Retrieved from http://www.ride.org.mx/docs/publicaciones/10/psicologia_y_educacion/E06_.pdf

    Google Scholar 

  7. Boven, D. (2013). The Next Game Changer: The Historical Antecedents of the MOOC Movement in Education. eLearning Papers, 33, 1–7. Retrieved from http://www.openeducationeuropa.eu/en/download/file/fid/26967

    Google Scholar 

  8. Breslow, L., Pritchard, D., DeBoer, J., Stump, G., Ho, A., & Seaton, D. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice in Assessment Journal, 8, 13–25. Retrieved from http://www.rpajournal.com/dev/wp-content/uploads/2013/05/SF2.pdf

    Google Scholar 

  9. Brown, S. (2013). Back to the future with MOOCs. ICICTE 2013 Proceedings, 237–246. Retrieved from http://www.icicte.org/Proceedings2013/Papers%202013/06-3-Brown.pdf

  10. Cabero, J., & Romero, R. (2007). Diseño y producción de TIC para la formación. Barcelona, Spain: UOC.

    Google Scholar 

  11. Chiecher, A., & Donolo, D. (2013). Trabajo grupal mediado por foros. Aportes para el análisis de la presencia social, cognitiva y didáctica en la comunicación asincrónica. In A. C. Chiecher, D. S. Donolo, & J. L. Córica (Eds.), Entornos virtuales y aprendizaje. Nuevas perspectivas de estudio e investigaciones (pp. 151–198). Mendoza, Argentina: Editorial Virtual Argentina.

    Google Scholar 

  12. Cross, S. (2013). Evaluation of the OLDS MOOC curriculum design course: participant perspectives, expectations and experiences. OLDS MOOC Project, Milton Keynes, United Kingdom.

  13. Domingo, M., & Marquès, P. (2011). Classroom 2.0 Experiences and Building on the Use of ICT in Teaching. Comunicar, 18(37), 169–174. http://dx.doi.org/10.3916/C37-2011-03-09

    Article  Google Scholar 

  14. EDUCAUSE (2011). 7 things you should know about MOOCs. EDUCAUSE Learning Initiative.

  15. European Foundation for Quality in e-Learning [EFQUEL] (2013). The MOOC Quality Project. Retrieved from http://mooc.efquel.org/the-mooc-quality-project/

  16. Franco-Casamitjana, M., Barbera, E., & Romero, M. (2013). A Methodological Definition for Time Regulation Patterns and Learning Efficiency in Collaborative Learning Contexts. eLC Research Paper Series, 6, 52–62. Retrieved from http://journals.uoc.edu/index.php/elcrps/article/download/1871/n6-franco-casamitjana

    Google Scholar 

  17. Friedman, T. (2013, January 26). Revolution hits the universities. The New York Times, The opinion pages. Retrieved from http://www.nytimes.com/

  18. Garrido, A. (2003). El aprendizaje como identidad de participación en la práctica de una comunidad virtual (Doctoral dissertation). Universitat Oberta de Catalunya, Barcelona.

    Google Scholar 

  19. Gómez-Zermeño, M. G. (2012). Bibliotecas digitales: recursos bibliográficos electrónicos en educación básica. Comunicar, 20(39), 119–128. Retrieved from http://www.revistacomunicar.com/index.php?contenido=detalles&numero=39&articulo=39-2012-14

    Article  Google Scholar 

  20. Gómez-Zermeño, M. G., & Alemán, L. Y. (2012). Administración de proyectos de capacitación basados en tecnología. Monterrey, Mexico: Editorial Digital del Tecnológico de Monterrey.

    Google Scholar 

  21. Gómez-Zermeño, M. G., Rodríguez, J. A., & Márquez, S. (2013). Estudio Exploratorio-Descriptivo “Curso Híbrido: Contabilidad V”. Revista de Investigación Educativa de la Escuela de Graduados en Educación, 4(7), 70–79. Retrieved from http://rieege.tecvirtual.mx/index.php/rieege/article/view/126

    Google Scholar 

  22. Gros, B., Barbera, E., & Kirschner, P. (2010). Time factor in e-Learning: impact literature review. eLC Research Paper Series, 0, 16–31. Retrieved from http://journals.uoc.edu/index.php/elcrps/article/download/issue0-gros-barbera-kirshner/issue0-gros-barbera-kirshner

  23. Guitert, M. (2011). Time management in virtual collaborative learning: the case of the Universitat Oberta de Catalunya (UOC). eLC Research Paper Series, 2, 5–16. Retrieved from http://journals.uoc.edu/index.php/elcrps/article/download/n2-guitert-catasus/n2-guitert-catasus

    Google Scholar 

  24. ITESM (2014). Formación que transforma vidas. Monterrey, Mexico: Tecnológico de Monterrey.

    Google Scholar 

  25. Liyanagunawardena, T., Adams, A., & Williams, S. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distance Learning, 14(3), 202–227. Retrieved from http://www.irrodl.org/index.php/irrodl/article/download/1455/2602

    Google Scholar 

  26. McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice. Digital ways of knowing and learning. Charlottetown, Canada: University of Prince Edward.

    Google Scholar 

  27. Menéndez, L. (2013). Estudiar ‘online’ y gratis. Escritura Pública, 80, 18–21. Retrieved from http://www.notariado.org/liferay/c/document_library/get_file?folderId=12092&name=DLFE-89971.pdf

    Google Scholar 

  28. Moore, M. G. (1983). Theory of transactional distance. In D. Keegan (Ed.), Theoretical Principles of Distance Education (pp. 22–38). New York, USA: Routledge.

    Google Scholar 

  29. Moore, M. G. (2013). The theory of transactional distance. In M. G. Moore (Ed.), Handbook of distance education (3rd ed., pp. 66–85). New York, USA: Routledge.

    Google Scholar 

  30. Observatorio de Innovación Educativa (May, 2014). Reporte Edu Trends. Monterrey, Mexico: Tecnológico de Monterrey. Retrieved from http://www.observatorioedu.com/redutrends

    Google Scholar 

  31. Penn GSE (2013). Study shows MOOCs have relatively few active users, with only a few persisting to course end. University of Pennsylvania: Graduate School of Education. Retrieved from http://www.gse.upenn.edu/pressroom/press-releases/2013/12/penn-gse-study-shows-moocs-have-relatively-few-active-users-only-few-persisti

  32. Rodriguez, O. (2012). MOOCs and the AI-Stanford like Courses: two successful and distinct course formats for massive open online courses. European Journal of Open, Distance, and E-Learning, 2012(2), 1–13. Retrieved from http://www.eurodl.org/materials/contrib/2012/Rodriguez.pdf

    Google Scholar 

  33. Roig, R., Flores, C., Álvarez, J., Blasco, J., Grau, S., Guarinos, I., … Tortosa, M. (2013). Características de los ambientes de aprendizaje on-line para una práctica docente de calidad. Indicadores de evaluación. Alicante, Spain: Universidad de Alicante.

    Google Scholar 

  34. Romero, M. (2011). The time factor in an online group course from the point of view of its students. eLC Research Paper Series, 2, 17–28. Retrieved from http://journals.uoc.edu/index.php/elcrps/article/download/n2-romero/n2-romero

    Google Scholar 

  35. Sangrà, A., & Wheeler, S. (2013). New Informal Ways of Learning: Or Are We Formalising the Informal? RUSC. Universities and Knowledge Society Journal, 10(1), 286–293. http://dx.doi.org/10.7238/rusc.v10i1.1689

    Article  Google Scholar 

  36. Schmelkes, S. (2001). La investigación en la innovación educativa. Mexico: CINVESTAV.

    Google Scholar 

  37. SCOPEO (2013). MOOC: Estado de la situación actual, posibilidades, retos y futuro. SCOPEO Report No 2, June 2013. Retrieved from http://scopeo.usal.es/wp-content/uploads/2013/06/scopeoi002.pdf

  38. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, USA: Houghton Mifflin.

    Google Scholar 

  39. Siemens, G. (2004). Connectivism. A learning theory for the digital age. International Journal of Instructional Technology and Distance learning, 2(1), 3–10. Retrieved from http://www.itdl.org/journal/jan_05/Jan_05.pdf

    Google Scholar 

  40. Skiba, D. (2013). MOOCs and the Future of Nursing. Nursing Education Perspectives, 34(3), 202–204. doi: 10.5480/1536-5026-34.3.202

    Article  MathSciNet  Google Scholar 

  41. The New Media Consortium & Universitat Oberta de Catalunya (2012). Perspectivas tecnológicas: educación superior en Iberoamérica 2012–2017: Un Análisis Regional del Informe Horizon del NMC y la UOC. Barcelona, Spain: UOC.

    Google Scholar 

  42. Tschofen, C., & Mackness, J. (2012). Connectivism and dimensions of individual experience. The International Review of Research in Open and Distance Learning, 13(1), 124–143. Retrieved from http://www.irrodl.org/index.php/irrodl/article/download/1143/2117

    Google Scholar 

  43. UNESCO (2012). 2012 Paris OER Declaration. 2012 World Open Educational Resources (OER) Congress. Paris, France: UNESCO.

    Google Scholar 

  44. Zapata-Ros, M. (2013). MOOCs, una visión crítica y una alternativa complementaria: La individualization del aprendizaje y de la ayuda pedagógica. Universidad de Alcalá de Henares, Spain. Retrieved from http://eprints.rclis.org/18658/7/MOOC_zapata_preprint.pdf

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Lorena Yadira Alemán de la Garza.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Alemán de la Garza, L.Y., Sancho-Vinuesa, T. & Gómez Zermeño, M.G. Indicators of pedagogical quality for the design of a Massive Open Online Course for teacher training. Int J Educ Technol High Educ 12, 104–118 (2015). https://doi.org/10.7238/rusc.v12i1.2260

Download citation

Keywords

  • Massive Open Online Courses
  • MOOC
  • evaluation
  • pedagogical quality
  • teacher training

Palabras clave

  • cursos en línea masivos y abiertos
  • MOOC
  • evaluación
  • calidad pedagógica
  • actualización docente
\