Integration of good practices of active methodologies with the reuse of student-generated content

Abstract

In this article, we present an integrated instructive methodological approach. We begin with a set of proposals for educational innovation oriented towards active learning that have been tested separately and implemented for various subjects in courses of different levels. The approach integrates the following elements: (1) the dynamic generation of digital content by students and their integration into shared knowledge bases of the subjects involved; (2) the systematic use of quality content, mainly in video format, distributed through online platforms as support for flipped classrooms; (3) peer evaluation to support the development of reflective and self-critical capacities; and (4) systematic collaboration with students and professors from other universities to develop the enumerated activities. The methodology has been tested in a variety of subjects, thanks to its flexibility. In all experienced cases, it has been shown that it is feasible for students to generate enough valuable and reusable content. In addition, students have expressed high levels of satisfaction with the implementation of the proposal.

Introduction

In recent years, a main educational goal has been to ensure that students assume a more significant role in the entire educational process and thus become active agents in configuring their education (Bovill, Cook-Sather, Felten, Millard, & Moore-Cherry, 2016; Seale, 2009). Consequently, different active teaching methodologies have been introduced, such as project-based learning, problem-based learning, and the flipped classroom (O’Flaherty & Phillips, 2015). However, the curricular and pedagogical design and grading remain largely being teachers’ tasks, though steps are being taken to involve students in the evaluation process through peer assessment, while incorporating formative goals.

Continuing this trend, efforts have been made to involve students in developing learning objects. As co-generators of subject content, students become co-creators too. In the broadest sense, co-creation is a collaborative, reciprocal process through which participants, namely, teachers and students, can contribute equally, though not necessarily identically, to curricular or pedagogical conceptualization, decision-making, implementation, research or analysis (Bovill et al., 2016).

The advantages of co-creation range from the design and planning of tasks by students taking higher-level courses to be conducted by students in lower-level courses (Bovill et al., 2016) to specific digital learning objects generally shared with the group, such as glossaries, blogs or wikis (Kalayci & Humiston, 2015; Yang, Guo, & Yu, 2016), concerns or questions (Aflalo, 2018; Yu & Wu, 2016) and videos (Chewar & Matthews, 2016; Orús et al., 2016). Co-creation requires developing higher order skills in Bloom’s taxonomy (Anderson & Krathwohl, 2001) because it requires deeper metacognitive understanding.

Our proposal does not use co-creation as an isolated element, rather combines it with other active learning methodologies, such as the flipped classroom and peer assessment. Peer assessment increases motivation, facilitating the development of assessment skills and encouraging learning, based on the observation of work performed by peers (Chen, Wei, Wua, & Uden, 2009; Gielen, Dochy, & Onghena, 2011) by contrasting one’s solutions with others’ solutions, either to emulate good practices or to avoid mistakes or inappropriate approaches. Some of these pedagogical reasons are also considered metacognitive and affective. In the current context, this peer assessment is unrelated to the evaluation of the subject itself.

The reuse in later courses of the materials created is another differentiating element of this proposal. This form of collaboration offers an additional benefit for students, as their self-confidence improves through their contribution to shaping the content of a course for future students.

Working with learning objects already created as reflection elements or those associated with certain activities is the foundation of the flipped classroom (O’Flaherty & Phillips, 2015).

The final step in implementing our proposed methodology involves collaboration among teachers and in different subjects. The experience of each person, including their mistakes and lessons learned, are important for implementing any methodological innovation. When collaboration with other universities is also possible, the experience is even more enriching.

In “Related work” section, we review the methodologies related to our proposal: the flipped classroom, peer assessment, the co-creation of new content and its reuse in subsequent courses. Proposed methodology section details our pedagogical approach to combining these four teaching techniques. Application of the methodology section explains how this approach has been used for six subjects during several academic years. Results section gathers and analyses the results of our pedagogical experience, which is discussed in the following section. The article finishes with a conclusions section.

Related work

Currently, there is a set of information and communication technologies (ICTs) available for improving student learning, whose use is considered essential to increase student satisfaction (O’Flaherty & Phillips, 2015). ICTs increase students’ ease of access to online resources before attending class (Ng, 2018). These resources can be generated by the teachers themselves, although freely accessible materials can also be used (e.g., YouTube, Khan Academy, Ted Talks, MIT Open Courseware). With the proliferation of Web 2.0 technologies, digital content can be reused and created in forms such as wikis, blogs, reports, videos and images. We can define user-generated content broadly, but in this context, we mean specifically student-generated content (SGC) (Yang et al., 2016). ICTs have also modified the evaluation process, from unidirectional to multidirectional (self, peer and teacher). Electronic questionnaires enable instant feedback and can also be used to ascertain students’ prior knowledge before attending class.

Flipped classroom

Since teaching no longer focuses on the teacher’s knowledge but on the active participation of the student, the flipped classroom (FC) methodology is increasingly used (Lundin, Rensfeldt, Hillman, Lantz-Andersson, & Peterson, 2018). This method is inspired by the idea of inverting traditional teaching by replacing a teacher-centred approach by an approach in which activities previously performed outside the classroom now occur within the classroom (Akçayır & Akçayır, 2018; Lundin et al., 2018; Nouri, 2016; O’Flaherty & Phillips, 2015) That is, instead of attending class and working at home afterwards, now, students first access the material (through formats such as videos, texts, links, and repositories) and then conduct different learning activities in the classroom (e.g., analysis, discussions, problem solving, answering questionnaires, learning advanced concepts). Hence, autonomous learning is encouraged through the completion of preparatory work, while class time is more interactive and productive (O’Flaherty & Phillips, 2015).

Many research and review articles have named the advantages of the FC, particularly in a recent work by Akçayır and Akçayır (2018). In terms of learning outcomes: the FC improves learning and enhances student satisfaction and, to a lesser extent, enhances students’ confidence, promotes their creativity and improves their problem-solving abilities. Student satisfaction improves because the FC allows greater interaction with the teacher and other students and facilitates more individualized learning. Most importantly, though, the FC modifies the students’ learning habits. Some studies highlight the fact that students are better prepared before class. However, the lack of preparation of certain students before class, either due to difficulties or lack of time or motivation, has also been identified as the most common disadvantage of the FC. To take advantage of the benefits of the FC and ensure adequate pre-class preparation, students must previously complete a brief task or a questionnaire (le Roux & Nagel, 2018). Some of the tasks associated with the FC according to our proposed methodology also involve evaluating the content consulted or proposing questions about the content.

This model requires a greater investment of time from both students and teachers. The use of pre-existing resources such as Khan Academy videos (Khan Academy Free Online Courses, Lessons & Practice, 2019) or material from other institutions or social media can reduce the teacher’s workload (O’Flaherty & Phillips, 2015). Our proposal also includes reusable SGC.

The effect of the FC depends on its specific implementation (Schwarzenberg, Navon, Nussbaum, Pérez-Sanagustín, & Caballero, 2018). An important task is to specify pre-class and class activities (Bishop & Verleger, 2013; Le Roux & Nagel, 2018). Students increase their commitment because of the activities conducted in class, not because of the FC (Le Roux & Nagel, 2018). That is, to ensure the FC is effective, it is more important to focus on the selection of active learning strategies to be used in class than on developing online videos or other out-of-class materials. It is necessary to consider which activities will be most conducive to learning and whether such activities are better performed before, during or after the face-to-face session, and the types of interactions with the material and between the students that are desirable. In our proposal, we use FC along with other techniques such as SGC and PR. For example, as a pre-class activity, the generation of questions can be proposed and as an activity to be carried out in class, the answer to the proposals made by other students. If the material used for FC has been made by other students, the PR can be proposed as a previous activity and the generation of new content as a subsequent activity.

Peer review

Peer assessment or peer review (PR) is the process whereby students evaluate the work or achievements of their peers. The assessment can consider the level, merit, quality or success of the products or the learning outcomes of colleagues in analogous situations (Topping, 2003). There is no single way to implement PR (Gielen et al., 2011). The objective can be formative but need not be. It can be performed quantitatively or qualitatively, with or without feedback. All students can be evaluators and can be evaluated, or only some students can evaluate all their peers or some of their peers, and evaluations may or may not be performed anonymously. In recent years, interest in peer- and self-review as a method to encourage collaboration among students and to promote greater student involvement in the learning-teaching process has grown (Agrawal & Rajapakse, 2018; Chang, Tseng, & Lou, 2012; Chen et al., 2009; Domínguez, Jaime, Sánchez, Blanco, & Heras, 2016; Ion, Barrera-Corominas, & Tomàs-Folch, 2016; Jaime et al., 2016; Li et al., 2016; Panadero & Brown, 2017; Wanner & Palmer, 2018).

The use of ICTs also favours implanting the PR through online tools that allow assigning, collecting and communicating the students’ assessments immediately (Li, Liu, & Steckelberg, 2010; Søndergaard & Mulder, 2012). In certain cases, ICTs are used for practical reasons because the PR enables providing sufficient feedback in a reasonable time when working with a considerable number of students, as in massive open online courses (MOOCs), or facilitating individual assessment within a work team (Falchikov, 2004; Willmot, Pond, Loddington, & Palermo, 2008). Beyond the interest in obtaining different peer grades and quickly, the PR is of interest for formative and collaborative learning (Søndergaard & Mulder, 2012; Strijbos & Sluijsmans, 2010). It helps participants build skills for discerning what constitutes quality performance (Van Zundert, Sluijsmans, & Van Merriënboer, 2010). These skills are necessary in Higher Education in the twenty-first century (Wanner & Palmer, 2018.) To perform such assessments responsibly, the learning environment must enable students to feel comfortable and involved in the learning process (McGarr & Clifford, 2013; Rotsaert, Panadero, & Schellens, 2018). These assessments moreover require that the design and implementation should be performed reflectively to ensure their effectiveness (Wanner & Palmer, 2018). In addition, performing the assessments requires practising (Panadero, 2016) to accomplish higher quality reviews. Another variant of the PR is the one concerning privacy. It can be done anonymously or not. Anonymity may reduce the pressure of the relationship with peers, but a non-anonymous assessment may require a more serious and reflective assessment and it is more in line with a formative use of the PR. It can be established an anonymous phase followed by a non-anonymous phase. When a transition from an anonymous to a non-anonymous peer assessment environment is facilitated, students’ peer feedback quality in the anonymous phase increases over time and the peer feedback quality in the non-anonymous sessions eventually becomes comparable (Rotsaert et al., 2018). Furthermore, an anonymous review does not reflect reality, as people give and receive feedback with known identities (Panadero & Brown, 2017). Our proposal thus supports non-anonymous assessments. Finally, students value the review process better if it is also valued (Wanner & Palmer, 2018).

Many studies conclude that PR exhibit a high level of validity. Although some nuances can be added, such as that peer and teacher ratings are more similar when it is an overall judgement based on well-defined criteria or rubrics (Falchikov & Goldfinch, 2000). The correlation is significantly higher when the course is graduate level rather than undergraduate or the PR is non-anoymous (Li et al., 2016), when the self-assessment is distinguished the validity tends to be a little lower and more variable (Topping, 2003). Despite the high level of reliability, some educators still have little confidence in their use (Agrawal & Rajapakse, 2018). Biases and competitive effects may result when, for example, students do not belong to the same university (Domínguez et al., 2016).

We use PR for strictly educational purposes. We use a non-anonymous form of PR to provide qualitative comments or quantitative values or both as part of the course requirements. Hence, we do not recommend that assessment received of their peers will affect the final grade of the evaluated students. These decisions were made with the intention of encouraging students to assess the quality responsibly, while minimizing their preoccupations regarding grades or the influence of students’ good or bad relationship with their peers. In our case, we do not value all the students’ work, but select a group that is not typically the most exemplary, as we agree with Verleger, Rodgers, and Diefes-Dux (2016) that experts should assess the work of students producing lower quality work to enhance learning and the evaluation process. The evaluations are commented on in a collective session. The students consider the teacher’s involvement in the process as crucial because they do not consider themselves experts (Wanner & Palmer, 2018).

Generating video content

The use of videos in education has evolved over time. A simple use is the recording or substitution of the teacher’s class, with the advantage that students can attend the class when and where they prefer and see what they do not understand again. Videos have also been incorporated as sources of external information. That is, they are used for the same purpose but are not made by teachers. As mentioned, in recent years, this ability of both teachers and students to produce videos has allowed videos to be incorporated into the classroom through the FC methodology (Bishop & Verleger, 2013).

Technological advances have enabled the production of good videos and their visualization at any time and in any place, in turn helping increase student participation in the learning process. Students have begun to be producers of videos in the academic context, and though this trend was initially more evident in the field of humanities and education, it is occurring equally in engineering classes (Greene, 2014). Some examples can be found in (Chewar & Matthews, 2016), for instance, a demo mode to replace the presentation of a computer project or a guided tour of a computer system or a presentation of students’ projects in education (Kerimkulova et al., 2016).

The creation of videos according to Bloom’s taxonomy adapted to the digital age (Churches, 2008) requires mobilizing higher-order, high-level cognitive thinking skills, which include understanding, planning, filming, direction, creation, combination and cooperation. Because university students, regardless of discipline, can be considered digital natives (Akçayır & Akçayır, 2018), they should not encounter difficulties in producing videos. Video production allows students to express themselves in a way that makes them feel more comfortable, thus increasing their motivation and stimulating more imaginative creations (Droumeva & Murphy, 2016). However, it requires practice and experience that not everyone possesses, as using video technology to socialize with friends is not identical to learning and developing critical thinking or communication skills. Another benefit of the use of video creation by students is that it can also facilitate experimentation, active engagement and reflection using subjective experience as a motor for learning (Van Noy, James, & Bedley, 2016).

Studies have already been conducted on students’ experience as video producers comparing them to other types of deliverables in university contexts. Thus, if students can choose between performing a written work or a video, those who choose the video option valued their experience better in certain cases (Khalid, 2014; Smith, 2014); equally, others may prefer written work (Genereux, 2014; Orús et al., 2016) due more to differences in the time and effort involved or how the work is factored into the evaluation than to technical difficulties.

Our proposal combines student video generation with peer review. With peer review students strive to create and do produce higher quality videos because of the social pressure of peer scrutiny (Greene, 2014; Smith, 2014). In our case, the reuse and evaluations by classmates of student-generated videos (SGV) both in the course itself and in subsequent academic courses is fundamental.

Generating questions

Student-generated questions (SGQ) are learning activities in which students generate a set of questions that correspond to specific prior instruction or experiences that they consider important and relevant from the educational perspective, focusing on self-assessment and PR goals (Yu & Wu, 2016). The creation of SGQs requires metacognitive strategies because creating questions requires locating materials that are considered important and relevant; adequately formulating questions; providing multiple responses or alternative answers; carefully formulating questions to address and evaluate specific learning outcomes; answering the questions that are generated; forging links between the current study material and previously learned subjects/units; creating examples of any focal concept; developing plans and strategies to generate questions according to specific criteria; monitoring one’s understanding; modifying plans and/or strategies to correct unsatisfactory learning in the generation of questions; and assessing one’s understanding of the study material (Yu & Wu, 2016). SGQs also help teachers assess the quality of students’ knowledge and reveal what students have learned, and uncover misunderstandings (Aflalo, 2018). ICTs also enable the collection and assessment of such information with tools such as PeerWise (Kay, Hardy, & Galloway, 2018). In addition, students can benefit from questions that are answered by their peers and a bank of questions that are available for reviewing knowledge before the exam (Aflalo, 2018). However, not all questions generated are quality questions, as training is required for achieving quality questions. In the Afalo study, 60% joined the course website. Students who had no practice generating questions were surprised to realize how difficult they found this activity (Aflalo, 2018; Barak & Asakle, 2018). In the Barak & Asakle study, most of the questions generated were classified as having a moderate level of difficulty.

This activity can be linked to the PR because the students can evaluate the questions contributed by other students. Yu and Wu (2016) found that students who provide quality feedback also generate high quality questions.

Generating other types of content

Many forms of digital learning content exist in addition to those described in the previous sections. The entries in a blog or wiki, for example, represent another format worth considering, particularly because such formats are often used as a source of active consultation in technical or health subjects.

Whereas blog content is generally performed by one person or a group of people, wikis are spaces where community members access, create, edit, modify, eliminate, and share information. The co-creation of knowledge extends through linked pages and enables collaborative writing. The most notorious example of the use of wikis is Wikipedia. In addition to possessing sufficient digital knowledge (Matthew, Felvegi, & Callaway, 2009), it is necessary to properly conduct student assessments to assure the quality of the selected content (Trentin, 2009) and to perform several iterations to be more effective (Zheng, Niiya, & Warschauer, 2015). Likewise (Zheng et al., 2015) echo other authors when they stress that good instructional design is vital to ensure the success of learning activities facilitated by this type of methodology.

Activities that include blogs and wikis improve students’ writing skills (de Almeida-Soares, 2008), foster linguistic improvement of second languages (Wang & Camilla, 2012), encourage more reflective and critical constructions about what students have learned and favour the application of learning to new situations, all of which are essential for succeeding in their competitive and changing future classes (Wheeler, 2010).

In computer science, the creation or collaborative maintenance of software through forums or blogs deepens the commitment to learning through the act of authorship because the awareness of an audience, regardless of how virtual or tentative, encourages more reflective contributions (Wheeler, Yeomans, & Wheeler, 2008). If, in addition, that audience is composed by other students of the same course or future classes, students may display greater self-motivation. Hence, this content creation system can be used to share information such as lessons learned (LL). The specific technology used, e.g., Twitter, wikis, forums, or blog posts, will depend on the learning context and subject.

Proposed methodology

This methodology combines several methods and techniques already tested separately in different subjects and integrates them into a global methodological proposal. According to it, during the progression of the subject, students transition from a knowledge-receiving role generated by peers to a creator and evaluator role according to the following five steps:

• 1. Access previous knowledge base of the subject: Students access content that forms part of the knowledge base of a subject (KBS) that is mainly SGC created by previous classes at their own or other universities. Specifically, we work with SGVs, blog entries and test batches that can be used online. Students learn about the subject by assimilating the knowledge collected by their predecessors in previous iterations of the methodology.

• 2. Peer Assessment and first reflection: Students assess the work done by their peers by following guidelines aimed at self-critical reflection and filtering the most useful knowledge. These evaluations are contrasted with those of students from other universities, professors or professional experts. Hence, students learn about not only the subject but also the evaluation process by assessing the quality of this content.

• 3. Co-creation and knowledge generation: Students, working together or individually, depending on the subject, become active agents in knowledge generation. They become aware that this knowledge should be useful beyond satisfying the requirements of subject evaluation and that this knowledge will be submitted to the assessment of peers who, on different occasions, may be from other universities and/or groups. Students encounter a new challenge in the creation of new content that adds value to previous content.

• 4. Peer Assessment and second reflection: The filtered, analysed and synthesized evaluations are returned in sessions to the interested parties to facilitate reflection on the quality and the active role of professionals in generating and disseminating knowledge within the organizations to which they belong. Collaborative reflection on the evaluations and the generated content is an important source of learning about the assessment process.

• 5. Incorporation of generated content to the knowledge base of the subject: Proposals of greater social and/or academic value, which result from the contrast between the students and faculty (of one or more groups and/or universities), are incorporated into the knowledge base, which is improved for future active learning cycles.

The iterative implementation of our methodology resembles the PDCA (Plan, Do, Check, Act) (Deming, 1986) continuous improvement cycles in the field of quality management. Our methodology aims to overcome the temporary barrier between one course and the next by transmitting some learning results from a class of students to subsequent ones, forming a virtual community of knowledge that transcends the academic year (Fig. 1).

Fig. 1

Proposed Methodology

Managing generated content

Given that a key element of this methodology is reusing the created materials, it is important to establish clear criteria regarding licences so that the copyright is not violated. The best option is to disseminate SGC through Creative Commons licences and insist on the proper use of images, audio or videos that are from public domains or created under this type of license.

This requirement is also necessary for initially broadcasting content. From the beginning, students must be aware that the material they produce can be shown to other students as an example of good practice or as a counterexample or for peer evaluation or discussion. Hence, student permission is required.

Analogously to how the authorship of SGC is public, the proposed PR is not conducted anonymously. In addition, independently of the academic evaluation, the assessment of the quality of our results is performed continuously by others, based on expectations that are dynamically modified and that we can try to influence but cannot determine.

For reuse in subsequent courses and to facilitate the creation of a real, sustainable and manageable knowledge base, a medium- and long-term policy of this information system is fundamental. In the case of written reports, the strategy can be easy, but it must be more developed for certain types of SGC, such as videos or webpages. In the latter cases, the location or maintenance of the creators should not be solely relied on them because they can disappear. For this reason, a means of persistent diffusion is required (e.g., a dedicated YouTube channel). A common strategy of use to these resources and among the teachers of the various groups involved (Orús et al., 2016) is also required. Additionally, the volume of SGC requiring management is growing with each course, so reflection on how to manage SGC is required.

Application of the methodology

Next, we detail how the methodology has been used in different subjects in the final academic years for courses in Computer Engineering (GII) offered by the Faculty of Computer Science at the University of the Basque Country (Universidad del País Vasco, UPV/EHU) (Spain) and in certain subjects of GII at the University of La Rioja (Universidad de la Rioja, UR) (Spain) and the University of Pau and the Pays de l’Adour (Université de Pau et des Pays de l'Adour (UPPA)) (France). Table 1 summarizes the pedagogical methods and tools used. Each subject is described in greater detail in this chapter.

Table 1 Teaching techniques used in the subjects

Database administration (DBA)

In DBA, this methodology is used to learn the requirements of being a database (DB) administrator. The DB administrator must work with large databases, interact with demanding clients to ensure that data is continuously and readily available, and solve indeterminate problems under pressure in a limited amount of time. We provoke reflection on the difficulties and demands of the work of a DB administrator using a set of videos selected from the subject KBS and following each of the five steps of the proposed methodology. The content of the videos reflects the experience of a professional expert, along with other videos made by previous classmates at UPV/EHU or UR, highlighting some aspects of the work of a database administrator from their perspective. These videos and a discussion with a DB administrator expert constitutes the source of inspiration and reflection for students who are instructed to work in groups of two or three to produce a SGV with similar characteristics (step 3). Before elaboration, the two steps (1 and 2) involve using peer review to consider and evaluate the videos available from previous years. The next step (step 4) entails evaluating the videos chosen for that course (peer assessment and second reflexion) and some of which will be incorporated into the KBS of the subject (step 5). Moreover, teachers and professional experts with more than 20 years of experience assess those videos, thus providing students with a different viewpoint important for their learning. Performing the activity simultaneously in two universities facilitates cross-evaluations among students from different areas, enriching peer evaluations and the work itself through competition (Domínguez et al., 2016).

Information security management systems (ISMS)

The breadth of subject content and the continuous stream of news and developments related to computer security that occur year after year make the development of approaches, involving greater student participation, particularly useful, even in selecting the issues to address. This role in the selection of topics and approaches is achieved through two lines of work. Students can dedicate 20% of the time they spend on ISMS to a project of their choice. They are also involved in defining and selecting the questions that most adequately reflect the topics they consider most important, given the content covered. Therefore, in this subject there are two types of SGC, reports and questions.

The reports obtained through the optional student projects are subjected to a conventional evaluation process. Projects that generate quality material and facilitate generalizable use become part of the KBS (step 5). Currently, some of the resources most valued by students are contributions by colleagues from previous courses (steps 1 and 2).

During the development of the subject, the methodology of team based learning (TBL) is systematically improved (Michaelsen & Sweet, 2008). For two-thirds of the course, the questions that compose the individual and group validation tests are originated from students’ proposals. After filtering and reviewing questions by peers (PR) and teachers, some of them are selected and added to a KBS of questions that are used in formal and informal evaluation processes to be used in the year that generates the questions and in the subsequent years (steps 1 to 5).

Project management (PM)

The competences to develop in this subject are based and built on intellectual maturity and are oriented to facilitate tools and work methods associated with the corresponding professional exercise and more specifically, with project management. The need for effective communication within organizations is a continuous challenge. The challenge concerns transferring and consolidating good practices and taking advantage of and reusing deliverables of all types (e.g., documents, libraries, manuals). Hence, developing a critical and self-critical capacity with the aim of selecting and filtering the approaches and materials that may improve the effectiveness of future work is fundamental. The need for this capacity is evident in various activities and actions, of which two are particularly effective for developing the subject: the development of a base of lessons learned (Niño et al., 2015) and the evaluation, selection and integration in new developments of products and results generated by students in previous projects and courses.

The repository of lessons learned (LLs) are managed through a blog (Project Knowledge 14, 2014) which content is reviewed, used, valued and increased (steps one to four) course after course. This blog is elaborated in different languages and by the students of two different universities and finally selected for their potential future usefulness by the subject’s teaching staff.

In PM three projects of increasing difficulty are carried out (Jaime et al., 2016) so that in different cycles, students must analyse, evaluate and select products (videos and video channels, being an example https://bit.ly/2Td4wyM) developed in previous cycles, for either themselves or their colleagues (steps 1 to 4). These SCVs constitute a catalogue of regulated use within a class, and after undergoing a filtering process by the teaching staff, they become indicative references for successive classes (step 5).

Web systems (WS)

Web Systems introduces main concepts and technologies to develop web applications. These are grouped into client technologies and server technologies. Throughout the semester and for 3 weeks, the FC method is used (step 1) to teach some of the concepts and technologies that are most relevant to the subject. Students are provided with a series of videos chosen and produced by the Khan Academy as the main source of information in which the speakers are authorities on the subject. Students perform weekly developments in groups incorporating functionalities in an initial website. After covering two-thirds of the programme, students accomplish a PR by completing an online rubric (step 2). Their evaluations, in addition to identifying mistakes in third-party products, serve as guides for developing their final project (step 3). Some of these websites, those previously selected by teachers, are incorporated to the KBS with the aim of being used in subsequent courses to demonstrate the functionalities requested to new students (step 5).

Additionally, this course students has been encouraged to create SGC (step 3), specifically videos lasting 2–3 min that describe any gaps or errors (their own or those of the groups evaluated) or difficulties that appeared during the development of their website worth mentioning, along with a possible solution. Students are informed that this material might be made available for further classes (step 5).

Human-computer interaction (HCI)

In HCI, the methodology is used to help in choosing a tool to produce a digital prototype of a responsive interface (i.e., interfaces capable of automatically adapting their structure, navigation and content to the device in use, be it a smartphone, tablet, laptop or desktop computer) for a web application. Students must design this interface, validate the design through prototypes and evaluate the usability of the final product implemented. This work is carried out in groups in three PR sessions (step 2 and 4). The objective of the tests is to confirm that the designs exhibit correct functionality and navigation and that the final product is usable by identifying problems, deficiencies or potential improvements in both students’ own products and others’ designs. It should be highlighted that at least one of the members of each group is studying WS simultaneously, and the digital prototyping is done on the websites they are developing in WS at the same time. In this way, websites, which are in turn SGC in SW, exceed the scope of the subject, as they are inputs in HCI.

Since there is a wide range of prototyping tools with multiple possible configurations, to choose the most appropriate one is not an easy task. At this point is where the videos of the previous years stored in KBS become a useful aid for students (step 1), since those videos were authored by students of similar profile after having faced a similar task with a concrete tool. The content of those videos reflect the capability and functionality of the tools used in the previous year, showing both the positive and negative aspects of them. Therefore, before elaboration students not only got help in the concrete task, but also got inspiration about the video they also had to develop, following the steps of the methodology (steps 1 to 3).

Tutored project (TP)

Students develop a micro-project in teams to create and publish a video on the Internet, based on given specification and some videos produced by former students (step 1) as guiding examples. The goals of this activity are to create a common experience among students, namely, the Video Project, to solve a minimally complex working task as a team within a limited timeframe and with limited resources, and to achieve an acceptable final product based on their previous level of technical skills. Hence, students are exposed, on a reduced scale, to professional problems and given an opportunity to later reflect on the experience, once they are aware of the problems and difficulties of working with others to achieve a result. We are not as interested in what students produce (step 3) but, rather, how they accomplish the task: we aim to help them learn about project management, not video editing.

SGVs are first evaluated by professors of the Faculty of Computer Science of UPV/EHU and the IUT of Bayonne based on pre-established quality indicators. Through this evaluation process, six videos of different quality levels (high, medium and low) are selected. Next, students assess the selected videos in a PR session (step 4) using the same quality indicators. The selected SGVs are included in the KBS (step 5) and reused in three subsequent subjects in which the experience of developing the Video Project is repeatedly used to illustrate basic concepts of project management and cooperative work methods, project concepts and teamwork.

Results

The analysis of the results obtained considers two aspects: the quantity and applicability, as possibility of reuse of the resources generated, and the level of satisfaction achieved with this methodological proposal, being excluded from this study the impact on academic performance.

In all cases, the methodology enabled the inclusion of SGC in the knowledge base of the subjects for reuse (see Table 2). The way in which the digital content has been obtained (licences) and its mode of dissemination (open access through the web) allows reuse of the content for not only a given subject but also other subjects, even at other universities.

Table 2 Number of student-generated contents (SGC)

Frequently, SGC that does not exhibit the quality necessary for its reuse, nonetheless, highlights erroneous assumptions about previously acquired knowledge or needs that teachers would otherwise have more difficulty detecting. The characterization of SGC and feedback from colleagues or teachers help to improve student understanding.

On average, 10 videos per course were made in ABD during three courses (7 at the UPV/EHU and 3 at the UR). From these, 6 videos were chosen for PR the first year (3 from each university), and 4 videos for the second and third years (2 from each university). The six most highly rated videos with quite different approaches, two per year, have been included in the KBS. In ISMS each student can produce between 15 and 20 questions (between 350 and 500 questions per class) throughout the course. At least 50 questions per course are potentially reusable due to their interest, quality and relevance to the aspects that, according to students, frequently pose greater difficulties in understanding. An average of 8 project reports per academic year is obtained and an average of 3 project reports per year become part of the KBS. In PM in each group, approximately 50 videos are developed, of which at least half are used to create an a posteriori shared resource, becoming integrated into independently developed channels. For each course, in WS, an average of 30 websites are made and reviewed by peers and, at present, its KBS has 5 websites and 4 videos. Currently, the dissemination of this SGC is limited to the academic scope of the subject. In two courses, in HCI, 8 videos have being made and gone through PR, and 4 videos with the highest ratings about different tools have initialized its KBS. In TP, in three courses, students have produced 74 videos. Each course, the six videos selected by teachers have also been reviewed by students and added to its KBS.

Regarding the level of student satisfaction, the objective was to maintain or exceed the results achieved with the methodology proposed in “Proposed methodology” section and applied to each subject as presented in “Application of the methodology” section. To this end, the results of the surveys conducted by UPV/EHU in each and every one of the subjects taught in its almost one hundred degrees have been taken into account. This survey includes questions on teaching methodology, but also on other aspects, so only those related to the methodology are considered here. Specifically, among the questions concerning the proposal of activities that foster autonomous learning, a reflective attitude and the encouragement of participation are highlighted (see Table 3). These items are assessed on a scale of 1 to 5. This table summarizes the satisfaction with the application of the methodology to the subjects, as detailed in the previous section and in cases where differences are discernible. According to those results, the employed teaching methodology is highly rated.

Table 3 Summary of results of student opinion surveys

As shown in Fig. 2, satisfaction averages are very stable, regardless of whether they are University, Faculty or department values that are being considered. Nevertheless, the comparative of the results indicate that the subjects analysed in this paper obtain substantially better results (see the last two bars labelled with “SUBJECTS” in Fig. 2 or the last row labelled with “average” in Table 3).

Fig. 2

Average survey results

Discussion

A key factor in the use of this methodology is to ensure the faculty trust students’ contributions. It is crucial to create shared objectives with clear, achievable and valuable perspectives for both students and teachers. That is, the learning environment must transmit the idea that what is performed interests the one who performs it and the one who receives it. Hence the importance of reusing SGC. By accessing and evaluating these materials, students can find a motivation to cooperate with good materials to the KBS. In this way, it is possible to obtain good outcomes of satisfaction as shown by the results. Their contributions can be very valuable and different, both from each other and from what it could be done by the faculty, due to the students’ direct and recent experience with the learning process. These ideas agree with those highlighted by Bovill et al. (2016) and Khalid (2014). Our experience also indicates that through good examples, new contributions improve. SGC can serve as inspiration for classmates in successive courses, and therefore it is more advantageous than problematic to repeat the same theme for SGC in different years, especially in issues that change so quickly.

The fear of increasing teachers’ workload in supervising student inputs can be an obstacle. One way to assuage this doubt is to opt for voluntary contributions or choose specific students and propose activities that require a reasonable effort by both parties and positive effect in their grades. It is easier to start with small contributions from students. Unlike Khalid (2014), in which study some students considered the experience frustrating because of its difficulty in creating videos, we have not encountered these problems.

As the results show, it is now possible to have a considerable amount of elements in the KBSs. These contents would have otherwise been difficult for teachers to produce due to the lack of time and, in some cases, limited technical skills in creating digital content. In addition, when students produce this content, in many cases, they present terminology or formats that are more accessible, familiar and motivating, allowing students to assimilate certain concepts more easily (Bovill et al., 2016; Chewar & Matthews, 2016).

As the volume of SGC is increasing, as result of the increase in the number of questions, videos, LLs and blog entries, it is required a specific organization to maintain, manage, and retrieve examples. We are starting to use YouTube channels as well as paid hosting to retain the different websites simultaneously and over time. Our experience indicates that the sooner the information system is designed and organized to fulfil the requirements associated with this proposal, the greater satisfaction and efficiency teachers will experience by using SGC, thus reducing errors, wasted efforts and repetition of tasks. The location and nomenclature of the digital objects will depend on the subsequent use, in both the search, selection and publication by teachers and the consumption and assessment by students. Likewise, during the processes of assessment, reflection and generation of new content, a multitude of metadata are generated (e.g., authorship, date, rating, score, theme, style, language(s), and resources). In many cases, managing the relationship between the specific content and the associated metadata is critical to achieve adequate system efficiency and scalability across years, subjects, teachers and students.

The presented methodology may be easily adapted to any subject and beneficial for students and teachers. As our application shows, it can be seen that the subjects and approaches are slightly different. It gives an idea of the flexibility of our methodology, and the use of the different types of SGC. Following its application to various subjects, students have become accustomed to generating digital content and participating comfortably in PR. In addition, for students of the current generation, giving and receiving peer reviews are common practices, evidenced in the “likes”, comments on blog posts or digital content, and the continuous feedback of online chats. Using the presented methodology, we performed PR in an orderly, systematic manner without authorship conflicts as in (Rotsaert et al., 2018), facilitating participants’ (self) critical thinking and the selection of reusable SGC.

Our methodology bears some similarity to that used by Yang et al. (2016), but in their case SGC are carried out under the supervision of teachers, while in our case they are generated autonomously. In addition, the use of PR has a different approach since they use it during the elaboration of SGC to introduce possible improvements while in our case it is applied for the valuation of SGC. The availability of SGC created in one academic year for the courses of the next generations is another differentiating feature.

The major limitation of our study refers to the lack of results with respect to the learning obtained. Although it is a promising line of future work, a significant limitation of our work is the lack of a systematic analysis of the learning outcomes obtained. Even though we think that the academic results might have improved, so far we only have partial results in one of the subjects (Jaime et al., 2016).

Conclusions

The educational methodology presented is a complement to various approaches that promote active involvement of students in the learning process. Our proposal integrates peer review into a strategic cycle of SGC. In this cycle, students benefit from SGC generated by previous students while critically reflecting on its usefulness. In a second stage, students themselves create content, knowing that it will be reviewed by their peers and, if circumstances arise, selected for use and inclusion in the KBS. In the final step, students and teachers conduct a review aimed, ultimately, at the selection of new content that will enrich the KBS.

The suitability of applying this method using various types of content (lessons learned, videos, questions and reports) has been tested in a variety of subjects, thanks to its flexibility. In all experienced cases, it has been shown that it is feasible for students to generate enough valuable and reusable content. In addition, students have expressed high levels of satisfaction with the implementation of the proposal. While not all students are able to generate new valuable content, many are able to appreciate it when their peers do. Being aware that they benefit from a legacy, while collaborating in its improvement, increases their co-responsibility in the various processes. Finally, for teachers it means a change of role, turning to train students to achieve publicly reusable results in the future.