Bearing in mind that this paper analyses the role of students in e-assessment activities centred in a concrete course and their perception about this process, the following research questions are considered:
How students appraise the different e-assessment strategies of the course?
What do these assessment strategies that involve an active role contribute to the learning process?
We developed a quantitative and qualitative research, the design of which is explained in the following sections.
Scenario
The research is conducted at the Open University of Catalonia that is a fully online university, with an e-learning model defined as “a mode of teaching and learning that can represent all or part of the educational model being applied, which exploits the media and electronic devices to facilitate access, evolution and improvement of the quality of education and training”(Sangrà et al. 2011, p.5). The university bases the assessment of its entire course on a continuous assessment model, which, according to Sangrà & Guàrdia (2005), has to be integrated in the course design from the beginning and thus meet the needs of students to build knowledge step by step.
The experience presented in this paper concretely focuses on the subject “ICT competencies” (ICTC) which is part of all the UOC university programs. This subject helps students to develop key methodologies and skills to work in digital environments from a rational and critical perspective, and its objective is that students begin in a gradual and integrated way with the acquisition of transversal competencies at the UOC; «Use and application of ICT in an academic and professional environment» and «Online team work» (Guitert et al. 2008). These competencies are outlined as: Search and selection of information online, Processing and development of digital information, Presentation and dissemination of digital information, Basic notions of digital technology, Work planning in a virtual environment, Management of a digital project, Communication strategies in the Net, Teamwork in an online environment and Digital attitude (Pérez-Mateo et al. 2014). The methodological approach is project-based learning (Railsback, 2002), concretely, the developing of a collaborative digital project is considered. To undertake it, students form groups of four, and have their own group space which integrates a variety of tools.
In order to develop their projects, students follow the process of the subject based on the subsequent phases (Author et al. 2012) (see Fig. 1).
The starting phase provides an environment to create working teams and perform the initial searches. This assists the students in setting out the theme of the project.
The second phase (structuring) involves making a deeper search for information to structure the project.
Subsequently, (developmental phase) the project is developed and the processing and the development of the gathered information are carried out. Through this step, the first version of the project is achieved.
Finally, the closing and dissemination of the project are done, as well as the sharing and discussing of the final version of the project.
In order to assess the acquisition of competencies during the subject’s development, some assessment criteria were defined and shown in the learning activities.
The assessment activities of most subjects at the UOC are based on continuous assessment and developed online but the final evaluation is carried out by a face-to-face test. In ICTC, all assessment activities (including the final ones) are developed online and from a formative and continuous perspective so it becomes a good example of continuous e-assessment. Bearing this model in mind, assessment becomes, following Delgado, A. Oliver R (2006), progressive, and the teacher can monitor the progress in student learning more and in a better way as it allows for comprehensive assessment.
In addition, the subject shown in this paper is appropriate for students to acquire a relevant role in the assessment process, in that the UOC’s educational model is student-centred. In fact, student learning is assessed from two dimensions provided by continuous assessment: on the one hand, the assessment of the process followed during the development of the activities based on the outcomes of each phase and process monitoring and, on the other hand, the assessment of the final outcome.
The fact of developing a digital project collaboratively facilitates the implication of the students not only in their individual learning process but also in the collective learning process. This is further corroborated when the definition of collaborative work is defined as a “shared, coordinated and interdependent process, in which students work together in order to achieve a common goal in a virtual environment and based on a process of activity, interaction and reciprocity between students, thus facilitating the collaborative construction of meanings and individual progress towards reaching higher levels of development” (Guitert & Pérez-Mateo, 2013, p.24).
This scenario boosts the development of a new teacher’s role as an advisor and facilitator of the learning process (Pérez-Mateo et al. 2014), providing a more active role of the students through reflecting on their own learning process and peer assessment.
Considering this perspective, the concept of 360° e-assessment can be formulated from the bases of 360° Communication, which is “considered as a state of constant dialogue in which organizations take on a Communicator role with their clients (internal and external)”, so it becomes a comprehensive strategy that connects companies constantly with their public, online and offline (López & Martinez 2012) Based on the 360° communication theory, the characteristics of 360° e-assessment can be described as the following (adapted from Curcoll, 2014) (see Fig. 2):
-
Strategic: considers the identification of the key elements for improvement based on the acquisition of competencies.
-
Integral: assures the integral acquisition of the competencies, considering the set of competencies to be acquired as a whole, and not as a segmented process.
-
Holistic: takes into account all the internal agents –understood as work groups-, external –the whole classroom- and the digital environment in order to understand collaborative learning as more than an addition of several parts,
-
Transversal: affects all of the learning actions and activities and the interactions that take place during the learning process,
-
Coherent: coordinates and gives sense to the whole teaching-learning process; being global, e-assessment takes into account the different processes as interrelated and not isolated, giving coherence to the assessment,
-
And dynamic: conceives the learning process as a living system, in constant change and dynamic in which all the agents participate. So assessment is understood as a process of constant interaction among students and between students and teachers that can be redefined depending on the inputs received.
Following these characteristics, the aim of 360° e-assessment is to reinforce and boost the students’ learning process in order to help them to acquire the competencies of the ICT competency course.
In the subsequent sections, some evidence of the active role of the student is given and some of the mentioned characteristics are reinforced such as those elements like being integral, dynamic and coherent.
Students’ involvement in the e-assessment process
In order to understand the role of students during the e-assessment process in ICTC, it is, first of all, necessary to describe the role of the teacher during this process.
During the development of the subject, the teacher carries out a process and final outcome assessment. The teacher monitors and assesses the work developed in each of the groups during the process in order to improve their dynamics. During this assessment, the active role of the students is quite relevant, since the self-assessment they develop in each phase of the digital project is one of the elements that determine the teacher’s post-assessment of their work.
The process assessment is finished with both individual and group feedback: individual feedback is based on the observation of the participation of each student within the group and the group one is based on the monitoring of the group’s dynamics and processes.
In the assessment of the final outcomes, the teacher assesses the final product of each phase following the predetermined criteria shown in the learning activities. This assessment is developed using a set of indicators that are a concretion of the afore-mentioned assessment criteria.
In this scenario, students subsequently develop the assessment activities.
Students participate in both the assessment of the process and the final outcome. The process assessment, developed in the framework of teamwork, is produced in a self-assessment process in which both the student and peer participation in a work group are analysed, thus developing a dynamic peer-assessment.
In addition, a process of reflection about the teamwork is carried out at the end of each phase of the project, based on an interactive process among the members of the group.
The assessment of the results focuses on an interactive assessment (Crisp, 2010) based on the analysis of the projects developed by other groups at the final stage of the course. During this phase, a defence process takes place during which every student participates individually as an evaluator and as part of a group participating in the development of the responses that their specific group receives from other individual evaluators.
The participation in the assessment process provides students with an awareness of their grade of competency acquisition, thereby allowing them to be an assessment agent of their own learning process from different perspectives or dimensions. In consequence, e-assessment becomes, within the ICTC’s subject framework, a very valuable resource to foster students’ implication in their own learning, from both the individual and collective perspective. In addition, the consideration of this dual perspective provides a higher level of personalization of student assessment: each student of the same team can obtain different marks according to the quality of their individual work in the framework of the team. In this context, a network assessment process is created.
The network assessment process becomes tangible during the different phases of the course because there is the group’s internal peer evaluation during the process and during the final phase of the project, and also each student becomes an evaluator of another group’s project, so the group class becomes a network of evaluators in order to improve their learning processes. In the subsequent figure, the different strategies to assure the students’ protagonism during the assessment project are laid out.
Data gathering and analysis
In order to gather the data, a questionnaire was designed and applied once at the end of the first semester of the course 2014–15. The questionnaire was composed of one section in which students rated the whole course (how it helps them to acquire the competencies of the course and its methodology) and another one based on ten statements that students had to rate their grade of agreement with them. In this last section, all items were drawn up considering their satisfaction with the different processes of e-assessment in the subject and with the active role that it implies. All items from both sections were rated using a 5 point Likert scale. In addition, open text questions were included in order to gather students’ testimonials. To validate the questionnaire’s content; it was previously revised by four experts in e-assessment and administered to a reduced sample of students. Once the validation process was completed, the questionnaire was administered to all students.
Considering that the course was undertaken by 2,303 students, we were able to gather a sample of 913 students thus reaching a representative sample of 39.6 %.
In order to complete the data gathered with some testimonials of the teachers, a focus group session involving 15 of them was developed during a coordination meeting.
Quantitative data has been analyzed with descriptive statistical methods (means and standard deviation) and, in order to determine the relationship between the items in the different sections, the Spearman’s r correlation coefficient has been calculated in some cases. The analysis of the items has been complemented with some of the students’ and teachers’ written testimonials.