- Research article
- Open Access
Students’ perceptions of a food safety and quality e-learning course: a CASE study for a MSC in food consumption
International Journal of Educational Technology in Higher Education volume 16, Article number: 37 (2019)
Life long training and education in food safety is a crucial issue in particular for professionals working in the field. At the same time there is a need to evaluate the effectiveness of that training in particular when is given in e-learning. The purpose of this explorative work is to evaluate the Food Safety and Quality e-learning course within a Master program offered in the National Distance Learning University in Portugal. Following a case study methodology a mixed methods approached was used, based on an online questionnaire survey to students that enrolled the course, followed by a Focus Group Discussion to better discuss the weak points addressed by the students in the questionnaire. The questionnaire aimed to assess students perceptions about the course organization, recourses, assignments, acquired competences, change attitudes and behaviour and link with the professional activity Students who enrolled the course during 2016/17 and 2017/18 were surveyed in the fourth semester of the curricular year, in the period of development of the master thesis dissertation. In addition a quantitative analysis was conducted in the temporal distributions of messages published on the Moodle platform (learning analytics) to evaluate students’ interactions and engagement during the course.
This study has shown that overall, students are very satisfied with the course, however they suggested improvements that are needed considering the pedagogical model used and their availability to learn as students with full time jobs, most related with the food science. This explorative research aimed to contribute to the improvement of food safety and quality training based also on recent tools and recommendations within e-learning in science education.
The food supply was “never so safe as today” at least in developed countries (Flynn et al., 2017). However, foodborne illness is still an important public health problem world wide and everyone in the world is at risk of this disease, whether in developing or developed countries (WHO, 2008).
wide and everyone in the world is at risk of this disease, whether in developing or developed countries (WHO, 2008). Although the improvements in water sanitation, hygiene, and the safety of the food supply have greatly reduced the number of deaths morbidity remains high (Scallan & Angulo, 2007). The consume of contaminated food is responsible for 600 million ill people/year. There are about 420,000 deaths, which occur among these annual victims and there are 125,000 deaths that occur with children under 5 years of age (WHO, 2015). Food safety is a serious component of sustainable development, and complications that occur in one country may put other countries at risk because the food globalization process has revealed a substantial impact on the food safety (Scott, 2003; FDA, 2009). Providentially, there are increasing governments’ involvement in monitoring and regulating the sector to guarantee food quality and decrease foodborne diseases (Zanin et al., 2017).
Food safety is essential to human nutrition and food security because poor nutrition and foodborne diseases frequently combine a vicious cycle of deterioration of health. Consequently, food safety must be methodically incorporated into policies and interventions to develop nutrition and food security. As the international community changes to the post–Millennium Development Goal stage with an emphasis on sustainable development goals, food safety is an essential element in improving global health and ensuring sustainable development. (WHO, 2013). Food safety is indeed an essential issue to address the Sustainable development goal n. 3 towards the insurance of healthy lives and promotion of well-being for all at all ages (UN, 2016, 2019).
European legislation demands food safety preparation in all EU countries. All food commercial technicians are required to ensure that all their staff involved in food handling activities are appropriately trained and/or coached in food hygiene (EC, 2004, Ovca et al., 2017). According to the European Commission (EC, 2012), the European education and training systems are insufficient in providing the right skills for employability and are not co-operating satisfactorily with business or employers to bring the learning experience closer to the reality of the working environments (EC, 2012).
The food safety risk assessment is a scientific field that is constantly advancing and, as such, a lot of educational programmes have been developed recently with a focus on keeping professionals up to date with suitable lifelong training programmes on different forms (Bosman et al., 2016). Many Programmes at European universities or other academic institutions provide specific programmes at Bachelor and Master’s levels, as well as doctoral education in food science, toxicology or other related areas that cover aspects of Food Safety and Food Risk Assessment (Kneifel, 2012). Nowadays, around 60 programmes in Europe formal education programmes and training) explicitly mention ‘Food Safety’ in their curriculum names (Bosman et al., 2016). Students that are already on their professional jobs related with food industry, search lifelong training on this field of food safety, because of the requirement in the Codex Alimentarius (WHO & FAO, 2009). Within this requirement on-going training in the field of food safety is necessary for all levels of employees and managers, as appropriate and adapted to their background level. Zanin et al. (2017), found in an integrative review, that the studies dealing with good manufacturing practices were conducted mainly in developing countries. Possibly, in countries where food safety regulation is more mature (e. g. Canada, USA, European Union countries), many of them with mandatory application of the HACCP (Hazard Analysis and Critical Control Point), system, the interest is greater in the application of management systems than the behavior of food handlers.
E-learning provides students with an educational alternative to face-to-face teaching, permitting students to proceed, at their own pace, and to identify their own personal course timeline while having at the same time full-time jobs (Azeiteiro et al., 2015, Lee et al., 2019). E-learning also leads to positive learning outcomes, such as a high level of learning achievement and higher-order thinking abilities, because it allows for leaners to actively engage in learning anytime and anywhere (Bacelar-Nicolau et al., 2015; Lee et al., 2019). In 2000, universities, companies and associations were seriously exploring the use of online education in an effort to reach adult learners unable to leave their work places (Matter & McNiel, 2008) and willing to acquire new skills. There was a virtual explosion in the number of online offerings among universities worldwide (Moura et al., 2010).
Despite the advantages of learning, one important problem in e-learning is the higher dropout rate and learning barriers working students have to face in this regime (Bacelar-Nicolau et al., 2015, 2012, Kim et al., 2017). To overlap these barriers student’s engagement and motivation are essential (Lee et al., 2019). Learning motivation of students can be induced by multiple factors, including intrinsic and extrinsic factors. For example, the motivation for good grades is an extrinsic factor and the preference for challenge is an intrinsic motivation (Lin et al., 2001; Law et al., 2019). The virtual environment involves many forms of media and materials that stimulate students’ interest in learning and results in higher satisfaction (Phungsuk et al., 2017).
Student engagement is defined as the level of effort or interaction between the time or the learning resources that develop learning outcome. When students are engaged in their learning they can improve their critical thinking, problem-solving and grades, and also applied their acquired knowledge in their jobs (Lee et al., 2019).
Learning Analytics is a fast-growing area in the field of Education, resulting in the generation of knowledge about agents and educational contexts, from large amounts of data (Data Mining), digital technologies and the Internet in learning Enhanced Learning (Ferguson et al., 2012). Learning Analytics facilitates the analysis of student learning processes based on the dynamics of student and teacher participation in online contexts (notably in LMS - Learning Management System). The evolution of student learning, as well as the prevention of dropout and failure, are the areas where the influence of the Learning Analytics has more evidenced (Papamitsiou & Economides, 2014; Lacave et al., 2018).
Besides some studies to evaluate e-learning efficiency at sciences courses (e.g. Azeiteiro et al., 2015, Bacelar-Nicolau et al., 2012, 2015, Moura et al., 2010, 2014), there is a dearth of research about the efficiency and students satisfaction of e-learning courses about food safety, an up to date life long learning topic. Also there is a lack of clear direction as to how the higher education system is accommodating or not nowadays students’ needs (in terms of innovation tools and new pedagogies) in science learning (Bidarra & Rusman, 2017).
The aim of this research is to evaluate student’s satisfaction about the course organization, pedagogical resources, evaluation activities, acquired competences, relation with their professional activity and students’ engagement, after enrolling a Food Safety and Quality course place in a context of e-learning Master’s in Food Consumption Sciences programme offered by a national distance learning university, Universidade Aberta in Portugal.
Case study: the Universidade Aberta (the Portuguese open distance university) and the master degree in food sciences
Universidade Aberta (UAb) was created in 1988 and is the only Public Distance Learning higher education institution in Portugal. All educational offers at UAb are incorporated into the Bologna European Process and are taught via e-learning since 2008. UAb is a reference Higher Education European institution in the area of online and advanced e-learning and learning, using the most innovative information and communication technologies, and a prominent Virtual Pedagogical Model (Pereira et al., 2008). This online model is based on four main pillars: student-centred learning, flexibility, interaction and digital inclusion. The UAb was, a few years ago, awarded internationally by Prize of European Foundation for Quality in E-learning and certification of The UNIQUE Quality Label for the use of Information and Communications Technology (ICT) in Higher Education, awarded with the 1st Level of Excellency of the European Foundation for Quality Management, and also awarded by the European Commission with the Diploma Supplement Label. UAb was also considered recently by a European Commission report as a key actor in the higher education system in Portugal and Europe (European Commission/EACEA/Eurydice, 2014; UAb, 2019a).
The formal educational offer of UAb is organised according to the European Credit Transfer and Accumulation System (ECTS).
The Master’s Degree in Food Science at Universidade Aberta is a unique programme that offers distance education and advanced training in e-learning in the area of food science (Moura et al., 2010) and its main objectives are to enable participants to i) interact along the supply chain in areas such food processing, retailing and consumption activities; ii) play a role in activities that promotes attitudes and behaviours changes in order to gather healthy consumption patterns; and play a role in decision-making processes concerning food product development (UAb, 2019b).
The Master programme is characterized by: i) strong motivation of students seeking a professional or intellectual updating; ii) independent, collaborative student-centered learning; iii) students are mainly in the labor market; iv) Flexibility is provided by the asynchronous model, without temporal imperatives and physical displacements; (v) training and accompanying students to enable them to use new technologies and work in networks; and vi) curricular structure designed to meet the students’ updating (Moura & Aires, 2012; Moura et al., 2014).
The curricular plan of the Master is divided in two semesters (in a total of 60 ECTS), defined as a period of twenty weeks each one, plus two more semesters for the preparation, orientation and presentation of an original dissertation which accounts for a further 60 ECTS Different learning strategies are developed from mainly individual work to more collaborative and cooperative learning where peer and teacher interaction becomes very important (Azeiteiro et al., 2015). This is achieved through the inclusion of a number of activities supervised by the teacher such as: surveys, quizzes, assignments and compulsory discussion groups.
The open source Moodle (http://elearning.uab.pt/) is the course management systems used. Each course from the MSc in Food Science and Consumption is organised in a set of topics and each topic is associated with one learning activity.
A 2 weeks online bootcamp module is available to students before the beginning of the 1st semester, which is intended to familiarise the students with the virtual environment and the e-learning tools, as well as promoting the acquisition of online communication and online social skills (Pereira et al., 2008).
The Food Safety and Quality course tries to update the demands that the European and world market has for the actors in the food chain. This course is organised into a set of seven topics and developed into six learning activities (see Table 1).
In first place, it is promoted an ice breaker where students and lectures introduced themselves, then a dialogue is promoted about course objectives, structure and topics/activities to be developed, that are chronologically presented in a document (e-book) called “learning Contract” to all students.
In order to stimulate students to become involved with the course and with the e-activities, practical and real examples were given (as shown in Table 1). Compulsory individual and teamwork activities were included. Support materials available to the students in the virtual space include e-books, research papers, internet sites, slide presentations, videos, original teachers’ documents and regulations (including legislation) case studies, news from media and original research are used to bring the subject to life and to help students to link theory to their practice in a e-learning regime, as defended by Moura et al. (2010, 2018).
Case study based on quantitative and qualitative approach according to a mixed method choice was used in this study (Saunders et al., 2007). Data collection was conducted through a student’s questionnaire survey, one focus group and learning analytics, during the school years of 2016/2017 and 2017/2018. Topics 3 and 4 (natural food toxins and contaminants) were not considered on this research, since were taught by a different teacher and data was not available on these contents.
All seventeen students who attended the course of Food Safety and Quality of the Master of Sciences in Food Consumption in the school year of 2016/2017 and 2017/2018, and who are still enrolled, developing the master thesis, were invited to answer an online questionnaire survey in order to evaluate the perceptions about the performance of the course. At the time of online survey, one student was not contactable by the University, therefore, he was not contacted to answer to the questionnaire. This survey was inspired on other research where e-learning courses were evaluated (e.g. Bacelar-Nicolau et al., 2009, 2012, 2015, Oliveira et al., 2016). The questionnaire built on Google Forms was divided in three groups of questions: the first concerns to the students course satisfaction itself and is constituted by four sections: i) organization of the course; ii) resources and activities of the course; iii) program contents of course; and iv) acquired competences (see in Additional file 1 the questionnaire). The second group consisted of questions related to the teachers, and the third group consisted of issues relating the course with the professional activity of students. In total, there were 42 questions using a Likert scale of 1 to 5, where 1 corresponds to a total disagreement and 5 corresponds to a total agreement. Descriptive statistics of the Likert based questions were conducted. Frequency distributions were converted to percentages to provide a standardised way of comparing the categories. For the open-ended question a transcription and translation to English of the content were made. This question consisted in a space to comments and suggestions (appendix 1).
Also, there was sociodemographic questions: age, sex and professional activity and an open space to comments and suggestions (appendix 1). A questionnaire pre-test was conducted and, after making the necessary adjustments, a final total of 46 items was obtained. All the questionnaire was translate from Portuguese to English (see appendix 1).
Focus group discussion (FGD)
All the students’ population invited to answer the questionnaire were after invited to attend a videoconference meeting for a focus group interview to explore the perceptions towards weaknesses identified in the previously applied questionnaire (see in appendix 1 the focus group guide). The Zoom-Colibri videoconference software was used (https://videoconf-colibri.zoom.us). Virtual focus group reduces the burden on participants by eliminating travel and minimizing preparation time (Rupert, et al., 2017). Focus group methodology employs an interviewing technique; is not a discussion, a problem solving session nor a decision making group (Paton, 1990; Krueger & Casey,2015). The prime objective is to obtain accurate data on a limited range of specific issues and within a social context where people consider their own views in relation to others (Wilkinson, 1998; Robinson, 1999).
The meeting was scheduled through Doodle. A focus group discussion was conducted with three participants (17.6% of total students and 37.5% of the online survey respondents). The interview lasted 32 min and all participants provided informed consent for their interviews to be recorded (see guide in Appendix 2).
Responses from Food Group Discussion were transcribed and translate from Portuguese to English. Participants were codified as FG#1 (male), FG#2 (female), and FG#3 (female). Data were then coded sentence by sentence. Common themes, patterns and clusters were identified and organized systematically (Braun & Clarke, 2006; Huberman & Miles, 1994). Data were then grouped together into themes and sub-themes.
The limitations associated with questionnaires and group interviews, e.g. validity, reliability and generalizability, such as those associated with participant and observer error and bias (Saunders et al., 2007) were taken into account in the discussion of results, and when drawing conclusions.
Learning analytics (LA)
In order to evaluate the students’ interactions and engagement in the course and its link with the topics activities, all “posted messages” in the Moodle platform were collected and recorded by student code. The number of messages posted in the course of Quality and Food Safety were identified with day and year from academic year of 2016/2017 until 2017/2018. This posted messages are public messages and is an indicator of participant’s involvement, as active users (Coelho et al., 2015), so an important indicator of students learning process.
All students that enrol the course were approved in their respective academic years, so the approval rate was 100% and the final rankings varied between 10 and 20 scores (in a total of 20). The average rating was 16.4 ± 2.25 sd (standard deviation).
The questionnaire response rate was 47% (n = 8). The average age of the respondents was 40.3 years, 25% were male and 75% female; 37.5% were single and 62.5% were married or in union. All the respondents have full time jobs and most of the students’ professional activity is related with the food sciences (Table 2).
Organization of the course
All the parameters evaluated on the organization of the course had positive evaluation except the statement: “The workload was adequate” (Fig. 1). Here, the perception of respondents was 12.5% “somewhat disagree”. In contrast to this perception, the remaining 87.5% received agreement (75% of Somewhat Agree and 12.5% of Strongly Agree). It was expected that the pace of learning had the same evaluation as the workload, however, it obtained 87.5% of favourable evaluations (12.5% of strong agree that pace of learning was adequate and 75% of partially agree). Neutrality was verified in 25% of the students. There was no negative evaluation here. The clear and adequate objectives received a very good evaluation: 100% of Agreement (somewhat or strongly). The programme/contents of the course were evaluated very positively, with 75% of general agreement (37.5% of strong and 37.5% of partially agree), with 25% of participants taking a neutral position. Also, the contents of the course were well appreciated, receiving 75% of general agreement (37.5% of strongly and 37.5% of partial agree) and 25% of neutral appreciation. The learning agreement plan had also good evaluation by students (37.5% of strong agree and 37.5% of somewhat agree, making a total of 75% of general concordance). The remaining participants (25%) positioned themselves in a neutral point. Regarding the balance of the activities, students gave a 75% agreement rating (25% of strong agree and 50% partially agree). Again, 25% of students assigned a median ranking. Both, the proposed assessment and the acquisition of competences initially proposed, had very good evaluation (87.5% of general agreement (37.5% of strong agree and 50% of somewhat agree). The remaining participants (12.5%) were positioned in the middle of the scale.
Concerning the issue of the workload, it was justified in the focus group due to much readings and accumulation with the other courses of the Masters.
… Let's say that Food Safety and Quality really does have a lot to study. Actually, the volume is very large considering all the legislation that is behind ( … ) (FG#2)
… taking into account that there are colleagues in this master who are not from the food professional area, those people may have experienced these difficulties (FG#2)
… the workload is linked to overworking with the other courses of the master. It's all accumulated. It has to do with that part too! (FG#1)
Pedagogical resources and assessment activities of the course
Regarding the students’ perceptions about the resources and assessment activities of the course, none of the statements obtained negative perceptions (Fig. 2). Three statements obtained 100% of agreement evaluation. “The Regulations presented as supporting material were interesting and appropriate”, “The power-point slides, used as support material, were adequate” and “Forums of interaction with teachers were useful” got the agreement, respectively, 50%, 62.5% and 75% to “I strongly agree” and 50%, 37.5% and 25% to “Somewhat agree”. The other seven statements, besides total and partial agreement, four obtained 12.5% of neutral annotations (to “The chapters of books presented as supporting material were adequate”, “The scientific articles presented as supporting material were useful”, “The individual activities proposed were interesting”, and to “Interaction Forums with colleagues were helpful”) and the other three statements obtained 25% of neutral annotations (to “The proposed group activities were interesting”, “The Moodle platform worked well” and “The Pedagogical Model used corresponds to the expectations”).
In the focus group interview, comments about the e-activities were given, namely related with some constraints of the teamwork in the e-learning teaching regime. Besides students’ satisfaction with this time of assessment, the problem is the conciliation between their intense professional occupation and time available to collaborate with the colleagues:
… the teamwork always promotes the debate, because each person has a different experience and, at the end, always can contribute with something (FG#3)
… I always preferred the individual works ( … .) but working in groups compels us to see other points of view and to think ( … .) I think the teamwork is very important. (FG#3)
… we have our own agendas. It becomes difficult for us to reconcile our lives with the lives of our colleagues in order to prepare for a teamwork. But we can do it! (FG#2)
… There is always a person who is more organized and puts some pressure on the others [of the group] with timings, with the readings of the themes, with participation and with doubts. (FG#3).
Although the regulations presented as support material have had good appreciation in the questionnaire survey, this issue deserved some reflection in the focus group interview:
… If everything were condensed in a simpler way, or drafted annually with a condensation of all the legislation, it might help (FG#1)
… at the level of legislation could be organized by themes ( … ) Because, it's like FG#1 said, not everyone is from the food area (FG#3).
Regarding the available material, such as the power point, students considered important to make some adjustments:
… The commented videos, in my opinion, work very well. In this case, with power-points commented. ( … ) we are also able to apprehend some things that only the slide itself, or just that slide, does not transmit us. (FG#2).
In the discussion of the focus group, also some tools were suggested to innovate the form of explanation of the lessons. Thus, students suggested commented power-point presentations. In this way, the student follows the teacher’s comments and has better study orientation. Students also suggested videos:
… in my opinion, commented power-points work very well. That is, as the matter is being introduced, we are also able to apprehend some things that only that slide cannot. I'll give you an example: I've done some training online and when they're written I have more difficulty because I'm alone. When the teacher is exposing the matter, that is to say, he is speaking, in a very fluent way, and he is explaining things to us in a simpler and more concrete way, I think it helps a lot this kind of exposure because we can assimilate more (FG#2)
… There is even the case of edX that make free courses. They make many presentations with videos and then, when they want to deepen a certain theme, they refer to the reading of certain documents, or even pages of the specialty. It's a good tool, I guess! (FG#1).
Contents of the course
None of the statements related to the contents of the course obtained negative appreciation (Fig. 3). The module of legislation and food safety policies received 100% of agreement. The percentage of strongly agreement was 37.5% and 50%, respectively and the percentage of somewhat agreement was, respectively 62.5% and 50%. Neither of these modules obtained neutral nor negative appreciation. The content on microbiology and microbial contamination received the highest percentage of “strongly agree” (62.5%) response, against 25% of “somewhat agree” and 12.5% of neutral perception. The content on food additives obtained 50% of strongly agreement against 37.5% of “somewhat agree” and 12.5% of neutral perception. In contrast, the content on Food Quality additives obtained 37.5% of strongly agreement against 50% of “somewhat agree” and 12.5% of neutral responses. The online questionnaire responses’ appear to be in line with the statement of a student in the focus group discussion:
… all contents are important (FG#1).
Acquisition of Competences
Student expectations are related to the acquisition of knowledge, research skills, and competences development. Regarding Food Safety and Quality acquisition of competences, students agreed that they had gained competences throughout the course (Fig. 4). In particular, the competence that allows to evaluate the main dangers to food safety and its influence on the quality of the food product and the competence that allows to understand the importance of food hygiene in good food production practices obtained 100% of agreement (both had 75% of strongly agreement and 25% of somewhat agreement). The recognition of good practices of food storage and food transport also gained 75% of strongly agreement and 12.5% of somewhat agreement. This statement obtained 12.5% of indecisive opinion (or neutral opinion). Clearly positive were also the remaining perceptions: “understanding the concept of food quality”; “knowing the methodology of implementation of quality systems in the agrifood industry” and “Knowing the implications associated with the legislation in the field of food safety”, all obtained, 62.5% strongly agree, 25% somewhat agree and 12.5% neutral opinion.
One of the students add a very positive comment related to student expectations about the overall course acquired competences:
… It was one of the most relevant courses! It opened my horizons in many aspects. It made me appreciate the industrial and business work in this [food] area and realize that there are unskilled workers that are also great professionals.( … ) The ethical and food product added- value brings benefits to the population. ( … ) THANK YOU TRULY. (Q4).
In the Food Safety Acquired Competences dimension, student focus group discussion reinforced their level of satisfaction:
… I, for my part yes. [The objectives of the course] were achieved. Yes, Yes, they were achieved! I have nothing to point out. (FG#3)
… I also think that everything went well (FG#1).
Teachers of the course
Students answered as 87.5% strongly agreement and 12.5% of partial agreement to the scientific competence of teachers of the course (Fig. 5). In relation to the guidelines and explanations given throughout the module, 25% strongly agree that were clear and 87.5% agree in part that they were clear. The sentence “Teachers stimulate the learning process” received 50% of strongly agreement, 37.5% of partial agreement and 12.5% of neutral or medium perception. Seventy-five percent of the respondents strongly agree that the activities proposed by the teachers facilitated the understanding of the subject; 12.5% agree partially and 12.5% have neutral perception. The statement “The feedback given to the activities carried out was fundamental” received 50% of strongly agreement, 25% of somewhat agreement, 12.5% of neutral response and 12.5% got negative perception.
Regarding this aspect, in the focus group discussion, students explained that the feedback to their assignments constitute an important tool for them.
… I'm looking for guidance. I'm trying to find out if I'm on the right way and if the methodology I'm using is the right one (FG#1)
… Yes, I agree with FG#1. I think feedback is important because when we do an activity, and sometimes we find that is to follow a certain orientation and then, in the end, it was to follow another. (FG#3).
Relation between the course and professional activity of students
From the total of respondents, 75% answered that the course is related with their professional activity (50% of strongly and 25% of somewhat related), and the remaining 25% not related (12.5% strongly not related and 12.5% somewhat not related) (Fig. 6).
It was stimulating to verify that 62.5% of the students considered that the course helped to update their knowledge related to professional activity (37.5% strongly agree and 25% somewhat agree). As expected, and considering that there are two students whose professional activity is not directly related to the course area, 12.5% had an undecided opinion and 25% considered that there was no update of the knowledge related to their professional activity (12.5% strongly disagree and 12.5% somewhat disagree). Seventy-five percent of students answered that the course changed their view on the issues of Food Safety and Quality (62.5% strongly changed and 12.5% changed partially). The remaining 25% of students were distributed by 12.5% neutral opinion and 12.5% considered that they did not change their view on food safety issues.
The percentage of students that believed the course gave them the necessary tools to their day-to-day life strongly was 62.5% (50% strongly believed and 12.5% partially believed). No one considered that did not get the necessary tools to their day-to-day life. Only 32.5% of neutral or undecided answers were obtained.
The results of the assessment of changing behaviour shows that half of students consider that this course made them change behaviour in their daily life. Only 12.5% considered that they did not change their behavior and 12.5 positioned themselves on an intermediate scale. Regarding the perceptions of changing attitudes in the daily life, 37.5% answered that they had not changed and 37.5% said that they had changed. In the intermediate position are the remaining 25% of the respondents.
In both academic years of 2016/2017 (n = 7) and 2017/2018 (n = 11), eighteen master students of the curricular unit of Food Safety and Quality were active in the Moodle Platform. The total posted messages are represented in Fig. 7.
In topic 1 it was proposed an individual work in which the student should explain the concepts of food quality and food safety (see Table 1). Individual work triggered a reduced number of interactions on the platform: 8 in the 2016/2017 and 14 in 2017/2018 (Fig. 8).
In Topic 2 an e-activity was proposed in which the students had to work in groups of 3 and 4 elements to solve a real problem (phase 1), and later, after submitting their proposal in the platform, the class would debate the proposals of each group (phase 2) (see Table 1). It is interesting to note that the greatest number of interactions happened near the end of the deadline for the delivery of teamwork for subsequent discussion in class (Fig. 9).
In Topic 6 students should choose a topic from the previous or the current one to be able to submit an abstract for a congress and construct one power point (see Table 1). The simulation of a call for papers for one congress was provided to students. This activity was a mixed of individual work with subsequent discussion in group.
Figure 10 shows the posted messages of topic 6 in both academic years of 2016/2017 and 2017/2018. In 2016/2017, students made only 20 interventions and submitted the work, with subsequent discussion, very close to the deadline for the development of the activity. The fact that there were fewer students (n = 7 versus n = 11) in this year, also conditioned the lowest number of public messages published on the Moodle platform. In contrast, the development of the activity in the academic year of 2017/2018 had 84 messages and a more regular work rate. In the academic year 2016/2017 one major deadline was placed by the teachers. In contrast, in the 2017/2018, two major deadlines were placed: the deadline for the production of the abstract and the power-point and the deadline for class debate. In Fig. 10 it can be seen a higher level of interactions on those dates. The fact that there were messages posted even after the deadline is due to students who maintained an interest in participating in the discussion after that period.
The total number of posted messages in the topic 7 were 20 in 2016/2017 and 34 in 2017/2018 (Fig. 11). Here too, there is a clear answer to the deadlines imposed by the teacher on the course. The activity proposed was individual (find solutions to solve a real problem) with subsequent discussion and reflexion of the solutions found by each student with the whole class.
Because of people’s busy lifestyles, online education is both an appealing and effective mean of educating the food service workers. Online education allows these workers to be able to proceed at their own pace and identify their own personal course timeline (Shanley et al., 2004).
This study focused on students’ perception of the course organization and contents, acquired competences and student’s engagement after enrolling a Food Safety and Quality course of the master’s degree in Food Science that is taught in e-learning at a distance learning Portuguese public university (Universidade Aberta).
Results from the students surveyed shown in general a high level of satisfaction with organization of the course, acquired competences, pedagogical resources, activities for assessment, pedagogical model, learning platform and teachers. Students also made positive evaluations regarding interactivity in forum (with colleagues and with teachers). These results are aligned with earlier research conducted in other formal programs at Universidade Aberta related with science education (Azeiteiro et al., 2015; Bacelar-Nicolau et al., 2009, 2012, 2015). In particular Azeiteiro et al. (2015), showed that students have a very high level of motivation and satisfaction with the programmes, and having reasons to return again for new training at Universidade Aberta. Also in this research, most students had the perception of having reached an effective learning outcome of knowledge, competences, values, attitudes and behaviour and an improvement on their professional skills related with food safety and quality.
Nevertheless, some weak points were highlighted and some improvements were suggested. These are discussed below.
In the focus group, there was an opportunity to scalp the students’ concerns a bit more, and it was found that they would like that the teachers did more timely feedback because they are looking for a guide to their study. Timely and supportive feedback on submitted coursework can let students know about their progress in learning when peer-induced participation is lacking (Law et al., 2019). Furthermore, a quality teaching presence offering clear directions can enhance the learning efficacy of students (Law et al., 2010), and this directly affects both the social and cognitive presences. Nevertheless, Palmer and Holt (2009), obtained lower satisfaction rates for the items that evaluated the feedback given to the activities developed, but this fact can be related with the quality and content and way feedback is given. During the study analysis (2016/17 and 2017/18) the teachers of the course were overloaded with other courses and one of the teachers had a personal reason issue during the 2016/17 that affected the learning process, what had as consequence a delay in a timely feedback to the students. Usual feedbacks given by the curse teachers are very personalized and detailed (to each student and an overall to the all class) what usually support students learning performance. Besides their periodicity and contents, they can also be given through the use of videos or audios what enhance and develop continual learning (Phungsuk et al., 2017).
Regarding the e-activities of the course (see Table 1), issues related to teamwork compared with individual work were addressed. It was found that the students enjoyed both teamwork and individual activities, but students sometimes prefer individual assignment since it only depend on their time availability. This issue is relevant for students with full time jobs and families to take care. Likewise, the student-student and student-teacher interaction forums were well classified. Learning analytics showed the greatest turnout of students in a teamwork activity (Topic 2) and when there was class debate. Students with stronger learning motivation also tend to engage in course tasks and group activities more actively (Lee et al., 2019). From this perspective, learning motivation improves learning effectiveness through its positive and direct effect on social presence and reflects on enrolment. Interaction between students and teacher is vital in any learning experience (Joo et al., 2011; Eyal, 2012). The effort engaged by learners in finding information is a crucial component of learning performance, which implies that learners who do not actively engage in information processing actively are liable to engage only in surface learning and achieving modest learning performance (Law et al., 2019). So, although teamwork is more time demanding from student’s point of view it should be encourage to improve the e-learning process. Indeed, one of the week points found by the students were the high workload of the course, what is in accordance with earlier students perceptions about e-learning courses (Azeiteiro et al., 2015; Bacelar-Nicolau et al., 2015; Martinho et al., 2016).
It is interesting to note that the module of Legislation (topic 7), which was one of the themes most frequently mentioned in the focus group interview (in terms of complexity, requirement and constant updating obligation), obtained a very high degree of content satisfaction. Likewise, the issue of food security policies (one of the most applicable European regulations in force) has also been of student’s satisfaction. The activity related to this topic, although it was not a group activity (which usually leads to greater interactivity in forum), got the 3rd highest frequency of posted messages in forum (54 public messages). This turnout corroborates the students’ satisfaction in the questionnaire survey and the general satisfaction with the course.
Beyond the electronic resources available to students: scientific papers, eBooks, real cases and personal presentations, students suggested that there should be commented power points to improve their study orientation as well as videos. The increasing availability of mobile devices, as well as the fact that videos and computer simulations have become available for a wide range of science subjects, have led to these tools becoming an integral part of many science curricula (Rutten et al., 2011). Also other improvement can be implemented to enhance the learning process in this course, for example, the implementation of game-based e-learning activity about food additives or other contents within the course. Gaming have been shown a significantly increased college students’ knowledge, for example about Good Manufacturing Practices (Stevenson et al., 2018). Eventually, stories may also be part of games, and, in the field of education, the application of games supporting learning processes has been developed into an increasing body of research (Bidarra et al., 2013). A common implementation is called ‘gamification’; it identifies the notion of using elements of video games, such as points, levels, badges and achievements, and their application in professional or educational contexts (Bidarra & Rusman, 2017). Yu et al. (2018), involved unmotivated students to the issues of food safety, by examining the effects of a tablet PC simulation game as a class activity on students’ motivation to learn food safety. This experience showed that there was a significant improvement for students who participated in the simulation game activity regarding learning motivation. Badges are also integrated in the Moodle platform and can be easily added in the course each time a student accomplished a certain task or assignment. Complemented or integrated with gaming, problem based learning and the use of social networking sites (like Facebook or Linkedin) can be implemented in the e-activities also contributing for student’s stimulation (Phungsuk et al., 2017).
Another stimulating system to improve a collaborative activity is using Wikispaces. Aguilar et al. (2013) found that most of the participants that had assessed the Wiki experience agreed that it was a very interesting system, not only for interaction between students on different programmes, but also for the dissemination of the work that had been done. It also fostered interaction between students who would not otherwise have mixed with each other, or would have done so in a relatively tenuous manner in the context of the virtual classrooms.
Besides at UAb the pedagogical strategies have been developed to engage students as active participants (Pereira et al., 2008), and this more up to date, interactive, rewarded and motivational strategies (Nah et al., 2014) should be put in place. Knowing that the characteristics of an online learning environment have a great impact on student satisfaction (Thurmond & Wambach, 2004), it is imperative that the resource used is “skilful”, otherwise no one will be interested in the e- learning application (Violante & Vezzetti, 2013).
In this research the overall course organization competences, attitudes and behaviours were reported by students enrolled in a Food Safety and Quality course offered by the Portuguese Distance Learning University (Universidade Aberta). Three types of approaches to data collection were considered. An online questionnaire survey, a focus group interview and a temporal distributions of messages published on the Moodle platform (learning analytics).
The surveyed students showed a very high level of motivation and satisfaction with the course and had the perception of having reached an effective learning outcome in knowledge, competences, attitudes and behaviours. Nevertheless, there are pedagogical and learning issues that should be improved on the Food and Safety Quality course, most related with type of assignments, management of workload, use of more interactive educative tools, and importance of timely and effective feedback to students.
The results showed that some students prefer individual activities rather than group activities because those activities only depend on their time and availability. In fact, this feature of individual activities is very relevant for students who work full time and who have their families to care for. However, that the results also showed that the greatest turnout of students on the platform was detected in the teamwork and, essentially, when there was a class debate. The fact that students indorsed high degree of satisfaction to teamwork activities (although they initially revealed that individual assignments were most desirable) leads to think that the more appealing the tools are for students, the better their performance are in the course. Some of the answers to success, motivation and students’ satisfaction are found in the use of tools and activities easy to use and edit.
Students suggested that there should be commented power points to improve their study orientation as well as videos. Other improvements can be implemented to enhance the learning process in this course, for example, the implementation of game-based e-learning activity (gamification) and the use of Wikispace.
The results of this study helped to realize that students need activities that give them satisfaction in the learning process but also that the role of the teacher is very important. For example, feedback to activities is a point that has to be improved in future editions of this course.
This research aimed to contribute to new knowledge related with the effectiveness of post-graduates courses taught in e-learning about food safety, through a case study and where good practices were highlighted. Education in this field is very relevant worldwide considering the United Nation Sustainable Development Goals related with healthy life’s and the need of update lifelong training of working professionals in the field. In addition, this study aimed to contribute to the lack of clear direction as to how the higher education system would need to change in order to accommodate students’ needs.
This study has limitations most related with the size of the sample of the survey and focus groups and existing bias by the fact that the teacher conducted the survey. Future developments should include other scholar years and other research tools, other than questionnaire surveys for the assessment of competences development and behavior changes. More complete learning analytics can also be conducted namely, using different data mining techniques, statistics and visual analytics techniques (Adejo & Connolly, 2017).
Availability of data and materials
Data and materials will be provided by the corresponding author upon request.
Adejo, O., & Connolly, T. (2017). Learning analytics in higher education Development: A roadmap. Journal of Education and Practice, 8(15), 156–163.
Aguilar, A., Medina, F. X., Pons, J. A., & Saigí, F. (2013). Challenges and opportunities of 2.0 tools for the interdisciplinary study of nutrition: The case of the Mediterranean diet wiki. Universities and Knowledge Society Journal (RUSC), 10(1), 210–225 https://doi.org/10.7238/rusc.v10i1.1341.
Azeiteiro, U. M., Bacelar-Nicolau, P., Caetano, F. J. P., & Caeiro, S. (2015). Education for sustainable development through e-learning in higher education: Experiences from Portugal. Journal of Cleaner Production, 106, 308–319 https://doi.org/10.1016/j.jclepro.2014.11.056.
Bacelar-Nicolau, P., Caeiro, S., Martinho, A. P., & Azeiteiro, U.M., (2015). Attitudes, barriers and motivators as factors for sustainability of higher education e-learning programmes at Universidade Aberta, Portugal. In: Leal Filho, W., Brandli, L., Kuznetsova, O., Paço, A. (eds.). Integrative Approaches to Sustainable Development at University Level: Making the links (Part II). World Sustainability Series. Springer. XII, 567–582 pp- ISBN 978–3–319-10689-2.
Bacelar-Nicolau, P., Caeiro, S., Martinho, A. P., Azeiteiro, U. M., & Amador, F. (2009). E-learning for environment. The Universidade Aberta (Portuguese open Distance University) experience in the environmental sciences post-graduate courses. International Journal of Sustainability in Higher Education, 10(4), 354–367.
Bacelar-Nicolau, P., Martinho, A.P., Amador, F., Caeiro, S., & Azeiteiro, U.M. (2012). Online learning for sustainability: the student perception in an environmental science post-graduation. In: Gonçalves, F., Fernando J., Pereira, R., Leal Filho, W., Azeiteiro, U.M., (Eds.). Contributions to Sustainability. Series Umweltbildung, Umweltkommunikation und Nachhaltigkeit - Environmental Education, Communication and Sustainability, Vol. 33. Peter Lang, Frankfurt. 281–294. ISBN 978–3–631-61347-
Bidarra, J., Rothschild, M., Squire, K., & Figueiredo, M. (2013). The AIDLET model: A framework for selecting games, simulations and augmented reality environments in mobile learning. International Journal of Web-Based Learning and Teaching Technologies, 8, 50–71.
Bidarra, J., & Rusman, E. (2017). Towards a pedagogical model for science education : Bridging educational contexts through a blended learning approach. Open Learning: The Journal of Open, Distance and e-Learning, 0513 (February), 1–15. https://doi.org/10.1080/02680513.2016.1265442
Bosman, A., Brent, P., Cocconcelli, P. S., Conole, G., Gombert, D., Hensel, A., & Zilliacus, J. (2016). Expertise for the future: Learning and training in the area of food safety risk assessment. EFSA Journal, 14, 1–10 https://doi.org/10.2903/j.efsa.2016.s0503.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
Coelho, J., Teixeira, A., Bacelar-Nicolau, P., Caeiro, S., & Rocio, V. (2015). iMOOC on climate change: Evaluation of a massive open online learning pilot experience. International Review of Research in Open and Distance Learning, 16(6), 152–173 Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-84950297680&partnerID=40&md5=12d35f6ec6e729ba34665a897922a369.
EC. (2004). Regulation (EC) No 852/2004 of the European Parliament and of the Council of 29 april 2004 on the hygiene of foodstuffs. Official Journal of the European Union, 47. L 139, 3e21
EC, European Commission, (2012). “Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions: Rethinking Education: investing in skills for better socio- economic outcomes. Strasbourg”, available at: http://eur-lex.europa.eu/procedure/EN/202132 (accessed 3 may 2019)
EC, European Commission/EACEA/Eurydice (2014). Modernisation of higher education in Europe: Access, retention and employability 2014. Publications Office of the European Union. Luxembourg: Eurydice Report.
Eyal, L. (2012). Digital assessment literacy — The core role of the teacher in a digital environment. Journal of Educational Technology & Society, 15(2), 37–49.
FDA, Food and Drug Administration (2009). FDA Retail Food Program Steering Committee. FDA report on the occurrence of foodborne illness risk factors in selected institutional foodservice, restaurant, and retail food store facility types. Available at: http://www.foodprotect.org/issues/packets/2012ScribePacket/attachments/I_024_b.pdf. (Retrieved April 25, 2019).
Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304 https://doi.org/10.1504/ijtel.2012.051816.
Flynn, K. M., Ho, P., Vieira, M. C., Pittia, P., & Dalla, M. (2017). Food science and technology students self-evaluate soft and technical skills. International Journal of Food Studies, 6(2), 129–138 https://doi.org/10.7455/ijfs/6.2.2017.a1.
Huberman, A. M., & Miles, M. B. (1994). Qualitative data Analyis (2nd ed.). Newsbury Park, CA: Sage.
Joo, Y. J., Lim, K. Y., & Kim, E. K. (2011). Online university students' satisfaction and persistence: Examining the perceived level of presence, usefulness and ease of use as predictors in a structural model. Computers & Education, 57(2), 1654–1664.
Kim, T. D., Yang, M. Y., Bae, J., Min, B. A., Lee, I., & Kim, J. (2017). Escape from infinite freedom: Effects of constraining user freedom on the prevention of dropout in an online learning context. Computers in Human Behaviour, 66, 217–231.
Kneifel, W., Apprich, S., & Kleboth, J. (2012). The MSc Programme: Safety in the food chain. Available at http://www.dlwt.boku.ac.at/fileadmin/data/H03000/?H75000/lehre75/SIFC_Pra%CC%88sentation_22.05.12_screen.pdf
Krueger, R. A. Casey, M. A. (2015). Focus groups: A practical guide for applied research. 5th edition. Thousand Oaks, CA: Sage publications.
Lacave, C., Molina, A. I., & Cruz-Lemus, J. A. (2018). Learning analytics to identify dropout factors of computer science studies through Bayesian networks. Behaviour & Information Technology, 37(10–11), 993–1007. https://doi.org/10.1080/0144929X.2018.1485053.
Law, K. M. Y., Geng, S., & Li, T. (2019). Student enrollment, motivation and learning performance in a blended learning environment: The mediating effects of social, teaching, and cognitive presence. Computers & Education, 136, 1–12 https://doi.org/10.1016/j.compedu.2019.02.021.
Law, K. M. Y., Lee, V. C. S., & Yu, Y. T. (2010). Learning motivation in e-learning facilitated computer programming courses. Computers & Education, 55(1), 218–228.
Lee, J., Song, H. D., & Hong, J. A. (2019). Exploring factors, and indicators for measuring students’ Sustainable engagement in e-learning. Sustainability, 11, 985. https://doi.org/10.3390/su11040985.
Lin, Y. G., McKeachie, W. J., & Kim, Y. C. (2001). College student intrinsic and/or extrinsic motivation and learning. Learning and Individual Differences, 13(3), 251–258. https://doi.org/10.1016/S1041-6080(02)00092-4
Martinho, A. P., Vaz-Fernandes, P., Padrel de Oliveira, C., Bacelar-Nicolau, P., Azeiteiro, U. M., & Caeiro, S. (2016). Strengths and weaknesses of an e-learning program in environmental sciences at Universidade Aberta, Portugal. In: Walter Leal Filho and Paul Pace (eds.) Teaching Education for Sustainable Development at University Level. World Sustainability Series, Springer, Berlin. 49–66.
Mather, E. C., & McNiel, P. A. (2008). The online professional master of science in food safety degree program at Michigan State University: An innovative graduate education in food safety. Journal of Veterinary Medical Education, 33(2), 272–278 https://doi.org/10.3138/jvme.33.2.272.
Moura, A. P., & Aires, L. (2012). How to learn about individual food choice criteria subject? A case study on e-learning environment module. In J. Ribeiro, & L. Aires (Eds.), Investigação e Variantes Curriculares do Ensino Online: Desafios da Interculturalidade na Era Tecnológica. Portugal: CEMERI, Universidade Aberta.
Moura, A. P., Cunha, L. M., Azeiteiro, U. M., Aires, L., Graça, P., & de Almeida, M. D. V. (2010). Food consumer science post-graduate courses: Comparison of face-to-face versus online delivery systems. British Food Journal, 112(5), 544–556 https://doi.org/10.1108/00070701011043781.
Moura, A. P. D., Aires, L. (2018). Food and Sustainability:. An Emerging Subject in Sustainable Environmental Sciences Education Applying to the e-Learning Environment. 109–130.
Moura, A. P. Fernandes, A. P., Caetano, F., Carapeto, C., & Oliveira, C. (2014). Avaliação do Curso de Mestrado Online em Ciências do Consumo Alimentar: Aplicação do Modelo de Análise SWOT. In M. J. (Organizadores) Aires, L, Moura AP, Seabra F (Ed.), Educação a Distância e Diversidade do Ensino Superior (pp. 69–80). Universidade Aberta. Retrieved from: https://www2.uab.pt/producao/eBooksArea/EaDDES/actasUAb.pdf.
Nah, F. F., Zeng, Q., Telaprolu, V. R., Ayyappa, A. P., & Eschenbrenner, B. (2014). Gamification of Education: A Review of Literature. In: F.F.H. Nah (Eds.). HCIB/HCII 2014, LNCS 8527, Springer, 401–409.
Oliveira, C., Carmo, H., Caeiro, S. (2016). Avaliação de uma unidade curricular do programa de doutoramento em Sustentabilidade Social e Desenvolvimento da Universidade Aberta de Portugal. In Maciel, C., Alonso, K. M., Peixoto, J. (eds.). Educação a Distância Experiências, Vivências e Realidades. Editora Ed UFMC, Mato Grosso, Brasil, 297–320. ISBN 978-85-327-0586-0.
Ovca, A., Jevšnik, M., & Raspor (2017). Food safety practices of future food handlers and their teachers, observed during practical lessons. British Food Journal, 120(3), 531–548.
Palmer, S. R., & Holt, D. M. (2009). Examining student satisfaction with wholly online learning. Journal of Computer Assisted Learning, 25(2), 101–113 https://doi.org/10.1111/j.1365-2729.2008.00294.x.
Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data Mining in Practice : A systematic literature review of empirical evidence the research questions, 17, 49–64.Ytics and educational data Mining in Practice : A. Educational Technology & Society, 17(October), 49–64 Retrieved from https://search.proquest.com/docview/1660157007?pq-origsite=summon.
Patton, M. Q. (1990). Qualitative evaluation and research methods. Sage, Newbury Park, California.
Pereira, A., Mendes, A.Q., Morgado, L., Amante, L., & Bidarra, J. (2008). Universidade Aberta's pedagogical model for distance education©. Universidade Aberta, Lisbon. Available at: https://repositorioaberto.uab.pt/bitstream/10400.2/2388/1/MPV_uaberta_english.pdf (retrieved April 26 2019).
Phungsuk, R., Viriyavejakul, C., & Ratanaolarn, T. (2017). Development of a problem-based learning model via a virtual learning environment. Kasetsart Journal of Social Sciences, 38(3), 297–306.
Robinson, N. (1999). The use of focus group methodology - with selected examples from sexual health research. Journal of Advanced Nursing, 29(4), 905–913. https://doi.org/10.1046/j.1365-2648.1999.00966.x.
Rupert, D. J., Poehlman, J. A., Hayes, J. J., Ray, S. E., & Moultrie, R. R. (2017). Virtual versus in-person focus groups: Comparison of costs, recruitment, and participant logistics. Journal of Medical Internet Research, 19(3), e80. https://doi.org/10.2196/jmir.6980.
Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2011). The learning effects of computer simulations in science education. Computers & Education, 58, 136–153.
Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students, fourth ed. Pearson Education Limited, Harlow, England, p. 624
Scallan, E., & Angulo, F. J. (2007). Surveillance for foodborne diseases. In N. M. M'ikanatha, R. Lynfield, C. A. V. Beneden, & H. Valk (Eds.), Infectious disease surveillance, (pp. 57–70). New Jersey: Blackwell Publishing.
Scott, E. (2003). Food safety and foodborne disease in 21st century homes. Canadian Journal of Infectious Diseases, 14(5), 277–280.
Shanley, E. L., Thompson, C. A., Leuchner, L. A., & Zhao, Y. (2004). Distance education is as effective as traditional education when teaching food safetys. Food Service Technology, 4(1), 1–8 https://doi.org/10.1111/j.1471-5740.2003.00071.x.
Stevenson, C. D., Porter, K., & Stevenson, K. T. (2018). Effects of a game-based e-learning module on undergraduate food science students’ planned behaviors concerning good manufacturing practices. Journal of Food Science Education, 17(4), 111–117 https://doi.org/10.1111/1541-4329.12148.
Thurmond, V., & Wambach, K. (2004). Towards an understanding of interactions in distance education. Online Journal of Nursing Informatics, (OJNI), 8(4).
UAb (2019a). Web Page of Universidade Aberta. Available at: https://portal.uab.pt/en/auab/ (Retrieved 3 may, 2019)
UAb (2019b), Web Page of Universidade Aberta. Available at: http://www2.uab.pt/guiainformativo/eng_detailcursos.php?curso=25 (Retrieved 3 may, 2019)
UN (2016). Transforming our world: The 2030 agenda for Sustainable Development. In A/RES/70/1. United: Nations.
UN Sustainable Development Goals. Available at: http://www.un.org/sustainabledevelopment/ sustainable-development-goals// (Retrieved April 30 2019).
Violante, M. G., & Vezzetti, E. (2013). Virtual interactive E-learning application: An evaluation of the student satisfaction. Computer Applications in Engineering Education, 23(1), 72–91 https://doi.org/10.1002/cae.21580.
WHO & FAO (2009). Food Hygiene (Basic Texts), 4th ed., World Health Organization, Food and Agriculture Organization of the United Nations, Rome.
WHO, World Health Organization (2008). Foodborne disease outbreaks: Guidelines for investigation and control. Available at: https://apps.who.int/iris/bitstream/handle/10665/43771/9789241547222_eng.pdf;jsessionid=218C69978B89CA8653571F5347774625?sequence=1. (Retrieved April 25, 2019)
WHO, World Health Organization (2013). Advancing Food Safety Initiatives: Strategic plan for food safety including foodborne zoonoses 2013–2022. (Available at: http://www.who.int/foodsafety/strategic-plan/en/. (Retrieved April 25, 2019).
WHO, World Health Organization (2015). WHO’s first ever global estimates of foodborne diseases find children under 5 account for almost one third of deaths. Available at: https://www.who.int/en/news-room/detail/03-12-2015-who-s-first-ever-global-estimates-of-foodborne-diseases-find-children-under-5-account-for-almost-one-third-of-deaths. (Retrieved April 25, 2019).
Wilkinson, S. (1998). Focus group methodology: A review. International Journal of Social Research Methodology, 1(3), 181–203. https://doi.org/10.1080/13645579.1998.10846874.
Yu, H., Sirsat, S. A., & Madera, J. M. (2018). Leisure , Sport & Tourism Education Enhancing hospitality students ’ motivation to learn food safety knowledge using tablet personal computer (TPC) -based simulation game class activity. Journal of Hospitality, Leisure, Sport & Tourism Education, 23(August), 82–94. https://doi.org/10.1016/j.jhlste.2018.08.002
Zanin, L. M., da Cunha, D. T., de Rosso, V. V., Capriles, V. D., & Stedefeldt, E. (2017). Knowledge, attitudes and practices of food handlers in food safety: An integrative review. Food Research International, 100(July), 53–62. https://doi.org/10.1016/j.foodres.2017.07.042
The authors would like to express their special thanks to the students that participate in the surveys and Ana Sofia Mil-Homens for allowing the dissemination of data from the Food Safety and Quality course of the MSc program in Food Science at Universidade Aberta. We also thank to Portuguese Science and Technology Foundation (FCT) for its funding (UID/AMB/04085/2019) to Center for Environmental and Sustainability Research (CENSE), NOVA School of Science and Technology, NOVA University and also for other funding (UID/CPO/00713/2019), to Center for Public Policy and Administration (CAPP), ISCSP/Universidade de Lisboa.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.