Sample
To facilitate external validity, the goal was to recruit participants with an intrinsic interest in food-related topics who would potentially use online video lectures to acquire relevant information. Therefore, German university students enrolled in nutrition science programs were chosen as participants. Participants were contacted via email and social network sites and received eight Euro for participating in the online experiment. Participants who indicated at the end of the study that they answered the questions honestly and completed it without interruption and technical problems were included in data analyses. 18 participants were excluded from data analyses because they stated in their comments that they did not meet the study eligibility requirements (e.g., they were no longer nutrition science students), participated in the study several times or took much longer than the average participant to complete the study (completion time > mean completion time + 2 x standard deviation). The final sample contained 189 participants (168 female, 21 male) with an average age of 22 years (M = 21.97, SD = 3.39). Furthermore, the average participant was enrolled in their study program for 4 semesters (M = 3.65, SD = 1.99) and took 18 min (M = 17.59, SD = 4.46) to complete the study.
Design and material
A 2 (Professional Affiliation: Scientist vs. Lobbyist) × 2 (Study Involvement: High vs. Low) between-subject experimental design was used, resulting in four experimental conditions. For each experimental condition, an online video lecture on the topic of organic food was developed that consisted of two parts. In the first part of each video lecture, a course instructor (male, 58 years) stated the main topic of the lecture and introduced the upcoming expert. During the introduction, the course instructor mentioned that the expert had earned a diploma and a doctoral degree in nutrition science to demonstrate his content knowledge. Furthermore, he mentioned the expert’s current employer. In the second part of the video, the expert (male, 31 years) described the results of scientific studies with seemingly sound methodologies that allegedly had shown that organic food is tastier, healthier and better for the environment than conventional food. The scientific studies were fictitious and solely designed for the propose of this study. However, this was not mentioned during the lecture. The duration of the video lectures was approximately six minutes.
Professional affiliation manipulation: Scientist vs. lobbyist
Depending on the experimental condition, the expert was described either (a) as a scientist, who currently works for a university, or (b) as a lobbyist, who currently works for an organic food lobbying organization. Note that the German word for lobbying organization (“Interessenvertretung”), in comparison to its English counterpart, is a rather neutral expression that does not necessarily carry negative associations. The expert’s professional affiliation was communicated in two ways. In the first part of the video lecture, the expert’s professional affiliation was mentioned by the course instructor during his introduction. In the second part of the video lecture, a continually visible banner at the bottom of the video displayed the expert’s name and professional affiliation.
Study involvement manipulation: High vs. low
Depending on the experimental condition, the expert said that either (a) he had conducted the mentioned scientific studies himself (e.g., “To answer this question, I conducted a study.”), or (b) that other scientists had conducted the mentioned scientific studies (e.g., “To answer this question, scientists conducted a study.”). A full manuscript of the video lectures can be obtained from the authors on request.
Procedure
To facilitate external validity, the experiment was conducted online using the Questback EFS Survey© platform for data collection. Before the experiment started, participants were told that the experiment was addressing the communication of scientific information in online video lectures. Furthermore, they were informed about the general procedure of the upcoming experiment and that they could end the experiment at any time. To start the experiment, participants had to indicate that they had read all provided information and that they agreed to take part in the experiment. After that, participants indicated their age, gender, the university where they studied nutrition science and the semester they were currently in. Furthermore, they answered the control measures (see section “Control measures”). Following this, participants were randomly assigned to one of the four experimental conditions and watched the corresponding online video lecture (see section “Design and material”). The online video lecture was embedded in the survey and participants were told that it was part of an online course on the topic of nutrition science. After watching the online video lecture, participants answered the dependent measures (see sections “Credibility measures”, “Trustworthiness measures”, and “Instructional quality measures”). At the end of the experiment, participants were debriefed. They were told about the manipulations of the experiment, that all presented studies and their results were fictions and that they could contact the leading scientist if they had any further questions or comments. Furthermore, they could choose to leave their information to get reimbursed for their participation. The study was designed to comply with the ethical guidelines developed by the America Psychological Association (APA) and the German Psychological Society (DGPs). The study was approved by the Ethics Committee of the Faculty of Psychology and Sports Science at the University of Münster and all participants provided informed consent to participate in the study.
Control measures
To analyze whether the experimental groups differed in regard to characteristics that could affect the study results, four control measures were included: (1) The participants’ general eco-friendly behavior in everyday situations, which could suggest particularly strong opinions in regard to organic food (Eco-Behavior), (2) their prior knowledge about organic food (Prior Knowledge), (3) how often they watch videos online (Video Consumption) and (4) how often they watch online videos for educational purposes (Educational Videos).
Eco-behavior
The Umweltschützende Verzichtsbereitschaften Scale (Montada, Kals, & Becker, 2014) was used to assess participants’ eco-friendly behavior in everyday situations. Participants indicated how much they agreed with five statements on a scale ranging from 1 (totally disagree) to 7 (totally agree), e.g. “In winter, I’m willing to keep windows and doors closed in order to save energy for the sake of the environment”. A total score was generated by calculating the mean.
Prior knowledge
To assess prior knowledge about organic food, participants answered the question “How much do you know about the topic of organic food?” on a scale ranging from 1 (very little) to 7 (very much).
Video consumption
To assess general online video consumption, participants answered the question “How often do you watch videos online?” on a scale ranging from 1 (very rarely) to 7 (very often).
Educational videos
To assess online video consumption for educational purposes, participants answered the question “How often do you use online video lectures / online courses to acquire knowledge / skills?” on a scale ranging from 1 (very rarely) to 7 (very often).
Credibility measures
Credibility is a complex construct that can be operationalized in diverse ways. For the purpose of the current study, two different credibility measures were used: (1) A general credibility measure that assessed the overall credibility of the provided information (Message Credibility) and (2) a specific credibility measure that assessed how much the participants agreed with specific statements from the video lecture (Organic Food Attitude). On the following scales, participants indicated how much they agreed with the provided statements on a scale ranging from 1 (totally disagree) to 7 (totally agree); for each scale, a total score was generated by calculating the mean.
Message credibility
The Message Credibility Scale (Appelman & Sundar, 2016) was translated and adapted to assess the credibility of the provided information. Participants indicated how much they agreed with three statements, e.g. “The provided information was accurate”.
Organic food attitude
Participants were asked how much they agreed with five main statements that were supposedly backed by scientific studies and presented during the video lecture, e.g. “People who mainly consume organic food have fewer health problems than people who mainly consume conventional food”.
Trustworthiness measures
Depending on the research setting and question, different trustworthiness measures can be appropriate. For the purpose of the current study, two different measures were used: (1) A measure that assessed the manipulative behavior of the expert (Machiavellianism) and (2) a measure that focused on three general aspects of trustworthiness (Expertise, Integrity, and Benevolence). On the following scales, if not otherwise mentioned, participants indicated how much they agreed with the provided statements on a scale ranging from 1 (totally disagree) to 7 (totally agree); for each scale, a total score was generated by calculating the mean.
Machiavellianism
The German version of the Machiavellianism Subscale from the Dirty Dozen Scale (Jonason & Webster, 2010; Küfner, Dufner, & Back, 2014) was adapted to assess how manipulative the lecturer was perceived. Participants indicated how much they agreed with four statements, e.g. “The lecturer has used deceit or lied to get his way”.
Expertise, integrity, and benevolence
The Muenster Epistemic Trustworthiness Inventory (Hendriks, Kienhues, & Bromme, 2015) was used to assess how trustworthy the expert was perceived. Fifteen items were rated on a scale ranging from 1 (not trustworthy at all) to 7 (very trustworthy). Six items measured expertise (e.g. “competent - incompetent”), four items measured benevolence (e.g., “considerate - inconsiderate”) and four items measured integrity (e.g., “honest - dishonest”).
Instructional quality measures
Instructional quality is a broad construct that can be differentiated into various subcategories. For the purpose of the current study, three common measures were used: (1) A measure that assesses the general likability of the expert (Likability), (2) a traditional and widely used instructional quality measure that assesses the enthusiasm of the expert (Enthusiasm) and (3) a measure that focuses on the participants’ subjectively perceived learning gain (Subjective Comprehension). On the following scales, participants indicated how much they agreed with the provided statements on a scale ranging from 1 (totally disagree) to 7 (totally agree); for each scale, a total score was generated by calculating the mean.
Likability
The Reysen Likability Scale (Reysen, 2005) was translated and adapted to assess how likable the expert was perceived. Participants indicated how much they agreed with eleven statements, e.g. “The lecturer is likable”.
Enthusiasm
The Enthusiasm Subscale from the Students’ Evaluations of Educational Quality Questionnaire (Marsh, 1982) was translated and adapted to assess how enthusiastic the expert was perceived. Participants indicated how much they agreed with four statements, e.g. “The lecturer’s style of presentation held my interest during the online video lecture”.
Subjective comprehension
The Subjective Comprehension Subscale from the Recipient Orientation Scale (Bromme, Jucks, & Runde, 2005) was adapted to assess the subjective learning gain of the participants. Participants indicated how much they agreed with five statements, e.g. “I have the feeling that I have learned something new by watching the online video lecture”.
Manipulation check
Two additional measures were included to assess whether the participants correctly remembered the expert’s professional affiliation and his involvement in the mentioned studies.
Professional affiliation
To assess whether the participants remembered the expert’s professional affiliation, they were asked “For whom did the lecturer work?”. Participants could choose between “A university”, “A lobbying organization”, and “I do not know”.
Study involvement
To assess whether the participants remembered who conducted the studies presented in the video, they were asked “Who conducted the studies that the lecturer presented?”. Participants could choose between “The lecturer conducted the studies himself”, “Other scientists conducted the studies”, and “I do not know”.