Scenarios in virtual learning environments for one-to-one communication skills training

Selecting relevant literature

Learning environments for communication skills are used in very different contexts, and collecting the available (descriptions of) environments is non-trivial. We searched for papers on learning environments for communication skills on Scopus by looking at the title, abstract, and keywords using a query and refined the query stepwise.

The first step is a search for use of a scenario in communication skills. This leads to 2000+ results in 27 subject areas ranging from Chemistry to Humanities. The first articles were published in the 1970’s; there are 100+ results each year since 2006, the number of results rising steadily till 2013 to 200+ articles each year and a slight decline in 2016 (176 articles).

In a meta-analysis of the cognitive and motivational effects of serious games, Wouters, Van Nimwegen, Van Oostendorp, and Van Der Spek (2013) find that serious games were found to be more effective in terms of learning and retention, but not more motivating than conventional instruction methods. We refine our search to the area of serious/applied games. This leads to 668 results in 23 subject areas. The first articles date from the 1980’s, and there are already ten articles in 2017.

For this paper we refine the search query to virtual environments and simulations. This leads to 19 results; the first article is from 2008 and there are fewer than 5 articles per year since. The papers are in 9 subject areas: Psychology, Earth sciences, Chemistry, Humanities, Medicine, Mathematics, Engineering, Social sciences and Computer science. Finally, we refine the search further to exclude environments that focus on agent approaches. This search leads to fourteen papers and the trend is the same as in the previous step.

The final search query is:

The use of scripted scenarios in virtual learning environments for communication skills training is a relatively new area used in selected domains.

Besides this query, we asked communication skills teachers from the faculties of Pharmacy, Medicine, Veterinary medicine, and Psychology at our university to provide references to research papers on learning environments for communication skills in their respective fields. After scanning the abstracts returned by our query and the suggestions from the communication skills teachers, we selected nine papers that describe a learning environment with scripted communication skills scenarios (Bosse and Provoost 2015; Bracegirdle and Chapman 2010; Cláudio et al. 2015; Gebhard et al. 2011; Guo et al. 2014; Jeuring et al. 2015; Leuski and Traum 2011; Réty et al. 2008; Wauters et al. 2012).

Analysing the literature

Bracegirdle et al. (2010) present a ‘programmable patient’, with a ‘brain’ that is essentially a scenario in our description. Clinical educators can add subjects to a scenario. A student asks or selects questions that correspond to nodes in a tree. If a question is not recognized, a student can choose to store the question together with an answer in this tree. Scenarios are not completely scripted: questions corresponding to nodes can be asked in various ways. A scenario does not affect the emotion of a programmable patient (virtual character) in this game. A programmable patient is standardised in comparison to an actor and this leads to equity in assessment.

Bosse et al. (2015) use a scenario in which a student chooses between several statements. In addition to statement choices for a student, a statement of a virtual character may also have choices. The latter choices depend on values of emotional parameters, which in turn may vary for different characters. A statement choice of a student influences the value of these parameters. The main technology in the implementation is a scenario with conditional branches, in which emotional parameters are used. Students find this game an effective learning tool.

Gebhard et al. (2011) present Visual SceneMaker, an authoring tool for creating interactive applications with multiple virtual characters. SceneMaker separates content, such as statements, from logic, such as conditional branching. Content is specified in a scenario in the form of statements and stage directions for controlling gestures, postures, and facial expressions. The logic of an interactive performance and interaction with virtual characters is controlled by a sceneflow. A sceneflow is a hierarchical scenario with nodes that structure content of an interactive presentation and edges that specify how this content is linked together. Visual SceneMaker is evaluated in experiments; the visual programming approach and modelling structures are comprehensible and let non-experts create virtual character applications in a rapid prototype fashion.

Claudio et al. (2015) demonstrate Virtual Humans (VH) in an interactive application, the goal of which is to train and assess self-medication consultation skills. A domain expert develops a scenario using an authoring tool called Dialogue Creator. A scenario is a graph implementation and consists of a sequence of statements between a VH patient and a player, with associated scores per answer. Each node in a graph corresponds to a statement that is articulated by a VH or a statement option for a player and its associated score. Seven pharmacy experts performed an evaluation of the application. These experts consider this application a valuable tool for training and assessing the over-the-counter counselling skills of a player, and recommend to follow such training with real interactions and live evaluation.

Leary’s Rose (2012) focusses on natural language recognition of player sentences. Input sentences are classified on two axes: the vertical axis indicates whether a speaker is dominant or submissive towards a listener, and the horizontal axis indicates a speaker’s willingness to cooperate with a listener. A scenario engine selects an appropriate response to a player’s input on the basis of identified keywords and relative positions of both player and Virtual Agent on the axes. A player’s input is fed to a finite state machine (FSM) and if an input matches a state, the FSM selects a reply from the available follow-up states.

Rety et al. (2008) present a framework for interactive narrative authoring in the domain of virtual storytelling where a user is provided with navigation choices in an interactive narrative. The framework is based on a generalised concept of a section, which is close to a subject in our description. A subject consists of hierarchically structured fragments (a statement in our description). A statement is defined as an atomic content element containing simple text and possibly other media. A statement may also be composed recursively and consist of a set of statements, a behaviour and a termination property. Behaviour can be set by an author as either deterministic to indicate that a user reads in the order of subjects or as non-deterministic to indicate that a user reads with no predefined order. A termination property is a condition that an author sets to determine when a subject terminates. A user then moves on to a following subject. A user reads an interactive narrative created in this framework by successively accessing statements in a subject.

Leuski and Traum (2011) describe a Non Player Character (NPC) editor, which includes natural language understanding and generation, and statement management. This authoring tool is designed primarily to develop question answering characters. The focus is to deliver an appropriate answer for a user’s question. An author creates a response handling strategy by populating a language database with specific answers to sample questions and assigning these answers to an individual character. Different characters may have different knowledge about an event and respond differently to the same question. At run-time, a statement manager matches a user’s question to an answer of an associated non player character. NPCEditor has been evaluated and is an effective and versatile system in applications for virtual question answering characters. As future work, the autors mention dialogue management as a specialized component.

Jeuring et al. (2015) describe Communicate, a serious game for practicing communication skills. An important aspect of Communicate is the de-coupling of scenario development by communication skills experts from the implementation. A communication expert iteratively develops a scenario as a directed acyclic graph of statements in an authoring tool. A statement has an incremental score, emotional effect, and feedback. Statements can be structured under subjects. A scenario parser uses this graph to generate a scenario specific reasoner. During gameplay, a player receives information about statement choices at each step in a series of interactions with a virtual character. At the end of a simulation a player receives a final score on communication skills.

Guo et al. (2014) present a game prototype for practicing communication skills in a medical consultation process. The communication skills module accesses knowledge content and a scenario through a restful web service. Knowledge is modelled as rules, which determine which player actions have a positive or a negative effect during a medical consultation, as well as the influence of each type of action on a patients stress level. A scenario models a virtual patient with some key characteristics, which are chosen from the knowledge content. A scenario also defines possible actions for a player with a virtual patient in each phase of a medical consultation process.

Dialogue authoring assets

Articy is a visual environment for the creation and organization of game content offering virtual characters, locations and a game development environment. Chatmapper has a basic free version available. Twine is an open-source tool for creating interactive, non-linear stories and resembles Interactive narratives (Réty et al. 2008).

Scenario definition

Bosse et al. (2015) present a game for aggression de-escalation training with a scenario simulation in a public transport tram. Bracegirdle et al. (2010) and Guo et al. (2014) present simulations in clinical communication skills which play in a hospital and a clinic respectively. Claudio et al. (2015) present a game which plays in a pharmacy and allows student engagement in a true-to-life situation in a controllable environment. Jeuring et al. (2015) present a simulation where a teacher can select from virtual characters and backgrounds in a context of a scenario. According to Realdon et al. (2006), an adequate scaffolding structure for the training of communicative competence in an e-learning environment entails fixing both contextual boundaries and degrees of freedom to let the learner have the opportunity to give sense to the perspective selected. We infer that a communication scenario has a context in which a student/user plays. Three possible components of a context are for example a location, a background image and a hair color for an interactive virtual character. We define these components as properties for a scenario that provide context during gameplay. A property is a static piece of information for a scenario, a statement or a virtual character. An example of a property for an entire scenario is a location, of a property for a statement is an intent of a player, and of a property for a virtual character is a name.

Since we search for literature on serious or applied games, most papers (Bosse and Provoost 2015; Bracegirdle and Chapman 2010; Cláudio et al. 2015; Gebhard et al. 2011; Guo et al. 2014; Jeuring et al. 2015; Wauters et al. 2012) mention learning goals. A player’s choices determine his/her score on these learning goals. A player can get feedback either during gameplay following a player choice or at the end of a scenario. In some of the games (Bosse and Provoost 2015; Jeuring et al. 2015; Wauters et al. 2012) an emotional state of a virtual character can change depending on a player statement choice. In papers that describe an authoring tool for scenarios (Bosse and Provoost 2015; Gebhard et al. 2011; Jeuring et al. 2015; Leuski and Traum 2011; Réty et al. 2008), a scenario writer may change such an emotional state or progress towards a learning goal by defining an effect on a parameter at a statement. We define a parameter as a dynamic piece of information of a scenario that can be modified per statement. Examples of parameters are a character’s emotion and a score for a scenario.

In interactive narratives (Réty et al. 2008) a user may read subjects in a hierarchy without any predefined order. To overcome predictable behaviour in a conversation with an agent, agents in conversational trees (Bosse and Provoost 2015) are endowed with an internal state to introduce variability in statements. In Communicate! (Jeuring et al. 2015), interleaving enhances expressiveness in a communication scenario by allowing students to navigate a conversation with a virtual character through subjects. If the order in which statements within two subjects (or more) are navigated is not important, then these subjects are interleaved. We infer that variability in navigating subjects in a scenario is important to add expressiveness to a scenario and partly overcomes predictable behaviour. We characterise this variability for two (or more) subjects as interleaved in a scenario where a player gets statement choices from within these interleaving subjects without a predetermined order. We define a scenario as a sequence of interleaved subjects.

XML schema for validating a scenario

We define the Utrecht University Dialogue Scenario Language (UUDSL) as an XML language for describing the structure of a scenario including a dialogue structure, virtual character(s), properties (e.g. location, character name) and parameters (e.g. emotion, score).

It is schematically represented in Fig. 1

The top-level element contains definitions, metadata and an element for sequencing interleaved subjects. Characters, properties and parameters are defined inside the definitions element for reference in the rest of the dialogue scenario. A property is a static piece of information with a scope. A property scope can be per-statement or independent of statements, which evaluates to a piece of information about a scenario. A parameter is, in contrast to a property, a dynamic piece of information and is by definition per-statement. A statement can change the value of a parameter. A character has an ID and a set of properties and parameters.

The metadata element contains all data that describes a scenario, but is not part of it (e.g. name, description). The metadata element also contains property values and initial parameter values. These initial parameter values are used to initialise a scenario’s parameters during game-play. Property values stored in metadata define properties of a scenario.

The sequence element contains a sequence of interleaved subjects. The interleave element delimits the interleaving dialogue elements, which represent a piece of dialogue about a subject. Inside a dialogue element there are statements representing the contents and effects of the dialogue. A statement element contains text, preconditions, property values, parameter effects, links to possible response statements and whether the statement belongs to a player or computer.

The UUDSL can be accessed at: https://uudsl.github.io/scenario/.

An authoring tool for creating a scenario

In this section we present a scenario editor, an authoring tool in which a communication expert (an author) iteratively develops a communication scenario as a sequence of interleaved subjects, consisting of statements with parameters per step. This editor has been developed in collaboration with communication teachers from our university and external users, and runs in a web-browser.

Figure 2 shows an example scenario, which consists of 5 subjects.

Subjects on the same horizontal level are interleaved. In Fig. 2, ‘EatingOut’, ‘Cooking’, and ‘Take away’ are interleaved subjects. This indicates that a player will get statement choices from within these subjects in no particular order during a simulation of this scenario. She can navigate these subjects in any order in a simulation.

A subject may be marked as optional, depicted in Fig. 2 with a question-mark, which means that the subject can be skipped altogether in a simulation.

Figure 3 below shows an example of a DAG for a subject (in this example the ‘EatingOut’ subject from Fig. 2).

The scenario editor is configurable: a game developer can specify a virtual character(s), properties (e.g. name of a character) and parameters (e.g. emotion and gesture) specific to a game. We provide a configuration XML schema to create a game-specific instance of the authoring tool. A scenario author, usually a non-programming expert, can use this instance of the scenario editor to create a scenario(s) specific to a game. The resulting scenarios and their associated properties and parameters can be used by the game developer for the game.

In an example configuration, the type of a property such as location can be specified by a game developer as an enumeration with the elements town, village and city. This enables a scenario author to select a location from the enumeration as a context for a scenario. The name property of a virtual character can be specified by a game developer as a value of the type string, which allows a scenario author to define the name of the character. The emotion for a virtual character can be specified by a game developer as an enumeration parameter consisting of elements angry, happy and sad. This parameter enables a scenario author to create an effect on the emotion of a virtual character at a statement. Additionally, a scenario author can also define a user parameter for a specific scenario, for instance a score as an integer, which can be set at a player statement. Figure 3 also displays the parameters of this example configuration of the authoring tool.

Our authoring tool is open-source and can be accessed at: https://github.com/UURAGE/ScenarioEditor.

An author can export a scenario from the authoring tool. The exported scenario is an XML document based on the UUDSL and can be validated using the UUDSL Schema. An exported scenario XML is integrated in a game.