1 of 45

Conversational AI Design and Practice

Preface

Jinho D. Choi

Artificial Intelligence (AI) has advanced to the point that it starts interacting with humans in natural language. This communication ability makes AI an integral part of human society as a collaborator and companion. Thus, it is essential to understand how AI can (and should) be designed to conduct meaningful conversations with humans. The main objectives of this course are:

To discover technical approaches to dialogue systems.
To conceive manifold use cases of dialogue systems.
To study effective ways of Human-Computer Interaction.
To develop a dialogue system using methods in Computational Linguistics.
To comprehend the limitations of your dialogue system through Statistical Analysis.

Students will have individual assignments and work in groups to build end-to-end dialogue systems of their choice. Toward the end of the semester, all teams will present the dialogue systems with live demonstrations.

LINC Course

For Spring 2023, this is selected as a course. Thus, it will include collaborative work with students taking .

Syllabus

Spring 2023

General

Book: https://emory.gitbook.io/conversational-ai
GitHub: https://github.com/emory-courses/conversational-ai
Time: MW 2:30PM - 3:45PM
Location: White Hall 112

Instructors

Jinho Choi Associate Professor of Computer Science Office Hours → MW 4PM - 5:30PM, MSC W302F
Talyn Fan Research Engineer at the Emory NLP Research Lab Office Hours → TuTh 1PM - 2:30PM, MSC W302B (https://emory.zoom.us/j/94134466856)
Benjamin Ascoli Ph.D. student of Computer Science and Informatics Office Hours → MW 11:30AM - 1PM, MSC W302B
Sichang Tu Ph.D. student of Computer Science and Informatics Office Hours → MW 1PM - 2:30PM, MSC W302B

Grading

1 + 7 topical quizzes: 55%
3 LINC discussions: 10%
Project proposal: 15%
Final project: 20%
Your work is governed by the Emory Honor Code. Honor code violations (e.g., copies from any source, including colleagues and internet sites) will be referred to the Emory Honor Council.
Excuses for exam absence/rescheduling and other serious personal events (health, family, personal related, etc.) that affect course performance must be accompanied by a letter from the Office of Undergraduate Education.

Topical Quizzes

For every topic, one quiz will be assigned to check if you keep up with the materials.
Quizzes must be submitted individually. Discussions are allowed; however, your work must be original.
Late submissions within a week will be accepted with a grading penalty of 15% and will not be accepted once the solutions are discussed in class.

LINC Discussions

You will meet with students from IDS 385W - Translation: Who, What, How three times:
- 02/08 (W): evening (60 mins), 6PM, Atwood 215
- 03/15 (W): evening (60 mins), 6PM, Atwood 215
- 04/24 (M): evening (75 mins), 6PM, Atwood 215
For the first two meetings, you will meet in small groups. These meetings will not replace our regular class hours.
For the third meeting, you will present your chatbot to LINC students. This meeting will replace our regular class hours.

Project Proposal

You are expected to:
- Group a team of 3-4 members.
- Give a presentation to propose your idea about the final project.
- Write a proposal that illustrates details about your proposed project.
Everyone in each group will receive the same grade for the project proposal.
Proposal Guidelines.

Final Project

You are expected to:
- Give a presentation about your final project.
- Give a demonstration of your system.
- Write a final report that illustrates details about your work.
Everyone in each group will receive the same grade for the final project.

Schedule

Spring 2023

Date

Topic

Assignment

0. Getting Started

This chapter helps you set up the development environment for this course.

0.1. Environment Setup

Setup a Python programming environment with GitHub and PyCharm.

Python

Install .
Lower versions of Python may not be compatible with this course.

GitHub

Login to (create an account if you do not have one).
Create a new repository called conversational-ai and make it private.
From the [Settings] menu, add the instructors as collaborators of this repository.
- Jinho Choi:
- Talyn Fan:
- Benjamin Ascoli:
- Sichang Tu:

PyCharm

Install on your local machine:

The following instructions assume that you have "PyCharm 2022.3.x Professional Edition".
You can get the professional version by applying for an .

Configure your GitHub account:

Go to [Preferences] - [Version Control] - [GitHub].
Press [+], select Log in via GitHub, and follow the procedure.

Create a new project:

Press the [Get from VCS] button on the Welcome prompt.
Choose [GitHub] on the left menu, select the conversational-ai repository, and press [Clone] (make sure the directory name is conversational-ai).

Setup an interpreter:

Go to [Preferences] - [Project: conversational-ai] - [Project Interpreter].
Click Add Interpreter and select Add Local Interpreter.
In the prompted window, choose [Virtualenv Environment] on the left menu, configure as follows, then press [OK]:
- Environment: New
- Location: LOCAL_PATH/conversational-ai/venv
- Base interpreter: Python 3.11

Install packages:

Open a terminal by clicking [Terminal] at the bottom (or go to [View] - [Terminal]).
If the terminal prompts "Successfully installed ...", the packages are installed on your machine.

0.2. Quiz

Quiz 0: Getting Started

Task

1. Create a package called src under the conversational-ai directory.

PyCharm may automatically create the __init__.py file under src, which is required for Python to treat the directory as a package, so leave the file as it is.

2. Create the quiz package under the src package.

3. Create a python file called quiz0.py under the quiz package and copy the code:

from emora_stdm import DialogueFlow

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        '[{good, fantastic}]': {
            '`Glad to hear that you are doing well :)`': {
                'error': {
                    '`See you later!`': 'end'
                }
            }
        },
        'error': {
            '`I hope your day gets better soon :(`': {
                'error': {
                    '`Take care!`': 'end'
                }
            }
        }
    }
}

df = DialogueFlow('start', end_state='end')
df.load_transitions(transitions)

if __name__ == '__main__':
    df.run()

If PyCharm prompts you to add quiz0.py to git, press [Add].

4. Run quiz0 by clicking [Run] - [Run 'quiz0']. An alternative way of running the program is to click the green triangle by the line #26 and select Run 'quiz0':

5. Conduct the following two dialogues with the chatbot you just created (S: system, U: user):

S: Hello. How are you?
U: I'm good
S: Glad to hear that you are doing well :)
U: Thanks
S: See you later!
==========================================
S: Hello. How are you?
U: It's not going well
S: I hope your day gets better soon :(
U: Hopefully
S: Take care!

This is what your directory structure may look like in PyCharm after finishing this assignment:

Submission

1. Create the .gitignore file under the conversational-ai directory and copy the content:

.idea/
venv/

2. Add the following files to git by right-clicking on them and selecting [Git] - [Add]:

quiz0.py
.gitignore

Once the files are added to git, they should turn green. If not, restart PyCharm and try to add them again.

3. Commit and push your changes to GitHub:

Right-click on conversational-ai.
Select [Git] - [Commit Directory].
Enter a commit message (e.g., Submit Quiz 0).
Press the [Commit and Push] button.

Make sure you both commit and push, not just commit.

4. Check if the above files are properly pushed to your GitHub repository.

5. Submit the address of your repository to Canvas.

1. Exploration

This chapter overviews dialogue systems, the technologies behind those systems, and their applications.

Content

Overview
Discussion
Quiz

Resource

Chapter 24: Chatbots and Dialogue Systems, Speech and Language Processing (3rd ed.), Jurafsky and Martin.

1.1. Overview

Explain components, properties, scopes, techniques, and assessments of dialogue systems.

Components

Genre

Conversation: interactive communication between two or more people.
Dialogue: a conversation, often between two people, with a specific goal in mind.

Dialog: a window that appears on a screen in computing contexts (e.g., dialog box).

Application

Dialogue System: a computer system that interacts with humans in natural language.
Conversational Agent: a dialogue system that interprets and responds to user statements.
Virtual Assistant: a dialogue system that performs tasks or services for user requests.
Chatbot: a dialogue system that simulates and processes human conversation.

Chatbots are typically understood to follow pre-defined dialogue flows for open-domain conversations without using sophisticated artificial intelligence technology.

Intelligence

Dialogue Management: a process of controlling the state and flow of the dialogue to conduct contextual communications.
Conversational AI: a type of Artificial Intelligence (AI) for a dialogue system to understand user inputs and respond properly to them, often processed by machine learning models.

What are examples of dialogue systems currently used in practical applications?
Are there applications that would greatly benefit from adopting dialogue systems?

Properties

Unit

Turn: a single contribution from one speaker to the dialogue.
Utterance: a natural unit of speech bounded by breaths or pauses.

For a text-based conversation, each turn is often considered an utterance.

Context

Speech Act: the action, either explicitly or implicitly, expressed by an utterance (e.g., answering, advising, greeting; see Switchboard Dialog Act Corpus).
Intent: the user's goal expressed by an utterance within the context of a conversation (e.g., making an appointment, requesting information).
Topic: the matter dealt with in an utterance (e.g., movie, family, midterm).

It is possible that one utterance expresses multiple speech acts and intents and also deals with various topics.

Classify each of the following utterances from Friends S1E1 using the dialogue acts: http://compprag.christopherpotts.net/swda.html

Ross: Hi.
Joey: This guy says hello, I wanna kill myself.
Monica: Are you okay, sweetie?
Ross: I just feel like someone reached down my throat, grabbed my small intestine, pulled it out of my mouth and tied it around my neck...
Chandler: Cookie?
Monica: Carol moved her stuff out today.
Joey: Ohh.
Monica: Let me get you some coffee.
Ross: Thanks.

Scopes

Task-oriented

Task-oriented dialogue systems have specific tasks to be accomplished:

The Second Dialog State Tracking Challenge, Henderson et al., SIGDIAL, 2014 (dataset).
Conditional Generation and Snapshot Learning in Neural Dialogue Systems, Wen et al., EMNLP 2016 (dataset).
Learning End-to-End Goal-Oriented Dialog, Bordes et al., ICLR, 2017 (dataset).
Key-Value Retrieval Networks for Task-Oriented Dialogue, Eric et al., SIGDIAL, 2017 (dataset).
MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling, Budzianowski et al., EMNLP, 2018 (dataset).
Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever, Qin et al., EMNLP, 2019 (dataset).
Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset, Rastogi et al., AAAI, 2020 (dataset).

Open-domain

Open-domain dialogue systems aim to talk about any topics without specific end goals:

Alexa Prize Socialbot Grand Challenge (Emora demo)
Meta BlenderBot (demo)
OpenAI ChatGPT (demo; requires login)
Google LaMDA (article; interview)

What kind of tasks are presented in the above task-oriented datasets?
Try the demos of BlenderBot and ChatGPT. What are their limitations?
What are the challenges in building task-oriented vs. open-domain dialogue systems?

Techniques

State Machine

A dialogue flow can be designed into a fine-state machine. Most commercial dialogue systems take this approach because it gives greater control over how the systems behave. Several platforms are available to facilitate the development of state machine-based dialogue systems:

End-to-End

Recent researches focus on developing end-to-end dialogue systems using sequence-to-sequence (S2S) models, which is a type of encoder-decoder model:

Sequence to Sequence Learning with Neural Networks, Sutskever et al., NeurIPS, 2014.

The current state-of-the-art S2S models use transformers such as BERT as their encoders:

Attention is All you Need, Vaswani et al., NeurIPS, 2017.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al., NAACL, 2019.

Three of the open-domain dialogue systems above, Meta BlenderBot, OpenAI ChatGPT, and Google LaMDA, are end-to-end systems based on S2S models.

Implementing an end-to-end system is beyond the scope of this course. Thus, we will use the state machine approach to develop dialogue systems, starting from Chapter 2.

Assessments

The primary objective of both task-oriented and open-domain dialogue systems is to satisfy users by communicating with them. For task-oriented, users are generally satisfied if the tasks are accomplished efficiently. For open-domain, however, user satisfaction is often highly subjective, so proper conversational analysis may need to be involved.

Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocols, Finch and Choi, SIGDIAL, 2020.
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges, Mehri et al., arXiv, 2022.
Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems, Finch et al., arXiv, 2022.

1.2. Project Ideas

Discuss project ideas in groups.

Before Discussion

Prepare your ideas about the questions in Quiz 1.

In-class Discussion

You will meet three groups during this class. Each team discussion will last 20 minutes, including the time to find group members:

Find 3-4 people you have not interacted with from your previous group(s).
Discuss your project idea with your group and check if anyone has a similar interest.

1.3. Quiz

Quiz 1: Exploration

Task 1

Create a PDF file quiz1.pdf that includes your ideas about the following questions:

What kind of a dialogue system do you want to develop for your team project?
Who will benefit from your dialogue system in what way?
What is the novelty of your dialogue system?
What are the expected challenges in developing your dialogue system?
How do you plan to evaluate your dialogue system?

Provide your ideas in detail, about 100 words (or more) per question.

Task 2

Group a team of 3-4 members for your project.

Submission

Submit quiz1.pdf to Canvas.
Go to [People → Team Projects] in Canvas and assign your team members to the same group.

2. Dialogue Graph

This chapter provides how to build a simple dialogue graph using Emora STDM.

Content

Resource

Source code:

2.1. Emora STDM

Introduce Emora State Transition Dialogue Manager.

Emora State Transition Dialogue Manager (STDM) is a dialogue system development framework that seamlessly integrates intent classification, pattern recognition, state machine, and information state approaches to dialogue management and natural language understanding. It supports rapid prototyping and long-term team-based development workflows, catering to a wide range of developer expertise.

Make sure you have the emora_stdm package installed from the Environment Setup.

References

Emora STDM: A Versatile Framework for Innovative Dialogue System Development, James D. Finch and Jinho D. Choi, Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue: System Demonstrations (SIGDIAL:DEMO), 2020.
Emora STDM GitHub Repository

2.2. State Transition

Create a dialogue flow using state transitions.

Dialogue Flow

Let us create a dictionary called transitions and name its initial state as start:

transitions = {'state': 'start'}

#1: transitions is a dictionary.

Add transitions to conduct a single-turn dialogue:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        'good': {
            '`Glad to hear that you are doing well :)`': 'end'
        }
    }
}

#3: the system begins the dialogue by saying, "Hello. How are you?".
#4: the system matches the user input with 'good'.
#5: the system responds to the user with "Glad to hear that ..." and recognizes that it is the final state end.

S: Hello. How are you?
U: Good!
S: Glad to hear that you are doing well :)

All keys and values must be in the string type, where literals need to be surrounded by reversed primes (e.g., '`Hello. How are you?`').

There are two ways to create a string in Python, using single quotes (e.g., 'Hello') and double quotes (e.g., "Hello"). When you use single quotes, any single quote inside the string needs to be escaped by a backslash (e.g., 'I\'m "good"!'). Similarly, when you use double quotes, any double quote needs to be escaped (e.g., "I'm \"good\"!").

Create the dialogue flow df that expects the initial state start and the final state end, which must match the names of the initial and final states in transitions:

from emora_stdm import DialogueFlow
df = DialogueFlow('start', end_state='end')

#1: the import statement should be on top of the source code (see dialogue_graph.py).
#2: keyword vs. positional arguments

Load the transitions to the dialogue flow df:

df.load_transitions(transitions)

Finally, run the dialogue flow:

if __name__ == '__main__':
    df.run()

#1: top-level code environment

Enter inputs such as "good", "Good", or "Good!" (separately) and see how the system responds. Does the system respond differently?
How does the system respond to inputs like "fantastic" or "bad"?

User inputs get automatically lowercased and tokenized before matching. Thus, the system gives the same response to those inputs.
If the input does not match 'good' exactly, the system throws an error because no exception is currently handled in our code.

Branching

Let us add a transition such that it can also handle the input 'bad':

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        'good': {
            '`Glad to hear that you are doing well :)`': 'end'
        },
        'bad': {
            '`I hope your day gets better soon :(`': 'end'
        }
    }
}

#7-9: a new transition for the 'bad' condition.

When you load the new transitions and run the dialogue flow, it now gives proper responses to both 'good' and 'bad':

S: Hello. How are you?
U: Good
S: Glad to hear that you are doing well :)

S: Hello. How are you?
U: Bad
S: I hope your day gets better soon :(

Error Handling

Even with branching, it still throws errors for other inputs. Let us add an error transition to set the default statement for all the other inputs:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        'good': {
            '`Glad to hear that you are doing well :)`': 'end'
        },
        'bad': {
            '`I hope your day gets better soon :(`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#10-12: an error transition to generate the default response.

S: Hello. How are you?
U: It could be better.
S: Sorry, I didn't understand you.

Make sure to put a default error statement for every branching; otherwise, it will throw an exception during runtime, which can be detrimental.

Code Snippet

from emora_stdm import DialogueFlow

def state_transition() -> DialogueFlow:
    transitions = {
        'state': 'start',
        '`Hello. How are you?`': {
            'good': {
                '`Glad to hear that you are doing well :)`': 'end'
            },
            'bad': {
                '`I hope your day gets better soon :(`': 'end'
            },
            'error': {
                '`Sorry, I didn\'t understand you.`': 'end'
            }
        }
    }

    df = DialogueFlow('start', end_state='end')
    df.load_transitions(transitions)
    return df

if __name__ == '__main__':
    state_transition().run()

#3: although Python is a dynamically-typed language, it allows you to indicate the type of a variable or a function using typing (since Python 3.5).

2.3. Matching Strategy

A taste of matching strategies provided in Emora STDM.

Multiple Matching

There is more than one way of expressing "good" or "bad". Let us use a to match multiple expressions per condition by surrounding them with curly brackets:

#4: matches the input with either 'good' or 'fantastic'.
#7: matches the input with either 'bad' or 'could be better'.

The above transitions can handle the following inputs (on top of the cases of good and bad):

A phrase can be used for matching, although it needs to match the entire input such that 'could be better' in #7 does not match an input like "It could be better".

Enter inputs such as "I'm good" or "Pretty bad" and see how the system responds.

It responds with the error statement because the string key (e.g., 'good', 'bad') does not match the entire input (e.g., "I'm good", "Pretty bad").

If you replace 'bad' in #7 with 'pretty bad', it matches the input "Pretty bad". However, it no longer matches the input "bad".

Currently, STDM does not allow you to insert a single quote (') in any condition such that if you replace 'good' in #4 with 'I\'m good', it will throw a runtime error. This bug will be fixed in the next version.

Partial Matching

#4: the keyword 'good' is surrounded by the square brackets (good -> [good]), and matched to any word in the input.
#7: the keyphrase 'could be better' is surrounded by the square brackets, and matched to any phrase in the input.

What are the differences between a set and a list in general data structures?

The above transitions yield the following dialogues:

Replace the condition in #4 with '[good, fantastic]'. Does it respond as expected?

All terms in a list must match some terms in the input sequentially. Thus, by replacing '[good]' with '[good, fantastic]', it no longer understands the inputs "good" or "fantastic":

Instead, it understands the input "good and fantastic" by matching "good" first, then "fantastic" later:

Nesting

To allow partial matching of multiple expressions, a set can be put in a list:

#4: the keywords 'good' and 'fantastic' are first put in a set for multiple matching and then in a list for partial matching.
#7: the keyword 'bad' and the keyphrase 'could be better' are first put in a set for multiple matching, then in a list for partial matching.

What are the meanings of nesting a set inside a list vs. a list inside a set?

What happens if '[{good, fantastic}]' in #4 is replaced with '{[good, fantastic]}'; in other words, put the keywords in a list first, then in a set?

It behaves the same way as the one in Exercise 2.

Sequential Matching

Keywords or keyphrases delimited by commas in a list are matched sequentially:

#10: first matches 'how', then matches 'you' sequentially so that "how" must be followed by "you" in the input for this condition to be true.

All terms in the list must be matched sequentially such that the conditions are false for the following cases:

Let us replace 'how' with '{how, and}' and 'you' with '{you, going}' in #10:

#10: first matches ('how' or 'and'), then matches ('you' or 'going') sequentially.

What other inputs does the condition in #10 match? Is there any consequence of matching that many expressions?

How does the system respond when the input is "Good, how are you"?

Since both the conditions in #5 and #11 match the input, it randomly chooses one of the responses such that it can nondeterministically generate either of the followings:

Code Snippet

2.4. Multi-turn Dialogue

Create a multi-turn dialogue flow.

All examples in the previous session are single-turn dialogues, where users get to interact with the system only once. Let us design a multi-turn dialogue:

#5: after the system says "Glad to ...", it matches the input with those conditions.
#6: if the condition in #5 is true, the system responds with "I feel superb ...".

The condition in #6 does not come with the default action indicated by an error transition. Let us create an error transition for it:

#10: prompts the error statement for any other inputs.

2.5. Quiz

Quiz 2: Dialogue Graph

Overview

You run a hair salon and recently adopted a dialogue system to take a call from a customer and book a service for the customer:

Your salon provides three services: haircut, hair coloring, and perms. Your system should reject any other service request from the customer:

Your system should understand the following dates: Monday ~ Saturday. The time is always followed by the date with the following format: <number><space><AM|PM>:

Only the following times and dates are available for the specific services:

Haircut:
- Monday 10 AM, 1 PM, 2 PM
- Tuesday: 2 PM
Hair coloring:
- Wednesday: 10 AM, 11 AM, 1 PM
- Thursday: 10 AM, 11 AM
Perms
- Friday: 10 AM, 11 AM, 1 PM, 2 PM
- Saturday: 10 AM, 2 PM

Thus, your system should not schedule an appointment for any other slots:

Make sure your system does not throw an exception in any situation.

Task 1

Update the transitions to design a dialogue flow for the above dialogue system.

Task 2

Create a PDF file quiz2.pdf that describes the limitations of your system.

Submission

Commit and push quiz2.py to your GitHub repository.
Submit quiz2.pdf to Canvas.

3. Contextual Understanding

This chapter explains how to improve contextual understanding using Natex.

Content

Natex
Ontology
Regular Expression
Macro
Quiz

Resource

Source code: contextual_understanding.py

3.1. Natex

Several matching strategies built in Natex.

Emora STDM supports several ways for interpreting the contexts of user inputs through Natex (Natural Langauge Expression), some of which you already experienced in Matching Strategy.

Literal

A literal is what you intend the system to say. A literal is represented by reversed primes (`..`):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': 'end'  # literal
}

#3: the system prompts the literal and ends the dialogue.

S: Hello. How are you?

Matching

Natex supports several ways of matching the input with key terms.

Term

The condition is true if the input exactly matches the term. A term is represented as a string and can have more than one token:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {         # literal
        'could be better': {           # term
            '`I hope your day gets better soon :(`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: matches the input with 'could be better'.
#7: error is a reserved term indicating the default condition of this conditional branching, similar to the wildcard condition (_) in a match statement.

S: Hello. How are you?
U: Could be better..
S: I hope your day gets better soon :(

S: Hello. How are you?
U: It could be better
S: Sorry, I didn't understand you.

Set

The condition is true if the input exactly matches any term in the set. A set is represented by curly brackets ({}):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {         # literal
        'could be better': {           # term
            '`I hope your day gets better soon :(`': 'end'
        },
        '{good, not bad}': {           # set
            '`Glad to hear that you are doing well :)`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#7: matches the input with either 'good' or 'not bad'.

S: Hello. How are you?
U: Good!!
S: Glad to hear that you are doing well :)

S: Hello. How are you?
U: Not bad..
S: Glad to hear that you are doing well :)

S: Hello. How are you?
U: I'm good
S: Sorry, I didn't understand you.

S: Hello. How are you?
U: Not so bad
S: Sorry, I didn't understand you.

Unordered List

The condition is true if some terms in the input match all terms in the unordered list, regardless of the order. An unordered list is represented by angle brackets (<>):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {         # literal
        'could be better': {           # term
            '`I hope your day gets better soon :(`': 'end'
        },
        '{good, not bad}': {           # set
            '`Glad to hear that you are doing well :)`': 'end'
        },
        '<very, good>': {              # unordered list
            '`So glad that you are having a great day!`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#10: matches the input with both 'very' and 'good' in any order.

S: Hello. How are you?
U: Very good!
S: So glad that you are having a great day!

S: Hello. How are you?
U: I'm very well and good
S: So glad that you are having a great day!

S: Hello. How are you?
U: Good, things are going very well!
S: So glad that you are having a great day!

S: Hello. How are you?
U: Good
S: Glad to hear that you are doing well :)

Ordered List

The condition is true if some terms in the input match all terms in the ordered list, a.k.a. sequence, in the same order. An ordered list is represented by square brackets ([]):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {         # literal
        'could be better': {           # term
            '`I hope your day gets better soon :(`': 'end'
        },
        '{good, not bad}': {           # set
            '`Glad to hear that you are doing well :)`': 'end'
        },
        '<very, good>': {              # unordered list
            '`So glad that you are having a great day!`': 'end'
        },
        '[so, good]': {                # ordered list (sequence)
            '`Things are just getting better for you!`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
    }
}

#13: matches the input with both 'so' and 'good' in that order.

S: Hello. How are you?
U: So good!
S: Things are just getting better for you!

S: Hello. How are you?
U: It's so wonderfully good!
S: Things are just getting better for you!

S: Hello. How are you?
U: It's good
S: Sorry, I didn't understand you.

S: Hello. How are you?
U: It's good so far
S: Sorry, I didn't understand you.

Currently, it matches the input "could be better" with the condition in #4, but does not match "it could be better" or "could be better for sure", where there are terms other than the ones indicated in the condition.

Update the condition such that it matches all three inputs.
How about matching inputs such as "could be much better" or "could be really better"?

'[could be better]'
'[could be, better]'

Rigid Sequence

The condition is true if all terms in the input exactly match all terms in the rigid sequence in the same order. A rigid sequence is represented by square brackets ([ ]), where the left bracket is followed by an exclamation mark (!):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {         # literal
        'could be better': {           # term
            '`I hope your day gets better soon :(`': 'end'
        },
        '{good, not bad}': {           # set
            '`Glad to hear that you are doing well :)`': 'end'
        },
        '<very, good>': {              # unordered list
            '`So glad that you are having a great day!`': 'end'
        },
        '[so, good]': {                # ordered list (sequence)
            '`Things are just getting better for you!`': 'end'
        },
        '[!hello, world]': {           # rigid sequence
            '`You\'re a programmer!`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
    }
}

#16: matches the input with both 'hello' and 'world' in that order.

S: Hello. How are you?
U: Hello World
S: You're a programmer!

S: Hello. How are you?
U: hello world to you
S: Sorry, I didn't understand you.

S: Hello. How are you?
U: It's hello world
S: Sorry, I didn't understand you.

There is no difference between matching a term (e.g., 'hello world') and matching a rigid sequence (e.g., '[!hello, world]'). The rigid sequence is designed specifically for negation, which will be deprecated in the next version.

Negation

The condition is true if all terms in the input exactly match all terms in the rigid sequence except for ones that are negated. A negation is represented by a hyphen (-):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {         # literal
        'could be better': {           # term
            '`I hope your day gets better soon :(`': 'end'
        },
        '{good, not bad}': {           # set
            '`Glad to hear that you are doing well :)`': 'end'
        },
        '<very, good>': {              # unordered list
            '`So glad that you are having a great day!`': 'end'
        },
        '[so, good]': {                # ordered list (sequence)
            '`Things are just getting better for you!`': 'end'
        },
        '[!hello, world]': {           # rigid sequence
            '`You\'re a programmer!`': 'end'
        },
        '[!-not, aweful]': {           # negation
            '`Sorry to hear that :(`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#19: matches the input with 'aweful' and zero to many terms prior to it that are not 'not'.

S: Hello. How are you?
U: Aweful!
S: Sorry to hear that :(

S: Hello. How are you?
U: It's so aweful..
S: Sorry to hear that :(

S: Hello. How are you?
U: Not aweful
S: Sorry, I didn't understand you.

S: Hello. How are you?
U: Not so aweful
S: Sorry to hear that :(

S: Hello. How are you?
U: Aweful and terrible
S: Sorry, I didn't understand you.

Nesting

It is possible to nest conditions for more advanced matching. Let us create a term condition that matches both "so good" and "very good" using a nested set:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        '{so, very} good': {
                '`Things are just getting better for you!`': 'end'
            },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: uses a set inside a term.

Does this condition match "good"?

No, because the outer condition uses term matching that requires the whole input to be the same as the condition.

However, it does not match when other terms are included in the input (e.g., "It's so good to be here"). To broaden the matching scope, you can put the condition inside a sequence:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        '[{so, very} good]': {
                '`Things are just getting better for you!`': 'end'
            },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: the term condition is inside the sequence.

What if we want the condition to match the above inputs as well as "fantastic"? You can put the condition under a set and add fantastic as another term:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        '{[{so, very} good], fantastic}': {
                '`Things are just getting better for you!`': 'end'
            },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: the sequence condition and the new term fantastic is inside the set.

S: Hello. How are you?
U: I'm very good, thank you!
S: Things are just getting better for you!

S: Hello. How are you?
U: It's so good to be here :)
S: Things are just getting better for you!

S: Hello. How are you?
U: Fantastic!!!
S: Things are just getting better for you!

S: Hello. How are you?
U: Good
S: Sorry, I didn't understand you.

S: Hello. How are you?
U: It's fantastic
S: Sorry, I didn't understand you.

The above transitions match "Fantastic" but not "It's fantastic". Update the condition such that it can match both inputs.

Put fantastic under a sequence such that '{[{so, very} good], [fantastic]}'.

Variable

Saving user content can be useful in many ways. Let us consider the following transitions:

transitions = {
    'state': 'start',
    '`What is your favorite animal?`': {
        '[{dogs, cats, hamsters}]': {
            '`I like them too!`': 'end'
        },
        'error': {
            '`I\'ve never heard of that animal.`': 'end'
        }
    }
}

S: What is your favorite animal?
U: I like dogs
S: I like them too!

Users may feel more engaged if the system says, "I like dogs too" instead of "them". Natex allows you to create a variable to store the matched term. A variable is represented by a string preceded (without spaces) by a dollar sign $:

transitions = {
    'state': 'start',
    '`What is your favorite animal?`': {
        '[$FAVORITE_ANIMAL={dogs, cats, hamsters}]': {
            '`I like` $FAVORITE_ANIMAL `too!`': 'end'
        },
        'error': {
            '`I\'ve never heard of that animal.`': 'end'
        }
    }
}

#4: creates a variable FAVORITE_ANIMAL storing the matched term from the user content.
#5: uses the value of the variable to generate the follow-up system utterance.

In #5, two literals, `I like` and `too!` surround the variable $FAVORITE_ANIMAL. If a variable were indicated inside a literal, STDM would throw an error.

S: What is your favorite animal?
U: I like dogs!!
S: I like dogs too!

S: What is your favorite animal?
U: Hamsters are my favorite!
S: I like hamsters too!

3.2. Ontology

How to use ontologies for matching in Natex.

Ontology

Let us create a dialogue flow to talk about animals:

transitions = {
    'state': 'start',
    '`What is your favorite animal?`': {
        '[{dog, ape, rat}]': {
            '`I love mammals!`': 'end'
        },
        '[{snake, lizard}]': {
            '`Reptiles are slick, haha`': 'end'
        },
        '[{frog, salamander}]': {
            '`Amphibians can be cute :)`': 'end'
        },
        'error': {
            '`I\'ve never heard of that animal.`': 'end'
        }
    }
}

S: What is your favorite animal?
U: I love frog
S: Amphibians can be cute :)

S: What is your favorite animal?
U: Cat
S: I've never heard of that animal.

S: What is your favorite animal?
U: Dogs
S: I've never heard of that animal.

For each type of animal, however, the list can be indefinitely long (e.g., there are over 5,400 mammal species). In this case, it is better to use an ontology (e.g., WordNet, FrameNet).

Let us create a JSON file, ontology_animal.json, containing an ontology of animals:

{
    "ontology": {
        "animal": ["mammal", "fish", "bird", "reptile", "amphibian"],
        "mammal": ["dog", "ape", "rat"],
        "reptile": ["snake", "lizard"],
        "amphibian": ["frog", "salamander"],
        "dog": ["golden retriever", "poodle"]
    }
}

#2: the key ontology is paired with a dictionary as a value.
#3: the key animal represents the category, and its subcategories are indicated in the list.
#4-6: each subcategory, mammal, reptile, and amphibian, has its own subcategory.
#7: the ontology hierarchy: animal -> mammal -> dog.

Given the ontology, the above transitions can be rewritten as follow:

transitions = {
    'state': 'start',
    '`What is your favorite animal?`': {
        '[#ONT(mammal)]': {
            '`I love mammals!`': 'end'
        },
        '[#ONT(reptile)]': {
            '`Reptiles are slick, haha`': 'end'
        },
        '[#ONT(amphibian)]': {
            '`Amphibians can be cute :)`': 'end'
        },
        'error': {
            '`I\'ve never heard of that animal.`': 'end'
        }
    }
}

#4: matches the key "mammal" as well as its subcategories: "dog", "ape", and "rat".
#5: matches the key "reptile" as well as its subcategories: "snake" and "lizard".
#6: matches the key "amphibian" as well as its subcategories: "frog" and "salamander".

S: What is your favorite animal?
U: I love frogs
S: Amphibians can be cute :)

Unlike set matching, ontology matching handles plurals (e.g., "frogs").

S: What is your favorite animal?
U: I love my golden retriever
S: I love mammals!

Although there is no condition specified for the category dog that includes "golden retriever", there is a condition for its supercategory mammal (#4), to which it backs off.

S: What is your favorite animal?
U: I cannot live without my puppy!
S: I've never heard of that animal.

Currently, ontology matching does not handle plurals for compound nouns (e.g., "golden retrievers"), which will be fixed in the following version.

Expression

It is possible that a category is mentioned in a non-canonical way; the above conditions do not match "puppy" because it is not introduced as a category in the ontology. In this case, we can specify the aliases as "expressions":

{
    "ontology": {
        "animal": ["mammal", "fish", "bird", "reptile", "amphibian"],
        "mammal": ["dog", "ape", "rat"],
        "reptile": ["snake", "lizard"],
        "amphibian": ["frog", "salamander"],
        "dog": ["golden retriever", "poodle"]
    },

    "expressions": {
        "dog": ["canine", "puppy"]
    }
}

#10: the key expressions is paired with a dictionary as a value.
#4: allows matching "canine" and "puppy" for the dog category.

Once you load the updated JSON file, it now understands "puppy" as an expression of "dog":

S: What is your favorite animal?
U: I cannot live without my puppy!
S: I love mammals!

It is possible to match "puppy" by adding the term as a category of "dog" (#7). However, it would not be a good practice as "puppy" should not be considered a subcategory of "dog".

Variable

Values matched by the ontology can also be stored in variables:

transitions = {
    'state': 'start',
    '`What is your favorite animal?`': {
        '[$FAVORITE_ANIMAL=#ONT(mammal)]': {
            '`I love` $FAVORITE_ANIMAL `!`': 'end'
        },
        '[$FAVORITE_ANIMAL=#ONT(reptile)]': {
            '$FAVORITE_ANIMAL `are slick, haha`': 'end'
        },
        '[$FAVORITE_ANIMAL=#ONT(amphibian)]': {
            '$FAVORITE_ANIMAL `can be cute :)`': 'end'
        },
        'error': {
            '`I\'ve never heard of that animal.`': 'end'
        }
    }
}

#4,7,10: the matched term gets stored in the variable FAVORITE_ANIMAL.
#5,8,11: the system uses the value of FAVORITE_ANIMAL to generate the response.

S: What is your favorite animal?
U: I love frogs
S: frogs can be cute :)

S: What is your favorite animal?
U: I can't live without my puppy!
S: I love puppy !

Loading

The custom ontology must be loaded to the knowledge base of the dialogue flow before it runs:

df = DialogueFlow('start', end_state='end')
df.knowledge_base().load_json_file('resources/ontology_animal.json')
df.load_transitions(transitions)

#1: loads the ontology in ontology_animal.json to the knowledge base of df.

Code Snippet

def natex_ontology() -> DialogueFlow:
    transitions = {
        'state': 'start',
        '`What is your favorite animal?`': {
            '[$FAVORITE_ANIMAL=#ONT(mammal)]': {
                '`I love` $FAVORITE_ANIMAL `!`': 'end'
            },
            '[$FAVORITE_ANIMAL=#ONT(reptile)]': {
                '$FAVORITE_ANIMAL `are slick, haha`': 'end'
            },
            '[$FAVORITE_ANIMAL=#ONT(amphibian)]': {
                '$FAVORITE_ANIMAL `can be cute :)`': 'end'
            },
            'error': {
                '`I\'ve never heard of that animal.`': 'end'
            }
        }
    }

    df = DialogueFlow('start', end_state='end')
    df.knowledge_base().load_json_file('resources/ontology_animal.json')
    df.load_transitions(transitions)
    return df
    
if __name__ == '__main__':
    natex_ontology().run()

3.4. Regular Expression

How to use regular expressions for matching in Natex.

Regular expressions provide powerful ways to match strings and beyond:

Chapter 2.1: Regular Expressions, Chapter 2.1, Speech and Language Processing (3rd ed.), Jurafsky and Martin.
Regular Expression HOWTO, Python Documentation
Regular Expresions 101

Syntax

Grouping

Syntax

Description

Repetitions

Syntax

Description

Non-greedy

Special Characters

Functions

Several functions are provided in Python to match regular expressions.

match()

Let us create a regular expression that matches "Mr." and "Ms.":

import re

RE_MR = re.compile(r'M[rs]\.')
m = RE_MR.match('Dr. Wayne')
print(m)

#1: imports the regular expression library.
#3: compiles the regular expression into the regex object RE_MR.
#4: matches the string "Dr. Choi" with RE_MR and saves the match object to m.

A regular expression is represented by r'expression' where the expression is in a string preceded by the special character r.

The above code prints None, indicating that the value of m is None, because the regular expression does not match the string.

m = RE_MR.match('Mr. Wayne')
print(m)
if m:
    print(m.group(), m.start(), m.end())

#1: since RE_MR matches the string, m is a match object.
#3: true since m is a match object.
#4: prints the matched substring, and the start (inclusive) and end (exclusive) indices of the substring with respect to the original string in #1.

<re.Match object; span=(0, 3), match='Mr.'>
Mr. 0 3

Currently, no groups are specified in RE_MR:

print(m.groups())

#1: returns an empty tuple ().

What are the differences between a list and a tuple in Python?

It is possible to group specific patterns using parentheses:

RE_MR = re.compile(r'(M[rs])(\.)')
m = RE_MR.match('Ms. Wayne')
print(m.groups())
print(m.group())
print(m.group(0))
print(m.group(1))
print(m.group(2))

#1: there are two groups in this regular expression, (M[rs]) and (\.).
#3: returns a tuple of matched substrings ('Ms', '.') for the two groups in #1.
#4,5: return the entire match "Ms.".
#6: returns "Ms" matched by the first group (M[rs]).
#7: returns "." matched by the second group (\.).

('Ms', '.')
Ms.
Ms.
Ms
.

The above RE_MR matches "Mr." and "Ms." but not "Mrs." Modify it to match all of them (Hint: use a non-capturing group and |).

RE_MR = re.compile(r'(M(?:[rs]|rs))(\.)')

The non-capturing group (?:[rs]|rs) matches "r", "s", or "rs" such that the first group matches "Mr", "Ms", and "Mrs", respectively.

Since we use the non-capturing group, the following code still prints a tuple of two strings:

print(RE_MR.match('Mrs. Wayne').groups())
--> ('Mrs', '.')

What if we use a capturing group instead?

RE_MR = re.compile(r'(M([rs]|rs))(\.)')

Now, the nested group ([rs]|rs) is considered the second group such that the match returns a tuple of three strings as follows:

print(RE_MR.match('Mrs. Wayne').groups())
--> ('Mr', 'rs', '.')

search()

Let us match the following strings with RE_MR:

s1 = 'Mr. and Ms. Wayne are here'
s2 = 'Here are Mr. and Mrs. Wayne'

print(RE_MR.match(s1))
print(RE_MR.match(s2))

#4: matches "Mr." but not "Ms."
#5: matches neither "Mr." nor "Mrs."

<re.Match object; span=(0, 3), match='Mr.'>
None

For s1, only "Mr." is matched because match() stops matching after finding the first pattern. For s2 on the other hand, even "Mr." is not matched because match() requires the pattern to be at the beginning of the string.

To match a pattern anywhere in the string, we need to search for the pattern instead:

print(RE_MR.search(s1))
print(RE_MR.search(s2))

search() returns a match object as match() does.

<re.Match object; span=(0, 3), match='Mr.'>
<re.Match object; span=(9, 12), match='Mr.'>

findall()

search() still does not return the second substrings, "Ms." and "Mrs.". The following shows how to find all substrings that match the pattern:

print(RE_MR.findall(s1))
print(RE_MR.findall(s2))

findall() returns a list of tuples where each tuple represents a group of matched results.

[('Mr', '.'), ('Ms', '.')]
[('Mr', '.'), ('Mrs', '.')]

finditer()

Since findall() returns a list of tuples instead of match objects, there is no definite way of locating the matched results in the original string. To return match objects instead, we need to interactively find the pattern:

for m in RE_MR.finditer(s1):
    print(m)

#1: finditer() returns an iterator that keeps matching the pattern until it no longer finds.

<re.Match object; span=(0, 3), match='Mr.'>
<re.Match object; span=(8, 11), match='Ms.'>

for m in RE_MR.finditer(s2):
    print(m)

<re.Match object; span=(9, 12), match='Mr.'>
<re.Match object; span=(17, 21), match='Mrs.'>

You can use a list comprehension to store the match objects as a list:

ms = [m for m in RE_MR.finditer(s1)]
print(ms)

#1: returns a list of all m (in order) matched by finditer().

[<re.Match object; span=(0, 3), match='Mr.'>, <re.Match object; span=(8, 11), match='Ms.'>]

How is the code above different from the one below?

ms = []
for m in RE_MR.finditer(s1):
    ms.append(m)

What are the advantages of using a list comprehension over a for-loop other than it makes the code shorter?

Write regular expressions to match the following cases:

Abbreviation: Dr., U.S.A.
Apostrophe: '80, '90s, 'cause
Concatenation: don't, gonna, cannot
Hyperlink: https://github.com/emory-courses/cs329/
Number: 1/2, 123-456-7890, 1,000,000
Unit: $10, #20, 5kg

RE_TOK = re.compile(r'([",.]|n\'t|\s+)')
RE_ABBR = re.compile(r'((?:Mr|Mrs|Ms|Dr)\.)|((?:[A-Z]\.){2,})')
RE_APOS = re.compile(r'\'(\d\ds?|cause)')
RE_CONC = re.compile(r'([A-Za-z]+)(n\'t)|(gon)(na)|(can)(not)')
RE_HYPE = re.compile(r'(https?://\S+)')
RE_NUMB = re.compile(r'(\d+/\d+)|(\d{3}-\d{3}-\d{4})|(\d(?:,\d{3})+)')
RE_UNIT = re.compile(r'([$#])?(\d+)([km]g)?')

Natex Integration

The nesting example in Section 3.1 has a condition as follows (#4):

'{[{so, very} good], fantastic}'

Write a regular expression that matches the above condition.

r'((?:so|very) good|fantastic)'

It is possible to use regular expressions for matching in Natex. A regular expression is represented by forward slashes (/../):

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        '/((?:so|very) good|fantastic)/': {
            '`Things are just getting better for you!`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: true if the entire input matches the regular expression.

S: Hello. How are you?
U: So good!!!
S: Things are just getting better for you!

S: Hello. How are you?
U: Fantastic :)
S: Things are just getting better for you!

S: Hello. How are you?
U: It's fantastic
S: Sorry, I didn't understand you.

You can put the expression in a sequence to allow it a partial match:

transitions = {
    'state': 'start',
    '`Hello. How are you?`': {
        '[/((?:so|very) good|fantastic)/]': {
            '`Things are just getting better for you!`': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: the regular expression is put in a sequence [].

S: Hello. How are you?
U: It's fantastic!!
S: Things are just getting better for you!

S: Hello. How are you?
U: I'm so good, thank you!
S: Things are just getting better for you!

When used in Natex, all literals in the regular expression (e.g., "so", "good" in #4) must be lowercase because Natex matches everything in lowercase. The design choice is made because users tend not to follow typical capitalization in a chat interface, whether it is text- or audio-based.

Variable

It is possible to store the matched results of a regular expression to variables. A variable in a regular expression is represented by angle brackets (<..>) inside a capturing group ((?..)).

The following transitions take the user name and respond with the stored first and last name:

transitions = {
    'state': 'start',
    '`Hello. What should I call you?`': {
        '[/(?<FIRSTNAME>[a-z]+) (?<LASTNAME>[a-z]+)/]': {
            '`It\'s nice to meet you,` $FIRSTNAME `. I know several people with the last name,` $LASTNAME': 'end'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        }
    }
}

#4: matches the first name and the last name in order and stores them in the variables FIRSTNAME and LASTNAME.
#5: uses FIRSTNAME and LASTNAME in the response.

S: Hello. What should I call you?
U: Jinho Choi
S: It's nice to meet you, jinho . I know several other choi .

3.5. Macro

How to use macro functions for matching in Natex.

The most powerful aspect of Natex is its ability to integrate pattern matching with arbitrary code. This allows you to integrate regular expressions, NLP models, or custom algorithms into Natex.

Creation

A macro can be defined by creating a class inheriting the Macro in STDM and the run method:

#1: imports Macro from STDM.
#2: imports type hints from the package in Python.
#4: creates the MacroGetName class inheriting Macro.
#5: overrides the run method declared in Macro.

Currently, the run method returns True no matter what the input is.

Integration

Let us create transitions using this macro. A macro is represented by an alias preceded by the pound sign (#):

#4: calls the macro #GET_NAME that is an alias of MacroGetName.
#13: creates a dictionary defining aliases for macros.
#14: creates an object of MacroGetName and saves it to the alias GET_NAME.

To call the macro, we need to add the alias dictionary macros to the dialogue flow:

#3: adds all macros defined in macros to the dialogue flow df.

Parameters

The run method has three parameters:

vars: is the variable dictionary, maintained by a DialogueFlow object, where the keys and values are variable names and objects corresponding to their values.
args: is a list of strings representing arguments specified in the macro call.

Let us modify the run method to see what ngrams and vars give:

#2: prints the original string of the matched input span before preprocessing.
#3: prints the input span, preprocessed by STDM and matched by the Natex.
#4: prints a set of n-grams.

When you interact with the the dialogue flow by running it (df.run()), it prints the followings:

The raw_text method returns the original input:

The text method returns the preprocessed input used to match the Natex:

The ngrams gives a set of all possible n-grams in text():

Finally, the vars gives a dictionary consisting of both system-level and user-custom variables (no user-custom variables are saved at the moment):

Implementation

Let us update the run method that matches the title, first name, and last name in the input and saves them to the variables $TITLE, $FIRSTNAME, and $LASTNAME, respectively:

#2: creates a regular expression to match the title, first name and last name.
#3: searches for the span to match.
#4: returns False if no match is found.
#6-18 -> exercise.
#20-22: saves the recognized title, first name, and last name to the corresponding variables.
#24: returns True as the regular expression matches the input span.

Given the updated macro, the above transitions can be modified as follow:

#5: uses the variables $FIRSTNAME and $LASTNAME retrieved by the macro to generate the output.

The followings show outputs:

Can macros be mixed with other Natex expressions?

3.5. Quiz

Quiz 3: Contextual Understanding

Overview

Your goal is to create a chatbot that talks about movies. Here is a sample dialogue:

Your chatbot aims to collect user information by asking the following:

The latest movie that the user watched (#3-4).
A contextualized question regarding the latest movie (#5-6).
A question regarding the genre of the latest movie (#7-10).

Your chatbot should give an appropriate response to every user response. For this assignment, you must use all of the following:

Task 1

Update them to design a dialogue flow for the chatbot.

Task 2

Create a PDF file quiz3.pdf that describes the following:

Sample dialogues that your chatbot can conduct.
Explanations of how the ontology, macro(s), and regular expression(s) are used for contextual understanding in your chatbot.

Submission

Commit and push quiz3.py to your GitHub repository.
Submit quiz3.pdf to Canvas.

4. Interaction Design

Content

Resource

Source code:

4.1. State Referencing

Start State

Let us create transitions for a virtual agent that handles time, weather, and playing music:

#5: shows the current time (although it is correct only twice a day at the moment).
#8: informs the current weather in Sunnyville.
#14: print the error message and references to the start state.

Notice that when the user input does not match any of the conditions, it prints the error message and loops back to the start state.

Custom States

It is possible to name any transition you create as a state:

#6: names the state as time.
#9: names the state as weather.
#13: names the state as play_raining_tacos.
#17: references to the play_raining_tacos state.

Exit State

State referencing can be abused to create transitions that never end (infinite loop). It is important to design an exit state so you can terminate the conversation without throwing errors:

#6,10,14,17: loops back to the start state.
#19-21: creates an exit state to terminate the dialogue.

What is the main difference between an error transition and an exit state?

An error transition defines a default action for uninterpretable user input, whereas an exit state terminates the current session of the dialogue.

4.2. Advanced Interaction

Built-in Macros

Besides #ONT for checking an ontology, STDM provides several macros useful for dialogue design.

#LEM

The following example shows a use case of the macro #LEM that uses the NLTK Lemmatizer to match lemmas of "raining tacos":

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '`It\'s raining tacos. From out of the sky ...`': 'start'
        },
        'error': {
            '`Sorry, I didn\'t understand you.`': 'end'
        },
    }
}

#5: maches the input with the lemma of "rain" or "rainy", then the lemma of "taco".

S: What can I do for you?
U: Play raining tacos
S: It's raining tacos. From out of the sky ... What can I do for you?
U: Play raining taco
S: It's raining tacos. From out of the sky ... What can I do for you?
U: Play rain taco
S: It's raining tacos. From out of the sky ... What can I do for you?
U: Play rainy taco
S: It's raining tacos. From out of the sky ... What can I do for you?
U: Bye
S: Sorry, I didn't understand you.

#UNX

The #UNX macro can be used as an alternative to the 'error' transition, which prepends a short acknowledgment phrase (e.g., "yeah", "sure") to the error statement:

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '`It\'s raining tacos. From out of the sky ...`': 'start'
        },
        '#UNX': {
            '`Thanks for sharing.`': 'end'
        },
    }
}

#8: prepends an acknowledgment phrase to the error statement, "Thanks for sharing" in #8.

S: What can I do for you?
U: My name is Jinho
S: Right. Thanks for sharing. What can I do for you?
U: I am a professor
S: Yeah. Thanks for sharing. What can I do for you?
U: Hello world
S:  Thanks for sharing. What can I do for you?
U: Play raining tacos
S: It's raining tacos. From out of the sky ...

#3,5: add acknowledgment phrases given the user inputs.
#7: does not add an acknowledgment phrase when the user input is short.

When the user input contains fewer than three tokens, #UNK does not add any acknowledgment.

Seldomly, #UNKgets selected even when conditions in other transitions match the input (due to an STDM bug). If you notice this during testing, assign a very slow score to the #UNK state to ensure it gets the lowest priority among other branches.

#SET & #IF

Let us update the above transitions so that it sometimes refuses to sing the same song:

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '#IF($RAINING_TACOS=True) `Don\'t make me sing this again!`': 'start',
            '`It\'s raining tacos. From out of the sky ...` #SET($RAINING_TACOS=True)': 'start',
        },
        '#UNX': {
            '`Thanks for sharing.`': 'start'
        },
    }
}

#6: can be picked if the variable $RAINING_TACOS equals to the string 'True'.
#7: sets $RAINING_TACOS to 'True' after prompting the system output.

S: What can I do for you?
U: Play raining tacos
S: It's raining tacos. From out of the sky ...   What can I do for you?
U: Play raining tacos
S:  Don't make me sing this again! What can I do for you?
...

Once $RAINING_TACOS is set to 'True', STDM randomly picks a statement in #6 or #7 as the system output.

Notice that the #SET macro only assigns a string value to the variable. It is possible to write a macro to set any variable to an actual boolean value (e.g., True, False):

class MacroSetBool(Macro):
    def run(self, ngrams: Ngrams, vars: Dict[str, Any], args: List[str]):
        if len(args) != 2:
            return False

        variable = args[0]
        if variable[0] == '$':
            variable = variable[1:]

        boolean = args[1].lower()
        if boolean not in {'true', 'false'}:
            return False

        vars[variable] = bool(boolean)
        return True

#3-4: checks if there are two arguments.
#6-8: retrieves the variable name.
#10-12: checks if the argument is a proper boolean value.
#14: stores the boolean value to the variable.

Given MacroSetBool, the above transitions can be updated as follow:

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '#IF($RAINING_TACOS) `Don\'t make me sing this again!`': 'start',
            '`It\'s raining tacos. From out of the sky ...` #SETBOOL(RAINING_TACOS, True)': 'start',
        },
        '#UNX': {
            '`Thanks for sharing.`': 'start'
        },
    }
}

macros = {
    'SETBOOL': MacroSetBool()
}

#6: is selected if $RAINING_TACOS is True.
#0: sets the variable $RAINING_TACOS to the boolean value True.

Currently, STDM randomly chooses the statements in #6 and #7 once $RAINING_TACOS becomes True. We can write another macro that prompts #7 only for the first time:

class MacroPlayRainingTacos(Macro):
    def run(self, ngrams: Ngrams, vars: Dict[str, Any], args: List[str]):
        return not vars.get('RAINING_TACOS', False)

#3: the get() method in dict returns the value corresponding to the key, RAINING_TACOS if exists; otherwise, False.

Given MacroPlayRainingTacos, the transitions can be updated as follow:

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '#IF($RAINING_TACOS) `Don\'t make me sing this again!`': 'start',
            '#IF(#PLAY_RAINING_TACOS) `It\'s raining tacos. From out of the sky ...` #SETBOOL(RAINING_TACOS, True)': 'start',
        },
        '#UNX': {
            '`Thanks for sharing.`': 'start'
        },
    }
}

macros = {
    'SETBOOL': MacroSetBool(),
    'PLAY_RAINING_TACOS': MacroPlayRainingTacos()
}

df = DialogueFlow('start', end_state='end')
df.load_transitions(transitions)
df.add_macros(macros)
df.run()

#7: is selected if the macro #PLAY_RAINING_TACOS returns True.
#17: adds #PLAY_RAINING_TACOS to the macro dictionary.

S: What can I do for you?
U: Play raining tacos
S:  It's raining tacos. From out of the sky ...    What can I do for you?
U: Play raining tacos
S:  Don't make me sing this again! What can I do for you?
U: Play raining tacos
S:  Don't make me sing this again! What can I do for you?
...

Music

It is possible to make our system actually sings instead of prompting the lyric. Let us first install the VLC package:

pip install python-vlc

Then, import the package and update the MacroPlayRainingTacos macro to play the MP3 file, resources/raining_tacos.mp3:

import vlc

class MacroPlayRainingTacos(Macro):
    def run(self, ngrams: Ngrams, vars: Dict[str, Any], args: List[str]):
        if not vars.get('RAINING_TACOS', False):
            vlc.MediaPlayer("resources/raining_tacos.mp3").play()
            return True
        return False

#1: imports the VLC package.
#6: creates a VLC media player and plays the specified MP3 file.

When you run the above dialogue flow, it now plays the MP3 file and prompts the lyric.

Time

The transitions in Section 4.1 prompt the same time (3PM) upon every request. Let us create a new macro that checks the current (system) time:

import time

class MacroTime(Macro):
    def run(self, ngrams: Ngrams, vars: Dict[str, Any], args: List[str]):
        current_time = time.strftime("%H:%M")
        return "It's currently {}.".format(current_time)

#1: imports the time package.
#5: retrieves the current time in the specified format using the strftime method.
#6: returns the current time using the str.format method.

The macro MacroTime can be called to generate the system output:

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '#IF($RAINING_TACOS) `Don\'t make me sing this again!`': 'start',
            '#IF(#PLAY_RAINING_TACOS) `It\'s raining tacos. From out of the sky ...` #SETBOOL(RAINING_TACOS, True)': 'start',
        },
        '[{time, clock}]': {
            'state': 'time',
            '#TIME': 'end'
        },
        '#UNX': {
            '`Thanks for sharing.`': 'start'
        },
    }
}

macros = {
    'SETBOOL': MacroSetBool(),
    'PLAY_RAINING_TACOS': MacroPlayRainingTacos(),
    'TIME': MacroTime()
}

#11: calls the TIME macro to generate the system output displaying the current time.
#22: adds #TIME to the macro dictionary.

S: What can I do for you?
U: What time is it now?
S: It's currently 05:27.

Weather

The transitions in Section 4.1 prompt the same weather (sunny) upon every request. Let us retrieve the latitude and the longitude of the system using Google Maps:

Then, use a web API provided by the National Weather Service to retrieve the grid correlates to the coordinate: https://api.weather.gov/points/33.7904,-84.3266

{
    ...,
    "properties": {
        "@id": "https://api.weather.gov/points/33.7904,-84.3266",
        ...,
        "forecast": "https://api.weather.gov/gridpoints/FFC/52,88/forecast",
        ...
    }
}

Look for the forecast field under properties: https://api.weather.gov/gridpoints/FFC/52,88/forecast

Write a macro that retrieves the current weather for the grid using another web API:

import json
import requests

class MacroWeather(Macro):
    def run(self, ngrams: Ngrams, vars: Dict[str, Any], args: List[Any]):
        url = 'https://api.weather.gov/gridpoints/FFC/52,88/forecast'
        r = requests.get(url)
        d = json.loads(r.text)
        periods = d['properties']['periods']
        today = periods[0]
        return today['detailedForecast']

#1: imports the json package.
#2: imports the requests package.
#6: specifies the forecast URL.
#7: retrieves the content from the URL in JSON.
#8: saves the JSON content to a dictionary.
#9: retrieves forecasts for all supported periods.
#10: retrieves the forecast for today.
#11: returns today's forecast.

Finally, update the transitions with the weather macro:

transitions = {
    'state': 'start',
    '`What can I do for you?`': {
        '[play, [!{#LEM(rain), rainy}, #LEM(taco)]]': {
            'state': 'play_raining_tacos',
            '#IF($RAINING_TACOS) `Don\'t make me sing this again!`': 'start',
            '#IF(#PLAY_RAINING_TACOS) `It\'s raining tacos. From out of the sky ...` #SETBOOL(RAINING_TACOS, True)': 'start',
        },
        '[{time, clock}]': {
            'state': 'time',
            '#TIME': 'end'
        },
        '[{weather, forecast}]': {
            'state': 'weather',
            '#WEATHER': 'end'
        },
        '#UNX': {
            '`Thanks for sharing.`': 'start'
        },
    }
}

macros = {
    'SETBOOL': MacroSetBool(),
    'PLAY_RAINING_TACOS': MacroPlayRainingTacos(),
    'TIME': MacroTime(),
    'WEATHER': MacroWeather()
}

#15: calls the WEATHER macro to generate the system output displaying today's weather.
#22: adds #WEATHER to the macro dictionary.

S: What can I do for you?
U: How is the weather today?
S: A chance of rain showers. Cloudy, with a high near 68. South wind 0 to 5 mph. Chance of precipitation is 30%.

The time and the weather retrieved by the above macros are oriented to the system, not the user. It is possible to anticipate the users' location if one's IP address is provided; however, this is not possible unless the user uses a specific device (e.g., Amazon Echo, smartphone) and agrees to send one's private information to our system.

4.3. Compound States

Multiple Topics

A dialogue flow can be complex when you start adding multiple topics. In this case, you can create a separate transition dictionary for each topic.

Let us create transitions talking about music and movies:

#3: directs to the music state.
#4: directs to the movie state.

The music and movie states can be defined in separate transition dictionaries:

#5: directs to the start state.
#8: directs to the movie state.

#5: directs to the start state.
#8: directs to the music state.

Finally, all three transition dictionaries can be loaded to the same dialogue flow:

When the dialogue flow runs, it randomly selects one of the 3 states, music, movie, and end:

#1: randomly selects the music state.
#3: switches to the movie state when it does not understand the user input.
#5: switches to the music state when it does not understand the user input.
#7: goes back to the start state when it understands the user input, and randomly selects the movie state.
#9: goes back to the start state when it understands the user input, and randomly selects the music state.
#11: goes back to the start state when it understands the user input, and randomly selects the end state.

#1: randomly selects the end state and terminates the dialogue.

Gating

The randomness in the above transitions can be quite annoying because it may end the dialogue immediately after it runs or repeats the same topic over and over. This can be alleviated by using the #GATE built-in macro:

#3: puts the music topic to an open gate.
#4: puts the movie topic to an open gate.
#4: has no gate to open (so it can be selected at any time).

Once an output is selected, #GATE closes the door for that output such that it will never be selected again.

It is important to have at least one output with no gate; otherwise, the system will crash unless one of the outputs leads to the end state.

Scoring

The gating prevents the system from repeating the same topic, but the end state can still be selected at any time without consuming all the other topics. To ensure that the end state gets selected last, we use scoring:

#6: indicates that this is the end state.
#7: assigns the score of this state to 0.1.

By default, all states receive a score of 1; thus, assigning a score below 1 would make that state not selected at all unless there are dynamic scoring changes through macros such as #GATE.

4.4. Global Transition

Exercise

Draw a diagram describing the following dialogue flow.
What is the role of MacroWhatElse?

Global Transitions

It is often the case that the user says something out of the topic that the system expects. One way of handling such occasions is by using global transitions that anticipate common cases:

#1-8: creates global transitions.
#10: adds the global transitions to the dialogue flow.

Notice that after prompting an output in the global transitions, it will direct to the good state defined in the regular transitions.

The global transitions are fired whenever the user content matches their conditions, which can cause multiple matching conditions as follow:

#5: matches the condition in the movie state.

#5: matches the condition in the global transition.

Thus, it is recommended to put lower scores on the conditions in global transitions such that local transitions are prioritized over them:

4.6. Quiz

Overview

You aim to create a chatbot that makes a movie or a song recommendation. Your system should conduct dialogue including the following aspects:

Greet with respect to the current time and/or weather.
Ask about the user's name (with a different way of asking each time the user logins).
If it is the first time the system sees the name, then prompt the first-time greeting. Otherwise, greet by asking about its previous recommendation.
Recommend a movie or a song, depending on the user's request. Do not make the same recommendation more than once for each genre.
If the user requests a different recommendation, make another suggestion.
Provide more information about the recommendation upon request.