arrow-left

All pages
gitbookPowered by GitBook
1 of 4

Loading...

Loading...

Loading...

Loading...

1.1. Overview

Explain components, properties, scopes, techniques, and assessments of dialogue systems.

hashtag
Components

hashtag
Genre

  • Conversation: interactive communication between two or more people.

  • Dialogue: a conversation, often between two people, with a specific goal in mind.

Dialog: a window that appears on a screen in computing contexts (e.g., dialog box).

hashtag
Application

  • Dialogue System: a computer system that interacts with humans in natural language.

  • Conversational Agent: a dialogue system that interprets and responds to user statements.

  • Virtual Assistant: a dialogue system that performs tasks or services for user requests.

circle-info

Chatbots are typically understood to follow pre-defined dialogue flows for open-domain conversations without using sophisticated artificial intelligence technology.

hashtag
Intelligence

  • Dialogue Management: a process of controlling the state and flow of the dialogue to conduct contextual communications.

  • Conversational AI: a type of Artificial Intelligence (AI) for a dialogue system to understand user inputs and respond properly to them, often processed by machine learning models.

  1. What are examples of dialogue systems currently used in practical applications?

  2. Are there applications that would greatly benefit from adopting dialogue systems?

hashtag
Properties

hashtag
Unit

  • Turn: a single contribution from one speaker to the dialogue.

  • Utterance: a natural unit of speech bounded by breaths or pauses.

circle-info

For a text-based conversation, each turn is often considered an utterance.

hashtag
Context

  • Speech Act: the action, either explicitly or implicitly, expressed by an utterance (e.g., answering, advising, greeting; see ).

  • Intent: the user's goal expressed by an utterance within the context of a conversation (e.g., making an appointment, requesting information).

  • Topic: the matter dealt with in an utterance (e.g., movie, family, midterm).

circle-info

It is possible that one utterance expresses multiple speech acts and intents and also deals with various topics.

Classify each of the following utterances from Friends S1E1 using the dialogue acts:

Ross: Hi.

Joey: This guy says hello, I wanna kill myself.

Monica: Are you okay, sweetie?

Ross: I just feel like someone reached down my throat, grabbed my small intestine, pulled it out of my mouth and tied it around my neck...

Chandler

hashtag
Scopes

hashtag
Task-oriented

Task-oriented dialogue systems have specific tasks to be accomplished:

  • , Henderson et al., SIGDIAL, 2014 ().

  • , Wen et al., EMNLP 2016 ().

  • , Bordes et al., ICLR, 2017 ().

hashtag
Open-domain

Open-domain dialogue systems aim to talk about any topics without specific end goals:

  • ()

  • ()

  • (; requires login)

  1. What kind of tasks are presented in the above task-oriented datasets?

  2. Try the demos of BlenderBot and ChatGPT. What are their limitations?

  3. What are the challenges in building task-oriented vs. open-domain dialogue systems?

hashtag
Techniques

hashtag
State Machine

A dialogue flow can be designed into a fine-state machine. Most commercial dialogue systems take this approach because it gives greater control over how the systems behave. Several platforms are available to facilitate the development of state machine-based dialogue systems:

hashtag
End-to-End

Recent researches focus on developing end-to-end dialogue systems using sequence-to-sequence (S2S) models, which is a type of encoder-decoder model:

  • , Sutskever et al., NeurIPS, 2014.

The current state-of-the-art S2S models use transformers such as BERT as their encoders:

  • , Vaswani et al., NeurIPS, 2017.

  • , Devlin et al., NAACL, 2019.

Three of the above, Meta BlenderBot, OpenAI ChatGPT, and Google LaMDA, are end-to-end systems based on S2S models.

circle-info

Implementing an end-to-end system is beyond the scope of this course. Thus, we will use the state machine approach to develop dialogue systems, starting from .

hashtag
Assessments

The primary objective of both task-oriented and open-domain dialogue systems is to satisfy users by communicating with them. For task-oriented, users are generally satisfied if the tasks are accomplished efficiently. For open-domain, however, user satisfaction is often highly subjective, so proper conversational analysis may need to be involved.

  • , Finch and Choi, SIGDIAL, 2020.

  • , Mehri et al., arXiv, 2022.

  • , Finch et al., arXiv, 2022.

Chatbot: a dialogue system that simulates and processes human conversation.

: Cookie?

Monica: Carol moved her stuff out today.

Joey: Ohh.

Monica: Let me get you some coffee.

Ross: Thanks.

Key-Value Retrieval Networks for Task-Oriented Dialoguearrow-up-right, Eric et al., SIGDIAL, 2017 (datasetarrow-up-right).

  • MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingarrow-up-right, Budzianowski et al., EMNLP, 2018 (datasetarrow-up-right).

  • Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retrieverarrow-up-right, Qin et al., EMNLP, 2019 (datasetarrow-up-right).

  • Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetarrow-up-right, Rastogi et al., AAAI, 2020 (datasetarrow-up-right).

  • Google LaMDAarrow-up-right (articlearrow-up-right; interviewarrow-up-right)

    Microsoft Azure Bot Servicearrow-up-right
  • Emora STDM

  • Switchboard Dialog Act Corpusarrow-up-right
    http://compprag.christopherpotts.net/swda.htmlarrow-up-right
    The Second Dialog State Tracking Challengearrow-up-right
    datasetarrow-up-right
    Conditional Generation and Snapshot Learning in Neural Dialogue Systemsarrow-up-right
    datasetarrow-up-right
    Learning End-to-End Goal-Oriented Dialogarrow-up-right
    datasetarrow-up-right
    Alexa Prize Socialbot Grand Challengearrow-up-right
    Emora demoarrow-up-right
    Meta BlenderBotarrow-up-right
    demoarrow-up-right
    OpenAI ChatGPTarrow-up-right
    demoarrow-up-right
    Amazon Lexarrow-up-right
    Google Dialogflowarrow-up-right
    IBM Watson Assistantarrow-up-right
    Sequence to Sequence Learning with Neural Networksarrow-up-right
    Attention is All you Needarrow-up-right
    BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarrow-up-right
    open-domain dialogue systems
    Chapter 2
    Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocolsarrow-up-right
    Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challengesarrow-up-right
    Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systemsarrow-up-right

    1.3. Quiz

    Quiz 1: Exploration

    hashtag
    Task 1

    Create a PDF file quiz1.pdf that includes your ideas about the following questions:

    1. What kind of a dialogue system do you want to develop for your team project?

    2. Who will benefit from your dialogue system in what way?

    3. What is the novelty of your dialogue system?

    4. What are the expected challenges in developing your dialogue system?

    5. How do you plan to evaluate your dialogue system?

    Provide your ideas in detail, about 100 words (or more) per question.

    hashtag
    Task 2

    Group a team of 3-4 members for your project.

    hashtag
    Submission

    1. Submit quiz1.pdf to Canvas.

    2. Go to [People → Team Projects] in Canvas and assign your team members to the same group.

    1.2. Project Ideas

    Discuss project ideas in groups.

    hashtag
    Before Discussion

    Prepare your ideas about the questions in Quiz 1.

    hashtag
    In-class Discussion

    You will meet three groups during this class. Each team discussion will last 20 minutes, including the time to find group members:

    1. Find 3-4 people you have not interacted with from your previous group(s).

    2. Discuss your project idea with your group and check if anyone has a similar interest.

    1. Exploration

    This chapter overviews dialogue systems, the technologies behind those systems, and their applications.

    hashtag
    Content

    1. Overview

    hashtag
    Resource

    • , Speech and Language Processing (3rd ed.), Jurafsky and Martin.

    Discussion
    Quiz
    Chapter 24: Chatbots and Dialogue Systemsarrow-up-right
    Individual Ideas, Spring 2023.