> For the complete documentation index, see [llms.txt](https://emory.gitbook.io/conversational-ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://emory.gitbook.io/conversational-ai/5.-lm-based-matching/5.4.-quiz.md).

# 5.4. Quiz

Revisit your [Quiz 2](/conversational-ai/2.-dialogue-graph/2.5.-quiz.md) and improve its language understanding capability using the large language model such as GPT.

* Use [ChatGPT](https://chat.openai.com) to figure out the right prompts.
* Use your trial credits from OpenAI to test the APIs.

## Task 1

* Create a python file [**`quiz5.py`**](https://github.com/emory-courses/conversational-ai/blob/main/src/quiz/quiz5.py) under the [`quiz`](https://github.com/emory-courses/conversational-ai/tree/main/src/quiz) package and copy the code.
* Update the code to design a dialogue flow for the assigned dialogue system.
* Create a PDF file **`quiz5.pdf`** that describes the approach (e.g., prompt engineering) and how the large language model improved over the limitations you described in Quiz 2.

## Task 2

Answer the following questions in **`quiz5.py`**:

1. What are the limitations of the *Bag-of-Words* representation?
2. Describe the *Chain Rule* and *Markov Assumption* and how they are used to estimate the probability of a word sequence.
3. Explain how the *Word2Vec* approach uses feed-forward neural networks to generate word embeddings. What are the advantages of the *Word2Vec* representation over the *Bag-of-Words* representation?
4. Explain what patterns are learned in the multi-head attentions of a *Transformer*. What are the advantages of the *Transformer* embeddings over the *Word2Vec* embeddings?