5.4. Quiz

Revisit your Quiz 2 and improve its language understanding capability using the large language model such as GPT.

• Use ChatGPT to figure out the right prompts.

• Use your trial credits from OpenAI to test the APIs.

Task 1

• Create a python file quiz5.py under the quiz package and copy the code.

• Update the code to design a dialogue flow for the assigned dialogue system.

• Create a PDF file quiz5.pdf that describes the approach (e.g., prompt engineering) and how the large language model improved over the limitations you described in Quiz 2.

Task 2

Answer the following questions in quiz5.py:

1. What are the limitations of the Bag-of-Words representation?

2. Describe the Chain Rule and Markov Assumption and how they are used to estimate the probability of a word sequence.

3. Explain how the Word2Vec approach uses feed-forward neural networks to generate word embeddings. What are the advantages of the Word2Vec representation over the Bag-of-Words representation?

4. Explain what patterns are learned in the multi-head attentions of a Transformer. What are the advantages of the Transformer embeddings over the Word2Vec embeddings?