Data Creation

When you create a dataset, the followings need to be clearly described:

• Data collection (e.g., sources of the data).

• Preprocessing if performed (e.g., scripts that you write, existing tools used).

• Annotation scheme and guidelines if conducted with justification.

• People involved in this process (e.g., annotators, survey subjects).

• Quality of the created data (e.g., inter-annotator agreement).

• Statistics and analysis of the original, preprocessed, annotated data.

Here are a few papers presenting new datasets:

• Competence-Level Prediction and Resume & Job Description Matching Using Context-Aware Transformer Models, Li et al., EMNLP 2020 (see Section 3).

• FriendsQA: Open-Domain Question Answering on TV Show Transcripts, Yang and Choi, SIGDIAL, 2019 (see Section 3).