Question Answering Data Formats
For question answering tasks, the input data can be in JSON files or in a Python list of dictionaries in the correct format. The structure of both formats is identical, i.e. the input may be a string pointing to a JSON file containing a list of dictionaries, or it the input may be a list of dictionaries itself.
Input Structure
The input data should be a single list of dictionaries (or path to a JSON file containing the same). A dictionary represents a single context and its associated questions.
Each such dictionary contains two attributes, the "context" and "qas".
context: The paragraph or text from which the question is asked.qas: A list of questions and answers (format below).
Questions and answers are represented as dictionaries. Each dictionary in qas has the following format.
id: (string) A unique ID for the question. Should be unique across the entire dataset.question: (string) A question.is_impossible: (bool) Indicates whether the question can be answered correctly from the context.answers: (list) The list of correct answers to the question.
A single answer is represented by a dictionary with the following attributes.
text: (string) The answer to the question. Must be a substring of the context.answer_start: (int) Starting index of the answer in the context.
Train Data Format
Used with train_model()
Train data can be in the form of a path to a JSON file or a list of dictionaries in the structure specified.
Note: There cannot be multiple correct answers to a single question during training. Each question must have a single answer (or an empty string as the answer with is_impossible=True).
List of dictionaries
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
train_data = [
{
"context": "Mistborn is a series of epic fantasy novels written by American author Brandon Sanderson.",
"qas": [
{
"id": "00001",
"is_impossible": False,
"question": "Who is the author of the Mistborn series?",
"answers": [
{
"text": "Brandon Sanderson",
"answer_start": 71,
}
],
}
],
},
{
"context": "The first series, published between 2006 and 2008, consists of The Final Empire,"
"The Well of Ascension, and The Hero of Ages.",
"qas": [
{
"id": "00002",
"is_impossible": False,
"question": "When was the series published?",
"answers": [
{
"text": "between 2006 and 2008",
"answer_start": 28,
}
],
},
{
"id": "00003",
"is_impossible": False,
"question": "What are the three books in the series?",
"answers": [
{
"text": "The Final Empire, The Well of Ascension, and The Hero of Ages",
"answer_start": 63,
}
],
},
{
"id": "00004",
"is_impossible": True,
"question": "Who is the main character in the series?",
"answers": [],
},
],
},
]
JSON file
1
train_data = "data/train.json"
Evaluation Data Format
Used with eval_model()
Evaluation data can be in the form of a path to a JSON file or a list of dictionaries in the structure specified.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
eval_data = [
{
"context": "The series primarily takes place in a region called the Final Empire "
"on a world called Scadrial, where the sun and sky are red, vegetation is brown, "
"and the ground is constantly being covered under black volcanic ashfalls.",
"qas": [
{
"id": "00001",
"is_impossible": False,
"question": "Where does the series take place?",
"answers": [
{
"text": "region called the Final Empire",
"answer_start": 38,
},
{
"text": "world called Scadrial",
"answer_start": 74,
},
],
}
],
},
{
"context": "\"Mistings\" have only one of the many Allomantic powers, while \"Mistborns\" have all the powers.",
"qas": [
{
"id": "00002",
"is_impossible": False,
"question": "How many powers does a Misting possess?",
"answers": [
{
"text": "one",
"answer_start": 21,
}
],
},
{
"id": "00003",
"is_impossible": True,
"question": "What are Allomantic powers?",
"answers": [],
},
],
},
]
Prediction Data Format
Used with predict()
The predict() method of a Simple Transformers model is typically used to get a prediction from the model when the true label/answer is not known. Reflecting this, the predict() method of the QuestionAnsweringModel class expects a list of dictionaries which contains only contexts, questions, and an unique ID for each question.
The prediction data should be in the following format.
1
2
3
4
5
6
7
8
9
10
11
12
to_predict = [
{
"context": "Vin is a Mistborn of great power and skill.",
"qas": [
{
"question": "What is Vin's speciality?",
"id": "0",
}
],
}
]
Lazy Loading Data Format
The training data (train_data) must be input as a path (str) to a JSONL file to use Lazy Loading.
The structure of the JSON object is identical to the normal Question Answering train data format.
Note: Currently, lazy loading is only supported for training. The full eval_data will be loaded to memory.