I am trying to generate a quiz from a text that look like this:
Text: "Mary has little lamb and John has a cow. The lamb is one month old. It eats grass and milk, which Mary brings from the the farm."
Keywords: "lamb", "cow", "one month old", "farm."
1. What does Mary have?
A. Lamb
B. Cow
C. Dog
D. Cat
A. Lamb
2. What does John have?
A. Cow
B. Lamb
C. Dog
D. Cat
A. Cow
3. How old is Mary's lamb?
A. One month old
B. One year old
C. Two months old
D. Two years old
etc.
It works perfectly when I don't give keywords, with multiple prompts, with all these answer options, correct answer, everything. The problem is when I want to generate these questions such that the correct answer is the keyword. I tried all kinds of prompts, even giving examples like what I did above, but it doesn't work.
Using GPT-J I tested the following:
Input:
Create a quiz about the following text according to the keywords: Mary has little lamb and John has a cow. The lamb is one month old. It eats grass and milk, which Mary brings from the the farm.
Keywords: "lamb", "cow", "one month old", "farm."
Question 1: What does Mary have?
A. Lamb
B. Cow
C. Dog
D. Cat
Answer: A. Lamb
Question 2:
Output:
Create a quiz about the following text according to the keywords: Mary has little lamb and John has a cow. The lamb is one month old. It eats grass and milk, which Mary brings from the the farm.
Keywords: "lamb", "cow", "one month old", "farm."
Question 1: What does Mary have?
A. Lamb
B. Cow
C. Dog
D. Cat
Answer: A. Lamb
Question 2: What does John have?
A. Lamb
B. Cow
C. Dog
D. Cat
Answer: B. Cow
Question 3: How old is the lamb?
A. 1 month
B. 1 year
C. 2 months
D. 1 year and a half
Answer: C. 2 months
Giving an example of what you want in the input (1 question + answer) can help GPT to understand the structure of the desired output. Explicitly explaining the GPT what the desired task is in the input (see line 1) can help it to understand the task in my experience.
It does not execute the task perfectly, but using GPT-3 might help and I hope this is a step in the right direction for you.