I am wondering if I can be able to use OpenAI GPT-3 for transfer learning in a text classification problem? If so, how can I get start on it using Tensorflow, Keras.
(i substituted hateful language with ******** in the following samples)
Given samples like:
("You look like ****** *** to me *******", true)
("**** you *********", true)
("**** my ****", true)
("hey my name is John can you help me?", false)
("hey my name is John, i think you ****** ***!", true)
("i have a problem with my network driver hpz-3332d", false)
GPT-3 can indeed then decide if a given input is hateful or not. GPT-3 actually is implementing filters that will very effectively tell if an arbitrary comment is hatefull or not. You would just enter the msg and let GPT3 autcomplete the , true|false) part at the end, setting tokens to about ~6 and temperature setting 90%.
Boolean-ish classification that also relies on more complex context (you can insult someone without using foul-language) id doeable with GPT3 and can also be done with GPT2.