Search code examples
openai-apichatgpt-api

OpenAI Chat Completions API: Can I fine-tune a GPT-3.5 model?

I have fine-tuned an openai language model (curie) and was able to access the model via the openai.Completion.create method, but I could not access the fine-tuned model via openai.ChatCompletion.create.

By researching a bit, I have found out that the problem is not in the fine-tuning but in the fact that the original curie model is not accessible via openai.ChatCompletion.create.

By looping over these models:

models = ['gpt-3.5-turbo', 'davinci', 'curie', 'babbage', 'ada']

I found out that only gpt-3.5-turbo model is accessible via openai.ChatCompletion.create and it is not accessible via openai.Completion.create. In contrast, the remaining four models are accessible via openai.Completion.create but are not accessible via openai.ChatCompletion.create.

So, my first question is: can someone confirm my finding? Is what I found out written somewhere in the OpenAI documentation?

My second question is: is it possible to fine-tune a model that supports chat?

For example, on the official page, I see that:

Fine-tuning is currently only available for the following base models: davinci, curie, babbage, and ada.

So, did I get it right that we can only fine-tune models that do not support chat?

Solution

  • Question 1:

    I found out that only gpt-3.5-turbo model is accessible via openai.ChatCompletion.create and it is not accessible via openai.Completion.create. In contrast, the remaining four models are accessible via openai.Completion.create but are not accessible via openai.ChatCompletion.create.

    So, my first question if someone can confirm my finding?

    Answer 1:

    Yes, correct. The reason why this is the case is that the gpt-3.5.-turbo model is a GPT-3.5 model. All the other models you mentioned (i.e., davinci, curie, babbage, and ada) are GPT-3 models.

    GPT-3.5 models use a different API endpoint than GPT-3 models. This is not explicitly written in the documentation, but it's very clear if you read the whole documentation.

    Question 2:

    My second question is if it is possible to fine-tune a model that supports Chat / Dialogue?

    Answer 2:

    No, it's not possible. You want to fine-tune a GPT-3.5 model, which is not possible as of March 2023. Also, it doesn't seem this will change in the near future, if ever. Why?

    I strongly recommend you to read the official OpenAI article on how ChatGPT's behavior is shaped to understand why you can't fine-tune a GPT-3.5 model. I want to emphasize that the article doesn't discuss specifically the fine-tuning of a GPT-3.5 model, or better yet, its inability to do so, but rather ChatGPT's behavior. It's important to emphasize that ChatGPT is not the same as the GPT-3.5 model, but ChatGPT uses chat models, which GPT-3.5 belongs to, along with GPT-4 models.

    As stated in the article:

    Unlike ordinary software, our models are massive neural networks. Their behaviors are learned from a broad range of data, not programmed explicitly. /.../ An initial “pre-training” phase comes first, in which the model learns to predict the next word in a sentence, informed by its exposure to lots of Internet text (and to a vast array of perspectives). This is followed by a second phase in which we “fine-tune” our models to narrow down system behavior.

    First, we “pre-train” models by having them predict what comes next in a big dataset that contains parts of the Internet. They might learn to complete the sentence “instead of turning left, she turned ___.” By learning from billions of sentences, our models learn grammar, many facts about the world, and some reasoning abilities. They also learn some of the biases present in those billions of sentences.

    Then, we “fine-tune” these models on a more narrow dataset that we carefully generate with human reviewers who follow guidelines that we provide them. /.../ Then, while they are in use, the models generalize from this reviewer feedback in order to respond to a wide array of specific inputs provided by a given user.

    Visual representation (source):

    Screenshot

    As you can see, chat models (i.e., GPT-3.5 and GPT-4 models) are already "fine-tuned" by the OpenAI. This is the reason why you can only fine-tune base models: davinci, curie, babbage, and ada. These are the original models that do not have any instruction following training.