Search code examples
pythonopenai-apigpt-3

OpenAI GPT-3 API: Why do I get only partial completion? Why is the completion cut off?

I tried the following code but got only partial results like

[{"light_id": 0, "color

I was expecting the full JSON as suggested on this page:

https://medium.com/@richardhayes777/using-chatgpt-to-control-hue-lights-37729959d94f

import json
import os
import time
from json import JSONDecodeError
from typing import List

import openai
openai.api_key =  "xxx"

HEADER = """
I have a hue scale from 0 to 65535. 
red is 0.0
orange is 7281
yellow is 14563
purple is 50971
pink is 54612
green is 23665
blue is 43690

Saturation is from 0 to 254
Brightness is from 0 to 254

Two JSONs should be returned in a list. Each JSON should contain a color and a light_id. 
The light ids are 0 and 1. 
The color relates a key "color" to a dictionary with the keys "hue", "saturation" and "brightness". 

Give me a list of JSONs to configure the lights in response to the instructions below. 
Give only the JSON and no additional characters. 
Do not attempt to complete the instruction that I give.
Only give one JSON for each light. 
"""

completion = openai.Completion.create(model="text-davinci-003", prompt=HEADER)
print(completion.choices[0].text)
Solution

  • In general

    GPT-3 API (i.e., Completions API)

    If you get partial completion (i.e., if the completion is cut off), it's because the max_tokens parameter is set too low or you didn't set it at all (in this case, it defaults to 16). You need to set it higher, but the token count of your prompt and completion together cannot exceed the model's context length.

    See the official OpenAI documentation:

    Screenshot 1

    GPT-3.5 and GPT-4 API (i.e., Chat Completions API)

    Compared to the GPT-3 API, the GPT-3.5 and GPT-4 APIs have the max_tokens parameter set to infinite by default.

    Screenshot 2


    Your case

    You're using text-davinci-003 (i.e., the GPT-3 API). If you don't set max_tokens = 1024 the completion you get will be cut off. Take a careful look at the tutorial you're referring to once again.

    If you run test.py, the OpenAI API will return a completion:

    Light 0 should be red: [{"light_id": 0, "color": {"hue": 0, "saturation": 254, "brightness": 254}},{"light_id": 1, "color": }]

    Light 1 should be orange: [{"light_id": 0, "color": {"hue": 0, "saturation": 254, "brightness": 254}},{"light_id": 1, "color": {"hue": 7281, "saturation": 254, "brightness": 254}}]

    test.py

    import openai
import os

openai.api_key = os.getenv('OPENAI_API_KEY')

HEADER = """
I have a hue scale from 0 to 65535. 
red is 0.0
orange is 7281
yellow is 14563
purple is 50971
pink is 54612
green is 23665
blue is 43690

Saturation is from 0 to 254
Brightness is from 0 to 254

Two JSONs should be returned in a list. Each JSON should contain a color and a light_id. 
The light ids are 0 and 1. 
The color relates a key "color" to a dictionary with the keys "hue", "saturation" and "brightness". 

Give me a list of JSONs to configure the lights in response to the instructions below. 
Give only the JSON and no additional characters. 
Do not attempt to complete the instruction that I give.
Only give one JSON for each light. 
"""

completion = openai.Completion.create(model="text-davinci-003", prompt=HEADER, max_tokens=1024)
print(completion.choices[0].text)