How to use the OpenAI stream=true property with a Django Rest Framework response, and still save the content returned?

I'm trying to use the stream=true property as follows.

completion = openai.Completion.create(
            model="text-davinci-003",
            prompt="Write me a story about dogs.",
            temperature=0.7,
            max_tokens=MAX_TOKENS,
            frequency_penalty=1.0,
            presence_penalty=1.0,
            stream=True,
        )

Unfortunately, I don't know what to do from here to return it to my React frontend. Typically, I've used standard response objects, setting a status and the serializer.data as the data. From my readings online, it seems I have to use the StreamingHttpResponse, but I'm not sure how to integrate that with the iterator object of completion, and actually save the outputted data once it is done streaming, as the view will end after returning the iterator to the endpoint. Any help?

Solution

You can use StreamingHttpResponse. Note that you can't see it stream live for most api clients like postman, but you can see it on your terminal. If you'd like to use it for react, you'll have to use fetch api.

@api_view(["POST"])
def generate_names(request):
    if request.method == 'POST':
        # Parse the request body and extract the prompt
        prompt = request.data.get('prompt')
        
        # Set up the OpenAI API client
        openai.api_key = OPENAI_API_KEY
        
        # Define a generator function to stream the response
        def generate_response():
            for chunk in openai.ChatCompletion.create(
                model="gpt-3.5-turbo",
                messages=[{
                    "role": "user",
                    "content": prompt
                }],
                stream=True,
            ):
                content = chunk["choices"][0].get("delta", {}).get("content")
                if content is not None:
                    
                    yield content
        
        # Return a streaming response to the client
        return StreamingHttpResponse(generate_response(), content_type='text/event-stream')
    
    # Return a JSON error if the request method is not POST
    return JsonResponse({'error': 'Method not allowed.'}, status=405)

To see it in your terminal use this Curl command http://127.0.0.1:8000/api/v1/askkk --header "Content-Type: application/json" --data '{"prompt": "How do I set up payment invoices?"}'