OpenAI_API chat returning null

Question

I'm attempting to stream the results from a call to ChatGPT using the OpenAI_API library.

public async IAsyncEnumerable<string> Stream(string prompt) {
    var chat = api.Chat.CreateConversation(new ChatRequest() {
        Model = this.LanguageModel,
        Temperature = 0.1,
        MaxTokens = 50
    });
    chat.AppendUserInput(prompt);
    Console.WriteLine(String.Format("Querying with prompt: {0}", prompt));

    int i = 0;
    await foreach (var resp in chat.StreamResponseEnumerableFromChatbotAsync()) {
        Console.WriteLine(i++);
        Console.WriteLine(chat.MostResentAPIResult);
        yield return resp;
    }
    Console.WriteLine("Last");
    Console.WriteLine(chat.MostResentAPIResult);
}

The documentation shows this usage:

var chat = api.Chat.CreateConversation();
chat.AppendUserInput("How to make a hamburger?");

await foreach (var res in chat.StreamResponseEnumerableFromChatbotAsync())
{
    Console.Write(res);
}

Unfortunately, when I test my implementation (without all the console logging) it returns an empty string.

public async Task IntegrationTestPromptStream() {
    IConversation conv = new OpenAIConversation(OpenAI_API.Models.Model.GPT4);
    var response = conv.Stream("Repeat back this sentence \"This is a multi word example that should chunk\"");
    var chunks = new List<string>();
    await foreach (var chunk in response) {
        chunks.Append(chunk);
    }

    Assert.That(
        String.Join(" ", chunks),
        Is.EqualTo("This is a multi word example that should chunk"),
        "Didn't receive expected response from OpenAI chat");
}

I added the console logging statements to work out what was being invoked and this is the output

Querying with prompt: Repeat back this sentence "This is a multi word example that should chunk"
0

NUnit Adapter 4.1.0.0: Test execution complete

If I attach the debugger and attempt to examine chat.MostResentAPIResult [sic], I can see an object, but notably the .ToString() method throws a null reference exception.

Finally, if I just keep hte index counter in but remove the statements to output the last response, I get this

Querying with prompt: Repeat back this sentence "This is a multi word example that should chunk"
0
1
2
3
4
5
6
7
8
9
Last

So It'll iterate through empty strings all day long but attempting to look at the response returns a null or a NullReferenceException.

What am I missing?

Answer 1

Example code

using System.Collections;
using System.Collections.Generic;


List<string> strings = new List<string>();

void OutputAdd()
{
    Console.WriteLine("Output for Add Mehtod ...");


    for (int i = 0; i < 10; i++)
    {
        strings.Add($"Testing using Add {i}");
    }

    foreach (var line in strings)
    {
        Console.WriteLine(line);
    }
}

void OutputAppend()
{
    Console.WriteLine("Output for Append Mehtod ...");


    for (int i = 0; i < 10; i++)
    {
        strings.Append($"Testing using Append {i}");
    }

    foreach (var line in strings)
    {
        Console.WriteLine(line);
    }
}

void OutputAppendFixed()
{
    Console.WriteLine("Output for Append Fixed Mehtod ...");


    for (int i = 0; i < 10; i++)
    {
        strings = strings.Append($"Testing using Append with added code {i}").ToList();
    }

    foreach (var line in strings)
    {
        Console.WriteLine(line);
    }
}

OutputAdd();

OutputAppend();

OutputAppendFixed();

If you run this, you'll see that OutputAppend() will only show what was added from OutputAdd(). Then when OutputAppendFixed() runs it will add using the Append method. This has to do with Append returning an IEnumerable<T>.

The Add modifies the original data while Append returns a new array without modifying the original data.