Langchain streaming in not working with Next.JS 13+ (Next.JS App Router)

Question

I am trying to implement Langchain (Conversational Retrieval QA stream) in my NextJS 13 (App router) application and am not able to stream data to FE, I am trying to use this NextResponse(stream); I want to do something like this

    res.writeHead(200, {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive',
      'Transfer-Encoding': 'chunked',
    });
    completion.data.on('data', (data) => {
      res.write('event: message\n');
      res.write('data: ' + JSON.stringify(contentValue) + '\n\n');
    });
    completion.data.on('end', () => {
      res.write('event: close\n');
      res.write('data: close\n\n');
      res.end(); // End the response stream
    });

this is a NodeJS example from my other open.ai project.

Answer 1

Try this:

api/example/route.ts

const stream = await chain.stream({
  chatHistory: userChatHistory,
  question: userQuestion,
});

return new NextResponse(stream, {
  headers: {
    'Content-Type': 'text/event-stream',
    'Connection': 'keep-alive',
    'Cache-Control': 'no-cache, no-transform',
  },
});

Client

const res = await fetch('/api/example', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
  body: yourPayload,
});


if (!res.ok) throw new Error(res.statusText);
    const data = res.body;
    if (!data) return;

    const reader = data.getReader();
    const decoder = new TextDecoder();
    let done = false;
    let accumulatedContent = '';
    while (!done) {
      
      const { value, done: doneReading } = await reader.read();
      done = doneReading;
      const chunkValue = decoder.decode(value);
      accumulatedContent += chunkValue;
      console.log(accumulatedContent);
    }

    if (done) {
      console.log(accumulatedContent);
    }