OpenAI Assistant Starter Kit Updated with AssistantStream

There have been several significant changes to the OpenAI APIs over the last month (April, 2024) and I’ve updated the OpenAI Assistant Starter Kit to take advantage of these changes. These changes have enabled me to significantly simplify the Starter Kit code.

What is the OpenAI Assistant Starter Kit?

As a reminder, the OpenAI Assistant Starter Kit is an open-source application that illustrates how you can easily start building OpenAI Large Language Model (LLM) applications using NextJS + TypeScript and OpenAI. You can view a live version of the Starter Kit (go chat with it right now – I will wait).

I first described the OpenAI Starter Kit in the following blog post:

Use the OpenAI Assistant Starter Kit to Quickly Build New OpenAI Apps

Using the New AssistantStream Class

The biggest change to the OpenAI Node library is the introduction of the fromReadableStream() method to the AssistantStream class. This method enables you to easily read a stream from the server on the client.

OpenAI uses Server-Sent Events to stream text responses from an Assistant. Instead of waiting for OpenAI to compose the entire response to a chat message, the response can be sent back to the browser in chunks. That way, the user is not left waiting while nothing happens. Here’s a screen capture of streaming from the OpenAI Assistant Starter Kit:

To get streaming to work, you need to add the right server-side code and the right client-side code. Here’s what the server-side code looks like to stream a response to the client:

// add new message to thread
await openai.beta.threads.messages.create(
    newMessage.threadId,
    {
        role: "user",
        content: newMessage.content
    }
);

// create a run
const stream = await openai.beta.threads.runs.create(
    newMessage.threadId, 
    {assistant_id: newMessage.assistantId, stream:true}
);

return new Response(stream.toReadableStream());

The code above adds a new message to an OpenAI conversation thread and then returns a stream by calling the threads.runs.create() method. Instead of returning the entire OpenAI chat response, this method streams the response in chunks.

The client code (in the ReactJS component) is equally simple:

// post new message to server and stream OpenAI Assistant response
const response = await fetch('/api/openai-assistant', {
    method: 'POST',
    body: JSON.stringify({
        assistantId: assistantId,
        threadId: threadId,
        content: prompt,
    }),
});


if (!response.body) {
    return;
}
const runner = AssistantStream.fromReadableStream(response.body);

runner.on('messageCreated', (message) => {
    setThreadId(message.thread_id);
});

runner.on('textDelta', (_delta, contentSnapshot) => {
    const newStreamingMessage = {
        ...streamingMessage,
        content: contentSnapshot.value,
    };
    setStreamingMessage(newStreamingMessage);
});

runner.on('messageDone', (message) => {
    // get final message content
    const finalContent =  message.content[0].type == "text" ? message.content[0].text.value : "";

    // add assistant message to list of messages
    messageId.current ++;
    setMessages( (prevMessages) =>
        [
            ...prevMessages, 
            {
                id: messageId.current.toString(),
                role: "assistant",
                content: finalContent,
                createdAt: new Date(),
            } 
        ]
    );

    // remove busy indicator
    setIsLoading(false);
});

The code above posts a new message to the server. The newly added OpenAI AssistantStream.fromReadableStream() method is used to consume the response from the server. Notice that two event handlers are used:

• messageCreated – this event is raised once when the new message is created. I am grabbing the thread Id in this event handler.
• textDelta – this event is raised when each chunk of text is sent from the server. The event handler updates the page to show the text as it is streamed from the server.
• messageDone – this event is raised after the entire message has been streamed. The code above adds the final message to the conversation history.

Conclusion

This latest update to the OpenAI APIs and OpenAI Node library enables you to more easily build a chat application. Use the latest version of the OpenAI Assistant Starter Kit to kickstart your OpenAI development.