Building a Streaming Chatbot

The useChatStream composable turns a plain API route into a conversational interface. It sends the user's message, reads the response as a stream of chunks, and keeps a reactive list of messages that your template can render as it grows.

Out of the box, the composable gives you:

• Incremental rendering: tokens are appended to the last message the moment they arrive, so the answer types itself out instead of appearing all at once.
• Request state: status, error and an abort() function are tracked for you, ready to bind to buttons and spinners.
• Bring your own UI: it returns data, not components. Render the messages with whatever markup your design system already uses.

This guide walks through a minimal chat page with real-time streaming. If your assistant needs to call functions, read the tool calling guide once you have the basics working.

Example

A single round trip looks like this:

1. The user submits the form and send() posts the conversation to /api/chat.
2. The server route forwards it to your model provider and pipes the response body straight back.
3. The composable decodes each chunk with a TextDecoder and appends it to the assistant's message.

<script setup lang="ts">
import { ref } from 'vue'
import { useChatStream } from '@/composables/use-chat-stream'

const input = ref('')
const { messages, send, status } = useChatStream({ api: '/api/chat' })

function submit() {
  send(input.value)
  input.value = ''
}
</script>

<template>
  <ChatMessage v-for="m in messages" :key="m.id" :message="m" />
  <form @submit.prevent="submit">
    <input v-model="input" :disabled="status === 'streaming'" />
  </form>
</template>

On the server, the route does nothing more than relay the stream:

// server/api/chat.post.ts
export default defineEventHandler(async (event) => {
  const { messages } = await readBody(event)

  const upstream = await fetch(process.env.MODEL_URL!, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ messages, stream: true }),
  })

  return sendStream(event, upstream.body!)
})

Note: keep the provider key on the server. The browser only ever talks to your own route, never to the model provider directly.

Options

Option	Type	Description
api	string	The endpoint that receives the conversation. Defaults to /api/chat.
initial	Message[]	Messages to show before the first request, such as a greeting.
onFinish	(m: Message) => void	Called once the stream closes, with the completed message.

Cancelling a response

Long answers are easy to regret. Call abort() to close the connection; whatever text has already arrived stays in the message, and status returns to idle.

---

Next steps

With streaming in place, the natural next step is persistence. Store each finished message in onFinish, and pass the stored list back as initial when the page loads.