Streaming

Set "stream": true to receive the response incrementally as server-sent events (SSE), rather than waiting for the full completion. Streaming uses the same /v1/chat/completions endpoint and is fully OpenAI-compatible, so any OpenAI SDK handles it for you.

Streaming a response

curl https://gateway.errorbar.ai/v1/chat/completions \
  -H "Authorization: Bearer $ERRORBAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-32B",
    "messages": [{ "role": "user", "content": "Count from 1 to 5." }],
    "stream": true
  }'

stream = client.chat.completions.create(
    model="Qwen/Qwen3-32B",
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

const stream = await client.chat.completions.create({
  model: "Qwen/Qwen3-32B",
  messages: [{ role: "user", content: "Count from 1 to 5." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0].delta.content ?? "");
}

The event format

Each event is a data: line containing a chunk in OpenAI’s streaming shape (object: "chat.completion.chunk"). Instead of a full message, each choice carries a delta with the incremental piece. The stream terminates with a literal data: [DONE] line:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"1"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":", 2"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Concatenate every choices[0].delta.content to reconstruct the full text. The final content-bearing chunk includes a finish_reason.

Usage on streamed requests

By default, streamed responses don’t include a usage object. Set stream_options: {"include_usage": true} to receive one final chunk carrying the authoritative token counts, so you don’t have to count tokens yourself.

stream = client.chat.completions.create(
    model="Qwen/Qwen3-32B",
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.usage:                       # only the final chunk has usage
        print(chunk.usage.total_tokens)
    elif chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

const stream = await client.chat.completions.create({
  model: "Qwen/Qwen3-32B",
  messages: [{ role: "user", content: "Count from 1 to 5." }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of stream) {
  if (chunk.usage) console.log(chunk.usage.total_tokens);
  else process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

The usage chunk comes just before data: [DONE] and has an empty choices array:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":9,"total_tokens":27}}

data: [DONE]

Billing and cancellation

Billing counts tokens actually produced, including partial output. If your client disconnects or cancels mid-stream, errorbar settles the tokens generated up to that point. You’re charged for what was produced, not the whole request, and never for nothing.

This matters for agents and UIs that stop generation early (for example, when a user navigates away). Cancelling saves you the tokens you would have received after the cancel point, but the tokens already streamed are billed.

When to stream

Stream when you’re rendering output to a user in real time (chat UIs, agents) so they see progress immediately. For batch jobs where you only need the final text, a non-streaming request is simpler to consume.

Get started

Capture

Observe

Grade & Judge

Evaluate

Improve

Run at scale

Platform

Streaming a response

The event format

Usage on streamed requests

Billing and cancellation

When to stream

​Streaming a response

​The event format

​Usage on streamed requests

​Billing and cancellation

​When to stream

Streaming a response

The event format

Usage on streamed requests

Billing and cancellation

When to stream