Skip to content

Streaming ​

Set "stream": true to receive tokens as they're generated, as server-sent events in OpenAI's chunk format:

text
data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hel"}}]}

data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"lo"}}]}

data: [DONE]

Add "stream_options": {"include_usage": true} to get a final chunk with token usage.

ts
const stream = await client.chat.completions.create({ model: "anthropic/claude-sonnet-5-5", messages, stream: true });
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta.content ?? "");
python
for chunk in client.chat.completions.create(model="anthropic/claude-sonnet-5-5", messages=messages, stream=True):
    print(chunk.choices[0].delta.get("content") or "", end="")

Failures ​

  • Before the first chunk: the request falls back to another provider as usual; you never see the failure.

  • After streaming has started: the stream ends with an error event and no [DONE]:

    text
    data: {"error":{"type":"provider_error","code":"provider_error","message":"…"}}

    The SDKs raise this as a ProviderError.

Billing for interrupted streams ​

If the provider reports usage, you pay for exactly that. If the stream is cut short (you disconnect, or the provider fails mid-stream) you pay for the input plus an estimate of the output actually delivered; such requests show status partial in Logs.