Streaming

Server-sent events, where usage arrives, and what happens when a stream comes apart.

Set stream: true and the same call answers text/event-stream instead of one JSON body. Nothing else about the request changes, and nothing about what you are billed changes either.

The event shape

One data: event per chunk, terminated by a literal data: [DONE]:

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"delta":{"content":"A distributed"}}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"delta":{"content":" store can hold"}}]}

data: {"id":"chatcmpl-…","choices":[{"delta":{},"finish_reason":"stop"}],"usage":{…}}

data: [DONE]

/v1/messages streams Anthropic’s named-event format instead — message_start, content_block_delta, message_delta, message_stop — because an Anthropic SDK is parsing it. Same transport, same guarantees below.

usage arrives whether or not you ask

The last data event carries the usage object. You do not need stream_options.include_usage to get it. That parameter is accepted for compatibility and changes nothing.

This is a deliberate difference from OpenAI, where omitting it means no usage at all on a stream. Streaming should not be the mode where you stop being able to see what you were charged.

If you are aggregating token counts yourself, read them from the final event rather than counting deltas. The final event is the number you are billed on; a delta count is your own estimate of it.

When a stream comes apart

Chunks already sent are not withdrawn — they were correct when they were sent. What follows is one more event carrying an error object, and then [DONE].

data: {"error":{"message":"The response was cut off because the model timed out.","type":"server_error","code":"PARTIAL_RESPONSE_TIMEOUT","param":null}}

data: [DONE]

So the rule on the client side is: a stream that ended without [DONE] is incomplete. Treat it as a failure even though you have partial text, because you cannot tell a finished answer from a dropped connection any other way.

What a broken stream costs

This is the one place where a failed call is billed, and it is worth being precise about why. Of the errors this API can return, exactly two produce a usage record, and both are streams:

Code	When
PARTIAL_RESPONSE_TIMEOUT	The model stopped producing output before it was finished.
REQUEST_CANCELLED	You disconnected — closed the connection, cancelled the request, hit a client timeout.

In both cases the model had already generated tokens. Those tokens exist, and the compute that produced them was spent, whether or not they reached you. Everything else on Errors is free when it fails.

Cancelling a stream is not a way to stop being charged for what has already been generated. If you are experimenting with long generations, cap max_tokens rather than disconnecting early.

Next

Reliability and error handling

Timeouts, retries, and which codes are worth retrying.

Usage and billing

What the numbers in usage mean and where to see them again.