Skip to main content
With stream: true, Nito sends the reply a few tokens at a time over one long-lived connection instead of one JSON body. Use it when a person is watching the output or the output is long. Skip it for batch jobs that wait for the full text anyway. The exact wire format is in the streaming format reference.

​
Turn It On

OpenAI SDKs expose this as stream=True and hand you an iterator of chunks.

​
What Arrives

  • Content chunks. object is chat.completion.chunk; new text is in choices[].delta.content. The assistant role arrives once, on the first chunk. Concatenate the deltas in order.
  • Heartbeats. A : keep-alive comment line roughly every 15 seconds. Ignore it.
  • Usage chunk. With stream_options.include_usage: true, a final chunk with an empty choices array and a usage object arrives before the end.
  • Terminator. data: [DONE].
Tool calls arrive as tool_calls fragments assembled by index (Tool Calling); reasoning arrives as reasoning_details deltas (Reasoning).

​
Mid-Stream Errors

Once streaming starts, 200 has already been sent. A later failure arrives as a chunk with a top-level error object and finish_reason: "error". Check each chunk for error before reading delta.content. A failure before any byte is written returns an ordinary HTTP error.

​
Privacy

Streaming works at every level. At Encrypted, the deltas arrive as ciphertext for your client to decrypt.

Chat Completions

The endpoint.

Tool Calling

Function calls.

Reasoning

Think first.

Stream Format Reference

Every chunk field.

Privacy Levels

The four levels.