Tool call streaming - Nito

A tool call arrives in fragments inside delta.tool_calls, keyed by index. The non-streaming shape and the round-trip are on Tool calling.

Start

The first fragment names the tool and assigns its id:

{ "index": 0, "id": "call_abc", "type": "function", "function": { "name": "get_weather" } }

Argument Deltas

Each fragment carries the index and a piece of arguments:

{ "index": 0, "function": { "arguments": "{\"city\":" } }

{ "index": 0, "function": { "arguments": " \"Paris\"}" } }

Reassembling

One buffer per index. Record id, type, and function.name from the start fragment; concatenate each function.arguments piece; when the terminal chunk arrives with finish_reason: tool_calls, parse each buffer as JSON. The gateway does not send a final echo of the assembled arguments.

{ "choices": [ { "index": 0, "delta": {}, "finish_reason": "tool_calls" } ] }

Then run the tools, append each result as a tool message with the matching tool_call_id, and call again. The gateway routes function tools; it does not execute them.