Skip to main content
A long, stable prefix (a system prompt, a document, a tool set) is processed on every call. Prompt caching lets the provider remember it. On Nito this is a passthrough: you mark cacheable content, Nito preserves the marker on routes that advertise cache support and reports the provider’s accounting in usage. Nito keeps no cache of its own. On routes without cache support the markers are dropped; explicit cache-retention requests the gateway cannot enforce are rejected.

​
Marking Content

Attach cache_control to a text, image, or file content part. Put stable content first, the variable part last.
prompt_cache_key is forwarded as the provider’s cache affinity key where the route supports it.

​
Accounting

When the provider reports it, cache accounting appears in usage, for example cached_tokens under the prompt-token details. The first call writes the prefix; later calls that hit it read it at the provider’s reduced rate.

​
Scope

Nito never returns a stored response for an identical request. Every request runs the model. Deduplicate identical requests in your own application layer.

Chat Completions

The endpoint.

Safe Retries

Retry without paying twice.

Cost Control

Spend guidance.