Skip to main content

​
What Nito Does

  • Failover within a model. A logical model can have several physical backends. Nito filters by your requirements and routes around a backend in trouble.
  • A circuit breaker. A provider that keeps failing is held out and probed again after a cooldown.
The privacy level never silently downgrades. A :confidential request that cannot be served as asked fails. Your fallback logic must never retry at a weaker level.

​
App-Level Fallback

Only your app knows which alternate model is acceptable. On 502 (provider_error) or 503 (unavailable), fall back to an alternate you selected from GET /v1/models at startup, at the same privacy level.

​
Retry With Backoff

  • Retry 502 and 503.
  • On 429, honor Retry-After as a floor. 429 also covers insufficient_quota, which retrying will not fix; check the code first.
  • Never retry 400, 401, or 403; they fail identically.
Exponential backoff with jitter, capped attempts.

​
Together

  1. Nito routes within the model and trips its breaker. Nothing to do.
  2. Transient 502/503: retry with backoff, honoring Retry-After.
  3. Retries exhausted: fall back to an alternate model at the same privacy level.
  4. Still failing: surface a clean error keyed by the request ID.
Full status handling: Error Format.

How the API Works

The request lifecycle.

Error Format

The error envelope.

Retry Behavior

Safe to retry, or not.

Rate Limits

The limit headers.