LLM Gateway

LLM Gateway

OpenAI-compatible HTTP gateway over rig. Supports streaming chat completions and a static model catalog. Auth: Hydra-issued OAuth2 access token (RS256, RFC 9068, aud=agent-api).

Owner
agents
Generator
utoipa
Version
0.1.0
Paths
3

Select an operation from the sidebar to view its parameters, request and response schemas, and example payloads. The raw spec is available at openapi.json.

Chat completions (streaming + non-streaming, with reasoning) POST

OpenAI-compatible chat completions over an upstream provider chosen by `model`. Set `"stream": true` for SSE; otherwise the response is a single JSON body. ### Reasoning Opt into model thinking with top-level `reasoning_effort` (`minimal | low | medium | high | xhigh | max`). Allowed values are per-model — query `/v1/models` for `reasoning_levels`. Reasoning is **streaming-only**: non-streaming responses drop it entirely. Streaming reasoning surfaces two channels per `delta`: - `reasoning_content` — display text for live UI (DeepSeek-style extension; concatenate across chunks like `content`). - `reasoning_meta` — structured snapshot at block close, carrying the `signature` (Anthropic) or `encrypted_content` (OpenAI) clients need to replay reasoning on subsequent turns. ### Multi-turn replay Persist `reasoning_meta` between turns and send it back as a `{"type": "reasoning", ...}` block on the assistant message — before text and tool calls (Anthropic enforces this order). Required on Anthropic for tool-using turns (the API 400s without the matching thinking block); recommended on OpenAI so the model doesn't re-derive its plan and re-bill reasoning tokens. OpenAI's Responses API requires `id` on every replayed reasoning block — the gateway 400s pre-flight if missing. Anthropic ignores `id` (signatures live per-block instead). Signatures and encrypted blobs are **model-bound**: persist `model` alongside the reasoning meta and refuse to replay across models. See `docs/llm-gateway/REASONING.md` for the full contract, JSONL persistence shape, and end-to-end curl examples.