Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
AI gateway and observability
Portkey (like Helicone and Kong AI Gateway) sees every call: logs, guardrails, caching and retries. OpenSmartRoute chooses the route across kinds with a cost/quality objective and learns from outcomes; run it behind the gateway you already have.
“Portkey observes and gates the traffic you already route; OpenSmartRoute chooses the target for each request and learns from the outcome.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Portkey |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost)Guardrails on the payload; routing rules are static configs | Yes | Partial |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependencies | Yes | Partial |
| Prompt-injection and steering-gadget defences on the routerGuardrail plugins on content, not on the routing decision | Yes | Partial |
| Output guard on responses (PII, secrets, injection: flag, redact or block) | Yes | Yes |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images) | Yes | Yes |
| Anthropic Messages API (/v1/messages)Through the Anthropic SDK pointed at the gateway | Yes | Partial |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiry | Yes | Yes |
| Per-target rate limits, budgets and circuit breakersLoad balancing and fallbacks per config; budgets are per key or workspace | Yes | Partial |
| Python and TypeScript SDKs | Yes | Yes |
| MCP server for IDEs and agentsMCP client support in the gateway; not an MCP server for routing | Yes | Partial |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callDecision trace, tokens, cost and outcome per request; prompt and answer stored when the workspace turns on request logging (redacted or full) | Yes | Yes |
Point the gateway's routing config at a single upstream: the OpenSmartRoute proxy with model osr/auto. Observability stays where it is.
client = OpenAI(base_url=f"{GATEWAY}/v1")
client.chat.completions.create(
model="osr/auto:cheap", # or auto:fast|quality|private|agentic, or a target id
messages=messages,
extra_body={"osr": {"constraints": {"region": "eu", "data_boundary": "private"}}},
)Run the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Cloudflare AI Gateway, Bifrost (Maxim AI), Kong AI Gateway, Helicone, Not Diamond, Martian, Unify, Requesty, Building it yourself.
Photo: Mario Gogh
