Can we run it inside our own network?

Yes. The router runs as a container or a Helm release on your infrastructure, and a routing rule keeps regulated data on the models you allow.

Will this lock us into another vendor?

No. The router is Apache-2.0 open source, your provider keys stay yours, and the API is OpenAI-compatible, so leaving is a change of base URL.

How much can we save on our LLM bill?

On the public benchmark the router saved 89% against sending everything to the reference model. Import last month's provider bill on the bill page and the estimate is computed on your own traffic before you change a line of code.

Use cases

The same router, a different first rule for every team.

Each page below starts from the situation a team is in - the tickets, the code, the account data, the patient letter, the contract, the campaign - says what changes once requests are routed under one policy, follows one request through, and ends with the two text files that produce it. The saving on every card is a live quote of a request of that kind.

Customer support
Answer the easy tickets cheaply. Escalate the hard ones before they get worse.
Problem
A support desk's traffic is mostly simple and occasionally serious. One model for all of it either pays frontier prices to answer 'where is my order' or lets a small model mishandle a furious customer - and the handover to a person is a rule someone wrote into the bot, not something the router can see.
Routing strategy
Complaint or chargeback detected, then Small model removed from the candidates.
Models it routes between
llm-smallllm-frontierhuman-escalation
What changes
FAQs and status questions answered by the cheapest model that meets your quality rules
Quoting a request of this kind against the public catalogue…
Software engineering
The right model for each coding task, from a one-line fix to a redesign.
Problem
An IDE assistant uses whatever model it was configured with for everything - a variable rename and a cross-module refactor cost the same and get the same attention. Switching is a setting a person flips, and nobody knows what the day's completions cost until the invoice.
Routing strategy
Code and complexity detected, then Prose is kept off the coding agent.
Models it routes between
coding-agent
What changes
Small edits and completions on a fast, cheap model; real code on the coding agent
Quoting a request of this kind against the public catalogue…
Financial services
Account data never leaves your boundary. Everything else routes on cost.
Problem
A bank's AI traffic mixes public questions with requests that carry account numbers, balances and names. Model risk, data residency and the regulator all ask the same thing - where did this request go, and why - and an application-level setting cannot answer it.
Routing strategy
Cloud models removed from the candidates, then On-prem model answers.
Models it routes between
llm-onpremllm-mid
What changes
Requests with account data pinned to the on-prem model by a rule no score can override
Quoting a request of this kind against the public catalogue…
Healthcare
Route sensitive healthcare workloads to approved models - and take the decision out of every application.
Problem
Clinical and administrative teams want the same AI tools everyone else has, and every application they use would need its own rule about what may leave the network. One missed rule is a breach. The decision has to live in one place, before any model is called.
Routing strategy
External models removed, then On-prem model selected.
Models it routes between
llm-onpremllm-frontierllm-midclinician-review
What changes
PHI detected and external models removed before a request is scored
Quoting a request of this kind against the public catalogue…
Legal
Confidential documents, the strongest reasoning for them, and a person when the model is unsure.
Problem
Legal work is long documents, high stakes and confidentiality. A cheap model misses a clause; a frontier model on every e-mail is wasteful; and the answer a lawyer must not rely on looks exactly like the one they can - unless the router says how sure it was and hands the doubtful ones to a person.
Routing strategy
Length and complexity measured, then Large-context model answers.
Models it routes between
llm-frontiercounsel-review
What changes
Contracts and long filings on large-context, high-reasoning models
Quoting a request of this kind against the public catalogue…
Retail & marketing
Ten thousand product descriptions on a cheap model. The campaign headline on the best one.
Problem
Marketing generates more AI text than anyone else in the company and most of it is bulk: variants, descriptions, translations. Paying frontier prices for all of it is the single easiest saving - and the one piece that must be perfect has to be allowed to cost more.
Routing strategy
Brand-critical flag read from the context, then Bulk copy to the cheap models within budget.
Models it routes between
llm-smallllm-midllm-frontier
What changes
Bulk copy on the cheapest models that meet the quality rules, inside a budget per team
Quoting a request of this kind against the public catalogue…

By what you are trying to do

For the business

What it does for the whole organisation.

Cloud, self-hosted or on-premises
Run it where the data is
The same router ships as a hosted platform, a container image and a Helm chart; the data boundary is a routing rule.
Deployment options
One URL, every vendor
No vendor lock-in
Point your OpenAI-compatible client at one address; add, swap or retire providers in the catalogue without touching application code.
Every vendor side by side
89% less
Lower cost per request
On the 150-prompt benchmark the router spent 89% less than sending every prompt to the reference model, and the judged quality went up, not down.
Read the benchmark

Can we run it inside our own network?: Yes. The router runs as a container or a Helm release on your infrastructure, and a routing rule keeps regulated data on the models you allow.
Will this lock us into another vendor?: No. The router is Apache-2.0 open source, your provider keys stay yours, and the API is OpenAI-compatible, so leaving is a change of base URL.
How much can we save on our LLM bill?: On the public benchmark the router saved 89% against sending everything to the reference model. Import last month's provider bill on the bill page and the estimate is computed on your own traffic before you change a line of code.

Start from the rules closest to your team, then change one line.

OpenSmartRoute is open source and the free plan keeps the full trace of every decision. Every use case above ends with its rules.yaml.

Create a free account All six, side by side

Free plan, no card. Fifteen thousand decisions a month with the full trace.

The same router, a different first rule for every team.

Answer the easy tickets cheaply. Escalate the hard ones before they get worse.

The right model for each coding task, from a one-line fix to a redesign.

Account data never leaves your boundary. Everything else routes on cost.

Route sensitive healthcare workloads to approved models - and take the decision out of every application.

Confidential documents, the strongest reasoning for them, and a person when the model is unsure.

Ten thousand product descriptions on a cheap model. The campaign headline on the best one.

By what you are trying to do

What it does for the whole organisation.

Run it where the data is

No vendor lock-in

Lower cost per request

Start from the rules closest to your team, then change one line.

The same router, a different first rule for every team.

Answer the easy tickets cheaply. Escalate the hard ones before they get worse.

The right model for each coding task, from a one-line fix to a redesign.

Account data never leaves your boundary. Everything else routes on cost.

Route sensitive healthcare workloads to approved models - and take the decision out of every application.

Confidential documents, the strongest reasoning for them, and a person when the model is unsure.

Ten thousand product descriptions on a cheap model. The campaign headline on the best one.

By what you are trying to do

What it does for the whole organisation.

Run it where the data is

No vendor lock-in

Lower cost per request

Start from the rules closest to your team, then change one line.