Can we run it inside our own network?

Yes. The router runs as a container or a Helm release on your infrastructure, and a routing rule keeps regulated data on the models you allow.

Will this lock us into another vendor?

No. The router is Apache-2.0 open source, your provider keys stay yours, and the API is OpenAI-compatible, so leaving is a change of base URL.

How much can we save on our LLM bill?

On the public benchmark the router saved 89% against sending everything to the reference model. Import last month's provider bill on the bill page and the estimate is computed on your own traffic before you change a line of code.

Financial services · use case

Account data never leaves your boundary. Everything else routes on cost.

A policy, not a promise: requests carrying account or customer data are pinned to the model you run yourself, statements and extractions go to a mid-tier model, and a hash-chained audit trail records why each request went where - with budgets per team enforced before a call is made.

Try OpenSmartRoute free Talk to sales about a pilot

Free plan, no card. Self-hosting is free forever; the rules below run the same way on both.

The situation

What the team is dealing with today

A bank's AI traffic mixes public questions with requests that carry account numbers, balances and names. Model risk, data residency and the regulator all ask the same thing - where did this request go, and why - and an application-level setting cannot answer it.

What changes

Requests with account data pinned to the on-prem model by a rule no score can override
Extraction and statement work on a mid-tier model at a fraction of frontier cost
A hash-chained audit trail of every decision for model-risk and compliance review

One request

What happens, step by step

Requests with account data pin to your on-prem model; the audit trail satisfies the regulator.

Account data detected
Cloud models removed from the candidates
On-prem model answers
Decision hash-chained in the audit trail

Route a request like this in the playground and read the trace

A financial services request, quoted live“Explain to the account holder why the transfer of 4,200 EUR from account IBAN DE89 3704 0044 0532 0130 00 was held for review, in plain language.”

Quoting every candidate in the public catalogue...

The policy

Two text files, not a rewrite

This is the rules.yaml that produces the behaviour above; targets.yaml names the models, agents and people it refers to. pin restricts the candidates instead of nudging them; the hash-chained audit records why.

rules.yaml - financial services (reference configuration)yaml

rules:
  - name: account-data-stays-onprem
    when:
      domains: [finance]
      contains_pii: true
    prefer: [llm-onprem]
    pin: true

  - name: statements-are-extraction
    when:
      domains: [finance]
      actions: [extract]
    prefer: [llm-mid]
    weight: 0.6

Example: Illustrative configuration or synthetic data, not a customer's.A reference configuration to start from - rules the router enforces as written, not a description of how a named customer runs it.

Other use cases

For the business

What it does for the whole organisation.

Cloud, self-hosted or on-premises
Run it where the data is
The same router ships as a hosted platform, a container image and a Helm chart; the data boundary is a routing rule.
Deployment options
One URL, every vendor
No vendor lock-in
Point your OpenAI-compatible client at one address; add, swap or retire providers in the catalogue without touching application code.
Every vendor side by side
89% less
Lower cost per request
On the 150-prompt benchmark the router spent 89% less than sending every prompt to the reference model, and the judged quality went up, not down.
Read the benchmark

Can we run it inside our own network?: Yes. The router runs as a container or a Helm release on your infrastructure, and a routing rule keeps regulated data on the models you allow.
Will this lock us into another vendor?: No. The router is Apache-2.0 open source, your provider keys stay yours, and the API is OpenAI-compatible, so leaving is a change of base URL.
How much can we save on our LLM bill?: On the public benchmark the router saved 89% against sending everything to the reference model. Import last month's provider bill on the bill page and the estimate is computed on your own traffic before you change a line of code.

Start with the financial services rules and change what you disagree with.

OpenSmartRoute is open source and the free plan keeps the full trace of every decision. Create a workspace, paste the rules, route a request.

Create a free account Read the benchmark

Free plan, no card. Fifteen thousand decisions a month with the full trace.

rules: - name: account-data-stays-onprem when: domains: [finance] contains_pii: true prefer: [llm-onprem] pin: true - name: statements-are-extraction when: domains: [finance] actions: [extract] prefer: [llm-mid] weight: 0.6

Account data never leaves your boundary. Everything else routes on cost.

What the team is dealing with today

What happens, step by step

Two text files, not a rewrite

What it does for the whole organisation.

Run it where the data is

No vendor lock-in

Lower cost per request

Start with the financial services rules and change what you disagree with.

Account data never leaves your boundary. Everything else routes on cost.

What the team is dealing with today

What happens, step by step

Two text files, not a rewrite

What it does for the whole organisation.

Run it where the data is

No vendor lock-in

Lower cost per request

Start with the financial services rules and change what you disagree with.