Routing

get/api/v1/audit

Audit

The workspace's hash-chained decision and outcome records, newest first, across every process's chain (chains); chain_ok is false when any chain is broken and problem names the file. tenant keeps only the records of one tenant slug (a tenant-pinned key always gets its own); before (the ts of the oldest record already seen) pages further back and next_before is the cursor for the following page.

Requires an API key

curl -s "$OSR_URL/api/v1/audit?limit=100" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/evaluations

Evaluations List

The workspace's kept evaluation runs, newest first (summaries; GET /{id} has the report).

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations?limit=50" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/evaluations

Evaluations Run

Route every row of a labelled dataset on this workspace's pool - nothing is executed or metered - and keep the report: accuracy with its interval, quality, price, latency, calibration, the mix, the misroutes and the baselines on the same rows. route scope.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "rows": [] }'

{
  "rows": []
}

get/api/v1/evaluations/datasets

Datasets List

The workspace's kept datasets, most recently changed first (row and label counts; GET /{id} has the rows).

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations/datasets" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/evaluations/datasets

Datasets Create

Save logged requests as an evaluation dataset - the activity page's *Add to dataset*: name request_ids, or a window over the prompts kept under log_payloads (days, limit, outcome good / bad / rated / any, target_id), or explicit rows. A good outcome labels the row with the target that answered. route scope; 409 when nothing matches.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/datasets" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "days": 1, "request_ids": [], "rows": [], "target_id": "llm-frontier" }'

{
  "days": 1,
  "request_ids": [],
  "rows": [],
  "target_id": "llm-frontier"
}

get/api/v1/evaluations/datasets/{dataset_id}

Datasets Get

One dataset with its rows; ?format=jsonl returns the file osr eval reads, ragas / deepeval / langfuse the same rows as a JSONL file in that framework's dataset shape (:data:DATASET_FORMATS).

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations/datasets/<dataset_id>?format=json" \
  -H "Authorization: Bearer $OSR_API_KEY"

delete/api/v1/evaluations/datasets/{dataset_id}

Datasets Delete

Requires an API key

curl -s -X DELETE "$OSR_URL/api/v1/evaluations/datasets/<dataset_id>" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/evaluations/datasets/{dataset_id}/results

Datasets Results

Read the scores another framework computed on this dataset back in - a Ragas result table, DeepEval test results, Langfuse run items, or plain {row | request_id | input, scores, passed} rows (:data:RESULT_FORMATS) - and keep them as an external run next to the router's own; rows that came from logged requests get the mean score as an outcome (source eval:<tool>). route scope; 409 when no row joins.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/datasets/<dataset_id>/results" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "rows": [] }'

{
  "rows": []
}

post/api/v1/evaluations/datasets/{dataset_id}/rows

Datasets Extend

Add rows to a dataset (same body as creation; a logged request already present is not added twice).

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/datasets/<dataset_id>/rows" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "days": 1, "request_ids": [], "rows": [], "target_id": "llm-frontier" }'

{
  "days": 1,
  "request_ids": [],
  "rows": [],
  "target_id": "llm-frontier"
}

post/api/v1/evaluations/datasets/{dataset_id}/run

Datasets Run

Evaluate a kept dataset on today's pool and keep the report (the body carries the run options - objective, constraints, tenant, baselines, frontier). route scope.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/datasets/<dataset_id>/run" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "constraints": {}, "objective": {}, "tenant": "string" }'

{
  "constraints": {},
  "objective": {},
  "tenant": "string"
}

get/api/v1/evaluations/gates

Gates List

The workspace's quality gates with their last verdict, newest first.

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations/gates" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/evaluations/gates

Gates Create

Keep a labelled dataset as a quality gate: it is routed now (the baseline) and again every interval and after every policy change; a run below min_accuracy or more than max_drop under the baseline files a warning in the workspace inbox. Nothing is executed or metered. route scope, admin role.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/gates" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "rows": [] }'

{
  "rows": []
}

delete/api/v1/evaluations/gates/{gate_id}

Gates Delete

Requires an API key

curl -s -X DELETE "$OSR_URL/api/v1/evaluations/gates/<gate_id>" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/evaluations/gates/{gate_id}/run

Gates Run

Route the gate now and judge it (route scope).

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/gates/<gate_id>/run" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/evaluations/judge

Judge Overview

The judge in force (the policy's judge block), what it graded over days per answering target, and the latest verdicts with their reasons.

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations/judge?days=7&limit=50" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/evaluations/judge/run

Judge Run

Grade one kept request now (the activity page's *Judge this answer*): a metered judge call whose verdict becomes the request's outcome. execute scope; 409 when the prompt and answer were not kept.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/judge/run" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "request_id": "7f3c0e2a9b1d4f6e8a0c2b4d6e8f0a1b" }'

{
  "request_id": "7f3c0e2a9b1d4f6e8a0c2b4d6e8f0a1b"
}

post/api/v1/evaluations/replay

Evaluations Replay

Re-route the prompts the workspace kept under its log_payloads policy through today's router and report what would change, what it would cost and how it agrees with the outcomes people reported. 409 when nothing was kept. route scope.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/evaluations/replay" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "objective": {} }'

{
  "objective": {}
}

get/api/v1/evaluations/sample

Evaluations Sample

A labelled sample dataset built from the catalogue, to try a run before you have your own rows.

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations/sample?n=40" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/evaluations/{evaluation_id}

Evaluations Get

Requires an API key

curl -s "$OSR_URL/api/v1/evaluations/<evaluation_id>" \
  -H "Authorization: Bearer $OSR_API_KEY"

delete/api/v1/evaluations/{evaluation_id}

Evaluations Delete

Requires an API key

curl -s -X DELETE "$OSR_URL/api/v1/evaluations/<evaluation_id>" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/feedback

Feedback

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/feedback" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "request_id": "7f3c0e2a9b1d4f6e8a0c2b4d6e8f0a1b", "success": true, "quality": 0.9 }'

{
  "request_id": "7f3c0e2a9b1d4f6e8a0c2b4d6e8f0a1b",
  "success": true,
  "quality": 0.9
}

get/api/v1/handoffs

Handoffs List

The workspace's hand-off tickets, newest first (status filters pending/resolved/cancelled); a tenant-pinned key sees the tickets of its own tenant's requests only.

Requires an API key

curl -s "$OSR_URL/api/v1/handoffs?limit=100" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/handoffs/{handoff_id}

Handoffs Get

Requires an API key

curl -s "$OSR_URL/api/v1/handoffs/<handoff_id>" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/handoffs/{handoff_id}/cancel

Handoffs Cancel

Withdraw a pending ticket; the router learns the hand-off did not deliver.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/handoffs/<handoff_id>/cancel" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/handoffs/{handoff_id}/resolve

Handoffs Resolve

A person answers the ticket; the resolution teaches the router that deferring was right.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/handoffs/<handoff_id>/resolve" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "answer": "string" }'

{
  "answer": "string"
}

post/api/v1/route

Route

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/route" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "objective": { "quality": 1, "cost": 0.3 }, "top_k": 3 }'

{
  "objective": {
    "quality": 1,
    "cost": 0.3
  },
  "top_k": 3
}

get/api/v1/shadow

Read

The running shadow candidate with what it would have changed so far (summary: agreement, spend, quality and latency deltas, the target moves, refusals, a verdict), the most recent comparisons and the earlier candidates. candidate is null when nothing is being shadowed.

Requires an API key

curl -s "$OSR_URL/api/v1/shadow?recent=20" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/shadow

Start

Start shadowing a candidate policy (admin role): objective weights (quality, cost, latency, quality_floor) and / or constraint overrides (the policy's request-shaping keys). A running candidate is stopped. Nothing is executed or charged for the shadow decisions.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/shadow" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "constraints": {}, "objective": {} }'

{
  "constraints": {},
  "objective": {}
}

delete/api/v1/shadow

Stop

Stop the running candidate (admin role); its comparisons stay readable under history.

Requires an API key

curl -s -X DELETE "$OSR_URL/api/v1/shadow" \
  -H "Authorization: Bearer $OSR_API_KEY"

post/api/v1/shadow/promote

Promote

Make the candidate the workspace policy (admin role): its constraint overrides are merged into the policy and its objective weights become the policy's objective (the default for requests that set none). The policy change runs every quality gate the workspace keeps.

Requires an API key

curl -s -X POST "$OSR_URL/api/v1/shadow/promote" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/shadow/{candidate_id}

One

One candidate (running or ended) with its summary and comparisons.

Requires an API key

curl -s "$OSR_URL/api/v1/shadow/<candidate_id>?recent=50" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/stats

Stats

Requires an API key

curl -s "$OSR_URL/api/v1/stats" \
  -H "Authorization: Bearer $OSR_API_KEY"

get/api/v1/targets

Targets

The deployment catalogue plus the workspace's installed marketplace listings (source: catalogue / marketplace), each with whether it can execute for this workspace.

Requires an API key

curl -s "$OSR_URL/api/v1/targets" \
  -H "Authorization: Bearer $OSR_API_KEY"