Embedding Studio fits a decision model to cases your team already decided, measures what its answers are worth on rows nothing was fitted on, and serves it behind an API. The thing it gives you that a confidence score does not is a promise you can state: the correct answer is in what comes back about 9 times in 10, where the 9 is yours to choose.
These docs cover what the product is for and how to reason about it. The request and response reference lives in the product's own documentation, which is where it stays current.
One decision, start to finish#
No project needed: the questions ride in the request, or a template name does.
curl -X POST $API/v1/decide \
-H "X-API-Key: $STUDIO_KEY" \
-H "content-type: application/json" \
-d '{"template": "support-routing",
"text": "I was charged twice, please refund one."}'A project is the same call with a slug in the path, POST /v1/projects/<slug>/decide, and it is what makes the answers yours: fitted on your labels, measured, replayable, audited.
{"answers": {"team": {"choice": "billing", "confidence": 0.94,
"prediction_set": ["billing"], "abstain": false}},
"route": "auto",
"audit": {"engine_sha256": "78f96be7...", "calibration_version": "aafbddab7e6f"}}The same surface drives from a terminal or a coding agent: npm install -g embedding-studio-cli, JSON out, non-zero exit on refusal. See Quick start and the CLI reference.
route is the field to branch on. It is auto only when every question cleared its band; anything else is review, because a record is something a person picks up. Build your control flow on that and you never invent a threshold.
The three things worth understanding#
- A score is not a probability. Raw confidence is ranked right and scaled wrong. Calibration corrects it: ECE 0.160 to 0.069 across six corpora, accuracy unchanged.
- The answer is allowed to be empty. When nothing clears a threshold measured on your own held-back cases, nothing comes back: escalation by policy, not by whoever remembered the
if. - The calibration is a file you keep. It runs on your own hardware through the open source engine, offline, no account. Cancel and it still works.
What it refuses to do#
- Serve a head that lost. A fitted head that fails to beat the engine on cross-validation is not served, and the screen says so.
- Approximate an unreachable target. Everything goes to review, and it names the best target your data supports.
- Load an edited calibration. The checksum covers everything that changes an answer, so a hand-edited file stops matching its version and is refused.
- Claim accuracy for unlabelled files. You get the answer mix and the confidence spread, which is what there is.
What it will not do#
No prose, no summaries, no conversation. If the answer is not one of a set you can write down in advance, this is the wrong tool. The three answer types are the whole surface:
| type | you define | you get back |
|---|---|---|
| choice | named options with one-line definitions | the winning option, plus a probability per option |
| boolean | a statement | the probability that it holds |
| score | an ordered list of level descriptions | an integer level, plus the expected level |
Note
You do not need labelled data to start.
The engine answers from written criteria alone. Labels buy two things later: dozens make the probabilities honest on your traffic, hundreds make the answers match your policy. Neither trains the language model. See Fit and calibrate.
Where to go next#
To understand what you are buying, read Coverage and abstain and then The review band. To decide whether it beats what you use now, the honest version is in Against a hosted API, including the benchmark where a hosted competitor beat us.
Embedding Studio
- Fit and calibrate · The two things one upload buys, what each costs in labelled rows, and why the file is cut 60 / 20 / 20 before anything is measured.
- Coverage and abstain · Where the threshold comes from, worked through on real numbers, and what it means when a decision comes back empty.
- The review band · Set the precision you need and find out what share of traffic can be automated at it, with a lower bound rather than a point estimate.
- The calibration is yours · Download the calibration as a file and run the same decisions on your own hardware with the open source engine. No account, no network, no expiry.
- Against a hosted API · The three things a hosted decision API does not sell you, drawn on capability rather than accuracy, and why we will not claim the accuracy win.
The engine
- Installation · The CLI in one line on any OS, and pip install vegaml for the engine.
- Quick start · Five CLI commands from a CSV to a served, calibrated API. Measured: 60 rows in 63 seconds.
- Questions and answers · The three answer types, how to write criteria that actually steer a decision, and every field on an answer including the one you must gate on.
- Readers and ports · One engine, any frozen reader. What a port is, how to pick a reader, and why the engine bytes are identical behind all of them.
- Long context and images · 73,728 tokens, prefix caching that makes twelve questions about one contract cost roughly one read, and decide_image for pixels.
- Fitting a head · A few hundred labelled rows turn a general reader into your policy, without training the language model. The modes, the knobs, and the two traps.
- Calibration · Refit the probabilities on your own outcomes. What temperature scaling and the conformal threshold each buy you, and when calibrating makes things worse.
- Attach your own reader · Point vegaml.attach at any instruction-tuned LLM and a file of your own text. No labels, no fine-tuning, no retraining of the engine.
- Evals on your labels · vegaml-eval against your own labelled decisions, what each number in the report means, and which fix each bad number points at.
Integrations
- Overview · Seven surfaces around one engine. Each is a single install extra and none of them changes a decision.
- Command line · Decisions from the shell, typed flags instead of JSON, and a JSONL batch that pays for one model load.
- HTTP server · vegaml-serve: POST /v1/decide returns exactly what decide() returns. Endpoints, errors, and how to scale it.
- MCP server · decide() as a tool your agent can gate on, over stdio. Two tools, one client entry, and the instruction to give the agent.
- LangChain and LangGraph · Decide inside a chain, route a graph on the winning option, or gate an input and raise instead of rewriting it.
- Pydantic models · A pydantic model as the question sheet and the validated answer, with every calibrated field still there beside it.
- Docker · The HTTP server as a container, with the checkpoint cached in a volume so a restart does not re-download it.
- ONNX and MLX · Export the settling computation and run it without torch: onnxruntime anywhere, MLX on Apple silicon.
Recipes
- Prompt-injection guard · A gateway guard as one question sheet, the code to put it in front of your model, and the measured catch rate at a real base rate.
- Cascade routing · Answer everything with the small reader and pay for the big one only on what the small one cannot settle.
Reference
- Benchmarks · Every published number, the split it came from, and the caveats that belong next to it.
- Troubleshooting · The errors we have actually hit, what each one means, and the fix. Out of memory, 401 from the Hub, a head that serves raw probabilities.