embedding
Against a hosted API

Docs/Against a hosted API

Against a hosted API

The three things a hosted decision API does not sell you, drawn on capability rather than accuracy, and why we will not claim the accuracy win.


The honest version of this page is shorter than the marketing version would be, and it starts with what we are not claiming.

We do not claim to be more accurate#

We run comparisons against hosted decision models and publish them either way. On one of our own guard benchmarks, a hosted competitor caught more attacks than our fitted head did: 79.1% against 73.0%, at a lower false refusal rate.

Accuracy is a per-task, per-corpus, per-base-rate question. Anybody who answers it in general is selling something, and a claim like that collapses the first time a prospect runs their own rows. So the comparison below is drawn entirely on capability: things that either exist in an API or do not, checkable against anybody's published documentation rather than against a benchmark we ran.

What is actually different#

a hosted decision APIEmbedding Studio
coverage guaranteenone. a score, with no stated relationship to how often it is rightconformal, at a risk level you set
can answer nothingno. it returns its best option whatever happensyes. an empty answer your code cannot act on
precision targetnot in the APIset it, and get the review rate that holds it
review rate before you shipdiscovered in productionmeasured on held-out rows
fitted to your labelsno. your data is not used to adapt the modela head fitted on your own decided cases
the artefactan endpoint you renta calibration file you download and keep
where records gotheir serversours, or yours, or never leaving your building

The one that is structural#

Most rows above are features that a hosted API could add next quarter if it wanted to. One of them is different in kind.

The exported calibration is not a feature a hosted-only product is missing. It is a thing it cannot be, because the business depends on the decision happening on their machine. A vendor whose revenue is per-call cannot hand you a file that keeps working after you stop calling.

Note

What you still pay us for.

Fitting, measuring, the held-out numbers, the drift watch and the hosted endpoint. Not the right to keep deciding with a calibration you already exported.

When a hosted API is the right answer#

Stated plainly, because a comparison page that cannot name its own weaknesses is an advertisement.

If you have no labelled cases at all and no way to get any, most of what is above does not apply to you: there is nothing to fit and nothing to calibrate against. You can still run cold off written criteria, but you will be choosing on raw quality, and that is a fair fight you should hold us to rather than take our word on.

If your volumes are tiny, the measurement machinery is overhead. A few dozen decisions a week does not need a review band.

And if the task is genuinely generative, summarising, drafting, extracting a free-form field, this is the wrong tool entirely and no amount of calibration fixes that.