Docs for an AI API that admits it’s probabilistic
- Client
- Parse (spec)
- Year
- 2026
- Niche
- AI docs
[ The brief ]
Parse extracts structured data from messy PDFs with an LLM under the hood. AI docs usually oversell certainty. The spec brief: document the model like the honest, slightly unpredictable colleague it is.
[ Craft decisions ]
- →Confidence scores documented as a decision tool, with thresholds and actions.
- →Prompt guidance shows a weak prompt and a strong one; diffs teach faster than rules.
- →Non-determinism addressed head-on: what varies, what doesn’t, how to test around it.
[ The sample ]
Working with confidence scores
Every extracted field returns a confidence between 0 and 1. This is not decoration. It’s the API telling you how to route the result. In production systems we recommend three lanes: ≥0.95 auto-accept, 0.80-0.95 human review queue, <0.80 reject and re-request with a hint (below). Teams that skip the middle lane end up rebuilding it after their first invoice with a handwritten total.
Hints: prompting Parse like you mean it
Weak hint: “extract the total.” Parse will find a total, possibly the subtotal, possibly the previous balance. Strong hint: “extract the final amount due after tax, usually the largest currency value in the bottom third of the last page.” Location, relationship, and disambiguation: that’s the whole craft.
{
"fields": [{
"name": "amount_due",
"hint": "final amount due after tax; bottom third of last page",
"type": "currency"
}]
}What’s deterministic and what isn’t
Identical documents with identical hints return identical values at temperature 0, but confidence scores may drift ±0.02 between model versions. Pin a model version in production (model: parse-2026-05), test against our golden-document set before upgrading, and never assert exact confidence values in your test suite. Assert the lane, not the number.
[ Read it like an editor ]
“Assert the lane, not the number” is the kind of sentence support teams frame. AI docs earn trust by documenting uncertainty as a feature with a workflow, not a footnote.