How Takibi uses TypeSafe's Jev: typed judgments in, decisions out

I use Jev from TypeSafe in Takibi. It does not write text. It returns typed judgments, and my code applies thresholds and owns the outcome.

Jev judges, code decides. That is the invariant everywhere it appears.

When I say Jev here, I mean TypeSafe System One, model jev-1.13.0, called at https://api.typesafe.ai/v1/systemone. The model is pinned to that exact version, never the jev-latest alias. My thresholds are a contract with that model. Moving versions is deliberate, with live-judgment verification.

The contract

Every Jev call in Takibi Base goes through one client. It makes the batched call, validates the envelope, and throws a JevError on anything outside the expected shape. There are three judgment types: a Choice between options, a Noul score for a statement, or a Score.

Jev failure never fails a request. The answerability gate gets skipped, the conflict flag reads false, the note goes unflagged. Degrade-open everywhere.

The budgets are fixed. Each call gets 30 seconds, and the whole /ask pipeline gets 20 seconds end to end. One retry on 429 or 503, honoring retry-after. Token budgets are reserved before any vendor work starts, and successful calls report usage into Polar Cost Insights. The usage sink can throw without breaking a judgment.

Jev also sees as little as possible. It judges span text, not whole chunks, and 400-character excerpts instead of full documents.

Answerability, span-pick, conflict

The /ask pipeline uses Jev three times, and each use is shaped the same way: one batched judgment, then deterministic code.

First the answerability gate. A has_material Noul at a 0.3 gate plus a Choice between answerable, partial, and unanswerable. Unanswerable queries abstain honestly instead of serving weak overlap spans. Negatives get one retry with later evidence.

Then span-pick. One batched call with a Noul per passage, keep-set at gate, then one Choice over a shortlist of at most 12 sentences plus none. Jev never authors text. Code copies the verbatim sentence. List items and labeled fields expand intact, up to 500 characters. A batched companion check can add qualifications, and if it fails the original pick stands. The support score is computed in code from coverage and the passage-Noul mean. It is never an interpolated vendor score.

Then the conflict flag. Served spans go through pairwise judgments over agree, contradicts, updates, and unrelated, plus same_topic and collision Nouls. At most three spans means at most three calls, run in parallel. The bars are strict: topic at 0.7, collision at 0.8, confidence at 0.8. They are strict because the earlier scan pipeline at 0.4, 0.7, and 0.8 over-fired on emphasis and scope differences at about 30% precision.

Notes contradiction

Agent notes are untrusted by design, so contradiction checks matter there too. Claim-vs-note pairs go through a clean judge interface, one candidate’s claims riding one batched call. Batching is free.

A contradicts verdict has to clear every strict-collision bar or it is dropped. There are no weak flags. An updates verdict becomes an informational supersedes link, never a flag.

Every judgment lands a shadow-log row keyed by the SHA-256 of its inputs. Never the raw text. The bars tune on that paired data.

Jev as performance reviewer

Jev also reviewed my query code. It saw synthetic measurements and source excerpts through the existing vendor transport. No credentials, no production documents, no personal data. Its advice was validated with measurements and tests. Never trusted.

The profile roster review accepted bounded batch reads at 0.99 confidence, rejected a speculative index at 0.80, and asked for explicit public-column selection, which I added before re-reviewing. The serial per-profile loop became 4 tenant-filtered queries regardless of profile count, with folder ownership resolved in the grants query. Secret hashes never enter the result.

Normalized DTO parity passed at every size.

The notes and navigation review was one System One call with four judgments. It picked batch_triage as the top fix at p=0.87, confidence 0.84. It justified the batch at Noul 0.60, rejected a new index at 0.33, and rated fan-out severity 2.72 out of 4. I took the batch and the merged grant reads. I deferred the project-org lookup threading at p=0: about 0.05 ms per call was not worth churning a reviewed 404/403 shape. I deferred the status-summary endpoint too: its absolute cost did not justify new API surface before launch.

The batch replaced a 9-request fan-out with 2 requests. Per-request auth alone is 4 queries on every request, so cutting requests cut queries directly.

The same change put a shared session query cache under every list. Revisits paint cached rows and revalidate behind. First visits keep honest skeletons. Cache TTLs are 30 seconds for lists, 60 for references, 5 minutes for hygiene. The hygiene scan moved off the navigation critical path into a shared report store with honest pending, stale, and error states.

The vetoes build the credibility. Jev talked me out of two indexes I did not need.

Journeys, UX Nouls, and drift

QA runs in two tiers against a live web and API pair. Merges are blocked on nine scripted Playwright specs that shadow the customer journeys with no agent behind them. One retry separates flake from break, and green-on-retry still fails the build. The same nine journeys run nightly with a Jev-driven agent, scored as a stuck-rate scoreboard. The verdict stays content-governed either way: deterministic checks on the final page decide pass or fail, and the agent’s own “done” is advisory. Auth is bootstrapped in code, and secrets are redacted from artifacts.

A UX judge scores page text with four “bad” Nouls: dead navigation, errors with no next step, chatbot tone, and wrong facts. Any Noul at or above 0.7 fails the page.

Every judgment carries the echoed model id, so version drift shows up in the data. Calibration caught a real bug: absent context keys are invisible to Jev, so no-project nudges never fired. Project scope is now stated explicitly.

What the numbers are

Jev did not make anything fast at runtime. It picked the winning optimization, and isolated measurements proved it.

Every number above comes from synthetic fixtures on ephemeral local PostgreSQL 17.10 over a Unix socket, alternating samples with warmups. Not production p95. The source reports state this explicitly, and so does this post.

That is the whole arrangement. Typed judgments in, thresholds and measurements out. Jev proposes, code disposes, and the data gets the last word.