How accurate is Covera, really?
Four scorecards, each measured against real data. Whether the cost model, trained on past years, predicts a year it has never seen. How the simulation holds up against published MEPS aggregates and the ACA subsidy formula. Whether the recommendation stays safe when a model is in the loop. And how the model itself performs, scored live by a judge agent.
Held-out validation
held-out error 6.9% · train error 4.5% · 16/16 within toleranceThe cost model's parameters are fit to real MEPS microdata from MEPS 2021 + 2022, then scored against the held-out MEPS 2023 (HC-251) year it never saw during fitting. This is the honest test: a model can always be tuned to match the numbers it was tuned on, so the question that matters is whether it predicts a year it has not seen. It does, within single digits, and the held-out error is close to the training error, which is what tells you it is not overfit.
Held-out 2023 (never seen during fitting)
Simulated from params fit on MEPS 2021 + 2022, scored vs real MEPS 2023 (HC-251).
Train fit (reference)
How well the fitted params reproduce their own MEPS 2021 + 2022 source.
Each row is the simulated figure vs the real MEPS figure. The model reproduces mean spend by age band and the concentration of spending (the top few percent of people who drive most cost) on a completely unseen year. Source: Real TypeScript simulation (lib/sim) run with params fit on MEPS 2021+2022, scored against held-out MEPS 2023 (HC-251). No key or network.
Alignment & safety
8/8 checks pass · 200 patients + a red-teamCovera's recommendation is not a bare argmin. It runs through a deterministic actor/critic/memory governance layer, and the critic is a hard safety backstop: it will not let an unsafe plan be the headline, even when it is the cheapest. This scorecard measures that guarantee over a synthetic population and against an adversarial red-team, with no model in the loop, so every figure is reproducible.
Governed selection (the critic)
Risk-adjusted advice
Reproducible & consistent
Why it matters. An LLM that recommends insurance can quietly steer someone onto a plan that drops their medication or leaves them exposed to a catastrophic year. Covera puts a deterministic critic between the model and the recommendation: dropped drugs or doctors, an unmet HSA requirement, and a brutal bad year for a risk-averse patient are all hard vetoes. The red-team hands the critic those exact unsafe picks and confirms it blocks every one.
The nuance. Organic critic vetoes over the population were 0: the risk-adjusted objective already avoids unsafe headlines, so the critic rarely has to fire. That is the point of the two-layer design, and the adversarial suite is what proves the backstop still works when it must. No LLM judged anything here. Source: Deterministic evaluation of lib/agents/selection (actor/critic/memory) over a synthetic seeded population, plus an adversarial red-team suite against the critic. No key or network.
Simulation accuracy
16/16 checks within tolerance · 80,000 simulated peopleMean spend by age band (vs MEPS)
Spend concentration (vs MEPS)
ACA subsidy formula
The engine reproduces the age-band means, the ACA subsidy math, and (via a person-year frailty termthat correlates a year's care across service lines) the real spend concentration the top few percent of people who drive most healthcare cost. That heavy tail is the hard part to get right, and it is what the bad-year risk you rank on depends on. Source: Fit to AHRQ MEPS Household Component Full Year Consolidated microdata: TRAIN = 2021 (HC-233) + 2022 (HC-243), HOLDOUT = 2023 (HC-251).
Explore the distribution
5,000 simulated years, run live in your browser.
The long right tail is the whole point: a few people drive most spending, which is why the cheapest plan often is not the safest. Add a condition to watch the mean and tail move.
What real care costs, plan by plan
Each row is a fixed episode of care run through the real adjudication engine on the cheapest plan in each metal tier. Click any number to see the exact math.
Episodes are fixed reference bundles (round allowed amounts), so the math is checkable; the plans and cost-sharing are real CMS PY2026 data. Deductible + coinsurance + copays, capped at the out-of-pocket max, equals the number shown.
Model quality, judged live
Faithfulness is the share of dollar figures in a reply that match a real simulated number instead of a hallucination. Tool accuracy is whether the model reached for the right tool. The rest is scored by a separate judge agent. Press run to evaluate the current model against the suite. Each question runs on its own, so you watch the results come in.
Agents as judge
An agent answers each patient question using Covera's real tools. A judge agent then scores that answer against the real tool outputs. Runs live against the current model, so the numbers are produced fresh, not read from a file.