Report · end-to-end cost
Phase 7B — Live JEV cost validation
Phase 7B reran the four provider/route arms with live JEV scoring inside each Evidence Layer arm: 160 answers, 400 JEV scores, 80 paired judgments.
| Measure | OpenAI | DeepSeek |
|---|---|---|
| Mean total tokens per answer | 510.35 → 384.05 −24.75% | 798.125 → 567.25 −28.93% |
| Estimated total API cost per 40-case arm | $0.070700 → $0.061188 −13.45% | $0.029040 → $0.025995 −10.49% |
| Mean end-to-end latency | 2,329 → 3,005 ms +676 ms | 4,144 → 4,159 ms +16 ms |
| Evidence coverage | 1.0000 → 0.9792 | 1.0000 → 0.9792 |
| Mean correctness / 2 | 2.00 → 2.00 | 1.95 → 2.00 |
| Paired total-cost interval | Reduction interval excludes zero | Includes zero; not conclusive on this set |
No general answer-quality improvement is established. Coverage falls by 0.0208 in both providers. Costs are estimated from provider usage and published rates; they are not invoices.
Limitations
- Exploratory automated judgments, not human review.
- The OpenAI judge overlaps with a tested provider.
- The exposed split is not blind.
- Cost and latency values must not be pooled with Phase 6 or Phase 7.