Benchmark · v1

Premium-grade accuracy,
at a fraction of the cost.

A reproducible, cross-vendor benchmark of DataLayer Nexus cost-aware routing on mixed-difficulty math reasoning. Real metered costs, programmatic scoring — no cherry-picking.

88.0%

Nexus accuracy (vs 92.0% premium)

46% cheaper

Nexus cost/question vs premium

+11 pts

Nexus accuracy vs the cheap model

dataset: GSM8K (easy) + GSM-Hard (hard) · n=100 (50 easy + 50 hard) · seed 42 · total spend $1.32 · OpenRouter (exact usage cost)

The result

Every config runs the identical task set. Accuracy is exact-match; cost is the real amount billed.

Configuration	Accuracy	Easy	Hard	Total cost	Cost / q	Perf / $
Direct · gpt-4o-mini (cheap)	77.0%	92.0%	62.0%	$0.014	$0.00014	32.74×
Plan-Execute · gpt-5.5 → deepseek	87.0%	94.0%	80.0%	$0.490	$0.00490	1.03×
Direct · gpt-5.5 (premium)	92.0%	96.0%	88.0%	$0.533	$0.00533	1.00×
Nexus (routed)	88.0%	94.0%	82.0%	$0.288	$0.00288	1.77×

Perf/$ normalized to premium (gpt-5.5 = 1.00×). Nexus routing mix: direct-cheap: 60, premium: 40.

The efficiency frontier

Up and to the left is better: higher accuracy, lower cost. Nexus sits where the premium model's accuracy meets the cheap model's price.

Where routing earns its keep

Accuracy split by difficulty. The cheap model collapses on hard problems; Nexus escalates only those, so it holds accuracy on both.

How it works

Nexus rates each query's difficulty with a cheap classifier, then routes:

Easy queries → a small, cheap model (near-free)
Hard queries → a frontier model (only when it's warranted)
Result: premium-level accuracy without paying premium prices on every call

Read honestly. This is a v1 result on math reasoning. Accuracy differences within a few points at n=100 are inside the confidence interval — the headline is the cost: Nexus reaches premium-grade accuracy for 46% less. The cheap model is cheaper still, but loses 11 points of accuracy it can't recover on hard problems.

Methodology