Through OrcaRouter the OrcaCyber Ultra 1.0 benchmarks section is empty. Not sparse — empty. The page carries a tagline, four descriptive sections, five FAQs, a complete pricing block and live latency statistics, and no scored evaluation of any kind [OURS]. Its cheaper siblings each publish four to six rows.
A buyer arriving from a comparison table has nothing to put in it. That is a problem to solve, not a reason to leave.
Three reasons a page ships without scores
Legitimate explanations exist, and they are worth separating from excuses.
The evaluation is not finished. Security benchmarks that mean anything take weeks to run properly, and a card released 2026-10-17 is young.
The scores are worse than the siblings’. A flagship that scored below the tier beneath it would be awkward to publish. This is the cynical reading and it is not unreasonable.
The evaluation does not fit the model’s purpose. Agentic security work depends on tooling a public harness may not provide.
The page does not say which applies, and that is the honest complaint: absence of evidence, not evidence of absence.
What you can still compare
Three properties are measurable today, and they are enough to place the card.
| Property | Ultra 1.0 | Zero 1.5 | Flash 1.0 |
| Input / M | $5.00 | $3.00 | $0.80 |
| Output / M | $10.00 | $7.50 | $1.60 |
| Context | 1,048,576 | 1,000,000 | 1,048,576 |
| Output ceiling | 131,072 | 128,000 | 393,216 |
| Throughput (7d) | 42.2 tok/s | 114.5 tok/s | 138.8 tok/s |
| Traffic (7d) | 6.01B | 30.63B | 21.97B |
| Published rows | 0 | 4 | 6 |
Read down the throughput column: the flagship is the slowest by a factor of two and a half. Read down the traffic column: it is the least used by a factor of three to five. Neither is damning — flagships are expensive and deep — but both are evidence, and they are the only evidence available.
The evaluation you can run instead
Without vendor rows, the only measurement is yours, and it does not take long.
Twenty cases from your backlog: ten you solved, ten you did not.
Three models: this one and the two cheaper tiers, all on the same key.
Four columns: found the issue, explained the chain, artifact usable, wall-clock time.
Cost for sixty runs at roughly 300,000 input and 20,000 output each: $1.70 per flagship run, $1.05 on the middle tier, $0.27 on the cheap one — about $60 for the whole comparison [OURS]. That is an afternoon and a budget line nobody needs to approve.
The output is not a score out of a hundred. It is a ranking on your own work, which is the only ranking that will ever matter.
What the empty table costs you in practice
Two real costs, beyond the inconvenience.
You cannot justify the flagship on merit. A procurement conversation that would normally cite a benchmark row now cites throughput and context window, which are weaker arguments.
You cannot detect a regression. Without a baseline, a new checkpoint that is worse will look the same as one that is better.
The second is the bigger problem for a long-lived pipeline. The mitigation is to build the baseline yourself — twenty cases, dated, with the model string recorded — so that the next release is measurable even though this one was not.
The case for buying it anyway
Three situations where the missing table does not matter.
Your workload is unlike any benchmark. If your cases are proprietary and the public harnesses do not resemble them, vendor rows would not have predicted your outcome anyway.
The decision hinges on the context window. A million tokens with structured output is a specific capability, and no benchmark row describes it.
You will evaluate regardless. A team that tests before deploying loses little from an absent vendor table.
For everyone else, the empty section is a genuine gap, and the cheapest response is to fill it yourself.
The table your procurement will ask for
An empty vendor section does not remove the requirement for a comparison; it moves the obligation to you. The table below is the one worth building, using only facts that exist today.
| Criterion | Source | Outcome |
| Context window sufficient | model page | yes / no |
| Output ceiling sufficient | model page | yes / no |
| Modalities cover the evidence | model page | yes / no |
| Blended price at your volume | arithmetic | value |
| Throughput compatible with consumer | measured telemetry | yes / no |
| Reliability adequate | measured telemetry | value |
| Quality on your cases | your pilot | value |
Six of seven rows are fillable within an hour. The seventh is the pilot, and on a card with no published evaluation it carries the whole decision.
That asymmetry — one row deciding everything — is uncomfortable and correct. It is also a reason to run the pilot properly rather than treating it as a formality.
The takeaway
No scores is not the same as bad scores, but it is the same as no evidence — and a flagship that asks for five dollars per million of input should expect the question.
Ask it, and answer it with twenty of your own cases. That is a better instrument than a table would have been, and it is the only one that will still be valid next month.
A closing thought on pleading: it is tempting to fill an empty benchmark section with adjacent evidence — the base model’s scores, the sibling’s rows, an article about the family. Resist it. A reader who is told “the sibling scores 88.1” will assume the flagship is at least that good, and the page does not say so.
The honest summary of this card’s evidence is two sentences long: it has a million-token window, and six billion tokens of weekly traffic at a 0.295% error rate. Everything else is inference, and inference is what the pilot is for.
The pilot is also the artefact that answers the question a year from now. When someone asks why the team chose one card over another, “we tested twenty cases and here is the file” is a better answer than any benchmark table would have been — including a full one.
And one practical note for anyone building the pilot: include at least two cases the cheaper tier got wrong. A pilot built only from successes measures whether the flagship agrees with what already worked, which is not the question anyone is asking.
Sourcing note: the absence of benchmark rows, the pricing, context, ceilings and throughput are from the OrcaRouter model page and production telemetry for the seven days ending 2026-10-26 [OURS]. The sibling rows referenced are those cards’ own published claims [VENDOR]. No benchmark figure is asserted for this model, because none is published.
