BYOK cost accounting explained: honest arithmetic, no fabrication

BYOK makes LLM cost accounting honest because the tenant can cross-check the gateway's numbers against the provider dashboard. The arithmetic, versioned pricing catalog, expensive-model gates, and the five questions to ask a vendor.

BYOK — bring-your-own-key — is the architectural choice that makes LLM cost accounting honest. This post explains what that means at the arithmetic level, why it matters, and what to check to verify a vendor's BYOK claim is real.

The two architectures

Reseller: the gateway operator holds the provider (OpenAI, Anthropic, etc.) keys. Your application talks to the gateway; the gateway pays the provider. You pay the gateway on your own invoice, and the gateway's price is provider price + margin.

BYOK: you hold your own provider keys. You register those keys in the gateway. Your application talks to the gateway; the gateway uses your keys to talk to the provider. Tokens bill to your provider account. The gateway charges you for the gateway service (typically a flat fee or per-request), not for the tokens.

Both models are legitimate. They are different products with different economics and different failure modes.

Why BYOK makes cost accounting honest

In BYOK, the number you see on your provider dashboard should match the number the gateway reports, within a small drift window. That is a checkable invariant. If the gateway says "you spent $1,247 on OpenAI last month" and your OpenAI dashboard says "$1,247 ± small drift," the gateway's numbers are trustworthy.

In reseller, the provider dashboard is the vendor's, not yours. You cannot cross-check. The gateway's cost number is authoritative by definition; whether it is accurate is a matter of trust.

The arithmetic

Real cost per call in BYOK is derived deterministically:

cost_usd = (tokens_in × price_in_per_million / 1_000_000)
         + (tokens_out × price_out_per_million / 1_000_000)

Where tokens_in and tokens_out come from the provider's usage response, and price_in_per_million and price_out_per_million come from a versioned pricing catalog maintained by the gateway.

The critical property: if the provider reports total_tokens = 0, the cost is 0. If usage is missing entirely, the row is marked missing. Nothing is fabricated.

What versioned pricing means

Provider prices change. When OpenAI cuts the price of a model, the change should apply from the effective date forward — not retroactively. When a new model launches, its price appears in the catalog on its launch date.

The gateway stores a table of the form:

model         | effective_from | effective_to | in_per_M | out_per_M
openai/gpt-4o | 2026-01-01     | 2026-04-14   | 5.00     | 15.00
openai/gpt-4o | 2026-04-15     | (open)       | 4.00     | 12.00

Every usage row is priced using the row that was active at the call's timestamp. Historical spend numbers do not silently shift when a price changes.

What "expensive model" gates look like

Some models are so much more costly than their siblings that a bare enable is a footgun. The pattern:

Cheap defaults (gpt-4o-mini, claude-haiku-4.5) are auto-seeded in new projects.
Mid-tier models (gpt-4o, claude-sonnet-4.5) require explicit enable.
Expensive models (o1-pro, claude-opus-4.5) return HTTP 403 with a typed error until an admin explicitly enables and acknowledges the price sheet.

This is not paternalism; it is the same design principle as requiring an explicit typed DROP TABLE in a database — the destructive operation should be hard to invoke by accident.

Model ids with the @key suffix

A workspace may want to route the same model through multiple provider keys (e.g., a primary OpenAI account and a fallback OpenAI account under a different billing entity). The @key suffix disambiguates:

openai/gpt-4o-mini@primary
openai/gpt-4o-mini@fallback

The gateway routes each to its designated BYOK key. Cost accounting attributes to the right account.

Verifying a vendor's BYOK claim

Ask the vendor these questions:

Do the tokens bill to my provider account? (If yes, ask them to show you a live request landing in the provider dashboard.)
What does the gateway charge me, and is that number independent of the token cost?
Can I rotate my provider key at any time without gateway involvement?
Can I revoke my provider key at the provider, breaking gateway traffic, without any dependency on the gateway operator?
Are your metering numbers reconcilable against my provider dashboard within a small drift?

All five should be "yes" for a real BYOK. If any answer is "well, actually…" you have a reseller wearing BYOK clothes.

Why we shipped BYOK

ThinkNEO ships BYOK because it makes the enforcement plane's cost decision anchored to a number the tenant can verify. Enforcement is only meaningful if the meter is honest. The meter is only honest if it is checkable. It is only checkable if the tenant sees both sides.