AI runtime enforcement checklist for enterprises (audit-ready)

A 36-item audit-ready checklist for enterprise AI runtime enforcement. Control-plane presence, identity, budget, quota, model scope, content policy, audit ledger, change management, BYOK hygiene, observability, governance-of-governance, sunset.

This is an audit-ready checklist for enterprises adopting AI runtime enforcement. Each item is either satisfied or it is not — no partial credit. Use it as a gap analysis or as an RFP requirements list.

1. Control-plane presence

[ ] Every LLM call from every application in your organization traverses a single enforcement plane. Applications that go direct-to-provider are known, listed, and scheduled for migration.
[ ] The enforcement plane is deployed with high availability (≥2 replicas across zones). Its outage does not silently degrade to direct-to-provider.
[ ] The enforcement plane's own control channel (admin API) is authenticated and audited separately from the data channel.

2. Identity and attribution

[ ] Every call is authenticated with a key that is scoped to a specific project (not a shared root key).
[ ] Keys can be rotated new-first (new key issued and adopted before old key is revoked) with zero downtime.
[ ] Every metered event records which key, which project, which workspace, which model, and which end user (via user field or a documented header) made the call.
[ ] Revoked keys stop working within a bounded, published time window (typically ≤60 seconds).

3. Budget and cost

[ ] Every workspace has a monthly USD budget with a documented enforcement mode (monitor or enforce).
[ ] Cost budget enforcement is fail-CLOSED: a stuck check blocks the call, does not let it through.
[ ] Cost is derived arithmetically from real tokens × versioned provider pricing. No fabricated tokens. No hidden markups. Provider dashboards match gateway metering.
[ ] Budget warnings fire at 50%, 75%, 90% of the period budget, via a documented notification channel.
[ ] Budget overrides for exceptional workloads require documented approval and are themselves audit rows.

4. Quota and rate limiting

[ ] Every workspace has requests-per-minute and tokens-per-minute limits appropriate for its tier.
[ ] Quota enforcement is fail-NEUTRAL: a stuck check degrades to monitor mode, does not block live user traffic.
[ ] Quota exhaustion returns HTTP 429 with Retry-After and a machine-readable rule id.

5. Model scope

[ ] Every project has an explicit allow-list of models. Requests for un-enabled models are rejected with a typed error.
[ ] Model ids are provider-qualified (openai/gpt-4o-mini, not gpt-4o-mini) to prevent silent provider swaps.
[ ] Deprecated or expensive models can be disabled organization-wide without redeploying application code.

6. Content policy (if applicable)

[ ] Prompt-side inspection for PII with a documented redaction or rejection policy.
[ ] Response-side inspection for jailbreak or unsafe-content markers with a documented action.
[ ] Content policy decisions are logged with the same rule-id discipline as enforcement decisions.

7. Audit ledger

[ ] Every enforcement decision (allow, deny, redirect) produces a row in an append-only, tamper-evident ledger.
[ ] Ledger rows include: timestamp (UTC, high resolution), key id, project, workspace, model, tokens in/out, cost USD, rule id, outcome, trace id.
[ ] Ledger is exportable to your existing SIEM in a standard format (JSON lines, OpenTelemetry, or FOCUS).
[ ] Ledger retention meets your compliance requirement (usually ≥12 months, often ≥7 years).
[ ] Ledger reads are separate credentials from ledger writes; nobody with a write credential can delete history.

8. Change management

[ ] Every policy change is a versioned, reviewable artefact.
[ ] The active policy version at any timestamp is recoverable from the audit ledger.
[ ] Emergency policy changes (a "break-glass" tighten) require documented approval and are annotated as such in the ledger.
[ ] Policy changes propagate to the enforcement plane in bounded time (≤60 seconds) with published SLA.

9. Provider key hygiene (BYOK)

[ ] Provider keys are stored encrypted at rest with a documented key-management chain.
[ ] Provider keys are never returned by any API to any caller after the initial registration.
[ ] Provider key rotation is a documented one-click operation.
[ ] Compromised provider keys can be revoked at the gateway without an OpenAI/Anthropic-side rotation (the gateway is a defense-in-depth layer).

10. Observability integration

[ ] Enforcement decision events export to your metrics platform (Prometheus, Datadog, Grafana) at the same resolution as the request rate.
[ ] A dashboard shows, at minimum: requests per minute, denied per minute (by rule id), budget consumption per workspace, and latency percentiles.
[ ] Alerts fire on: budget over 90%, quota denials spike, gateway error rate >1%, gateway latency p95 >100 ms overhead.

11. Governance-of-governance

[ ] The team that operates the enforcement plane is separate from the team that sets policy.
[ ] Policy changes require review by at least one person who did not author the change.
[ ] There is a documented incident-response playbook for: enforcement-plane outage, policy misconfiguration, key compromise, and audit-ledger integrity failure.

12. Sunset and rollback

[ ] A rollback path exists for any policy change (immediately, without waiting for the change author).
[ ] A path exists to safely disable enforcement (fall back to monitor) if the plane is misbehaving without cutting off all AI traffic.
[ ] Sunsetting a workspace or project preserves the audit ledger; deletion is soft and audited.

Scoring

Under 20 items: you have observability, not enforcement. This is not a moral judgement — it is the state of most enterprises in 2025 and half of them in 2026. Move to phase 2.

20–30 items: you have enforcement in production but ownership is unclear or one of the failure modes (audit ledger integrity, key rotation) is a known gap. Fix the gaps you can point to on this list.

All 36+ items: you have an audit-ready runtime enforcement program. Bring the checklist to your next SOC 2 or ISO 42001 review.