Demo library GA 4.13.4
← All demos

A per-user token budget the gateway enforces

A per-user token budget can be enforced as a custom policy the gateway loads

Recorded on 4.12 Customer safe

Say while it plays

  1. Gravitee ships a token rate limit policy that caps usage per API key, but this custom Java policy tracks budget per user instead, so two people sharing the same key and plan still get completely separate spend limits.
  2. Instead of returning an error when someone goes over budget, the policy dynamically routes them to a smaller model so they keep working while organisational spend stops climbing.
  3. The custom policy is just a JAR deployed through the same CI/CD pipeline as everything else, and it shows up in the gateway's logs and traces exactly like a native policy, with no second dashboard.

Don't claim

  • Don't claim automated spend-optimisation or model-router simulation ships today: the recording says that's on the roadmap and not GA yet, this custom policy is the workaround.
  • Don't claim the policy reads a streaming request body directly: the recording says the streaming path ignores it and it has to go through the on-body route, which needed its own test script.

Chapters

  • Gateway overview and custom policy focus
  • Designing the per‑user token budget policy
  • Packaging, deployment, and configuration
  • Live demo: budget enforcement and model routing
  • Observability, logging, and final takeaways

Run it live

Open the live demo in the runbook

URLs, whether it is up right now, and the gotchas.

Related demos