AI-generated image: White hexagons forming dispersing beam ending in bright magenta glow on dark background
Image generated by AI

Defending an LLM API Against Unbounded Consumption

Rate limiting by request count does not bound how much an LLM API can cost you. It only feels like it does, right up until one request comes in with 200,000 tokens of context and burns through what would normally take a thousand small requests to spend.

Unbounded Consumption rose four places in OWASP’s 2026 Top 10 for LLM Applications, published in early August. It is the plain version of a familiar problem: nobody breaks in, nobody steals a key, they just make your API spend money, one legitimate-looking request at a time. A denial-of-wallet situation does not need to be malicious either. A buggy client retrying a huge prompt in a loop does the same damage as an attacker who found the gap on purpose, and from the API’s side the two look almost identical: valid credentials, well-formed requests, a bill that keeps climbing.

Most API security thinking is built around requests that fail, an injected payload, a malformed input, an auth bypass. Unbounded Consumption is different because the request usually succeeds exactly as designed. The defense has to live at the layer of “how much did this cost,” not “was this request valid,” which is a different set of checks than most teams already have in place.

The trap with request counting

“100 requests per minute per API key” sounds like a rate limit until you remember that a request is not a fixed unit of cost. Gemini 3.1 Pro alone accepts up to 1,048,576 input tokens in a single call. A hundred small requests and one enormous one can carry the same request count and wildly different bills. If your limiter counts requests, an attacker, or a bug, only needs one very large request to get past it.

The unit that actually maps to cost is tokens, not requests. Everything below follows from that one substitution, and the three layers cover different moments in a request’s lifecycle on purpose: before the call is accepted, before the call generates output, and after the call is billed. One layer alone leaves a gap the other two would have caught.

Layer one: a per-caller token budget

Track tokens spent per caller over a rolling window, not requests per caller.

s f t u r n u c c t t w t b i i r t o o i i u f f e k k n o d t C e e d n g n b u a n n o e o b b u r r l s _ w c t w u u d a n l _ l _ h ( d d g i e u i r e = ) g g e s b r s m e c e e t e u B e i s k g > t t . d u d t e _ e . . t B g d : : t b t b t w o u e g _ u _ u o i k d t e i i a d b d k n e g t n n t g u g e d n e : t t : e d e n o s t t g t s w _ E t ( e . _ _ u x i c t w u r s c m a ( i s e e e e l c n e s d e s l a d d e d # # t e l o t + e a r l w = _ d r p m , e _ a r E o e p r r 0 t e r l r r ) e q r l e s = u o i c q e e r n a u t n s g l e _ o t , l s a w . e t t ( e c r ) : ) s u , : t r + i r p m e e W a n r I t t N e w D d w i O _ i n W t n d o d o k o w e w n s > b u d g e t . t o k e n _ l i m i t :

estimated_tokens has to include the input the caller is about to send, counted before the call goes out, not after. A budget check that only looks backward at what has already been spent will always let the request that busts the budget through, because it has not been charged yet at the moment you check it. Counting tokens before the call is cheap, most SDKs expose a tokenizer for exactly this, and it is the only place in the whole system where you can reject an oversized request without having already paid for it.

Layer two: an output cap enforced server side

A max token count on the response is usually a client-side parameter, max_tokens in the request body. That is a setting the client sets, and a compromised or careless client can set it to anything, or omit it entirely. The cap that actually matters is the one your own server enforces regardless of what the caller asked for.

f u n c t s e r r i e f e e o r f s t n v e p u e c o r c r t n n a _ i s l m v e r l a e e _ x _ = s m _ m p o o a m o d u x o n e t d s l p = e e ( u l c t m . a i c l = n a l ( l e p r l r o e ( , l q r i u e r c e q e y s u q _ t e u m . s e a m t s x a , t _ x ) o _ m : u t a t o x p k _ u e t t n o _ s k f , e o n r s s ( e = c r e a v f l e f l r e e _ c r m t ) a i x v _ e o _ u m t a p x u ) t )

The client’s max_tokens is a request. Your server’s policy_max_output_for is the number that actually bounds the bill, because it is the one value the caller cannot override. policy_max_output_for does not have to be one fixed number either, a free-tier caller and a paid one can carry different caps, but the important property is that it is decided by your code, keyed on who the caller is, and never simply trusted from the request body.

Layer three: a circuit breaker on spend rate

Error-rate circuit breakers are common, and they miss this failure mode entirely, because a denial-of-wallet request usually succeeds. It returns a normal response, it is just enormous. Trip the breaker on spend rate instead of error rate.

s f t u r n u c c s s t t t i t p p i r r f e e o a a S n n n c c t p d d k k r p a e _ _ r e e a a l n l t e r r c u e d a h c . k s r T s r o = s e e t r t e r p r _ _ a _ s d g e . c o c m h _ e n s a p k i o a t d p l s e n l n _ _ e l ( r u d d s l n e c : t : _ p a d r a e c e s _ ( l : f h n t l c l l e d _ a a e f o c _ m s l r l a k t i t l , o t ( r n _ e a c a u m r t t a c t i , r l k e n a l e u d c e r + t u k r ( = e r e , c a r a a > t . a l c i s c l t t o p t e u r n e u r a a = n a ) l c C d l _ k O _ _ c e O l c o r L a o s . D s s t s O t t p W _ ) e N m : n ) i d n _ u t t h e r ) e s h o l d :

This layer runs after the call, on the actual measured cost, and it exists to stop the second enormous request, not the first. That distinction matters for what comes next. A cooldown of a few minutes is usually enough to stop a runaway retry loop without permanently locking out a caller who had one legitimately large request in an otherwise normal pattern, and alert_ops is what turns a quiet cost spike into something a person actually looks at the same day instead of the same month.

What this does not fix

Per-request cost tracking, the thing that feeds the circuit breaker above, tells you what you spent after the request already finished. It cannot stop the one request that started the fire, because by the time you know its actual cost, the model has already generated the tokens and the bill has already happened. The thing that stops that first request is the output cap in layer two, enforced before the tokens get generated, not a spend tracker that notices afterward.

Smart Inference tracks the actual provider cost on every request that passes through it, which is the measured number layer three needs, and it gives you that number as a real figure per request instead of an estimate you have to trust. It still will not stop the first oversized request from generating, because no cost-tracking layer can undo tokens after the model has already produced them. That part is the server-side cap, and it has to sit in front of the call, in code you own, checked before the request goes out rather than reconciled after the invoice arrives.