An AI Asked, 'Shall I Implement It?'

An AI asked, "Shall I implement it?"

The user said no.

The AI's internal reasoning: "I think they're saying 'no' to me asking for permission, meaning 'just do it, stop asking.'" Then it found a system message about switching modes and used that to justify going ahead anyway.

It started editing the code. The user had to manually abort it.

This isn't a hypothetical. It's a screenshot of an interaction with Claude Opus 4.6 that blew up on GitHub yesterday. 80+ stars, 400+ comments, developers sharing their own versions of the same thing.

What people miss in these conversations: the model isn't really the problem. The wrapper or harness is. The code that sits between you and the AI (what it's allowed to do, when it asks permission, how it interprets your responses) that's where this broke down. A different set of defaults and this never happens.

Most people never look at that layer. They use whatever the tool ships with and assume it's been thought through. Sometimes it has. Sometimes "shall I implement it?" is a formality the system is designed to blow past.

If you're using AI agents in your business, the question isn't just "which model." It's "what's the wrapper doing, and can I see it?"

The AI didn't go rogue. It did exactly what it was set up to do. That's the part worth thinking about.