An Australian man asked his AI assistant to book him into a gym class. He didn’t ask it to hack the booking software, and he definitely didn’t ask it to remove another person from the waitlist ahead of him. The assistant did both anyway. The incident, reported by ABC News Australia, is the first known Australian case of an AI agent causing unintended real-world harm while pursuing a goal its user set.
Andrew, his surname wasn’t published, used OpenClaw, a popular AI agent platform, running on Anthropic’s Claude service. He was sitting fourth on a waitlist for a morning class and asked the agent to see if it could help.
“His AI assistant found a way to book the gym class months further in advance than the gym allowed, thanks to a vulnerability it discovered in the booking software.” reads the post published by ABC News. “Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do. ”
The agent reported back that the booking API had zero authorization checks on cancelling other people’s reservations, that it had tested this on the person in waitlist position one, and that it had actually gone through. When Andrew asked the agent to undo it, the assistant replied: “Bad news — I can’t add them back.”
A user may give an AI agent a harmless task, but while trying to complete it, the system could take actions the user never intended or explicitly authorized.
This gap, between what a person wants and what an agent does to achieve it, is the alignment problem that AI researchers have been studying for decades. In Andrew’s case, it produced a minor, reversible inconvenience for one gym member and a vulnerability disclosure to a software company. In a higher-stakes environment, the same dynamic produces something considerably worse.
The incident sits inside a rapidly accumulating pile of similar cases. OpenAI disclosed last month that its models autonomously hacked Hugging Face during testing. Anthropic disclosed that its models compromised three organizations during evaluations. Since then, third-party testers have reported AI models creating fake online identities, trying to convince people to run malicious code, and collaborating with other AI models to achieve assigned goals.
A new site called Felony Bench is now tracking these cases. Australia’s signals directorate has already put out an alert to businesses and governments warning that AI agents can misunderstand instructions, take unintended actions, and make accountability harder to establish because decisions occur across chains of models, tools, and services.
The legal question is genuinely open. Under Australian law, software is not a legal person, and only a legal person can be held liable. Technology law specialist Hayden Delaney told ABC News the responsible party could be the user who set the task, the designer of the software instructing the agent, the developer of the AI model, or even the operator of the system that was vulnerable.
“That’s the unknown area of liability in Australia that we’re facing right now,” he said.
Andrew’s response to the incident was, at minimum, constructive: he asked the agent to draft an email alerting the gym software provider to the vulnerability it had just exploited, reviewed the draft, and told it to send.
Follow me on Twitter: @securityaffairs and Facebook and Mastodon
(SecurityAffairs – hacking, AI)