AI agents are forcing enterprises to rethink security governance, human accountability and oversight as deployments scale.
AWS’s Reimagine 2026 argues that organizations need to build governance into their systems and keep humans accountable for outcomes. The findings come from confidential interviews of 45 to 60 minutes with 154 executives at 128 organizations in 23 industries, conducted over nine months.
A number of the organizations interviewed still apply review processes designed for six-month IT programs to work that takes days. If a two-week experiment waits a month for approval, some teams stop asking for permission. Policy in that situation is “pushing it underground,” the authors write.
One interviewed leader said that CIOs who spent years managing shadow IT are dealing with shadow AI at ten times the scale.
Security, privacy and data leakage are among the technical risks the authors identify. Organizations fear that proprietary data, customer personal information or confidential business intelligence will escape through AI systems, sometimes invisibly and often through third-party tools employees use without oversight.
A Strand Partners survey of European businesses, commissioned by AWS and cited in the document, found that more than half of SMEs and large enterprises use AI. Only 24% have a documented approach to responsible AI use, and 10% have a data governance strategy.
Chris Sedore, VP of Information Services and Technology and CIO at Boston University, estimates that 40 to 50 percent of people at the university use AI at least weekly. “Some of it in models and systems we provide, some of it going rogue,” he said.
When AI reads employee emails
Emails, chats and meeting notes make up what Reimagine 2026 calls the “human layer” of data, and the authors see value in mining it with AI. They note that accessing it raises privacy concerns.
At Houston Methodist Main Campus, a doctor challenged CEO and Chief Innovation Officer Roberta Schwartz, assuming an AI project meant leadership was reading his messages. That wasn’t the case. “I can barely get through my day with my own emails,” Schwartz said, recalling the exchange.
Staff needed a year of experience with the system before they trusted that it surfaced organizational patterns and left personal content alone. The principle that emerged from the interviews is that AI looks at group-level patterns and leaves individuals out.
Where salary data, HR decisions or personal communications were involved, organizations redacted them before processing so that “AI receives the signal without the identity.” Being open with employees about how their data was used was key to building trust.
Writing the rules into the system
Some organizations sort AI projects by risk before they start. Rafael Cavalcanti, Chief Data Officer and Director of Data and AI at Bradesco, built a classification tree that asks whether a use case involves personal data, whether it runs live or in batches, and whether a human must stay in the loop. Each combination of answers maps to a risk level and a set of controls.
For AI agents, the authors advise starting with human approval and expanding autonomy only after the agent shows it is reliable, while keeping the option to narrow it again. “Treat it like probation for a new hire,” they write. Security limits should be set outside the agent, since agents “can misinterpret or work around embedded rules.”
Regulatory differences between jurisdictions complicate things further. “When you’re in 22 countries, you don’t want a paradigm where 20 of the countries are impacted because two of the countries don’t allow certain things,” noted Duncan Macdonald, CTO of Cloud Enablement at Standard Bank.
Wide use, thin results
Usage figures say little about results. One organization observed early in its AI journey appeared 88 percent “adopted,” but produced better work in fewer than one in 5,000 sessions.
Saved time also needs a plan. “If I’m able to do four hours of work in one hour, if there is no structure for how the three hours that have been saved will be used, then there’s no benefit to the company,” stated Dr. Rashed Iqbal, CTO at RAK IDO.
The junior talent gap remains open. AI removes the repetitive work that used to build judgment in junior employees, and while organizations are experimenting with fixes, none of those interviewed claimed an answer.