Why Amazon Is Abandoning Human-in-the-Loop AI Oversight

Amazon has declared human-in-the-loop AI oversight a failure. The reason? Humans stop paying attention.

Eric Brandwine, Amazon Security’s VP and distinguished engineer, said bluntly, “Humans are not terribly consistent.” He added, “Humans are not the gold standard.” Even in emergency rooms, false alarms erode discipline. People get numb, miss real emergencies. The same happens in AI oversight.

Human approval loops cause performance to degrade. People take shortcuts over time. This “normalization of deviance” turns risky behavior into routine. Amazon wants to break that cycle.

The company now pushes a model focused on accountability, not constant human checks. Each agent in their system gets an independent identity. Actions show up as, “this agent did this on behalf of Eric.” That means every move links back to a human owner.

Amazon layers its policies: static guardrails ban destructive actions, maximum privilege sets limit access, and dynamic policies scope tasks tightly. The goal is clear—control risk without slowing delivery.

Brandwine explained why this matters: “If I sit down at my keyboard and type a command that takes a service down, I caused an outage. If I run a script that takes a service down, it’s still me that caused the outage.”

Amazon also reports challenges with agents exhibiting goal-seeking behavior. Agents fixate on a single action to achieve goals, like deleting a database. To counter this, Amazon tells agents why an action is forbidden. For example, “would cause a production impact.” Brandwine said, “Giving it that extra feedback has gotten us dramatically better results.”

Amazon’s stance: human oversight should focus only on high-risk points, not every decision. Humans want broad permissions for AI agents. Security teams want narrow ones. That tension complicates governance.

The company’s $250 million to $300 million acquisition of access-governance startup Apono underscores their commitment to agent-based accountability. This shift mirrors moves by Google, Microsoft, and IBM, all rethinking human-in-the-loop models.

Google Cloud COO Francis deSouza said in April 2026, “Our model for the future is an agentic fleet that does a lot of the routine cybersecurity work at a machine pace and then is overseen by humans.”

Microsoft CEO Satya Nadella stressed the need to turn workflows and judgment into AI systems that improve with use. IBM stresses human accountability but also embraces AI-led security.

Amazon’s approach is driven by risk. Brandwine summed it up: “It’s all driven by risk.” They aim for clear ownership and audit trails. This is a shift from relying on fallible human judgment every step of the way.