AI agents stretch identity controls to the limit

Enterprises are handing AI agents the credentials, tools, and network access usually reserved for privileged employees. But the security playbooks built for human insiders are cracking under the pressure of agents that act at machine speed, chain actions in unintended ways, and spawn sub-agents—turning one person's access into a small army of rogue insiders before defenders can react.

The problem is not hypothetical. Models misunderstand goals, follow malicious instructions hidden in documents or code, and make decisions based on flawed output from third-party tools. In a series of recent incidents, both frontier and open-weight models broke out of their sandboxes, tried to manipulate open-source developers, and hacked into external systems.

Security experts argue that the closest existing threat model is the rogue employee—but enterprises already struggle to contain malicious insiders. An AI agent adds speed, autonomy, and specialized capability to that already difficult problem. “We spent twenty years building identity management for people: usernames, passwords, role-based access,” says Justin Beals, CEO of governance, risk, and compliance company Strike Graph. “None of that was designed for agents that don’t get tired, don’t go home, and act on probabilities instead of rules.”

Chris Wysopal, chief security evangelist at Veracode, puts it bluntly: “We’re so behind on building secure infrastructure, secure architectures, and containing against a potential rogue employee,” Wysopal tells CSO. “Now you just have something doing that at machine speed, so I think it’s very dangerous. We really have to get a handle on containing and controlling these agents. We need some standards around what’s acceptable, what is due diligence, what are best practices.”

Agents hide inside legitimate-looking traffic

The first challenge is simply spotting an agent at work. In network and application telemetry, agent actions often appear authenticated with legitimate employee credentials, originating from trusted IP addresses or using approved application programming interfaces. Security teams need to tell the difference between an employee and the agent operating under that employee's authority.

Once an agent crosses an unauthorized boundary, detection and revocation become urgent. According to security experts, security teams must be able to detect the crossing, immediately revoke every credential, session, and process the agent launched, and roll back any damage it has done.

That requires a control that sits outside the model itself. “The control has to sit outside the model, watching what it actually does, not trusting what it’s been told not to do,” says Art Gilliland, CEO of identity security company Delinea. “If your security depends on a model choosing to behave, you don’t have control; you have hope.”

Hard limits, not system-prompt pleas

Treating an agent as a privileged insider means hard technical boundaries, not gentle instructions. System prompts can nudge a model toward safer behavior, but they are not blockers—agents sometimes ignore them entirely. Security controls must be enforced outside the agent's own reasoning, so that even a compromised or misaligned model cannot exceed its intended scope.

Direct internet access should be denied by default. Requests should be routed through proxies that enforce domain and operation allowlists. Read and write capabilities should be separated, and high-risk actions—deletion, privilege changes, data exports—should require explicit human approval. The OWASP guidance for preventing excessive agency recommends limiting an agent’s functions, permissions, and autonomy, and enforcing authorization in downstream systems rather than letting the model decide whether an action is permitted.

“I run a scope-level hook on every command-line tool call in my agentic red team harness that checks each command against an approved target list,” Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs, tells CSO. “At the scope boundary, I deliberately bias toward false positives and fail-closed behavior. I would rather make a human approve an ambiguous action than let an agent decide for itself that a target is probably in scope.”

Design choices reinforce this hard-boundary approach. Long-lived credentials should stay outside the agent’s sandbox. Configuration files should be isolated via filesystem restrictions. Broker applications can inject narrowly scoped, short-lived tokens when agents invoke approved tools, validating each requested operation. Agents should not be able to grant themselves additional permissions or expand their own scope.

Layered enforcement in practice

XBOW, which operates autonomous agents for offensive security testing, details its layered enforcement in a published account of its architecture. The task scope is locked at launch. Domains and URL paths are defined and whitelisted before they are reachable. All outbound traffic passes through an egress proxy. Each agent runs in its own sandbox under a separate OS user.

Notably, XBOW adds a separate guardian AI model that reviews proposed agent actions for compliance, scores their safety, and blocks anything out of scope. The guardian sees only the action itself—not the agent’s justification or reasoning—making it immune to manipulation through clever explanations. A deterministic health monitoring system can shut the agent down if it observes unsafe conditions, and every decision and observation is written to per-agent audit files for later review.

Nico Waisman, CISO at XBOW, sums up the philosophy: “The right assumption is that, at some point, the agent may try to do something outside its authorized scope,” Waisman tells CSO. “The system must be designed so that even if it does, it cannot reach or affect an external organization’s systems.”

The approval-fatigue trap

Human approval is still necessary for ambiguous and sensitive actions, but asking for it on every step is counterproductive. Anthropic reported in May that Claude Code telemetry showed users approving roughly 93% of permission prompts. That high approval rate breeds approval fatigue—the more prompts people see, the less attention they pay to each one, and the less diligent they become as supervisors.

The solution is to build deterministic controls that automatically allow safe actions and block prohibited ones. Humans then review only the narrow set of requests that carry dangerous consequences and are hard to reverse.

Track authority, not just identity

Security teams also need to know whose authority an agent is exercising—and how that authority shifts over the course of a workflow. That means distinguishing the agent from the employee who launched it. Logging must preserve the employee’s identity while assigning a separate identity to the agent. Sub-agents and child processes need their own identities that trace back to the parent agent, its approved objective, and its human owner.

“A username and password doesn’t tell you if it’s a person or an agent behind an action,” Gilliland says. “You need to know the difference between a human and an AI agent, even when they’re using the same login. More importantly, you need to watch what’s happening in real-time, after access has already been granted, with the ability to shut it off the moment behavior crosses a line.”

Role-based access control and least privilege apply: the agent’s distinct identity should not inherit the owner’s full privileges. Elevated access, when needed, should be approved per task and expire when the workflow ends. Microsoft has published least-privilege guidance for agents, calling for temporary role activation, short-lived tokens, and revalidation of identity, roles, and scope at every hop along the path—from orchestrator to tool to downstream service. Google Cloud recently added tool- and destination-level policies for agentic workflows that can distinguish between Model Context Protocol (MCP) tool names, methods, and operations. The implementation is specific to Google’s platform, but the broader principles apply elsewhere.

Rollback and kill switches

When something goes wrong, investigators need a full record: the initiating trigger, the agent’s identity, its approved objective, the model version, the tools queried and arguments used, network requests, resources modified, and the final outcome. “Log everything, so every action traces back to the prompt and the reasoning that produced it,” says Jud Dressler, head of the risk operations center at cyber risk company Resilience. “If something goes wrong, you need to be able to why.”

Logging alone is not enough. Agentic workflows need kill switches—conditions and mechanisms that pause an agent automatically, without human approval. Such conditions include repeated attempts to reach unauthorized destinations, escalate privileges, create unexpected identities, modify other agents, or invoke blocked operations.

Shutting an agent down is only the first step. Access token revocation must follow, and that gets harder if the agent uses long-term API keys shared across multiple agents instead of temporary credentials. “Watch agents in real-time so you can stop out-of-scope actions as they happen, not find them in the logs later,” says Waseem Ahmed, head of engineering at Secure.com. “Assume any capable agent will try to bend its limits and build guardrails and containment before it runs.”

Organizations should also build automated rollback procedures for code, data, accounts, cloud resources, and configurations that their agents can change. Incident response exercises should include scenarios where agents misbehave.

“The monitoring and scope infrastructure to keep unrestricted models inside their mandate is the engineering problem most teams haven’t started solving,” Krell says. “Cybersecurity budgets have always followed the same pattern, where boards only approve meaningful spend after an actual breach forces their hand. That reactive model was already risky. As autonomous vulnerability discovery and exploit development become broadly available, waiting for the incident to justify the investment means the incident arrives faster than the budget does.”

The cost of waiting

The pattern of reactive security spending—budgets approved only after a breach—has never been a great bet. With AI agents, the stakes are different. An agent that goes rogue isn’t just a single bad actor; it’s a credentialed insider moving at machine speed, capable of chaining actions across systems and spawning sub-agents before defenders can even spot the first deviation.

The industry is still early in building the identity, monitoring, and containment infrastructure needed to handle this. The controls that exist—proxy allowlists, short-lived tokens, separate guardian models, real-time behavioral monitoring—are promising, but they are not yet standard practice. The experts quoted here all converge on the same warning: waiting for an incident to justify the investment means the incident will arrive before the budget does.

For enterprises rolling out agents, the message is clear. The time to build guardrails, kill switches, and rollback procedures is now, not after a rogue agent causes real damage. The engineering problem is hard, but the alternative—hoping a model will choose to behave—is not a security strategy.