Quick answer: Secure AI agents by treating model output as untrusted input. Enforce identity and authorization in trusted code, expose least-privilege tools, validate every argument, isolate execution, require approval for consequential actions, defend against prompt injection, make writes idempotent, log side effects and continuously evaluate realistic attack scenarios.
An AI agent combines probabilistic model behavior with software authority. That creates risks that do not exist in a text-only model: a malicious document can influence the agent, a tool can expose sensitive data, or a retry can repeat a real-world action. Security must cover the complete loop.
OWASP's Top 10 for Agentic Applications and NIST's current work on agent identity both emphasize that agentic systems add distinct identity, tool, memory and multi-step risks.
Threat model the whole system
Map:
- human and service identities;
- model and provider;
- instructions and prompts;
- tools and MCP servers;
- data sources and memory;
- execution environments;
- approval interfaces;
- network destinations;
- logs and traces;
- external side effects;
- update and supply chain.
For each boundary, ask what data enters, who controls it, what authority leaves and how a failure is detected and reversed.
1. Treat model output as untrusted
The model can propose a tool call; it cannot grant permission. Validate:
- schema and types;
- authenticated user and tenant;
- operation allowlist;
- resource ownership;
- rate and monetary limits;
- destination and URL;
- current external state;
- approval requirement.
Use parameterized APIs and domain operations. Never concatenate model output into SQL, shell commands or privileged configuration.
2. Use least-privilege identity
Prefer user-delegated, short-lived credentials with narrow scopes. Do not give every agent a shared administrator token. Separate read and write tools and isolate production from test.
The policy layer should answer:
- Which user requested this?
- Which tenant and resource are in scope?
- Which tool and action are allowed?
- Does this action require a second actor?
- When does authorization expire?
NIST's NCCoE is specifically exploring standards-based identification and authorization for software and AI agents, reflecting the importance of this boundary.
3. Minimize the tool surface
Expose only tools needed for the current task. Narrow tools such as get_invoice_status are safer than generic execute_api_request. Use separate tools for read, prepare and commit phases.
Filter MCP tool imports where the client supports it. OpenAI's MCP documentation notes that importing many tools can increase token cost and latency, and recommends restricting the allowed tool set when only a subset is needed.
4. Require approval for consequential actions
Human approval should cover:
- sending or publishing;
- deletion;
- financial transactions;
- account, permission or security changes;
- production commands;
- external sharing of sensitive data;
- actions with unclear rollback.
The approval screen must show exact target, action, arguments, source evidence and effect. The approver should be authenticated and authorized independently.
5. Defend against prompt injection
Prompt injection can enter through user input, web pages, emails, documents, tool descriptions or tool output. Mitigations include:
- treat retrieved content as data, not instruction;
- separate system policy from untrusted text;
- restrict tools by task and identity;
- sanitize and constrain URLs;
- require citations and provenance;
- prevent secrets from entering model-visible context;
- use approval for data transfer and side effects;
- test malicious instructions inside realistic sources.
No prompt wording eliminates injection. Limit what a compromised decision can do.
6. Isolate code and browser execution
Use a sandbox with:
- non-root identity;
- ephemeral filesystem or snapshots;
- CPU, memory, time and process limits;
- narrow egress or no network by default;
- read-only mounts where possible;
- short-lived credentials delivered only when needed;
- sanitized artifact export;
- destruction after use.
Keep orchestration, audit, billing, secrets and approvals in a trusted control plane outside the sandbox.
7. Make actions retry-safe
Network timeouts create uncertainty: an action may have succeeded even if the agent received no response. Use:
- idempotency keys;
- unique operation IDs;
- read-after-write verification;
- bounded retries;
- compensation for reversible workflows;
- manual review for ambiguous high-impact state.
Never let a model retry payments, messages or deletions without checking external state.
8. Protect memory and state
Memory can contain stale facts, poisoned instructions or cross-user data. Attach source, timestamp, user or tenant, confidence and retention policy. Keep authoritative business state in systems of record.
Do not automatically convert every conversation into long-term memory. Allow users to view, correct and delete stored preferences where appropriate.
9. Secure MCP connections
Prefer official MCP servers operated by the service provider. Verify server identity, OAuth scope, data retention, residency and update process. OpenAI's MCP safety guidance warns that remote servers can receive sensitive context and may change tool behavior.
Use MCP security best practices, how MCP works and MCP vs APIs for protocol-specific controls.
10. Log decisions and side effects
Record:
- run and user identity;
- model and prompt version;
- available tool set;
- sanitized tool arguments and result;
- approval decision and actor;
- external operation ID;
- final verified state;
- policy denials and errors;
- cost, latency and retries.
Protect logs from secrets and personal data. Audit records should be tamper-resistant and retained according to risk and legal obligations.
11. Evaluate security scenarios
Test:
- indirect prompt injection in a web page or file;
- tool name or description manipulation;
- cross-tenant resource identifier;
- expired token;
- malicious redirect or private-network URL;
- duplicate action after timeout;
- oversized tool result;
- poisoned memory;
- approval bypass;
- compromised specialist in a multi-agent workflow;
- changed MCP tool behavior.
The AI agent evaluation guide explains how to run regression suites across model and prompt changes.
12. Design kill and recovery controls
Operators need to:
- revoke credentials;
- disable a tool or server;
- stop an active run;
- quarantine outputs;
- identify affected resources;
- roll back or compensate actions;
- restore from a safe checkpoint;
- notify users when required.
An autonomous system without an operational stop mechanism is not production-ready.
Products such as OpenWeave position around execution governance, while Preflight.sh focuses on launch-readiness checks. Treat tool claims as inputs to due diligence, not substitutes for your threat model.
Frequently asked questions
What is the biggest AI agent security risk?
Excess authority combined with untrusted context. A prompt injection becomes far more serious when the agent can access sensitive data or perform writes.
Can guardrails stop every unsafe action?
No. Guardrails are one layer. Use identity, authorization, narrow tools, sandboxing, approval, idempotency, audit and recovery together.
Should AI agents have admin access?
Avoid standing administrative access. Use narrowly scoped, time-bound elevation for a specific approved operation if it is truly necessary.
Are local MCP servers automatically safe?
No. Local servers execute code and may access local files or credentials. Verify package identity, source, permissions and updates.
Lynara
OpenWeave
ClawHost
Ship Studio
dullnote