AI Agent Architecture Explained: A Step-by-Step Guide to Tools, Memory, and Human Approval

Overview

An AI agent is an application that uses a language model to interpret a goal, decide what action to take, call tools, inspect results, and continue until it reaches a stopping condition. The model is only one component. A reliable agent also needs an orchestration layer, access controls, persistent state, validation, observability, and a clear way to stop.

A useful mental model is a controlled loop:

1. Receive: Accept a user request and identify the task, constraints, and required permissions.
2. Plan: Decide whether the task can be answered directly or requires one or more tools.
3. Act: Select a permitted tool and provide arguments that match its schema.
4. Observe: Record the tool result, errors, and relevant state changes.
5. Evaluate: Determine whether the goal is complete, whether another action is needed, or whether a person must review the next step.
6. Stop: Return a result, request clarification, escalate, or fail safely.

This loop may be implemented with a framework, a provider's tool-calling API, or application code written directly around a model API. The architecture matters more than the framework name. A framework can simplify routing and state management, but it does not automatically make tools safe or outputs correct.

Keep the first version narrow. A support triage agent with two read-only tools is easier to test than a general-purpose agent with access to email, payments, file deletion, and production systems. If the application needs external knowledge, separate retrieval from action. A RAG tutorial can help with the retrieval portion, while this architecture governs how the agent decides what to do with the retrieved information.

For structured tool calls and predictable handoffs, define explicit schemas and validate every argument. The guidance in this structured outputs guide is useful when an agent must produce machine-readable decisions rather than free-form text.

Checklist by scenario

Scenario 1: A read-only question-answering agent

This is the simplest useful agent pattern. It might search an approved knowledge base, retrieve documents, and compose an answer without changing external systems.

• Define the allowed data sources and the maximum retrieval scope.
• Give the search tool a narrow schema, such as query, filters, and result limit.
• Return source identifiers or document references with each result.
• Instruct the agent to distinguish retrieved information from assumptions.
• Set a limit on search iterations and total retrieved content.
• Log the query, selected documents, final answer, and any refusal or escalation.
• Test empty results, conflicting documents, malformed content, and prompt injection inside retrieved text.

Do not treat retrieved text as a trusted instruction. Documents are data, not system-level authority. For a deeper defensive checklist, see the prompt injection defense guide.

Scenario 2: An agent that performs low-risk updates

Examples include creating a draft ticket, updating a noncritical field, or preparing a calendar proposal. The agent can act, but the application should keep a clear boundary between preparing an action and committing it.

• Separate read tools, draft tools, and commit tools.
• Require the agent to show a concise action summary before a commit.
• Validate target identifiers and user authorization outside the model.
• Use idempotency keys or equivalent safeguards so a retry does not create duplicates.
• Record who requested the action, what the model proposed, and what the tool actually changed.
• Return a stable result such as created, updated, rejected, or needs approval.

A model should not be the final authority for access control. The application or service receiving the request must enforce permissions independently.

Scenario 3: A multi-step research or operations agent

Multi-step workflows need explicit state. Store the task identifier, current phase, completed actions, tool results, unresolved questions, and stopping reason. Avoid relying on the conversation transcript as the only database.

• Represent each step with a status such as pending, running, succeeded, failed, or cancelled.
• Give every tool call a unique correlation identifier.
• Set maximum steps, time, tokens, and cost according to the workflow.
• Persist checkpoints before and after side effects.
• Make retries aware of whether a side effect already occurred.
• Provide cancellation and timeout handling.
• Escalate when the agent encounters ambiguity, conflicting data, or an action outside its policy.

A simple state record might contain task_id, user_id, goal, current_step, approved_actions, tool_history, pending_questions, and status. Keep sensitive values out of prompts where possible, and restrict which state fields the model can read or modify.

Scenario 4: An agent with human approval

Human approval is most useful at a defined risk boundary, not as an informal instruction to “ask when unsure.” Identify the exact event that pauses the workflow: sending an external message, changing customer data, publishing content, spending money, or modifying an infrastructure resource.

• Show the proposed action, target, arguments, expected effect, and relevant evidence.
• Make approval specific to that action and its current state.
• Expire approvals when the underlying data or arguments change.
• Record the approver, timestamp, decision, and reason when provided.
• Allow rejection, editing, cancellation, and reassignment.
• Never interpret silence, a timeout, or an ambiguous response as approval.

The approval interface should be understandable without exposing the entire internal chain of reasoning. A concise, auditable action summary is usually more useful than an unfiltered model transcript.

What to double-check

Tool design and permissions

Each tool should do one well-defined job. Use descriptive names, explicit argument types, required fields, enumerated options, and clear error responses. Avoid a single unrestricted tool such as “run any command” when narrower operations are possible.

• Can the tool be replaced with a read-only version during testing?
• Does the server validate arguments independently of the model?
• Are permissions checked for the current user, tenant, resource, and operation?
• Are secrets withheld from prompts, logs, and tool results?
• Can the tool limit records, fields, file paths, network destinations, or execution time?

Memory and context

Short-term memory is the current task context: recent messages, tool results, and active decisions. Long-term memory is information retained between tasks, such as user preferences or approved organizational facts. Store long-term memory only when its purpose, retention, and deletion behavior are clear.

Before adding memory, ask whether the workflow actually needs it. Unfiltered memory can preserve incorrect assumptions, duplicate information, or sensitive data. Use typed records with provenance and timestamps rather than an undifferentiated text store. If the agent uses embeddings or retrieval, compare storage and retrieval choices with a vector database and RAG comparison.

Reliability and failure recovery

Plan for malformed tool arguments, unavailable services, rate limits, timeouts, partial results, duplicate requests, and model responses that do not follow the expected format. Retry only operations that are safe to retry, and use backoff with a maximum attempt count. A failure should produce a useful status for the user or operator rather than silently restarting the entire workflow.

Track latency, tool failures, validation failures, loop counts, escalations, cancellations, and completion outcomes. Review representative traces, but redact sensitive values. For provider-level failures, use the LLM API error-handling guide as a companion checklist.

Evaluation

Evaluate the complete workflow, not just the final prose. Create test cases for normal requests, ambiguous requests, adversarial instructions, missing permissions, unavailable tools, contradictory data, and recovery after a timeout. Measure whether the agent selected the correct tool, used valid arguments, respected limits, requested approval when required, and stopped appropriately.

Keep a small regression set whenever you change the model, system prompt, tool schema, retrieval source, or orchestration code. A prompt can be improved, but it should be tested as part of the whole system. The prompt engineering playbook provides a reusable approach for prompt versions and output checks.

Common mistakes

• Starting with too many tools: A large tool catalog increases selection errors and makes permission review harder. Begin with the smallest set that proves the workflow.
• Using prompts as security controls: Instructions such as “never access another user's data” are helpful context, but authorization must be enforced in application code and downstream services.
• Giving the agent unrestricted write access: Use separate capabilities for reading, drafting, approving, and committing.
• Letting the agent loop indefinitely: Set limits for steps, time, tool calls, and retries, with a clear escalation path.
• Confusing conversation history with durable state: Persist workflow state in a controlled store so tasks can resume, audit, and cancel reliably.
• Retrying side effects blindly: A timeout does not prove that an external action failed. Check status or use idempotency before retrying.
• Testing only successful examples: Failure paths often reveal the real architecture weaknesses. Include malformed inputs, stale data, denied permissions, and conflicting instructions.
• Changing providers without a compatibility review: Tool schemas, structured output behavior, context handling, and refusal patterns can differ. Use the LLM provider migration guide before switching model APIs.

When to revisit

Review the architecture before a seasonal planning cycle, before expanding the agent's permissions, and whenever the workflow or its connected tools change. A model update, provider migration, new retrieval source, altered schema, or new user group can change behavior even when the surrounding application code remains the same.

Use this practical review sequence:

1. Write down the agent's current goal, tools, data sources, permissions, limits, and approval points.
2. Run the regression set, including at least one failure case for every tool.
3. Inspect traces for unnecessary loops, unsupported assumptions, excessive context, and missing authorization checks.
4. Confirm that state can be resumed, cancelled, expired, and audited.
5. Recheck retention, redaction, secret handling, and prompt-injection defenses.
6. Review the highest-risk actions with the people responsible for those systems.
7. Document what changed, which tests passed, and when the next review is due.

As a starting implementation checklist, an agent should have a narrow purpose, explicit tool schemas, independent permission checks, bounded execution, durable state, validated outputs, observable traces, safe retries, human approval for consequential actions, and a tested failure path. Add capabilities only after the existing loop is predictable. That approach makes an AI agent easier to operate today and easier to revalidate when models, tools, and workflows evolve.