Types of AI Agents

In the last chapter, we defined what an agent is and walked through the fundamental agent loop. But saying "AI agent" is a bit like saying "vehicle" - technically accurate, but it covers everything from a skateboard to a 747. The architecture you choose for your agent matters enormously. Pick the wrong one and you'll either over-engineer a simple problem or under-engineer a complex one.

This chapter maps the landscape. We'll cover six major types of agents, show you when each one shines, and help you choose the right architecture for your specific use case.

---

1. Reactive Agents

The simplest possible agent. A reactive agent has no memory, no internal model, and no planning. It takes in the current input and immediately produces an action. Stimulus in, response out.

Think of it like a thermostat: the temperature drops below 68, the heater turns on. The temperature rises above 72, the heater turns off. No memory of past temperatures. No prediction of future ones. Just react to right now.

When to use reactive agents

• Simple automation tasks with clear trigger-response mappings
• High-speed decision making where latency matters more than sophistication
• Environments that are fully observable (you can see everything you need in the current input)

Code example

class ReactiveAgent:
    """A simple reactive agent that responds based on keyword matching."""

    def __init__(self):
        self.rules = {
            "price": self.check_price,
            "weather": self.check_weather,
            "news": self.fetch_news,
        }

    def act(self, user_input: str) -> str:
        """React to the current input. No memory, no planning."""
        user_input_lower = user_input.lower()

        for keyword, handler in self.rules.items():
            if keyword in user_input_lower:
                return handler(user_input)

        return "I don't know how to handle that request."

    def check_price(self, query: str) -> str:
        # Call a pricing API
        return fetch_stock_price(extract_ticker(query))

    def check_weather(self, query: str) -> str:
        return fetch_weather(extract_location(query))

    def fetch_news(self, query: str) -> str:
        return search_news(extract_topic(query))

This agent doesn't remember that you asked about Apple's stock price 30 seconds ago. Every request is a blank slate.

---

2. Model-Based (State-Based) Agents

Model-based agents maintain an internal representation of the world - a mental model. They remember things. They track how the world changes over time. This lets them handle situations where the current input alone isn't enough to make a good decision.

Think of a GPS navigation agent. It doesn't just know your current location - it knows where you started, where you're going, what roads you've already tried, and what traffic looks like. It maintains a model of the relevant world.

How they differ from reactive agents

A reactive customer support agent sees: "My order is late." It responds with a canned shipping-delay message.

A model-based agent sees the same message but also knows: this customer has contacted us three times about this order, the order has been delayed twice, and the customer is a premium subscriber. Its response accounts for all of that context.

Code example

class ModelBasedAgent:
    """An agent that maintains internal state about the world."""

    def __init__(self):
        self.world_model = {
            "customer_history": {},
            "active_orders": {},
            "system_status": {},
        }

    def update_model(self, observation: dict):
        """Update internal world model with new observation."""
        if "customer_id" in observation:
            cid = observation["customer_id"]
            if cid not in self.world_model["customer_history"]:
                self.world_model["customer_history"][cid] = []
            self.world_model["customer_history"][cid].append(observation)

    def act(self, user_input: str, context: dict) -> str:
        """Decide action based on current input AND world model."""
        self.update_model(context)

        customer_id = context.get("customer_id")
        history = self.world_model["customer_history"].get(customer_id, [])
        contact_count = len(history)

        if contact_count >= 3:
            return self.escalate_to_human(customer_id)
        elif context.get("is_premium"):
            return self.prioritized_response(user_input, history)
        else:
            return self.standard_response(user_input)

The crucial addition is self.world_model. The agent builds up knowledge over time and uses it to make better decisions.

---

3. Goal-Based Agents

Now we're getting into the territory where things feel genuinely "agentic." A goal-based agent doesn't just react to the present or track the past - it plans toward a future state. It has a goal, and it figures out the sequence of actions needed to achieve it.

This is where the LLM's reasoning ability becomes central. The agent needs to:

1. Understand the goal
2. Assess the current state
3. Identify the gap between current state and goal
4. Generate a plan to close that gap
5. Execute the plan step by step
6. Replan if something goes wrong

Example: a research agent

Goal: "Write a comprehensive summary of the latest developments in quantum computing."

The agent might plan:

1. Search for recent quantum computing news (last 3 months)
2. Identify the top 5 most significant developments
3. For each development, find the original source/paper
4. Extract key technical details from each source
5. Synthesize into a coherent summary with citations
6. Review for accuracy and completeness

No one told it those steps. It derived them from the goal.

---

4. Utility-Based Agents

A goal-based agent asks: "Does this action move me toward my goal?" A utility-based agent asks: "Of all the actions that move me toward my goal, which one is best?"

The difference is optimization. Utility-based agents assign scores to potential actions and pick the highest-scoring one. This matters when there are multiple valid paths and you care about quality, efficiency, cost, or risk.

Example: a travel-booking agent

A goal-based agent might book the first flight it finds that meets the criteria. A utility-based agent evaluates multiple options:

def score_flight(flight: dict, preferences: dict) -> float:
    """Score a flight option based on multiple utility dimensions."""
    score = 0.0

    # Price utility (lower is better, normalized)
    price_score = 1.0 - (flight["price"] / preferences["max_price"])
    score += price_score * preferences["price_weight"]  # e.g., 0.4

    # Duration utility (shorter is better)
    duration_score = 1.0 - (flight["duration_hrs"] / 24.0)
    score += duration_score * preferences["duration_weight"]  # e.g., 0.3

    # Departure time utility (closer to preferred time is better)
    time_diff = abs(flight["departure_hour"] - preferences["preferred_hour"])
    time_score = 1.0 - (time_diff / 12.0)
    score += time_score * preferences["time_weight"]  # e.g., 0.2

    # Airline preference
    if flight["airline"] in preferences["preferred_airlines"]:
        score += 0.1

    return score

The agent searches for flights, scores each one, and picks the winner. If the user says "I care most about price," the weights shift. If they say "I need to arrive by 2 PM," the scoring function adapts.

When utility-based agents shine

• Multi-criteria decision making (buying decisions, resource allocation)
• Situations where "good enough" isn't enough - you need optimal
• Competitive environments where the best action depends on evaluating alternatives

---

5. Learning Agents

Every agent type we've covered so far has a fixed strategy. A learning agent improves over time based on experience. It has four conceptual components:

• Performance element - decides what action to take (this is the agent itself)
• Critic - evaluates how well the agent is doing based on feedback
• Learning element - modifies the performance element based on the critic's feedback
• Problem generator - suggests exploratory actions to discover new knowledge

In the LLM agent world, learning typically happens through:

1. In-context learning - Including successful (and failed) past trajectories in the prompt so the LLM can learn from examples within a session.
2. Fine-tuning - Periodically retraining the model on successful agent trajectories.
3. Memory systems - Storing lessons learned in a retrievable knowledge base that the agent consults before acting.
4. Reflexion - Having the agent explicitly critique its own performance and generate improvement strategies.

class LearningAgent:
    """An agent that stores and retrieves lessons from past episodes."""

    def __init__(self, base_agent, memory_store):
        self.agent = base_agent
        self.memory = memory_store  # vector DB or similar

    def act(self, goal: str) -> str:
        # Retrieve relevant past experiences
        past_lessons = self.memory.search(goal, top_k=3)

        # Augment the goal with learned context
        enhanced_prompt = f"""Goal: {goal}

Lessons from similar past tasks:
{self.format_lessons(past_lessons)}

Apply these lessons to avoid past mistakes and use proven strategies."""

        result = self.agent.run(enhanced_prompt)

        # After completing, reflect and store new lessons
        reflection = self.reflect(goal, result)
        self.memory.store(reflection)

        return result

    def reflect(self, goal: str, result: str) -> str:
        """Generate a lesson from this episode."""
        return llm.chat(f"""
        Task: {goal}
        Result: {result}
        What worked well? What should be done differently next time?
        Summarize in 2-3 bullet points.
        """)

---

6. Hierarchical Agents

Hierarchical agents are agents that manage other agents. There's a "manager" agent that breaks down a high-level goal into subtasks, delegates each subtask to a specialized "worker" agent, and aggregates the results.

This maps naturally to how human organizations work. A CEO doesn't write code - they set strategy and delegate. A VP breaks strategy into projects. A team lead breaks projects into tasks. Individual contributors execute.

Architecture

┌─────────────────────────────┐
│       ORCHESTRATOR AGENT    │
│   (understands the big      │
│    picture, delegates)      │
└──────────┬──────────────────┘
           │
     ┌─────┼──────────┐
     │     │          │
     ▼     ▼          ▼
┌────────┐ ┌────────┐ ┌────────┐
│Research│ │Writing │ │Code    │
│Agent   │ │Agent   │ │Agent   │
└────────┘ └────────┘ └────────┘

When to use hierarchical agents

• Complex tasks that require multiple distinct skill sets (research + writing + coding)
• Tasks where parallelism helps (researching 5 topics simultaneously)
• When different subtasks need different models (a cheap fast model for simple lookups, a powerful expensive model for synthesis)

When NOT to use them

• Simple tasks that one agent can handle - adding a manager agent just adds latency and cost
• When the subtask boundaries aren't clear (the manager will struggle to decompose the work)
• When subtask results are highly interdependent (coordination overhead exceeds the benefit)

---

Comparison Table

Here's how all six types stack up:

Type	Complexity	Memory	Planning	Learning	Best For
Reactive	Very Low	None	None	None	Simple triggers, fast responses
Model-Based	Low-Medium	Yes (internal state)	Minimal	None	Context-dependent decisions
Goal-Based	Medium	Yes	Yes (plan toward goal)	None	Most general-purpose agent tasks
Utility-Based	Medium-High	Yes	Yes (with optimization)	None	Multi-criteria optimization
Learning	High	Yes (+ long-term)	Yes	Yes	Repeated tasks that improve over time
Hierarchical	Very High	Per-agent	Multi-level	Optional	Complex, multi-skill-set tasks

---

Single-Agent vs. Multi-Agent Systems

A question that comes up early in every agent project: should I use one agent or multiple?

Single-agent: when to use it

• The task has a clear, linear workflow - search, then process, then output
• One model can handle all required reasoning - you don't need specialized expertise
• Simplicity matters - debugging one agent is dramatically easier than debugging three
• Latency is critical - multi-agent systems have coordination overhead

Multi-agent: when to use it

• The task has distinct phases that require different expertise - a research phase and a coding phase and a review phase
• You want separation of concerns - each agent has a focused system prompt and toolset
• Subtasks can run in parallel for speed
• You need checks and balances - one agent writes, another reviews

---

Real-World Architecture Mapping

Which products use which architectures? Here's a rough mapping based on publicly available information:

Product	Primary Architecture	Notes
Claude Code	Goal-based, single agent	Clear goal (user task), plans, uses tools, iterates
Cursor	Goal-based with utility scoring	Ranks code completion candidates by likelihood
Perplexity	Goal-based with learning	Learns citation patterns, plans search queries
Devin	Hierarchical (planner + executor)	High-level planning agent delegates to coding sub-agents
AutoGPT	Goal-based (original), learning (later)	Added memory/reflection in later versions
CrewAI apps	Hierarchical (multi-agent)	Explicitly designed for agent teams
Customer support bots	Model-based (most), reactive (simple ones)	Track conversation state and customer history
Trading bots	Utility-based	Score opportunities by expected return, risk, liquidity

---

How to Choose: A Decision Flowchart

Walk through these questions in order:

Q1: Does the task require memory of past interactions or context?

• No → Reactive agent might be enough
• Yes → Continue

Q2: Does the task require planning - figuring out a sequence of steps to reach a goal?

• No → Model-based agent (maintains state but follows fixed logic)
• Yes → Continue

Q3: Are there multiple valid paths and do you need to pick the best one?

• No → Goal-based agent (plan and execute toward the goal)
• Yes → Utility-based agent (score and rank alternatives)

Q4: Will the agent perform this task repeatedly and should it improve over time?

• Yes → Add learning capabilities to your chosen architecture

Q5: Does the task require multiple distinct skill sets that one prompt/toolset can't cover?

• Yes → Consider a hierarchical / multi-agent architecture
• No → Stay single-agent

---

Architectural Patterns in 2025

Beyond the theoretical agent types, several practical architectural patterns have emerged in the agent-building community. These are the "design patterns" of the agent world.

ReAct (Reasoning + Acting)

The most common pattern. The agent alternates between thinking (reasoning about what to do) and acting (calling a tool). Each thought-action pair is followed by an observation (tool result), which feeds into the next thought.

Thought: I need to find the current stock price of NVIDIA.
Action: web_search("NVIDIA stock price today")
Observation: NVIDIA (NVDA) is trading at $847.23, up 3.2%...
Thought: Now I have the price. The user also asked about market cap.
Action: web_search("NVIDIA market cap 2025")
Observation: NVIDIA's market cap is approximately $2.1 trillion...
Thought: I have both pieces of information. Let me compose the answer.
Answer: NVIDIA is currently trading at $847.23 with a market cap of ~$2.1T...

Pros: Simple, transparent, easy to debug (you can read the trace). Cons: Can be slow (serial execution), may get stuck in loops.

Plan-and-Execute

The agent first creates a complete plan, then executes each step. If a step fails, it replans.

Plan:
1. Search for quarterly revenue data for Apple (last 4 quarters)
2. Search for quarterly revenue data for Microsoft (last 4 quarters)
3. Create a comparison table
4. Analyze trends and write summary

Executing step 1...
Executing step 2...
[Step 2 failed - search returned no results]
Replanning from step 2...
Revised step 2: Search for "Microsoft quarterly earnings 2024 2025"
Executing revised step 2...

Pros: More structured than ReAct, handles complex multi-step tasks well. Cons: Upfront planning can be wrong; replanning adds complexity.

LATS (Language Agent Tree Search)

Inspired by Monte Carlo Tree Search (used in game-playing AI). The agent explores multiple possible action paths, evaluates them, and backtracks when a path looks unpromising. Think of it as "try multiple approaches and keep the best one."

Pros: Finds better solutions for complex problems. Cons: Expensive (multiple LLM calls per step), complex to implement.

Reflexion

After completing a task, the agent critiques its own performance and stores the lessons. On the next similar task, it retrieves and applies those lessons. This is the "learning agent" pattern made concrete.

Task result: [agent's output]
Self-reflection: "I spent too many steps searching for information that
was available in the initial context. Next time, I should read the full
input before searching. Also, my final summary was too long - the user
asked for a brief answer."
Stored lesson: "Read full context before searching. Match output length
to request."

Pros: Genuine improvement over time. Cons: Requires persistent storage, reflection quality varies.

---

Building a Goal-Based Agent with Planning

Let's make this concrete with a more complete code example. Here's a goal-based agent that creates and follows an explicit plan:

import json
from dataclasses import dataclass

@dataclass
class Plan:
    steps: list[str]
    current_step: int = 0

class GoalBasedAgent:
    def __init__(self, llm, tools: dict):
        self.llm = llm
        self.tools = tools

    def create_plan(self, goal: str) -> Plan:
        """Ask the LLM to decompose the goal into steps."""
        response = self.llm.chat(
            f"""Break this goal into 3-7 concrete steps.
            Return as a JSON array of strings.
            Goal: {goal}"""
        )
        steps = json.loads(response)
        return Plan(steps=steps)

    def execute_step(self, step: str, context: str) -> str:
        """Execute a single step, possibly using tools."""
        response = self.llm.chat(
            messages=[
                {"role": "system", "content": f"Context so far: {context}"},
                {"role": "user", "content": f"Execute this step: {step}"},
            ],
            tools=self.tools,
        )

        if response.tool_call:
            tool_fn = self.tools[response.tool_call.name]
            return tool_fn(**response.tool_call.arguments)

        return response.content

    def run(self, goal: str) -> str:
        """Run the full goal-based agent loop."""
        plan = self.create_plan(goal)
        context_parts = []

        print(f"Plan ({len(plan.steps)} steps):")
        for i, step in enumerate(plan.steps):
            print(f"  {i+1}. {step}")

        for i, step in enumerate(plan.steps):
            print(f"\nExecuting step {i+1}: {step}")
            result = self.execute_step(step, "\n".join(context_parts))
            context_parts.append(f"Step {i+1} result: {result}")
            plan.current_step = i + 1

        # Final synthesis
        return self.llm.chat(
            f"""Based on these results, provide the final answer to: {goal}

            {chr(10).join(context_parts)}"""
        )

This is a practical starting point. In Chapter 3, we'll build something similar with real tools and a working LLM integration.

---

What's Next

You now have a map of the agent landscape - from simple reactive bots to sophisticated hierarchical systems. You know which architecture fits which problem, and you've seen how the commercial products you use implement these patterns.

But maps are useless if you never go anywhere. In Chapter 3, we're going to build an agent from scratch. No frameworks, no abstractions - just Python, API calls, and the patterns you've learned here. You'll have a working research agent by the end that can search the web, extract information, and write summaries.

Let's build.