context-engineering-intro/use-cases/pydantic-ai/README.md

Pydantic AI Context Engineering Template

A comprehensive template for building production-grade AI agents using Pydantic AI with context engineering best practices, tools integration, structured outputs, and comprehensive testing patterns.

🚀 Quick Start - Copy Template

Get started in 2 minutes:

# Clone the context engineering repository
git clone https://github.com/coleam00/Context-Engineering-Intro.git
cd Context-Engineering-Intro/use-cases/pydantic-ai

# 1. Copy this template to your new project
python copy_template.py /path/to/my-agent-project

# 2. Navigate to your project
cd /path/to/my-agent-project

# 3. Fill out PRPs/INITIAL.md with the agent you want to create

# 4. Start building with the PRP workflow
# Edit PRPs/INITIAL.md with your requirements, then:
/generate-pydantic-ai-prp PRPs/INITIAL.md
/execute-pydantic-ai-prp PRPs/generated_prp.md

If you are not using Claude Code, you can simply tell your AI coding assistant to use the generate-pydantic-ai-prp and execute-pydantic-ai-prp slash commands in .claude/commands as prompts.

📖 What is This Template?

This template provides everything you need to build sophisticated Pydantic AI agents using proven context engineering workflows. It combines:

• Pydantic AI Best Practices: Type-safe agents with tools, structured outputs, and dependency injection
• Context Engineering Workflows: Proven PRP (Problem Requirements Planning) methodology
• Production Patterns: Security, testing, monitoring, and deployment-ready code
• Working Examples: Complete agent implementations you can learn from and extend

🎯 PRP Framework Workflow

This template uses a 3-step context engineering workflow for building AI agents:

1. Define Requirements (PRPs/INITIAL.md)

Start by clearly defining what your agent needs to do:

# Customer Support Agent - Initial Requirements

## Overview
Build an intelligent customer support agent that can handle inquiries, 
access customer data, and escalate issues appropriately.

## Core Requirements
- Multi-turn conversations with context and memory
- Customer authentication and account access
- Account balance and transaction queries
- Payment processing and refund handling
...

2. Generate Implementation Plan

/generate-pydantic-ai-prp PRPs/INITIAL.md

This creates a comprehensive Problem Requirements Planning document that includes:

• Pydantic AI technology research and best practices
• Agent architecture design with tools and dependencies
• Implementation roadmap with validation loops
• Security patterns and production considerations

3. Execute Implementation

/execute-pydantic-ai-prp PRPs/your_agent.md

This implements the complete agent based on the PRP, including:

• Agent creation with proper model provider configuration
• Tool integration with error handling and validation
• Structured output models with Pydantic validation
• Comprehensive testing with TestModel and FunctionModel
• Security patterns and production deployment setup

📂 Template Structure

pydantic-ai/
├── CLAUDE.md                           # Pydantic AI global development rules
├── copy_template.py                    # Template deployment script
├── .claude/commands/
│   ├── generate-pydantic-ai-prp.md     # PRP generation for agents
│   └── execute-pydantic-ai-prp.md      # PRP execution for agents
├── PRPs/
│   ├── templates/
│   │   └── prp_pydantic_ai_base.md     # Base PRP template for agents
│   └── INITIAL.md                      # Example agent requirements
├── examples/
│   ├── basic_chat_agent/               # Simple conversational agent
│   │   ├── agent.py                    # Agent with memory and context
│   │   └── README.md                   # Usage guide
│   ├── tool_enabled_agent/             # Agent with external tools
│   │   ├── agent.py                    # Web search + calculator tools
│   │   └── requirements.txt            # Dependencies
│   └── testing_examples/               # Comprehensive testing patterns
│       ├── test_agent_patterns.py      # TestModel, FunctionModel examples
│       └── pytest.ini                  # Test configuration
└── README.md                           # This file

🤖 Agent Examples Included

1. Main Agent Reference (examples/main_agent_reference/)

The canonical reference implementation showing proper Pydantic AI patterns:

• Environment-based configuration with settings.py and providers.py
• Clean separation of concerns between email and research agents
• Tool integration with external APIs (Gmail, Brave Search)
• Production-ready error handling and logging

Key Files:

• settings.py: Environment configuration with pydantic-settings
• providers.py: Model provider abstraction with get_llm_model()
• research_agent.py: Multi-tool agent with web search and email integration
• email_agent.py: Specialized agent for Gmail draft creation

2. Basic Chat Agent (examples/basic_chat_agent/)

A simple conversational agent demonstrating core patterns:

• Environment-based model configuration (follows main_agent_reference)
• String output by default (no result_type unless needed)
• System prompts (static and dynamic)
• Conversation memory with dependency injection

Key Features:

• Simple string responses (not structured output)
• Settings-based configuration pattern
• Conversation context tracking
• Clean, minimal implementation

3. Tool-Enabled Agent (examples/tool_enabled_agent/)

An agent with tool integration capabilities:

• Environment-based configuration (follows main_agent_reference)
• String output by default (no unnecessary structure)
• Web search and calculation tools
• Error handling and retry mechanisms

Key Features:

• @agent.tool decorator patterns
• RunContext for dependency injection
• Tool error handling and recovery
• Simple string responses from tools

4. Structured Output Agent (examples/structured_output_agent/)

NEW: Shows when to use result_type for data validation:

• Environment-based configuration (follows main_agent_reference)
• Structured output with Pydantic validation (when specifically needed)
• Data analysis with statistical tools
• Professional report generation

Key Features:

• Demonstrates proper use of result_type
• Pydantic validation for business reports
• Data analysis tools with numerical statistics
• Clear documentation on when to use structured vs string output

5. Testing Examples (examples/testing_examples/)

Comprehensive testing patterns for Pydantic AI agents:

• TestModel for rapid development validation
• FunctionModel for custom behavior testing
• Agent.override() for test isolation
• Pytest fixtures and async testing

Key Features:

• Unit testing without API costs
• Mock dependency injection
• Tool validation and error scenario testing
• Integration testing patterns

🛠️ Core Pydantic AI Patterns

Environment-Based Configuration (from main_agent_reference)

# settings.py - Environment configuration
from pydantic_settings import BaseSettings
from pydantic import Field

class Settings(BaseSettings):
    llm_provider: str = Field(default="openai")
    llm_api_key: str = Field(...)
    llm_model: str = Field(default="gpt-4")
    llm_base_url: str = Field(default="https://api.openai.com/v1")
    
    class Config:
        env_file = ".env"

# providers.py - Model provider abstraction
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.models.openai import OpenAIModel

def get_llm_model() -> OpenAIModel:
    settings = Settings()
    provider = OpenAIProvider(
        base_url=settings.llm_base_url,
        api_key=settings.llm_api_key
    )
    return OpenAIModel(settings.llm_model, provider=provider)

Simple Agent (String Output - Default)

from pydantic_ai import Agent, RunContext
from dataclasses import dataclass

@dataclass
class AgentDependencies:
    """Dependencies for agent execution"""
    api_key: str
    session_id: str = None

# Simple agent - no result_type, defaults to string
agent = Agent(
    get_llm_model(),  # Environment-based configuration
    deps_type=AgentDependencies,
    system_prompt="You are a helpful assistant..."
)

Structured Output Agent (When Validation Needed)

from pydantic import BaseModel, Field

class AnalysisReport(BaseModel):
    """Use result_type ONLY when you need validation"""
    summary: str
    confidence: float = Field(ge=0.0, le=1.0)
    insights: list[str] = Field(min_items=1)

# Structured agent - result_type specified for validation
structured_agent = Agent(
    get_llm_model(),
    deps_type=AgentDependencies,
    result_type=AnalysisReport,  # Only when structure is required
    system_prompt="You are a data analyst..."
)

Tool Integration

@agent.tool
async def example_tool(
    ctx: RunContext[AgentDependencies], 
    query: str
) -> str:
    """Tool with proper error handling - returns string."""
    try:
        result = await external_api_call(ctx.deps.api_key, query)
        return f"API result: {result}"
    except Exception as e:
        logger.error(f"Tool error: {e}")
        return f"Tool temporarily unavailable: {str(e)}"

Testing with TestModel

from pydantic_ai.models.test import TestModel

def test_simple_agent():
    """Test simple agent with string output."""
    test_model = TestModel()
    with agent.override(model=test_model):
        result = agent.run_sync("Test message")
        assert isinstance(result.data, str)  # String output

def test_structured_agent():
    """Test structured agent with validation."""
    test_model = TestModel(
        custom_output_text='{"summary": "Test", "confidence": 0.8, "insights": ["insight1"]}'
    )
    with structured_agent.override(model=test_model):
        result = structured_agent.run_sync("Analyze this data")
        assert isinstance(result.data, AnalysisReport)  # Validated object
        assert 0.0 <= result.data.confidence <= 1.0

🎯 When to Use String vs Structured Output

Use String Output (Default) ✅

Most agents should use string output - don't specify result_type:

# ✅ Simple chat agent
chat_agent = Agent(get_llm_model(), system_prompt="You are helpful...")

# ✅ Tool-enabled agent  
tool_agent = Agent(get_llm_model(), tools=[search_tool], system_prompt="...")

# Result: agent.run() returns string
result = agent.run_sync("Hello")
print(result.data)  # "Hello! How can I help you today?"

When to use string output:

• Conversational agents
• Creative writing
• Flexible responses
• Human-readable output
• Simple tool responses

Use Structured Output (Specific Cases) 🎯

Only use result_type when you need validation:

# ✅ Data analysis requiring validation
analysis_agent = Agent(
    get_llm_model(), 
    result_type=AnalysisReport,  # Pydantic model with validation
    system_prompt="You are a data analyst..."
)

# Result: agent.run() returns validated Pydantic object
result = analysis_agent.run_sync("Analyze sales data")
print(result.data.confidence)  # 0.85 (validated 0.0-1.0)

When to use structured output:

• Data validation required
• API integrations needing specific schemas
• Business reports with consistent formatting
• Downstream processing requiring type safety
• Database insertion with validated fields

Key Rule 📏

Start with string output. Only add result_type when you specifically need validation or structure.

🔒 Security Best Practices

This template includes production-ready security patterns:

API Key Management

# Environment variables (never commit to code)
export LLM_API_KEY="your-api-key-here"
export LLM_MODEL="gpt-4"
export LLM_BASE_URL="https://api.openai.com/v1"

# Or use .env file (git-ignored)
echo "LLM_API_KEY=your-api-key-here" > .env
echo "LLM_MODEL=gpt-4" >> .env

Input Validation

from pydantic import BaseModel, Field

class ToolInput(BaseModel):
    query: str = Field(max_length=1000, description="Search query")
    max_results: int = Field(ge=1, le=10, default=5)

Error Handling

@agent.tool
async def secure_tool(ctx: RunContext[Deps], input_data: str) -> str:
    try:
        # Validate and sanitize input
        cleaned_input = sanitize_input(input_data)
        result = await process_safely(cleaned_input)
        return result
    except Exception as e:
        # Log error without exposing sensitive data
        logger.error(f"Tool error: {type(e).__name__}")
        return "An error occurred. Please try again."

🧪 Testing Your Agents

Development Testing (Fast, No API Costs)

from pydantic_ai.models.test import TestModel

# Test with TestModel for rapid iteration
test_model = TestModel()
with agent.override(model=test_model):
    result = agent.run_sync("Test input")
    print(result.data)

Custom Behavior Testing

from pydantic_ai.models.test import FunctionModel

def custom_response(messages, tools):
    """Custom function to control agent responses."""
    return '{"response": "Custom test response", "confidence": 0.9}'

function_model = FunctionModel(function=custom_response)
with agent.override(model=function_model):
    result = agent.run_sync("Test input")

Integration Testing

# Test with real models (use sparingly due to costs)
@pytest.mark.integration
async def test_agent_integration():
    result = await agent.run("Real test message")
    assert result.data.confidence > 0.5

🚀 Deployment Patterns

Environment Configuration

# settings.py - Production configuration
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    # LLM Configuration
    llm_api_key: str
    llm_model: str = "gpt-4"
    llm_base_url: str = "https://api.openai.com/v1"
    
    # Production settings
    app_env: str = "production"
    log_level: str = "INFO"
    retries: int = 3
    
    class Config:
        env_file = ".env"

# agent.py - Use environment settings
agent = Agent(
    get_llm_model(),  # From providers.py
    retries=settings.retries,
    system_prompt="Production agent..."
)

Docker Deployment

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .
EXPOSE 8000

CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

🎓 Learning Path

1. Start with Examples

• Run examples/basic_chat_agent/agent.py to see a simple agent
• Explore examples/tool_enabled_agent/ for tool integration
• Study examples/testing_examples/ for testing patterns

2. Use the PRP Workflow

• Edit PRPs/INITIAL.md with your agent requirements
• Generate a PRP: /generate-pydantic-ai-prp PRPs/INITIAL.md
• Execute the PRP: /execute-pydantic-ai-prp PRPs/generated_file.md

3. Build Your Own Agent

• Start with the basic chat agent pattern
• Add tools for external capabilities
• Implement structured outputs for your use case
• Add comprehensive testing and error handling

4. Production Deployment

• Implement security patterns from CLAUDE.md
• Add monitoring and logging
• Set up CI/CD with automated testing
• Deploy with proper scaling and availability

🤝 Common Gotchas & Solutions

Based on extensive Pydantic AI research, here are common issues and solutions:

Async/Sync Patterns

# ❌ Don't mix sync and async inconsistently
def bad_tool(ctx):
    return asyncio.run(some_async_function())  # Anti-pattern

# ✅ Be consistent with async patterns
@agent.tool
async def good_tool(ctx: RunContext[Deps]) -> str:
    result = await some_async_function()
    return result

Model Token Limits

# ✅ Handle different model capabilities
from pydantic_ai.models import FallbackModel

model = FallbackModel([
    "openai:gpt-4o",        # High capability, higher cost
    "openai:gpt-4o-mini",   # Fallback option
])

Tool Error Handling

# ✅ Implement proper retry and fallback
@agent.tool
async def resilient_tool(ctx: RunContext[Deps], query: str) -> str:
    for attempt in range(3):
        try:
            return await external_api_call(query)
        except TemporaryError:
            if attempt == 2:
                return "Service temporarily unavailable"
            await asyncio.sleep(2 ** attempt)

📚 Additional Resources

• Official Pydantic AI Documentation: https://ai.pydantic.dev/
• Model Provider Guides: https://ai.pydantic.dev/models/
• Tool Integration Patterns: https://ai.pydantic.dev/tools/
• Testing Strategies: https://ai.pydantic.dev/testing/
• Context Engineering Methodology: See main repository README

🆘 Support & Contributing

• Issues: Report problems with the template or examples
• Improvements: Contribute additional examples or patterns
• Questions: Ask about Pydantic AI integration or context engineering

This template is part of the larger Context Engineering framework. See the main repository for more context engineering templates and methodologies.

---

Ready to build production-grade AI agents? Start with python copy_template.py my-agent-project and follow the PRP workflow! 🚀