Python Testing

Comprehensive testing is essential for maintaining code quality and preventing regressions in Python applications. This skill covers pytest as the modern testing framework, fixture management for test setup/teardown, mocking external dependencies, parametrized tests for data-driven testing, code coverage measurement, and tox for testing across multiple Python versions. Mastering these tools enables confidence in refactoring, continuous integration, and production deployments.

Key Concepts

• pytest Framework - Modern testing framework with simple assert statements, powerful fixtures, and extensive plugin ecosystem
• Fixtures - Reusable test setup/teardown functions that provide dependencies and manage resources
• Parametrize - Data-driven testing by running the same test with multiple input sets
• Mocking - Isolating code under test by replacing external dependencies with controlled test doubles
• Test Discovery - Automatic test detection following naming conventions (test_*.py, *_test.py)
• Coverage Analysis - Measuring code execution during tests to identify untested paths
• Test Markers - Organizing tests by category (slow, integration, smoke) for selective execution
• Assertions - pytest’s introspection provides detailed failure messages without custom assert methods
• Test Doubles - Mocks, stubs, spies, and fakes for controlling test dependencies
• Test Isolation - Each test runs independently without shared state or side effects

Best Practices

1. Write clear test names - Use descriptive names that explain what is being tested and expected outcome
2. Follow AAA pattern - Structure tests as Arrange (setup), Act (execute), Assert (verify)
3. Use fixtures for setup - Extract common setup code into reusable fixtures instead of repeating in tests
4. Parametrize data-driven tests - Use @pytest.mark.parametrize for testing multiple inputs efficiently
5. Mock external dependencies - Isolate unit tests by mocking databases, APIs, file systems, and network calls
6. Aim for 80%+ coverage - Measure and maintain high test coverage, but focus on critical paths
7. Use conftest.py for shared fixtures - Place common fixtures in conftest.py for automatic discovery
8. Test edge cases and errors - Verify boundary conditions, invalid inputs, and exception handling
9. Keep tests fast - Unit tests should run in milliseconds; use markers to separate slow integration tests
10. Run tests in CI/CD - Automate test execution on every commit with coverage reports

Code Examples

Basic pytest Structure and AAA Pattern

# ✅ GOOD: Clear test structure with AAA pattern
from typing import Final
from decimal import Decimal

import pytest

from myapp.services import PaymentProcessor, PaymentError

class TestPaymentProcessor:
    """Test suite for PaymentProcessor."""
    
    def test_successful_payment_returns_transaction_id(self) -> None:
        """Test that successful payment returns valid transaction ID."""
        # Arrange
        processor = PaymentProcessor(api_key="test_key")
        amount = Decimal("99.99")
        currency = "USD"
        
        # Act
        result = processor.process_payment(amount, currency)
        
        # Assert
        assert result["status"] == "success"
        assert "transaction_id" in result
        assert result["amount_charged"] == amount
        assert len(result["transaction_id"]) > 0
    
    def test_negative_amount_raises_value_error(self) -> None:
        """Test that negative amount raises ValueError."""
        # Arrange
        processor = PaymentProcessor(api_key="test_key")
        
        # Act & Assert
        with pytest.raises(ValueError, match="Amount must be positive"):
            processor.process_payment(Decimal("-10.00"), "USD")
    
    def test_invalid_currency_raises_value_error(self) -> None:
        """Test that invalid currency code raises ValueError."""
        # Arrange
        processor = PaymentProcessor(api_key="test_key")
        
        # Act & Assert
        with pytest.raises(ValueError, match="Invalid currency code"):
            processor.process_payment(Decimal("10.00"), "INVALID")

# ❌ BAD: Unclear test names, missing structure
def test_payment():
    processor = PaymentProcessor(api_key="test_key")
    result = processor.process_payment(Decimal("99.99"), "USD")
    assert result["status"] == "success"

def test_error():
    processor = PaymentProcessor(api_key="test_key")
    try:
        processor.process_payment(Decimal("-10.00"), "USD")
        assert False
    except ValueError:
        pass

pytest Fixtures for Setup and Teardown

# ✅ GOOD: Reusable fixtures with proper scoping
import tempfile
from pathlib import Path
from typing import Iterator

import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import Session, sessionmaker

from myapp.database import Base
from myapp.models import User

# conftest.py - Shared fixtures

@pytest.fixture(scope="session")
def database_engine():
    """
    Create database engine for test session.
    
    Session-scoped fixture is created once per test session.
    """
    engine = create_engine("sqlite:///:memory:")
    Base.metadata.create_all(engine)
    yield engine
    Base.metadata.drop_all(engine)
    engine.dispose()

@pytest.fixture(scope="function")
def db_session(database_engine) -> Iterator[Session]:
    """
    Provide clean database session for each test.
    
    Function-scoped fixture creates new session per test.
    Automatically rolls back changes after test.
    """
    SessionLocal = sessionmaker(bind=database_engine)
    session = SessionLocal()
    
    try:
        yield session
        session.commit()
    except Exception:
        session.rollback()
        raise
    finally:
        session.close()

@pytest.fixture
def sample_user(db_session: Session) -> User:
    """Create and return a sample user for testing."""
    user = User(
        username="testuser",
        email="test@example.com",
        is_active=True
    )
    db_session.add(user)
    db_session.commit()
    db_session.refresh(user)
    return user

@pytest.fixture
def temp_directory() -> Iterator[Path]:
    """Provide temporary directory that is cleaned up after test."""
    with tempfile.TemporaryDirectory() as tmpdir:
        yield Path(tmpdir)
    # Automatically cleaned up

# test_user_service.py - Using fixtures

from myapp.services import UserService

def test_create_user_stores_in_database(db_session: Session) -> None:
    """Test that created user is persisted to database."""
    # Arrange
    service = UserService(db_session)
    
    # Act
    user = service.create_user(
        username="newuser",
        email="new@example.com"
    )
    
    # Assert
    assert user.id is not None
    retrieved = db_session.query(User).filter_by(id=user.id).first()
    assert retrieved is not None
    assert retrieved.username == "newuser"

def test_get_user_returns_existing_user(
    db_session: Session,
    sample_user: User
) -> None:
    """Test retrieving existing user by ID."""
    # Arrange
    service = UserService(db_session)
    
    # Act
    user = service.get_user(sample_user.id)
    
    # Assert
    assert user is not None
    assert user.id == sample_user.id
    assert user.username == sample_user.username

# ❌ BAD: Repeated setup code in each test
def test_create_user():
    engine = create_engine("sqlite:///:memory:")
    Base.metadata.create_all(engine)
    SessionLocal = sessionmaker(bind=engine)
    session = SessionLocal()
    # Repeated in every test...

Parametrized Tests for Data-Driven Testing

# ✅ GOOD: Parametrized tests with descriptive IDs
import pytest
from decimal import Decimal

from myapp.validators import validate_email, validate_amount, validate_password

class TestEmailValidation:
    """Test email validation with various inputs."""
    
    @pytest.mark.parametrize(
        "email,expected",
        [
            ("user@example.com", True),
            ("user.name@example.co.uk", True),
            ("user+tag@example.com", True),
            ("invalid", False),
            ("@example.com", False),
            ("user@", False),
            ("", False),
        ],
        ids=[
            "valid_simple",
            "valid_subdomain",
            "valid_with_plus",
            "invalid_no_at",
            "invalid_no_username",
            "invalid_no_domain",
            "invalid_empty",
        ]
    )
    def test_email_validation(self, email: str, expected: bool) -> None:
        """Test email validation with various formats."""
        result = validate_email(email)
        assert result == expected

class TestAmountValidation:
    """Test amount validation with boundary conditions."""
    
    @pytest.mark.parametrize(
        "amount,currency,should_pass",
        [
            (Decimal("0.01"), "USD", True),  # Minimum valid
            (Decimal("999999.99"), "USD", True),  # Maximum valid
            (Decimal("0.00"), "USD", False),  # Zero not allowed
            (Decimal("-10.00"), "USD", False),  # Negative not allowed
            (Decimal("0.001"), "USD", False),  # Too many decimals
            (Decimal("10.00"), "INVALID", False),  # Invalid currency
        ]
    )
    def test_amount_validation(
        self,
        amount: Decimal,
        currency: str,
        should_pass: bool
    ) -> None:
        """Test amount validation with various values."""
        if should_pass:
            validate_amount(amount, currency)  # Should not raise
        else:
            with pytest.raises(ValueError):
                validate_amount(amount, currency)

# Parametrize with fixtures
@pytest.fixture(params=["sqlite", "postgresql", "mysql"])
def database_url(request) -> str:
    """Parametrized fixture for testing multiple databases."""
    db_type = request.param
    if db_type == "sqlite":
        return "sqlite:///:memory:"
    elif db_type == "postgresql":
        return "postgresql://test:test@localhost/test"
    else:
        return "mysql://test:test@localhost/test"

def test_connection_works_for_all_databases(database_url: str) -> None:
    """Test database connection for all supported databases."""
    engine = create_engine(database_url)
    with engine.connect() as conn:
        result = conn.execute("SELECT 1")
        assert result.fetchone()[0] == 1

# ❌ BAD: Repeated test code for each case
def test_valid_email_simple():
    assert validate_email("user@example.com") == True

def test_valid_email_subdomain():
    assert validate_email("user.name@example.co.uk") == True

def test_invalid_email_no_at():
    assert validate_email("invalid") == False
# ... 10 more nearly identical tests

Mocking External Dependencies

# ✅ GOOD: Comprehensive mocking with unittest.mock
from unittest.mock import Mock, MagicMock, patch, call
from typing import Any

import pytest
import requests

from myapp.services import NotificationService, EmailProvider

class TestNotificationService:
    """Test notification service with mocked dependencies."""
    
    def test_send_notification_calls_email_provider(self) -> None:
        """Test that send_notification delegates to email provider."""
        # Arrange
        mock_provider = Mock(spec=EmailProvider)
        mock_provider.send_email.return_value = {"status": "sent", "id": "123"}
        
        service = NotificationService(email_provider=mock_provider)
        
        # Act
        result = service.send_notification(
            to="user@example.com",
            subject="Test",
            body="Test message"
        )
        
        # Assert
        mock_provider.send_email.assert_called_once_with(
            to="user@example.com",
            subject="Test",
            body="Test message"
        )
        assert result["status"] == "sent"
    
    def test_send_notification_retries_on_failure(self) -> None:
        """Test retry logic when email provider fails."""
        # Arrange
        mock_provider = Mock(spec=EmailProvider)
        # Fail twice, succeed on third attempt
        mock_provider.send_email.side_effect = [
            Exception("Network error"),
            Exception("Network error"),
            {"status": "sent", "id": "123"}
        ]
        
        service = NotificationService(
            email_provider=mock_provider,
            max_retries=3
        )
        
        # Act
        result = service.send_notification(
            to="user@example.com",
            subject="Test",
            body="Test"
        )
        
        # Assert
        assert mock_provider.send_email.call_count == 3
        assert result["status"] == "sent"
    
    @patch("requests.post")
    def test_http_notification_with_patched_requests(
        self,
        mock_post: Mock
    ) -> None:
        """Test HTTP notification with patched requests library."""
        # Arrange
        mock_response = Mock()
        mock_response.status_code = 200
        mock_response.json.return_value = {"success": True}
        mock_post.return_value = mock_response
        
        service = NotificationService()
        
        # Act
        result = service.send_http_notification(
            url="https://api.example.com/notify",
            data={"message": "test"}
        )
        
        # Assert
        mock_post.assert_called_once_with(
            "https://api.example.com/notify",
            json={"message": "test"},
            timeout=30
        )
        assert result["success"] is True

# Context manager for patching
@patch("myapp.services.external_api")
def test_with_context_manager(mock_api: Mock) -> None:
    """Test using patch as context manager."""
    mock_api.get_data.return_value = {"value": 42}
    
    from myapp.services import DataProcessor
    processor = DataProcessor()
    result = processor.process()
    
    assert result == {"value": 42}
    mock_api.get_data.assert_called_once()

# Mock async functions
@pytest.mark.asyncio
async def test_async_function_mocking() -> None:
    """Test mocking async functions."""
    from unittest.mock import AsyncMock
    
    mock_client = Mock()
    mock_client.get = AsyncMock(return_value={"data": "value"})
    
    result = await mock_client.get("https://api.example.com")
    
    assert result == {"data": "value"}
    mock_client.get.assert_awaited_once_with("https://api.example.com")

# ❌ BAD: Testing with real external dependencies
def test_send_email():
    service = NotificationService()
    result = service.send_notification(
        to="real@example.com",
        subject="Test",
        body="Test"
    )
    # Sends real email - slow, brittle, side effects!

Test Markers and Organization

# ✅ GOOD: Organized tests with markers
import pytest

# pytest.ini or pyproject.toml
"""
[tool.pytest.ini_options]
markers =
    unit: Unit tests (fast, isolated)
    integration: Integration tests (slower, external dependencies)
    slow: Slow tests (can be skipped during development)
    smoke: Smoke tests (critical functionality)
    requires_db: Tests requiring database
    requires_network: Tests requiring network access
"""

@pytest.mark.unit
class TestUserValidation:
    """Fast unit tests for user validation."""
    
    def test_validate_username_length(self) -> None:
        """Test username length validation."""
        from myapp.validators import validate_username
        
        assert validate_username("valid") is True
        assert validate_username("ab") is False  # Too short
        assert validate_username("a" * 51) is False  # Too long

@pytest.mark.integration
@pytest.mark.requires_db
class TestUserRepository:
    """Integration tests requiring database."""
    
    def test_save_and_retrieve_user(self, db_session) -> None:
        """Test saving and retrieving user from database."""
        from myapp.repositories import UserRepository
        
        repo = UserRepository(db_session)
        user_id = repo.save(username="test", email="test@example.com")
        
        user = repo.get_by_id(user_id)
        assert user.username == "test"

@pytest.mark.slow
@pytest.mark.requires_network
class TestExternalAPI:
    """Slow tests requiring network access."""
    
    @pytest.mark.timeout(30)
    def test_fetch_real_data(self) -> None:
        """Test fetching data from real API."""
        from myapp.clients import APIClient
        
        client = APIClient()
        data = client.fetch_data()
        assert data is not None

@pytest.mark.smoke
class TestCriticalPaths:
    """Smoke tests for critical application functionality."""
    
    def test_application_starts(self) -> None:
        """Test that application can start."""
        from myapp.main import create_app
        
        app = create_app()
        assert app is not None

# Running specific test categories
"""
# Run only unit tests
pytest -m unit

# Run integration and slow tests
pytest -m "integration or slow"

# Run everything except slow tests
pytest -m "not slow"

# Run smoke tests only
pytest -m smoke

# Skip tests requiring network
pytest -m "not requires_network"
"""

# ❌ BAD: All tests mixed without organization
class TestEverything:
    def test_validation(self):
        pass
    
    def test_database(self):
        pass
    
    def test_slow_api_call(self):
        pass

Code Coverage Measurement

# ✅ GOOD: Comprehensive coverage configuration
"""
# pyproject.toml
[tool.coverage.run]
source = ["src/myapp"]
omit = [
    "*/tests/*",
    "*/test_*.py",
    "*/__pycache__/*",
    "*/venv/*",
    "*/migrations/*",
]
branch = true  # Measure branch coverage
parallel = true  # Support parallel testing

[tool.coverage.report]
precision = 2
show_missing = true
skip_covered = false
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "if TYPE_CHECKING:",
    "raise AssertionError",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "@abstractmethod",
    "@abc.abstractmethod",
]

[tool.coverage.html]
directory = "htmlcov"

[tool.pytest.ini_options]
addopts = [
    "--cov=myapp",
    "--cov-report=term-missing",
    "--cov-report=html",
    "--cov-report=xml",
    "--cov-fail-under=80",
]
"""

# Running coverage
"""
# Run tests with coverage
pytest --cov=myapp --cov-report=term-missing

# Generate HTML report
pytest --cov=myapp --cov-report=html
open htmlcov/index.html

# Generate XML report for CI
pytest --cov=myapp --cov-report=xml

# Show coverage for specific module
pytest --cov=myapp.services --cov-report=term

# Fail if coverage below threshold
pytest --cov=myapp --cov-fail-under=80
"""

# Example coverage report
"""
Name                           Stmts   Miss Branch BrPart  Cover   Missing
--------------------------------------------------------------------------
myapp/__init__.py                  3      0      0      0   100%
myapp/services/user.py            45      2      8      1    94%   23, 45
myapp/services/payment.py         67      5     12      2    89%   34-38, 56
myapp/utils/validation.py         23      0      4      0   100%
--------------------------------------------------------------------------
TOTAL                            138      7     24      3    93%
"""

# ❌ BAD: No coverage measurement, unknown test gaps
"""
pytest tests/  # No coverage information
"""

Anti-Patterns

Testing Implementation Details

# ❌ Avoid: Testing private methods and internal state
def test_user_service_internal_cache():
    service = UserService()
    user = service.get_user(1)
    assert user.id in service._cache  # Testing private implementation

# ✅ Fix: Test public interface and observable behavior
def test_user_service_returns_cached_user():
    service = UserService()
    user1 = service.get_user(1)
    user2 = service.get_user(1)
    assert user1 == user2  # Test behavior, not implementation

Tests with Shared State

# ❌ Avoid: Tests affecting each other through shared state
shared_users = []

def test_create_user():
    user = create_user("test")
    shared_users.append(user)
    assert len(shared_users) == 1

def test_list_users():
    # Depends on previous test running first!
    assert len(shared_users) == 1

# ✅ Fix: Each test is independent with fixtures
@pytest.fixture
def users():
    return []

def test_create_user(users):
    user = create_user("test")
    users.append(user)
    assert len(users) == 1

def test_list_users(users):
    users.append(create_user("test"))
    assert len(users) == 1

Catching Too Broad Exceptions

# ❌ Avoid: Catching generic exceptions
def test_invalid_input_raises_error():
    with pytest.raises(Exception):  # Too broad!
        process_data(None)

# ✅ Fix: Catch specific exceptions with message matching
def test_invalid_input_raises_value_error():
    with pytest.raises(ValueError, match="Data cannot be None"):
        process_data(None)

Slow Tests Without Markers

# ❌ Avoid: Slow tests slowing down entire suite
def test_process_large_dataset():
    data = generate_large_dataset()  # Takes 30 seconds
    result = process(data)
    assert result is not None

# ✅ Fix: Mark slow tests for selective execution
@pytest.mark.slow
def test_process_large_dataset():
    data = generate_large_dataset()
    result = process(data)
    assert result is not None

# Run fast tests during development
# pytest -m "not slow"

Testing Strategies

Test Pyramid Strategy

"""
Unit Tests (70-80%)
├── Fast, isolated tests
├── Mock external dependencies
└── High coverage of business logic

Integration Tests (15-20%)
├── Test component interactions
├── Use TestContainers for databases
└── Verify data flow between layers

End-to-End Tests (5-10%)
├── Test complete user workflows
├── Use Selenium/Playwright
└── Critical paths only
"""

Testing Async Code

# tests/test_async_service.py
import pytest
from unittest.mock import AsyncMock

from myapp.services import AsyncUserService

@pytest.mark.asyncio
async def test_async_user_creation() -> None:
    """Test async user creation."""
    mock_db = AsyncMock()
    mock_db.save.return_value = {"id": 1, "username": "test"}
    
    service = AsyncUserService(db=mock_db)
    result = await service.create_user(username="test")
    
    assert result["id"] == 1
    mock_db.save.assert_awaited_once()

@pytest.mark.asyncio
async def test_concurrent_operations() -> None:
    """Test multiple async operations."""
    service = AsyncUserService()
    
    results = await asyncio.gather(
        service.get_user(1),
        service.get_user(2),
        service.get_user(3)
    )
    
    assert len(results) == 3
    assert all(r is not None for r in results)

Integration Testing with TestContainers

# tests/test_integration.py
import pytest
from testcontainers.postgres import PostgresContainer

from myapp.database import create_engine_from_url
from myapp.repositories import UserRepository

@pytest.fixture(scope="module")
def postgres_container():
    """Provide PostgreSQL container for integration tests."""
    with PostgresContainer("postgres:15") as postgres:
        yield postgres

@pytest.fixture
def db_engine(postgres_container):
    """Create database engine connected to test container."""
    url = postgres_container.get_connection_url()
    engine = create_engine_from_url(url)
    # Run migrations
    from myapp.database import Base
    Base.metadata.create_all(engine)
    yield engine
    Base.metadata.drop_all(engine)

def test_user_repository_with_real_database(db_engine) -> None:
    """Test repository with real PostgreSQL database."""
    repo = UserRepository(db_engine)
    
    user_id = repo.create(username="test", email="test@example.com")
    user = repo.get_by_id(user_id)
    
    assert user.username == "test"
    assert user.email == "test@example.com"

Performance Testing

# tests/test_performance.py
import time
import pytest

@pytest.mark.slow
def test_query_performance() -> None:
    """Ensure query completes within acceptable time."""
    start = time.time()
    
    result = expensive_query()
    
    duration = time.time() - start
    assert duration < 1.0, f"Query took {duration}s, expected < 1.0s"
    assert len(result) > 0

@pytest.mark.benchmark
def test_processing_throughput(benchmark) -> None:
    """Benchmark processing throughput."""
    def process():
        return process_batch(data)
    
    result = benchmark(process)
    
    # Benchmark plugin provides statistics
    # Mean: 100ms, StdDev: 5ms, Min: 95ms, Max: 110ms
    assert result is not None

References

• pytest Documentation
• pytest Fixtures
• unittest.mock
• pytest-cov Coverage Plugin
• pytest-asyncio
• TestContainers Python
• Coverage.py
• pytest-benchmark

Related Skills

• python-best-practices.md - Type hints improve test clarity
• async-python.md - Testing async functions with pytest-asyncio
• python-packaging.md - Packaging tests with tox for CI/CD
• pythonic-patterns.md - Testing context managers and decorators