Tool Calling

Overview

Tool calling (also known as function calling) enables LLMs to invoke external functions with structured arguments. Spring AI’s @Tool annotation transforms Java methods into typed, versioned tool definitions that LLMs can discover and execute. This is the foundation for agentic workflows where LLMs orchestrate complex multi-step tasks.

---

Key Concepts

Tool Calling Flow

┌─────────────────────────────────────────────────────────────┐
│                Tool Calling Execution Flow                   │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  1. USER QUERY                                               │
│     "What's the weather in San Francisco?"                   │
│                                                              │
│  2. LLM DECIDES TO CALL TOOL                                 │
│     Function: getWeather                                     │
│     Arguments: {location: "San Francisco", unit: "celsius"}  │
│                                                              │
│  3. SPRING AI VALIDATES SCHEMA                               │
│     Check: location is string, unit is enum                  │
│                                                              │
│  4. INVOKE JAVA METHOD                                       │
│     weatherService.getWeather("San Francisco", "celsius")    │
│                                                              │
│  5. RETURN RESULT TO LLM                                     │
│     {temp: 18, condition: "Partly cloudy"}                   │
│                                                              │
│  6. LLM GENERATES FINAL RESPONSE                             │
│     "It's 18°C and partly cloudy in San Francisco."          │
│                                                              │
└─────────────────────────────────────────────────────────────┘

@Tool Annotation

@Tool(
    description = "Get current weather for a location",
    name = "getWeather" // Optional: defaults to method name
)
public WeatherInfo getWeather(
    @ToolParam(description = "City name") String location,
    @ToolParam(description = "Temperature unit", 
               defaultValue = "celsius") String unit
) {
    // Implementation
}

---

Best Practices

1. Tools Must Be Idempotent

LLMs may call tools multiple times; ensure no side effects on repeated calls.

// ✅ GOOD: Read-only, idempotent
@Tool(description = "Get order status by order ID")
public OrderStatus getOrderStatus(@ToolParam String orderId) {
    return orderRepository.findById(orderId)
        .orElseThrow(() -> new OrderNotFoundException(orderId));
}

// ❌ BAD: Side effects
@Tool(description = "Cancel order") // DON'T ALLOW THIS
public void cancelOrder(@ToolParam String orderId) {
    orderRepository.deleteById(orderId); // State change!
}

Rule: Tools should be read-only or require explicit user confirmation for writes.

2. Validate Tool Arguments

Never trust LLM-provided arguments; validate types, ranges, and business rules.

@Tool(description = "Search products by price range")
public List<Product> searchProducts(
    @ToolParam(description = "Minimum price") double minPrice,
    @ToolParam(description = "Maximum price") double maxPrice
) {
    // Validate
    if (minPrice < 0 || maxPrice < 0) {
        throw new IllegalArgumentException("Prices must be positive");
    }
    if (minPrice > maxPrice) {
        throw new IllegalArgumentException("Min price cannot exceed max price");
    }
    if (maxPrice > 10000) {
        throw new IllegalArgumentException("Max price exceeds limit");
    }
    
    return productRepository.findByPriceBetween(minPrice, maxPrice);
}

3. Provide Rich Descriptions

LLM tool selection depends on clear, detailed descriptions.

// ❌ BAD: Vague description
@Tool(description = "Get data")
public Data getData(@ToolParam String id) { ... }

// ✅ GOOD: Specific, actionable
@Tool(description = """
    Retrieve customer order details including status, items, and shipping info.
    Use this when user asks about 'my order', 'order status', or 'where is my package'.
    """)
public OrderDetails getOrderDetails(
    @ToolParam(description = "Order ID (format: ORD-XXXXXX)") String orderId
) { ... }

4. Return Structured Data, Not Strings

LLMs work better with structured JSON responses.

// ❌ BAD: Unstructured string
@Tool(description = "Get weather")
public String getWeather(@ToolParam String location) {
    return "It's sunny in " + location; // LLM has to parse this
}

// ✅ GOOD: Structured response
@Tool(description = "Get current weather")
public WeatherInfo getWeather(@ToolParam String location) {
    return new WeatherInfo(
        location,
        18.5,
        "Partly Cloudy",
        65 // humidity %
    );
}

public record WeatherInfo(
    String location,
    double tempCelsius,
    String condition,
    int humidity
) {}

5. Version Tools Explicitly

Tool schemas evolve; version them to support gradual migration.

@Tool(
    description = "Get order status (v2: includes estimated delivery)",
    name = "getOrderStatus_v2"
)
public OrderStatusV2 getOrderStatusV2(@ToolParam String orderId) {
    // New version with additional fields
}

// Keep old version for backward compatibility
@Tool(description = "Get order status (deprecated, use v2)")
@Deprecated
public OrderStatus getOrderStatus(@ToolParam String orderId) {
    // Legacy version
}

---

Code Examples

Example 1: Simple Read-Only Tool

@Service
public class ProductTools {
    private final ProductRepository productRepository;
    
    @Tool(description = """
        Search products by name or category.
        Returns list of matching products with price and availability.
        """)
    public List<Product> searchProducts(
        @ToolParam(description = "Search query (product name or category)") 
        String query
    ) {
        return productRepository.searchByNameOrCategory(query);
    }
}

Usage:

@Service
public class ShoppingAssistant {
    private final ToolCallingChatModel chatModel;
    
    public String assist(String userMessage) {
        ChatResponse response = chatModel.call(
            new Prompt(
                userMessage,
                ChatOptions.builder()
                    .withTools(List.of("searchProducts"))
                    .build()
            )
        );
        return response.getResult().getOutput().getContent();
    }
}

✅ Good for: Product search, FAQ lookup, data retrieval
❌ Not good for: State-changing operations

---

Example 2: Tool with Enum Arguments

public enum TemperatureUnit {
    CELSIUS, FAHRENHEIT, KELVIN
}

@Tool(description = "Get current weather for a city")
public WeatherInfo getWeather(
    @ToolParam(description = "City name (e.g., 'San Francisco')") 
    String city,
    
    @ToolParam(description = "Temperature unit", 
               defaultValue = "CELSIUS") 
    TemperatureUnit unit
) {
    double temp = weatherApi.getTemperature(city);
    return new WeatherInfo(
        city,
        convertTemperature(temp, unit),
        unit
    );
}

✅ Good for: Constrained options, type safety
❌ Not good for: Open-ended string values

---

Example 3: Multi-Tool Agent

@Service
public class CustomerSupportAgent {
    private final ToolCallingChatModel chatModel;
    
    @Tool(description = "Get customer account balance")
    public BigDecimal getAccountBalance(@ToolParam String customerId) {
        return accountService.getBalance(customerId);
    }
    
    @Tool(description = "Get recent transactions for customer")
    public List<Transaction> getTransactions(
        @ToolParam String customerId,
        @ToolParam(defaultValue = "10") int limit
    ) {
        return transactionService.getRecent(customerId, limit);
    }
    
    @Tool(description = "Check if customer is eligible for refund")
    public RefundEligibility checkRefundEligibility(
        @ToolParam String customerId,
        @ToolParam String transactionId
    ) {
        return refundService.checkEligibility(customerId, transactionId);
    }
    
    public String handleQuery(String customerId, String query) {
        String systemPrompt = String.format(
            "You are a customer support agent helping customer %s.", 
            customerId
        );
        
        ChatResponse response = chatModel.call(
            new Prompt(
                List.of(
                    new SystemMessage(systemPrompt),
                    new UserMessage(query)
                ),
                ChatOptions.builder()
                    .withTools(List.of(
                        "getAccountBalance",
                        "getTransactions",
                        "checkRefundEligibility"
                    ))
                    .build()
            )
        );
        
        return response.getResult().getOutput().getContent();
    }
}

✅ Good for: Multi-step workflows, agentic behavior
❌ Not good for: When all tools are needed (use prompts instead)

---

Example 4: Tool with Error Handling

@Tool(description = "Get stock price for a ticker symbol")
public StockPrice getStockPrice(
    @ToolParam(description = "Stock ticker (e.g., AAPL)") String ticker
) {
    try {
        return stockApi.getPrice(ticker.toUpperCase());
    } catch (TickerNotFoundException e) {
        throw new ToolException(
            "Unknown ticker: " + ticker + ". Please verify the symbol.",
            e
        );
    } catch (ApiException e) {
        throw new ToolException(
            "Unable to fetch stock data. Please try again later.",
            e
        );
    }
}

✅ Good for: External API calls with failure modes
❌ Not good for: Retries (implement circuit breaker separately)

---

Example 5: Approval Required for Write Operations

@Service
public class OrderManagementAgent {
    private final ToolCallingChatModel chatModel;
    private final ApprovalService approvalService;
    
    @Tool(description = "REQUEST to cancel order (requires user approval)")
    public ApprovalRequest requestOrderCancellation(
        @ToolParam String orderId
    ) {
        // Don't actually cancel; return approval request
        return approvalService.createApprovalRequest(
            "Cancel order " + orderId,
            () -> orderService.cancel(orderId)
        );
    }
    
    public String handleQuery(String userId, String query) {
        ChatResponse response = chatModel.call(
            new Prompt(
                query,
                ChatOptions.builder()
                    .withTools(List.of("requestOrderCancellation"))
                    .build()
            )
        );
        
        String llmResponse = response.getResult().getOutput().getContent();
        
        // Check if approval is pending
        if (llmResponse.contains("approval required")) {
            return llmResponse + "\n\nType 'approve' to confirm.";
        }
        
        return llmResponse;
    }
}

✅ Good for: Destructive operations, financial transactions
❌ Not good for: Read-only queries (adds unnecessary friction)

---

Anti-Patterns

❌ Non-Idempotent Tools

// DON'T: Increments counter on each call
@Tool(description = "Increment page view counter")
public int incrementPageViews(@ToolParam String pageId) {
    return pageViewService.increment(pageId); // Side effect!
}

Why: LLM may call this multiple times, inflating counts.

✅ DO: Read-only or require explicit confirmation

@Tool(description = "Get current page view count")
public int getPageViews(@ToolParam String pageId) {
    return pageViewService.get(pageId);
}

---

❌ Returning Raw Exceptions to LLM

// DON'T: Expose stack traces
@Tool(description = "Get user details")
public User getUser(@ToolParam String userId) {
    return userRepository.findById(userId)
        .orElseThrow(() -> new RuntimeException("SQL error: ...")); // Leaks internals
}

Why: Sensitive information leakage; LLM may expose to user.

✅ DO: Return user-friendly error messages

@Tool(description = "Get user details")
public User getUser(@ToolParam String userId) {
    return userRepository.findById(userId)
        .orElseThrow(() -> new ToolException(
            "User not found: " + userId
        ));
}

---

❌ Vague Parameter Descriptions

// DON'T: Unclear what 'id' refers to
@Tool(description = "Get data")
public Data getData(@ToolParam String id) { ... }

Why: LLM may pass wrong type of ID.

✅ DO: Be explicit

@Tool(description = "Get customer account data")
public CustomerAccount getCustomerAccount(
    @ToolParam(description = "Customer UUID (format: xxxxxxxx-xxxx-...)") 
    String customerId
) { ... }

---

❌ Unbounded Result Sets

// DON'T: May return millions of records
@Tool(description = "Search all customers")
public List<Customer> searchCustomers(@ToolParam String query) {
    return customerRepository.findByNameContaining(query); // No limit!
}

Why: Exceeds LLM context window; slow response.

✅ DO: Limit and paginate

@Tool(description = "Search customers (max 10 results)")
public List<Customer> searchCustomers(
    @ToolParam String query,
    @ToolParam(defaultValue = "10") int limit
) {
    return customerRepository.findByNameContaining(query)
        .stream()
        .limit(Math.min(limit, 50)) // Hard cap at 50
        .collect(Collectors.toList());
}

---

Testing Strategies

Unit Testing Tools Independently

@Test
void shouldReturnWeatherInfo() {
    WeatherTools tools = new WeatherTools(mockWeatherApi);
    
    when(mockWeatherApi.getTemperature("Paris"))
        .thenReturn(22.0);
    
    WeatherInfo result = tools.getWeather("Paris", TemperatureUnit.CELSIUS);
    
    assertEquals("Paris", result.location());
    assertEquals(22.0, result.tempCelsius());
}

Integration Testing Tool Calls

@SpringBootTest
class ToolCallingIntegrationTest {
    @Autowired
    private ToolCallingChatModel chatModel;
    
    @Test
    void shouldCallGetWeatherTool() {
        String response = chatModel.call(
            new Prompt(
                "What's the weather in Tokyo?",
                ChatOptions.builder()
                    .withTools(List.of("getWeather"))
                    .build()
            )
        ).getResult().getOutput().getContent();
        
        assertTrue(response.contains("Tokyo"));
    }
}

Schema Validation Testing

@Test
void shouldRejectInvalidArguments() {
    assertThrows(ValidationException.class, () -> {
        tools.searchProducts(
            -10.0,  // Invalid: negative price
            50.0
        );
    });
}

---

Performance Considerations

Concern	Strategy
Latency	Tools add 1-3 seconds; cache tool results when possible
Cost	Each tool call is 2 LLM calls (decide + respond); minimize tools
Reliability	Tools can fail; implement retries and fallbacks
Context	Large tool responses consume context window; summarize results

---

Observability

Metrics to Track

@Component
@Aspect
public class ToolCallMetrics {
    private final MeterRegistry registry;
    
    @Around("@annotation(Tool)")
    public Object trackToolCall(ProceedingJoinPoint joinPoint, Tool tool) throws Throwable {
        Timer.Sample sample = Timer.start(registry);
        
        try {
            Object result = joinPoint.proceed();
            
            registry.counter("tool.calls.success", "name", tool.name())
                .increment();
            
            return result;
        } catch (Exception e) {
            registry.counter("tool.calls.failure", 
                "name", tool.name(),
                "error", e.getClass().getSimpleName())
                .increment();
            throw e;
        } finally {
            sample.stop(registry.timer("tool.call.duration", "name", tool.name()));
        }
    }
}

---

References

• Spring AI Documentation - Tool Calling
• OpenAI Function Calling Guide
• Anthropic Tool Use

---

Related Skills

• chat-models.md — LLM integration
• prompt-templates.md — System prompts for tool use
• observability.md — Tracking tool usage
• failure-handling.md — Error handling
• ai-ml/agentic-patterns.md — Agent orchestration