Building AI Agents with Function Calling in Python

The landscape of artificial intelligence is evolving at a breathtaking pace. While large language models (LLMs) have captivated the world with their ability to generate human-like text, their true power lies not just in conversation, but in their capacity to interact with the real world. This is where AI agents and the concept of “function calling” come into play. As a senior software engineer, I’ve seen firsthand how integrating LLMs with external tools and APIs can unlock unprecedented capabilities, transforming static chatbots into dynamic, problem-solving entities.

In this comprehensive guide, we’ll dive deep into building intelligent AI agents in Python that leverage function calling. We’ll explore the core concepts, walk through architectural patterns, provide practical code examples, and discuss real-world applications, best practices, and the challenges you might encounter. By the end, you’ll have a solid understanding of how to empower your LLMs to go beyond mere text generation and truly act upon your requests.

The Evolution: From Chatbots to AI Agents

Remember the early days of chatbots? They were often rule-based, rigid, and quickly hit their limits. Even advanced LLM-powered chatbots, while incredibly fluent, primarily operate within the confines of their training data. They can answer questions, summarize text, or even write code, but they can’t inherently *do* things in the real world – like check the weather in real-time, book a flight, or update a database entry.

Enter the AI agent. An AI agent is an autonomous, goal-oriented system that uses an LLM as its “brain” to reason, plan, and execute actions to achieve a specific objective. The key differentiator is its ability to interact with external tools and services. This is precisely where function calling becomes indispensable.

What is Function Calling?

Function calling (also known as “tool use” or “tool integration”) is a mechanism that allows an LLM to intelligently identify when to use a tool, dynamically generate the arguments for that tool, and then have an external system execute it. Essentially, it’s a way for an LLM to “call” a Python function, an API endpoint, or any other defined capability, based on a user’s prompt.

Here’s the magic: instead of simply responding to a query, the LLM analyzes the user’s intent and, if it determines that an external action is required, it will output a structured request (often in JSON format) specifying which function to call and with what parameters. Your application then intercepts this request, executes the actual function, and feeds the result back to the LLM. The LLM then uses this new information to formulate a comprehensive and accurate final response.

Why is Function Calling Important?

Function calling addresses several critical limitations of standalone LLMs:

1. Access to Real-time Information: LLMs are trained on historical data. Function calling allows them to fetch current information (e.g., live weather, stock prices, news headlines).
2. Performing Actions: Beyond generating text, agents can now perform actions like sending emails, managing calendars, interacting with databases, or controlling IoT devices.
3. Reducing Hallucinations: By relying on concrete tool outputs, agents can provide more factual and grounded responses, reducing the likelihood of making up information.
4. Enhanced Specificity and Accuracy: Tools can perform precise calculations or data lookups that an LLM alone might struggle with.
5. Extensibility: You can continually add new tools to your agent, expanding its capabilities without retraining the underlying LLM.

The Architecture of an AI Agent with Function Calling

To truly grasp function calling, let’s visualize the typical flow. Imagine the LLM as a highly intelligent but physically constrained entity. Function calling provides its “hands” and “eyes” to interact with the world.

Components and Their Interactions

1. User Prompt: The interaction begins with a user’s request (e.g., “What’s the weather like in London and how much is Apple stock?”).

2. Agent Orchestration Layer: This is your Python application, the “brain” that manages the conversation flow. It receives the user prompt and sends it to the LLM.

3. Large Language Model (LLM):

   • Receives the prompt and a list of available tools (function definitions).
   • Reasons about the user’s intent and decides if any of the provided tools are relevant.
   • If a tool is needed, it generates a structured “tool call” (e.g., JSON containing function name and arguments).
   • If no tool is needed, it generates a direct textual response.

4. Tool Definition (Function Schema): You define your tools as Python functions and then describe them to the LLM using a structured format (typically JSON Schema). This tells the LLM what functions are available, what they do, and what parameters they accept.

5. Tool Execution:

   • If the LLM suggests a tool call, the Agent Orchestration Layer intercepts this.
   • It parses the tool call (function name and arguments).
   • It then executes the actual Python function (e.g., calls an external weather API, queries a database).
   • The result of the tool execution is obtained.

6. Result Feedback to LLM: The Agent Orchestration Layer takes the output from the tool execution and sends it back to the LLM as part of the ongoing conversation history. This is crucial because the LLM needs to see the result to formulate its final, informed response.

7. LLM Final Response: With the tool’s output now available in its context, the LLM processes this information and generates a natural language response to the user’s original query.

Architectural Flow (Diagram in Words)

[User]
    |
    v
[User Prompt]  "What's the weather in London and the price of AAPL?"
    |
    v
[Python Agent Orchestration Layer]
    | (Sends prompt & tool definitions to LLM)
    v
[Large Language Model (LLM)]  (e.g., GPT-4, Llama 3)
    | (LLM identifies need for tools: `get_current_weather`, `get_stock_price`)
    v
[LLM Tool Call Suggestion]
    {
        "tool_calls": [
            {"function": {"name": "get_current_weather", "arguments": {"location": "London", "unit": "celsius"}}},
            {"function": {"name": "get_stock_price", "arguments": {"symbol": "AAPL"}}}
        ]
    }
    |
    v
[Python Agent Orchestration Layer]
    | (Parses tool calls)
    v
[Execute Tool: get_current_weather(London)] ------------> [External Weather API]
    | (Returns weather data)
    v
[Execute Tool: get_stock_price(AAPL)] ------------------> [External Stock API]
    | (Returns stock price)
    v
[Python Agent Orchestration Layer]
    | (Sends tool outputs back to LLM as part of conversation history)
    v
[Large Language Model (LLM)]
    | (LLM integrates tool outputs with original prompt)
    v
[LLM Final Response] "The current weather in London is 15°C and Apple stock (AAPL) is trading at $175.42."
    |
    v
[Python Agent Orchestration Layer]
    |
    v
[User]

Getting Started with Function Calling in Python: A Practical Guide

Let’s move from theory to practice. We’ll use the OpenAI API for our examples, as it offers a robust and well-documented function calling interface. However, the core principles apply to other LLM providers (e.g., Google’s Gemini, Anthropic’s Claude) and frameworks like LangChain or LiteLLM, which abstract these interactions.

Setting Up Your Environment

First, install the necessary library and set up your API key:

pip install openai python-dotenv

Create a .env file in your project root to store your OpenAI API key:

OPENAI_API_KEY="your_openai_api_key_here"

Then, in your Python script, load it:

import os
from dotenv import load_dotenv

load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

if not OPENAI_API_KEY:
    raise ValueError("OPENAI_API_KEY not found. Please set it in a .env file or as an environment variable.")

Step-by-Step: Basic Function Calling Example (Weather Agent)

Let’s build a simple agent that can tell us the current weather for a given location.

1. Define Your Tool as a Python Function

This is a regular Python function that performs a specific task. For demonstration, we’ll use a mock weather service.

# tools.py
import json

def get_current_weather(location: str, unit: str = "fahrenheit") -> str:
    """
    Get the current weather in a given location.

    Args:
        location (str): The city and state, e.g., "San Francisco, CA" or "London, UK".
        unit (str): The unit of temperature. Can be "celsius" or "fahrenheit".
                    Defaults to "fahrenheit".

    Returns:
        str: A JSON string containing the location, temperature, and description.
    """
    print(f"--- Calling get_current_weather for {location} in {unit} ---")
    # In a real application, this would call an external weather API.
    # For now, we'll use mock data.
    weather_data = {
        "San Francisco, CA": {"temperature": 68, "unit": "fahrenheit", "description": "Sunny"},
        "London, UK": {"temperature": 15, "unit": "celsius", "description": "Cloudy"},
        "Paris, FR": {"temperature": 22, "unit": "celsius", "description": "Partly cloudy"},
    }

    normalized_location = location.strip()
    if normalized_location in weather_data:
        data = weather_data[normalized_location]
        if unit.lower() != data["unit"].lower():
            # Simple conversion for demonstration
            if unit.lower() == "celsius" and data["unit"].lower() == "fahrenheit":
                data["temperature"] = round((data["temperature"] - 32) * 5/9)
                data["unit"] = "celsius"
            elif unit.lower() == "fahrenheit" and data["unit"].lower() == "celsius":
                data["temperature"] = round((data["temperature"] * 9/5) + 32)
                data["unit"] = "fahrenheit"
        return json.dumps({
            "location": location,
            "temperature": data["temperature"],
            "unit": data["unit"],
            "description": data["description"]
        })
    else:
        return json.dumps({"location": location, "error": "Weather data not available for this location."})

# A dictionary to map function names to their actual Python functions
available_functions = {
    "get_current_weather": get_current_weather,
}

2. Describe the Tool to the LLM (Function Schema)

The LLM needs to understand what your function does, its name, and its parameters. This is done using a JSON Schema format. We’ll define this structure.

# main.py (continued)
from openai import OpenAI
from tools import get_current_weather, available_functions # Import your tools

client = OpenAI(api_key=OPENAI_API_KEY)

# --- Define the tool schema for the LLM ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g., San Francisco, CA or London, UK",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

3. Implement the Agent Orchestration Logic

This is the core loop where you send the prompt, check for tool calls, execute them, and feed results back to the LLM.

# main.py (continued)
import json

def run_conversation():
    messages = [
        {"role": "user", "content": "What's the weather like in London?"}
    ]

    # First API call: send the user prompt and available tools to the LLM
    print("\n--- LLM Call 1: User Prompt ---")
    response = client.chat.completions.create(
        model="gpt-4o", # or "gpt-3.5-turbo", "gpt-4"
        messages=messages,
        tools=tools,
        tool_choice="auto", # Allows the LLM to decide whether to call a tool or respond directly
    )
    response_message = response.choices[0].message
    print(f"LLM Response (raw): {response_message}")

    # Check if the LLM decided to call a tool
    if response_message.tool_calls:
        print("\n--- LLM wants to call a tool! ---")
        messages.append(response_message)  # Extend conversation with LLM's tool call request

        # Step 2: Call the tool(s)
        for tool_call in response_message.tool_calls:
            function_name = tool_call.function.name
            function_to_call = available_functions.get(function_name)

            if not function_to_call:
                print(f"Error: Function {function_name} not found.")
                continue

            function_args = json.loads(tool_call.function.arguments)
            
            # Execute the function
            function_response = function_to_call(**function_args)
            print(f"Tool '{function_name}' executed. Output: {function_response}")

            # Step 3: Send tool output back to the LLM
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response,
                }
            )
        
        # Second API call: send the updated conversation (with tool outputs) to the LLM
        print("\n--- LLM Call 2: Tool Output Feedback ---")
        second_response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
        )
        final_response_message = second_response.choices[0].message
        print(f"LLM Final Response: {final_response_message.content}")
        return final_response_message.content
    else:
        # LLM responded directly without a tool call
        print("\n--- LLM responded directly ---")
        print(f"LLM Final Response: {response_message.content}")
        return response_message.content

if __name__ == "__main__":
    run_conversation()

When you run this, you’ll see the LLM first indicate it wants to call `get_current_weather`, then your Python code executes it, and finally, the LLM uses that result to form its natural language answer. This multi-turn interaction is the essence of function calling.

Handling Multiple Tools and Complex Scenarios

Real-world agents often need more than one tool. Let’s expand our agent to also get stock prices.

1. Add Another Tool

# tools.py (continued)
def get_stock_price(symbol: str) -> str:
    """
    Get the current stock price for a given stock symbol.

    Args:
        symbol (str): The stock ticker symbol, e.g., "AAPL", "MSFT".

    Returns:
        str: A JSON string containing the stock symbol and its current price.
    """
    print(f"--- Calling get_stock_price for {symbol} ---")
    stock_data = {
        "AAPL": 175.42,
        "MSFT": 420.10,
        "GOOG": 170.80,
    }
    normalized_symbol = symbol.upper().strip()
    if normalized_symbol in stock_data:
        return json.dumps({"symbol": symbol, "price": stock_data[normalized_symbol]})
    else:
        return json.dumps({"symbol": symbol, "error": "Stock data not available for this symbol."})

# Update available_functions
available_functions = {
    "get_current_weather": get_current_weather,
    "get_stock_price": get_stock_price,
}

2. Add the New Tool to the LLM Schema

# main.py (continued)
# --- Define ALL tool schemas for the LLM ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g., San Francisco, CA or London, UK",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_stock_price",
            "description": "Get the current stock price for a given stock symbol.",
            "parameters": {
                "type": "object",
                "properties": {
                    "symbol": {
                        "type": "string",
                        "description": "The stock ticker symbol, e.g., AAPL, MSFT",
                    },
                },
                "required": ["symbol"],
            },
        },
    }
]

3. Update `run_conversation` for Multiple Tool Calls

The beauty of the OpenAI API is that it can suggest multiple tool calls in a single response. Our existing loop already handles this, but let’s make the prompt more challenging.

# main.py (updated run_conversation)
def run_conversation_multi_tool():
    messages = [
        {"role": "user", "content": "What's the weather like in London and how much is Apple stock (AAPL)?"}
    ]

    print("\n--- LLM Call 1: User Prompt with multiple needs ---")
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
        tool_choice="auto",
    )
    response_message = response.choices[0].message
    print(f"LLM Response (raw): {response_message}")

    if response_message.tool_calls:
        print("\n--- LLM wants to call multiple tools! ---")
        messages.append(response_message) # Append LLM's tool_calls response

        for tool_call in response_message.tool_calls:
            function_name = tool_call.function.name
            function_to_call = available_functions.get(function_name)
            if not function_to_call:
                print(f"Error: Function {function_name} not found.")
                continue

            function_args = json.loads(tool_call.function.arguments)
            function_response = function_to_call(**function_args)
            print(f"Tool '{function_name}' executed. Output: {function_response}")

            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response,
                }
            )
        
        # Second API call: LLM processes all tool outputs
        print("\n--- LLM Call 2: Tool Outputs Feedback for multiple tools ---")
        second_response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
        )
        final_response_message = second_response.choices[0].message
        print(f"LLM Final Response: {final_response_message.content}")
        return final_response_message.content
    else:
        print("\n--- LLM responded directly ---")
        print(f"LLM Final Response: {response_message.content}")
        return response_message.content

if __name__ == "__main__":
    run_conversation_multi_tool()

Now, with a single user query, the agent can orchestrate two distinct tool calls and synthesize their results into a coherent response.

Tool Chaining and State Management

Sometimes, the output of one tool needs to be the input for another. This is “tool chaining.” For example, “Find me the product ID for ‘wireless mouse’ and then get its details.” This requires the agent to maintain state (the conversation history) and potentially perform multiple rounds of LLM interaction and tool execution.

Our `messages` list effectively manages the conversation state for the LLM. For tool chaining, the agent’s orchestration layer needs to be robust enough to handle sequential tool calls. Let’s imagine a scenario where we need to create a user and then send them an email.

Example: User Management and Notification Agent

# tools.py (add these)
def create_user(name: str, email: str) -> str:
    """
    Creates a new user in the system.

    Args:
        name (str): The name of the user.
        email (str): The email address of the user.

    Returns:
        str: A JSON string confirming user creation, including a generated user_id.
    """
    print(f"--- Calling create_user for {name} ({email}) ---")
    # In a real app, this would interact with a database
    user_id = f"user_{len(available_functions)+100}" # Mock ID
    users_db[user_id] = {"name": name, "email": email, "status": "active"}
    return json.dumps({"status": "success", "user_id": user_id, "name": name, "email": email})

def send_email(recipient_email: str, subject: str, body: str) -> str:
    """
    Sends an email to a specified recipient.

    Args:
        recipient_email (str): The email address of the recipient.
        subject (str): The subject line of the email.
        body (str): The main content of the email.

    Returns:
        str: A JSON string indicating the success or failure of the email sending.
    """
    print(f"--- Calling send_email to {recipient_email} ---")
    # In a real app, this would use an email service (e.g., SendGrid, SMTP)
    if "@" in recipient_email:
        return json.dumps({"status": "success", "message": f"Email sent to {recipient_email} with subject '{subject}'"})
    else:
        return json.dumps({"status": "failed", "message": "Invalid email address."})

# Global mock database for users
users_db = {}

# Update available_functions
available_functions.update({
    "create_user": create_user,
    "send_email": send_email,
})

# main.py (add to tools list)

# ... existing tools ...

    {

        "type": "function",

        "function": {

            "name": "create_user",

            "description": "Creates a new user in the system.",

            "parameters": {

                "type": "object",

                "properties": {

                    "name": {"type": "string", "description": "The name of the user."},

                    "email": {"type": "string", "description": "The email address of the user."},

                },

                "required": ["name", "email"],

            },

        },

    },

    {