Building Powerful AI Agents with Function Calling in Python

In the rapidly evolving landscape of artificial intelligence, Large Language Models (LLMs) have emerged as incredibly versatile and powerful tools. They can generate human-like text, answer questions, summarize documents, and even write code. However, for all their brilliance, standalone LLMs have a fundamental limitation: they exist in a vacuum. They can’t directly interact with the real world, fetch real-time data, or execute complex operations beyond text generation.

This is where the concept of “AI Agents” and “Function Calling” (also known as “Tool Use” or “Tool Utilization”) comes into play. By equipping an LLM with the ability to call external functions or tools, we transform it from a mere text generator into an intelligent agent capable of sensing, reasoning, and acting upon its environment. This article will guide you through the process of building such agents in Python, covering everything from fundamental concepts to advanced architectural patterns and real-world applications.

Understanding Function Calling: The Bridge to the Real World

What is Function Calling?

At its core, function calling is a mechanism that allows a Large Language Model to interact with external systems, services, or data sources. Imagine an LLM not just as a brilliant conversationalist, but as a skilled assistant who has access to a comprehensive toolbox. When faced with a request that requires information or actions beyond its inherent knowledge base (its training data), the LLM can “decide” to reach into its toolbox, pick the right tool, and use it.

For example, if you ask an LLM, “What’s the weather like in New York today?”, a standalone LLM might politely tell you it doesn’t have real-time information. An LLM empowered with function calling, however, would recognize that this query requires external data. It would then identify a “get weather” tool, call it with “New York” as the location, receive the real-time weather data, and then present that information back to you in a natural language response.

This capability fundamentally extends the utility of LLMs, enabling them to:

• Access Real-Time Information: Fetch current stock prices, news headlines, weather updates, or sports scores.
• Perform Calculations: Execute complex mathematical operations that are prone to errors when done purely via text generation.
• Interact with APIs: Send emails, update databases, control smart home devices, or trigger workflows in SaaS applications.
• Query Private Data: Search internal company documents, customer records, or specific datasets.
• Execute Code: Run Python scripts, SQL queries, or shell commands.

How it Works: The Mechanics Behind the Magic

The process of function calling involves a sophisticated interplay between your application logic and the LLM. Here’s a step-by-step breakdown:

1. Define Your Tools: You, as the developer, define a set of functions (e.g., Python functions) that your AI agent can use. Crucially, you also provide a detailed, natural language description and a JSON schema for each function’s parameters. This description tells the LLM what the tool does and what inputs it expects.
2. Send Tools to the LLM: When you make a request to the LLM, you don’t just send the user’s prompt; you also send the definitions of all available tools.
3. LLM’s Reasoning and Tool Selection: The LLM analyzes the user’s prompt in conjunction with the provided tool definitions. It determines if any of the tools are relevant to fulfill the user’s request. If it decides a tool is needed, it also extracts the necessary arguments from the user’s prompt (e.g., location for a weather tool, amount for a currency converter).
4. LLM Outputs a “Tool Call”: Instead of generating a natural language response, the LLM outputs a structured object indicating which tool to call and with what arguments. This is not the execution of the tool itself, but rather a *suggestion* to your application to execute it.
5. Your Application Executes the Tool: Your code intercepts this tool call, parses the tool name and arguments, and then executes the actual Python function corresponding to that tool.
6. Send Tool Output Back to the LLM: The result of the executed tool (e.g., the weather data, the stock price) is then sent back to the LLM as another message in the conversation history.
7. LLM Integrates and Responds: With the tool’s output now available, the LLM uses this information to formulate a coherent, natural language response to the user, or it might decide to call another tool based on the new information. This process can repeat multiple times in a single interaction, forming a chain of reasoning and action.

This iterative process allows the LLM to engage in multi-turn reasoning and problem-solving, making it far more capable than a simple Q&A bot.

Architecting Your AI Agent with Function Calling

Building a robust AI agent with function calling requires more than just knowing how to call an API. It involves designing an architecture that can manage conversational state, orchestrate tool interactions, handle errors, and integrate seamlessly with various external systems.

Core Components of an AI Agent

A typical AI agent architecture centered around function calling will consist of several key components:

1. The Large Language Model (LLM): This is the brain of your agent. It performs the natural language understanding, reasoning, and decision-making regarding tool use and response generation. Examples include OpenAI’s GPT models, Anthropic’s Claude, Google’s Gemini, or fine-tuned open-source models.
2. Tools/Functions: These are the specific capabilities your agent possesses. They are typically standard Python functions that wrap external API calls, database queries, or complex business logic. Each tool needs a clear description and a structured parameter schema for the LLM to understand how and when to use it.
3. Agent Orchestration Logic: This is the critical component that manages the entire lifecycle of an interaction. It’s responsible for:
   • Receiving user input.
   • Sending the input (along with tool definitions and conversation history) to the LLM.
   • Parsing the LLM’s response (checking for text or tool calls).
   • Executing requested tools.
   • Sending tool outputs back to the LLM.
   • Managing the conversational turn and determining when to present a final response to the user.
   • Handling errors and edge cases.
4. Memory (Optional but Highly Recommended): For agents that need to maintain context across multiple turns or conversations, a memory component is essential. This can be as simple as storing the last few messages in a list or as complex as using a vector database for semantic retrieval of past interactions or long-term user profiles.
5. External Systems/APIs: These are the targets of your tools – databases, third-party services, internal APIs, file systems, etc.

Architectural Flow: A Text-Based Diagram

Let’s visualize the flow of information and control in a typical function-calling agent interaction:

+-------------------+       +-----------------------+
|   User Interface  |       |    External Systems   |
| (Web App, Chatbot)|       | (Databases, APIs, etc.)|
+---------+---------+       +-----------+-----------+
          |                             ^
          | 1. User Query               | 5. Tool Execution
          v                             |
+------------------------------------------------+
|             Agent Orchestration Logic          |
| (Python Application - The Conductor)           |
+-----------------+------------------------------+
                  |
                  | 2. Prepare Request (Query, Tools, History)
                  v
+------------------------------------------------+
|              Large Language Model (LLM)        |
| (e.g., OpenAI API, Local LLM)                  |
+-----------------+------------------------------+
                  |
                  | 3. LLM Response:
                  |    - Text (Final Answer)
                  |    - Tool Call (Tool Name, Arguments)
                  v
+------------------------------------------------+
|             Agent Orchestration Logic          |
| (Python Application - The Conductor)           |
+-----------------+------------------------------+
                  |
                  | 4. Process LLM Response:
                  |    - If Text: Return to User Interface
                  |    - If Tool Call:
                  |        Parse Tool Call
                  |        Execute Corresponding Python Function (Tool)
                  |        (Go to Step 5: External Systems Interaction)
                  |        Receive Tool Output
                  |        Add Tool Output to History
                  |        (Go back to Step 2: Send Tool Output to LLM)
                  v
+-----------------+------------------------------+
|            Memory (Conversation History)       |
| (e.g., List of messages, Vector DB)            |
+------------------------------------------------+

Explanation of the Flow:

1. User Query: The user initiates an interaction through a UI.
2. Prepare Request: The Agent Orchestration Logic receives the query. It gathers the current conversation history (from Memory), the user’s new query, and the definitions of all available tools. This combined package is then sent to the LLM.
3. LLM Response: The LLM processes the request. It decides one of two things:
   • It has enough information to provide a direct natural language response.
   • It needs to use a tool to gather more information or perform an action. In this case, it generates a structured “tool call” object specifying which tool to use and with what parameters.
4. Process LLM Response: The Agent Orchestration Logic receives the LLM’s response.
   • If it’s a direct text response, the agent returns this to the User Interface as the final answer.
   • If it’s a tool call, the agent parses the tool name and arguments. It then executes the corresponding Python function (the actual tool) which interacts with an External System.
5. Tool Execution and Output: The executed tool interacts with the External System (e.g., fetches data from a database, calls an API). The result of this operation is the “tool output”.
6. Feedback to LLM: The Agent Orchestration Logic takes this tool output, adds it to the conversation history (in Memory), and then sends this updated history back to the LLM (returning to Step 2, but now with the tool’s result as context). The LLM can then use this new information to formulate a final answer or decide to call another tool.

This loop continues until the LLM provides a final, natural language response, ensuring that the agent can perform multi-step reasoning and action sequences.

Building a Simple AI Agent in Python

Let’s dive into the practical implementation. We’ll use OpenAI’s API for its robust function calling capabilities, but the principles apply broadly to other LLMs and frameworks like LangChain or LlamaIndex.

Prerequisites

Before you begin, ensure you have:

• Python 3.8+ installed.
• An OpenAI API key. You can get one from the OpenAI platform.
• The openai Python library installed: pip install openai

Defining Tools for the LLM

The first step is to define the functions your agent can use. These are regular Python functions, but they need to be accompanied by a structured description that the LLM can understand. OpenAI’s API expects these descriptions in a specific JSON schema format.

Let’s create a simple tool to get the current weather.

import openai
import json
import os

# Set your OpenAI API key
# It's best practice to load this from an environment variable
# export OPENAI_API_KEY='your_api_key_here'
openai.api_key = os.getenv("OPENAI_API_KEY")

if not openai.api_key:
    raise ValueError("OPENAI_API_KEY environment variable not set.")

# --- 1. Define your actual Python functions (the "tools") ---

def get_current_weather(location: str, unit: str = "fahrenheit") -> dict:
    """
    Get the current weather in a given location.
    
    Args:
        location (str): The city and state, e.g., "San Francisco, CA".
        unit (str, optional): The unit of temperature. Can be "celsius" or "fahrenheit".
                              Defaults to "fahrenheit".
    Returns:
        dict: A dictionary containing weather information (location, temperature, unit, description).
              Returns None if weather data cannot be fetched for the location.
    """
    print(f"--- Calling get_current_weather for {location} in {unit} ---")
    # In a real application, this would make an API call to a weather service.
    # For this example, we'll use mock data.
    weather_data = {
        "San Francisco, CA": {"temperature": 72, "unit": "fahrenheit", "description": "Sunny"},
        "Boston, MA": {"temperature": 65, "unit": "fahrenheit", "description": "Partly cloudy"},
        "London, UK": {"temperature": 18, "unit": "celsius", "description": "Cloudy with a chance of rain"},
        "Tokyo, JP": {"temperature": 25, "unit": "celsius", "description": "Warm and clear"},
    }
    
    data = weather_data.get(location)
    if data:
        if unit == "celsius" and data["unit"] == "fahrenheit":
            data["temperature"] = round((data["temperature"] - 32) * 5/9)
            data["unit"] = "celsius"
        elif unit == "fahrenheit" and data["unit"] == "celsius":
            data["temperature"] = round((data["temperature"] * 9/5) + 32)
            data["unit"] = "fahrenheit"
        return data
    else:
        return {"location": location, "error": "Weather data not available for this location."}


# --- 2. Create the tool definitions for the LLM ---
# This describes the function in a way the LLM can understand,
# including its purpose and its parameters' types and descriptions.

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location. Use this tool when the user asks for current weather conditions.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g., 'San Francisco, CA' or 'London, UK'",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature. Defaults to 'fahrenheit'.",
                    },
                },
                "required": ["location"],
            },
        },
    }
]

# A dictionary to easily map tool names to their actual Python functions
available_functions = {
    "get_current_weather": get_current_weather,
}

In the code above:

• We define a standard Python function `get_current_weather` that simulates fetching weather data.
• We then create a `tools` list. Each item in this list is a dictionary describing a tool. This dictionary includes:
  • `type: “function”`: Specifies it’s a function tool.
  • `function`: An object containing:
    • `name`: The name of the Python function (must match).
    • `description`: A natural language explanation of what the tool does. This is crucial for the LLM to decide when to use it.
    • `parameters`: A JSON schema defining the arguments the function expects. This tells the LLM what information to extract from the user’s prompt to pass to the function.
• `available_functions`: A Python dictionary mapping the tool names to their actual Python function objects, making it easy to call them dynamically.

The Agent Core Loop

Now, let’s put it all together into an agent that can interact with the user, call the LLM, execute tools, and respond.

# --- 3. Implement the Agent Core Loop ---

def run_conversation(user_message: str):
    messages = [{"role": "user", "content": user_message}]

    # Step 1: Send user message and available tools to the LLM
    print(f"\nUser: {user_message}")
    print("--- Sending message and tools to LLM ---")
    response = openai.chat.completions.create(
        model="gpt-4o", # or gpt-3.5-turbo, gpt-4-turbo, etc.
        messages=messages,
        tools=tools, # Pass the tool definitions here
        tool_choice="auto", # Allow the LLM to decide whether to call a tool
    )
    response_message = response.choices[0].message
    messages.append(response_message) # Add LLM's response to history

    print(f"--- LLM's initial response: {response_message.content if response_message.content else 'Tool call detected'} ---")

    # Step 2: Check if the LLM wants to call a tool
    if response_message.tool_calls:
        print("--- LLM requested tool call(s) ---")
        for tool_call in response_message.tool_calls:
            function_name = tool_call.function.name
            function_to_call = available_functions.get(function_name)
            
            if not function_to_call:
                print(f"Error: Function {function_name} not found.")
                continue

            function_args_str = tool_call.function.arguments
            try:
                function_args = json.loads(function_args_str)
            except json.JSONDecodeError:
                print(f"Error: Could not decode JSON arguments for {function_name}: {function_args_str}")
                continue

            # Step 3: Call the function with the LLM-provided arguments
            # Use **function_args to unpack dictionary arguments
            function_response = function_to_call(**function_args)
            
            # Step 4: Send the function's response back to the LLM
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": json.dumps(function_response), # Tool output must be a string
                }
            )
            print(f"--- Function '{function_name}' executed. Output: {function_response} ---")

        # Step 5: Get the final response from the LLM based on tool output
        print("--- Sending tool output back to LLM for final response ---")
        final_response = openai.chat.completions.create(
            model="gpt-4o",
            messages=messages,
        )
        final_response_message = final_response.choices[0].message
        messages.append(final_response_message) # Add final LLM response to history
        print(f"Agent: {final_response_message.content}")
        return final_response_message.content
    else:
        # If LLM didn't call a tool, its initial response is the final one
        print(f"Agent: {response_message.content}")
        return response_message.content

# --- Example Usage ---
if __name__ == "__main__":
    print("AI Weather Agent ready. Ask about the weather!")
    run_conversation("What's the weather in San Francisco, CA?")
    run_conversation("How about in London, UK in Celsius?")
    run_conversation("Tell me a joke.") # LLM should not call a tool here
    run_conversation("What's the weather in some random place?") # Tool error handling example
    run_conversation("What is the capital of France?") # LLM should answer directly

Explanation of the Agent Core Loop:

1. Initial Call to LLM: We start a `messages` list with the user’s query. We then call `openai.chat.completions.create`, passing both the `messages` and our `tools` definitions. `tool_choice=”auto”` tells the LLM it can choose to call a tool or respond directly.
2. Process LLM’s First Response: We check `response_message.tool_calls`.
   • If it’s empty, the LLM has provided a direct textual answer, which we print.
   • If `tool_calls` exist, the LLM wants to use one or more tools.
3. Execute Tool(s): For each `tool_call` suggested by the LLM:
   • We extract the `function_name` and its `arguments` (which are provided as a JSON string).
   • We use `json.loads()` to parse the arguments into a Python dictionary.
   • We retrieve the actual Python function from our `available_functions` map.
   • We execute the Python function using `**function_args` to unpack the dictionary into keyword arguments.
4. Send Tool Output Back: The result of the function call (`function_response`) is then added back to the `messages` list. Crucially, this message has a `role` of “tool” and includes the `tool_call_id` to link it back to the original tool request. The `content` must be a JSON string.
5. Final LLM Call: With the tool’s output now part of the conversation history, we make a *second* call to the LLM. This time, the LLM has the full context, including the user’s query, its own decision to call a tool, and the result of that tool’s execution. It can then generate a final, informed response to the user.

This simple example demonstrates the fundamental pattern. Real-world agents will involve more sophisticated state management, error handling, and potentially multiple chained tool calls.

Code Example 2: Simple Calculator Agent

Let’s extend our agent with multiple tools for basic arithmetic operations. This will demonstrate how the LLM can choose between different tools based on the user’s intent.

# --- Extend with more tools: Calculator functions ---

def add(a: float, b: float) -> float:

    """Adds two numbers."""

    print(f"--- Calling add({a}, {b}) ---")

    return a + b
def subtract(a: float, b: float) -> float:

    """Subtracts the second number from the first."""

    print(f"--- Calling subtract({a}, {b}) ---")

    return a - b
def multiply(a: float, b: float) -> float:

    """Multiplies two numbers."""

    print(f"--- Calling multiply({a}, {b}) ---")

    return a * b
def divide(a: float, b: float) -> float:

    """Divides the first number by the second. Handles division by zero."""

    print(f"--- Calling divide({a}, {b}) ---")

    if b == 0:

        return "Error: Division by zero is not allowed."

    return a / b
# Add these new tools to our 'tools' list for the LLM

tools.extend([

    {

        "type": "function",

        "function": {

            "name": "add",

            "description": "Adds two numbers together. Use for addition operations.",

            "parameters": {

                "type": "object",

                "properties": {

                    "a": {"type": "number", "description": "The first number."},

                    "b": {"type": "number", "description": "The second number."},

                },

                "required": ["a", "b"],

            },

        },

    },

    {

        "type": "function",

        "function": {

            "name": "subtract",

            "description": "Subtracts the second number from the first. Use for subtraction operations.",

            "parameters": {

                "type": "object",

                "properties": {

                    "a": {"type": "number", "description": "The number to subtract from."},

                    "b": {"type": "number", "description": "The number to subtract."},

                },

                "required": ["a", "b"],

            },

        },

    },

    {

        "type": "function",

        "function": {