Build AI Agents with Function Calling in Python

The landscape of Artificial Intelligence is evolving at an exhilarating pace, and at the forefront of this revolution are AI agents. These intelligent entities, powered by Large Language Models (LLMs), are designed not just to understand and generate human-like text, but to *act* in the world. However, LLMs alone, despite their impressive linguistic prowess, have inherent limitations: they lack real-time information, cannot directly interact with external systems, and are confined to the data they were trained on. This is where function calling emerges as a game-changer, bridging the gap between an LLM’s reasoning capabilities and the practical execution of tasks.

As a senior software engineer at Khadervali.com, I’ve seen firsthand how function calling transforms static LLMs into dynamic, problem-solving agents. In this comprehensive guide, we’ll dive deep into building robust AI agents in Python, leveraging the power of function calling to unlock their full potential. We’ll cover everything from the foundational concepts to practical implementation, architectural patterns, and real-world scenarios.

The Evolution of AI Agents and the Function Calling Paradigm

Before the advent of sophisticated LLMs, AI agents were often rule-based systems or relied on complex symbolic reasoning. While effective in narrow domains, they lacked the flexibility and generalization capabilities we now associate with AI. Modern AI agents, particularly those built with LLMs, represent a significant leap forward.

What Are AI Agents?

At their core, AI agents are systems that perceive their environment, make decisions, and take actions to achieve specific goals. Think of them as intelligent automatons that can reason about a situation, formulate a plan, and then execute that plan. The typical agentic loop looks something like this:

1. Perceive: Receive input (e.g., a user query, sensor data).
2. Plan: Interpret the input, reason about available tools/knowledge, and decide on a course of action.
3. Act: Execute the planned actions (e.g., call a function, generate a response).
4. Reflect/Learn: Evaluate the outcome and update internal state or knowledge (often implicitly in current LLM agents).

The challenge for early LLM-powered agents was that while LLMs excelled at “Perceive” and “Plan” (reasoning), their “Act” phase was limited to generating text. They couldn’t, for instance, actually *send an email* or *fetch real-time stock prices* without external help.

Enter Function Calling: The Bridge to Action

Function calling (sometimes referred to as “tool use” or “plugins”) is the mechanism that allows an LLM to interact with the outside world. Instead of just generating a text response, the LLM can now indicate that it needs to perform an action using a predefined external function. Here’s how it generally works:

1. Tool Definition: You provide the LLM with a description of available tools (functions), including their names, what they do, and their required parameters (with types).
2. Intent Recognition: When a user asks a question or gives a command, the LLM analyzes the request and determines if any of the defined tools could help fulfill it.
3. Function Call Generation: If a tool is relevant, the LLM generates a structured output (often JSON) that specifies the tool to be called and the arguments to pass to it, derived from the user’s input.
4. Orchestration and Execution: Your Python code (the agent’s orchestrator) receives this function call, executes the actual Python function corresponding to the LLM’s request, and captures its output.
5. Result Integration: The output from the executed function is then fed back to the LLM, which uses this new information to generate a final, informed, and actionable response to the user.

This iterative process allows the LLM to orchestrate complex tasks by chaining together multiple tool calls or deciding when to simply respond with text. It’s the “Aha!” moment that transforms LLMs from intelligent chatbots into powerful, actionable agents.

Architectural Blueprint of a Function-Calling Agent

Understanding the components and their interactions is crucial for building robust agents. Let’s visualize the architecture of a typical function-calling agent.

<

>

Agent Architecture Diagram (in words)

+-------------------+      +-------------------+
|    User Input     |<---->|    User Output    |
+-------------------+      +-------------------+
          |                          ^
          V                          |
+------------------------------------------------+
|           Agent Orchestrator (Python Code)     |
|   (Manages conversation, calls LLM, executes   |
|         functions, handles state/memory)       |
+------------------------------------------------+
          |                          ^
          V                          | (Function Output + Next turn messages)
+------------------------------------------------+
|       Large Language Model (LLM) Provider      |
|    (e.g., OpenAI API, Anthropic, Gemini)       |
|   (Equipped with Function Definitions/Schema)  |
+------------------------------------------------+
          | (Function Call Request - JSON)
          V
+------------------------------------------------+
|        Toolbox / Function Registry (Python)    |
|   (Collection of actual Python functions to    |
|           be called by the orchestrator)       |
+------------------------------------------------+
          | (Function Execution)
          V
+------------------------------------------------+
|           External Services / APIs             |
|   (e.g., Weather API, Calendar API, Databases, |
|           Email Service, Web Scrapers)         |
+------------------------------------------------+

Key Components Explained:

1. User Input/Output: The interface through which users interact with the agent (e.g., a chat application, command line).
2. Agent Orchestrator (Python Code): This is the brain of your agent. Written in Python, it’s responsible for:
   • Receiving user queries.
   • Maintaining conversational history (memory).
   • Calling the LLM with the current conversation state and available function definitions.
   • Parsing the LLM’s response (checking for text or function calls).
   • If a function call is detected, executing the corresponding Python function.
   • Sending the function’s output back to the LLM to get a final response.
   • Handling errors and edge cases.
3. Large Language Model (LLM) Provider: The core intelligence. You’ll interact with a service like OpenAI’s API, Anthropic’s Claude, or Google’s Gemini. The crucial aspect here is that you’ll pass not just the conversation messages, but also the *definitions* of the tools the LLM can use.
4. Function Definitions (Schema): These are structured descriptions (typically JSON schema) of the functions your agent can use. The LLM doesn’t execute the functions itself; it just reads these descriptions to understand *when* and *how* to request a function call.
5. Toolbox / Function Registry (Python): This is where your actual Python functions reside. These are the practical implementations of the tools described to the LLM. When the LLM requests a function, the orchestrator looks up and executes the corresponding function here.
6. External Services/APIs: Many of your Python functions will, in turn, interact with external services: weather APIs, databases, email services, calendar applications, custom internal tools, etc. This is how the agent truly “acts” in the real world.

This architecture decouples the LLM’s reasoning from the execution of real-world actions, making the system modular, scalable, and powerful.

Setting Up Your Python Environment

Before we dive into code, let’s ensure your Python environment is ready. We’ll use the OpenAI API for our examples, as it was one of the pioneers in popularizing function calling.

Virtual Environment

It’s always a good practice to work within a virtual environment to manage dependencies.

python -m venv agent_env
source agent_env/bin/activate  # On Windows: agent_env\Scripts\activate

Installing Dependencies

For OpenAI, you’ll need their Python client:

pip install openai python-dotenv

python-dotenv is useful for managing environment variables, especially your API key.

API Keys

Obtain an API key from your chosen LLM provider (e.g., OpenAI). Store it securely. For local development, create a .env file in your project root:

OPENAI_API_KEY="sk-YOUR_OPENAI_API_KEY_HERE"

And load it in your Python script:

import os
from dotenv import load_dotenv

load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

if not OPENAI_API_KEY:
    raise ValueError("OPENAI_API_KEY not found. Please set it in a .env file or as an environment variable.")

Practical Implementation: A Step-by-Step Guide

Let’s build a simple agent that can get the current weather and perform web searches.

Step 1: Define Your Tools (Functions)

These are standard Python functions that perform specific actions. They should be self-contained and ideally interact with external services.

# tools.py
import requests
import json

def get_current_weather(location: str, unit: str = "celsius") -> dict:
    """
    Get the current weather in a given location.
    
    Args:
        location (str): The city and state, e.g., "San Francisco, CA"
        unit (str): The unit of temperature. Can be "celsius" or "fahrenheit".
                    Defaults to "celsius".
    Returns:
        dict: A dictionary containing weather information or an error message.
    """
    # This is a placeholder for a real weather API call.
    # In a real application, you would use a service like OpenWeatherMap, AccuWeather, etc.
    print(f"DEBUG: Calling get_current_weather for {location} in {unit}")
    
    # Simulate API call with a hardcoded response for demonstration
    weather_data = {
        "San Francisco, CA": {"temperature": 18, "unit": "celsius", "description": "Cloudy"},
        "New York, NY": {"temperature": 25, "unit": "celsius", "description": "Sunny"},
        "London, UK": {"temperature": 15, "unit": "celsius", "description": "Rainy"},
    }

    if location in weather_data:
        data = weather_data[location]
        # Adjust unit if requested and different from stored
        if unit.lower() != data["unit"].lower():
            if unit.lower() == "fahrenheit" and data["unit"].lower() == "celsius":
                data["temperature"] = round((data["temperature"] * 9/5) + 32, 2)
                data["unit"] = "fahrenheit"
            elif unit.lower() == "celsius" and data["unit"].lower() == "fahrenheit":
                data["temperature"] = round((data["temperature"] - 32) * 5/9, 2)
                data["unit"] = "celsius"
        return data
    else:
        return {"error": f"Weather data not available for {location}. Try San Francisco, New York, or London."}

def search_web(query: str) -> str:
    """
    Performs a web search for the given query and returns a summary of results.
    
    Args:
        query (str): The search query.
        
    Returns:
        str: A summarized string of search results. In a real scenario, this
             would involve calling a search API (e.g., Google Search API, Bing, DuckDuckGo).
    """
    print(f"DEBUG: Calling search_web for query: '{query}'")
    
    # Placeholder for a real web search API call
    # Example using a mock API or simplified call
    try:
        # For a real implementation, you'd use a library like `duckduckgo_search` or a dedicated API
        # from duckduckgo_search import DDGS
        # with DDGS() as ddgs:
        #     results = ddgs.text(query, max_results=3)
        #     return "\n".join([f"- {r['title']}: {r['body']} ({r['href']})" for r in results])
        
        # Mock response for demonstration
        mock_responses = {
            "latest AI news": "Researchers at Google DeepMind announce new advancements in multi-modal LLMs. OpenAI releases GPT-4.5 Turbo preview. Anthropic unveils Claude 3 Opus.",
            "Python function calling": "Function calling allows LLMs to interact with external tools. Popular with OpenAI's API. Enables AI agents to perform real-world actions.",
            "Khadervali.com": "Khadervali.com is a developer portfolio and tech blog featuring articles on AI, software engineering, and web development.",
            "capital of France": "The capital of France is Paris, known for its iconic Eiffel Tower and rich history.",
            "how to cook pasta": "Boil water, add salt, cook pasta according to package directions, drain, and serve with your favorite sauce.",
        }
        
        # Simple string matching for mock responses
        for k, v in mock_responses.items():
            if k in query.lower():
                return v
        
        return f"Web search results for '{query}': No specific results found in mock data. In a real scenario, this would return actual search snippets."
    except Exception as e:
        return f"Error during web search: {e}"

# Mapping of function names to actual Python functions
AVAILABLE_TOOLS = {
    "get_current_weather": get_current_weather,
    "search_web": search_web,
}

Step 2: Describe Your Tools to the LLM

The LLM needs to know what tools are available, their purpose, and their parameters. This is done by providing a JSON schema for each function. The openai library expects a specific format.

# agent_core.py (or could be in tools.py, but for clarity, keeping separate)
from openai import OpenAI
import os
from dotenv import load_dotenv
import json
from tools import AVAILABLE_TOOLS, get_current_weather, search_web # Import actual functions

# Load environment variables
load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

if not OPENAI_API_KEY:
    raise ValueError("OPENAI_API_KEY not found. Please set it in a .env file or as an environment variable.")

client = OpenAI(api_key=OPENAI_API_KEY)

# Define the tools in the format expected by OpenAI
tools_definition = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g., 'San Francisco, CA'",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Performs a web search for the given query and returns a summary of results.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "The search query, e.g., 'latest AI news'",
                    },
                },
                "required": ["query"],
            },
        },
    },
]

Notice the detailed description for the function and its parameters. The LLM relies heavily on these descriptions to understand when to call a tool and what arguments to provide. The enum for unit is a great way to constrain the LLM’s output for specific parameters.

Step 3: The Agent Orchestration Loop (Core Logic)

This is where the magic happens. The orchestrator manages the conversation, calls the LLM, detects function calls, executes them, and feeds the results back to the LLM.

# agent_core.py (continued)

def run_conversation(user_message: str, conversation_history: list = None) -> str:
    """
    Manages the conversation with the LLM, including function calling.
    
    Args:
        user_message (str): The initial message from the user.
        conversation_history (list): A list of previous messages in OpenAI format.
    
    Returns:
        str: The final response from the LLM.
    """
    if conversation_history is None:
        conversation_history = []

    # Add the user's message to the conversation history
    conversation_history.append({"role": "user", "content": user_message})

    # Step 1: Send the conversation and available tools to the LLM
    print(f"\n--- User: {user_message}")
    
    response = client.chat.completions.create(
        model="gpt-4o", # Or "gpt-3.5-turbo", "gpt-4-turbo-preview" etc.
        messages=conversation_history,
        tools=tools_definition,
        tool_choice="auto",  # Allows the LLM to decide whether to call a tool or respond
    )
    
    response_message = response.choices[0].message
    print(f"--- LLM initial response: {response_message}")

    # Step 2: Check if the LLM wants to call a function
    if response_message.tool_calls:
        print(f"--- LLM requested tool call(s): {response_message.tool_calls}")
        # Add the LLM's tool call request to the conversation history
        conversation_history.append(response_message)
        
        # Execute each tool call requested by the LLM
        for tool_call in response_message.tool_calls:
            function_name = tool_call.function.name
            function_to_call = AVAILABLE_TOOLS.get(function_name)
            
            if not function_to_call:
                print(f"ERROR: Function {function_name} not found in AVAILABLE_TOOLS.")
                # You might want to handle this more gracefully, e.g., inform the LLM
                conversation_history.append({
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": f"Error: Function {function_name} not available.",
                })
                continue

            function_args = json.loads(tool_call.function.arguments)
            print(f"--- Executing function: {function_name} with args: {function_args}")
            
            try:
                # Step 3: Call the function and get its result
                function_output = function_to_call(**function_args)
                print(f"--- Function '{function_name}' output: {function_output}")

                # Step 4: Send the function output back to the LLM
                conversation_history.append(
                    {
                        "tool_call_id": tool_call.id,
                        "role": "tool",
                        "name": function_name,
                        "content": json.dumps(function_output), # Function output should be stringified
                    }
                )
            except Exception as e:
                print(f"ERROR: Function '{function_name}' failed with error: {e}")
                conversation_history.append({
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": f"Error executing function: {e}",
                })

        # After executing tools, make another call to the LLM to get a final response
        # The LLM now has the original user message, its decision to call tools, and the tool results
        print("\n--- Sending tool outputs back to LLM for final response...")
        second_response = client.chat.completions.create(
            model="gpt-4o",
            messages=conversation_history,
        )
        final_message = second_response.choices[0].message.content
        print(f"--- LLM final response: {final_message}")
        conversation_history.append({"role": "assistant", "content": final_message}) # Add final response to history
        return final_message
    else:
        # If no tool call was made, the LLM's initial response is the final one
        final_message = response_message.content
        print(f"--- LLM responded directly: {final_message}")
        conversation_history.append({"role": "assistant", "content": final_message}) # Add direct response to history
        return final_message

# Example usage
if __name__ == "__main__":
    
    # Initialize conversation history
    messages = []
    
    print("AI Agent Ready! Type 'exit' to quit.")
    
    while True:
        user_input = input("\nYou: ")
        if user_input.lower() == 'exit':
            break
        
        response = run_conversation(user_input, messages)
        print(f"Agent: {response}")

        # Keep only the last N messages to prevent context window overflow in longer conversations
        # For simplicity, we are accumulating all messages here.
        # In a real app, you'd manage memory carefully.

To run this, make sure you have tools.py and agent_core.py in the same directory, and your .env file is set up correctly.

Example Interaction:

You: What's the weather like in San Francisco?

--- User: What's the weather like in San Francisco?

--- LLM initial response: ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_wQG...', function=Function(arguments='{"location": "San Francisco, CA", "unit": "celsius"}', name='get_current_weather'), type='function')])

--- LLM requested tool call(s): [ChatCompletionMessageToolCall(id='call_wQG...', function=Function(arguments='{"location": "San Francisco, CA", "unit": "celsius"}', name='get_current_weather'), type='function')]

--- Executing function: get_current_weather with args: {'location': 'San Francisco, CA', 'unit': 'celsius'}

DEBUG: Calling get_current_weather for San Francisco, CA in celsius

--- Function 'get_current_weather' output: {'temperature': 18, 'unit': 'celsius', 'description': 'Cloudy'}

--- Sending tool outputs back to LLM for final response...

--- LLM final response: The current weather in San Francisco, CA is 18 degrees Celsius and cloudy.

Agent: The current weather in San Francisco, CA is 18 degrees Celsius and cloudy.
You: Search for the capital of France.

--- User: Search for the capital of France.

--- LLM initial response: ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_lqP...', function=Function(arguments='{"query": "capital of France"}', name='search_web'), type='function')])

--- LLM requested tool call(s): [ChatCompletionMessageToolCall(id='call_lqP...', function=Function(arguments='{"query": "capital of France"}', name='search_web'), type='function')]

--- Executing function: search_web with args: {'query': 'capital of France'}

DEBUG: Calling search_web for query: 'capital of France'

--- Function 'search_web' output: The capital of France is Paris, known for its iconic Eiffel Tower and rich history.
--- Sending tool outputs back to LLM for final response...

--- LLM final response: The capital of France is Paris.

Agent: The capital of France is Paris.
You: What's new in AI according to the web?

--- User: What's new in AI according to the web?

--- LLM initial response: ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_f8R...', function=Function(arguments='{"query": "latest AI news"}', name='search_web'), type='function')])

--- LLM requested tool call(s): [ChatCompletionMessageToolCall(id='call_f8R...', function=Function(arguments='{"query": "latest AI news"}', name='search_web'), type='function')]

---