API Gateway Patterns: Routing, Auth, Rate Limiting

In the evolving landscape of modern software architecture, particularly with the rise of microservices, managing the deluge of incoming requests and orchestrating interactions between numerous backend services can quickly become a tangled mess. This is where the API Gateway pattern emerges as a foundational and indispensable component. Acting as a single entry point for all client requests, an API Gateway centralizes crucial cross-cutting concerns, providing a streamlined and secure interface to your backend ecosystem.

At Khadervali.com, we champion elegant and efficient solutions for complex distributed systems. In this comprehensive technical article, we’ll dive deep into the core functionalities of an API Gateway, exploring three fundamental patterns: Request Routing, Authentication and Authorization, and Rate Limiting. We’ll examine the “why” behind each, discuss various implementation strategies, provide illustrative code examples, and outline real-world scenarios to solidify your understanding.

The API Gateway Pattern Explained

Before microservices became the de facto standard for building scalable applications, a typical monolithic application would expose its entire API surface directly to clients. While simple for smaller projects, this approach quickly reveals its limitations as the application grows:

• Tight Coupling: Client applications become directly coupled to specific backend service endpoints. Changes to backend services often necessitate changes in client code.
• Cross-Cutting Concerns Duplication: Each service might need to handle authentication, rate limiting, logging, and monitoring independently, leading to redundant code and inconsistent policies.
• Security Vulnerabilities: Exposing internal service details directly can create security risks.
• Complexity for Clients: Clients need to know the location and interface of multiple services to perform a single business operation, increasing client-side complexity.

The API Gateway solves these challenges by acting as a facade, abstracting the internal architecture of your microservices from the clients. It intercepts all incoming requests, processes them, and routes them to the appropriate backend service. This central point of control allows for the consolidation of various concerns:

• Abstraction: Hides the complexity and number of backend microservices from clients.
• Centralized Management: Provides a single place to manage policies, security, and traffic.
• Performance & Scalability: Can implement caching, load balancing, and connection pooling.
• Security: Enforces authentication, authorization, and other security policies before requests reach backend services.
• Monitoring & Observability: Offers a central point for logging, metrics collection, and tracing.

Consider the architectural shift:

    Without API Gateway:

    Client
    |---> Service A (Auth, Rate Limit, Logic)
    |---> Service B (Auth, Rate Limit, Logic)
    |---> Service C (Auth, Rate Limit, Logic)


    With API Gateway:

    Client
    |
    |---> API Gateway (Auth, Rate Limit, Logging, Routing)
            |
            |---> Service A (Logic)
            |---> Service B (Logic)
            |---> Service C (Logic)

The API Gateway typically comprises several key components:

• Proxy/Reverse Proxy: The core component that accepts incoming requests and forwards them to upstream services.
• Routing Engine: Determines which backend service should receive a request based on rules.
• Policy Engine: Applies cross-cutting policies like authentication, rate limiting, and transformations.
• Service Discovery Client: Integrates with a service discovery system to find available service instances.

This architectural pattern significantly simplifies client interactions, enhances security, and provides a robust foundation for building scalable and maintainable microservices applications.

Core Functionality 1: Request Routing

At its heart, an API Gateway is a sophisticated reverse proxy whose primary function is to intelligently route incoming client requests to the correct backend service. This seemingly simple task becomes critical in a microservices environment where multiple services, potentially running on different hosts and ports, need to be accessible through a single, unified entry point.

Basic Routing Strategies

The most common routing strategies are based on the incoming request’s URL path or host header.

Path-Based Routing

This is perhaps the most intuitive method. The API Gateway inspects the URL path of the incoming request and forwards it to a specific service based on a predefined mapping. For example:

• /users/* routes to the User Service
• /products/* routes to the Product Catalog Service
• /orders/* routes to the Order Service

This allows clients to interact with distinct business capabilities using a single base URL, e.g., api.example.com/users instead of users.example.com.

Host-Based Routing

Less common in a single-API-gateway setup, but relevant for multi-tenant or domain-specific routing. Requests arriving at different hostnames can be routed to different services or service versions. For instance:

• api.users.example.com routes to User Service
• api.products.example.com routes to Product Service

Advanced Routing Strategies

Beyond basic path or host matching, API Gateways can employ more sophisticated routing rules to support advanced deployment patterns and A/B testing.

Header and Query Parameter-Based Routing

Routing decisions can be made based on HTTP headers or query parameters. This is particularly useful for:

• API Versioning: Clients can specify a desired API version in a header (e.g., X-API-Version: 2) or query parameter (e.g., ?v=2), routing requests to different versions of a service.
• Tenant-Specific Routing: In multi-tenant applications, a X-Tenant-ID header could route requests to specific service instances or configurations tailored for that tenant.

Canary Releases and A/B Testing

These strategies are crucial for safely deploying new features and evaluating user experience:

• Canary Release: A small percentage of production traffic is routed to a new version of a service (the “canary”), while the majority still goes to the stable version. The API Gateway can route, for example, 5% of requests to Service A v2 and 95% to Service A v1. If the canary performs well, the traffic split can be gradually increased.
• A/B Testing: Different user segments are routed to different versions of a service or feature, allowing for direct comparison of their impact. This could be based on user ID, cookie, geographical location, or other attributes.

Service Discovery Integration

In dynamic microservices environments, service instances are often ephemeral, scaling up and down based on demand. Hardcoding service addresses in the API Gateway is impractical and fragile. This is where service discovery becomes vital.

An API Gateway typically integrates with a service discovery system (e.g., Consul, Eureka, Kubernetes Service, DNS). When a request needs to be routed to a service, the gateway queries the service discovery system to get the current network location (IP address and port) of an available instance of that service. This ensures that the gateway always routes to healthy and available service instances.

Example: Routing Logic in Go (Pseudo-code)

Here’s a simplified illustration of how an API Gateway might implement routing using Go’s net/http/httputil package for reverse proxying. In a real-world scenario, you’d integrate with a routing library like Gorilla Mux or Chi, and service discovery.

package main

import (
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
)

// ServiceRegistry maps paths to upstream service URLs
type ServiceRegistry map[string]*url.URL

// NewProxy creates a reverse proxy for a given target URL
func NewProxy(targetURL *url.URL) *httputil.ReverseProxy {
	proxy := httputil.NewSingleHostReverseProxy(targetURL)
	return proxy
}

// GatewayHandler handles incoming requests and routes them
func GatewayHandler(registry ServiceRegistry) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		// Basic path-based routing
		var target *url.URL
		path := r.URL.Path

		if path == "/users" || path == "/users/" {
			target = registry["users"]
		} else if path == "/products" || path == "/products/" {
			target = registry["products"]
		} else if path == "/orders" || path == "/orders/" {
			target = registry["orders"]
		} else {
			http.Error(w, "Not Found", http.StatusNotFound)
			return
		}

		if target == nil {
			http.Error(w, "Service Unavailable", http.StatusServiceUnavailable)
			return
		}

		log.Printf("Routing request from %s to %s for path %s", r.RemoteAddr, target.String(), path)

		// Create a new proxy for each request to ensure URL is correctly rewritten
		// In a real gateway, proxies might be cached or managed more efficiently.
		proxy := NewProxy(target)
		proxy.ServeHTTP(w, r)
	}
}

func main() {
	// In a real application, these URLs would come from service discovery
	userServiceURL, _ := url.Parse("http://localhost:8081")
	productServiceURL, _ := url.Parse("http://localhost:8082")
	orderServiceURL, _ := url.Parse("http://localhost:8083")

	registry := ServiceRegistry{
		"users":    userServiceURL,
		"products": productServiceURL,
		"orders":   orderServiceURL,
	}

	mux := http.NewServeMux()
	mux.HandleFunc("/", GatewayHandler(registry)) // Catch all paths for gateway handler

	log.Println("API Gateway listening on :8080")
	log.Fatal(http.ListenAndServe(":8080", mux))
}

This example demonstrates a rudimentary path-based router. In a production-grade gateway, the routing logic would be significantly more complex, involving regular expressions for path matching, dynamic rule loading, and integration with a service discovery client to resolve service URLs dynamically instead of hardcoding them.

Core Functionality 2: Authentication and Authorization

Security is paramount for any API, and an API Gateway serves as a critical enforcement point for both authentication (verifying a user’s identity) and authorization (determining what an authenticated user is allowed to do). Centralizing these concerns at the gateway offers significant advantages:

• Single Point of Enforcement: Ensures consistent security policies across all services.
• Offloads Backend Services: Backend services can focus purely on business logic, as they can trust that requests reaching them have already been authenticated and authorized.
• Simplified Client Development: Clients interact with a single security mechanism at the gateway.
• Reduced Attack Surface: Unauthorized requests are rejected at the edge of the network, protecting internal services.

Authentication Methods at the Gateway

An API Gateway can support various authentication mechanisms:

API Keys

The simplest form, where a unique key is sent with each request (e.g., in a header or query parameter). The gateway validates the key against a store (database, cache) and can associate it with a user or application. While easy to implement, API keys are less secure than token-based approaches as they are often long-lived and require careful management.

Basic Authentication

Uses a username and password encoded in Base64 in the Authorization header. The gateway can validate these credentials against an identity store. Suitable for internal APIs or when simplicity is prioritized over advanced security features.

JSON Web Tokens (JWT)

JWTs are a popular and robust method for authentication in distributed systems. A JWT is a compact, URL-safe means of representing claims to be transferred between two parties. The claims in a JWT are encoded as a JSON object that is digitally signed.

A JWT typically consists of three parts, separated by dots (.):

1. Header: Contains the type of token (JWT) and the signing algorithm (e.g., HS256, RS256).
2. Payload: Contains the claims (statements about an entity, typically the user, and additional data). Common claims include iss (issuer), exp (expiration time), sub (subject), and custom application-specific claims like user roles or IDs.
3. Signature: Created by taking the encoded header, the encoded payload, a secret (for HS256) or a private key (for RS256), and signing them.

When a client presents a JWT to the API Gateway, the gateway performs the following steps:

1. Verification: Decodes the header and payload.
2. Signature Validation: Validates the signature using a known public key (for asymmetric encryption) or a shared secret (for symmetric encryption). This ensures the token hasn’t been tampered with.
3. Claim Validation: Checks claims like expiration time (exp), issuer (iss), audience (aud) to ensure the token is valid and intended for this service.
4. Extract Claims: Extracts relevant claims (e.g., user ID, roles) from the payload.

After successful validation, the gateway can forward the request to the backend service. It might:

• Pass the original JWT.
• Strip the JWT and inject the extracted claims into new headers (e.g., X-User-ID, X-User-Roles) for the backend service to consume. This offloads JWT parsing from backend services.

OAuth 2.0 and OpenID Connect (OIDC)

OAuth 2.0 is an authorization framework that enables an application to obtain limited access to a user’s resources on an HTTP service. OpenID Connect is an identity layer built on top of OAuth 2.0, allowing clients to verify the identity of the end-user based on authentication performed by an authorization server.

In this scenario, the API Gateway acts as a resource server. It receives access tokens (which are often JWTs) issued by an OAuth 2.0 Authorization Server or an OIDC Identity Provider (e.g., Auth0, Okta, Keycloak). The gateway validates these tokens, often by calling the Authorization Server’s introspection endpoint or by validating the JWT signature and claims locally using the Authorization Server’s public keys.

Authorization at the Gateway

Once a user is authenticated, authorization determines if they have permission to access a specific resource or perform an action. The gateway can enforce authorization policies based on information extracted during authentication (e.g., user roles, permissions from JWT claims).

• Role-Based Access Control (RBAC): The gateway checks if the authenticated user’s role (e.g., “admin”, “customer”) is permitted to access the requested endpoint.
• Attribute-Based Access Control (ABAC): More granular, ABAC considers a combination of user attributes (e.g., department, geographical location), resource attributes (e.g., resource owner, sensitivity), and environmental attributes (e.g., time of day) to make authorization decisions.

Example: JWT Validation Middleware in Go

This Go example shows a simple middleware that validates a JWT found in the Authorization header. It assumes a shared secret for simplicity (HS256) but in production, RS256 with public/private key pairs is generally preferred for better security and scalability.

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"strings"
	"time"

	"github.com/golang-jwt/jwt/v5" // Using jwt v5 for modern usage
)

// Define a context key for passing user info downstream
type contextKey string
const userContextKey contextKey = "userInfo"

// Claims struct for our JWT
type MyCustomClaims struct {
	UserID string   `json:"user_id"`
	Roles  []string `json:"roles"`
	jwt.RegisteredClaims
}

// JWTSecret is the secret key for signing/verifying JWTs.
// In a real app, this would be loaded securely (e.g., from environment variable, vault).
var JWTSecret = []byte("supersecretjwtkeythatshouldbemoresecure")

// JWTAuthMiddleware validates the JWT token in the Authorization header.
func JWTAuthMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		authHeader := r.Header.Get("Authorization")
		if authHeader == "" {
			http.Error(w, "Authorization header required", http.StatusUnauthorized)
			return
		}

		parts := strings.Split(authHeader, " ")
		if len(parts) != 2 || parts[0] != "Bearer" {
			http.Error(w, "Authorization header format must be Bearer {token}", http.StatusUnauthorized)
			return
		}

		tokenString := parts[1]

		// Parse and validate the token
		token, err := jwt.ParseWithClaims(tokenString, &MyCustomClaims{}, func(token *jwt.Token) (interface{}, error) {
			// Validate the alg is what we expect
			if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
				return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
			}
			return JWTSecret, nil
		})

		if err != nil {
			log.Printf("JWT validation error: %v", err)
			http.Error(w, "Invalid or expired token", http.StatusUnauthorized)
			return
		}

		if claims, ok := token.Claims.(*MyCustomClaims); ok && token.Valid {
			log.Printf("Authenticated UserID: %s, Roles: %v", claims.UserID, claims.Roles)

			// Add user info to the request context for downstream services
			ctx := context.WithValue(r.Context(), userContextKey, claims)
			r = r.WithContext(ctx)

			next.ServeHTTP(w, r) // Proceed to the next handler
		} else {
			http.Error(w, "Invalid token claims", http.StatusUnauthorized)
		}
	})
}

// UserInfoFromContext retrieves user claims from the request context
func UserInfoFromContext(ctx context.Context) (*MyCustomClaims, bool) {
	claims, ok := ctx.Value(userContextKey).(*MyCustomClaims)
	return claims, ok
}

// ProtectedHandler is an example backend service handler
func ProtectedHandler(w http.ResponseWriter, r *http.Request) {
	claims, ok := UserInfoFromContext(r.Context())
	if !ok {
		http.Error(w, "User info not found in context", http.StatusInternalServerError)
		return
	}

	// Example authorization check: only admins can access
	isAdmin := false
	for _, role := range claims.Roles {
		if role == "admin" {
			isAdmin = true
			break
		}
	}

	if !isAdmin {
		http.Error(w, "Access forbidden: Admins only", http.StatusForbidden)
		return
	}

	fmt.Fprintf(w, "Welcome, admin user %s! You have access to protected data.", claims.UserID)
}

// Simulate token generation for testing (DO NOT use in production client-side)
func generateTestToken(userID string, roles []string) (string, error) {
	claims := MyCustomClaims{
		UserID: userID,
		Roles:  roles,
		RegisteredClaims: jwt.RegisteredClaims{
			ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour * 24)), // Token expires in 24 hours
			IssuedAt:  jwt.NewNumericDate(time.Now()),
			NotBefore: jwt.NewNumericDate(time.Now()),
			Issuer:    "khadervali.com",
			Subject:   userID,
			ID:        "unique-session-id",
			Audience:  []string{"api-gateway"},
		},
	}

	token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
	tokenString, err := token.SignedString(JWTSecret)
	return tokenString, err
}


func main() {
	// Setup a simple router
	mux := http.NewServeMux()

	// Apply JWT middleware to a protected route
	mux.Handle("/protected", JWTAuthMiddleware(http.HandlerFunc(ProtectedHandler)))

	// Example: Generate a test token for an admin
	adminToken, err := generateTestToken("adminUser123", []string{"admin", "user"})
	if err != nil {
		log.Fatalf("Error generating admin token: %v", err)
	}
	log.Printf("Generated Admin Token (use in Authorization: Bearer header):\n%s\n", adminToken)

	// Example: Generate a test token for a regular user (will be forbidden from /protected)
	userToken, err := generateTestToken("regularUser456", []string{"user"})
	if err != nil {
		log.Fatalf("Error generating user token: %v", err)
	}
	log.Printf("Generated Regular User Token:\n%s\n", userToken)


	log.Println("API Gateway with JWT Auth listening on :8080")
	log.Fatal(http.ListenAndServe(":8080", mux))
}

This code illustrates how the gateway can:

1. Intercept requests.
2. Extract and validate a JWT.
3. Add authenticated user information to the request context.
4. Perform basic authorization based on roles before passing the request to the actual backend handler (simulated here by ProtectedHandler).

Core Functionality 3: Rate Limiting

Rate limiting is a critical control mechanism that restricts the number of requests a user or client can make to an API within a given timeframe. Implementing rate limiting at the API Gateway is highly effective for several reasons:

• Prevent Abuse: Protects against malicious activities like brute-force attacks, DDoS (Distributed Denial of Service) attempts, and web scraping.
• Ensure Fair Usage: Prevents any single client from monopolizing resources, ensuring equitable access for all users.
• Control Costs: For cloud-based services, limiting requests can help manage infrastructure costs by preventing excessive scaling triggered by high, uncontrolled traffic.
• Maintain Stability: Protects backend services from being overwhelmed by unexpected traffic spikes.

When a client exceeds the defined rate limit, the API Gateway typically responds with an HTTP 429 Too Many Requests status code and often includes a Retry-After header, indicating how long the client should wait before making another request.

Rate Limiting Algorithms

Several algorithms are commonly used for rate limiting, each with its own characteristics and trade-offs:

Fixed Window Counter

This is the simplest algorithm. It divides time into fixed-size windows (e.g., 1 minute). Each window has a counter, which increments with every request. If the counter exceeds the limit within the window, subsequent requests are blocked until the next window starts.
Pros: Simple to implement, low memory usage.
Cons: Can suffer from a “bursty” problem at the window edges. If a client makes N requests just before the window ends and N more just after it starts, they effectively make 2N requests within a short period (close to the window boundary).

Sliding Log

This algorithm tracks a timestamp for every request made by a client. To check if a request is allowed, it counts all timestamps within the last T seconds. If the count exceeds the limit, the request is blocked. Old timestamps are eventually purged.
Pros: Very precise, no “bursty” problem.
Cons: High memory consumption, especially for high