Routing

Configure intelligent routing between AI providers with automatic fallbacks and load balancing.

Fallback Configuration

Define backup models that will be used if the primary model fails or is unavailable:

const response = await client.gateway.chat({
  model: "gpt-4",
  fallbackModels: [
    "claude-3-opus",
    "gemini-pro",
    "gpt-3.5-turbo"
  ],
  messages: [{ role: "user", content: "Hello!" }]
});

Load Balancing

Distribute requests across multiple providers based on various strategies:

Round Robin

Distribute requests evenly across all configured providers.

Weighted

Assign weights to providers to control traffic distribution.

Latency-Based

Route to the provider with the lowest response time.

Provider Priority

// Configure routing in your dashboard or via API
const routingConfig = {
  strategy: "weighted",
  providers: [
    { model: "gpt-4", weight: 70 },
    { model: "claude-3-opus", weight: 30 }
  ],
  fallbackOnError: true,
  maxRetries: 3
};