Caching

Enable response caching to reduce costs and improve response times.

Enable Caching

const response = await client.gateway.chat({
  model: "gpt-4",
  messages: [{ role: "user", content: "What is AI?" }],
  cache: {
    enabled: true,
    ttl: 3600,
    key: "custom-cache-key"
  }
});

// Check if response was from cache
console.log('From cache:', response.cached);

Cache Configuration

TTL (Time to Live)

How long to cache responses (in seconds).

Cache Key

Custom key for cache lookup. Default uses request hash.

Semantic Caching

Match semantically similar requests for cache hits.

Cache Statistics

const stats = await client.cache.stats();
console.log('Hit rate:', stats.hitRate);
console.log('Entries:', stats.entries);
console.log('Saved cost:', stats.savedCost);