Caching

​

Overview

1. Reducing Computational Overhead: Caching expensive retrieval operations and feature computations
2. Improving Response Times: Serving frequently accessed results from cache instead of recomputing
3. Optimizing Resource Usage: Minimizing redundant processing of the same queries
4. Enabling Scalability: Supporting high-throughput retrieval operations efficiently

1. Retriever-level Caching: Caches the results of specific retriever operations, particularly useful for:
   • Frequently repeated queries
   • Expensive vector similarity searches
   • Complex multimodal retrieval pipelines
   • High-traffic search endpoints
2. Collection-level Caching: Caches collection-specific data and metadata, beneficial for:
   • Frequently accessed documents
   • Stable reference data
   • Metadata that doesn’t change often
   • Feature store lookups

​

Cache Configuration

​

Retriever Cache Configuration

{
  "enabled": true,
  "ttl_override_seconds": 3600,
  "score_threshold": 0.8,
  "invalidation_events": [
    {
      "collection": "my_collection",
      "action": "update",
      "recompute_strategy": "BACKGROUND"
    }
  ],
  "max_entries_per_key": 5
}

​

Collection Cache Configuration

{
  "enabled": true,
  "ttl_seconds": 604800,
  "invalidation_strategy": "REPLACE_IF_BETTER",
  "max_entries_per_key": 5
}

​

Cache Invalidation Strategies

1. REPLACE_IF_BETTER: Replace cache entry only if new score is better
2. REPLACE_ALWAYS: Always replace cache entry on invalidation
3. KEEP_EXISTING: Keep existing cache entry regardless of new data

​

Recompute Strategies

1. IMMEDIATE: Recompute the result immediately
2. BACKGROUND: Recompute in the background
3. LAZY: Recompute only when the result is next accessed

​

Cache Entry Structure

• key: Unique identifier for the cache entry
• value: The cached result
• score: Score of the cached result
• created_at: Timestamp of creation
• expires_at: Optional expiration timestamp

​

Usage with Collections

1. Collection-level caching is controlled by CollectionCacheConfig
2. Cache entries are automatically invalidated when collection data changes
3. Collections can specify their own TTL and invalidation strategies

​

Usage with Retrievers

1. Retriever-level caching is controlled by RetrieverCacheConfig
2. Cache keys are generated based on retriever ID and source document ID
3. Results are cached only if they meet the score threshold
4. Cache invalidation can be triggered by specific events

​

Best Practices

1. Configure Appropriate TTLs: Set TTLs based on data volatility
2. Use Score Thresholds: Only cache high-quality results
3. Monitor Cache Statistics: Track hit ratios and adjust configurations
4. Implement Proper Invalidation: Configure invalidation events for data changes
5. Balance Cache Size: Use max_entries_per_key to prevent cache bloat

​

Example Usage

​

Configure Retriever Caching

curl -X POST https://api.mixpeek.com/v1/retrievers/{retriever_id}/cache \
  -H "Authorization: Bearer {api_key}" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "score_threshold": 0.8,
    "ttl_override_seconds": 3600,
    "invalidation_events": [
      {
        "collection": "my_collection",
        "action": "update",
        "recompute_strategy": "BACKGROUND"
      }
    ]
  }'

​

Configure Collection Caching

curl -X POST https://api.mixpeek.com/v1/collections/{collection_id}/cache \
  -H "Authorization: Bearer {api_key}" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "ttl_seconds": 86400,
    "invalidation_strategy": "REPLACE_IF_BETTER"
  }'

​

Get Cache Statistics

curl -X GET https://api.mixpeek.com/v1/cache/stats \
  -H "Authorization: Bearer {api_key}"

​

Invalidate Cache

curl -X POST https://api.mixpeek.com/v1/cache/invalidate \
  -H "Authorization: Bearer {api_key}" \
  -H "Content-Type: application/json" \
  -d '{
    "retriever_id": "my_retriever",
    "collection_id": "my_collection",
    "action": "update"
  }'

​

Cache Key Generation

• Retriever ID
• Query parameters
• Source document IDs
• Collection IDs
• Timestamp (for time-sensitive queries)

• Unique keys for different queries
• Consistent keys for identical queries
• Proper cache invalidation when data changes

​

Next Steps