Explain Retriever Execution Plan

curl --request POST \
  --url https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute/explain \
  --header 'Content-Type: application/json' \
  --data '{
  "inputs": {}
}'

{
  "retriever_id": "<string>",
  "retriever_name": "<string>",
  "estimated_cost": {},
  "total_estimated_stages": 1,
  "execution_plan": [
    {
      "stage_index": 1,
      "stage_name": "<string>",
      "stage_type": "<string>",
      "estimated_input": 1,
      "estimated_output": 1,
      "estimated_efficiency": 1,
      "estimated_cost_credits": 1,
      "estimated_duration_ms": 1,
      "cache_likely": true,
      "optimization_notes": [
        "<string>"
      ],
      "warnings": [
        "<string>"
      ]
    }
  ],
  "optimization_suggestions": [
    {}
  ],
  "bottleneck_stages": [
    "<string>"
  ],
  "optimization_level": "mvp",
  "optimization_applied": false,
  "optimization_details": {
    "decisions": [
      {
        "applied": true,
        "reason": "Moved attribute_filter before semantic_search to reduce search scope by 50%",
        "rule_type": "push_down_filters",
        "stages_after": 5,
        "stages_before": 5
      },
      {
        "applied": true,
        "reason": "Merged group_by into feature_filter for database-level grouping (10x faster)",
        "rule_type": "group_by_push_down",
        "stages_after": 3,
        "stages_before": 5
      }
    ],
    "optimization_time_ms": 8.2,
    "optimized_stage_count": 3,
    "original_stage_count": 5,
    "stage_reduction_pct": 40
  }
}

Retrievers

Explain Retriever Execution Plan

Get a detailed execution plan for a retriever without actually executing it. Similar to MongoDB’s explain plan or SQL’s EXPLAIN command, this endpoint helps you understand performance characteristics, identify bottlenecks, estimate costs, and troubleshoot retrieval issues before running expensive queries.

What This Returns:

Stage-by-stage execution plan (AFTER automatic optimizations)
Estimated costs (credits + time per stage)
Document flow projections (input/output counts per stage)
Efficiency metrics (selectivity ratios, cache likelihood)
Bottleneck identification (slowest/most expensive stages)
Optimization details (transformations applied by the optimizer)
Performance warnings and improvement suggestions

Key Features:

Cost Estimation: See how many credits and milliseconds each stage will consume
Bottleneck Detection: Identify which stages dominate execution time
Optimization Transparency: Understand how your pipeline was optimized
Cache Analysis: See which stages are likely to hit cache
Accuracy Troubleshooting: Analyze stage efficiency and document flow
Latency Analysis: Break down estimated duration by stage

Important: The execution_plan shows OPTIMIZED stages (after automatic transformations like filter push-down, stage fusion, and grouping optimization). Check optimization_details to understand what changed from your original configuration.

Use Cases:

Debug slow retrievers by identifying bottleneck stages
Estimate costs before running expensive queries
Understand how the optimizer transformed your pipeline
Troubleshoot accuracy issues by analyzing stage selectivity
Compare different retriever configurations
Plan budget allocation for production workloads

Example Response:

{
  "retriever_id": "ret_abc123",
  "retriever_name": "product_search",
  "execution_plan": [
    {
      "stage_index": 0,
      "stage_name": "attribute_filter",
      "stage_type": "filter",
      "estimated_input": 10000,
      "estimated_output": 5000,
      "estimated_efficiency": 0.5,
      "estimated_cost_credits": 0.01,
      "estimated_duration_ms": 20,
      "cache_likely": true,
      "optimization_notes": ["Pushed down from stage 2"],
      "warnings": []
    },
    {
      "stage_index": 1,
      "stage_name": "semantic_search",
      "stage_type": "filter",
      "estimated_input": 5000,
      "estimated_output": 100,
      "estimated_efficiency": 0.02,
      "estimated_cost_credits": 0.5,
      "estimated_duration_ms": 200,
      "cache_likely": false,
      "optimization_notes": [],
      "warnings": ["High cost stage - consider reducing limit"]
    }
  ],
  "estimated_cost": {
    "total_credits": 0.51,
    "total_duration_ms": 220
  },
  "bottleneck_stages": ["semantic_search"],
  "optimization_applied": true,
  "optimization_details": {
    "original_stage_count": 3,
    "optimized_stage_count": 2,
    "optimization_time_ms": 8.2,
    "stage_reduction_pct": 33.3,
    "decisions": [
      {
        "rule_type": "push_down_filters",
        "applied": true,
        "reason": "Moved attribute_filter before semantic_search to reduce search scope"
      }
    ]
  },
  "optimization_suggestions": [
    {
      "type": "reduce_limit",
      "stage": "semantic_search",
      "message": "Consider reducing limit to improve latency"
    }
  ]
}

POST

retrievers

{retriever_id}

execute

explain

Explain Retriever Execution Plan

curl --request POST \
  --url https://api.mixpeek.com/v1/retrievers/{retriever_id}/execute/explain \
  --header 'Content-Type: application/json' \
  --data '{
  "inputs": {}
}'

{
  "retriever_id": "<string>",
  "retriever_name": "<string>",
  "estimated_cost": {},
  "total_estimated_stages": 1,
  "execution_plan": [
    {
      "stage_index": 1,
      "stage_name": "<string>",
      "stage_type": "<string>",
      "estimated_input": 1,
      "estimated_output": 1,
      "estimated_efficiency": 1,
      "estimated_cost_credits": 1,
      "estimated_duration_ms": 1,
      "cache_likely": true,
      "optimization_notes": [
        "<string>"
      ],
      "warnings": [
        "<string>"
      ]
    }
  ],
  "optimization_suggestions": [
    {}
  ],
  "bottleneck_stages": [
    "<string>"
  ],
  "optimization_level": "mvp",
  "optimization_applied": false,
  "optimization_details": {
    "decisions": [
      {
        "applied": true,
        "reason": "Moved attribute_filter before semantic_search to reduce search scope by 50%",
        "rule_type": "push_down_filters",
        "stages_after": 5,
        "stages_before": 5
      },
      {
        "applied": true,
        "reason": "Merged group_by into feature_filter for database-level grouping (10x faster)",
        "rule_type": "group_by_push_down",
        "stages_after": 3,
        "stages_before": 5
      }
    ],
    "optimization_time_ms": 8.2,
    "optimized_stage_count": 3,
    "original_stage_count": 5,
    "stage_reduction_pct": 40
  }
}

Headers

Authorization

string

REQUIRED: Bearer token authentication using your API key. Format: 'Bearer sk_xxxxxxxxxxxxx'. You can create API keys in the Mixpeek dashboard under Organization Settings.

X-Namespace

string

REQUIRED: Namespace identifier for scoping this request. All resources (collections, buckets, taxonomies, etc.) are scoped to a namespace. You can provide either the namespace name or namespace ID. Format: ns_xxxxxxxxxxxxx (ID) or a custom name like 'my-namespace'

Path Parameters

retriever_id

string

required

Retriever ID or name to explain. The execution plan will show the OPTIMIZED version after automatic transformations.

Body

application/json

Optional hypothetical inputs for estimation. Provide sample inputs to see how the plan changes with different parameters (e.g., different top_k values, query complexity, filter combinations). If not provided, uses default/representative values.

inputs

Inputs · object

Hypothetical inputs for tailored execution plan estimation. These values are used to analyze stage behavior and estimate costs.

NOT REQUIRED - if omitted, default/representative values are used.

Common inputs:

'query': Search query text (for semantic search stages)
'top_k': Number of results to return (affects search scope)
Filter parameters: Category, price range, etc.

Examples:

{'query': 'laptop'} - Simple text query
{'query': 'laptop', 'top_k': 100} - Query with custom limit
{'query': 'laptop', 'category': 'electronics', 'price_max': 1000} - Query with filters

Note: Inputs are for estimation only. No actual search is performed.

Response

Detailed execution plan with stage-by-stage cost estimates, optimization details, bottleneck identification, and performance insights. Use this to troubleshoot slow queries, estimate costs, and understand optimizer transformations.

Execution plan analysis for a retriever.

Provides comprehensive diagnostics about retriever execution characteristics without actually running the query. Similar to MongoDB's explain plan or SQL's EXPLAIN command, this helps troubleshoot performance, estimate costs, and understand optimizer behavior.

Use Cases: - Identify bottleneck stages before execution - Estimate costs for budget planning - Debug slow retrievers by analyzing stage efficiency - Understand optimizer transformations - Compare different retriever configurations - Troubleshoot accuracy issues via document flow analysis

retriever_id

string

required

Unique identifier of the retriever being explained. REQUIRED.

retriever_name

string

required

Human-readable name of the retriever. REQUIRED.

estimated_cost

Estimated Cost · object

required

Estimated total cost breakdown for executing this retriever. Contains: 'total_credits' (credit cost), 'total_duration_ms' (latency). Sum of all stage costs. Use for budget planning. REQUIRED.

Show child attributes

estimated_cost.{key}

number

total_estimated_stages

integer

required

Total number of stages in the optimized execution plan. This may differ from your original stage count if optimizations were applied. Compare with optimization_details.original_stage_count to see reduction. REQUIRED.

Required range: x >= 0

execution_plan

ExplainStagePlan · object[]

Ordered list of stage execution plans showing the OPTIMIZED pipeline. Each entry shows cost, latency, document flow, and warnings for one stage. Stages execute in this order. REQUIRED (may be empty for invalid retrievers).

Show child attributes

execution_plan.stage_index

integer

required

Zero-based position of this stage in the execution pipeline. Stages execute sequentially in this order. REQUIRED.

Required range: x >= 0

execution_plan.stage_name

string

required

Human-readable name of this stage instance. Corresponds to the 'stage_name' field in your retriever configuration. Use this to map explain plan output back to your pipeline definition. REQUIRED.

execution_plan.stage_type

string

required

Stage type identifier indicating the category of operation. Common types: 'filter' (reduce documents), 'sort' (reorder), 'reduce' (aggregate), 'apply' (transform/enrich). REQUIRED.

execution_plan.estimated_input

integer

required

Estimated number of documents entering this stage. This is the output count from the previous stage (or initial collection size). Used to project document flow through the pipeline. REQUIRED.

Required range: x >= 0

execution_plan.estimated_output

integer

required

Estimated number of documents leaving this stage. For filter stages, this is typically less than estimated_input. For sort/reduce stages, this may be the same or less. REQUIRED.

Required range: x >= 0

execution_plan.estimated_efficiency

number

required

Stage selectivity ratio (estimated_output / estimated_input). Values closer to 0 indicate aggressive filtering. Values closer to 1 indicate most documents pass through. Use this to identify stages that might be too restrictive or too permissive. REQUIRED.

Required range: x >= 0

execution_plan.estimated_cost_credits

number

required

Estimated credit cost for executing this stage. Credits are consumed for inference (embeddings, LLM calls), vector searches, and other computational operations. Filter/sort stages typically have near-zero cost. REQUIRED.

Required range: x >= 0

execution_plan.estimated_duration_ms

number

required

Estimated latency contribution of this stage in milliseconds. High values indicate potential bottlenecks. Sum across stages gives total estimated execution time. REQUIRED.

Required range: x >= 0

execution_plan.cache_likely

boolean

required

Whether this stage is likely to hit cache based on recent execution history. True = cache hit likely (near-zero actual latency/cost). False = cache miss likely (full cost incurred). Use this to understand when queries will be fast vs slow. REQUIRED.

execution_plan.optimization_notes

string[]

Human-readable notes about optimizations applied to this stage. Examples: 'Pushed down from stage 2', 'Merged with previous filter', 'Grouping pushed to database layer'. Empty if no optimizations were applied. OPTIONAL.

execution_plan.warnings

string[]

Performance warnings or potential issues with this stage. Examples: 'High cost stage - consider reducing limit', 'Very low efficiency - may need filter tuning', 'LLM stage without prior filtering - expensive'. Empty if no warnings. OPTIONAL.

optimization_suggestions

Optimization Suggestions · object[]

Actionable suggestions for improving retriever performance. Each suggestion includes: 'type' (suggestion category), 'stage' (affected stage name), 'message' (human-readable description). Common types: 'reduce_limit', 'add_filter', 'reorder_stages', 'enable_cache'. OPTIONAL (empty if no suggestions).

bottleneck_stages

string[]

Names of stages expected to dominate execution time. Includes stages with duration >= 80%% of the slowest stage. Focus optimization efforts on these stages. OPTIONAL (empty if all stages have similar duration).

optimization_level

string

default:mvp

Optimization level applied by the optimizer. Values: 'none' (no optimization), 'mvp' (basic optimizations), 'advanced' (all optimizations). REQUIRED.

optimization_applied

boolean

default:false

Whether automatic pipeline optimizations were applied. When true, execution_plan shows OPTIMIZED stages (after transformations like filter push-down, stage fusion, grouping optimization). When false, execution_plan matches your original configuration. Check optimization_details to see what changed. REQUIRED.

optimization_details

Optimization Details · object

Detailed breakdown of optimization transformations applied. Only present when optimization_applied=true.

Fields:

original_stage_count: Stage count before optimization
optimized_stage_count: Stage count after optimization
optimization_time_ms: Time spent on optimization (typically <100ms)
stage_reduction_pct: Percentage reduction in stage count
decisions: Array of optimization decisions

Each decision contains:

rule_type: Optimization rule that fired
applied: Whether the rule was applied
reason: Human-readable explanation
stages_before/after: Stage counts before/after this rule

Common rule types:

push_down_filters: Move filters earlier to reduce downstream work
group_by_push_down: Push grouping to database layer (10-100x faster)
merge_consecutive_filters: Combine adjacent filters
eliminate_redundant_sorts: Remove duplicate sort operations

OPTIONAL (null when optimization_applied=false).

Example:

{
  "decisions": [
    {
      "applied": true,
      "reason": "Moved attribute_filter before semantic_search to reduce search scope by 50%",
      "rule_type": "push_down_filters",
      "stages_after": 5,
      "stages_before": 5
    },
    {
      "applied": true,
      "reason": "Merged group_by into feature_filter for database-level grouping (10x faster)",
      "rule_type": "group_by_push_down",
      "stages_after": 3,
      "stages_before": 5
    }
  ],
  "optimization_time_ms": 8.2,
  "optimized_stage_count": 3,
  "original_stage_count": 5,
  "stage_reduction_pct": 40
}

Get Execution List Available Retriever Stages

⌘I

Health

Organizations

Namespaces

Buckets

Feature Extractors

Collections

Retrievers

Taxonomies

Clusters

Templates

Analytics

Resource Search

Inference

Agent Sessions

Tasks

Webhooks

Notifications

Triggers

Explain Retriever Execution Plan

Headers

Path Parameters

Body

Response