Retrievers combine feature-aware search stages, structured filters, enrichment joins, and optional LLM post-processing into a single executable pipeline. Each retriever has an input schema, a list of target collections, and a deterministic set of stages executed in order.
Anatomy of a Retriever
{
"retriever_name": "product_search_v2",
"description": "Semantic + hybrid search with reranking",
"collection_ids": ["col_products"],
"input_schema": {
"properties": {
"query_text": { "type": "text", "required": true },
"max_price": { "type": "number" }
}
},
"stages": [
{
"stage_name": "hybrid_search",
"version": "v1",
"parameters": {
"dense_feature_address": "mixpeek://text_extractor@v1/text_embedding",
"sparse_feature_address": "mixpeek://splade_extractor@v1/splade_vector",
"input_mapping": { "text": "query_text" },
"limit": 200,
"dense_weight": 0.7,
"sparse_weight": 0.3,
"sort_by": [{ "field": "score", "direction": "desc" }]
}
},
{
"stage_name": "filter",
"version": "v1",
"parameters": {
"strategy": "structured",
"structured_filter": {
"AND": [
{ "field": "metadata.in_stock", "operator": "eq", "value": true },
{ "field": "metadata.price", "operator": "lte", "value": "{{INPUT.max_price}}" }
]
}
}
},
{
"stage_name": "rerank",
"version": "v1",
"parameters": {
"strategy": "cross_encoder",
"model": "bge-reranker-v2-m3",
"top_k": 50
}
}
],
"cache_config": {
"enabled": true,
"ttl_seconds": 300,
"cache_stage_names": ["hybrid_search"]
}
}
Stage Catalog
Retrieve the live registry with GET /v1/retrievers/stages. Each entry includes stage_id, category, icon, and parameter schema so you can dynamically build configuration UIs or validations.
curl -s --request GET \
--url "$MP_API_URL/v1/retrievers/stages" \
--header "Authorization: Bearer $MP_API_KEY" \
--header "X-Namespace: $MP_NAMESPACE"
| Category | Stage ID | Description |
|---|
| search | hybrid_search | Dense + lexical retrieval with reciprocal-rank and weighted-score fusion, plus score thresholds. |
| reduce | attribute_filter | Structured filtering with boolean logic, projections, and template-aware batch sizing. |
| reduce | llm_filter | LLM-powered screening that keeps or discards documents based on prompt-defined criteria. |
| rank | sort_attribute | Sort in-flight results by any payload field with case-insensitive and null handling options. |
| rank | sort_relevance | Sort by relevance score (or custom numeric field) with graceful fallbacks for missing scores. |
| enrich | document_enrichment | Join documents with another collection and merge selected fields. |
| enrich | taxonomy_enrichment | Attach taxonomy assignments via feature URIs and copy enrichment fields. |
| enrich | llm_enrichment | Generate new fields with an LLM, respecting per-document conditions, batching, and schemas. |
GET /v1/retrievers/stages whenever you need the latest metadata or parameter schemas.
Execution Lifecycle
- Validate Inputs – Mixpeek enforces the retriever’s
input_schema.
- Walk Stages – Each stage receives the current working set, runs, and outputs a new set.
- Apply Pagination –
limit, offset, cursor, or keyset pagination is handled after the final stage.
- Return Telemetry – Responses include
stage_statistics, budget, and optional presigned URLs.
Response headers include:
ETag – cache validator; pair with If-None-Match for 304 responses.Cache-Control – TTL derived from cache_config.X-Cache – HIT or MISS for query-level caching.
Filters & Templates
- Structured filters support comparison operators (
eq, gt, lte, in, etc.) and logical composition (AND, OR, NOT).
- Template variables let you reference namespaces:
{{INPUT.query_text}}{{DOC.metadata.category}}{{STAGE.hybrid_search.top_score}}{{CONTEXT.budget_remaining}}
Retrievers & Caching
- Query cache – caches entire responses keyed by inputs, filters, pagination, and collection index signatures.
- Stage cache – reuse outputs of expensive stages by listing them under
cache_stage_names.
- Inference cache – Engine deduplicates identical model calls.
Use GET /v1/analytics/retrievers/{id}/cache-performance to monitor hit rates and latency improvements.
| Method | Use Case |
|---|
offset | Simple pagination, supports limit + offset |
cursor | Stable iteration over large result sets |
scroll | Deep pagination for analytics workloads |
keyset | High-performance paginated browsing (requires sort key) |
pagination.method when executing a retriever.
Execute a Retriever
curl -sS -X POST "$MP_API_URL/v1/retrievers/<retriever_id>/execute" \
-H "Authorization: Bearer $MP_API_KEY" \
-H "X-Namespace: $MP_NAMESPACE" \
-H "Content-Type: application/json" \
-d '{
"inputs": {
"query_text": "wireless earbuds",
"max_price": 150
},
"filters": {
"field": "metadata.category",
"operator": "eq",
"value": "audio"
},
"limit": 10,
"return_urls": true,
"return_vectors": false,
"session_id": "sess_123"
}'
{
"execution_id": "exec_b8f31e0c",
"documents": [...],
"stage_statistics": {
"hybrid_search": { "duration_ms": 180, "cache_hit": true },
"filter": { "duration_ms": 8 },
"rerank": { "duration_ms": 120 }
},
"budget": {
"credits_used": 12.4,
"credits_limit": 100,
"time_elapsed_ms": 310
}
}
Maintenance & Versioning
- Use
PATCH /v1/retrievers/{id} to rename retrievers or adjust cache settings (stages and schema are immutable; create a new retriever for breaking changes).
- List retrievers with filters, search, and sort:
POST /v1/retrievers/list.
- Retrieve execution history:
GET /v1/retrievers/{id}/executions.
- Diagnose pipelines without executing:
POST /v1/retrievers/{id}/explain.
Interaction Feedback
Capture user feedback with /v1/retrievers/interactions to power downstream analytics, learning-to-rank, or personalized retrieval:
{
"feature_id": "doc_abc123",
"interaction_type": ["click", "long_view"],
"position": 2,
"metadata": { "duration_ms": 12000 },
"user_id": "user_456",
"session_id": "sess_xyz789"
}
Best Practices
- Start narrow – run a single search stage before adding rerankers or joins.
- Push filters early – stage-level filters shrink the candidate set before expensive operations.
- Use JOIN strategies wisely –
direct for key-based joins, retriever for similarity joins; set join_strategy to control merge behavior.
- Enable caching – stage caching plus query caching dramatically reduces latency for repeat queries.
- Monitor analytics – use retriever analytics endpoints to optimize parameters, detect slow stages, and understand cache ROI.
Retrievers turn Mixpeek’s primitives—features, taxonomies, clusters, and models—into end-user search experiences. Configure once, execute anywhere, and evolve the pipeline with confidence. 
