The Limit stage controls the number of documents returned from the pipeline. It simply takes the first N documents from the current result set, preserving their order.
Stage Category : REDUCE (Reduces document count)Transformation : N documents → min(N, limit) documents
When to Use Use Case Description Final result count Return top 10 results to user Pre-processing reduction Limit before expensive operations Pagination Control page sizes Resource management Prevent large result sets
When NOT to Use Scenario Recommended Alternative Want best results Use rerank first Need offset pagination Combine with offset logic Filtering by criteria Use structured_filter
Parameters Parameter Type Default Description limitinteger Required Maximum number of documents to return
Configuration Examples Simple Top 10
Top 5 Results
Dynamic Limit
Large Result Set
{ "stage_type" : "reduce" , "stage_id" : "limit" , "parameters" : { "limit" : 10 } }
Output The stage returns the first N documents in their current order:
Input (5 documents):[ { "document_id" : "doc_1" , "score" : 0.95 }, { "document_id" : "doc_2" , "score" : 0.90 }, { "document_id" : "doc_3" , "score" : 0.85 }, { "document_id" : "doc_4" , "score" : 0.80 }, { "document_id" : "doc_5" , "score" : 0.75 } ]
Output (limit: 3):[ { "document_id" : "doc_1" , "score" : 0.95 }, { "document_id" : "doc_2" , "score" : 0.90 }, { "document_id" : "doc_3" , "score" : 0.85 } ]
Metric Value Latency < 1ms Complexity O(1) Memory Minimal
Limit is essentially free. Use it liberally to control result sizes and reduce processing in downstream stages.
Common Pipeline Patterns Search + Rerank + Limit [ { "stage_type" : "filter" , "stage_id" : "semantic_search" , "parameters" : { "query" : "{{INPUT.query}}" , "vector_index" : "text_extractor_v1_embedding" , "top_k" : 100 } }, { "stage_type" : "sort" , "stage_id" : "rerank" , "parameters" : { "model" : "bge-reranker-v2-m3" , "top_n" : 20 } }, { "stage_type" : "reduce" , "stage_id" : "limit" , "parameters" : { "limit" : 5 } } ]
Limit Before Expensive Operations [ { "stage_type" : "filter" , "stage_id" : "hybrid_search" , "parameters" : { "query" : "{{INPUT.query}}" , "vector_index" : "text_extractor_v1_embedding" , "top_k" : 200 } }, { "stage_type" : "reduce" , "stage_id" : "limit" , "parameters" : { "limit" : 50 } }, { "stage_type" : "apply" , "stage_id" : "llm_enrichment" , "parameters" : { "model" : "gpt-4o-mini" , "prompt" : "Extract key entities" } } ]
Full RAG Pipeline [ { "stage_type" : "filter" , "stage_id" : "hybrid_search" , "parameters" : { "query" : "{{INPUT.query}}" , "vector_index" : "text_extractor_v1_embedding" , "top_k" : 50 } }, { "stage_type" : "sort" , "stage_id" : "rerank" , "parameters" : { "model" : "bge-reranker-v2-m3" , "top_n" : 15 } }, { "stage_type" : "reduce" , "stage_id" : "limit" , "parameters" : { "limit" : 10 } }, { "stage_type" : "apply" , "stage_id" : "rag_prepare" , "parameters" : { "max_tokens" : 8000 , "output_mode" : "single_context" } } ]
Progressive Reduction [ { "stage_type" : "filter" , "stage_id" : "semantic_search" , "parameters" : { "top_k" : 500 } }, { "stage_type" : "filter" , "stage_id" : "structured_filter" , "parameters" : { "conditions" : { "field" : "status" , "operator" : "eq" , "value" : "active" } } }, { "stage_type" : "reduce" , "stage_id" : "limit" , "parameters" : { "limit" : 100 } }, { "stage_type" : "sort" , "stage_id" : "rerank" , "parameters" : { "model" : "bge-reranker-v2-m3" , "top_n" : 20 } }, { "stage_type" : "reduce" , "stage_id" : "limit" , "parameters" : { "limit" : 5 } } ]
Limit vs top_k vs top_n Parameter Stage Description top_kSearch stages Initial retrieval limit top_nRerank Results after reranking limitLimit stage Final output control
Recommendation : Use generous top_k for recall, top_n for precision, and limit for final output size.Error Handling Scenario Behavior Limit > document count Returns all documents Limit = 0 Returns empty set Limit < 0 Error Missing limit Error
