Get Collection

Headers

Authorization

string

required

REQUIRED: Bearer token authentication using your API key. Format: 'Bearer sk_xxxxxxxxxxxxx'. You can create API keys in the Mixpeek dashboard under Organization Settings.

X-Namespace

string

required

REQUIRED: Namespace identifier for scoping this request. All resources (collections, buckets, taxonomies, etc.) are scoped to a namespace. You can provide either the namespace name or namespace ID. Format: ns_xxxxxxxxxxxxx (ID) or a custom name like 'my-namespace'

Path Parameters

collection_identifier

string

required

The ID or name of the collection to retrieve

Response

Successful Response

Response model for collection endpoints.

collection_name

string

required

REQUIRED. Human-readable name for the collection. Must be unique within the namespace. Used for: Display, lookups (can query by name or ID), organization. Format: Alphanumeric with underscores/hyphens, 3-100 characters. Examples: 'product_embeddings', 'video_frames', 'customer_documents'.

Required string length: 3 - 100

input_schema

BucketSchema · object

required

REQUIRED (auto-computed from source). Input schema defining fields available to the feature extractor. Source: bucket.bucket_schema (if source.type='bucket') OR upstream_collection.output_schema (if source.type='collection'). Determines: Which fields can be used in input_mappings and field_passthrough. This is the 'left side' of the transformation - what data goes IN. Format: BucketSchema with properties dict. Use for: Validating input_mappings, configuring field_passthrough.

Show child attributes

input_schema.properties

Properties · object

required

Show child attributes

input_schema.properties.{key}

BucketSchemaField · object

Schema field definition for bucket objects.

Show child attributes

input_schema.properties.{key}.type

enum<string>

required

Supported data types for bucket schema fields.

Types fall into two categories:

Explicit Types (Type-Safe):
- Provide type guarantees for extractors
- User must pre-classify content
- Compatible with type-specific extractors
Automatic Type (Flexible):
- No type guarantee (runtime detection)
- User doesn't pre-classify content
- Only compatible with multimodal extractors

Available options:

string,

number,

integer,

float,

boolean,

object,

array,

date,

datetime,

json,

file,

text,

image,

audio,

video,

pdf,

document,

spreadsheet,

presentation,

dense_vector,

sparse_vector,

int8_vector,

automatic

input_schema.properties.{key}.default

unknown

input_schema.properties.{key}.items

unknown

input_schema.properties.{key}.properties

Properties · object

input_schema.properties.{key}.examples

any[] | null

OPTIONAL. List of example values for this field. Used by Apps to show example inputs in the UI. Provide multiple diverse examples when possible.

input_schema.properties.{key}.description

string | null

input_schema.properties.{key}.enum

any[] | null

input_schema.properties.{key}.required

boolean | null

default:false

output_schema

BucketSchema · object

required

REQUIRED (auto-computed at creation). Output schema defining fields in final documents. Computed as: field_passthrough fields + extractor output fields (deterministic). Known IMMEDIATELY when collection is created - no waiting for documents! This is the 'right side' of the transformation - what data comes OUT. Use for: Understanding document structure, building queries, schema validation. Example: {title (passthrough), embedding (extractor output)} = output_schema.

Show child attributes

output_schema.properties

Properties · object

required

Show child attributes

output_schema.properties.{key}

BucketSchemaField · object

Schema field definition for bucket objects.

Show child attributes

output_schema.properties.{key}.type

enum<string>

required

Supported data types for bucket schema fields.

Types fall into two categories:

Explicit Types (Type-Safe):
- Provide type guarantees for extractors
- User must pre-classify content
- Compatible with type-specific extractors
Automatic Type (Flexible):
- No type guarantee (runtime detection)
- User doesn't pre-classify content
- Only compatible with multimodal extractors

Available options:

string,

number,

integer,

float,

boolean,

object,

array,

date,

datetime,

json,

file,

text,

image,

audio,

video,

pdf,

document,

spreadsheet,

presentation,

dense_vector,

sparse_vector,

int8_vector,

automatic

output_schema.properties.{key}.default

unknown

output_schema.properties.{key}.items

unknown

output_schema.properties.{key}.properties

Properties · object

output_schema.properties.{key}.examples

any[] | null

OPTIONAL. List of example values for this field. Used by Apps to show example inputs in the UI. Provide multiple diverse examples when possible.

output_schema.properties.{key}.description

string | null

output_schema.properties.{key}.enum

any[] | null

output_schema.properties.{key}.required

boolean | null

default:false

feature_extractor

FeatureExtractorConfig · object

required

REQUIRED. Single feature extractor configuration for this collection. Defines: extractor name/version, input_mappings, field_passthrough, parameters. Task 9 change: ONE extractor per collection (previously supported multiple). For multiple extractors: Create multiple collections and use collection-to-collection pipelines. Use field_passthrough to include additional source fields beyond extractor outputs.

Show child attributes

feature_extractor.feature_extractor_name

string

required

Name of the feature extractor

feature_extractor.version

string

required

Version of the feature extractor

feature_extractor.feature_extractor_id

string

required

Construct unique identifier for the feature extractor instance (name + version).

feature_extractor.parameters

Parameters · object

Parameters for the feature extractor (model name, thresholds, etc.). See extractor-specific documentation for available parameters.

Show child attributes

feature_extractor.parameters.{key}

ParameterConfig · object

A typed value for a feature extractor parameter.

Show child attributes

feature_extractor.parameters.{key}.type

enum<string>

required

The type of the parameter (e.g., 'fixed').

Available options:

fixed,

dynamic

feature_extractor.parameters.{key}.value

any

required

The value of the parameter.

feature_extractor.input_mappings

Input Mappings · object

Mapping from extractor input names to source field paths. Tells the extractor which source fields to process. Example: {'image': 'thumbnail_url', 'text': 'description'}

Input Mappings · object
InputMapping · object[]

Show child attributes

feature_extractor.input_mappings.{key}

string

Example:

{ "image": "product_image", "text": "title" }

feature_extractor.field_passthrough

FieldPassthrough · object[]

NOT REQUIRED. List of specific fields to pass through from source to output documents. These fields are included alongside extractor-computed features (embeddings, detections, etc.). Empty list = only extractor outputs in documents (default behavior). With entries = specified fields + extractor outputs in documents.

How It Works:

During processing, fields are extracted from source object/document
They appear in output documents at the root level
Field filtering happens automatically (only listed fields included)
Use target_path to rename fields for cleaner schemas

Common Use Cases:

Preserve identifiers: campaign_id, product_sku, order_id
Keep metadata: category, tags, author, created_at
Enable filtering: department, status, priority, region
Maintain context: title, description, source_url

Behavior:

Works with include_all_source_fields=False (default): ONLY these fields included
Works with include_all_source_fields=True: These configs used for renaming/defaults
Fields must exist in source bucket_schema or upstream collection output_schema
Missing optional fields are omitted (unless default provided)
Missing required fields cause processing errors

Output Schema: output_schema = field_passthrough fields + extractor output fields Example: ['title', 'category', 'text_extractor_v1_embedding']

Show child attributes

feature_extractor.field_passthrough.source_path

string

required

REQUIRED. Path to the source field to copy. Simple fields: Use field name directly (e.g., 'title', 'campaign_id'). Nested fields: Use dot notation (e.g., 'metadata.author', 'config.model.version'). The field must exist in the source bucket schema or upstream collection schema. Without target_path, nested fields are flattened: 'metadata.author' becomes 'author' in output.

feature_extractor.field_passthrough.target_path

string | null

OPTIONAL. Target field name in output document. If NOT PROVIDED: Uses source_path name (or last component for nested paths). - 'title' → 'title' - 'metadata.author' → 'author' If PROVIDED: Uses this exact name in output. - source_path='doc_title', target_path='title' → 'title' - source_path='metadata.author', target_path='contributor' → 'contributor' Use cases: - Rename fields for cleaner API schemas - Avoid name conflicts with extractor outputs - Standardize field names across different sources Constraints: - Must not conflict with system fields (document_id, collection_id, etc.) - Must not conflict with extractor output fields - Must be a valid field name (alphanumeric, underscores, hyphens)

Example:

"title"

feature_extractor.field_passthrough.default

any | null

OPTIONAL. Default value if source field doesn't exist or is None. If NOT PROVIDED and field missing: Field is omitted from output document. If PROVIDED and field missing: Field is included with this default value. Type should match expected field type (string, int, list, dict, etc.).

Example:

"Unknown"

feature_extractor.field_passthrough.required

boolean

default:false

OPTIONAL. Whether this field MUST exist in source. If True and field missing: Raises validation error, processing fails. If False and field missing: Field omitted (or default used if provided). Use True for: Critical identifiers, required business fields. Use False for: Optional metadata, nice-to-have fields. Default: False (field is optional).

feature_extractor.include_all_source_fields

boolean

default:false

NOT REQUIRED. Whether to include ALL fields from source object/document in output. Default: False (only field_passthrough fields included).

When False (RECOMMENDED):

Only fields listed in field_passthrough are included in output
Creates clean, predictable output schemas
Prevents data leakage of unwanted fields
Output = field_passthrough fields + extractor outputs

When True (USE WITH CAUTION):

ALL source fields are included in output documents
field_passthrough still used for renaming/defaults/requirements
Can result in large documents if source has many fields
Can leak sensitive or unnecessary data
Output = all source fields + extractor outputs

Use True When:

You want to preserve complete source data
Source has limited, well-defined fields
Downstream processing needs all context

Use False When (MOST CASES):

You want clean, controlled output schemas
Source has many fields you don't need
You want explicit field selection
You're concerned about document size

Examples: False: source={a,b,c,d} + passthrough=[a,b] → output={a,b,embedding} True: source={a,b,c,d} + passthrough=[a→x] → output={x,b,c,d,embedding}

source

SourceConfig · object

required

REQUIRED. Source configuration defining where data comes from. Type 'bucket': Process objects from one or more buckets (tier 1). Type 'collection': Process documents from upstream collection(s) (tier 2+). For multi-bucket sources, all buckets must have compatible schemas. For multi-collection sources, all collections must have compatible schemas. Determines input_schema and enables decomposition trees.

Show child attributes

source.type

enum<string>

required

REQUIRED. Type of source for this collection. 'bucket': Process objects from one or more buckets (first-stage processing). 'collection': Process documents from another collection (downstream processing). Use 'bucket' for initial data ingestion, 'collection' for decomposition trees.

Available options:

bucket,

collection,

taxonomy,

cluster

source.bucket_ids

string[] | null

List of bucket IDs when type='bucket'. REQUIRED when type='bucket'. NOT ALLOWED when type='collection'. Can specify one or more buckets to process. Single bucket: Use array with one element ['bkt_id']. Multiple buckets: All buckets MUST have compatible schemas. Schema compatibility validated at collection creation. Compatible schemas have: 1) Same field names, 2) Same field types, 3) Same required status. Documents will include root_bucket_id to track which bucket they came from. Use cases: multi-region data, multi-team consolidation, environment aggregation.

Minimum array length: 1

Example:

["bkt_marketing_videos"]

source.collection_id

string | null

Collection ID when type='collection' (single collection). Use this OR collection_ids (not both). REQUIRED when type='collection' and processing single collection. NOT ALLOWED when type='bucket'. The collection will process documents from this upstream collection. The upstream collection's output_schema becomes this collection's input_schema. This enables decomposition trees (multi-stage pipelines). Example: Process frames collection → create scenes collection.

Example:

"col_video_frames"

source.collection_ids

string[] | null

List of collection IDs when type='collection' (multiple collections). Use this OR collection_id (not both). REQUIRED when type='collection' and processing multiple collections. NOT ALLOWED when type='bucket'. Used for operations that consolidate multiple upstream collections. Example: Clustering across multiple collections → cluster output collection. All collections must have compatible schemas for consolidation operations.

Minimum array length: 1

Example:

["col_us_products", "col_eu_products"]

source.inherited_bucket_ids

string[] | null

List of original bucket IDs that source collections originated from. OPTIONAL. Only used when type='collection'. Tracks the complete lineage chain: buckets → collections → derived collections. Extracted from upstream collection metadata at collection creation time. Enables tracing derived collections (like cluster outputs) back to original data sources. Example: Cluster output collection inherits bucket IDs from its source collections. Format: List of bucket IDs with 'bkt_' prefix.

Example:

["bkt_marketing_videos"]

source.source_filters

SourceFilters · object

Optional filters to apply to source data. When specified, only objects/documents matching these filters will be processed by this collection. Filters are evaluated at batch creation time. Uses same LogicalOperator model as list APIs for consistency.

Show child attributes

source.source_filters.filters

LogicalOperator · object

Optional logical filters to apply to source data. Uses LogicalOperator model with AND/OR/NOT support. When specified, only objects/documents matching these filters will be processed by this collection. When null, all source data is processed (no filtering). Filters are consistent across all batch runs for this collection.

Show child attributes

source.source_filters.filters.AND

FilterCondition · object[] | null

Logical AND operation - all conditions must be true

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

source.source_filters.filters.AND.field

string

required

Field name to filter on

source.source_filters.filters.AND.value

required

Value to compare against

Show child attributes

source.source_filters.filters.AND.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

source.source_filters.filters.AND.value.type

string

default:dynamic

Allowed value: "dynamic"

source.source_filters.filters.AND.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "name",
    "operator": "eq",
    "value": "John"
  },
  {
    "field": "age",
    "operator": "gte",
    "value": 30
  }
]

source.source_filters.filters.OR

FilterCondition · object[] | null

Logical OR operation - at least one condition must be true

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

source.source_filters.filters.OR.field

string

required

Field name to filter on

source.source_filters.filters.OR.value

required

Value to compare against

Show child attributes

source.source_filters.filters.OR.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

source.source_filters.filters.OR.value.type

string

default:dynamic

Allowed value: "dynamic"

source.source_filters.filters.OR.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "status",
    "operator": "eq",
    "value": "active"
  },
  {
    "field": "role",
    "operator": "eq",
    "value": "admin"
  }
]

source.source_filters.filters.NOT

FilterCondition · object[] | null

Logical NOT operation - all conditions must be false

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

source.source_filters.filters.NOT.field

string

required

Field name to filter on

source.source_filters.filters.NOT.value

required

Value to compare against

Show child attributes

source.source_filters.filters.NOT.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

source.source_filters.filters.NOT.value.type

string

default:dynamic

Allowed value: "dynamic"

source.source_filters.filters.NOT.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "department",
    "operator": "eq",
    "value": "HR"
  },
  {
    "field": "location",
    "operator": "eq",
    "value": "remote"
  }
]

source.source_filters.filters.case_sensitive

boolean | null

default:false

Whether to perform case-sensitive matching

Example:

true

Example:

{
  "AND": [
    {
      "field": "blobs.type",
      "operator": "eq",
      "value": "video"
    }
  ]
}

Example:

{
  "description": "Filter only video content",
  "filters": {
    "AND": [
      {
        "field": "blobs.type",
        "operator": "eq",
        "value": "video"
      }
    ]
  }
}

collection_id

string

NOT REQUIRED (auto-generated). Unique identifier for this collection. Used for: API paths, document queries, pipeline references. Format: 'col_' prefix + 10 random alphanumeric characters. Stable after creation - use for all collection references.

description

string | null

NOT REQUIRED. Human-readable description of the collection's purpose. Use for: Documentation, team communication, UI display. Common pattern: Describe what the collection contains and what processing is applied.

Example:

"Video frames extracted at 1 FPS with CLIP embeddings"

source_bucket_schemas

Source Bucket Schemas · object

NOT REQUIRED (auto-computed). Snapshot of bucket schemas at collection creation. Only populated for multi-bucket collections (source.type='bucket' with multiple bucket_ids). Key: bucket_id, Value: BucketSchema at time of collection creation. Used for: Schema compatibility validation, document lineage, debugging. Schema snapshot is immutable - bucket schema changes after collection creation do not affect this. Single-bucket collections may omit this field (schema in input_schema is sufficient).

Show child attributes

source_bucket_schemas.{key}

BucketSchema · object

Schema definition for bucket objects.

IMPORTANT: The bucket schema defines what fields your bucket objects will have. This schema is REQUIRED if you want to:

Create collections that use input_mappings to process your bucket data
Validate object structure before ingestion
Enable type-safe data pipelines

The schema defines the custom fields that will be used in:

Blob properties (e.g., "content", "thumbnail", "transcript")
Object metadata structure
Blob data structures

Example workflow:

Create bucket WITH schema defining your data structure
Upload objects that conform to that schema
Create collections that map schema fields to feature extractors

Without a bucket_schema, collections cannot use input_mappings.

Show child attributes

source_bucket_schemas.{key}.properties

Properties · object

required

Show child attributes

source_bucket_schemas.{key}.properties.{key}

BucketSchemaField · object

Schema field definition for bucket objects.

Show child attributes

source_bucket_schemas.{key}.properties.{key}.type

enum<string>

required

Supported data types for bucket schema fields.

Types fall into two categories:

Explicit Types (Type-Safe):
- Provide type guarantees for extractors
- User must pre-classify content
- Compatible with type-specific extractors
Automatic Type (Flexible):
- No type guarantee (runtime detection)
- User doesn't pre-classify content
- Only compatible with multimodal extractors

Available options:

string,

number,

integer,

float,

boolean,

object,

array,

date,

datetime,

json,

file,

text,

image,

audio,

video,

pdf,

document,

spreadsheet,

presentation,

dense_vector,

sparse_vector,

int8_vector,

automatic

source_bucket_schemas.{key}.properties.{key}.default

unknown

source_bucket_schemas.{key}.properties.{key}.items

unknown

source_bucket_schemas.{key}.properties.{key}.properties

Properties · object

source_bucket_schemas.{key}.properties.{key}.examples

any[] | null

OPTIONAL. List of example values for this field. Used by Apps to show example inputs in the UI. Provide multiple diverse examples when possible.

source_bucket_schemas.{key}.properties.{key}.description

string | null

source_bucket_schemas.{key}.properties.{key}.enum

any[] | null

source_bucket_schemas.{key}.properties.{key}.required

boolean | null

default:false

Example:

null

source_lineage

SingleLineageEntry · object[]

NOT REQUIRED (auto-computed). Lineage chain showing complete processing history. Each entry contains: source_config, feature_extractor, output_schema for one tier. Length indicates processing depth (1 = tier 1, 2 = tier 2, etc.). Use for: Understanding multi-tier pipelines, visualizing decomposition trees.

Show child attributes

source_lineage.source_config

SourceConfig · object

required

Configuration of the source for this lineage entry

Show child attributes

source_lineage.source_config.type

enum<string>

required

Available options:

bucket,

collection,

taxonomy,

cluster

source_lineage.source_config.bucket_ids

string[] | null

Minimum array length: 1

Example:

["bkt_marketing_videos"]

source_lineage.source_config.collection_id

string | null

Example:

"col_video_frames"

source_lineage.source_config.collection_ids

string[] | null

Minimum array length: 1

Example:

["col_us_products", "col_eu_products"]

source_lineage.source_config.inherited_bucket_ids

string[] | null

Example:

["bkt_marketing_videos"]

source_lineage.source_config.source_filters

SourceFilters · object

Show child attributes

source_lineage.source_config.source_filters.filters

LogicalOperator · object

Show child attributes

source_lineage.source_config.source_filters.filters.AND

FilterCondition · object[] | null

Logical AND operation - all conditions must be true

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

source_lineage.source_config.source_filters.filters.AND.field

string

required

Field name to filter on

source_lineage.source_config.source_filters.filters.AND.value

required

Value to compare against

Show child attributes

source_lineage.source_config.source_filters.filters.AND.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

source_lineage.source_config.source_filters.filters.AND.value.type

string

default:dynamic

Allowed value: "dynamic"

source_lineage.source_config.source_filters.filters.AND.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "name",
    "operator": "eq",
    "value": "John"
  },
  {
    "field": "age",
    "operator": "gte",
    "value": 30
  }
]

source_lineage.source_config.source_filters.filters.OR

FilterCondition · object[] | null

Logical OR operation - at least one condition must be true

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

source_lineage.source_config.source_filters.filters.OR.field

string

required

Field name to filter on

source_lineage.source_config.source_filters.filters.OR.value

required

Value to compare against

Show child attributes

source_lineage.source_config.source_filters.filters.OR.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

source_lineage.source_config.source_filters.filters.OR.value.type

string

default:dynamic

Allowed value: "dynamic"

source_lineage.source_config.source_filters.filters.OR.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "status",
    "operator": "eq",
    "value": "active"
  },
  {
    "field": "role",
    "operator": "eq",
    "value": "admin"
  }
]

source_lineage.source_config.source_filters.filters.NOT

FilterCondition · object[] | null

Logical NOT operation - all conditions must be false

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

source_lineage.source_config.source_filters.filters.NOT.field

string

required

Field name to filter on

source_lineage.source_config.source_filters.filters.NOT.value

required

Value to compare against

Show child attributes

source_lineage.source_config.source_filters.filters.NOT.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

source_lineage.source_config.source_filters.filters.NOT.value.type

string

default:dynamic

Allowed value: "dynamic"

source_lineage.source_config.source_filters.filters.NOT.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "department",
    "operator": "eq",
    "value": "HR"
  },
  {
    "field": "location",
    "operator": "eq",
    "value": "remote"
  }
]

source_lineage.source_config.source_filters.filters.case_sensitive

boolean | null

default:false

Whether to perform case-sensitive matching

Example:

true

Example:

{
  "AND": [
    {
      "field": "blobs.type",
      "operator": "eq",
      "value": "video"
    }
  ]
}

Example:

{
  "description": "Filter only video content",
  "filters": {
    "AND": [
      {
        "field": "blobs.type",
        "operator": "eq",
        "value": "video"
      }
    ]
  }
}

source_lineage.feature_extractor

FeatureExtractorConfig · object

required

Single feature extractor applied at this stage

Show child attributes

source_lineage.feature_extractor.feature_extractor_name

string

required

Name of the feature extractor

source_lineage.feature_extractor.version

string

required

Version of the feature extractor

source_lineage.feature_extractor.feature_extractor_id

string

required

Construct unique identifier for the feature extractor instance (name + version).

source_lineage.feature_extractor.parameters

Parameters · object

Parameters for the feature extractor (model name, thresholds, etc.). See extractor-specific documentation for available parameters.

Show child attributes

source_lineage.feature_extractor.parameters.{key}

ParameterConfig · object

A typed value for a feature extractor parameter.

Show child attributes

source_lineage.feature_extractor.parameters.{key}.type

enum<string>

required

The type of the parameter (e.g., 'fixed').

Available options:

fixed,

dynamic

source_lineage.feature_extractor.parameters.{key}.value

any

required

The value of the parameter.

source_lineage.feature_extractor.input_mappings

Input Mappings · object

Mapping from extractor input names to source field paths. Tells the extractor which source fields to process. Example: {'image': 'thumbnail_url', 'text': 'description'}

Input Mappings · object
InputMapping · object[]

Show child attributes

source_lineage.feature_extractor.input_mappings.{key}

string

Example:

{ "image": "product_image", "text": "title" }

source_lineage.feature_extractor.field_passthrough

FieldPassthrough · object[]

How It Works:

During processing, fields are extracted from source object/document
They appear in output documents at the root level
Field filtering happens automatically (only listed fields included)
Use target_path to rename fields for cleaner schemas

Common Use Cases:

Preserve identifiers: campaign_id, product_sku, order_id
Keep metadata: category, tags, author, created_at
Enable filtering: department, status, priority, region
Maintain context: title, description, source_url

Behavior:

Works with include_all_source_fields=False (default): ONLY these fields included
Works with include_all_source_fields=True: These configs used for renaming/defaults
Fields must exist in source bucket_schema or upstream collection output_schema
Missing optional fields are omitted (unless default provided)
Missing required fields cause processing errors

Output Schema: output_schema = field_passthrough fields + extractor output fields Example: ['title', 'category', 'text_extractor_v1_embedding']

Show child attributes

source_lineage.feature_extractor.field_passthrough.source_path

string

required

source_lineage.feature_extractor.field_passthrough.target_path

string | null

Example:

"title"

source_lineage.feature_extractor.field_passthrough.default

any | null

Example:

"Unknown"

source_lineage.feature_extractor.field_passthrough.required

boolean

default:false

source_lineage.feature_extractor.include_all_source_fields

boolean

default:false

NOT REQUIRED. Whether to include ALL fields from source object/document in output. Default: False (only field_passthrough fields included).

When False (RECOMMENDED):

Only fields listed in field_passthrough are included in output
Creates clean, predictable output schemas
Prevents data leakage of unwanted fields
Output = field_passthrough fields + extractor outputs

When True (USE WITH CAUTION):

ALL source fields are included in output documents
field_passthrough still used for renaming/defaults/requirements
Can result in large documents if source has many fields
Can leak sensitive or unnecessary data
Output = all source fields + extractor outputs

Use True When:

You want to preserve complete source data
Source has limited, well-defined fields
Downstream processing needs all context

Use False When (MOST CASES):

You want clean, controlled output schemas
Source has many fields you don't need
You want explicit field selection
You're concerned about document size

Examples: False: source={a,b,c,d} + passthrough=[a,b] → output={a,b,embedding} True: source={a,b,c,d} + passthrough=[a→x] → output={x,b,c,d,embedding}

source_lineage.output_schema

BucketSchema · object

required

Output schema produced by this processing stage

Show child attributes

source_lineage.output_schema.properties

Properties · object

required

Show child attributes

source_lineage.output_schema.properties.{key}

BucketSchemaField · object

Schema field definition for bucket objects.

Show child attributes

source_lineage.output_schema.properties.{key}.type

enum<string>

required

Supported data types for bucket schema fields.

Types fall into two categories:

Explicit Types (Type-Safe):
- Provide type guarantees for extractors
- User must pre-classify content
- Compatible with type-specific extractors
Automatic Type (Flexible):
- No type guarantee (runtime detection)
- User doesn't pre-classify content
- Only compatible with multimodal extractors

Available options:

string,

number,

integer,

float,

boolean,

object,

array,

date,

datetime,

json,

file,

text,

image,

audio,

video,

pdf,

document,

spreadsheet,

presentation,

dense_vector,

sparse_vector,

int8_vector,

automatic

source_lineage.output_schema.properties.{key}.default

unknown

source_lineage.output_schema.properties.{key}.items

unknown

source_lineage.output_schema.properties.{key}.properties

Properties · object

source_lineage.output_schema.properties.{key}.examples

any[] | null

OPTIONAL. List of example values for this field. Used by Apps to show example inputs in the UI. Provide multiple diverse examples when possible.

source_lineage.output_schema.properties.{key}.description

string | null

source_lineage.output_schema.properties.{key}.enum

any[] | null

source_lineage.output_schema.properties.{key}.required

boolean | null

default:false

vector_indexes

any[]

NOT REQUIRED (auto-computed from extractor). Vector indexes for semantic search. Populated from feature_extractor.required_vector_indexes. Defines: Which embeddings are indexed, dimensions, distance metrics. Use for: Understanding search capabilities, debugging vector queries.

payload_indexes

any[]

NOT REQUIRED (auto-computed from extractor + namespace). Payload indexes for filtering. Enables efficient filtering on metadata fields, timestamps, IDs. Populated from: extractor requirements + namespace defaults. Use for: Understanding which fields support fast filtering.

enabled

boolean

default:true

NOT REQUIRED (defaults to True). Whether the collection accepts new documents. False: Collection exists but won't process new objects. True: Active and processing. Use for: Temporarily disabling collections without deletion.

metadata

Metadata · object

NOT REQUIRED. Additional user-defined metadata for the collection. Arbitrary key-value pairs for custom organization, tracking, configuration. Not used by the platform - purely for user purposes. Common uses: team ownership, project tags, deployment environment.

Example:

{
  "environment": "production",
  "project": "Q4_campaign",
  "team": "data-science"
}

created_at

string<date-time> | null

Timestamp when the collection was created. Automatically set by the system when the collection is first saved to the database.

updated_at

string<date-time> | null

Timestamp when the collection was last updated. Automatically updated by the system whenever the collection is modified.

document_count

integer | null

Number of documents in the collection

schema_version

integer

default:1

Version number for the output_schema. Increments automatically when schema is updated via document sampling. Used to track schema evolution and trigger downstream collection schema updates.

last_schema_sync

string<date-time> | null

Timestamp of last automatic schema sync from document sampling. Used to debounce schema updates (prevents thrashing).

schema_sync_enabled

boolean

default:true

Whether automatic schema discovery and sync is enabled for this collection. When True, schema is periodically updated by sampling documents. When False, schema remains fixed at creation time.

taxonomy_applications

TaxonomyApplicationConfig · object[] | null

NOT REQUIRED. List of taxonomies to apply to documents in this collection. Each entry specifies: taxonomy_id, execution_mode (materialize/on_demand), optional filters. Materialized: Enrichments persisted to documents during ingestion. On-demand: Enrichments computed during retrieval (via taxonomy_join stages). Empty/null if no taxonomies attached. Use for: Categorization, hierarchical classification.

Show child attributes

taxonomy_applications.taxonomy_id

string

required

ID of the TaxonomyModel to execute.

taxonomy_applications.execution_mode

enum<string>

default:on_demand

'on_demand' executes at query time; 'materialize' materialises during ingestion.

Available options:

on_demand,

materialize

taxonomy_applications.target_collection_id

string | null

Optional collection to persist results into when execution_mode is 'materialize'. If omitted, the source collection is updated in-place.

taxonomy_applications.scroll_filters

LogicalOperator · object

Additional filters applied when scrolling the source collection before enrichment.

Show child attributes

taxonomy_applications.scroll_filters.AND

FilterCondition · object[] | null

Logical AND operation - all conditions must be true

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

taxonomy_applications.scroll_filters.AND.field

string

required

Field name to filter on

taxonomy_applications.scroll_filters.AND.value

required

Value to compare against

Show child attributes

taxonomy_applications.scroll_filters.AND.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

taxonomy_applications.scroll_filters.AND.value.type

string

default:dynamic

Allowed value: "dynamic"

taxonomy_applications.scroll_filters.AND.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "name",
    "operator": "eq",
    "value": "John"
  },
  {
    "field": "age",
    "operator": "gte",
    "value": 30
  }
]

taxonomy_applications.scroll_filters.OR

FilterCondition · object[] | null

Logical OR operation - at least one condition must be true

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

taxonomy_applications.scroll_filters.OR.field

string

required

Field name to filter on

taxonomy_applications.scroll_filters.OR.value

required

Value to compare against

Show child attributes

taxonomy_applications.scroll_filters.OR.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

taxonomy_applications.scroll_filters.OR.value.type

string

default:dynamic

Allowed value: "dynamic"

taxonomy_applications.scroll_filters.OR.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "status",
    "operator": "eq",
    "value": "active"
  },
  {
    "field": "role",
    "operator": "eq",
    "value": "admin"
  }
]

taxonomy_applications.scroll_filters.NOT

FilterCondition · object[] | null

Logical NOT operation - all conditions must be false

Represents a single filter condition.

Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against

Show child attributes

taxonomy_applications.scroll_filters.NOT.field

string

required

Field name to filter on

taxonomy_applications.scroll_filters.NOT.value

required

Value to compare against

Show child attributes

taxonomy_applications.scroll_filters.NOT.value.field

string

required

The dot-notation path to the value in the runtime query request, e.g., 'inputs.user_id'

taxonomy_applications.scroll_filters.NOT.value.type

string

default:dynamic

Allowed value: "dynamic"

taxonomy_applications.scroll_filters.NOT.operator

enum<string>

default:eq

Comparison operator

Available options:

eq,

ne,

gt,

lt,

gte,

lte,

in,

nin,

contains,

starts_with,

ends_with,

regex,

exists,

is_null,

text

Example:

[
  {
    "field": "department",
    "operator": "eq",
    "value": "HR"
  },
  {
    "field": "location",
    "operator": "eq",
    "value": "remote"
  }
]

taxonomy_applications.scroll_filters.case_sensitive

boolean | null

default:false

Whether to perform case-sensitive matching

Example:

true

Example:

null

cluster_applications

ClusterApplicationConfig · object[] | null

NOT REQUIRED. List of clusters to automatically execute when batch processing completes. Each entry specifies: cluster_id, auto_execute_on_batch, min_document_threshold, cooldown_seconds. Clusters enrich source documents with cluster assignments (cluster_id, cluster_label, etc.). Empty/null if no clusters attached. Use for: Segmentation, grouping, pattern discovery.

Show child attributes

cluster_applications.cluster_id

string

required

ID of the cluster to execute (must exist and use this collection as input)

cluster_applications.auto_execute_on_batch

boolean

default:true

Automatically execute cluster when batch processing completes for this collection. If False, cluster must be executed manually via API.

cluster_applications.min_document_threshold

integer | null

Minimum number of documents required before executing cluster. If document_count < threshold, clustering is skipped. Useful to avoid clustering on small datasets.

cluster_applications.cooldown_seconds

integer

default:3600

Minimum time (in seconds) between automatic cluster executions. Prevents excessive re-clustering on frequent batch completions. Default: 3600 seconds (1 hour).

Example:

null

taxonomy_count

integer | null

Number of taxonomies connected to this collection

retriever_count

integer | null

Number of retrievers connected to this collection

Health

Namespaces

Buckets

Feature Extractors

Collections

Retrievers

Taxonomies

Clusters

Analytics

Tasks

Webhooks

Headers

Path Parameters

Response