Skip to main content
POST
/
v1
/
retrievers
/
evaluations
/
datasets
Create evaluation dataset
curl --request POST \
  --url https://api.mixpeek.com/v1/retrievers/evaluations/datasets \
  --header 'Authorization: <authorization>' \
  --header 'Content-Type: application/json' \
  --header 'X-Namespace: <x-namespace>' \
  --data '
{
  "dataset_name": "<string>",
  "queries": [
    {
      "query_id": "<string>",
      "query_input": {},
      "relevant_documents": [
        "<string>"
      ],
      "relevance_scores": {}
    }
  ],
  "description": "<string>",
  "metadata": {}
}
'
{
  "dataset_id": "<string>",
  "dataset_name": "<string>",
  "queries": [
    {
      "query_id": "<string>",
      "query_input": {},
      "relevant_documents": [
        "<string>"
      ],
      "relevance_scores": {}
    }
  ],
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "namespace_id": "<string>",
  "internal_id": "<string>",
  "query_count": 123,
  "description": "<string>",
  "metadata": {}
}

Headers

​
Authorization
string
required

REQUIRED: Bearer token authentication using your API key. Format: 'Bearer sk_xxxxxxxxxxxxx'. You can create API keys in the Mixpeek dashboard under Organization Settings.

​
X-Namespace
string
required

REQUIRED: Namespace identifier for scoping this request. All resources (collections, buckets, taxonomies, etc.) are scoped to a namespace. You can provide either the namespace name or namespace ID. Format: ns_xxxxxxxxxxxxx (ID) or a custom name like 'my-namespace'

Body

application/json

Request model for creating a new evaluation dataset.

​
dataset_name
string
required

Unique name for this dataset

Required string length: 1 - 200
​
queries
GroundTruthQuery · object[]
required

List of queries with ground truth relevance labels

Minimum array length: 1
​
description
string | null

Description of what this dataset measures

​
metadata
Metadata · object

Additional metadata

Response

Successful Response

Complete evaluation dataset with metadata.

An evaluation dataset is a collection of queries with ground truth relevance labels, used to measure retriever quality.

​
dataset_id
string
required

Unique dataset identifier

​
dataset_name
string
required

Human-readable dataset name

​
queries
GroundTruthQuery · object[]
required

List of queries with ground truth

​
created_at
string<date-time>
required

When dataset was created

​
updated_at
string<date-time>
required

Last update timestamp

​
namespace_id
string
required

Namespace this dataset belongs to

​
internal_id
string
required

Internal organization ID

​
query_count
integer
required

Number of queries in dataset

​
description
string | null

Dataset description

​
metadata
Metadata · object

Additional metadata (e.g., labeling instructions, version info)