The API Call stage enriches documents by calling external HTTP APIs. This enables integration with third-party services like Stripe, GitHub, weather APIs, and more to augment documents with real-time data.
Stage Category : APPLY (1-1 Enrichment)Transformation : N documents → N documents (same count, expanded schema)
When to Use Use Case Description Customer data lookup Enrich with Stripe billing data, CRM info Repository info Fetch GitHub commit stats, stars Real-time data Add weather, stock prices, currency rates Data validation Verify addresses, phone numbers External context Lookup additional context from any API
When NOT to Use Scenario Recommended Alternative Untrusted/user-provided URLs Major security risk (SSRF) API credentials can’t be secured Use organization secrets vault High-volume enrichment Rate limits apply Time-critical responses Network latency adds 100-500ms Internal-only APIs behind firewalls Use sql_lookup for databases
Parameters Required Parameters Parameter Type Description urlstring API endpoint URL. Supports {DOC.field} and {INPUT.field} templates. allowed_domainsstring[] Domain allowlist for SSRF protection. Never use * . output_fieldstring Dot-path where API response should be stored (e.g., metadata.stripe_data).
Optional Parameters Parameter Type Default Description methodstring GETHTTP method: GET, POST, PUT, PATCH, DELETE authobject nullAuthentication configuration (see below) headersobject {}Additional HTTP headers (supports templates) bodyobject nullRequest body for POST/PUT/PATCH (JSON) timeoutinteger 10Request timeout in seconds (1-60) max_response_sizeinteger 10485760Maximum response size in bytes (default: 10MB) response_pathstring nullJSONPath to extract specific field from response rate_limitobject nullRate limiting per domain whenobject nullConditional filter for selective enrichment on_errorstring skipError handling: skip, remove, or raise
Authentication Types Bearer Token
API Key
Basic Auth
OAuth 2.0, JWT tokens - Most modern APIs (GitHub, OpenAI, Stripe){ "auth" : { "type" : "bearer" , "secret_ref" : "stripe_api_key" } }
Adds header: Authorization: Bearer {secret_value} Header or query parameter - Weather APIs, Maps, etc.{ "auth" : { "type" : "api_key" , "key" : "X-API-Key" , "location" : "header" , "secret_ref" : "weather_api_key" } }
Field Description keyHeader name or query param name locationheader (recommended) or querysecret_refSecret name in vault
HTTP Basic Authentication - Legacy systems{ "auth" : { "type" : "basic" , "secret_ref" : "basic_credentials" } }
Secret format: username:password Adds header: Authorization: Basic {base64(username:password)} Error Handling Strategy Behavior Best For skipKeep document unchanged Optional enrichment removeRemove document from results Mandatory enrichment raiseFail entire pipeline Debugging, critical failures
Configuration Examples Stripe Customer Lookup
GitHub Repository Stats
POST with JSON Body
Conditional Enrichment
Rate Limited with JSONPath
{ "stage_type" : "apply" , "stage_id" : "api_call" , "parameters" : { "url" : "https://api.stripe.com/v1/customers/{DOC.metadata.stripe_id}" , "method" : "GET" , "allowed_domains" : [ "api.stripe.com" ], "auth" : { "type" : "bearer" , "secret_ref" : "stripe_api_key" }, "output_field" : "metadata.stripe_data" , "timeout" : 10 , "on_error" : "skip" } }
Security SSRF Protection Required This stage makes external HTTP requests which can be exploited for Server-Side Request Forgery attacks. Always use allowed_domains to whitelist permitted domains.
Security Best Practices
Never use * in allowed_domains - Explicitly list each domainNever store credentials in configuration - Always use auth.secret_ref to reference vault secretsSet rate limits - Prevent abuse and excessive costsUse HTTPS - HTTP URLs are automatically upgradedAudit configurations - Review before deployment to prevent data exfiltration
Storing Secrets # Create a secret in the organization vault curl -X POST " $MP_API_URL /v1/organizations/secrets" \ -H "Authorization: Bearer $MP_API_KEY " \ -H "Content-Type: application/json" \ -d '{ "secret_name": "stripe_api_key", "secret_value": "sk_live_..." }'
Then reference in configuration:
{ "auth" : { "type" : "bearer" , "secret_ref" : "stripe_api_key" } }
Metric Value Latency per request 100-500ms (network dependent) Timeout range 1-60 seconds Max response size 10MB (configurable) Parallelization Documents processed concurrently
Template Variables URLs, headers, and body values support template variables:
Namespace Description Example DOCCurrent document fields {DOC.metadata.customer_id}INPUTQuery inputs {INPUT.api_version}
