Adding Custom Integrations

Datafuse makes Adding Custom Integrations incredibly easy and efficient. It allows developers to import any HTTP API, Swagger/OpenAPI specification, or custom API configuration and compile it instantly into a schema-safe, runtime-ready integration that AI models can execute.

The Custom Integrations Pipeline

When you paste an API definition or write a custom integration config, Datafuse runs it through a multi-stage validation and compiler pipeline to generate tools:

• Ingestion: Supports both raw YAML configurations and fully qualified OpenAPI v3 specifications.
• Normalization: Standardizes authentication schemes (Bearer, OAuth2, API Key) and HTTP parameters into flat query/body JSON schemas.
• Compilation: Creates functional tool definitions, ensuring is_mutating rules match (POST, PUT, DELETE are mutating, while GET is read-only).

Defining Custom Integrations in YAML

To convert any standard third-party REST API into agent tools, you define the integration mappings.

Here is an example definition for adding a custom integration for Ably Realtime (ably.yaml), representing the actions and schemas directly in Datafuse:

key: ably
title: Ably Realtime
description: Build real-time messaging, collaboration, and pub/sub solutions.
category: Communication
runtime_type: rest

auth_schemes:
  - key: api_key
    type: api_key
    placement:
      location: header
      name: Authorization
      prefix: "Basic "
    credential_fields:
      - name: api_key
        label: Ably API Key
        required: true
        is_secret: true

metadata:
  base_url_template: https://rest.ably.io/
  ping_endpoint: /time

actions:
  - slug: publish_message
    name: Publish Message
    description: Publish a real-time message to a specific pub/sub channel.
    endpoint: /channels/{channel_id}/messages
    method: POST
    is_mutating: true
    input_schema:
      type: object
      properties:
        path:
          type: object
          required:
            - channel_id
          properties:
            channel_id:
              type: string
              description: The Ably channel ID.
        body:
          type: object
          required:
            - name
            - data
          properties:
            name:
              type: string
              description: The event name.
            data:
              type: string
              description: The message payload.

Action Parsing Mechanics

During compilation, Datafuse processes custom integrations to guarantee safe agent runtime invocation:

Path Parameter Interpolation

If your endpoint template contains path variables (e.g. /channels/{channel_id}/messages), Datafuse parses the path argument key from the agent payload and dynamically rewrites the URL at runtime.

Mutation Isolation

AI models must know if an action causes side-effects. Datafuse sets the is_mutating flag true for any HTTP methods other than GET. This allows you to restrict writing actions behind human-in-the-loop approvals at the SDK level.

Using the Dashboard to Add Custom Providers

To register a custom provider manually:

1. Open the Integrations page in the Datafuse dashboard (app.datafuse.xyz/integrations).
2. Click Add Custom Integration and fill in the key, title, category, and runtime type, then paste your OpenAPI JSON/YAML spec or Omnithium-style YAML actions into the schema field.
3. Save. This calls POST /api/v1/providers and parses your spec into IntegrationAction rows on the new provider.
4. Connect the provider (creates an Integration record for your user) and call POST /api/v1/integrations/{id}/resolve-tools to confirm the parsed actions came through as tools.

!NOTE Once you create a custom provider, you can connect to it and call its resolved tools right away, with no build or redeploy step. It stays private to your account.

Registering via the API Directly

You can skip the dashboard and call the provider API directly with a JSON body matching CustomProviderCreate:

curl -X POST "$DATAFUSE_API_URL/api/v1/providers" \
  -H "Authorization: Bearer $DF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "ably",
    "title": "Ably Realtime",
    "description": "Build real-time messaging and pub/sub solutions.",
    "category": "communication",
    "runtime_type": "rest",
    "discovery_mode": "static",
    "base_url": "https://rest.ably.io",
    "auth_schemes": "api_key",
    "schema_definition": "actions:\n  - slug: publish_message\n    name: Publish Message\n    endpoint: /channels/{channel_id}/messages\n    method: POST\n"
  }'

schema_definition accepts either the Omnithium-style actions: YAML shown above or a raw OpenAPI v3 document (JSON or YAML) with a paths object. The server auto-detects which shape it received and parses accordingly (server/catalog/schema_parser.py).

Privacy and Export

A custom provider is private to the account that created it. It appears in your catalog and nobody else's, and only you can connect integrations to it.

You can download the definition of a custom provider you created as YAML, to keep it in version control or move it to a self-hosted install:

curl https://api.datafuse.xyz/api/v1/providers/{key}/export \
  -H "Authorization: Bearer $DF_TOKEN"

The export never contains credentials. Built-in catalog providers cannot be exported.