Bring Your Own Parser or Retriever

Divinci's RAG pipeline has two slots you can fill with your own service:

| Slot | toolType | Runs | Timeout ceiling | |---|---|---|---| | Document parser | RAG/Chunker | once per file, during ingest | 120 s | | Search backend | RAG/Retriever | on every chat turn against a vector that selected it | 5 s |

You register an HTTPS endpoint. Divinci calls it over the network — it never runs your code, and there is nothing to upload or deploy to us.

A retriever sits in the chat path

A parser runs once per file inside an ingest job. A retriever runs on every visitor question, and its results are placed in the model's context. The 5-second ceiling is not a suggestion — it is the budget a chat turn can afford.

The contract

A parser receives the raw file and returns NDJSON — one chunk object per line.

A retriever receives a JSON body and returns a JSON object:

// request
{ "query": "how do refunds work?", "topK": 5, "minScore": 0.62,
  "tenant": "wt_8cccddd8febb192d8954b7fa41f3538b", "filters": {}, "config": {} }

// response
{ "results": [
  { "id": "doc-42", "text": "Refunds are issued…", "score": 0.78,
    "metadata": { "title": "Refunds", "sourceUrl": "https://…" } }
] }

score must be in 0–1. If your backend produces an unbounded score (BM25, for example), squash it monotonically before returning — minScore is passed through untouched and shown to operators, so a raw score makes it meaningless.

metadata.sourceUrl becomes a citation link. Omit it rather than inventing one; a missing link stays missing instead of becoming a link to nowhere.

`tenant` is not your customer's workspace id

Each registration gets an opaque wt_… token, sent as tenant on every call. Use it to partition your own storage. It is deliberately not the Divinci workspace id — you should not learn that, and it does not change if the workspace is renamed.

Register it

CLI

export ACME_TOKEN=…                       # never pass a secret as a flag
divinci external-tools create \
  --type RAG/Retriever \
  --slug acme-search \
  --title "Acme Search" \
  --endpoint https://search.acme.example/divinci \
  --auth-type bearer \
  --auth-secret-env ACME_TOKEN

--auth-secret-env names an environment variable to read. ps is world-readable on most systems, so a credential passed literally is visible to every process on the machine for the life of the command.

SDK

await divinci.externalTools.create("ws-123", {
  toolType: "RAG/Retriever",
  slug: "acme-search",
  title: "Acme Search",
  endpointUrl: "https://search.acme.example/divinci",
  authType: "bearer",
  authSecret: process.env.ACME_TOKEN,
});

Web app

Setup → Data Sources → External Tools → Register a tool.

MCP

external_tool_create registers a tool, but cannot set a credential — authSecret is not part of its schema. MCP runs inside an LLM context, and a secret passed through it is a secret in a transcript. Register over the CLI, SDK or REST API, then manage it from MCP.

The endpoint must be publicly resolvable — registration refuses a host that does not resolve rather than accepting it and failing quietly at the first call.

The slug is immutable: your tool id derives from it (ext:<workspaceId>:<slug>), and that id is what a vector stores.

Prove it before pointing a vector at it

divinci external-tools health-check <id>

The probe sends a fixed synthetic request — never customer data — and requires a contract-conforming answer. A host that is merely reachable still fails it, which is the point: a 200 carrying the wrong shape is the failure mode a status check cannot see.

It is operator-triggered only. Nothing in the ingest path probes you, so a broken tool never becomes a loop of health checks against your service.

What happens when your service is down

Nothing catastrophic, by design:

1. A failed call contributes zero results to that chat turn. The turn still returns 200 and the visitor still gets an answer — from whatever other vectors the release has.

2. After five consecutive failures the circuit opens for 60 seconds and Divinci stops calling you. Skipped calls are still recorded, so an outage stays visible instead of going quiet the moment we back off.

3. A passing health check closes the circuit immediately.

The honest consequence

A release whose only vector is external answers from no context at all while you are down, and the model behaves as if it knows nothing. If that matters, give the release a first-party vector too.

Limits you cannot raise

timeoutMs and maxResponseBytes are clamped to platform ceilings. A tool may lower them, never raise them — they are the two quantities you control that would otherwise be unbounded, and they are what bounds the blast radius of a single call.

What it costs

Nothing. Every external tool is free: registering one bills neither you nor the workspace, and Divinci takes no cut.

The pricing field is still accepted so existing registrations keep working, but a declared rate is not billed to anyone and produces no income. Do not plan revenue around it.

Before you register a parser

A parser receives raw customer documents. That is a data-processing relationship between the workspace and you, not between the workspace and Divinci — make sure the workspace owner knows what your service does with what it receives, and where.

Retrieved text is stamped with its origin (externalSource.toolId) so that downstream it is distinguishable from the workspace's own corpus. It was never ingested, chunked or moderated by Divinci.