Connectors

Manage data connectors without leaving your terminal — the same surface as the Connectors page in the web app. Today the CLI focuses on the web_crawl provider: point a connector at a website (or sitemap/URL feed) and it crawls into a chosen RAG vector, optionally on a recurring schedule.

Start a web crawl

# Crawl a site into a vector (follows sitemap.xml by default)
divinci connectors web-crawl https://docs.example.com --vector <vectorId>

# Re-crawl daily, and swap through a draft vector (see below)
divinci connectors web-crawl https://docs.example.com \
  --vector <vectorId> --recurring --swap

# Treat the URL as a sitemap/feed instead of a crawl root
divinci connectors web-crawl https://docs.example.com/sitemap-0.xml \
  --vector <vectorId> --feed

# Scope and shape the crawl
divinci connectors web-crawl https://example.com \
  --vector <vectorId> --limit 200 \
  --allowed-paths /docs,/blog --exclude-paths /docs/archive \
  --subdomains --product-catalog

Re-running the command for the same site updates the stored connector (settings are persisted), so it’s also how you correct a recurring connector’s configuration.

Caution

If the target vector is attached to a published release, pass --swap. Without it, the live-vector guard rejects every chunk write — the crawl reports success but nothing new becomes retrievable. With --swap, the crawl ingests into a fresh draft vector and atomically repoints your live release(s) to it only once the crawl succeeds.

List connectors & track syncs

divinci connectors list

divinci connectors sync-status <connectionId>

list shows each connection’s provider, status, recurrence, last sync, and source URL. sync-status shows the last 10 sync jobs with page counts and any errors.

See also

• RAG Knowledge Base — the vectors these crawls ingest into.
• MCP tool catalog — connector_web_crawl_start and friends for agentic IDEs.
• divinci rag crawl — a one-shot crawl without creating a recurring connector.