# Connectors

> Crawl a website or sitemap into a RAG vector from the terminal — divinci connectors web-crawl/list/sync-status.

Manage data connectors without leaving your terminal — the same surface as
the Connectors page in the web app. Today the CLI focuses on the
**web_crawl** provider: point a connector at a website (or sitemap/URL feed)
and it crawls into a chosen RAG vector, optionally on a recurring schedule.

## Start a web crawl

```bash
# Crawl a site into a vector (follows sitemap.xml by default)
divinci connectors web-crawl https://docs.example.com --vector <vectorId>

# Re-crawl daily, and swap through a draft vector (see below)
divinci connectors web-crawl https://docs.example.com \
  --vector <vectorId> --recurring --swap

# Treat the URL as a sitemap/feed instead of a crawl root
divinci connectors web-crawl https://docs.example.com/sitemap-0.xml \
  --vector <vectorId> --feed

# Scope and shape the crawl
divinci connectors web-crawl https://example.com \
  --vector <vectorId> --limit 200 \
  --allowed-paths /docs,/blog --exclude-paths /docs/archive \
  --subdomains --product-catalog
```

Re-running the command for the same site **updates** the stored connector
(settings are persisted), so it's also how you correct a recurring
connector's configuration.

<Aside type="caution">
  If the target vector is attached to a **published release**, pass `--swap`.
  Without it, the live-vector guard rejects every chunk write — the crawl
  reports success but nothing new becomes retrievable. With `--swap`, the
  crawl ingests into a fresh draft vector and atomically repoints your live
  release(s) to it only once the crawl succeeds.
</Aside>

## List connectors & track syncs

```bash
divinci connectors list

divinci connectors sync-status <connectionId>
```

`list` shows each connection's provider, status, recurrence, last sync, and
source URL. `sync-status` shows the last 10 sync jobs with page counts and
any errors.

## See also

- [RAG Knowledge Base](/server/rag/) — the vectors these crawls ingest into.
- [MCP tool catalog](/mcp/tool-catalog/#groups--access) — `connector_web_crawl_start` and friends for agentic IDEs.
- `divinci rag crawl` — a one-shot crawl without creating a recurring connector.