Parsing & Chunking

Chunking decides what retrieval can ever find. A parser that flattens a table into prose, or a chunker that splits mid-sentence, caps the quality of every answer downstream — no store or embedding model recovers it.

Chunkers

Tool ids are not npm package names

The @divinci-ai/... strings in the table are Divinci's internal tool ids, the values you pass when selecting a chunker. They are not packages on npm. The published package is @divinci-ai/langextract-ts.

Tool id	Vendor	Credentials	Status	Divinci usage
@divinci-ai/langextract Sentence-aware text chunking with three strategies respecting sentence boundaries. Fast, in-process — no external API calls. Built on Divinci's TypeScript port (@divinci-ai/langextract-ts) of Google's LangExtract.	Google LangExtract	Divinci-managed	available	—
@divinci-ai/liteparse-langextract LiteParse (LlamaIndex) parses the source document locally (layout, reading order, bounding boxes), then LangExtract (Google; Divinci TypeScript port) sentence-chunks the extracted text. Best of both: faithful binary-document parsing plus sentence-aware chunks.	LlamaIndex LiteParse	Divinci-managed	available	—
@divinci-ai/openparse [DEPRECATED] Use LangExtract instead. Open-Parse semantically converts PDF files to text chunks but has been deprecated due to memory limits on large PDFs.	OpenParse	Divinci-managed	⚠️ deprecated	—
@divinci-ai/record-chunker One chunk per record, split on an explicit delimiter. For pre-chunked structured corpora (tables, catalogues, record exports) where sentence- or layout-based chunking would destroy row boundaries.	Divinci AI	Divinci-managed	available	—
@unstructured Unstructured is a tool for converting raw files to text chunks.	Unstructured	BYOK	available	—

Usage figures for this category are not collected yet — the weekly job populates them.

LangExtract is the default for documents: it splits text on sentence boundaries rather than at a fixed character count, in-process and with no model call. It is built on Divinci's TypeScript port, @divinci-ai/langextract-ts, of Google's open-source LangExtract. LiteParse + LangExtract is for binary documents such as PDFs: LiteParse (from LlamaIndex) parses layout and reading order locally, then LangExtract chunks the extracted text. The record chunker is for structured rows — products, records, catalog entries — where one item is one chunk and splitting it would be wrong. Unstructured is the BYOK option for teams already standardised on it.

Cleaners

Tool id	Vendor	Credentials	Status	Divinci usage
@divinci-ai/gim-cleaner Removes garbage text, fixes incomplete text, tracks and replaces pronouns and splits mixed text	Divinci AI	Divinci-managed	available	—

Usage figures for this category are not collected yet — the weekly job populates them.

Cleaning runs before chunking and removes the parts of a page that are not content — navigation, cookie banners, footers repeated on every page.

Cross-page boilerplate is detected, not guessed

On a crawl, text that appears on nearly every page is identified as boilerplate by comparison across the crawl rather than by a selector list, so it works on sites whose markup you have never seen. The consequence worth knowing: crawling a single page gives the detector nothing to compare against, so boilerplate survives. Crawl the section, not the page.

Auto-fix

Tool id	Vendor	Credentials	Status	Divinci usage
@divinci-ai/flag-chunks Analyzes retrieved context chunks and flags problematic ones for removal (contradictions, irrelevant content, outdated info, etc.)	Divinci AI	Divinci-managed	available	—
@divinci-ai/generate-chunk Will use an llm to generate new chunks to fix low scores. WARNING! This may cause hallucinations.	Divinci AI	Divinci-managed	available	—
@divinci-ai/jev-select-passage Fixes low scores only with text already in your corpus: Jev picks the passage that states the answer and it is added verbatim. When no passage states it, the question is escalated to a human instead of being filled.	Divinci AI	Divinci-managed	available	—

Usage figures for this category are not collected yet — the weekly job populates them.

These are the tools behind chunk-level repair — flagging chunks that look wrong and generating replacements — rather than part of the ingest path.

See also

• RAG Knowledge Base
• Search & Scraping — what fetches the page in the first place.