Parsing & Chunking

Chunking decides what retrieval can ever find. A parser that flattens a table into prose, or a chunker that splits mid-sentence, caps the quality of every answer downstream — no store or embedding model recovers it.

Chunkers

Tool id	Vendor	Credentials	Status	Divinci usage
@divinci-ai/langextract Sentence-aware text chunking with three strategies respecting sentence boundaries. Fast, in-process — no external API calls.	Divinci AI	Divinci-managed	available	—
@divinci-ai/liteparse-langextract LiteParse parses the source document locally (layout, reading order, bounding boxes), then LangExtract sentence-chunks the extracted text. Best of both: faithful binary-document parsing plus sentence-aware chunks.	Divinci AI	Divinci-managed	available	—
@divinci-ai/openparse [DEPRECATED] Use LangExtract instead. Open-Parse semantically converts PDF files to text chunks but has been deprecated due to memory limits on large PDFs.	OpenParse	Divinci-managed	⚠️ deprecated	—
@divinci-ai/record-chunker One chunk per record, split on an explicit delimiter. For pre-chunked structured corpora (tables, catalogues, record exports) where sentence- or layout-based chunking would destroy row boundaries.	Divinci AI	Divinci-managed	available	—
@unstructured Unstructured is a tool for converting raw files to text chunks.	Unstructured	BYOK	available	—

Usage figures for this category are not collected yet — the weekly job populates them.

LangExtract is the default for documents: it uses a model to segment content along semantic boundaries rather than at a fixed character count. LiteParse + LangExtract is the lighter-weight pairing for content that is already close to plain text. The record chunker is for structured rows — products, records, catalog entries — where one item is one chunk and splitting it would be wrong. Unstructured is the BYOK option for teams already standardised on it.

Cleaners

Tool id	Vendor	Credentials	Status	Divinci usage
@divinci-ai/gim-cleaner Removes garbage text, fixes incomplete text, tracks and replaces pronouns and splits mixed text	Divinci AI	Divinci-managed	available	—

Usage figures for this category are not collected yet — the weekly job populates them.

Cleaning runs before chunking and removes the parts of a page that are not content — navigation, cookie banners, footers repeated on every page.

Cross-page boilerplate is detected, not guessed

On a crawl, text that appears on nearly every page is identified as boilerplate by comparison across the crawl rather than by a selector list, so it works on sites whose markup you have never seen. The consequence worth knowing: crawling a single page gives the detector nothing to compare against, so boilerplate survives. Crawl the section, not the page.

Auto-fix

Tool id	Vendor	Credentials	Status	Divinci usage
@divinci-ai/flag-chunks Analyzes retrieved context chunks and flags problematic ones for removal (contradictions, irrelevant content, outdated info, etc.)	Divinci AI	Divinci-managed	available	—
@divinci-ai/generate-chunk Will use an llm to generate new chunks to fix low scores. WARNING! This may cause hallucinations.	Divinci AI	Divinci-managed	available	—

Usage figures for this category are not collected yet — the weekly job populates them.

These are the tools behind chunk-level repair — flagging chunks that look wrong and generating replacements — rather than part of the ingest path.

See also

• RAG Knowledge Base
• Search & Scraping — what fetches the page in the first place.