De-Identification (PII Redaction)

De-identification detects personal information in an end user’s message and removes it before the message is stored, used for retrieval, moderated, or sent to a model. It is the control you reach for when a Release will handle health, financial, or otherwise regulated content, and it is off by default.

It is configured per Release on the deIdentification object, and it is one of three Safety controls — the other two, which block and escalate rather than rewrite, are on Safety: Moderation & Flaggers.

Where it runs

Position is the whole point: de-identification is the first step of the send path, ahead of everything that could otherwise persist or transmit the original.

user message
   │
   ├─▶ 1. de-identify          ← original text ends here
   │
   ├─▶ 2. RAG retrieval        ┐
   ├─▶ 3. moderation           ├─ all operate on the redacted text
   ├─▶ 4. model generation     │
   └─▶ 5. stored transcript    ┘

Retrieval is queried with the redacted text, moderators never see the original, and what lands in the transcript is the redacted form.

The transformation is one-way

The original text is discarded, not archived. Nothing in the pipeline writes it to storage, so a redacted message cannot be un-redacted later — by you, by Divinci, or by a subpoena. That is deliberate, and it is what lets a redacted transcript fall outside the scope of the regulation that motivated turning this on. Plan any workflow that needs the raw text (manual review, dispute resolution) around its absence.

Enabling it

Set it through the SDK, which works on drafts and published Releases alike:

await divinci.releases.updateInWorkspace(workspaceId, releaseId, {
  deIdentification: {
    enabled: true,
    engine: "presidio",
    strategy: "redact",
    piiCategories: ["PERSON", "EMAIL_ADDRESS", "PHONE_NUMBER", "US_SSN", "MEDICAL_RECORD"],
    preserveContext: true,
  },
});

There is also a dedicated endpoint, useful when you would rather not read the release first. The two release states take different body shapes:

# PUBLISHED release — the body IS the config object, and this route
# refuses drafts ("Release is a draft"). Bumps the release's minor version.
curl -X POST \
  https://api.divinci.app/white-label/$WORKSPACE/release/$RELEASE/deidentification \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{
    "enabled": true,
    "engine": "presidio",
    "strategy": "redact",
    "piiCategories": ["PERSON","EMAIL_ADDRESS","PHONE_NUMBER","US_SSN","MEDICAL_RECORD"],
    "preserveContext": true
  }'

# DRAFT release — nested under `deIdentification` in the full draft body.
curl -X POST https://api.divinci.app/white-label/$WORKSPACE/release/$RELEASE \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{ "...full draft body...", "deIdentification": { "enabled": true, "...": "..." } }'

Omitting it still preserves it

releases.updateInWorkspace() carries the stored deIdentification forward on every save, so an update that does not mention it leaves a configured Release untouched. Passing a value overrides that carry-forward — which older clients could not do, and silently discarded when asked.

Configuration

Field	Type	Default	Notes
enabled	boolean	false	Master switch
engine	presidio | stanford | obi-roberta	presidio	See Engines — one of these does not work
strategy	redact | replace | hybrid	redact	See Strategies
piiCategories	PIICategory[]	[]	Empty = detect everything
preserveContext	boolean	true	Keep sentences semantically readable after substitution
customPatterns	{name, pattern, replacement}[]	—	Domain-specific regexes on top of the engine’s detectors
storeOriginal	boolean	false	Accepted and stored — but see the caveat below
failClosed	boolean	true	Not settable via the API — see Failure behaviour

The request body is validated strictly: an unrecognised key is rejected with De-Identification Config validation failed at <key>, rather than being ignored. Omitted keys fall back to the defaults above rather than to the Release’s current values, so send the whole object every time.

`storeOriginal` does not archive anything

The flag is accepted, persisted on the config, and deliberately excluded from workspace export — but no code path writes the original text anywhere. It cannot currently be used to build a reversible audit trail, and should be treated as inert rather than as a retention switch. The pipeline’s discard of the original (above) is unconditional.

Engines

Engine	Status
presidio	✅ Default. Microsoft Presidio; the most accurate detector
stanford	✅ Available
obi-roberta	⛔ Accepted by validation, but not implemented

If the configured engine fails at request time, the other working engine is tried as a fallback — presidio → stanford, and stanford → presidio.

`obi-roberta` will break every message on the Release

obi-roberta is a member of the type union and passes config validation, so you can save it without any error. There is no engine registered under that name, so at request time the lookup throws Unknown de-identification engine. It also has no fallback chain entry, so nothing recovers — and because the failure mode is fail-closed (below), every message on the Release is rejected with a 503 for as long as it is selected. Use presidio or stanford.

Strategies

redact (default) replaces each detected entity with a type token. Nothing false is introduced, which is why it is the safe default:

Hi, I'm John Smith. My email is john@example.com and my SSN is 123-45-6789.
→ Hi, I'm [PERSON]. My email is [EMAIL_ADDRESS] and my SSN is [US_SSN].

replace substitutes realistic synthetic values. The sentence stays natural, which can help a model answer well — at the cost of putting plausible-but-false details in the transcript:

→ Hi, I'm Michael Johnson. My email is michael.johnson@email.com and my SSN is 987-65-4321.

hybrid replaces ordinary entities with synthetic values but keeps hard tokens for the most sensitive categories (SSN, credit card, passport, medical record).

`hybrid` is not the same on both engines

Presidio implements the genuine mix described above. Stanford treats hybrid exactly like redact. So a Release that fails over from Presidio to Stanford changes redaction style mid-incident — safely (Stanford is the more conservative of the two), but visibly. If consistent output matters more than availability, pin redact, which both engines implement identically.

PII categories

Leaving piiCategories empty detects all categories — the safe default. Narrow it only when a category causes false positives that harm answers (in practice DATE_TIME, LOCATION, and NRP are the usual culprits on domain-specific content).

Group	Categories
Identity	PERSON, EMAIL_ADDRESS, PHONE_NUMBER, AGE, NRP, SOCIAL_MEDIA, GAMING_ID
Government ID	SSN, US_SSN, US_PASSPORT, US_DRIVER_LICENSE, ID_NUMBER, CERTIFICATE
Financial	CREDIT_CARD, IBAN_CODE, ACCOUNT_NUMBER, CRYPTO_ADDRESS
Health	MEDICAL_RECORD, HEALTH_PLAN, BIOMETRIC, PHOTO
Location & time	LOCATION, DATE_TIME
Technical	IP_ADDRESS, URL, DEVICE_ID, VEHICLE_ID

The set covers the HIPAA Safe Harbor identifiers plus modern ones (crypto addresses, social handles, device ids) that predate no regulation but leak just as effectively.

Failure behaviour

When the engine errors or is unreachable, the Release fails closed: the message is rejected with a 503 carrying reason: "DE_IDENTIFICATION_UNAVAILABLE" and a message telling the user to retry shortly. Nothing is stored and nothing reaches the model.

This is the only correct default for a redaction feature — the alternative is passing the unredacted original through at exactly the moment the safety net is down — but it has a real operational consequence worth stating plainly:

If the redaction service is down, the Release stops answering

A de-identification-enabled Release depends on the engine being reachable. Cold starts count: the client’s timeout is PRESIDIO_TIMEOUT_MS (default 30s), and the engine should be run with a warm minimum instance so a scale-from-zero does not read as an outage. Budget for this before enabling on a high-traffic Release, and alert on the 503 rather than discovering it through users.

failClosed: false exists in the type as a legacy fail-open escape hatch, but the API’s config schema does not accept the key and rejects it outright — so in practice every Release is fail-closed and the unsafe mode cannot be turned on through the API. Treat the field as documentation of intent, not as a knob.

A failure is logged as [DE-ID-FAILURE] and a successful redaction as [DE-ID-SUCCESS] with entity counts by type. Neither ever logs the text — not even the engine’s error string, which can echo the original.

Gotchas

• Enabled is not the same as working. Saving a config validates its shape, never that the engine is reachable. The first evidence of a bad engine choice is user-visible 503s, so send a test message immediately after enabling.
• It is per Release, not per workspace. Forking a Release carries the config forward; creating a new one does not. A second Release against the same data is unprotected until you configure it too.
• Redaction happens before retrieval. If your RAG corpus is keyed on terms the redactor removes (names, dates, locations), recall drops once this is on. Test retrieval quality with de-identification enabled, not before.
• The transcript is the redacted text. Anything downstream that reads transcripts — QA suites, fine-tuning exports, analytics — sees redacted content. That is usually the point, but it means training data collected after enabling differs in kind from data collected before.
• preserveContext affects quality, not safety. It keeps substitutions grammatical so the model still understands the sentence; it never widens what is detected.

Related

• Safety: Moderation & Flaggers — the other two Safety controls, and where de-identification sits in the send pipeline relative to them
• Security & Abuse Protection — the gate, rate limits, and spend caps that protect a public Release
• Releases — the Release object these settings live on