What is Purple8 DocIntel?

Purple8 DocIntel (v0.12.5) is a self-hosted, MCP-native document intelligence engine that extracts structured entities, relationships, tables, and metadata from 70+ document formats across 75+ languages using local GLiNER v4.1 zero-shot NER and domain-aware two-pass LLM extraction.

Point an AI agent (via /mcp) or REST client at any document — PDF, IFC model, contract, invoice, scientific paper, CAD drawing, email, whiteboard sketch — and it returns a typed, confidence-calibrated extraction result as JSON, or commits it directly into Purple8 Hyper Graph (POST /ingest/commit).

The problem it solves

Enterprise documents come in dozens of formats — each with its own structure, conventions, and vocabulary. Existing tools either do generic RAG chunking (losing structure and tables) or require manual schema definition for every document type.

DocIntel handles all of it with zero configuration:

• 70+ formats across 6 categories parsed natively (353.4 pages/s, 70.7 tables/s on 8 CPU workers) — no pre-conversion
• 75+ languages via Lingua detection and per-language extraction conventions
• Native MCP Server (/mcp) — AI agents call docintel_ingest_url, docintel_ingest_text, and docintel_get_job over Streamable HTTP using X-API-Key or Authorization: Bearer
• 5 domain profiles (aec, contract, financial, scientific, general) with built-in ontologies — auto-selected from content
• Hybrid GLiNER v4.1 + Two-pass LLM extraction — entities first, then relationships
• Confidence calibration — temperature scaling (T=2.0) produces scores you can actually trust
• Embedded durable job store (p8l) — zero external Redis dependency
• Self-hosted container — your documents never leave your infrastructure

Core pipeline

Document bytes
    ↓
Format detection (extension → magic bytes fallback)
    ↓
Incremental check — SHA-256 content hash → skip if already processed ← NEW
    ↓
Format-specific parser → ParsedDocument (text + structure + metadata)
    ↓
Table injection — raw_tables → markdown → dedicated LLM windows  ← NEW
    ↓
Domain classifier → extraction profile selected
    (keyword heuristic fast path · LLM fallback)
    ↓
Chunker (paragraph / section / clause / table-aware — per profile)
    ↓
Pass 1: entity extraction (all windows, parallel)
Pass 2: relationship extraction (given entity list, parallel)
    ↓
Confidence calibration (temperature scaling T=2.0)               ← NEW
    ↓
Fuzzy dedup (rapidfuzz WRatio ≥ 88) + confidence filter
    ↓
EmitPayload → POST /ingest/commit → Purple8 Graph

Key design decisions

Decision	Rationale
Two-pass extraction	Pass 1 builds a complete entity list so pass 2 can extract relationships between named entities — not ambiguous noun phrases
Domain profiles	Different industries have different ontologies. A legal doc and a BIM model need different entity types, chunking strategies, and confidence thresholds
ModelRegistry	One place to change models across all profiles — never hardcode model strings in profile code
Confidence calibration	Temperature scaling (T=2.0) corrects systematic LLM overconfidence; calibrated scores enable reliable threshold filtering
Table extraction	Tables converted to markdown and injected as dedicated windows — structured data no longer lost in text flattening
Incremental processing	SHA-256 content hash deduplicates resubmitted documents — zero LLM cost for unchanged files
Multi-language prompts	75+ languages with per-language naming, transliteration, and date conventions — not just a language tag
Async job pipeline	Documents can be large; extraction runs in the background with SSE streaming for partial results
SSRF guard	URL submissions are blocked from hitting RFC 1918 / cloud metadata endpoints

Part of the Purple8 platform (optional)

DocIntel works great as a standalone API — structured JSON output, webhook delivery, SSE streaming, no external dependencies beyond your LLM provider.

When you're ready for more, connect to Purple8 Graph for knowledge graph storage, embeddings, hybrid search, and Cypher queries.