Changelog

All notable changes to Purple8 DocIntel are documented here. For the full changelog, see CHANGELOG.md on GitHub.

---

v0.12.5 — 2026-10-30

Universal MCP Auth, Lazy Worker Job Store & 353.4 Pages/s Benchmark Validation

• Streamable HTTP MCP Auth (X-API-Key + Authorization: Bearer): require_api_key() now accepts X-API-Key, Authorization: Bearer, and ?api_key= across /mcp and all REST endpoints for both production keys and Cloud Sandbox keys.
• Smart auto_commit Default for Standalone & Sandbox Agents: docintel_ingest_text and docintel_ingest_url default auto_commit=False when no target graph_base_url is configured, returning extracted JSON directly without failing on an unconfigured graph target.
• Zero-Overhead CPU Worker Spawning (_LazySessionContentStore): Child cpu_pool worker processes no longer replay /data/docintel_jobs.p8l on import, eliminating startup latency and redundant memory usage across multi-worker pools.
• Live Sandbox MCP Config Snippets (/try): The Sandbox UI now renders live copy-paste MCP configs for both purple8-docintel and purple8-graph.
• Track 1 Breakpoint Benchmark Verified: 35.34 docs/s (353.4 pages/s, 70.7 tables/s) parse+layout throughput at 8 CPU workers; 0.91 docs/s (3,276 docs/hr) local GLiNER v4.1 zero-shot NER; 7.11 docs/s coupled /ingest/commit into Purple8 Hyper Graph.

---

v0.8.0 — 2026-04-09

Agentic chunking wire format + 56-format rule coverage — DocIntel now derives and emits chunking metadata that Purple8 Graph's ChunkingAgent consumes directly, eliminating redundant heuristic classification in P8G.

✨ Agentic Chunking Hints (Stage 3c)

• 5 new ExtractionResult fields — doc_type, chunking_hint, section_boundaries, suggested_embedding_model, doc_type_confidence
• doc_type — P8G registry key (e.g. "legal_contract", "code", "financial_report") derived from DocumentCategory
• chunking_hint — P8G ChunkingAgent vocabulary: "late", "syntactic", "semantic", "fixed"
• section_boundaries — char-offset list of section starts extracted via O(n) prefix search over LLM windows
• suggested_embedding_model — advisory Voyage specialist model (voyage-4-large, voyage-code-3, voyage-finance-2, etc.)
• doc_type_confidence — 1.0 for rule-based path, 0.9 for LLM classification path
• Pipeline Stage 3c — runs after chunking, before LLM extraction; logged under pipeline.p8g_hints
• EmitPayload.metadata — all 5 fields POSTed to Purple8 Graph /ingest/commit, consumed by ChunkingAgent

🐛 56-Format Rule Coverage Fix

• 7 code/config formats missing from _RULE_MAP — CSS, ENV, YAML, YML, TOML, INI, CFG were falling through to keyword/LLM path and landing as GENERAL (wrong doc_type, wrong embedding model)
• Fixed — all 7 now route to DocumentCategory.CODE via rule-based classification (confidence 1.0)
• Full 56-format coverage confirmed — 34 rule-based + 22 text/LLM paths; all classifying correctly

🧪 Tests

• 241 tests passing, 1 skipped (zero regressions)

---

v0.7.0 — 2026-04-06

Standalone product release — licensing infrastructure, commercial packaging, community docs, and GitHub templates.

🔑 Licensing Infrastructure

• License models — LicenseTier (Beta / Pro / Enterprise), LicenseFeature (11 gated features), LicenseQuotas, LicenseKey, LicenseStatus
• HMAC-SHA256 signed keys — cryptographic license key generation and offline validation
• License middleware — FastAPI middleware enforces tier/feature/quota per request
• Quota tracker — in-memory + Redis-backed (Lua atomic increment), monthly auto-reset, unlimited tier support
• CLI (purple8-license) — generate, validate, info, status subcommands
• License server stub — lightweight FastAPI activation/heartbeat/revocation server (SQLite-backed) in deploy/license-server/

🔧 License Generator

• scripts/generate_license.py — standalone script for generating signed license keys
• Supports --org, --email, --tier, --months, --org-id, --email-template flags
• --email-template outputs a ready-to-send email with install instructions and the key

💰 Commercial Packaging

• Pricing page — coming-soon placeholder (pricing finalized after beta)
• Self-hosting guide — Docker, Docker Compose, Kubernetes, air-gapped deployment instructions
• Licensing page — private install, key activation, environment variable setup, tier features
• Homepage repositioned — standalone product branding, "Start Free Beta" CTA
• Introduction updated — standalone product positioning (no longer described as a microservice)

📋 GitHub Templates

• Bug report (bug_report.yml) — structured form: OS, Python version, DocIntel version, steps to reproduce, expected/actual behavior, logs
• Feature request (feature_request.yml) — structured form: area dropdown (12 categories), problem, proposed solution, alternatives
• Issue config (config.yml) — disables blank issues, links to security email and general questions
• Pull request template — description, change type checkboxes, contributor checklist

📖 Community Docs

• Contributing — development setup, code style (PEP 8, Ruff, type hints), PR process, code of conduct
• Reporting Issues — bug reports, feature requests, security vulnerabilities, where to find issues
• Security Policy — supported versions, private reporting process, response timelines, security architecture overview

🧪 Tests

• 34 new licensing tests (tiers, features, encoding, validation, quotas, status)
• 238 tests passing, 4 skipped
• All 4 benchmarks passing

---

v0.6.7 — 2026-04-06

Four-feature upgrade — confidence calibration, table extraction, incremental processing, and multi-language prompts.

✨ Confidence Calibration

Temperature scaling (T = 2.0) corrects systematic LLM overconfidence via logit → scale → sigmoid transform. Raw 0.90 → calibrated ~0.74. Invoice ECE improved from 0.30 → 0.13.

Read more →

✨ Table Extraction

Structured tables from PDFs, DOCX, and XLSX are converted to markdown and injected as dedicated LLM context windows. Large tables auto-split with header preservation. Line items, amounts, and dates in tabular form are no longer missed.

Read more →

✨ Incremental Processing

SHA-256 content hashing detects unchanged documents. Resubmitted files skip parse/extract/emit entirely and return the cached result instantly. Zero cost for duplicate submissions across all connectors.

Read more →

✨ Multi-Language Prompts

Rich extraction guidance for 75+ languages — German, French, Spanish, Chinese, Japanese, Korean, Arabic, Hindi, Russian, and more. Per-language conventions for entity naming, transliteration, dates, and JSON key normalisation.

Read more →

🧪 Tests

• 29 new tests across 4 test classes
• 204 tests passing, 4 skipped
• All 4 benchmarks passing (contracts, invoices, scientific_papers, bim_models)

---

v0.6.6 — 2026-04-06

Extraction quality overhaul — the pipeline was systematically missing dates, monetary values, reference numbers, and document names.

🐛 Fixed: General profile too restrictive

• Added 4 entity types to the general ontology: Date, Value, Reference, Document
• Added 4 relationship types: HAS_DATE, HAS_VALUE, IDENTIFIED_BY, REFERENCES
• Entity count: 8 → 12 | Relationship count: 6 → 10

🐛 Fixed: Prompt blocking legitimate entities

• Removed "DO NOT EXTRACT: Dates, amounts, or percentages as standalone entities" from the general entity prompt
• Added explicit positive extraction rules for dates, values, and reference codes
• Raised entity cap: 20 → 30 per chunk

🐛 Fixed: Classifier routing all text to general profile

• Added keyword heuristic fast path — scans first 3000 chars for domain signals (13–14 keywords per domain)
• Threshold: ≥3 keyword hits → confident classification without LLM call
• Contracts → contract profile, invoices → financial profile, scientific papers → scientific profile

🐛 Fixed: Sparse benchmark gold labels

• Expanded gold labels across all 4 domains to match legitimate extraction output
• Benchmark version: 0.5.3 → 0.5.4

---

v0.6.5 — 2026-04-06

Bug fix release — 5 server/pipeline fixes + port collision resolution.

🐛 Fixed

• Port collision: Default port changed from 8200 → 8201 to avoid conflict with Purple8 Graph
• SSE streaming: Fixed async for on non-async generator in job streaming endpoint
• Redis pub/sub: Fixed channel name mismatch between publisher and subscriber
• Connector auth: Fixed SharePoint OAuth token refresh using wrong grant type
• Error handling: Fixed unhandled KeyError when profile not found in registry
• SQLite WAL: Fixed journal mode not set on connection open

---

Roadmap

See the GitHub Issues for planned features and known issues.

Key areas of active development:

• Configurable temperature — expose T as an environment variable
• Profile-aware incremental — re-extract when profile changes for same content hash
• Additional languages — expanding from 15 to 30+ languages
• Table structure preservation — nested/merged cell handling
• Streaming table extraction — emit table entities as they're found