Job Persistence

Purple8 DocIntel (v0.12.5) includes an embedded durable job store (p8l log via SessionContentStore) out of the box — no external Redis or database sidecar is required.

Embedded Durable p8l Store (Default)

By default, DocIntel persists job state, extracted entities/relationships/tables, and SHA-256 content hashes in /data/docintel_jobs.p8l:

• Zero external dependencies: Runs in a single container without Redis or Postgres.
• Lazy worker initialisation (_LazySessionContentStore): Spawned CPU worker processes (SERVER__CPU_WORKERS) never replay /data/docintel_jobs.p8l on startup, keeping multi-core worker pools fast and memory-bounded.
• Incremental processing (get_by_hash): SHA-256 content hashes are indexed automatically so resubmitted documents return cached results in single-digit milliseconds with zero LLM calls.
• Sandbox session TTL: Jobs created with Cloud Sandbox keys are isolated per principal and automatically purged after SERVER__SANDBOX_SESSION_MINUTES (default 60 minutes).

Job lifecycle

queued → running → DONE
                 → FAILED
queued → cancelled (via DELETE /jobs/{id})

Status	Description
queued	Job created, waiting for a worker
running	Worker processing (parse → extract → emit)
DONE / completed	Extraction finished, results available
FAILED / failed	Extraction failed (error in job response)
cancelled	Cancelled via API

Incremental processing

New file bytes → SHA-256 hash
    ↓
get_by_hash(hash) → existing completed job?
    YES → copy result, mark completed, return instantly
    NO  → proceed with full extraction

API & MCP endpoints

Endpoint / Tool	Description
GET /jobs	List recent jobs (with limit, status, profile filters)
GET /jobs/{id}	Get job status + extraction results (entities, relationships, tables, metadata)
GET /jobs/{id}/stream	SSE stream of partial results
DELETE /jobs/{id}	Cancel or remove a job
MCP docintel_get_job	Poll or retrieve job results directly from an AI agent

See REST API Reference → and AI Agents & MCP Guide → for details.