← All work
AI / Data Engineer2025

DocuParse

DocuParse turns dense SEC 10-K/10-Q PDFs into structured, verified data — a DVC-managed pipeline that combines native text + OCR, dual-model layout detection, and IBM's Docling, then cross-checks every figure against authoritative XBRL filings.

GitHub
DocuParse — SEC filing extraction pipeline
676
Pages processed
~3 min
End to end
6
Pipeline stages
2
Layout models
3
Export formats
3
Filing types

The problem

Filings resist parsing

  • Tables are the value

    Financial statements live in tables with merged cells, spans and multi-column layouts

  • One tool is not enough

    Text extractors miss structure; vision models miss content

  • Reprocessing is expensive

    Long-running stages make a naive rerun costly every time a model changes

Six stages

Decisions, not descriptions

Arrow keys to step

Stage 01of 06

Download

SEC EDGAR API

  • 10-K annual, 10-Q quarterly, 8-K current reports
  • Filing metadata stored beside the document for traceability
  • DVC content hash gates re-download

Decision

Chose
Content-hash the source
Over
  • Re-download on every run

Because

  • Filings are immutable once published
  • A hash check turns a network fetch into a no-op

Cost

A stale hash would silently skip a genuinely changed filing.

3
filing types

Architecture

DVC-managed DAG
DocuParse pipeline architectureSix DVC-managed stages, grouped into ingestion, structure extraction, and understanding and output. Download pulls 10-K, 10-Q and 8-K filings from SEC EDGAR and content-hashes them. Text extraction reads the native PDF layer first and falls back to OCR only where none exists, achieving 99.56 percent native extraction across 676 pages in about three minutes. Table detection combines Camelot for ruled tables with pdfplumber for whitespace-aligned ones, yielding 187 tables. Layout analysis chains Detectron2, which locates region bounding boxes, into LayoutLMv3, which reads text and layout jointly to resolve merged cells and spans. Docling runs in parallel as an enterprise benchmark against the custom path. Export writes JSON, Markdown and CSV per table and cross-validates 2,954 XBRL concepts against authoritative filings. Because DVC content-hashes every stage, changing the LayoutLMv3 version reruns layout, Docling and export only, not download or detection.DVC PIPELINE · CONTENT-HASHEDdvc repro · incrementalINGESTIONraw filings to textSTRUCTURE EXTRACTIONwhere and whatUNDERSTANDING AND OUTPUTparsed to validated01 · DOWNLOADSEC EDGAR APIEMITSSource PDFsFiling metadata JSONCompany + fiscal year tags· 10-K annual· 10-Q quarterly· 8-K current· Configured in params.yamlTRADEOFFContent-hash the sourceFilings are immutable once publishedHash check turns a fetch into a no-opRisk: a stale hash skips a real change3 filing typestraceable to sourcePDF02 · TEXT EXTRACTIONNative PDF + OCR fallbackEMITSPage-level textPer-page extraction methodWord Error Rate per page· Text layer read first· OCR only where none exists· 676 pages in ~3 minTRADEOFFNative first, OCR secondFilings are mostly born-digitalOCR on clean text adds errorsCost: two code paths to maintain99.56% nativeOCR on the remainder onlytext03 · TABLE DETECTIONCamelot + pdfplumberEMITSTable candidatesCell gridsPrecision / recall per page· Camelot: ruled tables· pdfplumber: whitespace-aligned· Both appear in one filingTRADEOFFTwo libraries, not oneNeither covers both table stylesCost: duplicate candidates to reconcile187 tablesfinancial statementsgrids04 · LAYOUT ANALYSISDetectron2 → LayoutLMv3EMITSRegion bounding boxesRegion classesCell spans + hierarchy· Detectron2: where tables are· Classes: title, header, body, figure· LayoutLMv3: text + layout jointly· Merged cells, spans, nestingTRADEOFFBoth models, chainedDetectron2 finds regions, ignores contentLayoutLMv3 needs coordinates as inputNeither alone parses a merged cellCost: two models per page2 modelsspatial then structuralstructure05 · DOCUMENT AIDocling (IBM)EMITSDocling document modelParallel parse of same input· Run beside the custom path· Head-to-head comparison· Written up as a reportTRADEOFFBenchmark against enterpriseOnly way to know custom work is justifiedThe comparison is a deliverableCost: two overlapping pipelines1 comparisoncustom vs Doclingdoc model06 · EXPORT + VALIDATEJSON / Markdown / CSVEMITSOne file per tableTagged filing ID + pageXBRL verification report· JSON for downstream Python· Markdown for LLM ingestion· CSV for analysts· Regression thresholds enforcedTRADEOFFThree formats at exportEach consumer wants a different shapeNegligible next to detect and parseCost: three artefacts to keep in sync2,954 conceptsXBRL cross-validatedchange the LayoutLMv3 version → only stages right of this line rerunXBRL CROSS-VERIFICATION2,954 concepts against authoritative filingscatches a confident parse of the wrong numberBENCHMARKING · BUILD VS BUYruntime, memory, bottleneck, drift, regression$1.05 / 1,000 pages vs $1.50 cloud floorSEC EDGAR → 10-K / 10-Q / 8-KJSON · Markdown · CSV
Six content-hashed stages. Each card carries what it emits, the tradeoff taken, and the measured result.

Build versus buy

Per 1,000 pages
  • Azure Form Recognizer$10.00–$50
  • Google Document AI$1.50–$50
  • AWS Textract$1.50–$50
  • DocuParse$1.05

What building bought

  • Complete data privacy, nothing leaves the environment
  • No API rate limits
  • Optimised for financial documents specifically
  • 30 to 98% cost saving at 1,000 pages

The three managed services were priced against this pipeline rather than dismissed. That is the difference between choosing to build and defaulting to it: at 1,000 pages the open-source path costs $1.05 against a floor of $1.50 and a ceiling of $50, and the privacy and rate-limit gains come with it rather than instead of it.

Quality assurance

Measured, not asserted

XBRL validation

Cross-check extracted values against authoritative financial data

Accuracy metrics

Word Error Rate, table precision and recall

Regression testing

Automated quality-threshold validation on every run

Distribution monitoring

Detect data drift and anomalies between runs

Measured pipeline results
MetricResultNote
Native text extraction99.56%OCR needed on the remainder only
Tables extracted187financial statements
XBRL concepts validated2,954against authoritative filings
Pages per run676in roughly 3 minutes
Cost per 1,000 pages$1.05open-source infrastructure

Stack

Everything used

Layout and vision

  • Detectron2
  • LayoutLMv3

Document AI

  • Docling
  • pdfplumber
  • Camelot
  • OCR

Orchestration

  • DVC
  • params.yaml
  • Python 3.9+

Source and validation

  • SEC EDGAR API
  • XBRL parsing

Outputs

  • JSON
  • Markdown
  • CSV

Analysis

  • Benchmarking
  • Cost modelling
  • Drift monitoring
See these across every project