diff --git a/Code/TODO.md b/Code/TODO.md index cc897d6..b6c858b 100644 --- a/Code/TODO.md +++ b/Code/TODO.md @@ -23,18 +23,46 @@ Rationale: De-risk unknowns (Docling OCR quality, Ollama CPU inference speed) before building orchestrator. -### 1a. Docker Compose + Docling +### 1a. Docker Compose + Docling (DONE) -- [ ] Docker Compose base (Docling service) -- [ ] Test Docling API with sample PDFs (curl) -- [ ] Validate OCR quality on real PDFs + generated samples +- [x] Docker Compose base (Docling service) +- [x] Test Docling API with sample PDFs (curl) +- [x] Validate OCR quality on real PDFs + generated samples +- [x] Test OCR on images (PNG) — works with `force_ocr=true` -### 1b. Ollama +**Docling API Spec:** +- Image: `ds4sd/docling-serve:latest` +- Port: `5001` +- Endpoint: `POST /v1/convert/file` +- Form params: `files=@;type=application/pdf`, `to_formats=md`, `do_ocr=true` +- Response: `{ "document": { "md_content": "..." }, "status": "success" }` +- OCR engine: `easyocr` (default), supports CJK -- [ ] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode) -- [ ] Test structured extraction prompt (curl) -- [ ] Measure inference time per document -- [ ] Validate extraction accuracy (deadline, doc type, etc.) +**OCR Options:** +- `do_ocr=true` — enabled by default, processes bitmap content +- `force_ocr=true` — use for images/scanned PDFs (replaces existing text with OCR) +- `ocr_engine` — `easyocr` (default), `tesseract`, `rapidocr`, `tesserocr`, `ocrmac` +- `ocr_lang` — language codes (engine-specific), e.g. `ch_sim` for Simplified Chinese + +**Tested Formats:** +- PDF (Filing Receipt) — extracted text, tables, CJK content ✅ +- PNG (Email screenshot) — extracted subject, dates, recipient ✅ (CJK trademark garbled, may need `ocr_lang`) + +### 1b. Ollama (DONE) + +- [x] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode) +- [x] Test structured extraction prompt (curl) +- [x] Measure inference time per document (~15-30s on 4 vCPU) +- [x] Validate extraction accuracy (deadline, doc type, etc.) + +**Ollama API Spec:** +- Image: `ollama/ollama:0.9.3` +- Port: `11434` +- Model: `qwen3:1.7b` (~1.4 GB, auto-pulled on startup) +- Endpoint: `POST /api/generate` +- Body: `{"model":"qwen3:1.7b","prompt":"...","stream":false,"options":{"temperature":0}}` +- Response: `{ "response": "" }` (strip `...` tags in post-processing) +- Note: Qwen3 includes reasoning by default; `/no_think` leaves empty tags, so strip instead ### 1c. Python Tools Service