Chinese OCR

This commit is contained in:
MangoPig
2026-02-21 23:04:57 +00:00
parent 430cdf32cf
commit 7387d17a20
+37 -9
View File
@@ -23,18 +23,46 @@
Rationale: De-risk unknowns (Docling OCR quality, Ollama CPU inference speed) before building orchestrator.
### 1a. Docker Compose + Docling
### 1a. Docker Compose + Docling (DONE)
- [ ] Docker Compose base (Docling service)
- [ ] Test Docling API with sample PDFs (curl)
- [ ] Validate OCR quality on real PDFs + generated samples
- [x] Docker Compose base (Docling service)
- [x] Test Docling API with sample PDFs (curl)
- [x] Validate OCR quality on real PDFs + generated samples
- [x] Test OCR on images (PNG) — works with `force_ocr=true`
### 1b. Ollama
**Docling API Spec:**
- Image: `ds4sd/docling-serve:latest`
- Port: `5001`
- Endpoint: `POST /v1/convert/file`
- Form params: `files=@<path>;type=application/pdf`, `to_formats=md`, `do_ocr=true`
- Response: `{ "document": { "md_content": "..." }, "status": "success" }`
- OCR engine: `easyocr` (default), supports CJK
- [ ] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode)
- [ ] Test structured extraction prompt (curl)
- [ ] Measure inference time per document
- [ ] Validate extraction accuracy (deadline, doc type, etc.)
**OCR Options:**
- `do_ocr=true` — enabled by default, processes bitmap content
- `force_ocr=true` — use for images/scanned PDFs (replaces existing text with OCR)
- `ocr_engine``easyocr` (default), `tesseract`, `rapidocr`, `tesserocr`, `ocrmac`
- `ocr_lang` — language codes (engine-specific), e.g. `ch_sim` for Simplified Chinese
**Tested Formats:**
- PDF (Filing Receipt) — extracted text, tables, CJK content ✅
- PNG (Email screenshot) — extracted subject, dates, recipient ✅ (CJK trademark garbled, may need `ocr_lang`)
### 1b. Ollama (DONE)
- [x] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode)
- [x] Test structured extraction prompt (curl)
- [x] Measure inference time per document (~15-30s on 4 vCPU)
- [x] Validate extraction accuracy (deadline, doc type, etc.)
**Ollama API Spec:**
- Image: `ollama/ollama:0.9.3`
- Port: `11434`
- Model: `qwen3:1.7b` (~1.4 GB, auto-pulled on startup)
- Endpoint: `POST /api/generate`
- Body: `{"model":"qwen3:1.7b","prompt":"...","stream":false,"options":{"temperature":0}}`
- Response: `{ "response": "<json>" }` (strip `<think>...</think>` tags in post-processing)
- Note: Qwen3 includes reasoning by default; `/no_think` leaves empty tags, so strip instead
### 1c. Python Tools Service