Chinese OCR
This commit is contained in:
+37
-9
@@ -23,18 +23,46 @@
|
||||
|
||||
Rationale: De-risk unknowns (Docling OCR quality, Ollama CPU inference speed) before building orchestrator.
|
||||
|
||||
### 1a. Docker Compose + Docling
|
||||
### 1a. Docker Compose + Docling (DONE)
|
||||
|
||||
- [ ] Docker Compose base (Docling service)
|
||||
- [ ] Test Docling API with sample PDFs (curl)
|
||||
- [ ] Validate OCR quality on real PDFs + generated samples
|
||||
- [x] Docker Compose base (Docling service)
|
||||
- [x] Test Docling API with sample PDFs (curl)
|
||||
- [x] Validate OCR quality on real PDFs + generated samples
|
||||
- [x] Test OCR on images (PNG) — works with `force_ocr=true`
|
||||
|
||||
### 1b. Ollama
|
||||
**Docling API Spec:**
|
||||
- Image: `ds4sd/docling-serve:latest`
|
||||
- Port: `5001`
|
||||
- Endpoint: `POST /v1/convert/file`
|
||||
- Form params: `files=@<path>;type=application/pdf`, `to_formats=md`, `do_ocr=true`
|
||||
- Response: `{ "document": { "md_content": "..." }, "status": "success" }`
|
||||
- OCR engine: `easyocr` (default), supports CJK
|
||||
|
||||
- [ ] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode)
|
||||
- [ ] Test structured extraction prompt (curl)
|
||||
- [ ] Measure inference time per document
|
||||
- [ ] Validate extraction accuracy (deadline, doc type, etc.)
|
||||
**OCR Options:**
|
||||
- `do_ocr=true` — enabled by default, processes bitmap content
|
||||
- `force_ocr=true` — use for images/scanned PDFs (replaces existing text with OCR)
|
||||
- `ocr_engine` — `easyocr` (default), `tesseract`, `rapidocr`, `tesserocr`, `ocrmac`
|
||||
- `ocr_lang` — language codes (engine-specific), e.g. `ch_sim` for Simplified Chinese
|
||||
|
||||
**Tested Formats:**
|
||||
- PDF (Filing Receipt) — extracted text, tables, CJK content ✅
|
||||
- PNG (Email screenshot) — extracted subject, dates, recipient ✅ (CJK trademark garbled, may need `ocr_lang`)
|
||||
|
||||
### 1b. Ollama (DONE)
|
||||
|
||||
- [x] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode)
|
||||
- [x] Test structured extraction prompt (curl)
|
||||
- [x] Measure inference time per document (~15-30s on 4 vCPU)
|
||||
- [x] Validate extraction accuracy (deadline, doc type, etc.)
|
||||
|
||||
**Ollama API Spec:**
|
||||
- Image: `ollama/ollama:0.9.3`
|
||||
- Port: `11434`
|
||||
- Model: `qwen3:1.7b` (~1.4 GB, auto-pulled on startup)
|
||||
- Endpoint: `POST /api/generate`
|
||||
- Body: `{"model":"qwen3:1.7b","prompt":"...","stream":false,"options":{"temperature":0}}`
|
||||
- Response: `{ "response": "<json>" }` (strip `<think>...</think>` tags in post-processing)
|
||||
- Note: Qwen3 includes reasoning by default; `/no_think` leaves empty tags, so strip instead
|
||||
|
||||
### 1c. Python Tools Service
|
||||
|
||||
|
||||
Reference in New Issue
Block a user