MangoPig
168 lines
7.0 KiB
Markdown
168 lines
7.0 KiB
Markdown
# TODO
|
|
|
|
## Phase 0: Infrastructure (DONE)
|
|
|
|
- [x] Azure VMs created and configured (jt-ub, jt-win)
|
|
- [x] Ubuntu VM provisioned (Docker, Go, Python, rclone, zsh, dotfiles)
|
|
- [x] Directory structures on both VMs (William's layout)
|
|
- [x] Sample document generator (7 generators, Azure Claude content pools, CJK fonts)
|
|
- [x] Project restructure: Tracker/Setup.bat + Tracker/Code/{Sync,Tools,Tracker}
|
|
- [x] Windows rclone sync setup (Setup.bat, Win-Setup.ps1, sync.ps1, config.ini)
|
|
- rclone auto-download + self-contained install in _LLM/Code/Sync/rclone/
|
|
- Password-based SFTP auth (demo), config.ini for server details
|
|
- .sync_ignore generated on setup (excludes_LLM/, dotfiles, temp/lock files)
|
|
- 5-minute scheduled task (JingTian-Sync)
|
|
- Pull-first-then-push: bidirectional files pulled with --update, excluded from push
|
|
- Bidirectional file list in config.ini (e.g. Admin/Tracker.xlsx)
|
|
- Daily log rotation in _LLM/Code/Sync/logs/
|
|
- [x] Ubuntu VM SSH password auth enabled (Azure cloud-init was blocking it)
|
|
- [x] Both sync directions tested and working
|
|
- [x] Old JingTian-Rclone repo archived on Gitea
|
|
|
|
## Phase 1: Services (Services-First Approach)
|
|
|
|
Rationale: De-risk unknowns (Docling OCR quality, Ollama CPU inference speed) before building orchestrator.
|
|
|
|
### 1a. Docker Compose + Docling (DONE)
|
|
|
|
- [x] Docker Compose base (Docling service)
|
|
- [x] Test Docling API with sample PDFs (curl)
|
|
- [x] Validate OCR quality on real PDFs + generated samples
|
|
- [x] Test OCR on images (PNG) — works with `force_ocr=true`
|
|
|
|
**Docling API Spec:**
|
|
|
|
- Image: `ds4sd/docling-serve:latest`
|
|
- Port: `5001`
|
|
- Endpoint: `POST /v1/convert/file`
|
|
- Form params: `files=@<path>;type=application/pdf`, `to_formats=md`, `do_ocr=true`
|
|
- Response: `{ "document": { "md_content": "..." }, "status": "success" }`
|
|
- OCR engine: `easyocr` (default), supports CJK
|
|
|
|
**OCR Options:**
|
|
|
|
- `do_ocr=true` — enabled by default, processes bitmap content
|
|
- `force_ocr=true` — use for images/scanned PDFs (replaces existing text with OCR)
|
|
- `ocr_engine` — `easyocr` (default), `tesseract`, `rapidocr`, `tesserocr`, `ocrmac`
|
|
- `ocr_lang` — language codes (engine-specific), e.g. `ch_sim` for Simplified Chinese
|
|
|
|
**Tested Formats:**
|
|
|
|
- PDF (Filing Receipt) — extracted text, tables, CJK content ✅
|
|
- PNG (Email screenshot) — extracted subject, dates, recipient ✅
|
|
- PNG with `ocr_lang=ch_tra` — Traditional Chinese extracted correctly (官藥坊) ✅
|
|
|
|
**Portable Chinese OCR:**
|
|
|
|
- Auto-downloads `ch_sim` (Simplified) and `ch_tra` (Traditional) models on first run
|
|
- Models persist in `docling_models` volume
|
|
- First run takes ~1-2 min extra for model download; subsequent runs instant
|
|
- Use `ocr_lang=ch_tra` for HK/Taiwan, `ocr_lang=ch_sim` for Mainland China
|
|
|
|
### 1b. Ollama (DONE)
|
|
|
|
- [x] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode)
|
|
- [x] Test structured extraction prompt (curl)
|
|
- [x] Measure inference time per document (~15-30s on 4 vCPU)
|
|
- [x] Validate extraction accuracy (deadline, doc type, etc.)
|
|
|
|
**Ollama API Spec:**
|
|
|
|
- Image: `ollama/ollama:0.9.3`
|
|
- Port: `11434`
|
|
- Model: `qwen3:1.7b` (~1.4 GB, auto-pulled on startup)
|
|
- Endpoint: `POST /api/generate`
|
|
- Body: `{"model":"qwen3:1.7b","prompt":"...","stream":false,"options":{"temperature":0}}`
|
|
- Response: `{ "response": "<json>" }` (strip `<think>...</think>` tags in post-processing)
|
|
- Note: Qwen3 includes reasoning by default; `/no_think` leaves empty tags, so strip instead
|
|
|
|
### 1c. Python Tools Service (DONE)
|
|
|
|
- [x] FastAPI + openpyxl service
|
|
- [x] Generic Excel API (not tracker-specific)
|
|
- [x] Test with Invoice_Schedule_2026Q1.xlsx
|
|
|
|
**Tools API Spec:**
|
|
|
|
- Image: Custom Dockerfile (python:3.12-slim + fastapi + openpyxl)
|
|
- Port: `8000`
|
|
- Volume: `/data` → `/data/jingtian/BenjaminTeam`
|
|
|
|
**Endpoints:**
|
|
|
|
| Endpoint | Method | Description |
|
|
| -------------------- | ------ | ---------------------------- |
|
|
| `/health` | GET | Health check |
|
|
| `/excel/sheets` | GET | List sheet names in workbook |
|
|
| `/excel/schema` | GET | Get column headers |
|
|
| `/excel/read` | GET | Read all rows |
|
|
| `/excel/row/{n}` | GET | Get specific row (1-indexed) |
|
|
|
|
**Query Params:**
|
|
|
|
- `file_path` (required) — path relative to `/data/`
|
|
- `sheet_name` (required) — name of the sheet to read
|
|
- `header_row` (optional, default=1) — which row contains headers
|
|
|
|
**Tested:**
|
|
|
|
- Invoice_Schedule_2026Q1.xlsx with `header_row=4` — extracted 6 invoice rows ✅
|
|
- Columns: Client, Matter Ref, TM Number, Description, Amount (HKD), Due Date, Status
|
|
|
|
**Future Endpoints (Phase 2+):**
|
|
|
|
- `POST /excel/row` — Add new row
|
|
- `PUT /excel/row/{n}` — Update specific row
|
|
- `DELETE /excel/row/{n}` — Delete row
|
|
- `POST /excel/format` — Apply formatting
|
|
|
|
### 1d. Manual Integration Test (DONE)
|
|
|
|
- [x] Chain test: PDF → Docling → Ollama → JSON extraction
|
|
- [x] Chain test: PNG → Docling (Chinese OCR) → Ollama → JSON extraction
|
|
- [x] Chain test: DOCX → Docling → Ollama → JSON extraction
|
|
- [x] Chain test: XLSX → Tools API → Ollama → JSON summary
|
|
|
|
**Tested Files:**
|
|
|
|
| File | Type | Docling | Ollama | Key Extractions |
|
|
| --------------------------------- | ---- | ------- | ------ | ------------------------------------------------ |
|
|
| Filing_Receipt_307800905.pdf | PDF | ✅ | ✅ | TM#307800905, NOVA, Class 9, deadline 30-05-2026 |
|
|
| Email_Monee_...20260221.png | PNG | ✅ | ✅ | TM#306527151, 官藥坊, evidence of use |
|
|
| Letter_Re_TM306735835.docx | DOCX | ✅ | ✅ | TM#306735835, CALIFORNIA BABY, hearing 17-04-2026|
|
|
| Invoice_Schedule_2026Q1.xlsx | XLSX | ✅ | ✅ | HKD 474,000 total, 6 invoices, 3 sent/1 paid/2 draft |
|
|
|
|
**Notes:**
|
|
|
|
- Qwen3 1.7b correctly extracted Chinese trademark 官藥坊 from PNG
|
|
- All deadlines, TM numbers, client names extracted accurately
|
|
- Minor: XLSX currency guessed as USD (HKD in source) — fix with better prompting
|
|
|
|
## Phase 2: Go Orchestrator
|
|
|
|
- [ ] Init Go module in Tracker/
|
|
- [ ] Config loading (YAML)
|
|
- [ ] SQLite schema + migrations (documents, extractions, processing_log)
|
|
- [ ] Docling HTTP client
|
|
- [ ] Ollama HTTP client
|
|
- [ ] Tools service HTTP client
|
|
- [ ] File watcher (inotify on Ubuntu, 2-min stability check via SHA256 hash)
|
|
- [ ] State tracker integration (new/changed/skip logic)
|
|
- [ ] Pipeline orchestrator (read latest Tracker before writing, merge not overwrite)
|
|
|
|
## Phase 3: Integration + Testing
|
|
|
|
- [ ] End-to-end test (drop file on Windows → syncs to Ubuntu → pipeline → Tracker updates → syncs back)
|
|
- [ ] Smoke test with real PDFs + generated samples
|
|
- [ ] README documentation
|
|
|
|
## Future
|
|
|
|
- [ ] Key-based SFTP auth (replace password for production)
|
|
- [ ] Windows Go service (replace PowerShell scheduled task)
|
|
- [ ] File hash (SHA256) change detection on Windows side
|
|
- [ ] WhatsApp deadline notifications
|
|
- [ ] Processing retry logic
|
|
- [ ] Multi-client support
|
|
- [ ] Smarter bidirectional merge (cell-level, not file-level)
|