Initial Services

This commit is contained in:
MangoPig
2026-02-21 22:34:40 +00:00
parent 060629ca94
commit 72c22c2396
2 changed files with 91 additions and 9 deletions
+33 -9
View File
@@ -19,18 +19,42 @@
- [x] Both sync directions tested and working
- [x] Old JingTian-Rclone repo archived on Gitea
## Phase 1: Foundation
## Phase 1: Services (Services-First Approach)
- [ ] Init Go module
- [ ] SQLite schema + migrations (documents, extractions, processing_log)
Rationale: De-risk unknowns (Docling OCR quality, Ollama CPU inference speed) before building orchestrator.
### 1a. Docker Compose + Docling
- [ ] Docker Compose base (Docling service)
- [ ] Test Docling API with sample PDFs (curl)
- [ ] Validate OCR quality on real PDFs + generated samples
### 1b. Ollama
- [ ] Add Ollama to Docker Compose (Qwen3-1.7B, CPU mode)
- [ ] Test structured extraction prompt (curl)
- [ ] Measure inference time per document
- [ ] Validate extraction accuracy (deadline, doc type, etc.)
### 1c. Python Tools Service
- [ ] FastAPI + openpyxl service
- [ ] Endpoints: read tracker, write/update rows, get column schema
- [ ] Test with sample Tracker.xlsx
### 1d. Manual Integration Test
- [ ] Chain test: PDF → Docling → Ollama → Tools → Excel updated
- [ ] Document any issues / adjustments needed
## Phase 2: Go Orchestrator
- [ ] Init Go module in Tracker/
- [ ] Config loading (YAML)
- [ ] Docker Compose (Docling, Ollama, Python Tools, Go server)
## Phase 2: Pipeline Core
- [ ] SQLite schema + migrations (documents, extractions, processing_log)
- [ ] Docling HTTP client
- [ ] Ollama HTTP client + structured extraction prompt
- [ ] Tools service (FastAPI + openpyxl Excel endpoints)
- [ ] Ollama HTTP client
- [ ] Tools service HTTP client
- [ ] File watcher (inotify on Ubuntu, 2-min stability check via SHA256 hash)
- [ ] State tracker integration (new/changed/skip logic)
- [ ] Pipeline orchestrator (read latest Tracker before writing, merge not overwrite)