MangoPig
54 lines
2.3 KiB
Markdown
54 lines
2.3 KiB
Markdown
# JingTian-Tracker
|
|
|
|
Document processing pipeline for JingTian & Gongcheng IP/trademark workflow.
|
|
|
|
## Pipeline
|
|
|
|
```
|
|
Windows VM Ubuntu VM
|
|
(Docker Compose)
|
|
BenjaminTeam/
|
|
user drops files ──► rclone sync ──► inotifywait detects new files
|
|
│
|
|
State check (SQLite)
|
|
- new file → process
|
|
- changed file → re-process
|
|
- unchanged → skip
|
|
- tracker.xlsx → skip
|
|
│
|
|
Docling (text extraction + OCR)
|
|
│
|
|
Ollama / Qwen 3-1.7B (structured extraction)
|
|
→ deadline, doc type, assigned person, etc.
|
|
│
|
|
Tools service (openpyxl)
|
|
- read latest tracker.xlsx
|
|
- append new rows / update re-uploaded
|
|
- preserve manual edits
|
|
│
|
|
rclone sync tracker.xlsx back
|
|
│
|
|
tracker.xlsx updated ◄── rclone ◄── done
|
|
```
|
|
|
|
## Stack
|
|
|
|
| Service | Role | Language |
|
|
|-------------|-------------------------------|----------|
|
|
| `server` | Orchestrator, watcher, state | Go |
|
|
| `docling` | Text extraction, OCR | Python |
|
|
| `ollama` | LLM structured extraction | - |
|
|
| `tools` | Excel read/write, extensible | Python |
|
|
| `sqlite` | State tracking (file, no container) | - |
|
|
|
|
## Key Rules
|
|
|
|
- LLM only writes to Excel on **first processing** or **source doc re-upload**
|
|
- Mr. Choi can edit any cell freely — edits are preserved
|
|
- `_LLM/` directory is excluded from sync and file watching
|
|
- Windows watcher uses debounce (15s) and `.last_sync` to avoid feedback loops
|
|
|
|
## TODO
|
|
|
|
See `TODO.md`.
|