# RAG-Based Chatbot for Accountants
## Project Plan

**Version:** 2.0  
**Date:** June 25, 2026  
**Status:** Planning — not started  
**Deployment:** 100% local / on-premise — no cloud APIs, no GPU required

---

## 1. What We Are Building

A **local RAG chatbot** for accountants that:

- Answers questions about financial data, transactions, and accounting concepts
- Uses your own data — never sends it to the cloud
- Runs on a normal CPU server (no GPU needed)
- Improves over time through user feedback (thumbs up/down, corrections)

**Test frontend:** Avatar + live chat is being tested at  
`https://staging.vipankumar.in/avatar/sample-page/`

---

## 2. Goals

| Goal | Detail |
|------|--------|
| Accurate answers | Grounded in your data, with source citations |
| Private & local | All processing on your own server |
| Simple to run | CPU only, no GPU, minimal infrastructure |
| Easy to improve | Feedback loop so answers get better over time |
| Audit-ready | Log all queries and responses |

---

## 3. Who Will Use It

| User | Example Question |
|------|-----------------|
| Staff Accountant | "Show AP transactions over $500 in Q3 2023" |
| Senior Accountant | "What is the debt-to-equity ratio for oil drilling companies?" |
| Controller / CFO | "Summarize profit margins from our analytics data" |
| Admin | Manage knowledge base, review feedback |

---

## 4. Technology Stack

### Languages

| Language | Used For |
|----------|----------|
| **Python 3.11+** | Backend, RAG pipeline, data processing |
| **HTML / CSS / JavaScript** | Chat UI (can embed in WordPress avatar page) |
| **SQL** | Storing feedback, logs, and metadata |

### Models (All Local, CPU-Only)

| Role | Model | Ollama Command | Size |
|------|-------|----------------|------|
| **Chat / answers** | Phi-3 Mini 4K Instruct | `ollama pull phi3:mini` | ~2.3 GB |
| **Embeddings** | BGE-small-en-v1.5 | via `sentence-transformers` | ~130 MB |

> One small LLM handles everything — chat, summarization, and simple data queries. No second model, no re-ranker needed for MVP.

### Software

| Component | Tool | Why |
|-----------|------|-----|
| Backend API | **FastAPI** | Lightweight Python API |
| RAG framework | **LangChain** | Retrieval + prompt chains |
| LLM runtime | **Ollama** | Easy local model serving on CPU |
| Vector store | **ChromaDB** | Simple, file-based, no setup |
| Database | **SQLite** | Feedback, sessions, audit logs |
| Data processing | **pandas** | CSV / tabular queries |
| PDF ingestion | **pdfplumber** | Policy documents (later) |
| Frontend | **WordPress avatar page** (current) or simple HTML chat widget | Already in testing |

### What We Will NOT Use

| Excluded | Reason |
|----------|--------|
| OpenAI / GPT-4 / Claude | Cloud — data leaves your server |
| Pinecone / cloud vector DBs | Cloud — data leaves your server |
| GPU / CUDA / NVIDIA drivers | Not required — CPU-only setup |
| PostgreSQL / Redis / Celery | Too complex for MVP — SQLite is enough |
| React / TypeScript build chain | Unnecessary — embed chat in existing WordPress page |

---

## 5. System Architecture

```
User (WordPress Avatar Page / Chat Widget)
        │
        ▼
   FastAPI Backend  (Python, port 8000)
        │
        ├──► ChromaDB          (vector search over your data)
        ├──► SQLite            (feedback, logs, sessions)
        ├──► pandas            (filter/aggregate CSV data)
        └──► Ollama            (phi3:mini on CPU)
                │
                └──► sentence-transformers  (BGE-small embeddings)
```

### How a Query Works

```
1. User asks a question
2. System searches ChromaDB for relevant data chunks
3. Top 5 chunks + question sent to Phi-3 Mini (local)
4. LLM generates answer with source citations
5. User sees response; can rate it (thumbs up/down)
6. Feedback saved to SQLite for admin review
```

---

## 6. Dataset

### Current Data (Ready)

**`final_finance_dataset.csv`** — 105,708 records

| Category | Records |
|----------|---------|
| Accounting transactions | 100,000 |
| Company financial statements | 4,668 |
| Accounting analytics | 1,000 |
| Investment survey | 40 |

Each row has:
- `text_content` — used for search/embedding
- `metadata_json` — structured fields for filtering
- `data_category` — type of record

### Data to Add Later

- Internal accounting policies (PDF)
- Chart of accounts
- GAAP / IFRS reference summaries

---

## 7. Security

| Rule | How |
|------|-----|
| No cloud APIs | Block outbound internet on the server |
| Data stays local | Ollama + ChromaDB run on same machine |
| Encrypt in transit | HTTPS (TLS) on the API |
| Access control | Simple login (JWT) — expand to LDAP later if needed |
| Audit logs | Every query + response logged in SQLite |
| No hallucination | Bot says "I don't know" if no relevant data found |
| Disclaimers | Bot does not give tax/legal advice |

---

## 8. Feedback Loop

```
User rates answer (👍 / 👎)
        │
        ▼
Saved to SQLite
        │
        ├── 👎 + correction → Admin reviews → Add to knowledge base
        ├── Low-rated queries → Fix chunking or add missing data
        └── Monthly review → Update prompts, re-index if needed
```

| Feedback Type | What Happens |
|---------------|-------------|
| Thumbs up | Logged as positive — no action needed |
| Thumbs down | Flagged for admin review |
| Correction text | Admin approves → added to knowledge base |
| Repeated failures | Trigger data ingestion or prompt update |

---

## 9. Hardware Requirements (CPU Only)

| Component | Minimum |
|-----------|---------|
| CPU | 4+ cores (any modern Intel/AMD) |
| RAM | 16 GB (32 GB recommended) |
| Storage | 50 GB free (models + data + logs) |
| GPU | **Not required** |
| OS | Ubuntu 22.04 / Windows / macOS |
| Internet | Only needed once to download models |

### Expected Performance (CPU)

| Metric | Expected |
|--------|----------|
| First response | 10–30 seconds |
| Follow-up responses | 5–15 seconds |
| Embedding 105K records | ~30–60 minutes (one-time) |
| Concurrent users | 1–5 comfortably |

---

## 10. Python Dependencies

```
fastapi
uvicorn
langchain
langchain-community
chromadb
sentence-transformers
pandas
pdfplumber
python-jose
passlib
httpx
python-dotenv
```

Install Ollama separately: https://ollama.com

---

## 11. Implementation Phases

### Phase 1 — MVP (Weeks 1–3)
- Install Ollama + pull `phi3:mini`
- Ingest `final_finance_dataset.csv` into ChromaDB
- Build FastAPI `/chat` endpoint with RAG
- Connect to WordPress avatar/chat test page
- Basic prompt + citation formatting

### Phase 2 — Feedback & Security (Weeks 4–6)
- Thumbs up/down in chat UI
- SQLite feedback storage + simple admin view
- Login (JWT)
- Query/response audit logging

### Phase 3 — Improve & Expand (Weeks 7–10)
- Ingest policy PDFs and additional documents
- pandas-based filtering for transaction queries ✅ (`app/rag/structured.py`)
- Review feedback, fix bad answers, re-index
- User guide and deployment docs
- WordPress / avatar `POST /chat` integration

---

## 12. Risks

| Risk | Mitigation |
|------|------------|
| Slow responses on CPU | Use small model (Phi-3 Mini); set user expectations |
| Wrong financial figures | Always cite sources; refuse when data is missing |
| Small model quality | Good RAG context compensates; upgrade model later if needed |
| Sensitive data in logs | Encrypt logs; limit admin access |

---

## 13. Success Metrics

| Metric | Target |
|--------|--------|
| User satisfaction (thumbs up) | > 75% |
| Answers with citations | 100% |
| Data sent to cloud | 0 |
| Average response time | < 30 seconds |
| Audit log coverage | 100% of queries |

---

## 14. Open Decisions

| # | Question | Options |
|---|----------|---------|
| 1 | Auth method | Simple JWT / LDAP (enterprise) |
| 2 | Compliance content | US GAAP / IFRS / India-specific |
| 3 | Who reviews feedback? | Admin / Senior accountant |
| 4 | ERP integration (later) | QuickBooks / Xero / Tally / None |

---

## 15. Project Files

```
accountants_chatbot/
├── final_finance_dataset.csv    ← Main dataset (105,708 rows)
├── combine_datasets.py            ← Script to rebuild dataset
├── Finance_data.csv               ← Source files
├── financial_accounting.csv
├── archive (1)/accounting_data.csv
├── archive (2)/dataset/dataset/  ← Company financial CSVs
└── PROJECT_PLAN.md               ← This document
```

---

## 16. Quick Reference

| Item | Choice |
|------|--------|
| Language | Python |
| LLM | Phi-3 Mini (Ollama, CPU) |
| Embeddings | BGE-small-en-v1.5 |
| Vector DB | ChromaDB |
| Database | SQLite |
| API | FastAPI |
| Frontend | WordPress avatar page (testing) |
| GPU | Not needed |
| Cloud | Not used |
| Dataset | final_finance_dataset.csv (105K rows) |

---

*Planning document only. No implementation started.*