Finance-Accounts/plan.md

# AR Aging App — Production-Ready Plan

**Date:** 19 Aug 2026
**Goal:** Upload Jan file → Jan data is saved and shown. Upload Feb later → Feb becomes a NEW month dataset; all previous months stay saved and selectable (never merged). Hosted on AWS with backups, automatic exchange rates, and a 5-user login.

**Bottom line:**

| | |
|---|---|
| AWS hosting cost | **≈ $50–55 / month** |
| Exchange-rate API | **$0 / month** (Frankfurter, free) |
| Build effort | **~12–17 dev days**, 5 phases, each shippable on its own |

---

## 1. What is wrong today (why months "merge" or disappear)

The app already stores each month as its own closing session in the database — the foundation is right. Four defects break it:

1. **Double-count bug (verified in code):** re-uploading a file with the same name overwrites it on disk (`ar-aging-app/backend/app/api/routes/files.py:38`) but inserts a **second** database row (`files.py:74`). Processing then parses the same file twice and **doubles every number**.
2. **Nothing prevents two sessions for the same month** — creating "January" twice by accident produces two competing datasets, no warning.
3. **New months start from zero** — default `opening_mode="zero"` even though carry-forward logic already exists in `routes/ar.py`, so months lose continuity unless someone remembers to switch it.
4. **Previous months silently vanish from Accounts Summary** — re-processing clears journal approval (correct audit control), but the summary grid only shows approved months, so the old month disappears with no explanation.

## 2. Confirmed decisions

- **Exchange rates:** free **Frankfurter API** — $0, no API key, central-bank (ECB) rates, historical month-end dates. The human confirmation gate (Control C5) stays.
- **Login:** simple per-user login, **5 users**. Approver/reviewer names come from the logged-in user.
- **File sizes:** uploads are **300–500 MB** each → server sized with **8 GB RAM** for Excel parsing.

---

## 3. AWS architecture & monthly cost

### Recommended: one production server + S3 backups (≈ $50–55/mo)

**Lightsail 8 GB instance** ($44/mo: 2 vCPU, 8 GB RAM, 160 GB SSD, static IP included) — or equivalent EC2 `t4g.large` (≈ $61/mo with EBS + IPv4). Runs everything via a new `docker-compose.prod.yml`:

| Container | Role |
|---|---|
| **nginx** | Serves built React app, proxies `/api` to backend, HTTPS (Let's Encrypt), `client_max_body_size 2g` + long timeouts for big uploads |
| **backend** | FastAPI/uvicorn, **single worker** (background jobs are in-process — documented constraint, fine for 5 users) |
| **MySQL 8** | Data on instance disk. Code already supports MySQL (`AR_DB_BACKEND=mysql`) and ships `migrate_sqlite_to_mysql.py` |
| **backup cron** | Nightly `mysqldump` + `aws s3 sync` of uploads/exports → versioned S3 bucket (lifecycle → cold storage after 90 days), via IAM role (no keys on disk) |

### Cost breakdown

| Item | Monthly cost |
|---|---|
| Lightsail 8 GB (EC2 t4g.large route ≈ $61) | $44 |
| S3 backups (~60 GB year one, versioned) | $1.50–3 |
| Weekly instance snapshots | $2–4 |
| Route 53 hosted zone (optional domain) | $0.50 |
| Frankfurter FX API | $0 |
| **Total** | **≈ $50–55/mo** |

**Alternative (managed DB):** same server + **RDS MySQL db.t4g.small** ≈ **$85–100/mo** — buys automated patching + point-in-time restore. Not needed at this scale; upgrading later is a one-line config change (`MYSQL_HOST`).

**Storage decisions:**
- Uploads/exports stay on the **instance disk** — the parsing pipeline needs local file paths; round-tripping 500 MB files through S3 adds complexity for no benefit. S3 = durable **backup**, not primary storage.
- Data growth ~1 GB/month (~3.4M transaction rows) — well within sizing.
- Firewall inbound restricted to office IPs/VPN as defense-in-depth on top of login.

### Exchange-rate API pricing (researched)

| Provider | Free tier | Paid | Verdict |
|---|---|---|---|
| **Frankfurter** (chosen) | Unlimited, **no API key**, ECB rates, historical dates, self-hostable | — | ✅ $0, ideal for month-end closes |
| ExchangeRate-API | 1,500 req/mo, daily updates | Pro $10/mo (30k req, hourly) | fallback provider stub |
| Open Exchange Rates | 1,000 req/mo, USD base only | $12/mo | not needed |
| Fixer | 100 req/mo, no HTTPS on free | higher | ruled out |

App usage: ~13 marketplaces × a few fetches/month → even free tiers would never be exceeded.

---

## 4. Implementation phases

### Phase 1 — Month management & data separation (~3–4 days) ← HIGHEST PRIORITY

**1.1 Fix double-count bug** — `backend/app/api/routes/files.py`:
- Stream uploads to a temp `.part` name; move into place after hashing (also stops a failed upload corrupting an existing good file).
- **Content dedup:** SHA-256 match with an existing file in the session → skip, report "identical content already uploaded as X".
- **Same filename → UPDATE the existing row** instead of inserting a second one; if month already processed, flag `needs_reprocess` (existing banner handles messaging).
- Batch-friendly response `{files, skipped}` so one duplicate doesn't fail a 13-file upload; UI shows skipped/replaced notices.
- One-time `cli.py dedupe-files` cleanup for existing bad rows, then a unique index on (session_id, filename).

**1.2 One session per month (guide, don't hard-block)** — `routes/sessions.py`: 409 on duplicate `reporting_month` unless `allow_duplicate: true`; `NewClosing.tsx` shows "A closing for 2026-02 already exists — Open it | Create anyway".

**1.3 Default carry-forward** — `NewClosing.tsx`: default `openingMode="carry_forward"` when prior processed months exist (backend logic already exists); default month = month after latest close.

**1.4 Month selector UX ("show previous options"):**
- Sessions list ordered as a month timeline with a `journal_approved` flag per row.
- New `MonthSwitcher` dropdown in the closing header (month · name · status) — jump between months from any screen.
- Dashboard shows "published / processed-but-unpublished" and flags duplicate months.

**1.5 Stop previous months vanishing from Accounts Summary:**
- `accounts_summary.py` also returns pending months (processed/blocked but unapproved, incl. "approval cleared by re-processing").
- Grid renders them as greyed columns linking to the journal tab ("2026-01 processed but unpublished — re-approve the journal"). Audit control untouched.

**1.6 Read-only lock after completion** — new `ensure_editable()` guard on all mutating endpoints (409 on completed months); new `POST /sessions/{id}/reopen`; frontend disables edit controls on locked months.

**Tests:** filename replace, sha256 skip, duplicate-month 409, completed-session 409s.

### Phase 2 — Login, 5 users (~2 days)

- New `User` table (username, display name, bcrypt password hash); users created via CLI (`add-user` / `set-password`) — no self-signup.
- `POST /api/auth/login` → signed bearer token (12 h expiry, new `AR_SECRET_KEY` env); every route requires login except `/api/health` + login.
- **Real identity in sign-offs:** journal review/approve, FX confirm, control verify, payout entry all record the logged-in user — free-text "your name" boxes removed.
- Frontend: `Login.tsx`, token auto-attached, 401 → redirect to login, user + logout in sidebar.
- New deps: `passlib[bcrypt]`, `itsdangerous`.

### Phase 3 — AWS deployment (~3–5 days incl. migration dry-run)

1. Production backend image (no `--reload`, `--proxy-headers`, single worker documented).
2. Production frontend image: multi-stage `node:20` build → `nginx` serving static + API proxy.
3. New `docker-compose.prod.yml` + `example.env.production` (`AR_DB_BACKEND=mysql`, `AR_CORS_ORIGINS=https://<domain>`, `AR_SECRET_KEY`, FX vars); HTTPS via certbot; DNS → static IP.
4. **Data migration:** MySQL up → schema auto-created → run existing `migrate_sqlite_to_mysql.py` → verify per-table row counts + to-the-cent reconciliation spot check → cut over; archive SQLite file to S3. **Timed dry run first** (3.4M rows/month).
5. Backups: nightly dump + S3 sync, weekly snapshots, S3 versioning + lifecycle.
6. Deploy runbook: `git pull && docker compose -f docker-compose.prod.yml up -d --build`.

### Phase 4 — Exchange-rate service (~2 days)

- New `backend/app/services/fx_service.py`: provider abstraction — **Frankfurter** default (`api.frankfurter.dev/v1/{date}?base=USD&symbols=CAD,GBP,AUD,EUR,PLN,SEK,TRY`), paid-provider stub via `FX_PROVIDER` env.
- Rates **inverted to USD-per-local** (the app's convention) — documented + locked with a test; one inversion mistake would mis-state every non-USD receivable.
- Month-end fetch seeds `fx_rates` per session **unconfirmed** → Control C5 still blocks the close until a human confirms — workflow unchanged, just pre-filled. Daily fetch fills `fx_rates_daily`.
- Cache table makes re-fetch idempotent + works offline after first fetch; provider failure → clear 502 "enter rates manually", never a silent default.
- UI: "Fetch month-end rates" button in the C5 panel (`Controls.tsx`); "Fetch daily rates" in `ArLedger.tsx`. Update "no third-party egress" copy (only currency codes are sent, never financial data).
- ⚠️ Verify Frankfurter publishes TRY; if not, Turkey stays manual (C5 still enforces) or paid stub takes over.

### Phase 5 — Hardening (~1–2 days)

1. **Stale-job recovery:** months stuck "processing" after a restart → marked error with a re-run message (otherwise a deploy mid-job blocks the month forever).
2. **Retention:** auto-purge old generated **exports only** (`AR_RETENTION_DAYS`, currently dead config) — uploaded source files never auto-deleted (audit source).
3. Health check: DB ping + disk-writable.
4. Logging: request timing + job start/finish/fail, shipped via Docker logs.

Deferred nice-to-haves: CI pipeline, Sentry, login rate-limiting, Alembic, S3-primary storage, audit-log table, MFA.

---

## 5. Verification (acceptance checks)

- Upload same CSV twice → one file row, totals unchanged; renamed-identical file → skipped with message.
- Close Jan → create Feb → warns nothing, carries forward Jan balances; Jan stays selectable, read-only, unchanged; both months on Accounts Summary.
- Fetch month-end rates → unconfirmed → C5 fails → Confirm all → C5 passes; EUR rate checked against ECB published figure.
- Unauthenticated API call → 401; all 5 users can log in; approvals record the real approver name.
- Prod compose rehearsed locally with a 500 MB file (memory watched) **before buying the instance**; restart mid-processing recovers; nightly backup object appears in S3; restore drill succeeds.
- All existing tests pass, incl. Jan-2026 reconciliation integration test **to the cent** after MySQL migration.

## 6. Key risks

| Risk | Mitigation |
|---|---|
| SQLite→MySQL migration (~3.4M rows/month) | Timed dry run, per-table counts, cent-level reconciliation before cutover |
| FX rate orientation (USD-per-local vs local-per-USD) | Fixture test against a known EUR rate |
| Read-only lock missing an endpoint | Sweep all mutating routes during implementation |
| 8 GB RAM sizing assumption | Validate with the real 283 MB test workbook in local rehearsal before instance purchase |

---

*Pricing sources: [ExchangeRate-API](https://www.exchangerate-api.com/#pricing) · [Open Exchange Rates](https://openexchangerates.org/signup) · [Frankfurter](https://frankfurter.dev/) · AWS Lightsail/EC2/RDS public pricing, Aug 2026. Costs are USD estimates, on-demand.*