11 KiB

Raw Blame History

AR Aging App — Production-Ready Plan

Date: 19 Aug 2026 Goal: Upload Jan file → Jan data is saved and shown. Upload Feb later → Feb becomes a NEW month dataset; all previous months stay saved and selectable (never merged). Hosted on AWS with backups, automatic exchange rates, and a 5-user login.

Bottom line:


AWS hosting cost	≈ $50–55 / month
Exchange-rate API	$0 / month (Frankfurter, free)
Build effort	~12–17 dev days, 5 phases, each shippable on its own

1. What is wrong today (why months "merge" or disappear)

The app already stores each month as its own closing session in the database — the foundation is right. Four defects break it:

Double-count bug (verified in code): re-uploading a file with the same name overwrites it on disk (ar-aging-app/backend/app/api/routes/files.py:38) but inserts a second database row (files.py:74). Processing then parses the same file twice and doubles every number.
Nothing prevents two sessions for the same month — creating "January" twice by accident produces two competing datasets, no warning.
New months start from zero — default opening_mode="zero" even though carry-forward logic already exists in routes/ar.py, so months lose continuity unless someone remembers to switch it.
Previous months silently vanish from Accounts Summary — re-processing clears journal approval (correct audit control), but the summary grid only shows approved months, so the old month disappears with no explanation.

2. Confirmed decisions

Exchange rates: free Frankfurter API — $0, no API key, central-bank (ECB) rates, historical month-end dates. The human confirmation gate (Control C5) stays.
Login: simple per-user login, 5 users. Approver/reviewer names come from the logged-in user.
File sizes: uploads are 300–500 MB each → server sized with 8 GB RAM for Excel parsing.

3. AWS architecture & monthly cost

Recommended: one production server + S3 backups (≈ $50–55/mo)

Lightsail 8 GB instance ($44/mo: 2 vCPU, 8 GB RAM, 160 GB SSD, static IP included) — or equivalent EC2 t4g.large (≈ $61/mo with EBS + IPv4). Runs everything via a new docker-compose.prod.yml:

Container	Role
nginx	Serves built React app, proxies `/api` to backend, HTTPS (Let's Encrypt), `client_max_body_size 2g` + long timeouts for big uploads
backend	FastAPI/uvicorn, single worker (background jobs are in-process — documented constraint, fine for 5 users)
MySQL 8	Data on instance disk. Code already supports MySQL (`AR_DB_BACKEND=mysql`) and ships `migrate_sqlite_to_mysql.py`
backup cron	Nightly `mysqldump` + `aws s3 sync` of uploads/exports → versioned S3 bucket (lifecycle → cold storage after 90 days), via IAM role (no keys on disk)

Cost breakdown

Item	Monthly cost
Lightsail 8 GB (EC2 t4g.large route ≈ $61)	$44
S3 backups (~60 GB year one, versioned)	$1.50–3
Weekly instance snapshots	$2–4
Route 53 hosted zone (optional domain)	$0.50
Frankfurter FX API	$0
Total	≈ $50–55/mo

Alternative (managed DB): same server + RDS MySQL db.t4g.small ≈ $85–100/mo — buys automated patching + point-in-time restore. Not needed at this scale; upgrading later is a one-line config change (MYSQL_HOST).

Storage decisions:

Uploads/exports stay on the instance disk — the parsing pipeline needs local file paths; round-tripping 500 MB files through S3 adds complexity for no benefit. S3 = durable backup, not primary storage.
Data growth ~1 GB/month (~3.4M transaction rows) — well within sizing.
Firewall inbound restricted to office IPs/VPN as defense-in-depth on top of login.

Exchange-rate API pricing (researched)

Provider	Free tier	Paid	Verdict
Frankfurter (chosen)	Unlimited, no API key, ECB rates, historical dates, self-hostable	—	✅ $0, ideal for month-end closes
ExchangeRate-API	1,500 req/mo, daily updates	Pro $10/mo (30k req, hourly)	fallback provider stub
Open Exchange Rates	1,000 req/mo, USD base only	$12/mo	not needed
Fixer	100 req/mo, no HTTPS on free	higher	ruled out

App usage: ~13 marketplaces × a few fetches/month → even free tiers would never be exceeded.

4. Implementation phases

Phase 1 — Month management & data separation (~3–4 days) ← HIGHEST PRIORITY

1.1 Fix double-count bug — backend/app/api/routes/files.py:

Stream uploads to a temp .part name; move into place after hashing (also stops a failed upload corrupting an existing good file).
Content dedup: SHA-256 match with an existing file in the session → skip, report "identical content already uploaded as X".
Same filename → UPDATE the existing row instead of inserting a second one; if month already processed, flag needs_reprocess (existing banner handles messaging).
Batch-friendly response {files, skipped} so one duplicate doesn't fail a 13-file upload; UI shows skipped/replaced notices.
One-time cli.py dedupe-files cleanup for existing bad rows, then a unique index on (session_id, filename).

1.2 One session per month (guide, don't hard-block) — routes/sessions.py: 409 on duplicate reporting_month unless allow_duplicate: true; NewClosing.tsx shows "A closing for 2026-02 already exists — Open it | Create anyway".

1.3 Default carry-forward — NewClosing.tsx: default openingMode="carry_forward" when prior processed months exist (backend logic already exists); default month = month after latest close.

1.4 Month selector UX ("show previous options"):

Sessions list ordered as a month timeline with a journal_approved flag per row.
New MonthSwitcher dropdown in the closing header (month · name · status) — jump between months from any screen.
Dashboard shows "published / processed-but-unpublished" and flags duplicate months.

1.5 Stop previous months vanishing from Accounts Summary:

accounts_summary.py also returns pending months (processed/blocked but unapproved, incl. "approval cleared by re-processing").
Grid renders them as greyed columns linking to the journal tab ("2026-01 processed but unpublished — re-approve the journal"). Audit control untouched.

1.6 Read-only lock after completion — new ensure_editable() guard on all mutating endpoints (409 on completed months); new POST /sessions/{id}/reopen; frontend disables edit controls on locked months.

Tests: filename replace, sha256 skip, duplicate-month 409, completed-session 409s.

New User table (username, display name, bcrypt password hash); users created via CLI (add-user / set-password) — no self-signup.
POST /api/auth/login → signed bearer token (12 h expiry, new AR_SECRET_KEY env); every route requires login except /api/health + login.
Real identity in sign-offs: journal review/approve, FX confirm, control verify, payout entry all record the logged-in user — free-text "your name" boxes removed.
Frontend: Login.tsx, token auto-attached, 401 → redirect to login, user + logout in sidebar.
New deps: passlib[bcrypt], itsdangerous.

Phase 3 — AWS deployment (~3–5 days incl. migration dry-run)

Production backend image (no --reload, --proxy-headers, single worker documented).
Production frontend image: multi-stage node:20 build → nginx serving static + API proxy.
New docker-compose.prod.yml + example.env.production (AR_DB_BACKEND=mysql, AR_CORS_ORIGINS=https://<domain>, AR_SECRET_KEY, FX vars); HTTPS via certbot; DNS → static IP.
Data migration: MySQL up → schema auto-created → run existing migrate_sqlite_to_mysql.py → verify per-table row counts + to-the-cent reconciliation spot check → cut over; archive SQLite file to S3. Timed dry run first (3.4M rows/month).
Backups: nightly dump + S3 sync, weekly snapshots, S3 versioning + lifecycle.
Deploy runbook: git pull && docker compose -f docker-compose.prod.yml up -d --build.

Phase 4 — Exchange-rate service (~2 days)

New backend/app/services/fx_service.py: provider abstraction — Frankfurter default (api.frankfurter.dev/v1/{date}?base=USD&symbols=CAD,GBP,AUD,EUR,PLN,SEK,TRY), paid-provider stub via FX_PROVIDER env.
Rates inverted to USD-per-local (the app's convention) — documented + locked with a test; one inversion mistake would mis-state every non-USD receivable.
Month-end fetch seeds fx_rates per session unconfirmed → Control C5 still blocks the close until a human confirms — workflow unchanged, just pre-filled. Daily fetch fills fx_rates_daily.
Cache table makes re-fetch idempotent + works offline after first fetch; provider failure → clear 502 "enter rates manually", never a silent default.
UI: "Fetch month-end rates" button in the C5 panel (Controls.tsx); "Fetch daily rates" in ArLedger.tsx. Update "no third-party egress" copy (only currency codes are sent, never financial data).
⚠️ Verify Frankfurter publishes TRY; if not, Turkey stays manual (C5 still enforces) or paid stub takes over.

Phase 5 — Hardening (~1–2 days)

Stale-job recovery: months stuck "processing" after a restart → marked error with a re-run message (otherwise a deploy mid-job blocks the month forever).
Retention: auto-purge old generated exports only (AR_RETENTION_DAYS, currently dead config) — uploaded source files never auto-deleted (audit source).
Health check: DB ping + disk-writable.
Logging: request timing + job start/finish/fail, shipped via Docker logs.

Deferred nice-to-haves: CI pipeline, Sentry, login rate-limiting, Alembic, S3-primary storage, audit-log table, MFA.

5. Verification (acceptance checks)

Upload same CSV twice → one file row, totals unchanged; renamed-identical file → skipped with message.
Close Jan → create Feb → warns nothing, carries forward Jan balances; Jan stays selectable, read-only, unchanged; both months on Accounts Summary.
Fetch month-end rates → unconfirmed → C5 fails → Confirm all → C5 passes; EUR rate checked against ECB published figure.
Unauthenticated API call → 401; all 5 users can log in; approvals record the real approver name.
Prod compose rehearsed locally with a 500 MB file (memory watched) before buying the instance; restart mid-processing recovers; nightly backup object appears in S3; restore drill succeeds.
All existing tests pass, incl. Jan-2026 reconciliation integration test to the cent after MySQL migration.

6. Key risks

Risk	Mitigation
SQLite→MySQL migration (~3.4M rows/month)	Timed dry run, per-table counts, cent-level reconciliation before cutover
FX rate orientation (USD-per-local vs local-per-USD)	Fixture test against a known EUR rate
Read-only lock missing an endpoint	Sweep all mutating routes during implementation
8 GB RAM sizing assumption	Validate with the real 283 MB test workbook in local rehearsal before instance purchase

Pricing sources: ExchangeRate-API · Open Exchange Rates · Frankfurter · AWS Lightsail/EC2/RDS public pricing, Aug 2026. Costs are USD estimates, on-demand.

11 KiB Raw Blame History Unescape Escape