Finance-Accounts/ar-aging-app
Talha Ahmed d823a45cb2 Production readiness: month separation, auth, FX service, AWS deployment
Month management & data separation:
- Fix double-count bug: re-uploading a filename updates the existing
  session_files row in place; identical content (sha256) is skipped —
  a closing can never parse the same file twice
- Guard against duplicate closings per reporting month (409 unless
  explicitly overridden); dashboard flags duplicates
- Default new closings to carry-forward openings; month switcher in the
  closing header; publish state visible everywhere; Accounts Summary
  lists unpublished months with the reason instead of dropping them
- Completed closings are locked read-only with an explicit reopen

Authentication (stdlib only, no new deps):
- Per-user login (scrypt + HMAC tokens), AR_AUTH=auto turns on with the
  first user; manage.py add-user/set-password/deactivate-user
- Verified identity feeds reviewed_by/approved_by/confirmed_by

Exchange rates:
- fx_service with provider abstraction: Frankfurter (free, keyless,
  ECB) default, exchangerate-api stub; month-end + daily fetch
  endpoints and UI buttons; rates arrive unconfirmed so Control C5
  still gates the close; cache table; certifi CA bundle

Deployment & hardening:
- Production Docker stack: caddy (auto-HTTPS) + nginx + single-worker
  backend + mysql:8.4; per-context .dockerignore (images carry no
  financial data); .env.example with local+production sections
- deploy/DEPLOY.md runbook + nightly S3 backup script
- Stale-job recovery on startup; export retention (AR_RETENTION_DAYS);
  deep /api/health; request/job logging; Gitea Actions CI
- Repo reorganized: launchers in scripts/, dated lowercase docs,
  root README, .gitattributes for deterministic line endings

Tests: 152 passed (25+ new: dedup, month locking, auth, FX orientation)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 18:46:47 +05:00
..
backend Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
deploy Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
docs Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
frontend Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
scripts Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
.dockerignore Implement MySQL support in AR aging app. Update README with configuration instructions, modify requirements for PyMySQL and dotenv, and refactor database setup to use MySQL. Adjust models and queries for compatibility with MySQL, including column size specifications. Enhance Vite config for API proxying. 2026-07-29 18:52:37 +05:00
.env.example Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
.gitignore Stop tracking SQLite WAL/SHM runtime files. 2026-07-29 18:07:50 +05:00
Makefile Initial commit: AR aging app with root gitignore. 2026-07-29 18:07:21 +05:00
README.md Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
docker-compose.prod.yml Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00
docker-compose.yml Production readiness: month separation, auth, FX service, AWS deployment 2026-08-19 18:46:47 +05:00

README.md

Amazon Accounts Receivable Aging Dashboard

Automates the Finance team's month-end Amazon Accounts Receivable Aging workbook: upload the month's Amazon Custom Unified Transaction files → get a structurally- and financially-identical Accounts Receivable Aging workbook, with validation, settlement reconciliation, exceptions, drill-down, and an audit trail.

Amazon only (no Walmart/TikTok/Temu/Shein/eBay/Shopify). Reverse-engineered from the Jan-2026 files; the USA receivable reproduces to the penny (11,110,433 = Detail!D11). See docs/accounts-receivable-logic.md.

Layout

backend/
  app/core/        # streaming parser + receivable engine (stdlib + openpyxl only)
  app/api/         # FastAPI app: routes, auth (login), deps
  app/services/    # jobs, persistence, FX-rate fetch, controls, retention
  app/db/          # SQLAlchemy models (20 tables) + engine (SQLite / MySQL)
  tests/           # 150+ tests: engine, API, auth, FX, dedup, Jan-2026 integration
  cli.py           # process files from the command line
  manage.py        # admin: add-user / set-password / dedupe-files / …
  migrate_sqlite_to_mysql.py   # one-time data migration for the AWS cutover
frontend/          # React + TS + Vite dashboard (nginx-served in production)
scripts/           # launchers: start.ps1 / start.bat (Windows) · start.command (macOS)
deploy/            # DEPLOY.md runbook + backup.sh (nightly mysqldump + S3 sync)
docs/              # system guide · AR logic · audit reports
docker-compose.yml            # dev stack        docker-compose.prod.yml  # production
.env.example                  # every setting, LOCAL + PRODUCTION sections

Production deployment (AWS)

One 8 GB server runs the whole stack with automatic HTTPS, MySQL, per-user login, and nightly S3 backups — see deploy/DEPLOY.md for the full runbook (provisioning, user creation, SQLite→MySQL migration, backups, updates). ≈ $5055/month.

cp .env.example .env.production    # fill the PRODUCTION section (domain, passwords, AR_SECRET_KEY)
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
  python manage.py add-user <user> --name "Full Name"

Key production behaviors:

  • One closing per month — every month stays saved and selectable (month switcher in the closing header); creating a second closing for an existing month requires an explicit override.
  • Duplicate-proof uploads — re-uploading a filename replaces it; identical content under another name is skipped. A month can never count a file twice.
  • Login (AR_AUTH) — per-user accounts via manage.py add-user; journal review/approval and FX confirmations record the signed-in user's verified name.
  • Exchange rates — "Fetch month-end rates" pulls central-bank rates (Frankfurter, free, keyless; AR_FX_PROVIDER); fetched rates still require human confirmation (Control C5).
  • Completed closings are locked read-only; corrections need an explicit Reopen.
  • Crash-safe jobs — a restart mid-processing marks the closing as interrupted instead of leaving it stuck; generated exports are purged after AR_RETENTION_DAYS (uploads never are).

Run the app (development)

# 1. Configure the environment (SQLite by default — no database setup needed)
cp .env.example .env   # then fill the LOCAL section (AR_SECRET_KEY at minimum)

# Option A — Docker (backend + Vite hot reload)
docker compose up --build
# → http://localhost:5173  (API on :8000; uploads/exports on the ar_data volume)

# Option B — local processes
make install          # backend deps + npm install
make backend          # terminal 1 → FastAPI on :8000
make frontend         # terminal 2 → dashboard on http://localhost:5173

# Option C — Windows one-click (ports 8010/5174)
scripts\start.bat     # or: powershell -ExecutionPolicy Bypass -File scripts\start.ps1
# macOS one-click:
scripts/start.command

Then open http://localhost:5173New Closing → pick the month → drag in the three Amazon files → Run processing → review → Download Full A/R Aging Excel.

Uploads and exports are stored under AR_DATA_DIR (default backend/data locally, /data in Docker). The app connects to MySQL using MYSQL_* variables from .env.

Run the engine headless (CLI)

cd backend && pip install -r requirements.txt
python cli.py "/path/USA…01 to 10 January,2026.xlsx" \
              "/path/USA…11 to 20 January,2026.xlsx" \
              "/path/USA…21 to 31 January,2026.xlsx" \
              --month-end 2026-01-31 --lag 2 --reserve-standard 125.44 --out AR_Aging_Jan26.xlsx

Tests

make test        # 16 fast tests (engine, export, API, robustness)
make test-all    # + Jan-2026 reconciliation & sample-comparison (integration, ~4 min)
# point tests at the sample files if not in the repo root:
AR_SAMPLE_DIR="/path/to/samples" make test-all

Status — all phases complete (incl. v2 multi-market)

  • Engine — settlement classification + receivable; USA = 11,110,433 verified to the penny
  • Multi-marketplace — all 13 markets (CA/UK/AU/IE + localized FR/DE/IT/ES/NL/PL/SV/TR) reconcile to the penny vs the Jan-26 workbook; per-market currency & FX; settlement-owner logic for cross-market EU chains; helper-row & pivot-sheet detection
  • AR roll-forward — opening balance (auto carry-forward) + net revenue payouts = closing; AR Ledger · Finance Summary · Journal Entry (per-marketplace) · Reconciliation Control w/ sign-off
  • Header mapping — localized alias tables + admin rules UI (/api/mapping-rules); unmapped amounts are never silently excluded
  • Storage-fee detection — canonical + description-based (potential/missing storage exceptions)
  • Excel — Full audit workbook + Summary Finance pack
  • Tests — ~60 fast + integration (USA reconciliation, sample comparison, 13-market benchmark)