utopia-ai
/
Finance-Accounts
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: utopia-ai:main
Branches Tags
utopia-ai:main
utopia-ai:new-changes
..
compare: utopia-ai:new-changes
Branches Tags
utopia-ai:main
utopia-ai:new-changes

No commits in common. "main" and "new-changes" have entirely different histories.
main ... new-change

94 changed files with 422 additions and 7509 deletions
Show Stats Expand all files Collapse all files

28
.gitattributes vendored
View File

@ -1,28 +0,0 @@
# Deterministic line endings regardless of each developer's core.autocrlf.
# Shell scripts and Docker/compose files MUST be LF — CRLF breaks bash and can break
# Docker builds when the repo is cloned on the Linux server.
* text=auto
*.sh text eol=lf
*.command text eol=lf
Makefile text eol=lf
Dockerfile text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.py text eol=lf
*.ts text eol=lf
*.tsx text eol=lf
*.md text eol=lf
*.json text eol=lf
*.html text eol=lf
*.css text eol=lf
*.conf text eol=lf
# Windows launchers keep CRLF
*.bat text eol=crlf
*.ps1 text eol=crlf
# Never mangle binaries
*.png binary
*.ico binary
*.woff2 binary

49
.gitea/workflows/deploy-to-s3.yml
View File

@ -1,49 +0,0 @@
name: Deploy to S3
on:
push:
branches:
- main
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Configure AWS credentials
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DEVOPS_USER_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DEVOPS_USER_AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
run: |
echo "AWS credentials configured"
- name: Archive project
run: |
apt-get update -y
apt-get install -y zip
zip -r utopia-ai-finance-accounts.zip . \
-x ".git/*" \
-x ".gitea/*" \
-x ".gitignore/*" \
-x "*.DS_Store"
- name: Install AWS CLI
run: |
apt-get update -y
apt-get install -y curl unzip
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip -q awscliv2.zip
./aws/install
aws --version
- name: Upload files to S3
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DEVOPS_USER_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DEVOPS_USER_AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
run: |
echo "Uploading repo contents to S3..."
aws s3 cp utopia-ai-finance-accounts.zip s3://utopia-ai-s3-repo-bucket/utopia-ai-finance-accounts.zip

1
.gitignore vendored
View File

@ -20,7 +20,6 @@
*.xls
*.csv
*.tsv
Test Files/
Amazon Transactions reports*/
Accounts Receivable*/
!ar-aging-app/backend/tests/fixtures/*.xlsx

40
README.md
View File

@ -1,40 +0,0 @@
# Finance-Accounts
Finance team tooling for **Utopia Brands**. The main (currently only) application is the
**Amazon Accounts Receivable Aging Dashboard** — it turns the month's Amazon *Custom
Unified Transaction* exports into the month-end AR workbook, with per-user login
(self-service password reset by emailed code), month-end controls, central-bank
exchange-rate fetching, and a full audit trail.
## Repository layout
```
ar-aging-app/ The application (FastAPI backend · React frontend)
│
├── backend/ Python API + calculation engine + tests
├── frontend/ React + TypeScript dashboard
├── scripts/ Launchers: start.ps1 / start.bat (Windows) · start.command (macOS)
├── deploy/ Production runbook (DEPLOY.md) + backup script
├── docs/ System guide, AR logic, audit reports
├── docker-compose.yml local Docker stack (dev)
├── docker-compose.prod.yml production stack (AWS: HTTPS + MySQL + backups)
└── .env.example every setting, local + production sections
plan.md Production-readiness plan (architecture, AWS costs, phases)
sample data (local) "Test Files/" — real Amazon exports; gitignored, never committed
```
## Quick start
| I want to… | Do this |
|---|---|
| Run the app on this PC | double-click `ar-aging-app/scripts/start.bat` → http://localhost:5174 |
| Understand the app | [ar-aging-app/README.md](ar-aging-app/README.md) |
| Deploy to AWS | [ar-aging-app/deploy/DEPLOY.md](ar-aging-app/deploy/DEPLOY.md) (~$50–55/month) |
| See how figures are calculated | [ar-aging-app/docs/system-guide.md](ar-aging-app/docs/system-guide.md) |
| Read the production plan | [plan.md](plan.md) |
| Reset a forgotten password | login screen → "Forgot password?" (emailed code) — or admin: `manage.py set-password` |
| Manage users / fix data | `python ar-aging-app/backend/manage.py --help` |
Financial data never enters git: spreadsheets, databases, uploads, and `.env*` secrets are
all ignored (see `.gitignore`). The only template committed is `ar-aging-app/.env.example`.

93
ar-aging-app/.env.example
View File

@ -1,93 +0,0 @@
# ==============================================================================
# AR Aging — environment template (this file IS committed; real copies are NOT)
#
# Local development: cp .env.example .env -> fill the LOCAL section
# Production (AWS): cp .env.example .env.production -> fill the PRODUCTION section
#
# .env and .env.production are gitignored — secrets never enter git.
# ==============================================================================
# ------------------------------------------------------------------ LOCAL (dev)
# Database: leave MYSQL_* unset and the app uses a local SQLite file —
# backend/data/ar_aging.db (zero setup; this is the local database "name")
# Force it explicitly if you like:
AR_DB_BACKEND=sqlite
# Signs login tokens (sessions survive backend restarts). Generate:
# python -c "import secrets; print(secrets.token_hex(32))"
AR_SECRET_KEY=
# Login: auto = required as soon as users exist (create with: python manage.py add-user)
AR_AUTH=auto
# Vite dev server origins
AR_CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173,http://localhost:5174,http://127.0.0.1:5174
# Exchange rates: frankfurter = free, keyless, central-bank rates
AR_FX_PROVIDER=frankfurter
# Processing auto-fetches the provider's DAILY rates over each closing's transaction
# span, so dated movements convert at their own transaction date's rate. Set 0 to
# disable (the AR Ledger's "Fetch daily rates" button still works).
#AR_FX_AUTO_DAILY=1
# Email (optional) — enables "email me a code" for password resets.
# Preferred: the company's internal Mail API (bearer token; ask Talha/IT for the values).
#AR_MAIL_API_URL=
#AR_MAIL_API_TOKEN=
# Fallback: any SMTP account (used only if AR_MAIL_API_URL is unset):
# Office365: smtp.office365.com : 587 Gmail: smtp.gmail.com : 587 (app password)
# Both unset -> passwords change via current password / admin reset instead.
#AR_SMTP_HOST=
#AR_SMTP_PORT=587
#AR_SMTP_USER=
#AR_SMTP_PASSWORD=
#AR_SMTP_FROM=
# Generated exports older than this are purged (uploads are NEVER auto-deleted). 0 = keep.
AR_RETENTION_DAYS=90
# ------------------------------------------------------------- PRODUCTION (AWS)
# Used by docker-compose.prod.yml. Fill these in .env.production on the server.
# Domain — DNS A record must point at the server; HTTPS certificate is automatic.
#AR_DOMAIN=ar.utopiabrands.com
# MySQL (the database is created automatically on first start).
# MYSQL_HOST is set to the compose service name by docker-compose.prod.yml.
#MYSQL_PORT=3306
#MYSQL_DATABASE=account_finance
#MYSQL_USER=ar_app
#MYSQL_PASSWORD= <- strong generated password
#MYSQL_ROOT_PASSWORD= <- different strong generated password
#MYSQL_SLOW_QUERY_MS=500
#MYSQL_POOL_SIZE=10
#MYSQL_POOL_RECYCLE=3600
# Auth — REQUIRED in production. Different key than local!
#AR_SECRET_KEY= <- openssl rand -hex 32
#AR_AUTH=on
#AR_AUTH_TOKEN_HOURS=12
# Same-origin behind nginx/caddy; still set exactly.
#AR_CORS_ORIGINS=https://ar.utopiabrands.com
#AR_FX_PROVIDER=frankfurter
# AR_FX_PROVIDER=exchangerate-api # paid fallback ($10/mo) — then set:
# AR_FX_API_KEY=
# Email for password codes (see the LOCAL section for the transports)
#AR_MAIL_API_URL=
#AR_MAIL_API_TOKEN=
#AR_SMTP_HOST=
#AR_SMTP_PORT=587
#AR_SMTP_USER=
#AR_SMTP_PASSWORD=
#AR_SMTP_FROM=
#AR_RETENTION_DAYS=90
# AR_MAX_UPLOAD_BYTES=2147483648 # 2 GB default
# Nightly backups (deploy/backup.sh) — S3 bucket; instance IAM role grants access.
#AR_BACKUP_S3_BUCKET=s3://utopia-ar-backups

98
ar-aging-app/README.md
View File

@ -13,55 +13,17 @@ See [docs/accounts-receivable-logic.md](docs/accounts-receivable-logic.md).
```
backend/
app/core/ # streaming parser + receivable engine (stdlib + openpyxl only)
app/api/ # FastAPI app: routes, auth (login), deps
app/services/ # jobs, persistence, FX-rate fetch, controls, retention
app/db/ # SQLAlchemy models (20 tables) + engine (SQLite / MySQL)
tests/ # 150+ tests: engine, API, auth, FX, dedup, Jan-2026 integration
app/api/ # FastAPI app (uploads, jobs, endpoints) — Phase 3
tests/ # unit + Jan-2026 reconciliation integration test
cli.py # process files from the command line
manage.py # admin: add-user / set-password / dedupe-files / …
migrate_sqlite_to_mysql.py # one-time data migration for the AWS cutover
frontend/ # React + TS + Vite dashboard (nginx-served in production)
scripts/ # launchers: start.ps1 / start.bat (Windows) · start.command (macOS)
deploy/ # DEPLOY.md runbook + backup.sh (nightly mysqldump + S3 sync)
docs/ # system guide · AR logic · audit reports
docker-compose.yml # dev stack docker-compose.prod.yml # production
.env.example # every setting, LOCAL + PRODUCTION sections
frontend/ # React + TS + Vite dashboard — Phase 4
docs/
```
## Production deployment (AWS)
One 8 GB server runs the whole stack with automatic HTTPS, MySQL, per-user login, and
nightly S3 backups — see **[deploy/DEPLOY.md](deploy/DEPLOY.md)** for the full runbook
(provisioning, user creation, SQLite→MySQL migration, backups, updates). ≈ $50–55/month.
## Run the app
```bash
cp .env.example .env.production # fill the PRODUCTION section (domain, passwords, AR_SECRET_KEY)
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
python manage.py add-user <user> --name "Full Name"
```
Key production behaviors:
- **One closing per month** — every month stays saved and selectable (month switcher in the
closing header); creating a second closing for an existing month requires an explicit override.
- **Duplicate-proof uploads** — re-uploading a filename *replaces* it; identical content
under another name is skipped. A month can never count a file twice.
- **Login** (`AR_AUTH`) — per-user accounts via `manage.py add-user`; journal review/approval
and FX confirmations record the signed-in user's verified name.
- **Password self-service** — Settings changes the password with the current one; a
forgotten password is recovered from the login screen ("Forgot password?") via a 6-digit
code emailed to the account address (company Mail API `AR_MAIL_API_*`, SMTP fallback);
`manage.py set-password` remains the admin override.
- **Exchange rates** — "Fetch month-end rates" pulls central-bank rates (Frankfurter, free,
keyless; `AR_FX_PROVIDER`); fetched rates still require human confirmation (Control C5).
- **Completed closings are locked** read-only; corrections need an explicit Reopen.
- **Crash-safe jobs** — a restart mid-processing marks the closing as interrupted instead of
leaving it stuck; generated exports are purged after `AR_RETENTION_DAYS` (uploads never are).
## Run the app (development)
```bash
# 1. Configure the environment (SQLite by default — no database setup needed)
cp .env.example .env # then fill the LOCAL section (AR_SECRET_KEY at minimum)
# 1. Configure MySQL (hosted RDS) + data paths
cp example.env .env # then fill in MYSQL_* credentials
# Option A — Docker (backend + Vite hot reload)
docker compose up --build
@ -71,17 +33,12 @@ docker compose up --build
make install # backend deps + npm install
make backend # terminal 1 → FastAPI on :8000
make frontend # terminal 2 → dashboard on http://localhost:5173
# Option C — Windows one-click (ports 8010/5174)
scripts\start.bat # or: powershell -ExecutionPolicy Bypass -File scripts\start.ps1
# macOS one-click:
scripts/start.command
```
Then open the dashboard → **New Closing** → pick the month → drag in the month's Amazon
Then open http://localhost:5173 → **New Closing** → pick the month → drag in the three Amazon
files → **Run processing** → review → **Download Full A/R Aging Excel**.
Uploads and exports are stored under `AR_DATA_DIR` (default `backend/data` locally, `/data` in
Docker). The database is a local SQLite file by default; set `MYSQL_*` in `.env` to use MySQL.
Uploads and exports are stored under `AR_DATA_DIR` (default `backend/data` locally, `/data` in Docker).
The app connects to MySQL using `MYSQL_*` variables from `.env`.
## Run the engine headless (CLI)
```bash
@ -94,28 +51,21 @@ python cli.py "/path/USA…01 to 10 January,2026.xlsx" \
## Tests
```bash
make test # 155 fast tests: engine, API, auth, FX, upload dedup, month locking
make test # 16 fast tests (engine, export, API, robustness)
make test-all # + Jan-2026 reconciliation & sample-comparison (integration, ~4 min)
# point the integration tests at the sample files if not in the repo root:
# point tests at the sample files if not in the repo root:
AR_SAMPLE_DIR="/path/to/samples" make test-all
```
The suite runs against an isolated temporary database — it can never touch real data
(a session-scoped guard asserts the isolation before anything runs).
## Feature summary
**Calculation engine**
- Settlement classification + receivable; USA Jan-26 = **11,110,433** verified to the penny
- All 13 marketplaces reconcile to the penny vs the Finance workbook — per-market currency
& FX, settlement-owner logic for cross-market EU chains, helper-row/pivot-sheet detection
- AR roll-forward (opening + net revenue − payouts = closing) with auto carry-forward,
AR Ledger, Finance Summary, per-marketplace Journal Entry, Reconciliation Control
- Header mapping with localized alias tables + admin rules UI; unmapped amounts are never
silently excluded; storage-fee detection; full Excel audit workbook + Finance pack
**Operations & security**
- Per-user login with emailed password codes; verified names on every sign-off
- Six month-end controls (C1–C6) block publication of any untrusted figure
- Month-end FX fetched from central-bank data, gated by human confirmation
- Duplicate-proof uploads, one-closing-per-month guard, completed-month locking
- Production Docker stack (auto-HTTPS · MySQL · nightly S3 backups); CI/CD owned by DevOps
## Status — all phases complete (incl. v2 multi-market)
- ✅ **Engine** — settlement classification + receivable; USA = **11,110,433** verified to the penny
- ✅ **Multi-marketplace** — all 13 markets (CA/UK/AU/IE + localized FR/DE/IT/ES/NL/PL/SV/TR)
reconcile **to the penny** vs the Jan-26 workbook; per-market currency & FX; settlement-owner
logic for cross-market EU chains; helper-row & pivot-sheet detection
- ✅ **AR roll-forward** — opening balance (auto carry-forward) + net revenue − payouts = closing;
AR Ledger · Finance Summary · Journal Entry (per-marketplace) · Reconciliation Control w/ sign-off
- ✅ **Header mapping** — localized alias tables + admin rules UI (`/api/mapping-rules`); unmapped
amounts are never silently excluded
- ✅ **Storage-fee detection** — canonical + description-based (potential/missing storage exceptions)
- ✅ **Excel** — Full audit workbook + Summary Finance pack
- ✅ **Tests** — ~60 fast + integration (USA reconciliation, sample comparison, 13-market benchmark)

18
ar-aging-app/backend/.dockerignore
View File

@ -1,18 +0,0 @@
# Docker only reads the .dockerignore INSIDE the build context (this folder) — the one at
# the app root does not apply to `build: ./backend`. Without this file the image would
# bake in backend/data: the live SQLite database and uploaded financial files.
data
.pytest_cache
__pycache__
**/__pycache__
*.pyc
.venv
venv
*.db
*.db-shm
*.db-wal
*.xlsx
*.xls
*.csv
.env
.env.*

9
ar-aging-app/backend/Dockerfile
View File

@ -4,7 +4,6 @@ WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
default-mysql-client \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
@ -14,10 +13,4 @@ COPY . .
EXPOSE 8000
# Production default: single worker, proxy-aware, no reload.
# SINGLE WORKER IS A CONSTRAINT, NOT A TUNABLE: processing/export jobs run in-process
# (FastAPI BackgroundTasks) and progress state lives in that process. More workers or
# replicas would split jobs from their status polling. Fine for a ~5-user finance team.
# The dev compose file overrides this command with --reload.
CMD ["uvicorn", "app.api.main:app", "--host", "0.0.0.0", "--port", "8000", \
"--workers", "1", "--proxy-headers", "--forwarded-allow-ips", "*"]
CMD ["uvicorn", "app.api.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

418
ar-aging-app/backend/app/api/auth.py
View File

@ -1,418 +0,0 @@
"""
Authentication: per-user login with signed bearer tokens. Stdlib only — no new deps.
Design (deliberately minimal for a ~5-user internal finance tool):
* Passwords: hashlib.scrypt (OpenSSL), per-user random salt, constant-time compare.
* Tokens: HMAC-SHA256-signed JSON (user id, username, display name, expiry) — the
same shape as a JWT but without the dependency. Signed with AR_SECRET_KEY;
when unset, an ephemeral key is generated and a warning logged (every
restart then logs everyone out — fine on a laptop, wrong on a server).
* Enforcement: an HTTP middleware guards every /api/* route except the open set below.
AR_AUTH=auto (default) requires login as soon as at least one user exists,
so a fresh dev checkout and the test suite run without ceremony while
creating the first real user turns authentication on by itself.
* Identity: the verified display name feeds reviewed_by / approved_by / confirmed_by
via actor_name(), replacing free-text name fields.
Users are created with `python manage.py add-user` — there is no self-signup endpoint.
"""
from __future__ import annotations
import base64
import datetime as dt
import hashlib
import hmac
import json
import logging
import secrets
import time
from dataclasses import dataclass
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from sqlalchemy.orm import Session as OrmSession
from ..config import AUTH_MODE, AUTH_TOKEN_HOURS, SECRET_KEY
from ..db import models
from ..db.database import SessionLocal
from .deps import db_dep
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/auth", tags=["auth"])
# Paths reachable without a token: health probes, login itself, the "is auth on?" check,
# and the forgot-password code flow (which by definition happens while locked out).
OPEN_PATHS = {"/api/health", "/api/auth/login", "/api/auth/status",
"/api/auth/request-code", "/api/auth/verify-code",
"/api/auth/reset-password"}
if SECRET_KEY:
_SECRET = SECRET_KEY.encode()
else:
_SECRET = secrets.token_bytes(32)
logger.warning(
"AR_SECRET_KEY is not set — using an ephemeral signing key. Login sessions will "
"not survive a restart. Set AR_SECRET_KEY in production."
)
# --------------------------------------------------------------------- password hashing
_SCRYPT_N, _SCRYPT_R, _SCRYPT_P = 16384, 8, 1
def hash_password(password: str) -> str:
salt = secrets.token_bytes(16)
digest = hashlib.scrypt(password.encode(), salt=salt,
n=_SCRYPT_N, r=_SCRYPT_R, p=_SCRYPT_P, dklen=32)
return (f"scrypt${_SCRYPT_N}${_SCRYPT_R}${_SCRYPT_P}"
f"${salt.hex()}${digest.hex()}")
def verify_password(password: str, stored: str) -> bool:
try:
algo, n, r, p, salt_hex, hash_hex = stored.split("$")
if algo != "scrypt":
return False
digest = hashlib.scrypt(password.encode(), salt=bytes.fromhex(salt_hex),
n=int(n), r=int(r), p=int(p),
dklen=len(bytes.fromhex(hash_hex)))
return hmac.compare_digest(digest, bytes.fromhex(hash_hex))
except (ValueError, TypeError):
return False
# ----------------------------------------------------------------------------- tokens
def _b64(data: bytes) -> str:
return base64.urlsafe_b64encode(data).rstrip(b"=").decode()
def _unb64(data: str) -> bytes:
return base64.urlsafe_b64decode(data + "=" * (-len(data) % 4))
def create_token(user: models.User) -> str:
payload = json.dumps({
"uid": user.id, "u": user.username, "dn": user.display_name,
"exp": int(time.time()) + AUTH_TOKEN_HOURS * 3600,
}, separators=(",", ":")).encode()
sig = hmac.new(_SECRET, payload, hashlib.sha256).digest()
return f"{_b64(payload)}.{_b64(sig)}"
def parse_token(token: str) -> dict | None:
"""The signed payload, or None if the token is malformed, forged, or expired."""
try:
payload_b64, sig_b64 = token.split(".")
payload = _unb64(payload_b64)
expected = hmac.new(_SECRET, payload, hashlib.sha256).digest()
if not hmac.compare_digest(expected, _unb64(sig_b64)):
return None
data = json.loads(payload)
if data.get("exp", 0) < time.time():
return None
return data
except (ValueError, TypeError, json.JSONDecodeError):
return None
# ------------------------------------------------------------------------- enforcement
@dataclass
class AuthUser:
id: int
username: str
display_name: str
# auto mode asks "do any users exist?" — cached briefly so it isn't a query per request.
_users_exist_cache: tuple[float, bool] = (0.0, False)
_USERS_CACHE_TTL_S = 10.0
def _users_exist() -> bool:
global _users_exist_cache
ts, val = _users_exist_cache
now = time.time()
if now - ts < _USERS_CACHE_TTL_S:
return val
db = SessionLocal()
try:
val = db.query(models.User.id).filter(
models.User.is_active == True).first() is not None # noqa: E712
except Exception: # noqa: BLE001 — table may not exist mid-migration; fail open once
val = False
finally:
db.close()
_users_exist_cache = (now, val)
return val
def invalidate_users_cache() -> None:
global _users_exist_cache
_users_exist_cache = (0.0, False)
def auth_required() -> bool:
if AUTH_MODE == "off":
return False
if AUTH_MODE == "on":
return True
return _users_exist() # auto
def _user_from_request(request: Request) -> AuthUser | None:
header = request.headers.get("Authorization", "")
if not header.startswith("Bearer "):
return None
data = parse_token(header[7:].strip())
if data is None:
return None
return AuthUser(id=data["uid"], username=data["u"], display_name=data["dn"])
async def auth_middleware(request: Request, call_next):
"""Guards every /api/* route except OPEN_PATHS. Registered in api/main.py."""
path = request.url.path.rstrip("/") or "/"
if path.startswith("/api"):
# Identity is attached whenever a valid token is present — including on open
# paths, so e.g. a signed-in password-code request knows who is asking.
user = _user_from_request(request)
request.state.user = user
if user is None and path not in OPEN_PATHS and auth_required():
return JSONResponse({"detail": "Not signed in (or the session expired). "
"Sign in to continue."}, status_code=401)
return await call_next(request)
def current_user(request: Request) -> AuthUser | None:
"""The signed-in user, or None when auth is off/auto-without-users (dev, tests)."""
return getattr(request.state, "user", None)
def actor_name(request: Request, provided: str = "") -> str:
"""The name that lands in accountability fields (reviewed_by / approved_by / …).
The verified identity always wins; the body-provided name is only honoured when no
one is signed in (auth off / auto without users), which keeps dev and tests working."""
user = current_user(request)
if user is not None and user.display_name:
return user.display_name
return (provided or "").strip()
# ----------------------------------------------------------------------------- routes
class LoginIn(BaseModel):
username: str
password: str
@router.get("/status")
def auth_status() -> dict:
"""Whether the frontend must show a login screen, and whether email codes work."""
from ..config import email_enabled
return {"auth_required": auth_required(), "email_enabled": email_enabled()}
@router.post("/login")
def login(body: LoginIn, db: OrmSession = Depends(db_dep)) -> dict:
user = db.query(models.User).filter(
models.User.username == body.username.strip().lower()).first()
if (user is None or not user.is_active
or not verify_password(body.password, user.password_hash)):
# One message for both wrong-user and wrong-password: don't confirm usernames.
raise HTTPException(401, "Wrong username or password.")
logger.info("login: %s", user.username)
db.add(models.AuditLog(username=user.username, display_name=user.display_name,
action="login"))
db.commit()
return {
"token": create_token(user),
"user": {"username": user.username, "display_name": user.display_name,
"is_admin": bool(user.is_admin)},
"expires_in_hours": AUTH_TOKEN_HOURS,
}
def is_admin(request: Request, db: OrmSession) -> bool:
"""Whether the signed-in user holds the admin flag — read from the DB every time, so a
revoke takes effect immediately rather than at token expiry. With auth off (dev/tests
before the first user) everyone counts as admin, matching AR_AUTH=auto's philosophy."""
user = current_user(request)
if user is None:
return not auth_required()
row = db.get(models.User, user.id)
return bool(row is not None and row.is_active and row.is_admin)
def require_admin(request: Request, db: OrmSession) -> None:
if not is_admin(request, db):
raise HTTPException(403, "Admin access required.")
@router.get("/me")
def me(request: Request, db: OrmSession = Depends(db_dep)) -> dict:
user = current_user(request)
if user is None:
if auth_required():
raise HTTPException(401, "Not signed in.")
return {"authenticated": False, "auth_required": False}
return {"authenticated": True, "auth_required": True,
"username": user.username, "display_name": user.display_name,
"is_admin": is_admin(request, db)}
# ------------------------------------------------------------- emailed password codes
# Usernames ARE email addresses, so the code goes to the account's own address. The code
# is stored as an HMAC (never plaintext), lives 10 minutes, works once, and the account
# locks the flow after 5 wrong attempts (request a fresh code to retry).
CODE_TTL_MINUTES = 10
CODE_MAX_ATTEMPTS = 5
def _hash_code(code: str) -> str:
return hmac.new(_SECRET, f"pwcode:{code}".encode(), hashlib.sha256).hexdigest()
class RequestCodeIn(BaseModel):
username: str = "" # optional when signed in (defaults to the session's account)
@router.post("/request-code")
def request_password_code(body: RequestCodeIn, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
"""Email a 6-digit password code to the account's address.
Deliberately explicit for this small internal team: an unregistered address gets a
clear 404 instead of an anti-enumeration non-answer."""
from ..config import email_enabled
from ..services.mailer import MailerError, send_password_code
if not email_enabled():
raise HTTPException(503, "Email is not set up on this server — ask the "
"administrator to reset your password instead.")
me_user = current_user(request)
username = (me_user.username if me_user else body.username).strip().lower()
if not username:
raise HTTPException(400, "Enter your username (email address).")
user = db.query(models.User).filter(models.User.username == username).first()
if user is None or not user.is_active:
logger.info("password code requested for unknown/inactive account: %s", username)
raise HTTPException(404, f"{username} isn't a registered account — check the "
f"address, or ask the administrator to create it.")
# Light resend throttle: one code per minute (a resend invalidates the previous code).
now = dt.datetime.utcnow()
if user.reset_code_expires:
issued_at = user.reset_code_expires - dt.timedelta(minutes=CODE_TTL_MINUTES)
if now - issued_at < dt.timedelta(seconds=60):
raise HTTPException(429, "A code was just sent — check your inbox, or try "
"again in a minute.")
code = f"{secrets.randbelow(1_000_000):06d}"
user.reset_code_hash = _hash_code(code)
user.reset_code_expires = now + dt.timedelta(minutes=CODE_TTL_MINUTES)
user.reset_code_attempts = 0
db.commit()
try:
send_password_code(user.username, code, CODE_TTL_MINUTES)
except MailerError as e:
# Roll the code back — a code nobody received must not stay live.
user.reset_code_hash = ""
user.reset_code_expires = None
db.commit()
raise HTTPException(502, f"{e} Ask the administrator to reset your password.")
return {"sent": True,
"detail": f"Code sent to {username} — it expires in {CODE_TTL_MINUTES} minutes."}
def _user_with_valid_code(db: OrmSession, username: str, code: str) -> models.User:
"""The account IF the code is currently valid — one generic error otherwise (never
confirms which part was wrong). A wrong code counts toward the attempt lockout."""
generic = HTTPException(400, "That code is wrong, expired, or already used — "
"request a fresh one.")
if not username or not code.strip():
raise generic
user = db.query(models.User).filter(models.User.username == username).first()
now = dt.datetime.utcnow()
if (user is None or not user.is_active or not user.reset_code_hash
or not user.reset_code_expires or user.reset_code_expires < now
or user.reset_code_attempts >= CODE_MAX_ATTEMPTS):
raise generic
if not hmac.compare_digest(_hash_code(code.strip()), user.reset_code_hash):
user.reset_code_attempts += 1
db.commit()
raise generic
return user
class VerifyCodeIn(BaseModel):
username: str = "" # optional when signed in
code: str
@router.post("/verify-code")
def verify_password_code(body: VerifyCodeIn, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
"""Step check for the reset UI: is this code valid? Does NOT consume the code — the
reset itself re-validates and burns it. Wrong guesses still count toward lockout."""
me_user = current_user(request)
username = (me_user.username if me_user else body.username).strip().lower()
_user_with_valid_code(db, username, body.code)
return {"valid": True}
class ResetPasswordIn(BaseModel):
username: str = "" # optional when signed in
code: str
new_password: str
@router.post("/reset-password")
def reset_password_with_code(body: ResetPasswordIn, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
"""Set a new password using the emailed code (works signed-in and from the login
screen). One generic failure message — never confirms which part was wrong."""
me_user = current_user(request)
username = (me_user.username if me_user else body.username).strip().lower()
if len(body.new_password) < 8:
raise HTTPException(400, "The new password must be at least 8 characters.")
user = _user_with_valid_code(db, username, body.code)
user.password_hash = hash_password(body.new_password)
user.reset_code_hash = "" # single use
user.reset_code_expires = None
user.reset_code_attempts = 0
db.commit()
logger.info("password reset via email code: %s", user.username)
return {"changed": True}
class ChangePasswordIn(BaseModel):
current_password: str
new_password: str
@router.post("/change-password")
def change_password(body: ChangePasswordIn, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
"""Signed-in users change their own password (admins reset others via manage.py).
Requires the current password so a walked-away-from session can't be hijacked into a
permanent account takeover. Existing tokens stay valid until their normal expiry."""
user = current_user(request)
if user is None:
raise HTTPException(401, "Sign in to change your password.")
row = db.get(models.User, user.id)
if row is None or not row.is_active:
raise HTTPException(401, "Account not found or deactivated.")
if not verify_password(body.current_password, row.password_hash):
raise HTTPException(400, "The current password is wrong.")
if len(body.new_password) < 8:
raise HTTPException(400, "The new password must be at least 8 characters.")
if body.new_password == body.current_password:
raise HTTPException(400, "The new password must be different from the current one.")
row.password_hash = hash_password(body.new_password)
db.commit()
logger.info("password changed: %s", row.username)
return {"changed": True}

14
ar-aging-app/backend/app/api/deps.py
View File

@ -49,20 +49,6 @@ def ensure_not_blocked(s: models.Session) -> None:
)
def ensure_editable(s: models.Session) -> None:
"""Guard every mutating endpoint: a completed closing is locked history.
Its figures were signed off and possibly booked — silently editing them would make the
record disagree with what was published. Corrections go through an explicit reopen
(POST /sessions/{id}/reopen), which is visible and deliberate."""
if s.status == "completed":
raise HTTPException(
status_code=409,
detail="This closing is completed and locked (read-only). "
"Reopen it first if a correction is genuinely needed.",
)
_SAFE = re.compile(r"[^A-Za-z0-9 ._,()\-]+")

68
ar-aging-app/backend/app/api/main.py
View File

@ -1,66 +1,28 @@
"""FastAPI application entrypoint."""
from __future__ import annotations
import asyncio
import logging
import time
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from sqlalchemy import text
from .. import APP_NAME, APP_VERSION
from ..config import CORS_ORIGINS, DATA_DIR
from ..db.database import ENGINE, init_db
from . import auth
from ..config import CORS_ORIGINS
from ..db.database import init_db
from .routes import (
sessions, files, processing, results, settings as settings_routes, export, ar, control,
analytics, controls, payouts, accounts_summary, fx, audit,
analytics, controls, payouts, accounts_summary,
)
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(_app: FastAPI):
init_db()
# A restart mid-job leaves a closing stuck on "processing" forever — recover it.
from ..services.jobs import recover_stale_jobs
recover_stale_jobs()
# Daily retention sweep (exports only; uploads are the audit source and are kept).
from ..services.retention import retention_loop
sweeper = asyncio.create_task(retention_loop())
yield
sweeper.cancel()
app = FastAPI(title=APP_NAME, version=APP_VERSION, lifespan=lifespan)
@app.middleware("http")
async def _request_log(request: Request, call_next):
"""One line per API request: method, path, status, duration."""
if not request.url.path.startswith("/api"):
return await call_next(request)
t0 = time.perf_counter()
response = await call_next(request)
ms = (time.perf_counter() - t0) * 1000
# /status is polled sub-second during processing; logging it would drown everything.
if not request.url.path.endswith("/status"):
logger.info("%s %s -> %d (%.0f ms)",
request.method, request.url.path, response.status_code, ms)
return response
# Registered BEFORE CORSMiddleware so CORS stays outermost (Starlette applies middleware
# in reverse registration order) and 401 responses still carry CORS headers.
app.middleware("http")(auth.auth_middleware)
app.add_middleware(
CORSMiddleware,
allow_origins=CORS_ORIGINS,
@ -72,27 +34,9 @@ app.add_middleware(
@app.get("/api/health")
def health() -> dict:
"""Liveness + readiness: DB reachable and the data dir writable — deep enough for a
load balancer / monitor probe, fast enough to hit every few seconds."""
checks = {"db": "ok", "data_dir": "ok"}
status = "ok"
try:
with ENGINE.connect() as conn:
conn.execute(text("SELECT 1"))
except Exception as e: # noqa: BLE001
checks["db"] = f"error: {type(e).__name__}"
status = "degraded"
try:
probe = DATA_DIR / ".health-probe"
probe.write_text("ok")
probe.unlink()
except OSError as e:
checks["data_dir"] = f"error: {type(e).__name__}"
status = "degraded"
return {"status": status, "app": APP_NAME, "version": APP_VERSION, "checks": checks}
return {"status": "ok", "app": APP_NAME, "version": APP_VERSION}
app.include_router(auth.router)
app.include_router(sessions.router)
app.include_router(files.router)
app.include_router(processing.router)
@ -107,5 +51,3 @@ app.include_router(analytics.router)
app.include_router(controls.router)
app.include_router(payouts.router)
app.include_router(accounts_summary.router)
app.include_router(fx.router)
app.include_router(audit.router)

37
ar-aging-app/backend/app/api/routes/accounts_summary.py
View File

@ -77,42 +77,6 @@ def accounts_summary(db: OrmSession = Depends(db_dep)) -> dict:
"receivable": accrual_total, # Dr A/R (net revenue accrued)
})
# Months that HAVE results but are not published, so they don't just vanish from this
# view without a word (the #1 "my previous month disappeared" confusion): processed or
# blocked closings whose journal is not approved — including sign-offs cleared by a
# re-process — are listed with the reason and a link target.
published_ids = {m["session_id"] for m in months}
pending: list[dict] = []
candidates = db.query(models.Session).filter(
models.Session.status.in_(("processed", "blocked", "completed"))).all()
journals = {j.session_id: j for j in db.query(models.JournalEntry).filter(
models.JournalEntry.session_id.in_([s.id for s in candidates]))} if candidates else {}
for s in candidates:
if s.id in published_ids:
continue
j = journals.get(s.id)
if is_blocked(s):
reason = f"blocked by a failed month-end control — {s.blocked_reason}"
elif j is None or not j.data:
reason = "no journal entry yet — re-process the closing"
elif j.approved_by:
reason = "approved, but the closing is blocked or has no journal data"
elif j.entry_no and not j.reviewed_by:
# An entry number exists but both sign-offs are empty: the usual cause is a
# re-process, which deliberately withdraws review/approval.
reason = ("sign-off was cleared (typically by re-processing) — "
"review and approve the journal again to re-publish")
else:
reason = "journal not approved yet — approval is what publishes a month here"
pending.append({
"month": s.reporting_month or (s.month_end_date.isoformat()[:7]
if s.month_end_date else f"session-{s.id}"),
"session_id": s.id,
"session_name": s.name,
"reason": reason,
})
pending.sort(key=lambda p: p["month"])
return {
"available": bool(months),
"line_keys": line_keys,
@ -120,5 +84,4 @@ def accounts_summary(db: OrmSession = Depends(db_dep)) -> dict:
"months": months,
"marketplaces": sorted(marketplaces),
"cells": cells,
"pending": pending,
}

95
ar-aging-app/backend/app/api/routes/analytics.py
View File

@ -12,7 +12,6 @@ opening/payout inputs the AR Ledger uses, so every tab ties back to the Overview
"""
from __future__ import annotations
import bisect
import datetime as dt
import json
from collections import defaultdict
@ -24,7 +23,7 @@ from sqlalchemy.orm import Session as OrmSession
from ...core.i18n import currency_for_region, default_fx_for_region
from ...db import models
from ..deps import db_dep, ensure_editable, get_session_or_404
from ..deps import db_dep, get_session_or_404
from .ar import _market_list, _movement_for, _payouts_for, fx_for
router = APIRouter(prefix="/api/sessions", tags=["analytics"])
@ -81,34 +80,6 @@ def _fx_for(db: OrmSession, session_id: int, marketplace: str) -> tuple[float, d
return month_rate, daily
def _effective_rate(month_rate: float, daily: dict[dt.date, tuple[float, str]]):
"""(rate, source) effective on a transaction date.
Resolution order:
1. that exact date's daily row (a provider fixing, or a hand-entered rate);
2. the most recent PROVIDER fixing before it — a weekend/holiday has no fixing,
so the previous banking day's rate is still in effect. Hand-entered rates are
deliberate single-date overrides and never carry forward;
3. the marketplace month rate (also used for undated rows and the opening balance,
which have no transaction date).
"""
fixing_dates = sorted(d for d, (_r, src) in daily.items() if (src or "") != "manual")
def resolve(d: dt.date | None) -> tuple[float, str]:
if d is not None:
hit = daily.get(d)
if hit is not None:
return hit
i = bisect.bisect_left(fixing_dates, d) - 1
if i >= 0:
prev = fixing_dates[i]
rate, src = daily[prev]
return rate, f"{src} {prev.isoformat()} (previous banking day)"
return month_rate, "month rate"
return resolve
def _daily_rows(db: OrmSession, session_id: int, marketplace: str,
frm: dt.date | None, to: dt.date | None) -> list[tuple[dt.date, float, int]]:
"""Per-day (date, revenue_total, row_count) for one marketplace — NON-transfer rows.
@ -206,23 +177,10 @@ def ledger_detail(session_id: int, marketplace: str | None = None, granularity:
frm, to = _parse_date(date_from, "date_from"), _parse_date(date_to, "date_to")
rows = _daily_rows(db, session_id, mkt, frm, to)
# Both currencies: USD is converted at the rate EFFECTIVE ON EACH TRANSACTION DATE —
# that date's fixing (auto-fetched from the provider at processing), the previous
# banking day's fixing for weekends/holidays, the month rate as last resort. The
# opening balance has no transaction date, so it converts at the month rate — the
# closing's official rate.
month_rate, daily = _fx_for(db, session_id, mkt)
effective = _effective_rate(month_rate, daily)
def rate_of(d: dt.date | None) -> float:
return effective(d)[0]
def new_bucket(key: str, label: str) -> dict:
return {"key": key, "label": label, "revenue": 0.0,
"payouts_received": 0.0, "payouts_in_transit": 0.0,
"bank_dated": 0.0, "rows": 0,
"revenue_usd": 0.0, "payouts_received_usd": 0.0,
"payouts_in_transit_usd": 0.0}
"bank_dated": 0.0, "rows": 0}
# Revenue buckets by transaction date; payouts by their EFFECTIVE date — the bank
# receipt's date when Finance entered one, Amazon's transfer date otherwise.
@ -231,7 +189,6 @@ def ledger_detail(session_id: int, marketplace: str | None = None, granularity:
key, label = _bucket(d, granularity)
b = buckets.setdefault(key, new_bucket(key, label))
b["revenue"] += revenue
b["revenue_usd"] += revenue * rate_of(d)
b["rows"] += n
for d, amount, received, bank_dated in _payout_events(db, s, mkt):
if frm and (d is None or d < frm):
@ -242,21 +199,16 @@ def ledger_detail(session_id: int, marketplace: str | None = None, granularity:
b = buckets.setdefault(key, new_bucket(key, label))
if amount:
b["payouts_received" if received else "payouts_in_transit"] += amount
b["payouts_received_usd" if received else "payouts_in_transit_usd"] += \
amount * rate_of(d)
if bank_dated:
b["bank_dated"] += amount
b["rows"] += 1
opening = mv["opening"]
running = opening
opening_usd = opening * month_rate
running_usd = opening_usd
out = []
for key in sorted(buckets):
b = buckets[key]
running += b["revenue"] + b["payouts_received"]
running_usd += b["revenue_usd"] + b["payouts_received_usd"]
out.append({
"key": b["key"], "label": b["label"],
"revenue": round(b["revenue"], 2),
@ -265,10 +217,6 @@ def ledger_detail(session_id: int, marketplace: str | None = None, granularity:
"bank_dated": round(b["bank_dated"], 2), # payout amounts placed by bank date
"rows": b["rows"],
"balance": round(running, 2),
"revenue_usd": round(b["revenue_usd"], 2),
"payouts_received_usd": round(b["payouts_received_usd"], 2),
"payouts_in_transit_usd": round(b["payouts_in_transit_usd"], 2),
"balance_usd": round(running_usd, 2),
})
filtered = bool(frm or to)
@ -287,12 +235,6 @@ def ledger_detail(session_id: int, marketplace: str | None = None, granularity:
"session_closing": mv["closing"],
"filtered": filtered,
"in_transit_total": round(sum(p["payouts_in_transit"] for p in out), 2),
"month_rate": month_rate,
"opening_usd": round(opening_usd, 2),
# Roll-forward valued at transaction-date rates; differs from closing × month rate
# whenever daily overrides exist — that spread is the FX effect of the month.
"closing_usd": round(running_usd, 2),
"in_transit_total_usd": round(sum(p["payouts_in_transit_usd"] for p in out), 2),
}
@ -309,7 +251,7 @@ def fx_daily(session_id: int, marketplace: str | None = None,
date_from: str | None = None, date_to: str | None = None,
db: OrmSession = Depends(db_dep)) -> dict:
"""Per-date local value, the USD rate applied, and the USD equivalent."""
s = get_session_or_404(session_id, db)
get_session_or_404(session_id, db)
mv = _movement_for(db, session_id, marketplace)
if not mv.get("available"):
return {"available": False}
@ -318,38 +260,19 @@ def fx_daily(session_id: int, marketplace: str | None = None,
month_rate, daily = _fx_for(db, session_id, mkt)
frm, to = _parse_date(date_from, "date_from"), _parse_date(date_to, "date_to")
# Revenue by transaction date; payouts by their EFFECTIVE date (bank receipt when one
# was entered, Amazon's transfer date otherwise) — the same placement the ledger uses,
# so this table converts exactly the movement the ledger shows.
per: dict[dt.date, list[float]] = defaultdict(lambda: [0.0, 0.0, 0]) # revenue, payouts, rows
for d, revenue, n in _daily_rows(db, session_id, mkt, frm, to):
if d is None:
continue
slot = per[d]
slot[0] += revenue
slot[2] += n
for d, amount, _received, _bank_dated in _payout_events(db, s, mkt):
if d is None or (frm and d < frm) or (to and d > to):
continue
slot = per[d]
slot[1] += amount
slot[2] += 1
effective = _effective_rate(month_rate, daily)
rows = []
tot_local = tot_usd = 0.0
for d in sorted(per):
revenue, payout, n = per[d]
for d, revenue, payout, n in _daily_rows(db, session_id, mkt, frm, to):
if d is None:
continue
local = revenue + payout
# The rate effective on the transaction date; the source column discloses a
# previous-banking-day carry-forward, so the conversion stays auditable.
rate, source = effective(d)
rate, source = daily.get(d, (month_rate, "month rate"))
usd = local * rate
tot_local += local
tot_usd += usd
rows.append({
"date": d.isoformat(), "local": round(local, 2), "rate": rate,
"usd": round(usd, 2), "source": source, "rows": int(n),
"usd": round(usd, 2), "source": source, "rows": n,
"revenue": round(revenue, 2), "payouts": round(payout, 2),
})
return {
@ -367,7 +290,7 @@ def fx_daily(session_id: int, marketplace: str | None = None,
@router.put("/{session_id}/fx-daily")
def put_fx_daily(session_id: int, items: list[DailyFxIn],
db: OrmSession = Depends(db_dep)) -> dict:
ensure_editable(get_session_or_404(session_id, db))
get_session_or_404(session_id, db)
for it in items:
d = _parse_date(it.rate_date, "rate_date")
row = db.query(models.FxRateDaily).filter(

6
ar-aging-app/backend/app/api/routes/ar.py
View File

@ -10,8 +10,7 @@ from sqlalchemy.orm import Session as OrmSession
from ...core.i18n import currency_for_region, default_fx_for_region
from ...core.movement import compute_movement
from ...db import models
from ..deps import (blocked_payload, db_dep, ensure_editable, get_session_or_404, is_blocked,
to_dict)
from ..deps import blocked_payload, db_dep, get_session_or_404, is_blocked, to_dict
router = APIRouter(prefix="/api/sessions", tags=["ar"])
@ -51,7 +50,6 @@ def put_openings(session_id: int, items: list[OpeningIn],
db: OrmSession = Depends(db_dep)) -> list[dict]:
"""Set one or more marketplaces' opening balances (only the ones sent are touched)."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
existing = {o.marketplace: o for o in db.query(models.OpeningBalance).filter(
models.OpeningBalance.session_id == session_id)}
for it in items:
@ -367,7 +365,6 @@ def carry_forward(session_id: int, body: CarryForwardIn | None = None,
db: OrmSession = Depends(db_dep)) -> dict:
"""Copy a prior closing's per-marketplace closing balance into this closing's opening."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
src_id = (body.from_session_id if body else None) or s.opening_source_session_id
if src_id is None:
cands = opening_candidates(session_id, db)["candidates"]
@ -405,7 +402,6 @@ def carry_forward(session_id: int, body: CarryForwardIn | None = None,
def reset_openings(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
"""Set every opening balance to zero (the default for a first-ever closing)."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
for o in db.query(models.OpeningBalance).filter(
models.OpeningBalance.session_id == session_id):
o.amount = 0.0

41
ar-aging-app/backend/app/api/routes/audit.py
View File

@ -1,41 +0,0 @@
"""Audit-log read API — admins only (users.is_admin, granted via `manage.py set-admin`)."""
from __future__ import annotations
from fastapi import APIRouter, Depends, Request
from sqlalchemy.orm import Session as OrmSession
from ...db import models
from ..auth import require_admin
from ..deps import db_dep
router = APIRouter(prefix="/api/audit", tags=["audit"])
@router.get("")
def list_audit(request: Request, limit: int = 100, offset: int = 0,
session_id: int | None = None, action: str = "",
db: OrmSession = Depends(db_dep)) -> dict:
"""Newest first. Filter by closing and/or action; page with limit/offset."""
require_admin(request, db)
limit = max(1, min(limit, 500))
q = db.query(models.AuditLog)
if session_id is not None:
q = q.filter(models.AuditLog.session_id == session_id)
if action:
q = q.filter(models.AuditLog.action == action)
total = q.count()
rows = (q.order_by(models.AuditLog.at.desc(), models.AuditLog.id.desc())
.offset(offset).limit(limit).all())
return {
"total": total,
"entries": [{
"id": r.id,
"at": r.at.isoformat() if r.at else None,
"username": r.username or "",
"display_name": r.display_name or "",
"action": r.action,
"session_id": r.session_id,
"session_name": r.session_name or "",
"detail": r.detail or "",
} for r in rows],
}

19
ar-aging-app/backend/app/api/routes/control.py
View File

@ -4,14 +4,13 @@ from __future__ import annotations
import datetime as dt
import json
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy.orm import Session as OrmSession
from ...core.money import USD, Total, to_usd
from ...db import models
from ..auth import actor_name
from ..deps import db_dep, ensure_editable, ensure_not_blocked, get_session_or_404
from ..deps import db_dep, ensure_not_blocked, get_session_or_404
router = APIRouter(prefix="/api/sessions", tags=["control"])
@ -124,7 +123,7 @@ def _get_or_create(db: OrmSession, session_id: int) -> models.FinanceControl:
@router.put("/{session_id}/reconciliation-control")
def put_control(session_id: int, body: ControlIn, db: OrmSession = Depends(db_dep)) -> dict:
ensure_editable(get_session_or_404(session_id, db))
get_session_or_404(session_id, db)
fc = _get_or_create(db, session_id)
data = body.model_dump(exclude_unset=True)
# A sign-off attests to specific numbers. If any control figure or the tolerance changes,
@ -144,19 +143,15 @@ def put_control(session_id: int, body: ControlIn, db: OrmSession = Depends(db_de
class VerifyIn(BaseModel):
verified_by: str = "" # ignored when signed in — the verified identity wins
verified_by: str
comment: str = ""
@router.post("/{session_id}/reconciliation-control/verify")
def verify_control(session_id: int, body: VerifyIn, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
ensure_editable(get_session_or_404(session_id, db))
who = actor_name(request, body.verified_by)
if not who:
raise HTTPException(400, "verified_by is required — the control is verified by a person.")
def verify_control(session_id: int, body: VerifyIn, db: OrmSession = Depends(db_dep)) -> dict:
get_session_or_404(session_id, db)
fc = _get_or_create(db, session_id)
fc.verified_by = who
fc.verified_by = body.verified_by
fc.verified_at = dt.datetime.utcnow()
if body.comment:
fc.comment = body.comment

23
ar-aging-app/backend/app/api/routes/controls.py
View File

@ -3,14 +3,13 @@ from __future__ import annotations
import datetime as dt
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy.orm import Session as OrmSession
from ...db import models
from ...services.controls_run import payload, run_and_persist
from ..auth import actor_name
from ..deps import db_dep, ensure_editable, get_session_or_404
from ..deps import db_dep, get_session_or_404
router = APIRouter(prefix="/api/sessions", tags=["controls"])
@ -39,12 +38,11 @@ class FxConfirmIn(BaseModel):
marketplace: str
rate: float | None = None # optionally correct the rate while confirming it
currency: str | None = None
confirmed_by: str = "" # ignored when signed in — the verified identity wins
confirmed_by: str
@router.post("/{session_id}/fx/confirm")
def confirm_fx(session_id: int, body: FxConfirmIn, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
def confirm_fx(session_id: int, body: FxConfirmIn, db: OrmSession = Depends(db_dep)) -> dict:
"""
Record that a human confirmed this marketplace's rate FOR THIS REPORTING MONTH.
@ -52,9 +50,7 @@ def confirm_fx(session_id: int, body: FxConfirmIn, request: Request,
snapshot and would otherwise value any later month at January's rates in silence.
"""
s = get_session_or_404(session_id, db)
ensure_editable(s)
who = actor_name(request, body.confirmed_by)
if not who:
if not body.confirmed_by.strip():
raise HTTPException(400, "confirmed_by is required — a rate is confirmed by a person.")
row = db.query(models.FxRate).filter(
models.FxRate.session_id == session_id,
@ -66,7 +62,7 @@ def confirm_fx(session_id: int, body: FxConfirmIn, request: Request,
row.rate = body.rate
if body.currency:
row.currency = body.currency
row.confirmed_by = who
row.confirmed_by = body.confirmed_by.strip()
row.confirmed_at = dt.datetime.utcnow()
row.confirmed_month = s.reporting_month or ""
row.source = f"confirmed by {row.confirmed_by}"
@ -75,16 +71,15 @@ def confirm_fx(session_id: int, body: FxConfirmIn, request: Request,
class FxConfirmAllIn(BaseModel):
confirmed_by: str = "" # ignored when signed in — the verified identity wins
confirmed_by: str
@router.post("/{session_id}/fx/confirm-all")
def confirm_all_fx(session_id: int, body: FxConfirmAllIn, request: Request,
def confirm_all_fx(session_id: int, body: FxConfirmAllIn,
db: OrmSession = Depends(db_dep)) -> dict:
"""Confirm every rate on the closing as-is (after reviewing them on the Settings tab)."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
who = actor_name(request, body.confirmed_by)
who = body.confirmed_by.strip()
if not who:
raise HTTPException(400, "confirmed_by is required — a rate is confirmed by a person.")
now = dt.datetime.utcnow()

25
ar-aging-app/backend/app/api/routes/export.py
View File

@ -3,12 +3,11 @@ from __future__ import annotations
import os
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
from fastapi.responses import FileResponse
from sqlalchemy.orm import Session as OrmSession
from ...db import models
from ...services.audit import record as audit
from ...services.jobs import run_export, run_summary_export
from ..deps import db_dep, ensure_not_blocked, get_session_or_404
@ -16,25 +15,14 @@ router = APIRouter(prefix="/api/sessions", tags=["export"])
@router.post("/{session_id}/export")
def start_export(session_id: int, background: BackgroundTasks, request: Request,
kind: str = "full", db: OrmSession = Depends(db_dep)) -> dict:
def start_export(session_id: int, background: BackgroundTasks, kind: str = "full",
db: OrmSession = Depends(db_dep)) -> dict:
"""kind='summary' → compact Finance pack (fast); kind='full' → complete audit workbook."""
if kind not in ("full", "summary"):
raise HTTPException(400, "kind must be 'full' or 'summary'.")
s = get_session_or_404(session_id, db)
# An export is the number leaving the building — never generate one from a blocked close.
ensure_not_blocked(s)
# The full workbook RE-COMPUTES its marketplace tabs from the source files using the
# current bank receipts, while the AR Ledger / Finance Summary sheets bound into the same
# file come from the last processing run. With unapplied receipts those two halves
# disagree — the tabs would show one receivable and the ledger sheet another.
if s.needs_reprocess:
raise HTTPException(
409,
"Bank receipts or the payout mode changed after the last run. Re-process the "
"closing first — otherwise the workbook's marketplace tabs and its AR Ledger "
"sheet would report different receivables.",
)
if s.status not in ("processed", "exporting", "completed"):
raise HTTPException(400, "Process the session before exporting.")
if s.status == "exporting":
@ -47,7 +35,6 @@ def start_export(session_id: int, background: BackgroundTasks, request: Request,
s.progress_rows_total = 0
s.eta_seconds = 0
s.error = ""
audit(db, request, "export_generate", session=s, detail=f"kind={kind}")
db.commit()
background.add_task(run_summary_export if kind == "summary" else run_export, session_id)
return {"started": True, "kind": kind}
@ -65,8 +52,7 @@ def list_exports(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict
@router.get("/{session_id}/export/download")
def download_export(session_id: int, request: Request, kind: str = "full",
db: OrmSession = Depends(db_dep)):
def download_export(session_id: int, kind: str = "full", db: OrmSession = Depends(db_dep)):
s = get_session_or_404(session_id, db)
# A workbook generated before a control started failing must not keep circulating.
ensure_not_blocked(s)
@ -76,9 +62,6 @@ def download_export(session_id: int, request: Request, kind: str = "full",
r = q.order_by(models.ExportRecord.generated_at.desc()).first()
if not r or not r.path or not os.path.exists(r.path):
raise HTTPException(404, "No export available; generate it first.")
audit(db, request, "export_download",
session=s, detail=f"kind={kind} '{os.path.basename(r.path)}'")
db.commit()
return FileResponse(
r.path, filename=os.path.basename(r.path),
media_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",

134
ar-aging-app/backend/app/api/routes/files.py
View File

@ -4,15 +4,13 @@ from __future__ import annotations
import hashlib
import os
from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
from sqlalchemy.orm import Session as OrmSession
from ...config import ALLOWED_EXTENSIONS, MAX_UPLOAD_BYTES, UPLOAD_DIR
from ...core.readers import make_reader
from ...core.xlsx_reader import ParseError
from ...core.xlsx_reader import TransactionReader, ParseError
from ...db import models
from ...services.audit import record as audit
from ..deps import db_dep, ensure_editable, file_dict, get_session_or_404, sanitize_filename
from ..deps import db_dep, file_dict, get_session_or_404, sanitize_filename
router = APIRouter(prefix="/api/sessions", tags=["files"])
@ -24,68 +22,22 @@ def list_files(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict]:
return [file_dict(f) for f in rows]
def _validate(rec: models.SessionFile, path: str) -> None:
"""Light validation: detect sheet/header + required columns (no full row scan)."""
try:
reader = make_reader(path)
reader.detect()
rec.data_sheet = reader.sheet_name
rec.status = "invalid" if reader.column_mapping.missing_required else "parsed"
rec.message = (f"missing required columns: {reader.column_mapping.missing_required}"
if reader.column_mapping.missing_required else "")
# Surface marketplace / date span when cheap (CSV already has rows in memory;
# for xlsx this stays blank until processing).
meta = reader.file_meta
if getattr(meta, "currency", None):
rec.currency = meta.currency
reader.close()
except ParseError as e:
rec.status = "invalid"
rec.message = str(e)
@router.post("/{session_id}/files")
async def upload_files(session_id: int, request: Request, files: list[UploadFile] = File(...),
db: OrmSession = Depends(db_dep)) -> dict:
"""
Upload one or more source files into a closing.
Duplicate protection (both were real double-count bugs):
* same filename again -> the existing row is UPDATED in place (the file is replaced),
never a second row pointing at the same path — two rows would make the pipeline
parse and sum the file twice.
* same content under a different name -> skipped and reported, for the same reason.
Returns {"files": [...saved/replaced...], "skipped": [{"filename", "reason"}]} so one
duplicate in a 13-file batch doesn't fail the other twelve.
"""
async def upload_files(session_id: int, files: list[UploadFile] = File(...),
db: OrmSession = Depends(db_dep)) -> list[dict]:
session = get_session_or_404(session_id, db)
ensure_editable(session)
dest_dir = UPLOAD_DIR / f"session_{session_id}"
dest_dir.mkdir(parents=True, exist_ok=True)
existing = db.query(models.SessionFile).filter(
models.SessionFile.session_id == session_id).all()
by_name = {f.filename: f for f in existing}
by_sha = {f.sha256: f for f in existing if f.sha256}
was_processed = session.status in ("processed", "blocked")
out: list[models.SessionFile] = []
skipped: list[dict] = []
changed = False
out = []
for uf in files:
safe = sanitize_filename(uf.filename or "upload.xlsx")
ext = os.path.splitext(safe)[1].lower()
if ext not in ALLOWED_EXTENSIONS:
raise HTTPException(400, f"Unsupported file type: {safe} ({ext})")
path = dest_dir / safe
# Stream to a temp name first: the hash decides whether this upload is kept, and a
# failed/oversized upload must never clobber a good file already on disk.
tmp = dest_dir / (safe + ".part")
h = hashlib.sha256()
size = 0
with open(tmp, "wb") as fh:
with open(path, "wb") as fh:
while True:
chunk = await uf.read(1 << 20)
if not chunk:
@ -93,72 +45,35 @@ async def upload_files(session_id: int, request: Request, files: list[UploadFile
size += len(chunk)
if size > MAX_UPLOAD_BYTES:
fh.close()
os.remove(tmp)
os.remove(path)
raise HTTPException(413, f"File too large: {safe}")
h.update(chunk)
fh.write(chunk)
sha = h.hexdigest()
same_name = by_name.get(safe)
same_content = by_sha.get(sha)
if same_name is not None and same_name.sha256 == sha:
os.remove(tmp)
skipped.append({"filename": safe,
"reason": "identical file already uploaded — unchanged"})
continue
if same_content is not None and (same_name is None or same_content.id != same_name.id):
os.remove(tmp)
skipped.append({"filename": safe,
"reason": f"identical content already uploaded as "
f"'{same_content.filename}'"})
continue
os.replace(tmp, path)
changed = True
audit(db, request, "file_upload", session=session,
detail=f"'{safe}' ({size:,} bytes)"
+ (" — replaced the existing file" if same_name is not None else ""))
if same_name is not None:
# Replace in place: update the existing row rather than adding a second one.
rec = same_name
rec.stored_path = str(path)
rec.size_bytes = size
rec.sha256 = sha
rec.status = "uploaded"
rec.message = ""
rec.imported_rows = 0
rec.min_date = None
rec.max_date = None
rec.marketplace = None
rec.sheet_last_row = 0
rec.blank_rows_skipped = 0
rec.helper_rows_skipped = 0
else:
rec = models.SessionFile(
session_id=session_id, filename=safe, stored_path=str(path),
size_bytes=size, sha256=sha, status="uploaded",
size_bytes=size, sha256=h.hexdigest(), status="uploaded",
)
# light validation: detect sheet + required columns (no full row scan)
try:
reader = TransactionReader(str(path))
reader.detect()
rec.data_sheet = reader.sheet_name
rec.status = "invalid" if reader.column_mapping.missing_required else "parsed"
if reader.column_mapping.missing_required:
rec.message = f"missing required columns: {reader.column_mapping.missing_required}"
reader.close()
except ParseError as e:
rec.status = "invalid"
rec.message = str(e)
db.add(rec)
_validate(rec, str(path))
by_name[safe] = rec
by_sha[sha] = rec
out.append(rec)
if changed:
session.status = "draft"
if was_processed:
# The stored results no longer reflect the files on disk.
session.needs_reprocess = True
db.commit()
return {"files": [file_dict(f) for f in out], "skipped": skipped}
return [file_dict(f) for f in out]
@router.delete("/{session_id}/files/{file_id}")
def delete_file(session_id: int, file_id: int, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
s = get_session_or_404(session_id, db)
ensure_editable(s)
def delete_file(session_id: int, file_id: int, db: OrmSession = Depends(db_dep)) -> dict:
f = db.get(models.SessionFile, file_id)
if not f or f.session_id != session_id:
raise HTTPException(404, "File not found")
@ -167,9 +82,6 @@ def delete_file(session_id: int, file_id: int, request: Request,
os.remove(f.stored_path)
except OSError:
pass
audit(db, request, "file_delete", session=s, detail=f"'{f.filename}'")
db.delete(f)
if s.status in ("processed", "blocked"):
s.needs_reprocess = True
db.commit()
return {"deleted": file_id}

48
ar-aging-app/backend/app/api/routes/fx.py
View File

@ -1,48 +0,0 @@
"""Fetch exchange rates from the configured provider (Frankfurter by default).
Fetched rates arrive UNCONFIRMED: Control C5 still blocks the close until a person
confirms them for the reporting month — this endpoint only replaces typing rates by hand."""
from __future__ import annotations
import datetime as dt
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy.orm import Session as OrmSession
from ...services.fx_service import FxProviderError, seed_daily_fx, seed_session_fx
from ..deps import db_dep, ensure_editable, get_session_or_404
router = APIRouter(prefix="/api/sessions", tags=["fx"])
@router.post("/{session_id}/fx/fetch")
def fetch_month_end_rates(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
"""Pre-fill this closing's FX table with the provider's month-end rates."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
try:
return seed_session_fx(db, s)
except FxProviderError as e:
raise HTTPException(502, f"{e} — enter the rates manually on the Controls tab.")
class DailyFetchIn(BaseModel):
marketplace: str | None = None # default: every non-USD marketplace in the closing
date_from: dt.date | None = None # default: the closing's earliest dated transaction
date_to: dt.date | None = None # default: month-end (or the latest transaction)
@router.post("/{session_id}/fx/fetch-daily")
def fetch_daily_rates(session_id: int, body: DailyFetchIn | None = None,
db: OrmSession = Depends(db_dep)) -> dict:
"""(Re-)fetch the per-date FX table from the provider. Processing already does this
automatically; the explicit fetch also replaces hand-entered overrides."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
body = body or DailyFetchIn()
try:
return seed_daily_fx(db, s, marketplace=body.marketplace,
date_from=body.date_from, date_to=body.date_to)
except FxProviderError as e:
raise HTTPException(502, f"{e} — enter daily rates manually on the AR Ledger tab.")

117
ar-aging-app/backend/app/api/routes/payouts.py
View File

@ -19,43 +19,19 @@ from __future__ import annotations
import datetime as dt
from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import func
from sqlalchemy.orm import Session as OrmSession
from ...config import MAX_UPLOAD_BYTES
from ...core.bank_import import BankImportError, match_payouts, parse_disbursements
from ...core.i18n import currency_for_region
from ...db import models
from ..auth import actor_name
from ..deps import db_dep, ensure_editable, get_session_or_404
from ..deps import db_dep, get_session_or_404
router = APIRouter(prefix="/api/sessions", tags=["payouts"])
TRANSFER = "Transfer"
def _payout_rows(db: OrmSession, session_id: int, marketplace: str | None = None):
"""One row per (marketplace, account stream, settlement id) — the key the engine
classifies on: (mkt, acct, sid, max(posted_date), sum(total), count)."""
q = db.query(
models.Transaction.marketplace,
models.Transaction.account_type,
models.Transaction.settlement_id,
func.max(models.Transaction.posted_date),
func.sum(models.Transaction.total),
func.count(),
).filter(
models.Transaction.session_id == session_id,
models.Transaction.txn_type_en == TRANSFER,
)
if marketplace:
q = q.filter(models.Transaction.marketplace == marketplace)
return q.group_by(models.Transaction.marketplace, models.Transaction.account_type,
models.Transaction.settlement_id)
@router.get("/{session_id}/payouts")
def list_payouts(session_id: int, marketplace: str | None = None,
db: OrmSession = Depends(db_dep)) -> dict:
@ -67,7 +43,21 @@ def list_payouts(session_id: int, marketplace: str | None = None,
Finance recorded it as received.
"""
s = get_session_or_404(session_id, db)
q = _payout_rows(db, session_id, marketplace)
q = db.query(
models.Transaction.marketplace,
models.Transaction.account_type,
models.Transaction.settlement_id,
func.max(models.Transaction.posted_date),
func.sum(models.Transaction.total),
func.count(),
).filter(
models.Transaction.session_id == session_id,
models.Transaction.txn_type_en == TRANSFER,
)
if marketplace:
q = q.filter(models.Transaction.marketplace == marketplace)
q = q.group_by(models.Transaction.marketplace, models.Transaction.account_type,
models.Transaction.settlement_id)
receipts = {(r.marketplace, r.account_type, r.settlement_id): r
for r in db.query(models.PayoutReceipt).filter(
@ -128,12 +118,11 @@ class ReceiptIn(BaseModel):
@router.put("/{session_id}/payouts/receipts")
def put_receipts(session_id: int, items: list[ReceiptIn], request: Request,
def put_receipts(session_id: int, items: list[ReceiptIn],
db: OrmSession = Depends(db_dep)) -> dict:
"""Batch upsert bank receipts. Only the payouts sent are touched; a null bank_date
deletes that payout's receipt (it reverts to the mode's default rule)."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
if s.status == "processing":
raise HTTPException(409, "This closing is still processing — wait for it to finish.")
existing = {(r.marketplace, r.account_type, r.settlement_id): r
@ -160,8 +149,7 @@ def put_receipts(session_id: int, items: list[ReceiptIn], request: Request,
row.bank_date = bank_date
row.bank_amount = it.bank_amount
row.note = it.note or ""
# The signed-in user's name wins; the free-text field only counts without auth.
row.entered_by = actor_name(request, it.entered_by)
row.entered_by = it.entered_by or ""
saved += 1
if saved or removed:
# The stored classification no longer reflects the receipts until a re-process.
@ -170,72 +158,6 @@ def put_receipts(session_id: int, items: list[ReceiptIn], request: Request,
return {"saved": saved, "removed": removed, "needs_reprocess": bool(s.needs_reprocess)}
@router.post("/{session_id}/payouts/receipts/import")
async def import_receipts(session_id: int, file: UploadFile = File(...),
window_days: int = 14,
db: OrmSession = Depends(db_dep)) -> dict:
"""
Parse a bank disbursements workbook (sheet 'Payouts': Date/Debit/FCY/Party Name…) and
propose bank receipts for this closing's payouts. READ-ONLY: nothing is saved — the
client applies accepted matches through PUT /payouts/receipts, which keeps that
endpoint's semantics (upsert, needs_reprocess, entered_by) in one place.
"""
s = get_session_or_404(session_id, db)
name = (file.filename or "").lower()
if not name.endswith((".xlsx", ".xls")):
raise HTTPException(400, "Upload the bank disbursements Excel file (.xlsx).")
if not 1 <= window_days <= 60:
raise HTTPException(400, "window_days must be between 1 and 60.")
data = await file.read()
if len(data) > MAX_UPLOAD_BYTES:
raise HTTPException(400, "File too large.")
try:
rows, problems = parse_disbursements(data)
except BankImportError as e:
raise HTTPException(400, str(e))
def _date(v) -> dt.date | None:
if isinstance(v, dt.datetime):
return v.date()
if isinstance(v, dt.date):
return v
try:
return dt.date.fromisoformat(str(v)[:10]) if v else None
except ValueError:
return None
payouts = [
{"marketplace": mkt, "account_type": acct, "settlement_id": sid,
"amazon_date": _date(d), "amount": round(amount or 0.0, 2)}
for mkt, acct, sid, d, amount, _n in _payout_rows(db, session_id)
]
receipts = {(r.marketplace, r.account_type, r.settlement_id): r.bank_date
for r in db.query(models.PayoutReceipt).filter(
models.PayoutReceipt.session_id == session_id)}
# Session FX rows carry the marketplace's currency (confirmed by Finance); fall back
# to the built-in region -> currency table.
currencies = {mkt: currency_for_region(mkt)
for (mkt,) in db.query(models.Transaction.marketplace).filter(
models.Transaction.session_id == session_id).distinct()}
for fx in db.query(models.FxRate).filter(models.FxRate.session_id == session_id):
if fx.marketplace and fx.currency:
currencies[fx.marketplace] = fx.currency
m = match_payouts(rows, payouts, s.month_end_date, window_days=window_days,
receipts=receipts, currency_by_marketplace=currencies)
return {
"total_rows": len(rows),
"window_days": window_days,
"matched": m.matched,
"ambiguous": m.ambiguous,
"unmatched_bank_rows": m.unmatched,
"unknown_party": m.unknown_party,
"out_of_scope": m.out_of_scope,
"problems": problems + m.problems,
}
class ModeIn(BaseModel):
mode: str
@ -244,7 +166,6 @@ class ModeIn(BaseModel):
def put_mode(session_id: int, body: ModeIn, db: OrmSession = Depends(db_dep)) -> dict:
"""auto = bank date wins, clearing-lag fallback · manual = bank dates only, no heuristic."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
if body.mode not in ("auto", "manual"):
raise HTTPException(400, "mode must be 'auto' or 'manual'.")
if s.status == "processing":

10
ar-aging-app/backend/app/api/routes/processing.py
View File

@ -1,22 +1,20 @@
"""Start processing (background) and poll status/progress."""
from __future__ import annotations
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
from sqlalchemy.orm import Session as OrmSession
from ...db import models
from ...services.audit import record as audit
from ...services.jobs import run_processing
from ..deps import db_dep, ensure_editable, get_session_or_404, session_dict
from ..deps import db_dep, get_session_or_404, session_dict
router = APIRouter(prefix="/api/sessions", tags=["processing"])
@router.post("/{session_id}/process")
def start_processing(session_id: int, background: BackgroundTasks, request: Request,
def start_processing(session_id: int, background: BackgroundTasks,
db: OrmSession = Depends(db_dep)) -> dict:
s = get_session_or_404(session_id, db)
ensure_editable(s)
if s.month_end_date is None:
raise HTTPException(400, "Set the month-end date before processing.")
valid_files = db.query(models.SessionFile).filter(
@ -31,8 +29,6 @@ def start_processing(session_id: int, background: BackgroundTasks, request: Requ
s.progress_stage = "Queued"
s.progress_pct = 0.0
s.error = ""
audit(db, request, "process_run", session=s,
detail=f"{len(valid_files)} file(s)")
db.commit()
background.add_task(run_processing, session_id)
return {"started": True, "session_id": session_id}

51
ar-aging-app/backend/app/api/routes/results.py
View File

@ -4,16 +4,14 @@ from __future__ import annotations
import datetime as dt
import json
from fastapi import APIRouter, Body, Depends, HTTPException, Query, Request
from fastapi import APIRouter, Body, Depends, HTTPException, Query
from sqlalchemy import func
from sqlalchemy.orm import Session as OrmSession
from ...core.receivable import AGING_SCHEMES, aging_bands, classify_aging
from ...core.receivable import AGING_BANDS, classify_aging
from ...core.settlements import RECEIVABLE_ACCOUNT_TYPES
from ...db import models
from ..auth import actor_name
from ..deps import (blocked_payload, db_dep, ensure_editable, get_session_or_404, is_blocked,
to_dict)
from ..deps import blocked_payload, db_dep, get_session_or_404, is_blocked, to_dict
router = APIRouter(prefix="/api/sessions", tags=["results"])
@ -178,7 +176,7 @@ def journal(session_id: int, marketplace: str | None = None,
@router.put("/{session_id}/journal/entry-no")
def set_journal_entry_no(session_id: int, entry_no: str = Body(..., embed=True),
db: OrmSession = Depends(db_dep)) -> dict:
ensure_editable(get_session_or_404(session_id, db))
get_session_or_404(session_id, db)
j = db.query(models.JournalEntry).filter(
models.JournalEntry.session_id == session_id).first()
if j:
@ -196,35 +194,28 @@ def _journal_row_or_400(session_id: int, db: OrmSession) -> models.JournalEntry:
@router.post("/{session_id}/journal/review")
def review_journal(session_id: int, request: Request, name: str = Body("", embed=True),
def review_journal(session_id: int, name: str = Body(..., embed=True),
db: OrmSession = Depends(db_dep)) -> dict:
"""Step 1 of the sign-off: a person confirms they reviewed this month's journal.
The signed-in user's display name is recorded; the body `name` only counts when no
one is signed in (auth off — dev and tests)."""
ensure_editable(get_session_or_404(session_id, db))
who = actor_name(request, name)
if not who:
"""Step 1 of the sign-off: a person confirms they reviewed this month's journal."""
get_session_or_404(session_id, db)
if not name.strip():
raise HTTPException(400, "A reviewer name is required.")
j = _journal_row_or_400(session_id, db)
j.reviewed_by = who
j.reviewed_by = name.strip()
j.reviewed_at = dt.datetime.utcnow()
db.commit()
return journal(session_id, None, db)
@router.post("/{session_id}/journal/approve")
def approve_journal(session_id: int, request: Request, name: str = Body("", embed=True),
def approve_journal(session_id: int, name: str = Body(..., embed=True),
db: OrmSession = Depends(db_dep)) -> dict:
"""Step 2: approval — this is what publishes the month to the Accounts Summary.
Requires a prior review, and a closing that isn't blocked by a month-end control:
an unverified number must never become part of the cross-month accounts view.
The signed-in user's display name is recorded (body `name` only without auth)."""
an unverified number must never become part of the cross-month accounts view."""
s = get_session_or_404(session_id, db)
ensure_editable(s)
who = actor_name(request, name)
if not who:
if not name.strip():
raise HTTPException(400, "An approver name is required.")
if is_blocked(s):
raise HTTPException(409, f"This closing is blocked by a failed month-end control — "
@ -232,7 +223,7 @@ def approve_journal(session_id: int, request: Request, name: str = Body("", embe
j = _journal_row_or_400(session_id, db)
if not j.reviewed_by:
raise HTTPException(400, "The journal must be reviewed before it can be approved.")
j.approved_by = who
j.approved_by = name.strip()
j.approved_at = dt.datetime.utcnow()
db.commit()
return journal(session_id, None, db)
@ -241,7 +232,7 @@ def approve_journal(session_id: int, request: Request, name: str = Body("", embe
@router.post("/{session_id}/journal/reset-signoff")
def reset_journal_signoff(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
"""Withdraw the sign-off (removes the month from the Accounts Summary)."""
ensure_editable(get_session_or_404(session_id, db))
get_session_or_404(session_id, db)
j = _journal_row_or_400(session_id, db)
j.reviewed_by = ""
j.reviewed_at = None
@ -252,8 +243,7 @@ def reset_journal_signoff(session_id: int, db: OrmSession = Depends(db_dep)) ->
@router.get("/{session_id}/aging")
def aging(session_id: int, scheme: str = "monthly",
db: OrmSession = Depends(db_dep)) -> dict:
def aging(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
"""
Real aging, banded by days **past due** — not days since the transaction.
@ -268,12 +258,9 @@ def aging(session_id: int, scheme: str = "monthly",
the entire point of an aging report. (Banding by transaction date instead would push a
normal biweekly settlement into 1-30 and make the report meaningless.)
"""
if scheme not in AGING_SCHEMES:
raise HTTPException(400, f"scheme must be one of: {', '.join(AGING_SCHEMES)}.")
s = get_session_or_404(session_id, db)
if is_blocked(s):
return blocked_payload(s)
bands = aging_bands(scheme)
rows = db.query(models.ReceivableResultRow).filter(
models.ReceivableResultRow.session_id == session_id,
models.ReceivableResultRow.account_type == "TOTAL").all()
@ -293,12 +280,12 @@ def aging(session_id: int, scheme: str = "monthly",
days_overdue = (month_end - due).days
else:
days_overdue = 0
band = classify_aging(days_overdue, scheme)
by_mkt.setdefault(st.marketplace, {b: 0.0 for b in bands})[band] += st.order_total
band = classify_aging(days_overdue)
by_mkt.setdefault(st.marketplace, {b: 0.0 for b in AGING_BANDS})[band] += st.order_total
matrix = []
for r in rows:
local_bands = by_mkt.get(r.marketplace) or {b: 0.0 for b in bands}
local_bands = by_mkt.get(r.marketplace) or {b: 0.0 for b in AGING_BANDS}
composed = sum(local_bands.values())
# The receivable is ROUND(reserve + additional sales); the reserve and that rounding
# belong to the current period, so the residual lands in Current and the row still
@ -310,7 +297,7 @@ def aging(session_id: int, scheme: str = "monthly",
total = round(sum(band_usd.values()), 2)
matrix.append({"marketplace": r.marketplace, "currency": r.currency,
**band_usd, "Total": total})
return {"bands": list(bands), "scheme": scheme, "rows": matrix,
return {"bands": list(AGING_BANDS), "rows": matrix,
"basis": (f"days past due at month-end — a settlement becomes due "
f"{SETTLEMENT_CYCLE_DAYS} days after its last activity plus the "
f"{lag}-day clearing lag")}

87
ar-aging-app/backend/app/api/routes/sessions.py
View File

@ -1,22 +1,18 @@
"""Session (month-end closing) CRUD and parameters."""
from __future__ import annotations
import logging
from datetime import date
from datetime import date, datetime
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy.orm import Session as OrmSession
from ...config import DEFAULT_CLEARING_LAG_DAYS, DEFAULT_TOLERANCE
from ...db import models
from ...services.audit import record as audit
from ..deps import db_dep, get_session_or_404, session_dict
router = APIRouter(prefix="/api/sessions", tags=["sessions"])
logger = logging.getLogger(__name__)
class SessionCreate(BaseModel):
name: str
@ -28,9 +24,6 @@ class SessionCreate(BaseModel):
# zero (default) | carry_forward | manual
opening_mode: str = "zero"
opening_source_session_id: int | None = None
# Two closings for one month is almost always an accident (two competing datasets for
# the same period); creating a second one requires this explicit flag.
allow_duplicate: bool = False
class SessionUpdate(BaseModel):
@ -46,57 +39,19 @@ class SessionUpdate(BaseModel):
opening_source_session_id: int | None = None
def _approved_session_ids(db: OrmSession) -> set[int]:
rows = db.query(models.JournalEntry.session_id).filter(
models.JournalEntry.approved_by != "").all()
return {r[0] for r in rows}
@router.get("")
def list_sessions(db: OrmSession = Depends(db_dep)) -> list[dict]:
"""Every closing, newest month first — the dashboard reads as a month timeline."""
rows = db.query(models.Session).all()
# reporting_month is "YYYY-MM" so string sort == chronological; sessions without a
# month (never given a month-end date) sort last, newest created first.
rows.sort(key=lambda s: (s.reporting_month or "",
s.created_at.isoformat() if s.created_at else ""), reverse=True)
approved = _approved_session_ids(db)
months_seen: dict[str, int] = {}
for s in rows:
if s.reporting_month:
months_seen[s.reporting_month] = months_seen.get(s.reporting_month, 0) + 1
out = []
for s in rows:
d = session_dict(s)
# "Published" = the journal is approved, which is what puts the month on the
# cross-month Accounts Summary.
d["journal_approved"] = s.id in approved
d["duplicate_month"] = bool(s.reporting_month
and months_seen.get(s.reporting_month, 0) > 1)
out.append(d)
return out
rows = db.query(models.Session).order_by(models.Session.created_at.desc()).all()
return [session_dict(s) for s in rows]
@router.post("")
def create_session(body: SessionCreate, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
def create_session(body: SessionCreate, db: OrmSession = Depends(db_dep)) -> dict:
me = body.month_end_date
month = me.strftime("%Y-%m") if me else None
if month and not body.allow_duplicate:
clash = db.query(models.Session).filter(
models.Session.reporting_month == month,
models.Session.status != "error").first()
if clash is not None:
raise HTTPException(
409,
f"A closing for {month} already exists ('{clash.name}', id {clash.id}). "
f"Open that closing instead — or pass allow_duplicate to deliberately "
f"create a second one.",
)
s = models.Session(
name=body.name,
month_end_date=me,
reporting_month=month,
reporting_month=me.strftime("%Y-%m") if me else None,
reporting_currency=body.reporting_currency,
clearing_lag_days=body.clearing_lag_days,
rounding_tolerance=body.rounding_tolerance,
@ -106,9 +61,6 @@ def create_session(body: SessionCreate, request: Request,
status="draft",
)
db.add(s)
db.flush() # assign s.id so the audit row can reference it
audit(db, request, "session_create", session=s,
detail=f"month {month or '(none)'}")
db.commit()
from .ar import seed_opening_from_prior
seed_opening_from_prior(db, s)
@ -125,9 +77,6 @@ def update_session(session_id: int, body: SessionUpdate,
db: OrmSession = Depends(db_dep)) -> dict:
s = get_session_or_404(session_id, db)
data = body.model_dump(exclude_unset=True)
if s.status == "completed" and set(data) - {"name"}:
raise HTTPException(409, "This closing is completed and locked — only the name can "
"be changed. Reopen it first for anything else.")
for k, v in data.items():
setattr(s, k, v)
if "month_end_date" in data and s.month_end_date:
@ -136,33 +85,11 @@ def update_session(session_id: int, body: SessionUpdate,
return session_dict(s)
@router.post("/{session_id}/reopen")
def reopen_session(session_id: int, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
"""Unlock a completed closing for corrections. Deliberate and logged — the opposite of
silently editing published history."""
s = get_session_or_404(session_id, db)
if s.status != "completed":
raise HTTPException(409, "Only a completed closing can be reopened.")
s.status = "blocked" if s.blocked_reason else "processed"
audit(db, request, "session_reopen", session=s)
db.commit()
logger.warning("closing %s (%s, %s) reopened for corrections",
s.id, s.name, s.reporting_month or "no month")
return session_dict(s)
@router.delete("/{session_id}")
def delete_session(session_id: int, request: Request,
db: OrmSession = Depends(db_dep)) -> dict:
def delete_session(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
"""Delete a closing and every row/file that belongs to it."""
s = get_session_or_404(session_id, db)
if s.status in ("processing", "exporting"):
raise HTTPException(409, "This closing is still processing — wait for it to finish.")
# Recorded up front (audit rows carry no FK, so they survive the purge); committed
# here so the entry exists even though purge_session manages its own transaction.
audit(db, request, "session_delete", session=s,
detail=f"month {s.reporting_month or '(none)'}, status {s.status}")
db.commit()
from ...services.store import purge_session
return purge_session(db, session_id)

5
ar-aging-app/backend/app/api/routes/settings.py
View File

@ -9,7 +9,7 @@ from sqlalchemy.orm import Session as OrmSession
from ...core.column_map import FIELD_ORDER, normalize_header
from ...db import models
from ..deps import db_dep, ensure_editable, get_session_or_404, to_dict
from ..deps import db_dep, get_session_or_404, to_dict
router = APIRouter(prefix="/api/sessions", tags=["settings"])
rules_router = APIRouter(prefix="/api/mapping-rules", tags=["mapping"])
@ -91,7 +91,7 @@ def get_reserves(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict
@router.put("/{session_id}/reserves")
def put_reserves(session_id: int, items: list[ReserveIn],
db: OrmSession = Depends(db_dep)) -> list[dict]:
ensure_editable(get_session_or_404(session_id, db))
get_session_or_404(session_id, db)
db.query(models.Reserve).filter(models.Reserve.session_id == session_id).delete()
for it in items:
db.add(models.Reserve(session_id=session_id, marketplace=it.marketplace,
@ -110,7 +110,6 @@ def get_fx(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict]:
@router.put("/{session_id}/fx")
def put_fx(session_id: int, items: list[FxIn], db: OrmSession = Depends(db_dep)) -> list[dict]:
s = get_session_or_404(session_id, db)
ensure_editable(s)
# Upsert per marketplace — NOT delete-all-then-insert. This used to wipe every rate not
# named in the body, so a partial PUT silently removed the other marketplaces' rates and
# the close fell back to the hardcoded Jan-26 defaults without a word.

103
ar-aging-app/backend/app/config.py
View File

@ -1,9 +1,4 @@
"""Application configuration (env-overridable).
Data egress: none, with ONE deliberate exception — the exchange-rate fetch
(services/fx_service.py) calls the configured FX provider (Frankfurter by default) with
currency codes and dates only. No financial figures, filenames, or transaction data ever
leave the server."""
"""Application configuration (env-overridable). No third-party data egress."""
from __future__ import annotations
import os
@ -22,15 +17,7 @@ DATA_DIR = Path(os.environ.get("AR_DATA_DIR", str(BASE_DIR / "data")))
UPLOAD_DIR = DATA_DIR / "uploads"
EXPORT_DIR = DATA_DIR / "exports"
# --------------------------------------------------------------------------- database
# The store is switchable so the app is never blocked on infrastructure:
# AR_DB_BACKEND=sqlite a single local file — zero setup, good for a laptop or a demo
# AR_DB_BACKEND=mysql the shared server, for real multi-user month-end work
# Unset, it picks MySQL when a real MYSQL_HOST is configured and SQLite otherwise, so a
# machine with no database installed still runs instead of failing at import.
_PLACEHOLDER_HOSTS = {"", "your-mysql-host.example.com", "changeme", "todo"}
SQLITE_PATH = Path(os.environ.get("AR_SQLITE_PATH", str(DATA_DIR / "ar_aging.db")))
# MySQL (required)
MYSQL_HOST = os.environ.get("MYSQL_HOST", "")
MYSQL_PORT = int(os.environ.get("MYSQL_PORT", "3306"))
MYSQL_USER = os.environ.get("MYSQL_USER", "")
@ -41,23 +28,11 @@ MYSQL_POOL_SIZE = int(os.environ.get("MYSQL_POOL_SIZE", "10"))
MYSQL_POOL_RECYCLE = int(os.environ.get("MYSQL_POOL_RECYCLE", "3600"))
def _mysql_configured() -> bool:
return (MYSQL_HOST.strip().lower() not in _PLACEHOLDER_HOSTS
and bool(MYSQL_USER) and bool(MYSQL_DATABASE))
DB_BACKEND = (os.environ.get("AR_DB_BACKEND")
or ("mysql" if _mysql_configured() else "sqlite")).strip().lower()
if DB_BACKEND not in ("sqlite", "mysql"):
raise RuntimeError(f"AR_DB_BACKEND must be 'sqlite' or 'mysql' (got {DB_BACKEND!r}).")
def mysql_url() -> str:
if not _mysql_configured():
if not all((MYSQL_HOST, MYSQL_USER, MYSQL_DATABASE)):
raise RuntimeError(
"MYSQL_HOST, MYSQL_USER, and MYSQL_DATABASE are required for the mysql "
"backend. Copy .env.example to .env and fill in real credentials, or set "
"AR_DB_BACKEND=sqlite to use a local file."
"MYSQL_HOST, MYSQL_USER, and MYSQL_DATABASE are required. "
"Copy example.env to .env and fill in credentials."
)
user = quote_plus(MYSQL_USER)
password = quote_plus(MYSQL_PASSWORD)
@ -68,20 +43,6 @@ def mysql_url() -> str:
)
def database_url() -> str:
if DB_BACKEND == "mysql":
return mysql_url()
SQLITE_PATH.parent.mkdir(parents=True, exist_ok=True)
return f"sqlite:///{SQLITE_PATH}"
def database_label() -> str:
"""Human-readable target, for logs and the launcher — never includes the password."""
if DB_BACKEND == "mysql":
return f"MySQL {MYSQL_USER}@{MYSQL_HOST}:{MYSQL_PORT}/{MYSQL_DATABASE}"
return f"SQLite {SQLITE_PATH}"
# Retention: temp uploads/exports older than this are purged (0 = keep forever).
RETENTION_DAYS = int(os.environ.get("AR_RETENTION_DAYS", "30"))
@ -97,60 +58,6 @@ CORS_ORIGINS = os.environ.get(
"AR_CORS_ORIGINS", "http://localhost:5173,http://127.0.0.1:5173"
).split(",")
# --------------------------------------------------------------------------- auth
# AR_AUTH: on | off | auto (default).
# auto — login is required as soon as at least one user exists (create users with
# `python manage.py add-user`), and the API is open while there are none.
# A fresh dev checkout and the test suite therefore run without ceremony,
# while creating the first real user turns authentication on by itself.
# on — login is always required (production; set it in .env.production).
# off — never required (explicit opt-out; never use on a reachable server).
AUTH_MODE = os.environ.get("AR_AUTH", "auto").strip().lower()
if AUTH_MODE not in ("on", "off", "auto"):
raise RuntimeError(f"AR_AUTH must be 'on', 'off' or 'auto' (got {AUTH_MODE!r}).")
# Signs login tokens. REQUIRED in production — without it a random ephemeral key is used
# and every restart logs everyone out (fine for a laptop, wrong for a server).
SECRET_KEY = os.environ.get("AR_SECRET_KEY", "")
# Token lifetime (hours).
AUTH_TOKEN_HOURS = int(os.environ.get("AR_AUTH_TOKEN_HOURS", "12"))
# --------------------------------------------------------------------------- email
# Used ONLY for password codes ("email me a code" on the login/Settings screens).
# Unset -> the email-code flow is hidden and passwords change via the current-password
# form (or manage.py set-password by the admin).
#
# Preferred transport: the company's internal Mail API (the same service the TikTok
# dashboard uses for its verification codes) — a bearer-token multipart POST.
MAIL_API_URL = os.environ.get("AR_MAIL_API_URL", "")
MAIL_API_TOKEN = os.environ.get("AR_MAIL_API_TOKEN", "")
# Fallback transport: any standard SMTP account
# (Office365: smtp.office365.com:587, Gmail: smtp.gmail.com:587 with an app password).
SMTP_HOST = os.environ.get("AR_SMTP_HOST", "")
SMTP_PORT = int(os.environ.get("AR_SMTP_PORT", "587"))
SMTP_USER = os.environ.get("AR_SMTP_USER", "")
SMTP_PASSWORD = os.environ.get("AR_SMTP_PASSWORD", "")
SMTP_FROM = os.environ.get("AR_SMTP_FROM", SMTP_USER)
SMTP_STARTTLS = os.environ.get("AR_SMTP_STARTTLS", "true").strip().lower() != "false"
def email_enabled() -> bool:
return bool(MAIL_API_URL) or bool(SMTP_HOST and SMTP_FROM)
# --------------------------------------------------------------------------- FX provider
# frankfurter (default; free, keyless, central-bank rates) | exchangerate-api (paid, needs
# FX_API_KEY). Rates fetched are suggestions: Control C5 still requires a human to confirm
# them for the reporting month before the close can publish.
FX_PROVIDER = os.environ.get("AR_FX_PROVIDER", "frankfurter").strip().lower()
FX_API_KEY = os.environ.get("AR_FX_API_KEY", "")
FX_TIMEOUT_S = float(os.environ.get("AR_FX_TIMEOUT_S", "15"))
# Processing auto-fetches the provider's DAILY rates over the closing's transaction span,
# so dated movements convert at the rate effective on their own transaction date. Set to 0
# to disable the automatic fetch (the AR Ledger's "Fetch daily rates" button still works).
FX_AUTO_DAILY = os.environ.get("AR_FX_AUTO_DAILY", "1").strip().lower() not in ("0", "false", "no")
def ensure_dirs() -> None:
for d in (DATA_DIR, UPLOAD_DIR, EXPORT_DIR):

301
ar-aging-app/backend/app/core/bank_import.py
View File

@ -1,301 +0,0 @@
"""
Bank disbursements import: parse the finance team's bank-deposit workbook and propose
bank receipts for the session's Amazon payouts.
The workbook (one row per bank credit) looks like:
Company Link | Type | B. Acc | FCY | Date | Month | Text | Debit | Credit | Net | Party Name
`Party Name` identifies the marketplace ("Amazon US", "Amazon Germany", ...), `Date` is
the day the money reached the bank, `Debit` the amount credited in the bank account's
currency. Matching is deliberately conservative: a bank row is only auto-matched when it
points at exactly ONE payout; anything else is surfaced as ambiguous/unmatched for a human.
Currency wrinkle: some deposits arrive converted (Australia payouts land as USD), so the
amount check only runs when the row's FCY equals the marketplace's currency — otherwise
the match is date-only and flagged (`amount_checked: False`).
Pure module: no ORM, no FastAPI — unit-testable with plain lists/dicts.
"""
from __future__ import annotations
import datetime as dt
from dataclasses import dataclass, field
from io import BytesIO
from typing import Any
# Bank narrative party -> engine marketplace label (regions.py). Casefolded lookup keys
# absorb the inconsistent casing seen in real files ("Amazon sweden").
PARTY_TO_MARKETPLACE: dict[str, str] = {
"amazon us": "USA", "amazon usa": "USA",
"amazon uk": "UK",
"amazon canada": "Canada",
"amazon australia": "Australia",
"amazon germany": "Germany",
"amazon france": "France",
"amazon italy": "Italy",
"amazon spain": "Spain",
"amazon netherlands": "Netherlands",
"amazon belgium": "Belgium",
"amazon ireland": "Ireland",
"amazon poland": "Poland",
"amazon sweden": "Sweden",
"amazon turkey": "Turkey",
}
_REQUIRED_HEADERS = ("date", "debit", "party name")
_EXCEL_EPOCH = dt.date(1899, 12, 30)
class BankImportError(ValueError):
"""The uploaded workbook is not a recognizable disbursements file."""
@dataclass
class BankRow:
sheet_row: int # 1-based row in the sheet, for human cross-reference
party: str
marketplace: str | None # None = unknown party
currency: str # FCY column, uppercased ("" if absent)
bank_date: dt.date
narrative: str
debit: float
credit: float
net: float
def _as_date(value: Any) -> dt.date | None:
if isinstance(value, dt.datetime):
return value.date()
if isinstance(value, dt.date):
return value
if isinstance(value, (int, float)) and value > 0: # raw Excel serial
return _EXCEL_EPOCH + dt.timedelta(days=float(value))
if isinstance(value, str):
try:
return dt.date.fromisoformat(value.strip()[:10])
except ValueError:
return None
return None
def _as_float(value: Any) -> float:
if isinstance(value, (int, float)):
return float(value)
if isinstance(value, str):
try:
return float(value.replace(",", "").strip() or 0.0)
except ValueError:
return 0.0
return 0.0
def parse_disbursements(data: bytes) -> tuple[list[BankRow], list[str]]:
"""All Deposit rows of the workbook's Payouts sheet, plus per-row problems."""
from python_calamine import CalamineWorkbook
try:
wb = CalamineWorkbook.from_filelike(BytesIO(data))
except Exception as e: # noqa: BLE001 — calamine raises library-specific errors
raise BankImportError(f"Could not open the workbook: {e}") from e
# The sheet named "Payouts" (any casing), else the first sheet with the right headers.
sheet = None
for name in wb.sheet_names:
if name.strip().casefold() == "payouts":
sheet = name
break
if sheet is None:
for name in wb.sheet_names:
head = wb.get_sheet_by_name(name).to_python(nrows=1)
labels = {str(c).strip().casefold() for c in (head[0] if head else [])}
if all(h in labels for h in _REQUIRED_HEADERS):
sheet = name
break
if sheet is None:
raise BankImportError(
"No disbursements sheet found — expected a sheet named 'Payouts' (or one whose "
"first row has 'Date', 'Debit' and 'Party Name' columns).")
grid = wb.get_sheet_by_name(sheet).to_python()
if not grid:
raise BankImportError(f"Sheet '{sheet}' is empty.")
header = [str(c).strip().casefold() for c in grid[0]]
col = {label: i for i, label in enumerate(header)}
missing = [h for h in _REQUIRED_HEADERS if h not in col]
if missing:
raise BankImportError(f"Sheet '{sheet}' is missing columns: {', '.join(missing)}.")
def cell(row: list, label: str) -> Any:
i = col.get(label)
return row[i] if i is not None and i < len(row) else None
rows: list[BankRow] = []
problems: list[str] = []
for idx, raw in enumerate(grid[1:], start=2):
party = str(cell(raw, "party name") or "").strip()
row_type = str(cell(raw, "type") or "").strip()
if not party and not any(str(c).strip() for c in raw):
continue # blank row
if row_type and row_type.casefold() != "deposit":
continue # only bank credits are receipts
bank_date = _as_date(cell(raw, "date"))
if bank_date is None:
problems.append(f"row {idx}: unreadable Date {cell(raw, 'date')!r} — skipped")
continue
if not party:
problems.append(f"row {idx}: empty Party Name — skipped")
continue
debit = _as_float(cell(raw, "debit"))
rows.append(BankRow(
sheet_row=idx,
party=party,
marketplace=PARTY_TO_MARKETPLACE.get(party.casefold()),
currency=str(cell(raw, "fcy") or "").strip().upper(),
bank_date=bank_date,
narrative=str(cell(raw, "text") or "").strip(),
debit=debit if debit else _as_float(cell(raw, "net")),
credit=_as_float(cell(raw, "credit")),
net=_as_float(cell(raw, "net")),
))
return rows, problems
@dataclass
class MatchResult:
matched: list[dict] = field(default_factory=list)
ambiguous: list[dict] = field(default_factory=list)
unmatched: list[dict] = field(default_factory=list)
unknown_party: list[dict] = field(default_factory=list)
out_of_scope: int = 0
problems: list[str] = field(default_factory=list)
def _amount_tolerance(amount: float) -> float:
# Small bank fees / rounding: 0.5% capped from below at 5 cents.
return max(0.05, 0.005 * abs(amount))
def match_payouts(
rows: list[BankRow],
payouts: list[dict],
month_end: dt.date | None,
window_days: int = 14,
receipts: dict[tuple[str, str, str], dt.date] | None = None,
currency_by_marketplace: dict[str, str] | None = None,
) -> MatchResult:
"""
payouts: [{marketplace, account_type, settlement_id, amazon_date: date|None, amount}]
receipts: existing PayoutReceipt bank dates keyed (marketplace, account_type, settlement_id).
Matching is one-to-one: bank rows are processed in (bank_date, sheet_row) order and a
payout consumed by an earlier row is no longer available to later ones.
"""
receipts = receipts or {}
currencies = currency_by_marketplace or {}
result = MatchResult()
by_marketplace: dict[str, list[dict]] = {}
dated = []
for p in payouts:
by_marketplace.setdefault(p["marketplace"], []).append(p)
if p.get("amazon_date"):
dated.append(p["amazon_date"])
scope_start = (min(dated) - dt.timedelta(days=3)) if dated else None
scope_end = (month_end + dt.timedelta(days=window_days)) if month_end else None
consumed: dict[tuple[str, str, str], int] = {} # payout key -> bank sheet_row that took it
def key(p: dict) -> tuple[str, str, str]:
return (p["marketplace"], p["account_type"], p["settlement_id"])
for row in sorted(rows, key=lambda r: (r.bank_date, r.sheet_row)):
if row.marketplace is None:
result.unknown_party.append({
"bank_row": row.sheet_row, "party": row.party,
"bank_date": row.bank_date.isoformat(), "amount": row.debit,
})
continue
in_scope = (scope_start is None or scope_end is None
or scope_start <= row.bank_date <= scope_end)
candidates = []
taken = [] # would match, but already consumed
for p in by_marketplace.get(row.marketplace, []):
d = p.get("amazon_date")
if d is None or not (d <= row.bank_date <= d + dt.timedelta(days=window_days)):
continue
(taken if key(p) in consumed else candidates).append(p)
amount_checked = bool(row.currency) and currencies.get(row.marketplace) == row.currency
if amount_checked:
confirmed = [p for p in candidates
if abs(row.debit - abs(p["amount"])) <= _amount_tolerance(p["amount"])]
else:
confirmed = []
chosen = None
if len(confirmed) == 1:
chosen = confirmed[0]
elif len(confirmed) > 1:
pass # genuinely ambiguous on amount
elif len(candidates) == 1:
chosen = candidates[0] # date-only (fee variance or FX-converted)
def _cand(p: dict) -> dict:
return {"settlement_id": p["settlement_id"], "account_type": p["account_type"],
"amazon_date": p["amazon_date"].isoformat() if p.get("amazon_date") else None,
"amount": p["amount"]}
if chosen is not None:
k = key(chosen)
consumed[k] = row.sheet_row
existing = receipts.get(k)
delta = (round(abs(row.debit - abs(chosen["amount"])), 2)
if amount_checked else None)
result.matched.append({
"marketplace": chosen["marketplace"],
"account_type": chosen["account_type"],
"settlement_id": chosen["settlement_id"],
"amazon_date": chosen["amazon_date"].isoformat() if chosen.get("amazon_date") else None,
"amazon_amount": chosen["amount"],
"bank_date": row.bank_date.isoformat(),
"bank_amount": row.debit,
"currency": row.currency,
"amount_checked": amount_checked,
"delta": delta,
"bank_row": row.sheet_row,
"already_had_receipt": existing is not None,
"existing_bank_date": existing.isoformat() if existing else None,
"note": f"Imported from bank file row {row.sheet_row}"
+ ("" if amount_checked else f" ({row.currency} {row.debit:,.2f})"),
})
continue
pool = confirmed or candidates
if pool:
result.ambiguous.append({
"bank_row": row.sheet_row, "party": row.party,
"marketplace": row.marketplace,
"bank_date": row.bank_date.isoformat(), "amount": row.debit,
"reason": f"{len(pool)} payouts match within {window_days} days",
"candidates": [_cand(p) for p in pool],
})
elif taken:
result.ambiguous.append({
"bank_row": row.sheet_row, "party": row.party,
"marketplace": row.marketplace,
"bank_date": row.bank_date.isoformat(), "amount": row.debit,
"reason": f"payout already matched by row {consumed[key(taken[0])]}",
"candidates": [_cand(p) for p in taken],
})
elif not in_scope:
result.out_of_scope += 1
else:
result.unmatched.append({
"bank_row": row.sheet_row, "party": row.party,
"marketplace": row.marketplace,
"bank_date": row.bank_date.isoformat(), "amount": row.debit,
"reason": f"no {row.marketplace} payout within {window_days} days before this date",
})
return result

18
ar-aging-app/backend/app/core/calamine_reader.py
View File

@ -32,7 +32,6 @@ class CalamineReader:
self.header_row = 0 # 1-based (Excel)
self.column_mapping: ColumnMapping | None = None
self._field_to_idx: dict[str, int] = {}
self._sum_field_idx: dict[str, list[int]] = {}
self.file_meta = FileMeta(filename=self.filename)
# -- lifecycle --
@ -99,17 +98,11 @@ class CalamineReader:
self._field_to_idx = {
fld: _letter_to_idx(col) for col, fld in mapping.col_to_field.items()
}
# Extra amount columns folded into an already-mapped field (ColumnMapping.sum_cols).
self._sum_field_idx = {
fld: [_letter_to_idx(col) for col, _hdr in cols]
for fld, cols in mapping.sum_cols.items()
}
self.file_meta.data_sheet = name
self.file_meta.header_row = self.header_row
self.file_meta.unmapped_headers = mapping.unmapped
self.file_meta.missing_required = mapping.missing_required
self.file_meta.duplicate_fields = mapping.duplicate_fields
self.file_meta.summed_fields = mapping.sum_cols
self.file_meta.sheet_last_row = self._safe_height(name) # control C1
return mapping
@ -133,15 +126,10 @@ class CalamineReader:
self.detect()
assert self._sheet is not None
idx_map = self._field_to_idx
sum_map = self._sum_field_idx
if only_fields is not None:
idx_map = {f: i for f, i in idx_map.items() if f in only_fields}
sum_map = {f: v for f, v in sum_map.items() if f in only_fields}
items = list(idx_map.items())
sum_items = list(sum_map.items())
mapped_idx = set(self._field_to_idx.values())
for _idxs in self._sum_field_idx.values():
mapped_idx.update(_idxs)
unmapped_sums = self.file_meta.unmapped_amount_sums
hdr = self.header_row # 1-based; data starts at hdr+1 (Excel) => row index hdr (0-based)
min_d: date | None = None
@ -165,12 +153,6 @@ class CalamineReader:
# this reader emitted trailing blank rows the other reader dropped.
if v not in (None, ""):
has_value = True
for fld, idxs in sum_items:
for i in idxs:
v = row[i] if i < len(row) else None
if v not in (None, ""):
rec[fld] = (rec.get(fld) or 0.0) + _conv_cal(fld, v)
has_value = True
if not has_value:
self.file_meta.blank_rows_skipped += 1
continue

67
ar-aging-app/backend/app/core/column_map.py
View File

@ -100,14 +100,12 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
("account_type", "text", ("account type", "accounttype")),
("fulfillment", "text", (
"fulfillment", "fulfilment", "fulfillment channel",
"shipping/fulfillment", "fulfillment/shipping", # ES / TR English-form
"expédition", "traitement", # FR / BE
"versand", # DE
"gestione", # IT
"gestión logística", # ES
"realizacja", # PL
"leverans", # SV
"gönderim", # TR
)),
("order_city", "text", (
"order city", "city",
@ -118,13 +116,9 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"bestelling stad",
"miejscowość zamówienia",
"stad för beställning",
"sipariş şehri", # TR
)),
("order_state", "text", (
"order state", "state",
"order state/province", "state/province", # IE/NL/PL/SV/BE, DE
"order region/province", "order province/state", # FR, IT
"order region/autonomous community", # ES
"état de la commande", "région d'où provient la commande",
"bundesland",
"provincia di provenienza dell'ordine",
@ -132,11 +126,9 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"status bestelling",
"stan zamówienia",
"delstat för beställning",
"sipariş durumu", # TR (order-state column, per reference workbook)
)),
("order_postal", "text", (
"order postal", "postal", "postal code", "zip",
"order postal code", # FR/IE/IT/NL/PL/ES/SV/BE/TR English-form
"commande postale", "code postal de la commande",
"postleitzahl",
"cap dell'ordine",
@ -144,7 +136,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"bestelling per post",
"przekaz pocztowy",
"postadress för beställning",
"sipariş postası", # TR
)),
("tax_collection_model", "text", (
"tax collection model", "tax collection responsible party",
@ -155,7 +146,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
)),
("product_sales", "amount", (
"product sales", "sales",
"ürün satışları", # TR
"ventes de produits", # FR / BE
"umsätze", # DE
"vendite", # IT
@ -173,8 +163,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
)),
("shipping_credits", "amount", (
"shipping credits", "shipping", "postage credits",
"shipping credit", # DE (singular English-form)
"kargo kredileri", # TR
"crédits d'expédition", "crédits d’expédition",
"gutschrift für versandkosten",
"accrediti per le spedizioni",
@ -185,8 +173,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
)),
("shipping_credits_tax", "amount", (
"shipping credits tax",
"tax on shipping credits", # FR / IT / ES English-form
"tax on shipping credit", # DE (singular English-form)
"taxe sur les crédits d'expédition", "taxe sur les crédits d’expédition",
"steuer auf versandgutschrift",
"imposta accrediti per le spedizioni",
@ -194,7 +180,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
)),
("gift_wrap_credits", "amount", (
"gift wrap credits", "gift wrap", "giftwrap credits",
"gift wrap credit", # DE (singular English-form)
"crédits d'emballage-cadeau", "crédits d’emballage-cadeau",
"crédits sur l'emballage cadeau",
"gutschrift für geschenkverpackung",
@ -206,8 +191,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
)),
("giftwrap_credits_tax", "amount", (
"giftwrap credits tax", "gift wrap credits tax",
"tax on gift wrap credits", # FR / IT / ES English-form
"tax on gift wrap credit", # DE (singular English-form)
"taxes sur les crédits cadeaux",
"steuer auf geschenkverpackungsgutschriften",
"imposta sui crediti confezione regalo",
@ -217,8 +200,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
("tax_on_regulatory_fee", "amount", ("tax on regulatory fee",)),
("promotional_rebates", "amount", (
"promotional rebates", "promotional rebate",
"promotional discounts", # FR/DE/IE/IT/NL/PL/ES/SV/TR English-form
"total discounts", # BE
"rabais promotionnels", "total des réductions",
"rabatte aus werbeaktionen",
"sconti promozionali",
@ -226,11 +207,9 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"promotiekortingen",
"rabaty promocyjne",
"kampanjrabatter",
"promosyon indirimleri", # TR
)),
("promotional_rebates_tax", "amount", (
"promotional rebates tax", "promotional rebate tax",
"tax on promotional discounts", # FR / DE / IT / ES English-form
"taxes sur les remises promotionnelles",
"steuer auf aktionsrabatte",
"imposta sugli sconti promozionali",
@ -246,7 +225,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
)),
("marketplace_withheld_tax", "amount", (
"marketplace withheld tax",
"marketplace withheld vat", # IT English-form
"marketplace facilitator tax", "marketplace facilitator tax",
"taxe marketplace facilitator", # BE (FR)
"taxes retenues sur le site de vente", # FR
@ -266,7 +244,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"verkoopkosten", # NL
"opłaty za sprzedaż", # PL
"försäljningsavgifter", # SV
"satış ücretleri", # TR
)),
("fba_fees", "amount", (
"fba fees", "fba fee", "fulfillment fees",
@ -278,7 +255,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"fba-vergoedingen",
"opłaty za fba",
"fba-avgifter",
"amazon lojistik ücretleri", # TR
)),
("other_transaction_fees", "amount", (
"other transaction fees", "other transaction fee",
@ -289,7 +265,6 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"overige transactiekosten",
"inne opłaty transakcyjne",
"övriga transaktionsavgifter",
"diğer işlem ücretleri", # TR
)),
("other", "amount", (
"other",
@ -300,31 +275,9 @@ CANONICAL_FIELDS: list[tuple[str, str, tuple[str, ...]]] = [
"overige", # NL
"inne", # PL
"övrigt", # SV
"diğer", # TR
)),
("transaction_status", "text", (
"transaction status",
"statut de la transaction", # FR / BE CSV
"transaktionsstatus", # DE
"stato della transazione", # IT
"estado de la transacción", # ES
"transactiestatus", # NL
"status transakcji", # PL
"İşlem durumu", # TR
)),
("transaction_release_date", "date", (
"transaction release date",
"date de délivrance de la transaction", # FR / BE CSV
"date de sortie de la transaction", # FR
"transaktionsfreigabedatum", # DE
"freigabedatum der transaktion", # DE
"data di rilascio della transazione", # IT
"fecha de liberación de la transacción", # ES
"publicatiedatum van transactie", # NL
"data zrealizowania transakcji", # PL
"transaktionens utgivningsdatum", # SV
"İşlem çıkış tarihi", # TR
)),
("transaction_status", "text", ("transaction status",)),
("transaction_release_date", "date", ("transaction release date",)),
("total", "amount", (
"total", "total amount", "amount",
"gesamt", # DE
@ -383,11 +336,6 @@ class ColumnMapping:
# Only the first is used, so the second column's amounts would vanish from the journal.
# Surfaced as an error rather than silently demoted to `unmapped`.
duplicate_fields: dict[str, list[tuple[str, str]]] = field(default_factory=dict)
# Extra AMOUNT columns folded into an already-mapped field: field -> [(col, header)].
# Amazon splits one concept across columns in some schemas (AU: "sales tax collected"
# + "low value goods", both inside the row `total`), so readers SUM these instead of
# dropping them.
sum_cols: dict[str, list[tuple[str, str]]] = field(default_factory=dict)
header_row: int = 0
@property
@ -421,15 +369,8 @@ def build_mapping(
m.col_to_field[col] = fld
m.field_to_col[fld] = col
elif fld:
if FIELD_KIND.get(fld) == "amount":
# A second amount column for the same concept (AU "low value goods" next
# to "sales tax collected") — readers ADD it into the field, because the
# row `total` includes both and dropping it fails control C2.
m.sum_cols.setdefault(fld, []).append((col, str(text)))
else:
# Collision on a non-amount field: first column wins and this one is
# dropped. Record both so the close can raise, instead of quietly
# excluding a whole column.
# Collision: first column wins and this one is dropped. Record both so the close
# can raise, instead of quietly excluding a whole amount column.
first_col = m.field_to_col[fld]
m.duplicate_fields.setdefault(fld, [(first_col, "")]).append((col, str(text)))
m.unmapped[col] = str(text)

285
ar-aging-app/backend/app/core/csv_reader.py
View File

@ -1,285 +0,0 @@
"""
Amazon Custom Unified Transaction reports delivered as CSV (UTF-8, often with BOM).
Same public interface as TransactionReader / CalamineReader:
detect() / iter_records() / file_meta / column_mapping / close()
Amazon CSVs typically begin with a short preamble (scope, currency, definitions) before the
real header row. Amounts use a European decimal comma in localized EU reports ("13,49").
"""
from __future__ import annotations
import csv
import os
import re
from datetime import date
from typing import Iterator
from .column_map import FIELD_KIND, ColumnMapping, build_mapping
from .dates import parse_amazon_date_fast
from .xlsx_reader import FileMeta, ParseError
_DETECT_REQUIRED = {"settlement_id", "total", "date_time"}
_CURRENCY_RE = re.compile(
r"\b(USD|EUR|GBP|CAD|AUD|PLN|SEK|TRY|JPY)\b", re.IGNORECASE
)
# Amazon's localized reports are not all ASCII: Sweden writes negatives with a real
# MINUS SIGN (U+2212, "\u221278 690,40") and several locales group thousands with
# non-breaking / narrow spaces. float() rejects U+2212, and the fallback below would
# silently turn the cell into 0.0 \u2014 which dropped every negative amount (fees, taxes,
# transfers) of an entire Swedish month while the positives kept adding up.
_AMOUNT_CLEANUP = str.maketrans({
"\u2212": "-", "\u2010": "-", "\u2011": "-", "\u2013": "-", # minus / dash variants
"\u00a0": None, "\u202f": None, "\u2009": None, " ": None, # space variants
})
def parse_amount(raw) -> float:
"""Parse Amazon amount cells, including European '1.234,56' / '13,49' forms."""
if raw is None or raw == "":
return 0.0
if isinstance(raw, bool):
return 0.0
if isinstance(raw, (int, float)):
return float(raw)
s = str(raw).strip().translate(_AMOUNT_CLEANUP)
if not s:
return 0.0
# European: decimal comma, optional thousands dots / spaces.
if "," in s and "." in s:
if s.rfind(",") > s.rfind("."):
s = s.replace(".", "").replace(",", ".")
else:
s = s.replace(",", "")
elif "," in s:
# "13,49" or "1.234" — if one comma and digits after, treat as decimal.
left, _, right = s.partition(",")
if right.isdigit() and 1 <= len(right) <= 2:
s = f"{left.replace('.', '')}.{right}"
else:
s = s.replace(",", "")
try:
return float(s)
except ValueError:
return 0.0
def _index_to_col(idx: int) -> str:
idx += 1
s = ""
while idx:
idx, r = divmod(idx - 1, 26)
s = chr(65 + r) + s
return s
def _convert(field_name: str, raw):
kind = FIELD_KIND.get(field_name, "text")
if kind == "amount":
return parse_amount(raw)
if kind == "int":
if raw in (None, ""):
return None
try:
return int(parse_amount(raw))
except (TypeError, ValueError):
return None
if raw is None:
return None
s = str(raw).strip()
if kind == "id" and s.endswith(".0"):
s = s[:-2]
return s or None
class CsvReader:
def __init__(self, path: str, saved_overrides: dict[str, str] | None = None):
self.path = path
self.filename = os.path.basename(path)
self.saved_overrides = saved_overrides
self._rows: list[list[str]] | None = None
self.sheet_name = "CSV"
self.header_row = 0 # 1-based, matching Excel readers
self.column_mapping: ColumnMapping | None = None
self._field_to_idx: dict[str, int] = {}
self._sum_field_idx: dict[str, list[int]] = {}
self.file_meta = FileMeta(filename=self.filename)
def open(self) -> None:
if self._rows is not None:
return
try:
raw = open(self.path, "rb").read()
except OSError as e:
raise ParseError(f"'{self.filename}' could not be read: {e}") from e
if not raw:
raise ParseError(f"'{self.filename}' is empty.")
# Strip UTF-8 BOM; fall back through common Amazon encodings.
if raw.startswith(b"\xef\xbb\xbf"):
text = raw.decode("utf-8-sig")
else:
text = None
for enc in ("utf-8", "utf-16", "cp1252", "latin-1"):
try:
text = raw.decode(enc)
break
except UnicodeDecodeError:
continue
if text is None:
raise ParseError(f"'{self.filename}' is not a readable text/CSV file.")
# Sniff delimiter from the densest early line (comma vs semicolon EU exports).
sample = "\n".join(text.splitlines()[:40])
try:
dialect = csv.Sniffer().sniff(sample, delimiters=",;\t")
delimiter = dialect.delimiter
except csv.Error:
delimiter = ";" if sample.count(";") > sample.count(",") else ","
self._rows = list(csv.reader(text.splitlines(), delimiter=delimiter))
self.file_meta.size_bytes = os.path.getsize(self.path)
self.file_meta.worksheets = [self.sheet_name]
# Currency hint from the preamble ("Tous les montants sont en EUR…").
for row in self._rows[:15]:
joined = " ".join(row)
m = _CURRENCY_RE.search(joined)
if m and ("montant" in joined.lower() or "amount" in joined.lower()
or "currency" in joined.lower() or "en " in joined.lower()):
self.file_meta.currency = m.group(1).upper()
break
def close(self) -> None:
self._rows = None
def __enter__(self):
self.open()
return self
def __exit__(self, *exc):
self.close()
def detect(self) -> ColumnMapping:
self.open()
assert self._rows is not None
best = None
best_score = -1
# Scan the first ~40 rows for the real Amazon header (skip preamble / definitions).
for r_idx, row in enumerate(self._rows[:40]):
cells = [(_index_to_col(i), str(v)) for i, v in enumerate(row) if str(v).strip()]
if len(cells) < 5:
continue
mapping = build_mapping(cells, r_idx + 1, self.saved_overrides)
if _DETECT_REQUIRED.issubset(set(mapping.field_to_col)):
score = len(mapping.field_to_col)
if score > best_score:
best, best_score = (r_idx, mapping), score
if not best:
raise ParseError(
f"'{self.filename}': could not find an Amazon transaction header row "
f"(need columns: date/time, settlement id, total)."
)
r_idx, mapping = best
self.header_row = r_idx + 1
self.column_mapping = mapping
self._field_to_idx = {
fld: _col_to_idx(col) for col, fld in mapping.col_to_field.items()
}
# Extra amount columns folded into an already-mapped field (ColumnMapping.sum_cols).
self._sum_field_idx = {
fld: [_col_to_idx(col) for col, _hdr in cols]
for fld, cols in mapping.sum_cols.items()
}
self.file_meta.data_sheet = self.sheet_name
self.file_meta.header_row = self.header_row
self.file_meta.unmapped_headers = mapping.unmapped
self.file_meta.missing_required = mapping.missing_required
self.file_meta.duplicate_fields = mapping.duplicate_fields
self.file_meta.summed_fields = mapping.sum_cols
self.file_meta.sheet_last_row = len(self._rows)
return mapping
def iter_records(self, only_fields: set[str] | None = None) -> Iterator[dict]:
if self.column_mapping is None:
self.detect()
assert self._rows is not None
idx_map = self._field_to_idx
sum_map = self._sum_field_idx
if only_fields is not None:
idx_map = {f: i for f, i in idx_map.items() if f in only_fields}
sum_map = {f: v for f, v in sum_map.items() if f in only_fields}
items = list(idx_map.items())
sum_items = list(sum_map.items())
mapped_idx = set(self._field_to_idx.values())
for _idxs in self._sum_field_idx.values():
mapped_idx.update(_idxs)
unmapped_sums = self.file_meta.unmapped_amount_sums
hdr = self.header_row
min_d: date | None = None
max_d: date | None = None
marketplace: str | None = None
count = 0
want_date = "date_time" in self._field_to_idx
for excel_row, row in enumerate(self._rows, start=1):
if excel_row <= hdr:
continue
rec = {
"_source_file": self.filename,
"_source_sheet": self.sheet_name,
"_source_row": excel_row,
}
has_value = False
for fld, i in items:
v = row[i] if i < len(row) else None
rec[fld] = _convert(fld, v)
if v not in (None, ""):
has_value = True
for fld, idxs in sum_items:
for i in idxs:
v = row[i] if i < len(row) else None
if v not in (None, ""):
rec[fld] = (rec.get(fld) or 0.0) + parse_amount(v)
has_value = True
if not has_value:
self.file_meta.blank_rows_skipped += 1
continue
if excel_row <= hdr + 2:
raw_sid = row[self._field_to_idx["settlement_id"]] \
if self._field_to_idx.get("settlement_id", 99999) < len(row) else None
sid = str(raw_sid or "").strip()
d_probe = parse_amazon_date_fast(rec.get("date_time")) \
if rec.get("date_time") else None
if d_probe is None and not sid.replace(".", "").isdigit():
self.file_meta.helper_rows_skipped += 1
continue
if len(row) > len(mapped_idx):
for i, v in enumerate(row):
if i not in mapped_idx and v not in (None, ""):
amt = parse_amount(v)
if amt:
col = _index_to_col(i)
unmapped_sums[col] = unmapped_sums.get(col, 0.0) + amt
if want_date and rec.get("date_time"):
d = parse_amazon_date_fast(rec["date_time"])
rec["_date"] = d
if d:
if min_d is None or d < min_d:
min_d = d
if max_d is None or d > max_d:
max_d = d
if marketplace is None and rec.get("marketplace"):
marketplace = rec["marketplace"]
count += 1
yield rec
self.file_meta.imported_rows = count
self.file_meta.min_date = min_d
self.file_meta.max_date = max_d
self.file_meta.marketplace = marketplace
def _col_to_idx(letters: str) -> int:
n = 0
for ch in letters:
n = n * 26 + (ord(ch) - 64)
return n - 1

166
ar-aging-app/backend/app/core/excel_export.py
View File

@ -107,10 +107,7 @@ class SheetLayout:
data_start: int = 0
data_rows: int = 0
subtotal_cells: dict[str, str] = field(default_factory=dict) # account_type -> "AD####"
# account_type -> ["AD9", "AD10", …]: a month can have SEVERAL received payouts per
# stream, so this is a list. It used to be one cell per account, which silently showed
# only the boundary payout and omitted every earlier one from the workbook.
transfer_cells: dict[str, list[str]] = field(default_factory=dict)
transfer_cells: dict[str, str] = field(default_factory=dict) # account_type -> "AD####"
@property
def data_end(self) -> int:
@ -127,10 +124,11 @@ class MarketplaceLayout:
return [f"'{s.name}'!{s.subtotal_cells[account_type]}"
for s in self.sheets if account_type in s.subtotal_cells]
def transfer_refs(self, account_type: str) -> list[str]:
"""Every received-payout cell for an account stream (a month can have several)."""
return [f"'{s.name}'!{c}"
for s in self.sheets for c in s.transfer_cells.get(account_type, [])]
def transfer_ref(self, account_type: str) -> str | None:
for s in self.sheets:
if account_type in s.transfer_cells:
return f"'{s.name}'!{s.transfer_cells[account_type]}"
return None
def _sheet_names(marketplace: str, n: int) -> list[str]:
@ -145,18 +143,10 @@ def compute_layouts(result: ProcessResult, row_limit: int = EXCEL_ROW_LIMIT) ->
cls = result.classification
assert agg is not None and cls is not None
# Order-row counts and receivable account types per marketplace. Only OPEN (receivable)
# settlements contribute rows: once a settlement's payout has reached the bank it is
# closed, and its raw transactions are deliberately left out of the workbook (they are
# listed in summary form on the "Settled Settlements" sheet instead).
# order-row counts and receivable account types per marketplace
per_mkt_orders: dict[str, int] = {}
per_mkt_accts: dict[str, list[str]] = {}
all_mkt_accts: dict[str, list[str]] = {}
for (mkt, acct, sid), st in agg.settlements.items():
if acct.lower() in RECEIVABLE_ACCOUNT_TYPES:
seen = all_mkt_accts.setdefault(mkt, [])
if acct not in seen:
seen.append(acct)
if st.status != "receivable" or acct.lower() not in RECEIVABLE_ACCOUNT_TYPES:
continue
orders = st.row_count - st.transfer_count
@ -165,33 +155,17 @@ def compute_layouts(result: ProcessResult, row_limit: int = EXCEL_ROW_LIMIT) ->
if acct not in accts:
accts.append(acct)
# A marketplace whose payouts were ALL received has nothing outstanding and so no order
# rows — but it still gets a tab. An absent tab is indistinguishable from a marketplace
# whose file failed to upload; an empty one with its receipts and a 0.00 subtotal proves
# the market was processed and legitimately had nothing open.
for mkt in sorted(agg.marketplaces_seen):
per_mkt_orders.setdefault(mkt, 0)
# EVERY received payout per (marketplace, account stream) — not just the boundary one.
# Using cls.boundary_transfer here showed a single payout per stream, so a month with
# several bank receipts silently omitted all but the last from the workbook.
received_tx: dict[tuple[str, str], list] = {}
for t in agg.transfers:
if not t.received:
continue # in transit: its settlement is still open
owner = cls.settlement_owner.get(t.settlement_id, t.marketplace)
received_tx.setdefault((owner, t.account_type), []).append(t)
for lst in received_tx.values():
lst.sort(key=lambda t: (t.txn_date or date.min, t.settlement_id))
# boundary (receipt) transfers per marketplace/account
boundary_tx: dict[tuple[str, str], object] = {}
for k, t in cls.boundary_transfer.items():
if t is not None:
boundary_tx[k] = t
layouts: dict[str, MarketplaceLayout] = {}
for mkt, order_total in per_mkt_orders.items():
accts = sorted(per_mkt_accts.get(mkt, []) or all_mkt_accts.get(mkt, []),
accts = sorted(per_mkt_accts.get(mkt, []),
key=lambda a: (0 if a.lower() == "standard orders" else 1, a))
# Payout rows are keyed by the account type Amazon tagged them with, which is blank
# ("(unspecified)") everywhere except the USA — include those streams too.
transfers = [t for (m, _a), lst in received_tx.items() if m == mkt for t in lst]
transfers.sort(key=lambda t: (t.txn_date or date.min, t.settlement_id))
transfers = [boundary_tx[(mkt, a)] for a in accts if (mkt, a) in boundary_tx]
# capacity of the first sheet (accounts for preamble+header+transfers+subtotals)
subtotal_block = 1 + 2 * max(len(accts), 1) # gap + one subtotal row per account
@ -218,20 +192,13 @@ def compute_layouts(result: ProcessResult, row_limit: int = EXCEL_ROW_LIMIT) ->
# rows: preamble 1-7, header 8, transfers 9.., data start after transfers
sl.data_start = 8 + len(tlist) + 1
sl.data_rows = nrows
# transfer cell addresses (rows 9..) — several payouts can share an account stream
# transfer cell addresses (rows 9..)
for j, t in enumerate(tlist):
sl.transfer_cells.setdefault(t.account_type, []).append(f"{TOTAL_COL}{9 + j}")
# Subtotal rows after the data: one blank gap row, then ONE row per account
# stream, consecutively — this must mirror _finalize_marketplace_subtotals()
# exactly, because Detail/Summary reference these planned addresses.
#
# This used to stride by 2 while the writer strides by 1, so every stream after
# the first pointed at an empty cell. USA is the only marketplace with two
# streams, so Detail and Summary silently dropped the whole Invoiced Orders
# receivable (Jan-2026: 67,854.71) while Reconciliation and COA showed it.
sl.transfer_cells[t.account_type] = f"{TOTAL_COL}{9 + j}"
# subtotal rows after data (one blank gap, then one row per account)
base = sl.data_end + 2
for j, acct in enumerate(accts):
sl.subtotal_cells[acct] = f"{TOTAL_COL}{base + j}"
sl.subtotal_cells[acct] = f"{TOTAL_COL}{base + 2 * j}"
ml.sheets.append(sl)
layouts[mkt] = ml
return layouts
@ -260,8 +227,7 @@ class WorkbookBuilder:
saved_column_overrides: dict[str, str] | None = None,
row_limit: int = EXCEL_ROW_LIMIT,
progress: "Callable[[float, int, int], None] | None" = None,
summary: dict | None = None, journal: dict | None = None,
payout_receipts: dict[tuple[str, str, str], str] | None = None):
summary: dict | None = None, journal: dict | None = None):
self.result = result
self.files = list(files)
self.reserves = reserves or {}
@ -271,8 +237,6 @@ class WorkbookBuilder:
self._progress = progress
self.summary = summary or {}
self.journal = journal or {}
# (marketplace, account bucket, settlement id) -> "YYYY-MM-DD · entered by"
self.payout_receipts = payout_receipts or {}
self.layouts = compute_layouts(result, row_limit)
self.wb = Workbook(write_only=True)
self._mkt_ws: dict[str, list] = {} # marketplace -> [ws per sheet]
@ -289,7 +253,6 @@ class WorkbookBuilder:
self._create_marketplace_sheets()
self._stream_marketplace_rows()
self._finalize_marketplace_subtotals()
self._build_settled_settlements()
self._build_reconciliation()
self._build_exceptions()
self._build_audit_trail()
@ -350,14 +313,11 @@ class WorkbookBuilder:
font=BOLD, border=BORDER) for col in "BCDEFG"],
])
ws.append([])
ar = total_row + 2 # the Allowance row, appended next
ar = total_row + 2
ws.append([_c(ws, "Allowance for Sales Returns", font=BOLD),
*[None] * 5, _c(ws, self.allowance, number_format=FMT_USD0, font=BOLD)])
# Net Receivable = TOTAL + Allowance (the allowance is entered negative, as in the
# manual workbook). This referenced G{ar+1} — its own row — so Excel opened the
# workbook with a circular-reference warning and showed 0.
ws.append([_c(ws, "Net Receivable", font=BOLD),
*[None] * 5, _c(ws, f"=G{total_row}+G{ar}", number_format=FMT_USD0, font=BOLD)])
*[None] * 5, _c(ws, f"=G{total_row}+G{ar + 1}", number_format=FMT_USD0, font=BOLD)])
ws.freeze_panes = "A6"
def _detail_receivable_usd_ref(self, mkt: str) -> str:
@ -652,84 +612,6 @@ class WorkbookBuilder:
for note in r.notes:
ws.append([_c(ws, "Note"), _c(ws, note)])
# -- Settled Settlements (what was deliberately left out) --
def _build_settled_settlements(self):
"""
Every settlement whose raw rows were EXCLUDED, and why.
The marketplace tabs carry only open settlements — once Amazon's payout has reached
the bank the settlement is closed and its transactions are not repeated here. Without
this sheet a reader cannot tell a deliberately-omitted settled month from a file that
failed to upload, so the omission is listed line by line and reconciled: excluded
order rows + included order rows = every order row in the source files.
"""
agg, cls = self.result.aggregation, self.result.classification
if agg is None or cls is None:
return
ws = self.wb.create_sheet("Settled Settlements")
for col, w in zip("ABCDEFGHIJ", (16, 18, 18, 13, 13, 10, 18, 18, 14, 18)):
ws.column_dimensions[col].width = w
ws.append([_c(ws, "Settled settlements — raw rows deliberately excluded",
font=TITLE_FONT)])
ws.append([_c(ws, "Amazon's payout for each settlement below reached the bank on or "
"before month-end, so the settlement is closed and its transactions "
"are summarised here instead of listed in the marketplace tabs.")])
ws.append([])
ws.append([_c(ws, h, font=HDR_FONT, fill=HDR_FILL) for h in (
"Marketplace", "Account stream", "Settlement ID", "First date", "Last date",
"Rows", "Order total", "Payout amount", "Amazon paid", "Bank received")])
# Bank receipt per (marketplace, account stream, settlement id), when Finance entered one.
receipts = self.payout_receipts or {}
# Payout facts per settlement bucket.
pay: dict[tuple[str, str, str], list] = {}
for t in agg.transfers:
slot = pay.setdefault((t.marketplace, t.account_type, t.settlement_id),
[0.0, None])
slot[0] += t.amount
if t.txn_date and (slot[1] is None or t.txn_date > slot[1]):
slot[1] = t.txn_date
n_rows = 0
excluded_total = 0.0
settled = sorted(
((k, st) for k, st in agg.settlements.items()
if st.status != "receivable" and (st.row_count - st.transfer_count) > 0),
key=lambda kv: (kv[0][0], kv[0][1], kv[0][2]))
for (mkt, acct, sid), st in settled:
orders = st.row_count - st.transfer_count
amount, paid_on = pay.get((mkt, acct, sid), (None, None))
rec = receipts.get((mkt, acct, sid))
n_rows += orders
excluded_total += st.order_total
ws.append([
_c(ws, mkt), _c(ws, "—" if acct == "(unspecified)" else acct), _c(ws, sid),
_c(ws, st.first_date.isoformat() if st.first_date else ""),
_c(ws, st.last_date.isoformat() if st.last_date else ""),
_c(ws, orders),
# round(): summing millions of floats leaves noise like 5000.000000000001,
# which reads as a data problem in an audit workbook.
_c(ws, round(st.order_total, 2), number_format=FMT_ACCT2),
_c(ws, round(amount, 2), number_format=FMT_ACCT2)
if amount is not None else _c(ws, ""),
_c(ws, paid_on.isoformat() if paid_on else ""),
_c(ws, rec or "clearing-lag rule"),
])
ws.append([])
included = sum(st.row_count - st.transfer_count for k, st in agg.settlements.items()
if st.status == "receivable")
included_total = sum(st.order_total for k, st in agg.settlements.items()
if st.status == "receivable")
for label, rows_n, amount_v in (
("Excluded (settled) order rows", n_rows, excluded_total),
("Included (open) order rows — in the marketplace tabs", included, included_total),
("Total order rows in the source files", n_rows + included,
excluded_total + included_total),
):
ws.append([_c(ws, label, font=BOLD), _c(ws, ""), _c(ws, ""), _c(ws, ""), _c(ws, ""),
_c(ws, rows_n, font=BOLD),
_c(ws, round(amount_v, 2), number_format=FMT_ACCT2, font=BOLD)])
# -- Exceptions --
def _build_exceptions(self):
ws = self.wb.create_sheet("Exceptions")
@ -819,11 +701,9 @@ def export_workbook(result: ProcessResult, files: Iterable[str], output_path: st
saved_column_overrides: dict[str, str] | None = None,
row_limit: int = EXCEL_ROW_LIMIT,
progress: Callable[[float, int, int], None] | None = None,
summary: dict | None = None, journal: dict | None = None,
payout_receipts: dict[tuple[str, str, str], str] | None = None) -> str:
summary: dict | None = None, journal: dict | None = None) -> str:
builder = WorkbookBuilder(result, files, reserves=reserves,
allowance_for_returns=allowance_for_returns,
saved_column_overrides=saved_column_overrides, row_limit=row_limit,
progress=progress, summary=summary, journal=journal,
payout_receipts=payout_receipts)
progress=progress, summary=summary, journal=journal)
return builder.build(output_path)

10
ar-aging-app/backend/app/core/readers.py
View File

@ -1,4 +1,4 @@
"""Reader factory: CSV, fast calamine (xlsx), or streaming iterparse fallback."""
"""Reader factory: fast calamine engine by default, streaming iterparse as fallback."""
from __future__ import annotations
import os
@ -19,15 +19,9 @@ def _calamine_available() -> bool:
def make_reader(path: str, saved_overrides: dict[str, str] | None = None):
"""
Return a reader with the TransactionReader interface (detect/iter_records/file_meta/close).
CSV reports (Amazon "Custom Unified Transaction" downloads) use CsvReader. Spreadsheets
use python-calamine when available (much faster, higher peak memory); otherwise the
Uses python-calamine when available (much faster, higher peak memory); otherwise the
low-memory streaming reader. Set AR_USE_CALAMINE=0 to force the streaming reader.
"""
ext = os.path.splitext(path)[1].lower()
if ext == ".csv":
from .csv_reader import CsvReader
return CsvReader(path, saved_overrides=saved_overrides)
if _calamine_available():
from .calamine_reader import CalamineReader
return CalamineReader(path, saved_overrides=saved_overrides)

39
ar-aging-app/backend/app/core/receivable.py
View File

@ -23,27 +23,6 @@ from .settlements import AggregationResult, Classification, RECEIVABLE_ACCOUNT_T
# month-end it is always "Current"; the day-based bands are kept for completeness.
AGING_BANDS = ("Current", "1-30", "31-60", "61-90", "91-Over")
# Selectable band widths for the aging report. Each value lists the inclusive upper edge
# (in days past due) of every closed band; whatever exceeds the last edge falls into the
# open-ended "-Over" band. "monthly" reproduces AGING_BANDS.
AGING_SCHEMES: dict[str, tuple[int, ...]] = {
"weekly": (7, 14, 21, 28),
"monthly": (30, 60, 90),
"half_year": (180, 360, 540),
"yearly": (365, 730, 1095),
}
def aging_bands(scheme: str = "monthly") -> tuple[str, ...]:
"""Band labels for a scheme, e.g. monthly -> Current, 1-30, 31-60, 61-90, 91-Over."""
edges = AGING_SCHEMES.get(scheme, AGING_SCHEMES["monthly"])
labels, lo = ["Current"], 1
for e in edges:
labels.append(f"{lo}-{e}")
lo = e + 1
labels.append(f"{lo}-Over")
return tuple(labels)
@dataclass
class AccountReceivable:
@ -163,17 +142,17 @@ def compute_receivable(
return result
def classify_aging(days_outstanding: int | None, scheme: str = "monthly") -> str:
"""Day-based aging band for the scheme (default matches the classic monthly bands)."""
def classify_aging(days_outstanding: int | None) -> str:
"""Day-based aging band (kept for completeness; Amazon receivable is 'Current')."""
if days_outstanding is None or days_outstanding <= 0:
return "Current"
edges = AGING_SCHEMES.get(scheme, AGING_SCHEMES["monthly"])
lo = 1
for e in edges:
if days_outstanding <= e:
return f"{lo}-{e}"
lo = e + 1
return f"{lo}-Over"
if days_outstanding <= 30:
return "1-30"
if days_outstanding <= 60:
return "31-60"
if days_outstanding <= 90:
return "61-90"
return "91-Over"
def aging_summary(result: ReceivableResult, band: str = "Current") -> dict[str, dict[str, float]]:

31
ar-aging-app/backend/app/core/xlsx_reader.py
View File

@ -69,9 +69,6 @@ class FileMeta:
missing_required: list[str] = field(default_factory=list)
# canonical field -> the columns that both claimed it (only the first is used)
duplicate_fields: dict[str, list] = field(default_factory=dict)
# canonical AMOUNT field -> extra [(col, header)] whose amounts were ADDED into it
# (Amazon splits one concept across columns, e.g. AU "low value goods" tax).
summed_fields: dict[str, list] = field(default_factory=dict)
# Finance-added translation/helper header rows found below the real header and skipped.
helper_rows_skipped: int = 0
# column-letter -> Σ of numeric values seen in columns with NO mapped field.
@ -256,7 +253,6 @@ class TransactionReader:
self.file_meta.unmapped_headers = mapping.unmapped
self.file_meta.missing_required = mapping.missing_required
self.file_meta.duplicate_fields = mapping.duplicate_fields
self.file_meta.summed_fields = mapping.sum_cols
self.file_meta.sheet_last_row = self._declared_last_row(part)
return mapping
@ -288,9 +284,6 @@ class TransactionReader:
assert self.column_mapping is not None and self._zip is not None
shared = self._shared_strings()
col_to_field = self.column_mapping.col_to_field
# Extra amount columns folded into an already-mapped field (see ColumnMapping.sum_cols).
sum_col_to_field = {c: f for f, cols in self.column_mapping.sum_cols.items()
for c, _hdr in cols}
want = only_fields
min_d: date | None = None
max_d: date | None = None
@ -310,14 +303,6 @@ class TransactionReader:
for col, val in cells.items():
fld = col_to_field.get(col)
if not fld:
sfld = sum_col_to_field.get(col)
if sfld is not None:
# Cells iterate in column order, so the field's primary column has
# already been converted (when present) — add, don't assign.
if (want is None or sfld in want) and val not in (None, ""):
rec[sfld] = (rec.get(sfld) or 0.0) + _convert(sfld, val)
has_value = True
continue
# No amount is silently excluded: sum numeric data in unmapped columns.
if val not in (None, ""):
try:
@ -365,20 +350,10 @@ class TransactionReader:
def quick_expected_rows(path: str) -> int:
"""
Fast estimate of data-row count for the progress bar.
Spreadsheets: reads the _xlnm._FilterDatabase defined name (or the largest sheet's
<dimension>) without loading sharedStrings. CSV: line count minus a small preamble
allowance (exact count comes later from the reader).
Fast (KB-sized) estimate of data-row count without loading sharedStrings, used to drive
the progress bar. Reads the _xlnm._FilterDatabase defined name (or the largest sheet's
<dimension>) to find the last row.
"""
if path.lower().endswith(".csv"):
try:
with open(path, "rb") as fh:
# Cheap line count; header/preamble typically ≤ 15 rows.
n = sum(1 for _ in fh)
return max(0, n - 12)
except OSError:
return 0
try:
z = zipfile.ZipFile(path)
except Exception:

93
ar-aging-app/backend/app/db/database.py
View File

@ -1,11 +1,4 @@
"""
Database setup (SQLAlchemy) — MySQL for shared use, SQLite for a laptop or a demo.
Which one is used comes from `config.DB_BACKEND`. Everything above this layer is written
against SQLAlchemy and is dialect-agnostic; the two places that are not — the raw bulk
INSERT in services/store.py and the column-migration below — ask the engine which dialect
it is rather than assuming.
"""
"""MySQL database setup (SQLAlchemy)."""
from __future__ import annotations
import logging
@ -16,7 +9,6 @@ from sqlalchemy import create_engine, event, text
from sqlalchemy.orm import declarative_base, sessionmaker
from ..config import (
DB_BACKEND,
MYSQL_DATABASE,
MYSQL_HOST,
MYSQL_PASSWORD,
@ -25,19 +17,17 @@ from ..config import (
MYSQL_PORT,
MYSQL_SLOW_QUERY_MS,
MYSQL_USER,
database_label,
database_url,
ensure_dirs,
mysql_url,
)
ensure_dirs()
logger = logging.getLogger(__name__)
IS_MYSQL = DB_BACKEND == "mysql"
def _ensure_database() -> None:
"""Create MYSQL_DATABASE if it does not exist yet (MySQL only)."""
"""Create MYSQL_DATABASE if it does not exist yet."""
user = quote_plus(MYSQL_USER)
password = quote_plus(MYSQL_PASSWORD)
server_url = (
@ -57,32 +47,15 @@ def _ensure_database() -> None:
server_engine.dispose()
if IS_MYSQL:
_ensure_database()
ENGINE = create_engine(
database_url(),
mysql_url(),
pool_size=MYSQL_POOL_SIZE,
pool_recycle=MYSQL_POOL_RECYCLE,
pool_pre_ping=True,
future=True,
)
else:
# check_same_thread=False: processing runs in a background thread with its own session.
ENGINE = create_engine(
database_url(), future=True,
connect_args={"check_same_thread": False, "timeout": 30},
)
@event.listens_for(ENGINE, "connect")
def _sqlite_pragmas(dbapi_conn, _rec):
cur = dbapi_conn.cursor()
cur.execute("PRAGMA journal_mode=WAL") # readers don't block the writer
cur.execute("PRAGMA foreign_keys=ON") # cascade deletes behave like MySQL
cur.execute("PRAGMA busy_timeout=30000") # bulk insert vs progress updates
cur.execute("PRAGMA synchronous=NORMAL")
cur.close()
logger.info("database: %s", database_label())
if MYSQL_SLOW_QUERY_MS > 0:
@event.listens_for(ENGINE, "before_cursor_execute")
@ -113,30 +86,16 @@ def init_db() -> None:
def _migrate() -> None:
"""
Add columns introduced after a DB was first created (create_all won't alter).
MySQL DDL rules that differ from SQLite and silently broke this list during the port:
* VARCHAR **must** carry a length — a bare `VARCHAR` is a syntax error. Lengths here
must match the model's String(n) or the column ends up a different width.
* TEXT/BLOB columns cannot take a literal DEFAULT before MySQL 8.0.13, so
`TEXT DEFAULT ''` fails. Declare plain TEXT and let the ORM default apply on insert.
"""
"""Add columns introduced after a DB was first created (create_all won't alter)."""
added = {
"users": [
("reset_code_hash", "VARCHAR(255) DEFAULT ''"),
("reset_code_expires", "DATETIME"),
("reset_code_attempts", "INTEGER DEFAULT 0"),
("is_admin", "BOOLEAN DEFAULT 0"),
],
"sessions": [
("progress_rows_done", "INTEGER DEFAULT 0"),
("progress_rows_total", "INTEGER DEFAULT 0"),
("eta_seconds", "INTEGER DEFAULT 0"),
("opening_mode", "VARCHAR(32) DEFAULT 'zero'"),
("opening_mode", "VARCHAR(255) DEFAULT 'zero'"),
("opening_source_session_id", "INTEGER"),
("blocked_reason", "TEXT"),
("payout_mode", "VARCHAR(32) DEFAULT 'auto'"),
("blocked_reason", "TEXT DEFAULT ''"),
("payout_mode", "VARCHAR DEFAULT 'auto'"),
("needs_reprocess", "BOOLEAN DEFAULT 0"),
],
"session_files": [
@ -145,14 +104,14 @@ def _migrate() -> None:
("helper_rows_skipped", "INTEGER DEFAULT 0"),
],
"fx_rates": [
("confirmed_by", "VARCHAR(255) DEFAULT ''"),
("confirmed_by", "VARCHAR DEFAULT ''"),
("confirmed_at", "DATETIME"),
("confirmed_month", "VARCHAR(32) DEFAULT ''"),
("confirmed_month", "VARCHAR DEFAULT ''"),
],
"journal_entries": [
("reviewed_by", "VARCHAR(255) DEFAULT ''"),
("reviewed_by", "VARCHAR DEFAULT ''"),
("reviewed_at", "DATETIME"),
("approved_by", "VARCHAR(255) DEFAULT ''"),
("approved_by", "VARCHAR DEFAULT ''"),
("approved_at", "DATETIME"),
],
"reconciliation": [
@ -170,13 +129,9 @@ def _migrate() -> None:
("storage_flag", "BOOLEAN DEFAULT 0"),
],
}
# Each ALTER runs on its own connection scope: MySQL auto-commits DDL, so wrapping the
# whole loop in one transaction gives no rollback anyway — and one bad statement would
# otherwise abort every later migration for the rest of the run.
with ENGINE.connect() as conn:
db_name = conn.execute(text("SELECT DATABASE()")).scalar() if IS_MYSQL else None
with ENGINE.begin() as conn:
db_name = conn.execute(text("SELECT DATABASE()")).scalar()
for table, cols in added.items():
if IS_MYSQL:
existing = {
r[0]
for r in conn.execute(
@ -187,23 +142,9 @@ def _migrate() -> None:
{"schema": db_name, "table": table},
)
}
else:
existing = {r[1] for r in conn.execute(text(f"PRAGMA table_info({table})"))}
if not existing:
continue # table not created yet — create_all owns it
for name, decl in cols:
if name in existing:
continue
# SQLite has no VARCHAR length limit and rejects some MySQL type spellings;
# its dynamic typing makes the declared type advisory anyway.
sql_decl = decl.replace("VARCHAR(255)", "VARCHAR").replace(
"VARCHAR(32)", "VARCHAR") if not IS_MYSQL else decl
try:
conn.execute(text(f"ALTER TABLE `{table}` ADD COLUMN `{name}` {sql_decl}"))
conn.commit()
except Exception: # noqa: BLE001
conn.rollback()
logger.exception("migration failed: %s.%s %s", table, name, decl)
if name not in existing:
conn.execute(text(f"ALTER TABLE `{table}` ADD COLUMN `{name}` {decl}"))
def get_db():

63
ar-aging-app/backend/app/db/models.py
View File

@ -15,45 +15,6 @@ def _now() -> dt.datetime:
return dt.datetime.utcnow()
class User(Base):
"""A named person who can sign in. Created via `python manage.py add-user` — there is no
self-signup. The display name is what lands in reviewed_by / approved_by / confirmed_by,
so accountability fields carry a verified identity instead of free text."""
__tablename__ = "users"
id = Column(Integer, primary_key=True)
username = Column(String(64), unique=True, nullable=False)
display_name = Column(String(255), nullable=False)
password_hash = Column(String(512), nullable=False)
is_active = Column(Boolean, default=True)
# Admins can read the audit log (/api/audit). Granted via `manage.py set-admin`.
is_admin = Column(Boolean, default=False)
created_at = Column(DateTime, default=_now)
# Emailed password code (usernames are email addresses). Stored as an HMAC, never the
# code itself; single-use, expires, and locks after too many wrong attempts.
reset_code_hash = Column(String(128), default="")
reset_code_expires = Column(DateTime)
reset_code_attempts = Column(Integer, default=0)
class AuditLog(Base):
"""Append-only record of who did what: uploads, deletions, processing runs, exports,
closing lifecycle, logins. session_id is a plain integer (no FK) so history survives
the closing being deleted. Written via services/audit.py; read via /api/audit (admins)."""
__tablename__ = "audit_log"
id = Column(Integer, primary_key=True)
at = Column(DateTime, default=_now, nullable=False)
username = Column(String(64), default="") # "" = auth off (dev) or unknown
display_name = Column(String(255), default="")
action = Column(String(64), nullable=False)
session_id = Column(Integer)
session_name = Column(String(255), default="")
detail = Column(Text, default="")
Index("ix_audit_at", AuditLog.at)
Index("ix_audit_session", AuditLog.session_id)
class Session(Base):
__tablename__ = "sessions"
id = Column(Integer, primary_key=True)
@ -293,30 +254,8 @@ class ReconciliationRow(Base):
all_payouts = Column(Float, default=0.0) # Σ all transfers (negative)
class FxProviderRate(Base):
"""Cache of rates fetched from the FX provider (Frankfurter by default).
One row per (provider, date, currency); `rate` is USD per 1 unit of local currency —
the same orientation as FxRate.rate, i.e. usd = local * rate. Caching makes a re-fetch
idempotent and keeps month-end seeding working offline once fetched."""
__tablename__ = "fx_provider_rates"
id = Column(Integer, primary_key=True)
provider = Column(String(32), nullable=False, default="frankfurter")
rate_date = Column(Date, nullable=False)
currency = Column(String(16), nullable=False)
rate = Column(Float, nullable=False)
fetched_at = Column(DateTime, default=_now)
__table_args__ = (
Index("ix_fx_provider_key", "provider", "rate_date", "currency", unique=True),
)
class FxRateDaily(Base):
"""Per-date FX rate — auto-fetched from the provider at processing, hand-editable.
Dated movements convert at the rate effective on their transaction date: this exact
date's row, else the previous banking day's provider fixing, else the month rate
(see analytics._effective_rate)."""
"""Optional per-date FX override. Falls back to the marketplace's month rate."""
__tablename__ = "fx_rates_daily"
id = Column(Integer, primary_key=True)
session_id = Column(Integer, ForeignKey("sessions.id"), nullable=False)

34
ar-aging-app/backend/app/services/audit.py
View File

@ -1,34 +0,0 @@
"""
Audit trail: one append-only row per business action, attributed to the signed-in user.
`record()` only ADDS the row to the caller's ORM session — the caller's own `db.commit()`
persists it atomically with the action itself, so a failed action never leaves a phantom
audit entry (and a recorded action is never lost to a second commit failing).
"""
from __future__ import annotations
from fastapi import Request
from sqlalchemy.orm import Session as OrmSession
from ..db import models
def record(db: OrmSession, request: Request | None, action: str, detail: str = "",
session: models.Session | None = None) -> models.AuditLog:
"""Attach an audit row to the caller's transaction.
Identity comes from the verified bearer token; with auth off (dev / tests before the
first user) the row is still written with an empty username, so the trail's shape is
the same everywhere."""
from ..api.auth import current_user # late import: auth imports models too
user = current_user(request) if request is not None else None
row = models.AuditLog(
username=user.username if user else "",
display_name=user.display_name if user else "",
action=action,
session_id=session.id if session is not None else None,
session_name=session.name if session is not None else "",
detail=(detail or "")[:2000],
)
db.add(row)
return row

367
ar-aging-app/backend/app/services/fx_service.py
View File

@ -1,367 +0,0 @@
"""
Exchange-rate fetching (the ONE deliberate network egress in the app — currency codes and
dates only, never financial data).
Providers
frankfurter (default) free, keyless, central-bank (ECB) reference rates, historical
dates supported. Weekend/holiday dates snap to the previous
banking day — exactly the month-end convention Finance uses.
exchangerate-api paid fallback (AR_FX_PROVIDER=exchangerate-api + AR_FX_API_KEY).
ORIENTATION — the #1 way to corrupt every non-USD receivable:
The app stores USD per 1 unit of LOCAL currency (usd = local * FxRate.rate; see
core/money.to_usd and store.py). Providers return the opposite (local per 1 USD when
base=USD), so every provider here INVERTS before returning. test_fx_service.py pins
this with a known EUR fixture.
Fetched rates are SUGGESTIONS: seeding writes them unconfirmed, so Control C5 still blocks
the close until a person reviews and confirms them for the reporting month — identical to
the manual-entry workflow, just pre-filled with a real rate instead of the Jan-26 snapshot.
Daily rates: processing auto-fetches the provider's daily fixings across the closing's
transaction span (auto_seed_daily_fx), so every dated movement converts at the rate
effective on ITS OWN transaction date — see api/routes/analytics.py for the resolution
order (exact fixing → previous banking day's fixing → month rate).
Failure policy: a provider error raises FxProviderError (the route answers 502 "enter rates
manually"). DEFAULT_FX_USD is never written silently — the existing merge in jobs.py is
already the fallback and C5 already flags unconfirmed defaults.
"""
from __future__ import annotations
import datetime as dt
import json
import logging
import ssl
import urllib.error
import urllib.parse
import urllib.request
from sqlalchemy import func as sa_func
from sqlalchemy.orm import Session as OrmSession
from ..config import FX_API_KEY, FX_PROVIDER, FX_TIMEOUT_S
from ..core.i18n import currency_for_region
from ..db import models
logger = logging.getLogger(__name__)
class FxProviderError(RuntimeError):
"""The provider could not supply rates (network, quota, unknown currency...)."""
def _ssl_context() -> ssl.SSLContext | None:
"""Prefer certifi's CA bundle: on some Windows machines loading the OS certificate
store fails outright (ssl [ASN1: NOT_ENOUGH_DATA]), which would break every fetch.
Fall back to the default context when certifi isn't installed (Linux containers)."""
try:
import certifi
return ssl.create_default_context(cafile=certifi.where())
except ImportError:
return None
def _http_get_json(url: str) -> dict:
req = urllib.request.Request(url, headers={"User-Agent": "ar-aging-app/1.0"})
try:
with urllib.request.urlopen(req, timeout=FX_TIMEOUT_S,
context=_ssl_context()) as resp:
return json.loads(resp.read().decode("utf-8"))
except urllib.error.HTTPError as e:
raise FxProviderError(f"FX provider answered HTTP {e.code} for {url.split('?')[0]}") from e
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError, OSError, ssl.SSLError) as e:
raise FxProviderError(f"Could not reach the FX provider: {e}") from e
class FrankfurterProvider:
"""https://frankfurter.dev — GET /v1/{date}?base=USD&symbols=EUR,GBP,..."""
name = "frankfurter"
_BASE = "https://api.frankfurter.dev/v1"
def rates_on(self, on: dt.date, currencies: set[str]) -> tuple[dict[str, float], dt.date]:
"""{currency: USD-per-local}, plus the banking day the provider actually used."""
symbols = sorted(c for c in currencies if c and c != "USD")
if not symbols:
return {}, on
url = (f"{self._BASE}/{on.isoformat()}"
f"?base=USD&symbols={urllib.parse.quote(','.join(symbols))}")
data = _http_get_json(url)
raw = data.get("rates") or {}
# base=USD → provider returns LOCAL per USD; the app stores USD per LOCAL. Invert.
out = {ccy: 1.0 / v for ccy, v in raw.items() if v}
actual = dt.date.fromisoformat(data["date"]) if data.get("date") else on
return out, actual
def rates_series(self, date_from: dt.date, date_to: dt.date,
currencies: set[str]) -> dict[dt.date, dict[str, float]]:
"""{date: {currency: USD-per-local}} for every banking day in the range."""
symbols = sorted(c for c in currencies if c and c != "USD")
if not symbols:
return {}
url = (f"{self._BASE}/{date_from.isoformat()}..{date_to.isoformat()}"
f"?base=USD&symbols={urllib.parse.quote(','.join(symbols))}")
data = _http_get_json(url)
out: dict[dt.date, dict[str, float]] = {}
for day, raw in (data.get("rates") or {}).items():
out[dt.date.fromisoformat(day)] = {c: 1.0 / v for c, v in raw.items() if v}
return out
class ExchangeRateApiProvider:
"""https://www.exchangerate-api.com — paid fallback. Needs AR_FX_API_KEY."""
name = "exchangerate-api"
_BASE = "https://v6.exchangerate-api.com/v6"
def __init__(self) -> None:
if not FX_API_KEY:
raise FxProviderError(
"AR_FX_PROVIDER=exchangerate-api requires AR_FX_API_KEY.")
def rates_on(self, on: dt.date, currencies: set[str]) -> tuple[dict[str, float], dt.date]:
symbols = {c for c in currencies if c and c != "USD"}
if not symbols:
return {}, on
# History endpoint (paid plans); falls back to latest when the date is today.
if on >= dt.date.today():
url = f"{self._BASE}/{FX_API_KEY}/latest/USD"
else:
url = f"{self._BASE}/{FX_API_KEY}/history/USD/{on.year}/{on.month}/{on.day}"
data = _http_get_json(url)
if data.get("result") != "success":
raise FxProviderError(f"exchangerate-api: {data.get('error-type', 'error')}")
raw = data.get("conversion_rates") or {}
return {c: 1.0 / raw[c] for c in symbols if raw.get(c)}, on
def rates_series(self, date_from: dt.date, date_to: dt.date,
currencies: set[str]) -> dict[dt.date, dict[str, float]]:
out: dict[dt.date, dict[str, float]] = {}
day = date_from
while day <= date_to:
try:
rates, actual = self.rates_on(day, currencies)
out[actual] = rates
except FxProviderError:
pass # weekends/holidays have no fixing
day += dt.timedelta(days=1)
return out
def get_provider():
if FX_PROVIDER == "exchangerate-api":
return ExchangeRateApiProvider()
if FX_PROVIDER == "frankfurter":
return FrankfurterProvider()
raise FxProviderError(f"Unknown AR_FX_PROVIDER {FX_PROVIDER!r} "
f"(use 'frankfurter' or 'exchangerate-api').")
# --------------------------------------------------------------------------- caching
def _cached_rates(db: OrmSession, provider_name: str, on: dt.date,
currencies: set[str]) -> dict[str, float] | None:
"""All requested currencies from the cache, or None on any miss."""
want = {c for c in currencies if c != "USD"}
if not want:
return {}
rows = db.query(models.FxProviderRate).filter(
models.FxProviderRate.provider == provider_name,
models.FxProviderRate.rate_date == on,
models.FxProviderRate.currency.in_(want)).all()
got = {r.currency: r.rate for r in rows}
return got if set(got) >= want else None
def _cache_rates(db: OrmSession, provider_name: str, on: dt.date,
rates: dict[str, float]) -> None:
existing = {r.currency for r in db.query(models.FxProviderRate).filter(
models.FxProviderRate.provider == provider_name,
models.FxProviderRate.rate_date == on)}
for ccy, rate in rates.items():
if ccy not in existing:
db.add(models.FxProviderRate(provider=provider_name, rate_date=on,
currency=ccy, rate=rate))
db.commit()
def rates_for_date(db: OrmSession, on: dt.date,
currencies: set[str]) -> tuple[dict[str, float], str]:
"""{currency: USD-per-local} for a date — cache first, provider on miss.
Returns (rates, source_label). The label names the provider and the banking day the
rates are actually for, so an FxRate row's `source` explains itself."""
provider = get_provider()
cached = _cached_rates(db, provider.name, on, currencies)
if cached is not None:
return cached, f"{provider.name} {on.isoformat()} (cached)"
rates, actual = provider.rates_on(on, currencies)
# Cache under both the requested date and the provider's actual banking day, so a
# weekend month-end (snapped to Friday) is served from cache next time as well.
_cache_rates(db, provider.name, actual, rates)
if actual != on:
_cache_rates(db, provider.name, on, rates)
return rates, f"{provider.name} {actual.isoformat()}"
# --------------------------------------------------------------------------- seeding
def _session_fx_targets(db: OrmSession, session: models.Session) -> list[models.FxRate]:
"""The session's existing FX rows — the marketplaces this close actually involves.
Rows are created during processing for every marketplace that appears in the files
(jobs.py), so 'process first' is the natural precondition; seeding rates for
marketplaces the close doesn't contain would only widen what C5 asks Finance to
confirm."""
return db.query(models.FxRate).filter(
models.FxRate.session_id == session.id).all()
def seed_session_fx(db: OrmSession, session: models.Session) -> dict:
"""Fetch month-end rates and pre-fill the session's FX table (UNCONFIRMED).
Existing confirmations are cleared — same withdrawal semantics as editing a rate by
hand (settings.put_fx): a confirmation attests to a specific number."""
if session.month_end_date is None:
raise FxProviderError("Set the month-end date first.")
rows = _session_fx_targets(db, session)
if not rows:
raise FxProviderError(
"No FX rows exist yet for this closing — process it first so its "
"marketplaces are known.")
currencies = {(r.currency or currency_for_region(r.marketplace)) for r in rows}
fetched, source = rates_for_date(db, session.month_end_date, currencies)
updated, missing = [], []
for r in rows:
ccy = r.currency or currency_for_region(r.marketplace)
if ccy == "USD":
new_rate = 1.0
elif ccy in fetched:
new_rate = round(fetched[ccy], 6)
else:
missing.append(f"{r.marketplace} ({ccy})")
continue
r.rate = new_rate
r.currency = ccy
r.rate_date = session.month_end_date
r.source = source
# A fetched rate is a suggestion — it must be confirmed for THIS month (C5).
r.confirmed_by = ""
r.confirmed_at = None
r.confirmed_month = ""
updated.append({"marketplace": r.marketplace, "currency": ccy, "rate": new_rate})
db.commit()
if session.status in ("processed", "blocked", "completed"):
from .controls_run import run_and_persist
run_and_persist(db, session.id)
logger.info("fx seed: session %s, %d rate(s) from %s, %d missing",
session.id, len(updated), source, len(missing))
return {"updated": updated, "missing": missing, "source": source,
"rate_date": session.month_end_date.isoformat()}
def _transaction_span(db: OrmSession, session_id: int) -> tuple[dt.date | None, dt.date | None]:
"""Earliest/latest dated transaction of the closing ((None, None) when nothing is dated)."""
lo, hi = db.query(sa_func.min(models.Transaction.posted_date),
sa_func.max(models.Transaction.posted_date)).filter(
models.Transaction.session_id == session_id,
models.Transaction.posted_date.isnot(None)).one()
def _d(v):
return v if (v is None or isinstance(v, dt.date)) else dt.date.fromisoformat(str(v))
return _d(lo), _d(hi)
def seed_daily_fx(db: OrmSession, session: models.Session, marketplace: str | None = None,
date_from: dt.date | None = None, date_to: dt.date | None = None,
overwrite_manual: bool = True) -> dict:
"""Fill fx_rates_daily from the provider.
Default range: the span of dates the files actually contain (earliest dated
transaction through month-end, extended to any later transaction), widened to the
start of the reporting month — so every transaction converts at its own date's rate.
Clamped to a year before / a month after month-end, so one mis-parsed date can't
request a decade of history.
Daily rows are what the ledger converts dated movements with (analytics),
marked source=provider so hand-entered rows are distinguishable. With
overwrite_manual=False (the automatic post-processing seed), rows a person typed
stay untouched; the explicit Fetch button replaces them."""
if session.month_end_date is None:
raise FxProviderError("Set the month-end date first.")
month_end = session.month_end_date
if date_from is None or date_to is None:
lo, hi = _transaction_span(db, session.id)
if date_from is None:
date_from = min(lo or month_end.replace(day=1), month_end.replace(day=1))
date_from = max(date_from, month_end - dt.timedelta(days=366))
if date_to is None:
date_to = max(hi or month_end, month_end)
date_to = min(date_to, month_end + dt.timedelta(days=31))
if date_from > date_to:
raise FxProviderError("date_from is after date_to.")
rows = _session_fx_targets(db, session)
targets = [(r.marketplace, r.currency or currency_for_region(r.marketplace))
for r in rows
if (marketplace is None or r.marketplace == marketplace)]
targets = [(m, c) for m, c in targets if c != "USD"]
if not targets:
raise FxProviderError(
"No non-USD marketplace to fetch daily rates for — process the closing "
"first (or this closing is USD-only).")
provider = get_provider()
series = provider.rates_series(date_from, date_to, {c for _, c in targets})
existing = {(r.marketplace, r.rate_date): r for r in db.query(models.FxRateDaily).filter(
models.FxRateDaily.session_id == session.id)}
saved = 0
for day, per_ccy in sorted(series.items()):
for mkt, ccy in targets:
rate = per_ccy.get(ccy)
if not rate:
continue
row = existing.get((mkt, day))
if row is None:
row = models.FxRateDaily(session_id=session.id, marketplace=mkt,
rate_date=day)
db.add(row)
existing[(mkt, day)] = row
elif not overwrite_manual and (row.source or "") == "manual":
continue # a person typed this rate — keep it
row.rate = round(rate, 6)
row.source = provider.name
saved += 1
db.commit()
logger.info("fx daily seed: session %s, %d row(s) %s..%s",
session.id, saved, date_from, date_to)
return {"saved": saved, "date_from": date_from.isoformat(),
"date_to": date_to.isoformat(), "provider": provider.name,
"marketplaces": sorted({m for m, _ in targets})}
def auto_seed_daily_fx(db: OrmSession, session: models.Session) -> dict:
"""Post-processing daily-rate fetch, so every dated movement converts at the rate
effective on its own transaction date without anyone clicking anything.
Advisory by design — it NEVER raises: a provider outage must not fail the close
(conversion falls back to the last available fixing, then the month rate, and
jobs.py surfaces the shortfall as an exception). Hand-entered daily rates are
preserved; only provider rows are refreshed."""
try:
rows = _session_fx_targets(db, session)
if not any((r.currency or currency_for_region(r.marketplace)) != "USD"
for r in rows):
return {"skipped": "USD-only closing", "saved": 0}
return seed_daily_fx(db, session, overwrite_manual=False)
except FxProviderError as e:
logger.warning("daily FX auto-seed failed for session %s: %s", session.id, e)
return {"error": str(e), "saved": 0}
except Exception as e: # noqa: BLE001 — advisory; never fail the close over FX
logger.exception("daily FX auto-seed crashed for session %s", session.id)
db.rollback()
return {"error": f"{type(e).__name__}: {e}", "saved": 0}

59
ar-aging-app/backend/app/services/jobs.py
View File

@ -1,44 +1,15 @@
"""Background processing job: run the engine over a session's files and persist results."""
from __future__ import annotations
import logging
import time
import traceback
from ..config import FX_AUTO_DAILY
from ..core.pipeline import process
from ..core.i18n import CURRENCY_BY_REGION, DEFAULT_FX_USD, currency_for_region, default_fx_for_region
from ..db import models
from ..db.database import SessionLocal
from .store import TransactionSink, persist_aggregates, clear_session_results
logger = logging.getLogger(__name__)
def recover_stale_jobs() -> None:
"""Called once at startup. Jobs run in-process (single worker), so a session still
marked processing/exporting at boot was killed mid-run by a restart or deploy — without
this it stays stuck forever and the 409 "already processing" guard blocks every re-run."""
db = SessionLocal()
try:
stale = db.query(models.Session).filter(
models.Session.status.in_(("processing", "exporting"))).all()
for s in stale:
was = s.status
s.status = "error" if was == "processing" else "processed"
s.error = ("Interrupted by a server restart before it finished — run it again."
if was == "processing"
else "Export was interrupted by a server restart — export again.")
s.progress_stage = "Interrupted"
logger.warning("recovered stale job: session %s (%s) was '%s'", s.id, s.name, was)
if stale:
db.commit()
except Exception: # noqa: BLE001 — recovery must never prevent startup
db.rollback()
logger.exception("stale-job recovery failed")
finally:
db.close()
def load_mapping_rules(db) -> dict[str, str]:
"""Admin-saved header rules (normalized header -> canonical field)."""
@ -135,26 +106,6 @@ def run_processing(session_id: int) -> None:
source="default (Jan-26 workbook)"))
db.commit()
# Daily FX from the provider, covering the span of dates the files actually
# contain, so every dated movement converts at the rate effective on ITS OWN
# transaction date (ledger / fx-daily). Advisory: a provider outage never blocks
# the close — conversion falls back to the last available fixing, then the month
# rate, and the shortfall is surfaced below as an exception.
if FX_AUTO_DAILY:
progress("Fetching daily FX rates", 0.97)
from .fx_service import auto_seed_daily_fx
fx_daily_out = auto_seed_daily_fx(db, session)
if fx_daily_out.get("error"):
db.add(models.Exception_(
session_id=session_id, category="fx_daily_unavailable",
severity="warning",
detail=(f"Daily exchange rates could not be fetched from the provider "
f"({fx_daily_out['error']}). Dated movements convert at "
f"previously fetched daily rates or the month rate until "
f"'Fetch daily rates' on the AR Ledger tab succeeds."),
source="fx provider"))
db.commit()
# Journal-entry decomposition (separate pass; part of the close).
try:
progress("Building journal entry", 0.98)
@ -405,18 +356,10 @@ def run_export(session_id: int) -> None:
EXPORT_DIR.mkdir(parents=True, exist_ok=True)
month = session.reporting_month or "output"
out_path = str(EXPORT_DIR / f"AR_Aging_{month}_session{session_id}.xlsx")
# Bank dates for the "Settled Settlements" sheet, so the workbook records WHY each
# excluded settlement was excluded and who said so.
receipt_notes = {
(r.marketplace, r.account_type, r.settlement_id):
f"{r.bank_date}" + (f" · {r.entered_by}" if r.entered_by else "")
for r in receipts if r.bank_date
}
export_workbook(result, paths, out_path, reserves=reserves,
allowance_for_returns=session.allowance_for_returns or 0.0,
saved_column_overrides=mapping_rules,
progress=write_progress, summary=_summary, journal=_journal,
payout_receipts=receipt_notes)
progress=write_progress, summary=_summary, journal=_journal)
_finalize_export(db, session_id, out_path, "full")
session.status = "processed"

107
ar-aging-app/backend/app/services/mailer.py
View File

@ -1,107 +0,0 @@
"""Outbound email — used ONLY for password codes. Stdlib only, no new dependencies.
Transports (first configured wins; see config.py):
* Mail API — the company's internal mail service (bearer-token multipart POST; the
same service the TikTok dashboard uses for its verification codes).
* SMTP — any standard account (Office365 / Gmail app password / relay).
When neither is configured, callers get MailerError and the UI falls back to the
current-password / admin-reset flows."""
from __future__ import annotations
import json
import logging
import smtplib
import ssl
import urllib.error
import urllib.request
import uuid
from email.message import EmailMessage
from ..config import (MAIL_API_TOKEN, MAIL_API_URL, SMTP_FROM, SMTP_HOST, SMTP_PASSWORD,
SMTP_PORT, SMTP_STARTTLS, SMTP_USER, email_enabled)
logger = logging.getLogger(__name__)
class MailerError(RuntimeError):
"""Email could not be sent (unconfigured, auth failure, network...)."""
def _send_via_mail_api(to: str, subject: str, body: str) -> None:
"""POST multipart/form-data to the team mail service (same field set the TikTok
dashboard's mailer sends: subject, body, to, cc, bcc, content_type, save_to_sent)."""
boundary = f"----ar-aging-{uuid.uuid4().hex}"
fields = {"subject": subject, "body": body, "to": to, "cc": "", "bcc": "",
"content_type": "text", "save_to_sent_items": "true"}
parts = []
for name, value in fields.items():
parts.append(f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="{name}"\r\n\r\n'
f"{value}\r\n")
payload = ("".join(parts) + f"--{boundary}--\r\n").encode("utf-8")
req = urllib.request.Request(MAIL_API_URL, data=payload, method="POST", headers={
"Content-Type": f"multipart/form-data; boundary={boundary}",
"Authorization": f"Bearer {MAIL_API_TOKEN}",
"accept": "application/json",
})
try:
with urllib.request.urlopen(req, timeout=25) as resp:
raw = resp.read().decode("utf-8", "replace")
logger.info("mail api sent to %s: %s (%s)", to, subject, raw[:120])
except urllib.error.HTTPError as e:
detail = e.read().decode("utf-8", "replace")[:200]
raise MailerError(f"Mail service answered HTTP {e.code}: {detail}") from e
except (urllib.error.URLError, TimeoutError, OSError, json.JSONDecodeError) as e:
raise MailerError(f"Could not reach the mail service: {e}") from e
def _ssl_context() -> ssl.SSLContext:
"""certifi CA bundle when available — the Windows OS cert store is unreliable on some
machines (same workaround as the FX service)."""
try:
import certifi
return ssl.create_default_context(cafile=certifi.where())
except ImportError:
return ssl.create_default_context()
def send_email(to: str, subject: str, body: str) -> None:
if not email_enabled():
raise MailerError("Email is not configured on this server "
"(AR_MAIL_API_* or AR_SMTP_* settings).")
if MAIL_API_URL:
_send_via_mail_api(to, subject, body)
return
msg = EmailMessage()
msg["From"] = SMTP_FROM
msg["To"] = to
msg["Subject"] = subject
msg.set_content(body)
try:
if SMTP_STARTTLS:
with smtplib.SMTP(SMTP_HOST, SMTP_PORT, timeout=20) as s:
s.starttls(context=_ssl_context())
if SMTP_USER:
s.login(SMTP_USER, SMTP_PASSWORD)
s.send_message(msg)
else: # implicit TLS (port 465)
with smtplib.SMTP_SSL(SMTP_HOST, SMTP_PORT, timeout=20,
context=_ssl_context()) as s:
if SMTP_USER:
s.login(SMTP_USER, SMTP_PASSWORD)
s.send_message(msg)
logger.info("email sent to %s: %s", to, subject)
except (smtplib.SMTPException, OSError) as e:
raise MailerError(f"Could not send the email: {e}") from e
def send_password_code(to: str, code: str, minutes: int) -> None:
send_email(
to,
"Your password code — Amazon A/R Aging",
f"Your verification code is:\n\n {code}\n\n"
f"It expires in {minutes} minutes and works once.\n\n"
f"If you didn't request a password change, ignore this email — "
f"your password has not been changed.",
)

62
ar-aging-app/backend/app/services/retention.py
View File

@ -1,62 +0,0 @@
"""
Retention: purge generated export workbooks older than AR_RETENTION_DAYS.
EXPORTS ONLY, by design. Uploaded source files are never auto-deleted — they are the audit
source for every published figure, and re-generating the full workbook re-parses them.
Export files are pure derivatives: anything purged can be regenerated with one click, and
the DB row is kept so the Exports list still shows what was generated and when
(`available: false` once the file is gone).
AR_RETENTION_DAYS=0 keeps everything forever.
"""
from __future__ import annotations
import asyncio
import datetime as dt
import logging
import os
from ..config import RETENTION_DAYS
from ..db import models
from ..db.database import SessionLocal
logger = logging.getLogger(__name__)
_SWEEP_INTERVAL_S = 24 * 3600
def purge_old_exports(retention_days: int | None = None) -> int:
"""Delete export files older than the retention window. Returns files removed."""
days = RETENTION_DAYS if retention_days is None else retention_days
if days <= 0:
return 0
cutoff = dt.datetime.utcnow() - dt.timedelta(days=days)
removed = 0
db = SessionLocal()
try:
rows = db.query(models.ExportRecord).filter(
models.ExportRecord.generated_at < cutoff).all()
for r in rows:
if not r.path:
continue
try:
if os.path.exists(r.path):
os.remove(r.path)
removed += 1
except OSError:
logger.warning("retention: could not remove %s", r.path)
if removed:
logger.info("retention: removed %d export file(s) older than %d days",
removed, days)
except Exception: # noqa: BLE001 — a failed sweep must never take the app down
logger.exception("retention sweep failed")
finally:
db.close()
return removed
async def retention_loop() -> None:
"""Daily sweep, started from the app lifespan. Cancelled cleanly on shutdown."""
while True:
await asyncio.to_thread(purge_old_exports)
await asyncio.sleep(_SWEEP_INTERVAL_S)

23
ar-aging-app/backend/app/services/store.py
View File

@ -16,12 +16,10 @@ _TXN_COLS = (
"settlement_id", "order_id", "sku", "txn_type", "txn_type_en", "account_type",
"posted_date", "total", "currency", "storage_flag",
)
# Raw DBAPI, so the placeholder style is the driver's, not SQLAlchemy's: PyMySQL wants %s,
# sqlite3 wants ?. Ask the engine which dialect it is instead of hardcoding either.
_PARAM = "%s" if ENGINE.dialect.name == "mysql" else "?"
# PyMySQL uses %-style placeholders for raw DBAPI executemany.
_INSERT_SQL = (
f"INSERT INTO transactions ({', '.join(_TXN_COLS)}) "
f"VALUES ({', '.join([_PARAM] * len(_TXN_COLS))})"
f"VALUES ({', '.join(['%s'] * len(_TXN_COLS))})"
)
@ -77,16 +75,14 @@ class TransactionSink:
recv = 1 if (st.status == "receivable"
and acct.lower() in RECEIVABLE_ACCOUNT_TYPES) else 0
cur.execute(
f"UPDATE transactions SET settlement_status={_PARAM}, "
f"receivable_flag={_PARAM} WHERE session_id={_PARAM} "
f"AND settlement_id={_PARAM} AND marketplace={_PARAM} "
f"AND account_type={_PARAM}",
"UPDATE transactions SET settlement_status=%s, receivable_flag=%s "
"WHERE session_id=%s AND settlement_id=%s AND marketplace=%s AND account_type=%s",
(st.status, recv, self.session_id, sid, mkt, acct),
)
# transfer rows: never receivable (canonical type covers localized names)
cur.execute(
f"UPDATE transactions SET receivable_flag=0 "
f"WHERE session_id={_PARAM} AND txn_type_en='Transfer'",
"UPDATE transactions SET receivable_flag=0 "
"WHERE session_id=%s AND txn_type_en='Transfer'",
(self.session_id,),
)
cur.close()
@ -287,13 +283,6 @@ def _exceptions_from(result: ProcessResult) -> list[dict]:
f"used, so the others are excluded from every total — "
f"correct the header mapping before relying on this close."),
"source": m.filename})
for fld, cols in (getattr(m, "summed_fields", None) or {}).items():
cols_txt = ", ".join(f"{c}{f' ({t})' if t else ''}" for c, t in cols)
out.append({"category": "summed_column_mapping", "severity": "info",
"detail": (f"Column(s) {cols_txt} were added into '{fld}' — Amazon "
f"splits this concept across columns and the row total "
f"includes both."),
"source": m.filename})
for col, s in (getattr(m, "unmapped_amount_sums", None) or {}).items():
if abs(s) > 0.005:
out.append({"category": "unmapped_amounts", "severity": "error",

208
ar-aging-app/backend/manage.py
View File

@ -1,208 +0,0 @@
"""
Admin commands (run on the server, next to the app):
python manage.py add-user <username> --name "Display Name" # prompts for password
python manage.py set-password <username> # prompts for password
python manage.py list-users
python manage.py deactivate-user <username>
python manage.py set-admin <username> [--revoke] # audit-log access
python manage.py dedupe-files [--apply] # fix double-counted uploads
There is deliberately no self-signup: the 5-or-so finance users are created here.
`dedupe-files` is the one-time cleanup for the historical upload bug where re-uploading a
file created a second session_files row pointing at the same stored file — which made the
pipeline parse and sum that file twice. Run it once after deploying the fix; affected
closings are flagged needs_reprocess so the corrected totals are one click away.
"""
from __future__ import annotations
import argparse
import getpass
import sys
from app.db.database import SessionLocal, init_db
from app.db import models
def _prompt_password() -> str:
pw = getpass.getpass("Password: ")
if len(pw) < 8:
sys.exit("Password must be at least 8 characters.")
if pw != getpass.getpass("Repeat password: "):
sys.exit("Passwords do not match.")
return pw
def cmd_add_user(args) -> int:
from app.api.auth import hash_password
db = SessionLocal()
try:
username = args.username.strip().lower()
if db.query(models.User).filter(models.User.username == username).first():
print(f"User '{username}' already exists — use set-password to change it.")
return 1
pw = _prompt_password()
db.add(models.User(username=username,
display_name=(args.name or username).strip(),
password_hash=hash_password(pw), is_active=True))
db.commit()
print(f"Created user '{username}' ({args.name or username}). "
f"Login is now required (AR_AUTH=auto turns on with the first user).")
return 0
finally:
db.close()
def cmd_set_password(args) -> int:
from app.api.auth import hash_password
db = SessionLocal()
try:
user = db.query(models.User).filter(
models.User.username == args.username.strip().lower()).first()
if user is None:
print(f"No user '{args.username}'.")
return 1
user.password_hash = hash_password(_prompt_password())
user.is_active = True
db.commit()
print(f"Password updated for '{user.username}'.")
return 0
finally:
db.close()
def cmd_list_users(_args) -> int:
db = SessionLocal()
try:
rows = db.query(models.User).order_by(models.User.username).all()
if not rows:
print("No users yet — the API is open until the first one is created "
"(AR_AUTH=auto).")
return 0
for u in rows:
flag = "" if u.is_active else " [DEACTIVATED]"
admin = " [ADMIN]" if u.is_admin else ""
print(f" {u.username:<20} {u.display_name}{admin}{flag}")
return 0
finally:
db.close()
def cmd_set_admin(args) -> int:
db = SessionLocal()
try:
user = db.query(models.User).filter(
models.User.username == args.username.strip().lower()).first()
if user is None:
print(f"No user '{args.username}'.")
return 1
user.is_admin = not args.revoke
db.commit()
state = "revoked from" if args.revoke else "granted to"
print(f"Admin (audit-log access) {state} '{user.username}'.")
return 0
finally:
db.close()
def cmd_deactivate_user(args) -> int:
db = SessionLocal()
try:
user = db.query(models.User).filter(
models.User.username == args.username.strip().lower()).first()
if user is None:
print(f"No user '{args.username}'.")
return 1
user.is_active = False
db.commit()
print(f"Deactivated '{user.username}' — existing tokens stop working within "
f"their normal expiry; new logins are refused immediately.")
return 0
finally:
db.close()
def cmd_dedupe_files(args) -> int:
"""Collapse session_files rows that point at the same stored file (or share a name)
within one closing. Keeps the NEWEST row (it matches the bytes on disk — uploads
overwrote the file), deletes the rest, flags the closing for re-processing."""
db = SessionLocal()
try:
rows = db.query(models.SessionFile).order_by(
models.SessionFile.session_id, models.SessionFile.id).all()
by_key: dict[tuple, list[models.SessionFile]] = {}
for f in rows:
by_key.setdefault((f.session_id, f.stored_path or f.filename), []).append(f)
dupes = {k: v for k, v in by_key.items() if len(v) > 1}
if not dupes:
print("No duplicated file rows found — nothing to do.")
return 0
affected_sessions: set[int] = set()
removed = 0
for (session_id, path), group in sorted(dupes.items()):
keep = group[-1] # newest row matches the bytes on disk
print(f"session {session_id}: '{keep.filename}' has {len(group)} rows "
f"-> keeping id {keep.id}, "
f"removing {[g.id for g in group if g.id != keep.id]}")
for g in group:
if g.id == keep.id:
continue
if args.apply:
db.delete(g)
removed += 1
affected_sessions.add(session_id)
if not args.apply:
print(f"\nDRY RUN: would remove {removed} duplicate row(s) across "
f"{len(affected_sessions)} closing(s). Re-run with --apply to fix.")
return 0
for sid in affected_sessions:
s = db.get(models.Session, sid)
if s is not None and s.status in ("processed", "blocked", "completed"):
s.needs_reprocess = True
db.commit()
print(f"\nRemoved {removed} duplicate row(s). {len(affected_sessions)} closing(s) "
f"flagged needs_reprocess — re-process them so the corrected totals persist.")
return 0
finally:
db.close()
def main(argv: list[str]) -> int:
ap = argparse.ArgumentParser(description="AR Aging admin commands")
sub = ap.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("add-user", help="create a user (prompts for password)")
p.add_argument("username")
p.add_argument("--name", default="", help="display name (lands in sign-off fields)")
p.set_defaults(fn=cmd_add_user)
p = sub.add_parser("set-password", help="reset a user's password")
p.add_argument("username")
p.set_defaults(fn=cmd_set_password)
p = sub.add_parser("list-users", help="list users")
p.set_defaults(fn=cmd_list_users)
p = sub.add_parser("deactivate-user", help="disable a user's login")
p.add_argument("username")
p.set_defaults(fn=cmd_deactivate_user)
p = sub.add_parser("set-admin", help="grant (or --revoke) audit-log access")
p.add_argument("username")
p.add_argument("--revoke", action="store_true", help="remove the admin flag")
p.set_defaults(fn=cmd_set_admin)
p = sub.add_parser("dedupe-files", help="fix double-counted duplicate upload rows")
p.add_argument("--apply", action="store_true", help="actually delete (default: dry run)")
p.set_defaults(fn=cmd_dedupe_files)
args = ap.parse_args(argv)
init_db()
return args.fn(args)
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))

389
ar-aging-app/backend/migrate_sqlite_to_mysql.py
View File

@ -1,389 +0,0 @@
#!/usr/bin/env python3
"""
Migrate the legacy SQLite database into MySQL.
The app stored everything in `backend/data/ar_aging.db` before the MySQL move. That file
still holds the real month-end closings (Jan-2026: 3,399,517 transaction rows), and MySQL
starts empty, so the closings have to be copied across once.
python3 migrate_sqlite_to_mysql.py --dry-run # inspect the source, touch nothing
python3 migrate_sqlite_to_mysql.py # migrate
python3 migrate_sqlite_to_mysql.py --force # migrate into a non-empty MySQL
What it does
* copies every table in foreign-key order, so a child row never precedes its session
* converts SQLite's text dates / 0-1 booleans to real MySQL DATE, DATETIME and BOOLEAN
* copies only the columns both schemas share, and reports any it had to skip
* streams in batches, so 3.4M rows never sit in memory
* verifies afterwards: row counts per table AND financial checksums (Σ transaction
totals, per-marketplace receivable) must match the source exactly
It never deletes anything from SQLite — the file is opened read-only.
"""
from __future__ import annotations
import argparse
import datetime as dt
import os
import sqlite3
import sys
from pathlib import Path
BACKEND = Path(__file__).resolve().parent
sys.path.insert(0, str(BACKEND))
BATCH = 5000
# Tables whose contents are re-derivable by re-processing, but copied anyway so the
# migrated database is byte-identical in what the dashboard shows.
SKIP_TABLES: set[str] = set()
def log(msg: str = "") -> None:
print(msg, flush=True)
# --------------------------------------------------------------------------- source
def sqlite_tables(conn: sqlite3.Connection) -> set[str]:
return {r[0] for r in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'")}
def sqlite_columns(conn: sqlite3.Connection, table: str) -> list[str]:
return [r[1] for r in conn.execute(f"PRAGMA table_info({table})")]
def sqlite_count(conn: sqlite3.Connection, table: str) -> int:
return conn.execute(f"SELECT COUNT(*) FROM `{table}`").fetchone()[0]
def open_sqlite(path: Path) -> sqlite3.Connection:
"""Open read-only. A stale -wal is checkpointed into a COPY, never the original."""
if not path.exists():
raise SystemExit(f"SQLite file not found: {path}")
# immutable=0 so an existing -wal is still applied; mode=ro keeps us from writing.
conn = sqlite3.connect(f"file:{path}?mode=ro", uri=True)
conn.row_factory = sqlite3.Row
return conn
# --------------------------------------------------------------------------- convert
def make_converter(col_type) -> "callable":
"""Return a function turning a SQLite value into something MySQL accepts."""
name = col_type.__class__.__name__
if name == "Date":
def conv(v):
if v in (None, ""):
return None
if isinstance(v, dt.date) and not isinstance(v, dt.datetime):
return v
if isinstance(v, dt.datetime):
return v.date()
try:
return dt.date.fromisoformat(str(v)[:10])
except ValueError:
return None
return conv
if name == "DateTime":
def conv(v):
if v in (None, ""):
return None
if isinstance(v, dt.datetime):
return v
s = str(v).replace("T", " ")
for fmt in ("%Y-%m-%d %H:%M:%S.%f", "%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
try:
return dt.datetime.strptime(s[:26], fmt)
except ValueError:
continue
return None
return conv
if name == "Boolean":
def conv(v):
if v is None:
return None
if isinstance(v, bool):
return v
return bool(int(v)) if str(v).strip() in ("0", "1") else bool(v)
return conv
if name in ("Integer", "BigInteger", "SmallInteger"):
def conv(v):
if v in (None, ""):
return None
try:
return int(v)
except (TypeError, ValueError):
return None
return conv
if name in ("Float", "Numeric"):
def conv(v):
if v in (None, ""):
return None
try:
return float(v)
except (TypeError, ValueError):
return None
return conv
# String / Text: MySQL columns are sized, so over-long values would be truncated or
# rejected. Trim to the declared length and report it rather than failing the batch.
length = getattr(col_type, "length", None)
def conv(v):
if v is None:
return None
s = v if isinstance(v, str) else str(v)
return s[:length] if length and len(s) > length else s
return conv
# --------------------------------------------------------------------------- migrate
def migrate() -> int:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--sqlite", default=str(BACKEND / "data" / "ar_aging.db"),
help="path to the legacy SQLite file")
ap.add_argument("--dry-run", action="store_true",
help="inspect the source and print the plan; do not connect to MySQL")
ap.add_argument("--force", action="store_true",
help="migrate even if the MySQL tables already contain rows")
args = ap.parse_args()
src_path = Path(args.sqlite)
src = open_sqlite(src_path)
have = sqlite_tables(src)
log("")
log(" Migrate SQLite → MySQL")
log(f" source: {src_path} ({src_path.stat().st_size / 1e9:.2f} GB)")
wal = src_path.with_name(src_path.name + "-wal")
if wal.exists() and wal.stat().st_size > 0:
log(f" note: a {wal.stat().st_size / 1e6:.0f} MB write-ahead log is present and "
f"will be read as part of the database")
log("")
# ---- source inventory (works with no MySQL at all) ----
log(" Source contents")
src_counts: dict[str, int] = {}
for t in sorted(have):
src_counts[t] = sqlite_count(src, t)
for t, n in sorted(src_counts.items(), key=lambda kv: -kv[1]):
if n:
log(f" {t:22} {n:>10,}")
empty = [t for t, n in src_counts.items() if not n]
if empty:
log(f" ({len(empty)} empty: {', '.join(sorted(empty))})")
src_checks = financial_checksums_sqlite(src)
log("")
log(" Financial checksums to preserve")
for k, v in src_checks.items():
log(f" {k:34} {v}")
if args.dry_run:
log("")
log(" Dry run — MySQL was not contacted and nothing was written.")
log(" Fill in ar-aging-app/.env, then re-run without --dry-run.")
return 0
# ---- target ----
try:
from app.db.database import ENGINE, init_db
from app.db import models # noqa: F401
except Exception as e: # noqa: BLE001
log("")
log(f" Could not connect to MySQL: {e}")
log(" Check MYSQL_* in ar-aging-app/.env and that the server is reachable.")
return 1
log("")
log(" Creating the MySQL schema (safe if it already exists)…")
init_db()
meta = models.Base.metadata
ordered = [t for t in meta.sorted_tables if t.name in have and t.name not in SKIP_TABLES]
missing_in_sqlite = [t.name for t in meta.sorted_tables if t.name not in have]
if missing_in_sqlite:
log(f" tables absent from the SQLite file (created empty): "
f"{', '.join(missing_in_sqlite)}")
raw = ENGINE.raw_connection()
cur = raw.cursor()
# Existing rows?
non_empty = []
for t in ordered:
cur.execute(f"SELECT COUNT(*) FROM `{t.name}`")
n = cur.fetchone()[0]
if n:
non_empty.append((t.name, n))
if non_empty and not args.force:
log("")
log(" MySQL already contains data — refusing to migrate on top of it:")
for name, n in non_empty:
log(f" {name:22} {n:>10,} rows")
log("")
log(" Re-run with --force to add these rows anyway (duplicates are possible),")
log(" or empty the MySQL database first.")
return 1
log("")
log(" Copying tables (foreign-key order)")
cur.execute("SET FOREIGN_KEY_CHECKS=0")
cur.execute("SET UNIQUE_CHECKS=0")
truncated: list[str] = []
copied: dict[str, int] = {}
try:
for table in ordered:
name = table.name
total = src_counts.get(name, 0)
if not total:
copied[name] = 0
continue
sq_cols = set(sqlite_columns(src, name))
cols = [c for c in table.columns if c.name in sq_cols]
dropped = [c.name for c in table.columns if c.name not in sq_cols]
extra = sq_cols - {c.name for c in table.columns}
convs = [make_converter(c.type) for c in cols]
names = [c.name for c in cols]
placeholders = ", ".join(["%s"] * len(names))
collist = ", ".join(f"`{n}`" for n in names)
sql = f"INSERT INTO `{name}` ({collist}) VALUES ({placeholders})"
done = 0
batch: list[tuple] = []
for row in src.execute(f"SELECT {', '.join(f'`{n}`' for n in names)} "
f"FROM `{name}`"):
vals = []
for i, conv in enumerate(convs):
v = conv(row[i])
vals.append(v)
batch.append(tuple(vals))
if len(batch) >= BATCH:
cur.executemany(sql, batch)
raw.commit()
done += len(batch)
batch.clear()
if total > 50000:
pct = 100.0 * done / total
print(f" {name:22} {done:>10,} / {total:,} ({pct:5.1f}%)",
end="\r", flush=True)
if batch:
cur.executemany(sql, batch)
raw.commit()
done += len(batch)
copied[name] = done
note = ""
if dropped:
note += f" [not in source: {', '.join(dropped)}]"
if extra:
note += f" [source-only, skipped: {', '.join(sorted(extra))}]"
truncated.append(name)
print(" " * 78, end="\r")
log(f" {name:22} {done:>10,}{note}")
finally:
cur.execute("SET FOREIGN_KEY_CHECKS=1")
cur.execute("SET UNIQUE_CHECKS=1")
raw.commit()
# ---- verify ----
log("")
log(" Verifying")
ok = True
for name, n in sorted(copied.items()):
cur.execute(f"SELECT COUNT(*) FROM `{name}`")
got = cur.fetchone()[0]
want = src_counts.get(name, 0)
if got != want:
ok = False
log(f" ✗ {name:22} MySQL {got:,} != SQLite {want:,}")
if ok:
log(f" ✓ row counts match on all {len(copied)} tables")
dst_checks = financial_checksums_mysql(cur)
for k, want in src_checks.items():
got = dst_checks.get(k)
if str(got) != str(want):
ok = False
log(f" ✗ {k}: MySQL {got} != SQLite {want}")
if ok:
log(" ✓ financial checksums match")
cur.close()
raw.close()
src.close()
log("")
if ok:
log(" Migration complete. Start the dashboard (scripts/start.ps1 on Windows, "
"scripts/start.command on macOS, or docker compose in production).")
return 0
log(" Migration finished with MISMATCHES — do not rely on the MySQL data until")
log(" they are explained. The SQLite file is untouched.")
return 1
# --------------------------------------------------------------------------- checksums
def financial_checksums_sqlite(conn: sqlite3.Connection) -> dict[str, str]:
out: dict[str, str] = {}
tables = sqlite_tables(conn)
def one(sql: str, default="—") -> str:
try:
r = conn.execute(sql).fetchone()
return "—" if r is None or r[0] is None else str(r[0])
except sqlite3.Error:
return default
if "sessions" in tables:
out["sessions"] = one("SELECT COUNT(*) FROM sessions")
if "transactions" in tables:
out["transaction rows"] = one("SELECT COUNT(*) FROM transactions")
out["Σ transactions.total"] = one("SELECT ROUND(SUM(total),2) FROM transactions")
out["receivable-flagged rows"] = one(
"SELECT COUNT(*) FROM transactions WHERE receivable_flag=1")
if "receivable_results" in tables:
out["USA receivable_local (TOTAL)"] = one(
"SELECT ROUND(receivable_local) FROM receivable_results "
"WHERE marketplace='USA' AND account_type='TOTAL'")
out["Σ receivable_usd (TOTAL rows)"] = one(
"SELECT ROUND(SUM(receivable_usd),2) FROM receivable_results "
"WHERE account_type='TOTAL'")
return out
def financial_checksums_mysql(cur) -> dict[str, str]:
out: dict[str, str] = {}
def one(sql: str) -> str:
try:
cur.execute(sql)
r = cur.fetchone()
return "—" if r is None or r[0] is None else str(r[0])
except Exception: # noqa: BLE001
return "—"
out["sessions"] = one("SELECT COUNT(*) FROM sessions")
out["transaction rows"] = one("SELECT COUNT(*) FROM transactions")
out["Σ transactions.total"] = one("SELECT ROUND(SUM(total),2) FROM transactions")
out["receivable-flagged rows"] = one(
"SELECT COUNT(*) FROM transactions WHERE receivable_flag=1")
out["USA receivable_local (TOTAL)"] = one(
"SELECT ROUND(receivable_local) FROM receivable_results "
"WHERE marketplace='USA' AND account_type='TOTAL'")
out["Σ receivable_usd (TOTAL rows)"] = one(
"SELECT ROUND(SUM(receivable_usd),2) FROM receivable_results "
"WHERE account_type='TOTAL'")
return out
if __name__ == "__main__":
try:
raise SystemExit(migrate())
except KeyboardInterrupt:
log("\n Interrupted. The SQLite source is unchanged.")
raise SystemExit(130)

2
ar-aging-app/backend/requirements.txt
View File

@ -16,8 +16,6 @@ SQLAlchemy==2.0.36
PyMySQL==1.1.1
cryptography>=42.0.0
python-dotenv==1.0.1
# CA bundle for the FX-rate fetch — the Windows OS cert store is unreliable on some machines
certifi>=2024.2.2
# Data helpers (optional / analysis)
pandas>=2.2

46
ar-aging-app/backend/tests/conftest.py
View File

@ -8,30 +8,20 @@ from pathlib import Path
import pytest
# ---------------------------------------------------------------------------------------
# Redirect ALL test data away from production — BEFORE anything imports app.config, which
# reads these variables once at module load and caches them.
# Redirect ALL test data to a throwaway directory — BEFORE anything imports app.config,
# which reads these variables once at module load and caches the paths.
#
# The suite creates AND DELETES closings, so pointing it at the live database would destroy
# Finance's data. That already happened once under SQLite (75 test sessions accumulated in
# the production file), and the blast radius is larger now that the store is a shared MySQL
# server rather than a local file.
# Without this the suite runs against the real production database: `app/config.py` falls
# back to `backend/data/ar_aging.db`, so every test that created a closing was writing into
# Finance's live data (75 sessions had accumulated there). Tests must never be able to touch
# a real closing.
#
# `load_dotenv()` in app/config.py does not override variables already present in the
# environment, so setting MYSQL_DATABASE here wins over .env. The database is created
# automatically by database._ensure_database().
# The names must match app/config.py exactly — AR_DB_PATH / AR_DATA_DIR. A near-miss such as
# "AR_DB_URL" silently does nothing and the tests quietly hit production again.
# ---------------------------------------------------------------------------------------
_TEST_DATA_DIR = Path(tempfile.mkdtemp(prefix="ar-aging-tests-"))
os.environ["AR_DATA_DIR"] = str(_TEST_DATA_DIR)
# Processing auto-fetches daily FX rates from the provider (jobs.FX_AUTO_DAILY); tests
# must never touch the network, so the automatic fetch is forced off for the whole suite.
# The FX tests exercise seeding explicitly through a mocked HTTP layer — including one
# integration test that re-enables the flag with monkeypatch (test_fx_service.py).
os.environ["AR_FX_AUTO_DAILY"] = "0"
_PROD_DB = os.environ.get("MYSQL_DATABASE", "")
TEST_DB_NAME = os.environ.get("AR_TEST_MYSQL_DATABASE", "ar_aging_pytest")
os.environ["MYSQL_DATABASE"] = TEST_DB_NAME
os.environ["AR_DB_PATH"] = str(_TEST_DATA_DIR / "test.db")
# Default: the project root two levels above ar-aging-app/backend.
_DEFAULT_SAMPLE_DIR = Path(__file__).resolve().parents[3]
@ -68,19 +58,13 @@ def _never_touch_production_data():
"""
Hard stop if the redirect above ever fails.
The suite creates and deletes closings, so running against the live database would
destroy Finance's data. Assert the isolation actually took effect rather than trusting
it — this fixture is the reason a renamed config variable can't silently re-point the
tests at production.
The suite creates and deletes closings, so pointing at the real database would destroy
Finance's data. Assert the isolation actually took effect rather than trusting it.
"""
from app.config import DATA_DIR, MYSQL_DATABASE
assert MYSQL_DATABASE == TEST_DB_NAME, (
f"tests are pointed at MySQL database {MYSQL_DATABASE!r} — expected "
f"{TEST_DB_NAME!r}. app/config.py reads MYSQL_DATABASE; check that name."
)
assert not _PROD_DB or MYSQL_DATABASE != _PROD_DB, (
f"the test database is the same as the configured production database "
f"({_PROD_DB!r}). Set AR_TEST_MYSQL_DATABASE to a separate name."
from app.config import DATA_DIR, DB_PATH
assert str(DB_PATH).startswith(str(_TEST_DATA_DIR)), (
f"tests are pointed at {DB_PATH} — expected a temp path under {_TEST_DATA_DIR}. "
f"app/config.py reads AR_DB_PATH / AR_DATA_DIR; check those names."
)
assert str(DATA_DIR).startswith(str(_TEST_DATA_DIR)), (
f"tests would write uploads/exports to {DATA_DIR}, not a temp directory."

7
ar-aging-app/backend/tests/test_api.py
View File

@ -24,14 +24,12 @@ def test_full_api_flow():
sid = c.post("/api/sessions", json={
"name": "test", "month_end_date": "2026-01-31", "clearing_lag_days": 2,
"allow_duplicate": True, # suite shares one DB; the guard has its own test
}).json()["id"]
with open(synth, "rb") as fh:
up = c.post(f"/api/sessions/{sid}/files", files={"files": ("USA synthetic.xlsx", fh)})
assert up.status_code == 200
assert up.json()["files"][0]["status"] == "parsed"
assert up.json()["skipped"] == []
assert up.json()[0]["status"] == "parsed"
c.put(f"/api/sessions/{sid}/reserves",
json=[{"marketplace": "USA", "account_type": "Standard Orders", "amount": 0.0}])
@ -69,8 +67,7 @@ def test_invalid_file_rejected():
with open(bad, "w") as f:
f.write("not a spreadsheet")
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "bad", "month_end_date": "2026-01-31",
"allow_duplicate": True}).json()["id"]
sid = c.post("/api/sessions", json={"name": "bad", "month_end_date": "2026-01-31"}).json()["id"]
with open(bad, "rb") as fh:
r = c.post(f"/api/sessions/{sid}/files", files={"files": ("bad.txt", fh)})
assert r.status_code == 400 # unsupported extension

114
ar-aging-app/backend/tests/test_audit.py
View File

@ -1,114 +0,0 @@
"""Audit trail: business actions land in audit_log attributed to the verified user, the
history survives a closing's deletion, and /api/audit is readable by admins only."""
from __future__ import annotations
import os
import tempfile
import pytest
from fastapi.testclient import TestClient
from app.api import auth as auth_mod
from app.api.main import app
from app.db import models
from app.db.database import SessionLocal, init_db
from tests.test_excel_export import make_amazon_xlsx
_TMP = tempfile.mkdtemp(prefix="ar_audit_test_")
def _entries(session_id: int) -> list[models.AuditLog]:
db = SessionLocal()
try:
return (db.query(models.AuditLog)
.filter(models.AuditLog.session_id == session_id)
.order_by(models.AuditLog.id).all())
finally:
db.close()
@pytest.fixture()
def clean_users():
"""Users flip AR_AUTH=auto to 'required' for the WHOLE shared test DB — always remove
them again so the rest of the suite keeps running unauthenticated."""
init_db()
yield
db = SessionLocal()
try:
db.query(models.User).delete()
db.commit()
finally:
db.close()
auth_mod.invalidate_users_cache()
def _add_user(username: str, display: str, password: str, admin: bool = False) -> None:
db = SessionLocal()
try:
db.add(models.User(username=username, display_name=display,
password_hash=auth_mod.hash_password(password),
is_active=True, is_admin=admin))
db.commit()
finally:
db.close()
auth_mod.invalidate_users_cache()
def test_actions_recorded_and_history_survives_delete():
init_db()
path = os.path.join(_TMP, "USA audit.xlsx")
make_amazon_xlsx(path, order_rows=4)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={
"name": "audit-me", "month_end_date": "2027-03-31",
"allow_duplicate": True}).json()["id"]
with open(path, "rb") as fh:
r = c.post(f"/api/sessions/{sid}/files",
files={"files": ("USA audit.xlsx", fh)})
assert r.status_code == 200
rows = _entries(sid)
assert [e.action for e in rows][:2] == ["session_create", "file_upload"]
assert "USA audit.xlsx" in rows[1].detail
# With auth off (no users) the row is still written, just unattributed.
assert rows[1].username == ""
# Deleting the closing records the deletion and KEEPS the history (no FK).
assert c.delete(f"/api/sessions/{sid}").status_code == 200
actions = [e.action for e in _entries(sid)]
assert "session_delete" in actions and "session_create" in actions
def test_audit_endpoint_admin_only_and_logins_attributed(clean_users):
_add_user("admin@x.com", "Admin A", "pw-longenough", admin=True)
_add_user("user@x.com", "User U", "pw-longenough2")
with TestClient(app) as c:
tok_admin = c.post("/api/auth/login", json={
"username": "admin@x.com", "password": "pw-longenough"}).json()["token"]
tok_user = c.post("/api/auth/login", json={
"username": "user@x.com", "password": "pw-longenough2"}).json()["token"]
assert c.get("/api/audit").status_code == 401 # not signed in
r = c.get("/api/audit", headers={"Authorization": f"Bearer {tok_user}"})
assert r.status_code == 403 # not an admin
r = c.get("/api/audit", headers={"Authorization": f"Bearer {tok_admin}"})
assert r.status_code == 200
logins = [e for e in r.json()["entries"] if e["action"] == "login"]
assert {e["username"] for e in logins} >= {"admin@x.com", "user@x.com"}
def test_login_and_me_carry_admin_flag(clean_users):
_add_user("admin2@x.com", "Admin B", "pw-longenough", admin=True)
_add_user("user2@x.com", "User V", "pw-longenough2")
with TestClient(app) as c:
res = c.post("/api/auth/login", json={
"username": "admin2@x.com", "password": "pw-longenough"}).json()
assert res["user"]["is_admin"] is True
me = c.get("/api/auth/me",
headers={"Authorization": f"Bearer {res['token']}"}).json()
assert me["is_admin"] is True
res = c.post("/api/auth/login", json={
"username": "user2@x.com", "password": "pw-longenough2"}).json()
assert res["user"]["is_admin"] is False

227
ar-aging-app/backend/tests/test_auth.py
View File

@ -1,227 +0,0 @@
"""Login: AR_AUTH=auto turns on with the first user; identity feeds sign-off fields."""
from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from app.api import auth as auth_mod
from app.api.main import app
from app.db import models
from app.db.database import SessionLocal, init_db
@pytest.fixture()
def clean_users():
"""Users flip AR_AUTH=auto to 'required' for the WHOLE shared test DB — always remove
them again so the rest of the suite keeps running unauthenticated."""
init_db()
yield
db = SessionLocal()
try:
db.query(models.User).delete()
db.commit()
finally:
db.close()
auth_mod.invalidate_users_cache()
def _add_user(username: str, display: str, password: str) -> None:
db = SessionLocal()
try:
db.add(models.User(username=username, display_name=display,
password_hash=auth_mod.hash_password(password), is_active=True))
db.commit()
finally:
db.close()
auth_mod.invalidate_users_cache()
def test_password_hash_roundtrip():
h = auth_mod.hash_password("s3cret-pw!")
assert h.startswith("scrypt$")
assert auth_mod.verify_password("s3cret-pw!", h)
assert not auth_mod.verify_password("wrong", h)
assert not auth_mod.verify_password("s3cret-pw!", "garbage")
def test_token_roundtrip_and_tamper():
user = models.User(id=7, username="jane", display_name="Jane D",
password_hash="x", is_active=True)
tok = auth_mod.create_token(user)
data = auth_mod.parse_token(tok)
assert data and data["uid"] == 7 and data["dn"] == "Jane D"
payload, sig = tok.split(".")
assert auth_mod.parse_token(f"{payload}x.{sig}") is None # tampered payload
assert auth_mod.parse_token(f"{payload}.{sig[:-2]}aa") is None # tampered signature
def test_api_open_with_no_users(clean_users):
auth_mod.invalidate_users_cache()
with TestClient(app) as c:
assert c.get("/api/auth/status").json()["auth_required"] is False
assert c.get("/api/sessions").status_code == 200
assert c.get("/api/auth/me").json()["authenticated"] is False
def test_first_user_turns_auth_on_and_login_works(clean_users):
_add_user("talha", "Talha Ahmed", "correct-horse-9")
with TestClient(app) as c:
assert c.get("/api/auth/status").json()["auth_required"] is True
# Locked out without a token; health stays open for probes.
assert c.get("/api/sessions").status_code == 401
assert c.get("/api/health").status_code == 200
assert c.post("/api/auth/login", json={
"username": "talha", "password": "nope"}).status_code == 401
assert c.post("/api/auth/login", json={
"username": "ghost", "password": "correct-horse-9"}).status_code == 401
r = c.post("/api/auth/login", json={"username": "TALHA", # case-insensitive
"password": "correct-horse-9"})
assert r.status_code == 200
token = r.json()["token"]
assert r.json()["user"]["display_name"] == "Talha Ahmed"
hdr = {"Authorization": f"Bearer {token}"}
assert c.get("/api/sessions", headers=hdr).status_code == 200
me = c.get("/api/auth/me", headers=hdr).json()
assert me["authenticated"] and me["display_name"] == "Talha Ahmed"
def test_signed_in_identity_overrides_body_name(clean_users):
"""Accountability fields record the VERIFIED identity, not whatever the body claims."""
_add_user("ayesha", "Ayesha K", "another-pw-123")
with TestClient(app) as c:
token = c.post("/api/auth/login", json={
"username": "ayesha", "password": "another-pw-123"}).json()["token"]
hdr = {"Authorization": f"Bearer {token}"}
sid = c.post("/api/sessions", json={"name": "identity", "month_end_date": "2029-01-31",
"allow_duplicate": True}, headers=hdr).json()["id"]
r = c.post(f"/api/sessions/{sid}/reconciliation-control/verify",
json={"verified_by": "Somebody Else", "comment": "spoof attempt"},
headers=hdr)
assert r.status_code == 200
db = SessionLocal()
try:
fc = db.query(models.FinanceControl).filter_by(session_id=sid).first()
assert fc.verified_by == "Ayesha K" # not "Somebody Else"
finally:
db.close()
c.delete(f"/api/sessions/{sid}", headers=hdr)
def test_change_password_flow(clean_users):
_add_user("changer", "Change Person", "old-password-1")
with TestClient(app) as c:
token = c.post("/api/auth/login", json={
"username": "changer", "password": "old-password-1"}).json()["token"]
hdr = {"Authorization": f"Bearer {token}"}
# Wrong current password / too short / unchanged are all refused.
assert c.post("/api/auth/change-password", headers=hdr, json={
"current_password": "nope", "new_password": "new-password-2"}).status_code == 400
assert c.post("/api/auth/change-password", headers=hdr, json={
"current_password": "old-password-1", "new_password": "short"}).status_code == 400
assert c.post("/api/auth/change-password", headers=hdr, json={
"current_password": "old-password-1",
"new_password": "old-password-1"}).status_code == 400
# Not signed in -> refused.
assert c.post("/api/auth/change-password", json={
"current_password": "old-password-1",
"new_password": "new-password-2"}).status_code == 401
r = c.post("/api/auth/change-password", headers=hdr, json={
"current_password": "old-password-1", "new_password": "new-password-2"})
assert r.status_code == 200 and r.json()["changed"] is True
# Old password dead, new one works, existing token still valid until expiry.
assert c.post("/api/auth/login", json={
"username": "changer", "password": "old-password-1"}).status_code == 401
assert c.post("/api/auth/login", json={
"username": "changer", "password": "new-password-2"}).status_code == 200
assert c.get("/api/auth/me", headers=hdr).status_code == 200
def test_email_code_reset_flow(clean_users, monkeypatch):
"""Emailed 6-digit code: request -> reset. Mailer mocked; email config forced on."""
from app.services import mailer
from app.api import auth as auth_module
sent: dict = {}
def fake_send(to, code, minutes):
sent["to"], sent["code"] = to, code
monkeypatch.setattr("app.config.email_enabled", lambda: True)
monkeypatch.setattr(mailer, "send_password_code", fake_send)
_add_user("coder@utopiabrands.com", "Code Person", "first-password-1")
with TestClient(app) as c:
# Unknown account: explicit 404 (deliberate for this small internal team), no email.
r = c.post("/api/auth/request-code", json={"username": "ghost@utopiabrands.com"})
assert r.status_code == 404 and "code" not in sent
assert "registered" in r.json()["detail"]
r = c.post("/api/auth/request-code", json={"username": "coder@utopiabrands.com"})
assert r.status_code == 200
assert sent["to"] == "coder@utopiabrands.com" and len(sent["code"]) == 6
# Immediate resend is throttled.
assert c.post("/api/auth/request-code",
json={"username": "coder@utopiabrands.com"}).status_code == 429
# Wrong code refused; attempts count up.
bad = "000000" if sent["code"] != "000000" else "111111"
assert c.post("/api/auth/reset-password", json={
"username": "coder@utopiabrands.com", "code": bad,
"new_password": "second-password-2"}).status_code == 400
# Step-2 verify: wrong code 400, right code valid — and NOT consumed by verifying.
assert c.post("/api/auth/verify-code", json={
"username": "coder@utopiabrands.com", "code": bad}).status_code == 400
r = c.post("/api/auth/verify-code", json={
"username": "coder@utopiabrands.com", "code": sent["code"]})
assert r.status_code == 200 and r.json()["valid"] is True
# Right code sets the new password and is single-use.
r = c.post("/api/auth/reset-password", json={
"username": "coder@utopiabrands.com", "code": sent["code"],
"new_password": "second-password-2"})
assert r.status_code == 200 and r.json()["changed"] is True
assert c.post("/api/auth/reset-password", json={
"username": "coder@utopiabrands.com", "code": sent["code"],
"new_password": "third-password-3"}).status_code == 400
assert c.post("/api/auth/login", json={
"username": "coder@utopiabrands.com",
"password": "first-password-1"}).status_code == 401
assert c.post("/api/auth/login", json={
"username": "coder@utopiabrands.com",
"password": "second-password-2"}).status_code == 200
assert auth_module.CODE_MAX_ATTEMPTS >= 3 # sanity: lockout exists
def test_request_code_without_email_configured(clean_users, monkeypatch):
# Force the unconfigured state — the dev .env may carry real mail settings.
monkeypatch.setattr("app.config.email_enabled", lambda: False)
_add_user("noemail@utopiabrands.com", "No Email", "some-password-1")
with TestClient(app) as c:
r = c.post("/api/auth/request-code", json={"username": "noemail@utopiabrands.com"})
assert r.status_code == 503
assert "administrator" in r.json()["detail"]
def test_inactive_user_cannot_login(clean_users):
_add_user("gone", "Gone Person", "some-pw-12345")
db = SessionLocal()
try:
db.query(models.User).filter_by(username="gone").first().is_active = False
db.commit()
finally:
db.close()
auth_mod.invalidate_users_cache()
with TestClient(app) as c:
assert c.post("/api/auth/login", json={
"username": "gone", "password": "some-pw-12345"}).status_code == 401

227
ar-aging-app/backend/tests/test_bank_import.py
View File

@ -1,227 +0,0 @@
"""
Bank disbursements import: parsing the bank workbook, matching deposits to payouts, and
the read-only import endpoint + apply-via-PUT flow.
Bank fixture mirrors the real file: sheet "Payouts" with
Company Link | Type | B. Acc | FCY | Date | Month | Text | Debit | Credit | Net | Party Name
Amazon fixture (make_amazon_xlsx, month-end 2026-01-31): USA transfers
sid 200 (Jan 6, -1000, Standard) · sid 300 (Jan 30, -2000, Standard)
sid 250 (Jan 12, -300, Invoiced)
"""
from __future__ import annotations
import datetime as dt
import os
import tempfile
import openpyxl
from fastapi.testclient import TestClient
from app.api.main import app
from app.core.bank_import import BankRow, match_payouts, parse_disbursements
from app.db.database import init_db
from tests.test_excel_export import make_amazon_xlsx
_TMP = tempfile.mkdtemp(prefix="ar_bank_import_test_")
_HEADERS = ["Company Link", "Type", "B. Acc", "FCY", "Date", "Month", "Text",
"Debit", "Credit", "Net", "Party Name"]
def make_disbursements_xlsx(path: str, rows: list[tuple], sheet: str = "Payouts") -> None:
"""rows: (party, fcy, date, debit) or (party, fcy, date, debit, type)."""
wb = openpyxl.Workbook()
ws = wb.active
ws.title = sheet
ws.append(_HEADERS)
for r in rows:
party, fcy, date, debit = r[:4]
row_type = r[4] if len(r) > 4 else "Deposit"
ws.append(["Utopia Brands Inc.", row_type, "5887", fcy, date, None,
"ORIG CO NAME=Amazon", debit, 0, debit, party])
wb.save(path)
def _read(path: str) -> bytes:
with open(path, "rb") as fh:
return fh.read()
# --------------------------------------------------------------------------- parsing
def test_parse_disbursements_dates_and_types():
p = os.path.join(_TMP, "parse.xlsx")
make_disbursements_xlsx(p, [
("Amazon US", "USD", dt.datetime(2026, 1, 8, 0, 0), 1000.0),
("Amazon UK", "GBP", # raw Excel serial float
float((dt.date(2026, 1, 13) - dt.date(1899, 12, 30)).days), 250.0),
("Amazon US", "USD", dt.datetime(2026, 1, 9), 50.0, "Charge"), # non-deposit: skipped
])
rows, problems = parse_disbursements(_read(p))
assert problems == []
assert [(r.marketplace, r.bank_date, r.debit) for r in rows] == [
("USA", dt.date(2026, 1, 8), 1000.0),
("UK", dt.date(2026, 1, 13), 250.0),
]
def test_parse_rejects_wrong_workbook():
p = os.path.join(_TMP, "wrong.xlsx")
wb = openpyxl.Workbook()
wb.active.append(["Just", "Some", "Columns"])
wb.save(p)
try:
parse_disbursements(_read(p))
assert False, "expected BankImportError"
except ValueError as e:
assert "No disbursements sheet" in str(e)
# --------------------------------------------------------------------------- matching
def _row(party, mkt, date, debit, currency="USD", sheet_row=2):
return BankRow(sheet_row=sheet_row, party=party, marketplace=mkt, currency=currency,
bank_date=date, narrative="", debit=debit, credit=0.0, net=debit)
_PAYOUT = {"marketplace": "USA", "account_type": "Standard Orders", "settlement_id": "200",
"amazon_date": dt.date(2026, 1, 6), "amount": -1000.0}
_MONTH_END = dt.date(2026, 1, 31)
_CCY = {"USA": "USD", "Sweden": "SEK", "Australia": "AUD"}
def test_match_exact_amount_and_window():
m = match_payouts([_row("Amazon US", "USA", dt.date(2026, 1, 8), 1000.0)],
[_PAYOUT], _MONTH_END, currency_by_marketplace=_CCY)
assert len(m.matched) == 1 and not m.ambiguous and not m.unmatched
got = m.matched[0]
assert got["settlement_id"] == "200" and got["amount_checked"] and got["delta"] == 0.0
assert got["bank_date"] == "2026-01-08" and not got["already_had_receipt"]
def test_party_mapping_case_insensitive_and_unknown():
sweden = {"marketplace": "Sweden", "account_type": "(unspecified)", "settlement_id": "9",
"amazon_date": dt.date(2026, 1, 10), "amount": -70.0}
rows, _ = parse_disbursements(_read(_mk("party.xlsx", [
("Amazon sweden", "SEK", dt.datetime(2026, 1, 12), 70.0),
("Some Vendor", "USD", dt.datetime(2026, 1, 12), 10.0),
])))
m = match_payouts(rows, [sweden], _MONTH_END, currency_by_marketplace=_CCY)
assert len(m.matched) == 1 and m.matched[0]["marketplace"] == "Sweden"
assert len(m.unknown_party) == 1 and m.unknown_party[0]["party"] == "Some Vendor"
def _mk(name: str, rows: list[tuple]) -> str:
p = os.path.join(_TMP, name)
make_disbursements_xlsx(p, rows)
return p
def test_match_ambiguous_unmatched_and_out_of_scope():
twin_a = dict(_PAYOUT, settlement_id="201")
twin_b = dict(_PAYOUT, settlement_id="202")
m = match_payouts([
_row("Amazon US", "USA", dt.date(2026, 1, 8), 1000.0, sheet_row=2), # two equal payouts
_row("Amazon US", "USA", dt.date(2026, 1, 25), 555.0, sheet_row=3), # nothing near
_row("Amazon US", "USA", dt.date(2026, 5, 12), 94.42, sheet_row=4), # other month
], [twin_a, twin_b], _MONTH_END, currency_by_marketplace=_CCY)
assert not m.matched
assert len(m.ambiguous) == 1 and len(m.ambiguous[0]["candidates"]) == 2
assert len(m.unmatched) == 1 and m.unmatched[0]["bank_row"] == 3
assert m.out_of_scope == 1
def test_one_to_one_consumption():
m = match_payouts([
_row("Amazon US", "USA", dt.date(2026, 1, 8), 1000.0, sheet_row=2),
_row("Amazon US", "USA", dt.date(2026, 1, 9), 1000.0, sheet_row=3), # same payout again
], [_PAYOUT], _MONTH_END, currency_by_marketplace=_CCY)
assert len(m.matched) == 1 and m.matched[0]["bank_row"] == 2
assert len(m.ambiguous) == 1 and "already matched by row 2" in m.ambiguous[0]["reason"]
def test_currency_mismatch_matches_by_date_only():
au = {"marketplace": "Australia", "account_type": "(unspecified)", "settlement_id": "77",
"amazon_date": dt.date(2026, 1, 10), "amount": -140.0} # AUD
m = match_payouts([_row("Amazon Australia", "Australia", dt.date(2026, 1, 13), 94.42)],
[au], _MONTH_END, currency_by_marketplace=_CCY) # bank row is USD
assert len(m.matched) == 1
got = m.matched[0]
assert got["amount_checked"] is False and got["delta"] is None
assert "USD" in got["note"] # FCY amount preserved in the note, not bank_amount
# --------------------------------------------------------------------------- endpoint
def _fresh(c, name: str) -> int:
sid = c.post("/api/sessions", json={
"name": name, "reporting_month": "2026-01",
"month_end_date": "2026-01-31", "clearing_lag_days": 2,
"allow_duplicate": True,
}).json()["id"]
path = os.path.join(_TMP, f"USA {name}.xlsx")
make_amazon_xlsx(path, order_rows=4)
with open(path, "rb") as fh:
assert c.post(f"/api/sessions/{sid}/files",
files={"files": (os.path.basename(path), fh)}).status_code == 200
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
return sid
def test_import_endpoint_end_to_end():
init_db()
with TestClient(app) as c:
sid = _fresh(c, "bank import e2e")
bank = _mk("e2e.xlsx", [
("Amazon US", "USD", dt.datetime(2026, 1, 8), 1000.0), # -> sid 200 (Standard)
("Amazon US", "USD", dt.datetime(2026, 1, 14), 300.0), # -> sid 250 (Invoiced)
("Amazon US", "USD", dt.datetime(2026, 2, 3), 2000.0), # -> sid 300 (Standard)
("Amazon US", "USD", dt.datetime(2026, 6, 20), 461.77), # other month
])
with open(bank, "rb") as fh:
r = c.post(f"/api/sessions/{sid}/payouts/receipts/import",
files={"file": ("bank.xlsx", fh)})
assert r.status_code == 200, r.text
res = r.json()
assert res["total_rows"] == 4 and res["out_of_scope"] == 1
assert {m["settlement_id"]: m["bank_date"] for m in res["matched"]} == {
"200": "2026-01-08", "250": "2026-01-14", "300": "2026-02-03"}
assert all(m["amount_checked"] and m["delta"] == 0.0 for m in res["matched"])
# Apply through the existing PUT — the endpoint itself must not have written.
payload = [{"marketplace": m["marketplace"], "account_type": m["account_type"],
"settlement_id": m["settlement_id"], "bank_date": m["bank_date"],
"bank_amount": m["bank_amount"] if m["amount_checked"] else None,
"note": m["note"]} for m in res["matched"]]
put = c.put(f"/api/sessions/{sid}/payouts/receipts", json=payload).json()
assert put["saved"] == 3 and put["needs_reprocess"]
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
payouts = c.get(f"/api/sessions/{sid}/payouts").json()["payouts"]
by_sid = {p["settlement_id"]: p for p in payouts}
assert by_sid["200"]["received_now"] is True # bank Jan 8 <= month-end
assert by_sid["250"]["received_now"] is True
assert by_sid["300"]["received_now"] is False # bank Feb 3 > month-end
# Re-import: matches flagged as already having receipts (idempotent workflow).
with open(bank, "rb") as fh:
res2 = c.post(f"/api/sessions/{sid}/payouts/receipts/import",
files={"file": ("bank.xlsx", fh)}).json()
assert all(m["already_had_receipt"] for m in res2["matched"])
def test_import_rejects_wrong_file():
init_db()
with TestClient(app) as c:
sid = _fresh(c, "bank import reject")
r = c.post(f"/api/sessions/{sid}/payouts/receipts/import",
files={"file": ("bank.csv", b"a,b,c")})
assert r.status_code == 400
wrong = os.path.join(_TMP, "not-bank.xlsx")
wb = openpyxl.Workbook()
wb.active.append(["Random", "Header"])
wb.save(wrong)
with open(wrong, "rb") as fh:
r = c.post(f"/api/sessions/{sid}/payouts/receipts/import",
files={"file": ("not-bank.xlsx", fh)})
assert r.status_code == 400 and "disbursements" in r.json()["detail"]

109
ar-aging-app/backend/tests/test_csv_reader.py
View File

@ -1,109 +0,0 @@
"""CSV Amazon transaction reader — Belgium/FR June sample shape."""
from __future__ import annotations
from pathlib import Path
import pytest
from app.core.csv_reader import CsvReader, parse_amount
from app.core.readers import make_reader
def test_parse_amount_european():
assert parse_amount("13,49") == 13.49
assert parse_amount("1.234,56") == 1234.56
assert parse_amount("1,234.56") == 1234.56
assert parse_amount("-6,24") == -6.24
assert parse_amount("0") == 0.0
assert parse_amount("") == 0.0
def test_parse_amount_unicode_minus_and_spaces():
"""Sweden writes negatives with U+2212 and groups thousands with (narrow) NBSP.
float() rejects both, and the parser's fallback returned 0.0 — every negative
kronor amount (fees, taxes, transfers) of the Jan-2026 file silently vanished
while the positives kept adding up (control C2 caught the +165,810.53 drift)."""
assert parse_amount("−35,70") == -35.70 # −35,70
assert parse_amount("−78 690,40") == -78690.40 # −78 690,40 (NBSP)
assert parse_amount("−1 234,56") == -1234.56 # narrow NBSP thousands
assert parse_amount("1 234,56") == 1234.56
assert parse_amount("–6,24") == -6.24 # en dash used as minus
def test_second_amount_column_is_summed_not_dropped(tmp_path: Path):
"""Australia carries BOTH 'sales tax collected' and 'low value goods' (LVIG GST);
the row `total` includes both. Dropping the second column failed control C2 by its
sum. Amount-field collisions are summed; only non-amount collisions stay errors."""
body = (
'"preamble"\n'
'"date/time","settlement ID","type","order ID","sales tax collected",'
'"low value goods","total"\n'
'"1 Jan 2026 00:00:00 UTC","123","Order","o-1","10,00","-6,60","3,40"\n'
)
p = tmp_path / "2026JanMonthlyTransaction.csv"
p.write_text(body, encoding="utf-8-sig")
reader = CsvReader(str(p))
mapping = reader.detect()
assert mapping.sum_cols == {"sales_tax_collected": [("F", "low value goods")]}
assert not mapping.duplicate_fields
assert "F" not in mapping.unmapped
rows = list(reader.iter_records())
assert len(rows) == 1
assert rows[0]["sales_tax_collected"] == pytest.approx(3.40) # 10.00 + (-6.60)
assert rows[0]["total"] == 3.40
assert reader.file_meta.summed_fields == mapping.sum_cols
assert not reader.file_meta.unmapped_amount_sums
reader.close()
def test_make_reader_routes_csv(tmp_path: Path):
p = tmp_path / "sample.csv"
p.write_text(
"preamble\n"
"date/time,settlement id,type,total\n"
"1 Jun 2026 00:00:00 UTC,123,Order,10.00\n",
encoding="utf-8",
)
r = make_reader(str(p))
assert type(r).__name__ == "CsvReader"
def test_csv_reader_belgium_headers(tmp_path: Path):
# Minimal FR/BE Custom Unified Transaction CSV (preamble + header + 1 row).
body = (
'"Comprend les transactions Amazon Marketplace"\n'
'"Tous les montants sont en EUR, sauf indication contraire"\n'
'"date/heure","Identifiant du paiement","type","Numéro de la commande","SKU",'
'"description","quantité","site de vente","expédition","ville de la commande",'
'"état de la commande","commande postale","ventes de produits",'
'"crédits d’expédition","crédits d’emballage-cadeau","Total des réductions",'
'"taxe de ventes prélevée","Taxe Marketplace Facilitator","frais de vente",'
'"Frais pour le service Expédié par Amazon","autres frais de transaction",'
'"autres","total","Statut de la transaction","Date de délivrance de la transaction"\n'
'"31 mai 2026 22:00:44 UTC","27177484042","Commande","405-9354558-3629905",'
'"SKU1","desc","2","amazon.com.be","Amazon","Enines","","1350","29,74","0","0","0",'
'"6,24","-6,24","-4,68","-11,57","0","0","13,49","Effectuée","8 juin 2026 15:26:40 UTC"\n'
)
p = tmp_path / "2026JunMonthlyTransaction.csv"
p.write_text(body, encoding="utf-8-sig")
reader = CsvReader(str(p))
mapping = reader.detect()
assert not mapping.missing_required
assert reader.sheet_name == "CSV"
assert reader.file_meta.currency == "EUR"
rows = list(reader.iter_records())
assert len(rows) == 1
rec = rows[0]
assert rec["settlement_id"] == "27177484042"
assert rec["txn_type"] == "Commande"
assert rec["marketplace"] == "amazon.com.be"
assert rec["total"] == 13.49
assert rec["product_sales"] == 29.74
assert rec["_date"].isoformat() == "2026-05-31"
assert reader.file_meta.imported_rows == 1
reader.close()

81
ar-aging-app/backend/tests/test_engine_unit.py
View File

@ -4,7 +4,7 @@ from __future__ import annotations
from datetime import date
from app.core.dates import parse_amazon_date, parse_amazon_datetime
from app.core.column_map import build_mapping, normalize_header, resolve_field
from app.core.column_map import build_mapping, normalize_header
from app.core.settlements import aggregate, classify
from app.core.receivable import compute_receivable, classify_aging
@ -40,67 +40,6 @@ def test_column_mapping_missing_required():
assert "total" in m.missing_required
def test_reference_workbook_aliases_resolve():
"""Every header variant from the finance team's per-marketplace reference workbook
must resolve. Doubles as a collision guard: _ALIAS_TO_FIELD is first-wins, so an
alias later hijacked by an earlier field makes the expected mapping here fail."""
expected = [
# promotional rebates family
("Promotional Discounts", "promotional_rebates"),
("Total Discounts", "promotional_rebates"),
("promosyon indirimleri", "promotional_rebates"),
("Tax on Promotional Discounts", "promotional_rebates_tax"),
# shipping credits family
("Shipping Credit", "shipping_credits"),
("kargo kredileri", "shipping_credits"),
("Tax on Shipping Credit", "shipping_credits_tax"),
("Tax on Shipping Credits", "shipping_credits_tax"),
# gift wrap family
("Gift Wrap Credit", "gift_wrap_credits"),
("Tax on Gift Wrap Credit", "giftwrap_credits_tax"),
("Tax on Gift Wrap Credits", "giftwrap_credits_tax"),
# other amount columns
("Marketplace Withheld VAT", "marketplace_withheld_tax"),
("ürün satışları", "product_sales"),
("satış ücretleri", "selling_fees"),
("Amazon Lojistik ücretleri", "fba_fees"),
("diğer işlem ücretleri", "other_transaction_fees"),
("diğer", "other"),
# transaction release date translations
("Freigabedatum der Transaktion", "transaction_release_date"),
("Date de sortie de la transaction", "transaction_release_date"),
("Data di rilascio della transazione", "transaction_release_date"),
("Fecha de liberación de la transacción", "transaction_release_date"),
("Publicatiedatum van transactie", "transaction_release_date"),
("Data zrealizowania transakcji", "transaction_release_date"),
("Transaktionens utgivningsdatum", "transaction_release_date"),
("İşlem çıkış tarihi", "transaction_release_date"),
# transaction status translations
("Transactiestatus", "transaction_status"),
("Status transakcji", "transaction_status"),
("İşlem durumu", "transaction_status"),
# location / fulfillment variants
("Order State/Province", "order_state"),
("State/Province", "order_state"),
("Order Region/Province", "order_state"),
("Order Province/State", "order_state"),
("Order Region/Autonomous Community", "order_state"),
("sipariş durumu", "order_state"),
("Order Postal Code", "order_postal"),
("sipariş postası", "order_postal"),
("sipariş şehri", "order_city"),
("Shipping/Fulfillment", "fulfillment"),
("Fulfillment/Shipping", "fulfillment"),
("gönderim", "fulfillment"),
]
for header, want in expected:
assert resolve_field(header) == want, f"{header!r} -> {resolve_field(header)!r}, want {want!r}"
# Pre-existing aliases that must not be hijacked by the additions above.
assert resolve_field("shipping") == "shipping_credits"
assert resolve_field("Transaktionsstatus") == "transaction_status"
assert resolve_field("total des réductions") == "promotional_rebates"
# --------------------------------------------------------------------------- aging
def test_aging_bands():
assert classify_aging(None) == "Current"
@ -111,24 +50,6 @@ def test_aging_bands():
assert classify_aging(120) == "91-Over"
def test_aging_band_schemes():
from app.core.receivable import AGING_BANDS, aging_bands
assert aging_bands("monthly") == AGING_BANDS # default stays the classic bands
assert aging_bands("weekly") == ("Current", "1-7", "8-14", "15-21", "22-28", "29-Over")
assert aging_bands("half_year") == ("Current", "1-180", "181-360", "361-540", "541-Over")
assert aging_bands("yearly") == ("Current", "1-365", "366-730", "731-1095", "1096-Over")
assert aging_bands("nonsense") == AGING_BANDS # unknown scheme falls back
assert classify_aging(5, "weekly") == "1-7"
assert classify_aging(14, "weekly") == "8-14"
assert classify_aging(35, "weekly") == "29-Over"
assert classify_aging(120, "half_year") == "1-180"
assert classify_aging(400, "yearly") == "366-730"
assert classify_aging(2000, "yearly") == "1096-Over"
assert classify_aging(0, "weekly") == "Current"
# --------------------------------------------------------------------------- settlements
def _rec(total, ttype, acct, sid, d, mkt="USA"):
return {

84
ar-aging-app/backend/tests/test_excel_export.py
View File

@ -182,87 +182,3 @@ def test_sheet_split_when_over_row_limit(synth_file):
add = wb["Detail"]["B5"].value
for s in usa_sheets:
assert f"'{s}'!" in add
# ------------------------------------------------------------------ workbook must tie
def test_detail_references_every_account_stream(synth_file):
"""
Detail/Summary must reference EVERY account stream's subtotal cell.
compute_layouts() planned the subtotal rows with a stride of 2 while
_finalize_marketplace_subtotals() writes them consecutively, so every stream after the
first pointed at an empty cell. USA is the only marketplace with two streams, so the
whole Invoiced Orders receivable silently vanished from Detail and Summary (Jan-2026:
67,854.71) while Reconciliation and COA in the same workbook showed it.
"""
result = process([synth_file], month_end=date(2026, 1, 31), clearing_lag_days=2)
out = synth_file.replace(".xlsx", "-streams.xlsx")
export_workbook(result, [synth_file], out)
wb = openpyxl.load_workbook(out)
ws = wb["USA"]
# Where the subtotal formulas actually landed.
actual = {}
for row in ws.iter_rows():
for c in row:
if isinstance(c.value, str) and c.value.startswith("=SUMIFS"):
actual[ws.cell(row=c.row, column=c.column - 1).value] = c.coordinate
streams = set(result.receivable.marketplaces["USA"].accounts)
assert set(actual) == streams, f"a stream has no subtotal row: {actual} vs {streams}"
detail_formula = wb["Detail"]["B5"].value
for stream, coord in actual.items():
assert f"'USA'!{coord}" in detail_formula, (
f"Detail!B5 ({detail_formula}) does not reference the {stream} subtotal at {coord}"
)
def test_summary_net_receivable_is_not_circular(synth_file):
"""'Net Receivable' referenced its own cell, so Excel warned and showed 0."""
result = process([synth_file], month_end=date(2026, 1, 31), clearing_lag_days=2)
out = synth_file.replace(".xlsx", "-net.xlsx")
export_workbook(result, [synth_file], out, allowance_for_returns=-100.0)
ws = openpyxl.load_workbook(out)["Summary"]
rows = {ws.cell(row=r, column=1).value: r for r in range(1, ws.max_row + 1)}
net_row = rows["Net Receivable"]
formula = ws.cell(row=net_row, column=7).value
assert f"G{net_row}" not in formula, f"circular reference: G{net_row} in {formula}"
assert f"G{rows['TOTAL']}" in formula and f"G{rows['Allowance for Sales Returns']}" in formula
def test_every_received_payout_appears_on_the_tab(synth_file):
"""
A month can have several received payouts per stream; the workbook used to lift only the
boundary one, so earlier bank receipts were absent from the entire file and the tab could
not be hand-footed against the bank statement.
"""
result = process([synth_file], month_end=date(2026, 1, 31), clearing_lag_days=2)
out = synth_file.replace(".xlsx", "-payouts.xlsx")
export_workbook(result, [synth_file], out)
ws = openpyxl.load_workbook(out)["USA"]
in_book = sorted(r[TOTAL_IDX] for r in ws.iter_rows(min_row=9, values_only=True)
if r and r[FIELD_ORDER.index("txn_type")] == "Transfer")
received = sorted(t.amount for t in result.aggregation.transfers if t.received)
assert in_book == received, f"workbook payouts {in_book} != received payouts {received}"
def test_settled_settlements_sheet_reconciles(synth_file):
"""The excluded settlements are listed and their rows + the included rows = every row."""
result = process([synth_file], month_end=date(2026, 1, 31), clearing_lag_days=2)
out = synth_file.replace(".xlsx", "-settled.xlsx")
export_workbook(result, [synth_file], out)
ws = openpyxl.load_workbook(out)["Settled Settlements"]
labels = {ws.cell(row=r, column=1).value: r for r in range(1, ws.max_row + 1)}
excluded = ws.cell(row=labels["Excluded (settled) order rows"], column=6).value
included = ws.cell(row=labels["Included (open) order rows — in the marketplace tabs"],
column=6).value
total = ws.cell(row=labels["Total order rows in the source files"], column=6).value
assert excluded + included == total
engine_total = sum(st.row_count - st.transfer_count
for st in result.aggregation.settlements.values())
assert total == engine_total, f"sheet says {total} order rows, engine has {engine_total}"
# Every settled settlement is named, so the omission is documented rather than silent.
listed = {ws.cell(row=r, column=3).value for r in range(5, ws.max_row + 1)}
settled = {sid for (m, a, sid), st in result.aggregation.settlements.items()
if st.status != "receivable" and (st.row_count - st.transfer_count) > 0}
assert settled <= listed, f"settled settlements missing from the sheet: {settled - listed}"

7
ar-aging-app/backend/tests/test_frontend_guards.py
View File

@ -26,7 +26,7 @@ def _sources() -> list[Path]:
def test_no_native_browser_dialogs():
offenders: list[str] = []
for path in _sources():
for i, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
for i, line in enumerate(path.read_text().splitlines(), 1):
stripped = line.strip()
if stripped.startswith("//") or stripped.startswith("*"):
continue
@ -40,15 +40,14 @@ def test_no_native_browser_dialogs():
def test_confirm_dialog_component_exists():
# encoding pinned: sources are UTF-8; Windows' default cp1252 chokes on curly quotes
ui = (SRC / "components" / "ui.tsx").read_text(encoding="utf-8")
ui = (SRC / "components" / "ui.tsx").read_text()
assert "export function ConfirmDialog" in ui
def test_destructive_actions_use_confirm_dialog():
"""Anything calling deleteSession must route through the in-app dialog."""
for path in _sources():
text = path.read_text(encoding="utf-8")
text = path.read_text()
if "deleteSession" in text and "api.ts" not in path.name and "client.ts" not in path.name:
assert "ConfirmDialog" in text, (
f"{path.relative_to(SRC)} deletes a closing without <ConfirmDialog>"

105
ar-aging-app/backend/tests/test_fx_dashboard.py
View File

@ -1,105 +0,0 @@
"""
Dual-currency dashboard figures.
The AR Ledger shows every movement in the marketplace's local currency AND in USD,
converted at each TRANSACTION DATE's exchange rate (a daily override when one exists,
the marketplace month rate otherwise). The opening balance has no transaction date, so
it converts at the month rate.
Also the regression for the fx-daily endpoint: after payouts moved out of _daily_rows
into _payout_events, fx_daily still unpacked 4-tuples and crashed on every session with
data — the "Daily exchange rates" table never rendered.
Fixture dates (make_amazon_xlsx, USA, month-end 2026-01-31, lag 2 → cutoff Jan 29):
revenue: Jan 5 +1000 · Jan 10 +300 · Jan 15 +2000 · Jan 20 +80 · Jan 31 +500
payouts: Jan 6 −1000 (received) · Jan 12 −300 (received) · Jan 30 −2000 (in transit)
→ closing = 3880 − 1300 = 2580
"""
from __future__ import annotations
from fastapi.testclient import TestClient
from app.api.main import app
from app.db.database import init_db
from tests.test_payout_receipts import _fresh
def test_fx_daily_no_longer_crashes_and_covers_payout_dates():
init_db()
with TestClient(app) as c:
sid = _fresh(c, "fx daily regression")
r = c.get(f"/api/sessions/{sid}/fx-daily")
assert r.status_code == 200
data = r.json()
assert data["available"] is True
by_date = {row["date"]: row for row in data["rows"]}
# Revenue sits on its transaction date…
assert by_date["2026-01-15"]["revenue"] == 2000.0
# …and payouts on their effective date, in the same table.
assert by_date["2026-01-06"]["payouts"] == -1000.0
assert by_date["2026-01-30"]["payouts"] == -2000.0
# USA converts 1:1 — USD total equals local total = 3880 − 3300 net movement.
assert data["month_rate"] == 1.0
assert data["total_local"] == 580.0
assert data["total_usd"] == data["total_local"]
def test_ledger_detail_shows_usd_at_transaction_date_rates():
init_db()
with TestClient(app) as c:
sid = _fresh(c, "dual currency ledger")
# Daily override for the big revenue day; every other date uses the month rate.
assert c.put(f"/api/sessions/{sid}/fx-daily", json=[
{"marketplace": "USA", "rate_date": "2026-01-15", "rate": 1.25},
]).status_code == 200
d = c.get(f"/api/sessions/{sid}/ledger-detail").json()
per = {p["key"]: p for p in d["periods"]}
# Jan 15's revenue converts at ITS OWN day's rate…
assert per["2026-01-15"]["revenue"] == 2000.0
assert per["2026-01-15"]["revenue_usd"] == 2500.0
# …every other date at the month rate (1.0 for USA).
assert per["2026-01-05"]["revenue_usd"] == per["2026-01-05"]["revenue"] == 1000.0
assert per["2026-01-06"]["payouts_received_usd"] == -1000.0
assert per["2026-01-30"]["payouts_in_transit_usd"] == -2000.0
# Opening has no transaction date → month rate; the USD running balance then
# absorbs the daily-rate spread: closing_usd = closing + 2000 × (1.25 − 1).
assert d["month_rate"] == 1.0
assert d["opening_usd"] == 0.0
assert d["closing"] == 2580.0
assert d["closing_usd"] == 3080.0
assert d["in_transit_total_usd"] == -2000.0
# The local-currency figures are untouched by the daily override.
assert per["2026-01-15"]["balance"] == per["2026-01-15"]["balance_usd"] - 500.0
def test_weekend_transactions_use_the_previous_banking_days_fixing():
"""2026-01-10 is a Saturday — no fixing is published. The rate effective on it is the
previous banking day's PROVIDER fixing (Friday the 9th), not the month rate. Manual
rates never carry forward: they are deliberate single-date overrides (which is also
why the test above sees the month rate everywhere but Jan 15)."""
init_db()
with TestClient(app) as c:
sid = _fresh(c, "weekend carry forward")
assert c.put(f"/api/sessions/{sid}/fx-daily", json=[
{"marketplace": "USA", "rate_date": "2026-01-09", "rate": 1.5,
"source": "frankfurter"},
]).status_code == 200
d = c.get(f"/api/sessions/{sid}/ledger-detail").json()
per = {p["key"]: p for p in d["periods"]}
# Saturday's revenue converts at Friday's fixing: 300 × 1.5.
assert per["2026-01-10"]["revenue_usd"] == 450.0
fxd = c.get(f"/api/sessions/{sid}/fx-daily").json()
by_date = {r["date"]: r for r in fxd["rows"]}
assert by_date["2026-01-10"]["rate"] == 1.5
assert "2026-01-09" in by_date["2026-01-10"]["source"] # carry-forward disclosed
# Dates before the first fixing still fall back to the month rate.
assert by_date["2026-01-05"]["rate"] == 1.0
assert by_date["2026-01-05"]["source"] == "month rate"

283
ar-aging-app/backend/tests/test_fx_service.py
View File

@ -1,283 +0,0 @@
"""FX rate service: orientation (the #1 risk), confirmation withdrawal, caching, failure.
The provider is mocked — no network in tests. Frankfurter with base=USD returns LOCAL per
USD; the app stores USD per LOCAL (usd = local * rate), so the service must invert."""
from __future__ import annotations
import datetime as dt
import pytest
from fastapi.testclient import TestClient
from app.api.main import app
from app.db import models
from app.db.database import SessionLocal, init_db
from app.services import fx_service
@pytest.fixture()
def fake_frankfurter(monkeypatch):
"""Replace the HTTP layer with a fixture: 1 USD = 0.85 EUR on 2029-06-29 (Friday)."""
calls = {"n": 0, "urls": []}
def fake_get(url: str) -> dict:
calls["n"] += 1
calls["urls"].append(url)
if ".." in url: # time-series request
return {"base": "USD", "rates": {
"2029-06-28": {"EUR": 0.86},
"2029-06-29": {"EUR": 0.85},
}}
return {"base": "USD", "date": "2029-06-29", "rates": {"EUR": 0.85}}
monkeypatch.setattr(fx_service, "_http_get_json", fake_get)
return calls
def _session_with_fx(c, name: str, month_end: str) -> int:
sid = c.post("/api/sessions", json={"name": name, "month_end_date": month_end,
"allow_duplicate": True}).json()["id"]
db = SessionLocal()
try:
db.add(models.FxRate(session_id=sid, marketplace="Germany", currency="EUR",
rate=1.185665, source="default (Jan-26 workbook)",
confirmed_by="Old Confirmer", confirmed_month="2026-01"))
db.add(models.FxRate(session_id=sid, marketplace="USA", currency="USD", rate=1.0,
source="default"))
db.commit()
finally:
db.close()
return sid
def test_fetch_inverts_to_usd_per_local_and_clears_confirmation(fake_frankfurter):
init_db()
with TestClient(app) as c:
sid = _session_with_fx(c, "fx orient", "2029-06-30")
r = c.post(f"/api/sessions/{sid}/fx/fetch")
assert r.status_code == 200, r.text
body = r.json()
de = next(u for u in body["updated"] if u["marketplace"] == "Germany")
# 1 USD = 0.85 EUR -> 1 EUR = 1/0.85 USD. The inverse (0.85) would mis-state
# every EUR receivable — this assertion pins the orientation.
assert de["rate"] == pytest.approx(1 / 0.85, abs=1e-6)
usa = next(u for u in body["updated"] if u["marketplace"] == "USA")
assert usa["rate"] == 1.0
assert "frankfurter" in body["source"]
assert "2029-06-29" in body["source"] # the provider's banking day
rows = c.get(f"/api/sessions/{sid}/fx").json()
de_row = next(x for x in rows if x["marketplace"] == "Germany")
assert "frankfurter" in de_row["source"]
db = SessionLocal()
try:
fx = db.query(models.FxRate).filter_by(session_id=sid,
marketplace="Germany").first()
# A fetched rate is a suggestion: the old confirmation no longer applies (C5).
assert fx.confirmed_by == "" and fx.confirmed_month == ""
finally:
db.close()
def test_second_fetch_is_served_from_cache(fake_frankfurter):
init_db()
with TestClient(app) as c:
sid = _session_with_fx(c, "fx cache", "2029-06-30")
assert c.post(f"/api/sessions/{sid}/fx/fetch").status_code == 200
n_after_first = fake_frankfurter["n"]
r = c.post(f"/api/sessions/{sid}/fx/fetch")
assert r.status_code == 200
assert fake_frankfurter["n"] == n_after_first # no second HTTP call
assert "(cached)" in r.json()["source"]
def test_provider_failure_is_a_502_never_a_silent_default(monkeypatch):
init_db()
def boom(url: str) -> dict:
raise fx_service.FxProviderError("provider down")
monkeypatch.setattr(fx_service, "_http_get_json", boom)
with TestClient(app) as c:
sid = _session_with_fx(c, "fx down", "2029-08-31")
r = c.post(f"/api/sessions/{sid}/fx/fetch")
assert r.status_code == 502
assert "manually" in r.json()["detail"]
# The stored rate is untouched — not overwritten with anything.
de = next(x for x in c.get(f"/api/sessions/{sid}/fx").json()
if x["marketplace"] == "Germany")
assert de["rate"] == 1.185665
def test_fetch_without_processing_explains_the_precondition():
init_db()
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "fx bare", "month_end_date": "2029-09-30",
"allow_duplicate": True}).json()["id"]
r = c.post(f"/api/sessions/{sid}/fx/fetch")
assert r.status_code == 502
assert "process" in r.json()["detail"].lower()
def test_daily_fetch_fills_fx_rates_daily(fake_frankfurter):
init_db()
with TestClient(app) as c:
sid = _session_with_fx(c, "fx daily", "2029-06-30")
r = c.post(f"/api/sessions/{sid}/fx/fetch-daily",
json={"date_from": "2029-06-28", "date_to": "2029-06-29"})
assert r.status_code == 200, r.text
assert r.json()["saved"] == 2 # two banking days, EUR only
db = SessionLocal()
try:
rows = db.query(models.FxRateDaily).filter_by(
session_id=sid, marketplace="Germany").all()
by_date = {row.rate_date: row for row in rows}
assert by_date[dt.date(2029, 6, 29)].rate == pytest.approx(1 / 0.85, abs=1e-6)
assert by_date[dt.date(2029, 6, 28)].rate == pytest.approx(1 / 0.86, abs=1e-6)
assert all(row.source == "frankfurter" for row in rows)
finally:
db.close()
def test_daily_fetch_defaults_to_the_transaction_span(fake_frankfurter):
"""No explicit range → the provider is asked for the span the files actually cover
(earliest dated transaction through month-end), so pre-month rows convert at their
own date's rate too."""
init_db()
with TestClient(app) as c:
sid = _session_with_fx(c, "fx daily span", "2029-06-30")
db = SessionLocal()
try:
db.add(models.Transaction(session_id=sid, marketplace="Germany",
posted_date=dt.date(2029, 5, 20), total=100.0))
db.add(models.Transaction(session_id=sid, marketplace="Germany",
posted_date=dt.date(2029, 6, 12), total=50.0))
db.commit()
finally:
db.close()
r = c.post(f"/api/sessions/{sid}/fx/fetch-daily", json={})
assert r.status_code == 200, r.text
body = r.json()
assert body["date_from"] == "2029-05-20" # earliest dated transaction
assert body["date_to"] == "2029-06-30" # through month-end
series_url = next(u for u in fake_frankfurter["urls"] if ".." in u)
assert "2029-05-20..2029-06-30" in series_url
def test_auto_seed_preserves_manual_daily_overrides(fake_frankfurter):
"""The automatic post-processing seed refreshes provider rows but never clobbers a
rate a person typed; only the explicit Fetch button replaces manual overrides."""
init_db()
with TestClient(app) as c:
sid = _session_with_fx(c, "fx auto manual", "2029-06-30")
db = SessionLocal()
try:
db.add(models.FxRateDaily(session_id=sid, marketplace="Germany",
rate_date=dt.date(2029, 6, 29), rate=2.0,
source="manual"))
db.commit()
s = db.get(models.Session, sid)
out = fx_service.auto_seed_daily_fx(db, s)
assert "error" not in out
rows = {r.rate_date: r for r in db.query(models.FxRateDaily).filter_by(
session_id=sid, marketplace="Germany")}
assert rows[dt.date(2029, 6, 29)].rate == 2.0 # manual kept
assert rows[dt.date(2029, 6, 29)].source == "manual"
assert rows[dt.date(2029, 6, 28)].rate == pytest.approx(1 / 0.86, abs=1e-6)
assert rows[dt.date(2029, 6, 28)].source == "frankfurter"
finally:
db.close()
def test_auto_seed_is_advisory_when_the_provider_is_down(monkeypatch):
init_db()
def boom(url: str) -> dict:
raise fx_service.FxProviderError("provider down")
monkeypatch.setattr(fx_service, "_http_get_json", boom)
with TestClient(app) as c:
sid = _session_with_fx(c, "fx auto down", "2029-06-30")
db = SessionLocal()
try:
s = db.get(models.Session, sid)
out = fx_service.auto_seed_daily_fx(db, s) # must not raise
assert "provider down" in out["error"]
finally:
db.close()
def test_auto_seed_skips_usd_only_closings(monkeypatch):
"""A USD-only close has nothing to fetch — no HTTP request, no warning."""
init_db()
def no_network(url: str) -> dict:
raise AssertionError(f"unexpected FX fetch for a USD-only closing: {url}")
monkeypatch.setattr(fx_service, "_http_get_json", no_network)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "fx usd only",
"month_end_date": "2029-06-30",
"allow_duplicate": True}).json()["id"]
db = SessionLocal()
try:
db.add(models.FxRate(session_id=sid, marketplace="USA", currency="USD",
rate=1.0, source="default"))
db.commit()
s = db.get(models.Session, sid)
out = fx_service.auto_seed_daily_fx(db, s)
assert out.get("skipped")
finally:
db.close()
def test_processing_auto_seeds_daily_rates_from_the_api(tmp_path, monkeypatch):
"""End-to-end: processing fetches the provider's daily fixings for the file's span,
and the daily FX table converts each date at the rate effective on it — the exact
fixing when one exists, the previous banking day's fixing otherwise."""
init_db()
from app.services import jobs
from tests.test_multimarket import _make_dutch_file
monkeypatch.setattr(jobs, "FX_AUTO_DAILY", True)
urls: list[str] = []
def fake_get(url: str) -> dict:
urls.append(url)
assert ".." in url, "auto-seed must use a single series request"
return {"base": "USD", "rates": {
"2026-01-02": {"EUR": 0.8},
"2026-01-15": {"EUR": 0.9},
}}
monkeypatch.setattr(fx_service, "_http_get_json", fake_get)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={
"name": "auto daily fx", "reporting_month": "2026-01",
"month_end_date": "2026-01-31", "clearing_lag_days": 2,
"allow_duplicate": True}).json()["id"]
path = tmp_path / "Netherlands Amazon Transactions January, 2026.xlsx"
_make_dutch_file(str(path))
with open(path, "rb") as fh:
assert c.post(f"/api/sessions/{sid}/files",
files={"files": (path.name, fh)}).status_code == 200
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
assert c.get(f"/api/sessions/{sid}/status").json()["status"] in (
"processed", "blocked") # blocked = unconfirmed C5, fine
# One series request, widened to the whole reporting month.
assert any("2026-01-01..2026-01-31" in u for u in urls)
fxd = c.get(f"/api/sessions/{sid}/fx-daily?marketplace=Netherlands").json()
by_date = {r["date"]: r for r in fxd["rows"]}
# Jan 2 converts at Jan 2's fixing (1 USD = 0.80 EUR → 1.25 USD per EUR)…
assert by_date["2026-01-02"]["rate"] == pytest.approx(1.25, abs=1e-6)
assert by_date["2026-01-02"]["source"] == "frankfurter"
# …Jan 6 has no fixing, so the previous banking day's rate is in effect…
assert by_date["2026-01-06"]["rate"] == pytest.approx(1.25, abs=1e-6)
assert "2026-01-02" in by_date["2026-01-06"]["source"]
# …and Jan 20 carries Jan 15's fixing.
assert by_date["2026-01-20"]["rate"] == pytest.approx(1 / 0.9, abs=1e-6)
assert "2026-01-15" in by_date["2026-01-20"]["source"]

117
ar-aging-app/backend/tests/test_month_locking.py
View File

@ -1,117 +0,0 @@
"""One closing per month (guarded), completed closings are read-only, reopen unlocks."""
from __future__ import annotations
import os
import tempfile
from fastapi.testclient import TestClient
from app.api.main import app
from app.db import models
from app.db.database import SessionLocal, init_db
from tests.test_excel_export import make_amazon_xlsx
_TMP = tempfile.mkdtemp(prefix="ar_lock_test_")
def _processed(c, name: str, month_end: str) -> int:
sid = c.post("/api/sessions", json={"name": name, "month_end_date": month_end,
"allow_duplicate": True}).json()["id"]
path = os.path.join(_TMP, f"USA {name}.xlsx")
make_amazon_xlsx(path, order_rows=5)
with open(path, "rb") as fh:
assert c.post(f"/api/sessions/{sid}/files",
files={"files": (os.path.basename(path), fh)}).status_code == 200
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
return sid
def _force_complete(sid: int) -> None:
db = SessionLocal()
try:
db.get(models.Session, sid).status = "completed"
db.commit()
finally:
db.close()
def test_duplicate_month_is_refused_unless_explicit():
init_db()
with TestClient(app) as c:
first = c.post("/api/sessions", json={"name": "dup guard A",
"month_end_date": "2028-01-31"})
assert first.status_code == 200, first.text
again = c.post("/api/sessions", json={"name": "dup guard B",
"month_end_date": "2028-01-31"})
assert again.status_code == 409
assert "2028-01" in again.json()["detail"]
assert "dup guard A" in again.json()["detail"] # names the existing closing
forced = c.post("/api/sessions", json={"name": "dup guard C",
"month_end_date": "2028-01-31",
"allow_duplicate": True})
assert forced.status_code == 200
# The listing flags both sessions as sharing a month.
rows = c.get("/api/sessions").json()
flagged = [s for s in rows if s["reporting_month"] == "2028-01"]
assert len(flagged) == 2 and all(s["duplicate_month"] for s in flagged)
def test_listing_is_month_ordered_with_publish_flag():
init_db()
with TestClient(app) as c:
c.post("/api/sessions", json={"name": "older", "month_end_date": "2028-02-29",
"allow_duplicate": True})
c.post("/api/sessions", json={"name": "newer", "month_end_date": "2028-03-31",
"allow_duplicate": True})
rows = c.get("/api/sessions").json()
months = [s["reporting_month"] for s in rows if s["reporting_month"]]
assert months == sorted(months, reverse=True)
assert all("journal_approved" in s for s in rows)
def test_completed_closing_is_locked_and_reopen_unlocks():
init_db()
with TestClient(app) as c:
sid = _processed(c, "lock me", "2028-04-30")
_force_complete(sid)
# Every mutating surface answers 409 while completed.
path = os.path.join(_TMP, "USA lock me.xlsx")
with open(path, "rb") as fh:
up = c.post(f"/api/sessions/{sid}/files",
files={"files": ("late file.xlsx", fh)})
assert up.status_code == 409
assert c.post(f"/api/sessions/{sid}/process").status_code == 409
assert c.put(f"/api/sessions/{sid}/fx",
json=[{"marketplace": "USA", "currency": "USD", "rate": 1.0}]).status_code == 409
assert c.put(f"/api/sessions/{sid}/opening-balances",
json=[{"marketplace": "USA", "amount": 1.0}]).status_code == 409
assert c.post(f"/api/sessions/{sid}/journal/reset-signoff").status_code == 409
assert c.patch(f"/api/sessions/{sid}",
json={"clearing_lag_days": 5}).status_code == 409
# …but a rename stays allowed, and reads still work.
assert c.patch(f"/api/sessions/{sid}", json={"name": "renamed"}).status_code == 200
assert c.get(f"/api/sessions/{sid}/summary").status_code == 200
# Reopen restores editability.
assert c.post(f"/api/sessions/{sid}/reopen").status_code == 200
assert c.get(f"/api/sessions/{sid}").json()["status"] == "processed"
assert c.put(f"/api/sessions/{sid}/opening-balances",
json=[{"marketplace": "USA", "amount": 1.0}]).status_code == 200
# Reopen on a non-completed closing is refused.
assert c.post(f"/api/sessions/{sid}/reopen").status_code == 409
def test_accounts_summary_lists_unpublished_months_as_pending():
init_db()
with TestClient(app) as c:
sid = _processed(c, "pending month", "2028-05-31")
summ = c.get("/api/accounts-summary").json()
mine = [p for p in summ["pending"] if p["session_id"] == sid]
assert len(mine) == 1
assert mine[0]["month"] == "2028-05"
assert "approv" in mine[0]["reason"] # explains HOW to publish it

1
ar-aging-app/backend/tests/test_payout_receipts.py
View File

@ -32,7 +32,6 @@ def _fresh(c, name: str) -> int:
sid = c.post("/api/sessions", json={
"name": name, "reporting_month": "2026-01",
"month_end_date": "2026-01-31", "clearing_lag_days": 2,
"allow_duplicate": True, # suite shares one DB; the guard has its own test
}).json()["id"]
path = os.path.join(_TMP, f"USA {name}.xlsx")
make_amazon_xlsx(path, order_rows=4)

1
ar-aging-app/backend/tests/test_per_market.py
View File

@ -102,7 +102,6 @@ def test_per_market_endpoints():
sid = c.post("/api/sessions", json={
"name": "multi", "reporting_month": "2026-01",
"month_end_date": "2026-01-31", "clearing_lag_days": 2,
"allow_duplicate": True, # suite shares one DB; the guard has its own test
}).json()["id"]
for path in (usa, nl):
with open(path, "rb") as fh:

5
ar-aging-app/backend/tests/test_session_lifecycle.py
View File

@ -23,10 +23,7 @@ from tests.test_excel_export import make_amazon_xlsx # noqa: E402
def _new(c, name: str, month_end: str, **kw) -> int:
# allow_duplicate: the suite reuses months across tests on one shared DB; the
# duplicate-month guard itself is covered in test_month_locking.py.
body = {"name": name, "month_end_date": month_end, "clearing_lag_days": 2,
"allow_duplicate": True, **kw}
body = {"name": name, "month_end_date": month_end, "clearing_lag_days": 2, **kw}
r = c.post("/api/sessions", json=body)
assert r.status_code == 200, r.text
return r.json()["id"]

104
ar-aging-app/backend/tests/test_upload_dedup.py
View File

@ -1,104 +0,0 @@
"""Upload duplicate protection — the historical double-count bug.
Re-uploading a file with the same name used to overwrite it on disk but insert a SECOND
session_files row pointing at the same path, so processing parsed and summed the file
twice. Same content under a different name was equally unguarded."""
from __future__ import annotations
import os
import tempfile
from fastapi.testclient import TestClient
from app.api.main import app
from app.db import models
from app.db.database import SessionLocal, init_db
from tests.test_excel_export import make_amazon_xlsx
_TMP = tempfile.mkdtemp(prefix="ar_dedup_test_")
def _upload(c, sid: int, path: str, as_name: str | None = None):
with open(path, "rb") as fh:
return c.post(f"/api/sessions/{sid}/files",
files={"files": (as_name or os.path.basename(path), fh)})
def _file_rows(sid: int) -> list[models.SessionFile]:
db = SessionLocal()
try:
return db.query(models.SessionFile).filter_by(session_id=sid).all()
finally:
db.close()
def test_same_filename_reupload_updates_row_not_duplicates():
init_db()
a = os.path.join(_TMP, "USA jan.xlsx")
make_amazon_xlsx(a, order_rows=6)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "dedup-name", "month_end_date": "2027-01-31",
"allow_duplicate": True}).json()["id"]
first = _upload(c, sid, a).json()
assert len(first["files"]) == 1 and first["skipped"] == []
first_id = first["files"][0]["id"]
# Different bytes, same filename -> the existing row is replaced, never doubled.
b = os.path.join(_TMP, "USA jan v2.xlsx")
make_amazon_xlsx(b, order_rows=9)
second = _upload(c, sid, b, as_name="USA jan.xlsx").json()
assert len(second["files"]) == 1 and second["skipped"] == []
assert second["files"][0]["id"] == first_id # updated in place
rows = _file_rows(sid)
assert len(rows) == 1
assert rows[0].sha256 == second["files"][0]["sha256"]
def test_identical_bytes_same_name_is_skipped():
init_db()
a = os.path.join(_TMP, "USA feb.xlsx")
make_amazon_xlsx(a, order_rows=6)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "dedup-same", "month_end_date": "2027-02-28",
"allow_duplicate": True}).json()["id"]
assert _upload(c, sid, a).status_code == 200
again = _upload(c, sid, a).json()
assert again["files"] == []
assert len(again["skipped"]) == 1
assert "unchanged" in again["skipped"][0]["reason"]
assert len(_file_rows(sid)) == 1
def test_identical_content_under_new_name_is_skipped():
init_db()
a = os.path.join(_TMP, "USA mar.xlsx")
make_amazon_xlsx(a, order_rows=6)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "dedup-bytes", "month_end_date": "2027-03-31",
"allow_duplicate": True}).json()["id"]
assert _upload(c, sid, a).status_code == 200
renamed = _upload(c, sid, a, as_name="USA mar COPY.xlsx").json()
assert renamed["files"] == []
assert "already uploaded as" in renamed["skipped"][0]["reason"]
assert len(_file_rows(sid)) == 1
def test_double_upload_no_longer_doubles_the_totals():
"""End to end: upload, process, re-upload the SAME file, re-process — totals unchanged."""
init_db()
a = os.path.join(_TMP, "USA apr.xlsx")
make_amazon_xlsx(a, order_rows=8)
with TestClient(app) as c:
sid = c.post("/api/sessions", json={"name": "dedup-e2e", "month_end_date": "2027-04-30",
"allow_duplicate": True}).json()["id"]
assert _upload(c, sid, a).status_code == 200
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
before = c.get(f"/api/sessions/{sid}/summary").json()["closing_receivable_usd"]
assert _upload(c, sid, a).status_code == 200 # skipped as unchanged
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
after = c.get(f"/api/sessions/{sid}/summary").json()["closing_receivable_usd"]
assert after == before

129
ar-aging-app/deploy/DEPLOY.md
View File

@ -1,129 +0,0 @@
# Production deployment (AWS, one server)
One 8 GB server runs everything via `docker-compose.prod.yml`:
**caddy** (automatic HTTPS) → **web** (nginx: React build + `/api` proxy) → **backend**
(FastAPI, single worker) + **mysql** (data on the instance disk), with nightly backups to S3.
8 GB RAM is not optional: uploaded Amazon exports are 300–500 MB and expand to multi-GB
while parsing. Validate with your largest real file before buying anything smaller.
Monthly cost: **≈ $50–55** — Lightsail 8 GB $44 (or EC2 `t4g.large` ≈ $61 with EBS + IPv4),
S3 backups $1.50–3, weekly snapshots $2–4, Route 53 $0.50, Frankfurter FX API $0.
---
## 1. Provision
1. **Lightsail**: 8 GB / 2 vCPU / 160 GB SSD instance, Ubuntu 24.04. Attach the included
static IP. (EC2 route: `t4g.large` + 100 GB gp3 EBS + Elastic IP.)
2. Firewall: allow 22 (your office IPs only), 80, 443. Everything else closed.
3. Install Docker + AWS CLI:
```bash
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER # re-login after this
sudo apt-get install -y awscli # or the AWS CLI v2 bundle
```
4. **S3 bucket** for backups: create `company-ar-backups`, enable **versioning**, add a
lifecycle rule (transition to Glacier/IA after 90 days). Attach an IAM **role** to the
instance allowing `s3:PutObject`, `s3:GetObject`, `s3:ListBucket` on that bucket —
no access keys on disk.
5. **DNS**: A record `ar.<company>.com` → the static IP. Caddy then issues and renews the
TLS certificate automatically — there is no certbot step.
## 2. Configure & start
```bash
sudo mkdir -p /opt/ar-aging && sudo chown $USER /opt/ar-aging
cd /opt/ar-aging && git clone <repo-url> . && cd ar-aging-app
cp .env.example .env.production
nano .env.production # fill the PRODUCTION section: AR_DOMAIN, passwords,
# AR_SECRET_KEY (openssl rand -hex 32), backup bucket
# (a pre-filled .env.production with generated credentials already exists on the
# dev machine — copy it to the server instead of re-generating)
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
curl -s https://ar.<company>.com/api/health # {"status":"ok",...}
```
> Every `docker compose ... -f docker-compose.prod.yml` command below also needs
> `--env-file .env.production` — set an alias once and forget it:
> `alias dcp='docker compose --env-file .env.production -f docker-compose.prod.yml'`
## 3. Create the users (5 logins)
```bash
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
python manage.py add-user talha --name "Talha Ahmed"
# repeat per user; passwords are prompted, never stored in shell history
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend python manage.py list-users
```
`AR_AUTH=on` means the API refuses everything except login/health until users exist.
Password resets: `manage.py set-password <username>`. Leavers: `manage.py deactivate-user`.
## 4. Migrate the existing SQLite data (one-time)
The current data lives in `backend/data/ar_aging.db` on the dev machine. **Do a timed dry
run first** — January alone is ~3.4M transaction rows.
```bash
# copy the SQLite file to the server first (scp), then from ar-aging-app/:
docker compose --env-file .env.production -f docker-compose.prod.yml cp ./ar_aging.db backend:/tmp/ar_aging.db
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
python migrate_sqlite_to_mysql.py --sqlite /tmp/ar_aging.db --dry-run
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
python migrate_sqlite_to_mysql.py --sqlite /tmp/ar_aging.db
```
Verify before anyone uses it: per-table row counts printed by the script must match, and a
spot check to the cent — open the January closing and compare `/api/sessions/{id}/reconciliation`
`final_receivable_usd` against the dev machine. Copy `backend/data/uploads/` into the
`ar_data` volume the same way (`compose cp ./uploads backend:/data/`), then archive the
SQLite file to S3 and retire the dev copy.
**One-time cleanup for the historical double-count bug** (duplicate upload rows):
```bash
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend python manage.py dedupe-files # dry run
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend python manage.py dedupe-files --apply
# then re-process the closings it flagged
```
## 5. Backups
```bash
chmod +x deploy/backup.sh
crontab -e
# 30 2 * * * /opt/ar-aging/ar-aging-app/deploy/backup.sh >> /var/log/ar-backup.log 2>&1
```
Three layers: nightly `mysqldump` + uploads/exports → versioned S3 (the script), weekly
instance snapshots (Lightsail console → enable automatic snapshots), and MySQL's own volume
on the instance disk. **Run the restore drill quarterly** — commands are at the bottom of
`backup.sh`.
## 6. Deploying updates
```bash
cd /opt/ar-aging/ar-aging-app
git pull
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
```
Deploy outside a processing run when possible. If a restart does land mid-run, the closing
is auto-marked as interrupted at startup (never stuck on "processing") — just re-run it.
## 7. Operating notes
- **Single backend worker, single instance — by design.** Jobs and their progress live
in-process. Do not add `--workers` or replicas.
- Logs: `docker compose --env-file .env.production -f docker-compose.prod.yml logs -f backend` (requests, jobs, FX
fetches, logins). Add the CloudWatch agent if you want them off-box.
- Health: `GET /api/health` checks the DB and data-dir and is unauthenticated — point
Lightsail/CloudWatch monitoring at it.
- Exchange rates: Frankfurter (free, keyless). The only outbound call the app makes;
currency codes and dates only. Fetched rates still require in-app confirmation (C5).
- Upgrade path (not needed at this scale): move MySQL to RDS `db.t4g.small` (+~$30/mo,
point-in-time restore) by setting `MYSQL_HOST` to the RDS endpoint and removing the
mysql service; move exports to S3-primary with presigned URLs if the disk ever tightens.

44
ar-aging-app/deploy/backup.sh
View File

@ -1,44 +0,0 @@
#!/usr/bin/env bash
# Nightly backup: MySQL dump + uploads/exports -> S3 (versioned bucket).
#
# Install on the server (as the user that runs docker):
# crontab -e
# 30 2 * * * /opt/ar-aging/ar-aging-app/deploy/backup.sh >> /var/log/ar-backup.log 2>&1
#
# Requires: aws cli v2 on the host, an instance IAM role with s3:PutObject/ListBucket on
# the bucket (no access keys on disk), and .env.production next to docker-compose.prod.yml.
set -euo pipefail
APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$APP_DIR"
# shellcheck disable=SC1091
set -a; source .env.production; set +a
BUCKET="${AR_BACKUP_S3_BUCKET:?AR_BACKUP_S3_BUCKET not set in .env.production}"
STAMP="$(date +%Y-%m-%d_%H%M)"
COMPOSE="docker compose --env-file .env.production -f docker-compose.prod.yml"
echo "[$STAMP] backup starting"
# 1) MySQL dump (single transaction: consistent snapshot without locking the app out).
$COMPOSE exec -T mysql sh -c \
'exec mysqldump --single-transaction --quick --routines \
-u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' \
| gzip > "/tmp/ar_aging_${STAMP}.sql.gz"
aws s3 cp "/tmp/ar_aging_${STAMP}.sql.gz" "$BUCKET/mysql/ar_aging_${STAMP}.sql.gz"
rm -f "/tmp/ar_aging_${STAMP}.sql.gz"
# 2) Uploaded source files + generated exports (the audit trail).
# The ar_data volume is mounted by the backend container; sync straight from it.
DATA_MOUNT="$(docker volume inspect -f '{{ .Mountpoint }}' \
"$(basename "$APP_DIR" | tr '[:upper:]' '[:lower:]' | tr -cd 'a-z0-9')_ar_data" 2>/dev/null \
|| docker volume inspect -f '{{ .Mountpoint }}' ar-aging-app_ar_data)"
aws s3 sync "$DATA_MOUNT/uploads" "$BUCKET/data/uploads" --only-show-errors
aws s3 sync "$DATA_MOUNT/exports" "$BUCKET/data/exports" --only-show-errors
echo "[$STAMP] backup finished"
# Restore drill (run quarterly — a backup you never restored is a hope, not a backup):
# aws s3 cp "$BUCKET/mysql/<latest>.sql.gz" - | gunzip | \
# docker compose --env-file .env.production -f docker-compose.prod.yml exec -T mysql \
# sh -c 'exec mysql -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"'

92
ar-aging-app/docker-compose.prod.yml
View File

@ -1,92 +0,0 @@
# Production stack: [your reverse proxy] -> web (nginx: SPA + /api proxy) -> backend + mysql.
#
# cp .env.example .env.production # fill the PRODUCTION section first
# docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
#
# --env-file is REQUIRED: the ${AR_DOMAIN} / ${MYSQL_ROOT_PASSWORD} references below are
# resolved from it (env_file: alone only feeds the containers, not this YAML).
#
# SHARED SERVER (default): the app's only host port is 127.0.0.1:81 (the web UI). Point
# the server's reverse proxy for ar.utopiabrands.com at http://127.0.0.1:81 with
# client_max_body_size 2g; proxy_read_timeout 600s; proxy_request_buffering off;
# All other ports (backend 8000, mysql 3306) are container-internal and can never
# conflict with other apps on the box.
#
# DEDICATED SERVER: nothing else on 80/443? Start the bundled auto-HTTPS front instead:
# docker compose --env-file .env.production -f docker-compose.prod.yml --profile caddy up -d --build
#
# Sized for one 8 GB server (300-500 MB Excel parsing needs the RAM). Backend runs ONE
# worker by design — jobs and their progress live in-process. See deploy/DEPLOY.md.
services:
mysql:
image: mysql:8.4
restart: unless-stopped
env_file: .env.production # uses MYSQL_PASSWORD / MYSQL_DATABASE / MYSQL_USER
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?set MYSQL_ROOT_PASSWORD in .env.production}
command:
- --innodb-buffer-pool-size=1G
- --max-allowed-packet=256M
volumes:
- mysql_data:/var/lib/mysql
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-p${MYSQL_ROOT_PASSWORD}"]
interval: 10s
timeout: 5s
retries: 12
# Not exposed to the host network — only the backend reaches it.
backend:
build: ./backend
restart: unless-stopped
env_file: .env.production
environment:
AR_DATA_DIR: /data
MYSQL_HOST: mysql
AR_DB_BACKEND: mysql
volumes:
- ar_data:/data
depends_on:
mysql:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c",
"import urllib.request;urllib.request.urlopen('http://localhost:8000/api/health', timeout=5)"]
interval: 30s
timeout: 10s
retries: 3
web:
build:
context: ./frontend
target: prod
restart: unless-stopped
depends_on:
- backend
ports:
# Loopback-only: reachable by the server's own reverse proxy, never the internet.
# Host port 81 avoids clashing with anything else on a shared box.
- "127.0.0.1:81:80"
# OPTIONAL auto-HTTPS front for a DEDICATED server (--profile caddy). Not started by
# default: on a shared box another proxy usually owns 80/443 already.
caddy:
image: caddy:2-alpine
restart: unless-stopped
profiles: ["caddy"]
ports:
- "80:80"
- "443:443"
environment:
AR_DOMAIN: ${AR_DOMAIN:?set AR_DOMAIN in .env.production}
command: caddy reverse-proxy --from "https://${AR_DOMAIN}" --to web:80
volumes:
- caddy_data:/data
- caddy_config:/config
volumes:
mysql_data:
ar_data:
caddy_data:
caddy_config:

7
ar-aging-app/docker-compose.yml
View File

@ -5,17 +5,14 @@ services:
environment:
AR_DATA_DIR: /data
ports:
# Host 8001 avoids clashing with Ahmed's app on 8000.
- "8001:8000"
- "8000:8000"
volumes:
- ./backend:/app
- ar_data:/data
command: uvicorn app.api.main:app --host 0.0.0.0 --port 8000 --reload
frontend:
build:
context: ./frontend
target: dev
build: ./frontend
ports:
- "5173:5173"
volumes:

0
ar-aging-app/docs/audit-2026-07-31-code-review.md → ar-aging-app/docs/AUDIT-REPORT.md
View File

4
ar-aging-app/docs/system-guide.md → ar-aging-app/docs/SYSTEM-GUIDE.md
View File

@ -215,7 +215,7 @@ the ledger, a wrong one can.
Resolve a block on the **Controls** tab (e.g. confirm the FX rates for the month), then re-run.
Every control result travels with the workbook on its own *Month-End Controls* sheet.
See [`audit-2026-07-31-code-review.md`](audit-2026-07-31-code-review.md) for the audit these controls came out of.
See [`AUDIT-REPORT.md`](AUDIT-REPORT.md) for the audit these controls came out of.
---
@ -313,7 +313,7 @@ via `cli.py`, which is what the integration tests exercise.
| `market_payouts` | Per-marketplace received / total payouts, attributed to each settlement's **owner** |
| `opening_balances` | Opening AR per marketplace, with source (manual / carried-forward) and reason |
| `fx_rates` | Month FX rate + currency per marketplace |
| `fx_rates_daily` | Per-date FX rates — auto-fetched from the provider at processing (hand-editable); dated movements convert at the rate effective on their transaction date (exact fixing → previous banking day's fixing → month rate) |
| `fx_rates_daily` | Optional per-date FX override |
| `reserves` | Net Closing Balance per marketplace and account |
| `journal_entries` | The GL decomposition JSON (primary + `per_marketplace`) and entry number |
| `finance_control` | Finance's control-sheet amounts, tolerance, sign-off and comments |

55
ar-aging-app/docs/audit-2026-08-19.md
View File

@ -1,55 +0,0 @@
# Local audit — 19 Aug 2026
Full audit of the local database, files, and the running app before the team enters the
first production month. **Verdict: system healthy and ready; three data-cleanup items for
the team below.**
## What was checked
| Check | Result |
|---|---|
| Database integrity (`PRAGMA integrity_check`) | ✅ ok — `ar_aging.db`, 295 MB, 20 tables |
| Foreign keys / orphaned rows | ✅ zero violations, zero orphans |
| Transactions | ✅ 883,930 rows across 4 processed closings, none missing currency or marketplace |
| Uploaded files vs database | ✅ all 4 files present on disk, sizes match, SHA-256 recorded |
| Duplicate upload rows (historical double-count bug) | ✅ none found — no closing ever double-counted a file |
| Orphan files on disk | ✅ none |
| Users | ✅ 4 active accounts (login verified for each) |
| Full test suite | ✅ 152 passed, 0 failed (12 skipped — large sample files) |
| Live API (running app, port 8010) | ✅ health deep-check ok; no token → 401; wrong password → 401; login ok; month-ordered listing; pending-months explanations; controls 5/6 on Jan |
| Frontend (port 5174) | ✅ serving, proxying to the API |
| Exchange-rate provider (live call) | ✅ Frankfurter reachable; EUR→USD 2026-06-30 = **1.139406**, 2026-01-30 = 1.191895 |
One environment fix made during the audit: this Windows machine's OS certificate store is
corrupted (Python `ssl [ASN1: NOT_ENOUGH_DATA]`), which blocked HTTPS calls. The FX service
now uses the `certifi` CA bundle instead (added to requirements) — affects nothing else.
## Findings for the team (data, not code)
### 1. ⚠️ June closings are valued at January's exchange rate — ≈ $50k overstated
Closings **#3, #4, #5** (all 2026-06) carry EUR→USD = **1.185665**, the January-2026
workbook snapshot, and it was *confirmed* at that value. The actual ECB rate on
2026-06-30 was **1.139406** — the June receivable of $1,290,921 is overstated by roughly
**$50,000**. Fix on whichever June closing is kept: Controls tab → **Fetch month-end
rates** → review → Confirm → re-run controls. This is precisely the failure mode the new
FX fetch exists to prevent.
### 2. ⚠️ Three identical June closings + three empty drafts
Closings #3 ("July finance report"), #4 ("june"), #5 ("Test Case - Germany Jun-2026") are
the **same June file processed three times** — identical 214,166 transactions and identical
receivable. Keep one, delete the other two. Drafts #2, #6 (2026-06) and #7 (2026-01) are
empty and can be deleted. The dashboard now flags all of these with a duplicate-month ⚠.
Going forward the app blocks accidental month duplicates at creation.
### 3. ℹ️ No month is published yet
No journal has been approved, so the Accounts Summary is empty — the summary page now
lists each processed month with the reason ("journal not approved yet") and a link. When
January is final: Journal Entry tab → Mark reviewed → Approve (records the signed-in
user's name).
## State after cleanup (recommended target)
- One closing per month: `2026-01` (#1) and one `2026-06`, both with fetched + confirmed
June/January rates, journals approved, then **Complete** to lock them read-only.
- First production month gets entered by the team on the deployed server per
`deploy/DEPLOY.md`; this local database migrates there as-is.

13
ar-aging-app/example.env Normal file
View File

@ -0,0 +1,13 @@
MYSQL_HOST=your-mysql-host.example.com
MYSQL_PORT=3306
MYSQL_USER=your_mysql_user
MYSQL_PASSWORD=your_mysql_password
MYSQL_DATABASE=account_finance
MYSQL_SLOW_QUERY_MS=500
MYSQL_POOL_SIZE=10
MYSQL_POOL_RECYCLE=3600
# Uploads/exports. Docker Compose sets AR_DATA_DIR=/data (named volume).
# Leave unset locally to use backend/data.
# AR_DATA_DIR=/data
AR_CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173

7
ar-aging-app/frontend/.dockerignore
View File

@ -1,7 +0,0 @@
# Docker only reads the .dockerignore INSIDE the build context (this folder). Without it,
# `COPY . .` would overwrite the image's freshly installed node_modules with the host's
# (built for a different OS) and drag in stale dist output.
node_modules
dist
.env
.env.*

18
ar-aging-app/frontend/Dockerfile
View File

@ -1,22 +1,12 @@
# Multi-stage: `dev` target = Vite dev server (docker-compose.yml),
# default/`prod` target = static build served by nginx (docker-compose.prod.yml).
FROM node:20-alpine
FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm install
COPY . .
# ---- dev: hot-reload server (dev compose bind-mounts the source over /app) ----
FROM deps AS dev
EXPOSE 5173
CMD ["npm", "run", "dev", "--", "--host", "0.0.0.0", "--port", "5173"]
# ---- prod: type-check + build, then serve the static bundle with nginx ----
FROM deps AS build
RUN npm run build
FROM nginx:1.27-alpine AS prod
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80

40
ar-aging-app/frontend/nginx.conf
View File

@ -1,40 +0,0 @@
# Production frontend: serve the built SPA, proxy /api to the backend container.
# TLS is terminated in front of this (caddy service in docker-compose.prod.yml).
server {
listen 80;
listen [::]:80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Uploads are 300-500 MB Amazon exports; the app enforces its own 2 GB cap.
client_max_body_size 2g;
gzip on;
gzip_types text/plain text/css application/json application/javascript image/svg+xml;
location /api {
proxy_pass http://backend:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
# Big uploads and long-running processing/status calls.
proxy_request_buffering off;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
# SPA routing: every non-file path renders index.html.
location / {
try_files $uri /index.html;
}
# Hashed assets can cache forever; index.html must not.
location /assets/ {
add_header Cache-Control "public, max-age=31536000, immutable";
}
location = /index.html {
add_header Cache-Control "no-cache";
}
}

37
ar-aging-app/frontend/src/App.tsx
View File

@ -1,14 +1,10 @@
import { NavLink, Route, Routes } from "react-router-dom";
import { LayoutDashboard, FilePlus2, LogOut, ScrollText, Settings as SettingsIcon, Landmark, Table2, UserCircle2 } from "lucide-react";
import { LayoutDashboard, FilePlus2, Settings as SettingsIcon, Landmark, Table2 } from "lucide-react";
import Dashboard from "./pages/Dashboard";
import AccountsSummary from "./pages/AccountsSummary";
import NewClosing from "./pages/NewClosing";
import Closing from "./pages/Closing";
import Settings from "./pages/Settings";
import AuditLog from "./pages/AuditLog";
import Login from "./pages/Login";
import { Spinner } from "./components/ui";
import { useAuth } from "./auth";
function SideLink({ to, icon: Icon, children, end }: {
to: string; icon: typeof LayoutDashboard; children: string; end?: boolean;
@ -32,16 +28,6 @@ function SideLink({ to, icon: Icon, children, end }: {
}
export default function App() {
const { loading, authRequired, user, logout } = useAuth();
if (loading)
return (
<div className="h-full bg-canvas flex items-center justify-center gap-2 text-subink">
<Spinner /> Loading…
</div>
);
if (authRequired && !user) return <Login />;
return (
<div className="h-full bg-canvas p-3 sm:p-4">
<div className="flex h-full min-h-0 bg-panel rounded-3xl shadow-pop overflow-hidden border border-line">
@ -60,26 +46,10 @@ export default function App() {
<SideLink to="/accounts" icon={Table2}>Accounts Summary</SideLink>
<SideLink to="/new" icon={FilePlus2}>New Closing</SideLink>
<SideLink to="/settings" icon={SettingsIcon}>Settings</SideLink>
{user?.is_admin && <SideLink to="/audit" icon={ScrollText}>Audit Log</SideLink>}
</nav>
{user && (
<div className="m-3 p-3 rounded-2xl bg-canvas/70 space-y-2.5">
<div className="flex items-center gap-2.5">
<UserCircle2 size={22} className="text-primary shrink-0" />
<div className="flex-1 min-w-0">
<div className="text-xs font-semibold text-ink truncate">{user.display_name}</div>
<div className="text-[11px] text-muted truncate">{user.username}</div>
<div className="m-3 p-3.5 rounded-2xl bg-canvas/70 text-[11px] text-muted leading-relaxed">
Amazon‑only · processed locally on your server · no third‑party egress.
</div>
</div>
<button
className="w-full inline-flex items-center justify-center gap-2 px-3 py-2 rounded-xl
text-sm font-medium text-subink bg-panel border border-line
hover:text-bad hover:border-bad/30 hover:bg-badbg/40 transition-colors"
onClick={logout}>
<LogOut size={15} /> Sign out
</button>
</div>
)}
</aside>
<main className="flex-1 min-w-0 overflow-y-auto">
@ -89,7 +59,6 @@ export default function App() {
<Route path="/new" element={<NewClosing />} />
<Route path="/closing/:id/*" element={<Closing />} />
<Route path="/settings" element={<Settings />} />
<Route path="/audit" element={<AuditLog />} />
</Routes>
</main>
</div>

168
ar-aging-app/frontend/src/api/client.ts
View File

@ -1,20 +1,12 @@
const BASE = "/api";
const TOKEN_KEY = "ar_token";
export const getToken = () => localStorage.getItem(TOKEN_KEY);
export const setToken = (t: string) => localStorage.setItem(TOKEN_KEY, t);
export const clearToken = () => localStorage.removeItem(TOKEN_KEY);
/** Set by the auth provider: called on a 401 so the app can drop to the login screen. */
export let onUnauthorized: (() => void) | null = null;
export const setOnUnauthorized = (fn: (() => void) | null) => { onUnauthorized = fn; };
async function req<T>(path: string, opts: RequestInit = {}): Promise<T> {
const headers: Record<string, string> = {};
if (opts.body && !(opts.body instanceof FormData)) headers["Content-Type"] = "application/json";
const token = getToken();
if (token) headers["Authorization"] = `Bearer ${token}`;
const res = await fetch(`${BASE}${path}`, { headers, ...opts });
const res = await fetch(`${BASE}${path}`, {
headers: opts.body && !(opts.body instanceof FormData)
? { "Content-Type": "application/json" }
: undefined,
...opts,
});
if (!res.ok) {
let detail = res.statusText;
try {
@ -22,10 +14,6 @@ async function req<T>(path: string, opts: RequestInit = {}): Promise<T> {
} catch {
/* ignore */
}
if (res.status === 401 && !path.startsWith("/auth/")) {
clearToken();
onUnauthorized?.();
}
throw new Error(detail);
}
const ct = res.headers.get("content-type") ?? "";
@ -61,38 +49,6 @@ export interface SessionT {
payout_mode?: string;
/** Bank receipts / payout mode changed after the last run — re-process to apply. */
needs_reprocess?: boolean;
/** Journal approved = published to the Accounts Summary (list endpoint only). */
journal_approved?: boolean;
/** Another closing exists for the same reporting month (list endpoint only). */
duplicate_month?: boolean;
}
export interface UploadResultT {
files: FileT[];
skipped: { filename: string; reason: string }[];
}
export interface AuthUserT {
username: string;
display_name: string;
/** Admin = can read the audit log (granted via manage.py set-admin). */
is_admin?: boolean;
}
export interface AuditEntryT {
id: number;
at: string | null;
username: string;
display_name: string;
action: string;
session_id: number | null;
session_name: string;
detail: string;
}
export interface AuditLogT {
total: number;
entries: AuditEntryT[];
}
export interface PayoutT {
@ -110,9 +66,6 @@ export interface PayoutT {
received_next_run: boolean;
}
/** Aging band width for the A/R aging report. */
export type AgingSchemeT = "weekly" | "monthly" | "half_year" | "yearly";
export interface PayoutsT {
payout_mode: string;
clearing_lag_days: number;
@ -121,45 +74,6 @@ export interface PayoutsT {
payouts: PayoutT[];
}
/** One bank-file row auto-matched to a payout by the disbursements import. */
export interface PayoutImportMatchT {
marketplace: string;
account_type: string;
settlement_id: string;
amazon_date: string | null;
amazon_amount: number;
bank_date: string;
bank_amount: number;
currency: string;
amount_checked: boolean;
delta: number | null;
bank_row: number;
already_had_receipt: boolean;
existing_bank_date: string | null;
note: string;
}
export interface PayoutImportRowT {
bank_row: number;
party: string;
marketplace?: string;
bank_date: string;
amount: number;
reason?: string;
candidates?: { settlement_id: string; account_type: string; amazon_date: string | null; amount: number }[];
}
export interface PayoutImportT {
total_rows: number;
window_days: number;
matched: PayoutImportMatchT[];
ambiguous: PayoutImportRowT[];
unmatched_bank_rows: PayoutImportRowT[];
unknown_party: PayoutImportRowT[];
out_of_scope: number;
problems: string[];
}
/** One month-end control (core/controls.py). Distinct from ControlRowT, which is a row of
* the Finance reconciliation control sheet. */
export interface MonthEndControlT {
@ -329,9 +243,6 @@ export interface AccountsSummaryT {
month: string; session_id: number; marketplace: string; currency: string; fx_rate: number;
values: Record<string, number>; receivable: number;
}[];
/** Months with results that are NOT published (unapproved / blocked), with the reason —
* so a month never silently vanishes from this view. */
pending?: { month: string; session_id: number; session_name: string; reason: string }[];
}
export interface ComponentT {
@ -368,9 +279,6 @@ export interface FinanceSummaryT extends BlockableT {
export interface LedgerPeriodT {
key: string; label: string; revenue: number; payouts_received: number;
payouts_in_transit: number; rows: number; balance: number;
/** USD equivalents, converted at each transaction date's FX rate. */
revenue_usd: number; payouts_received_usd: number;
payouts_in_transit_usd: number; balance_usd: number;
}
export interface LedgerDetailT {
available: boolean;
@ -378,13 +286,6 @@ export interface LedgerDetailT {
granularity?: string; date_from?: string | null; date_to?: string | null;
opening?: number; periods?: LedgerPeriodT[]; closing?: number;
session_closing?: number; filtered?: boolean; in_transit_total?: number;
/** The marketplace month rate; the opening balance converts at this rate. */
month_rate?: number;
opening_usd?: number;
/** Roll-forward valued at transaction-date rates — differs from closing × month rate
* whenever daily overrides exist. */
closing_usd?: number;
in_transit_total_usd?: number;
}
export interface FxDailyRowT {
@ -549,48 +450,19 @@ const q = (o: Record<string, string | undefined>) =>
export const api = {
health: () => req<{ status: string; version: string }>("/health"),
authStatus: () => req<{ auth_required: boolean; email_enabled: boolean }>("/auth/status"),
login: (username: string, password: string) =>
req<{ token: string; user: AuthUserT }>("/auth/login", {
method: "POST", body: JSON.stringify({ username, password }),
}),
me: () => req<{ authenticated: boolean; username?: string; display_name?: string;
is_admin?: boolean }>("/auth/me"),
auditLog: (opts: { limit?: number; offset?: number; action?: string } = {}) =>
req<AuditLogT>(`/audit?${q({
limit: opts.limit?.toString(), offset: opts.offset?.toString(), action: opts.action,
})}`),
changePassword: (current_password: string, new_password: string) =>
req<{ changed: boolean }>("/auth/change-password", {
method: "POST", body: JSON.stringify({ current_password, new_password }),
}),
requestPasswordCode: (username = "") =>
req<{ sent: boolean; detail: string }>("/auth/request-code", {
method: "POST", body: JSON.stringify({ username }),
}),
verifyPasswordCode: (code: string, username = "") =>
req<{ valid: boolean }>("/auth/verify-code", {
method: "POST", body: JSON.stringify({ username, code }),
}),
resetPassword: (code: string, new_password: string, username = "") =>
req<{ changed: boolean }>("/auth/reset-password", {
method: "POST", body: JSON.stringify({ username, code, new_password }),
}),
listSessions: () => req<SessionT[]>("/sessions"),
createSession: (body: Partial<SessionT> & { allow_duplicate?: boolean }) =>
createSession: (body: Partial<SessionT>) =>
req<SessionT>("/sessions", { method: "POST", body: JSON.stringify(body) }),
getSession: (id: number) => req<SessionT>(`/sessions/${id}`),
updateSession: (id: number, body: Partial<SessionT>) =>
req<SessionT>(`/sessions/${id}`, { method: "PATCH", body: JSON.stringify(body) }),
deleteSession: (id: number) => req<void>(`/sessions/${id}`, { method: "DELETE" }),
reopenSession: (id: number) => req<SessionT>(`/sessions/${id}/reopen`, { method: "POST" }),
listFiles: (id: number) => req<FileT[]>(`/sessions/${id}/files`),
uploadFiles: (id: number, files: File[]) => {
const fd = new FormData();
files.forEach((f) => fd.append("files", f));
return req<UploadResultT>(`/sessions/${id}/files`, { method: "POST", body: fd });
return req<FileT[]>(`/sessions/${id}/files`, { method: "POST", body: fd });
},
deleteFile: (id: number, fileId: number) =>
req<void>(`/sessions/${id}/files/${fileId}`, { method: "DELETE" }),
@ -618,9 +490,9 @@ export const api = {
deleteMappingRule: (ruleId: number) =>
req<MappingRulesT>(`/mapping-rules/${ruleId}`, { method: "DELETE" }),
reconciliation: (id: number) => req<ReconT>(`/sessions/${id}/reconciliation`),
aging: (id: number, scheme: AgingSchemeT = "monthly") =>
req<BlockableT & { bands: string[]; scheme?: string; rows: Record<string, number | string>[] }>(
`/sessions/${id}/aging?scheme=${scheme}`),
aging: (id: number) =>
req<BlockableT & { bands: string[]; rows: Record<string, number | string>[] }>(
`/sessions/${id}/aging`),
journal: (id: number, marketplace?: string) =>
req<JournalT>(`/sessions/${id}/journal${marketplace ? `?marketplace=${encodeURIComponent(marketplace)}` : ""}`),
reviewJournal: (id: number, name: string) =>
@ -675,13 +547,6 @@ export const api = {
bank_date: string | null; bank_amount?: number | null; note?: string; entered_by?: string;
}[]) => req<{ saved: number; removed: number; needs_reprocess: boolean }>(
`/sessions/${id}/payouts/receipts`, { method: "PUT", body: JSON.stringify(items) }),
importPayoutReceipts: (id: number, file: File, windowDays = 14) => {
const fd = new FormData();
fd.append("file", file);
return req<PayoutImportT>(
`/sessions/${id}/payouts/receipts/import?window_days=${windowDays}`,
{ method: "POST", body: fd });
},
putPayoutMode: (id: number, mode: "auto" | "manual") =>
req<{ payout_mode: string; needs_reprocess: boolean }>(
`/sessions/${id}/payouts/mode`, { method: "PUT", body: JSON.stringify({ mode }) }),
@ -698,18 +563,9 @@ export const api = {
putReserves: (id: number, items: { marketplace: string; account_type: string; amount: number }[]) =>
req(`/sessions/${id}/reserves`, { method: "PUT", body: JSON.stringify(items) }),
getFx: (id: number) =>
req<{ marketplace: string; currency: string; rate: number;
source: string; rate_date: string | null }[]>(`/sessions/${id}/fx`),
req<{ marketplace: string; currency: string; rate: number }[]>(`/sessions/${id}/fx`),
putFx: (id: number, items: { marketplace: string; currency: string; rate: number }[]) =>
req(`/sessions/${id}/fx`, { method: "PUT", body: JSON.stringify(items) }),
fetchFx: (id: number) =>
req<{ updated: { marketplace: string; currency: string; rate: number }[];
missing: string[]; source: string; rate_date: string }>(
`/sessions/${id}/fx/fetch`, { method: "POST" }),
fetchFxDaily: (id: number, body: { marketplace?: string; date_from?: string; date_to?: string } = {}) =>
req<{ saved: number; date_from: string; date_to: string; provider: string;
marketplaces: string[] }>(
`/sessions/${id}/fx/fetch-daily`, { method: "POST", body: JSON.stringify(body) }),
startExport: (id: number, kind: "full" | "summary" = "full") =>
req<{ started: boolean; kind: string }>(`/sessions/${id}/export?kind=${kind}`, { method: "POST" }),

85
ar-aging-app/frontend/src/auth.tsx
View File

@ -1,85 +0,0 @@
import { ReactNode, createContext, useContext, useEffect, useMemo, useState } from "react";
import { api, AuthUserT, clearToken, getToken, setOnUnauthorized, setToken } from "./api/client";
/**
* Login state for the whole app.
*
* The backend decides whether login is required (/auth/status): with AR_AUTH=auto it turns
* on as soon as the first user is created, so a dev checkout keeps working with no ceremony
* while production requires sign-in. The signed-in display name is also what the backend
* records in review/approve/confirm fields — the UI shows it instead of a free-text box.
*/
interface AuthState {
loading: boolean;
authRequired: boolean;
/** Server can send password codes by email (AR_SMTP_* configured). */
emailEnabled: boolean;
user: AuthUserT | null;
login: (username: string, password: string) => Promise<void>;
logout: () => void;
}
const AuthCtx = createContext<AuthState>({
loading: true, authRequired: false, emailEnabled: false, user: null,
login: async () => undefined, logout: () => undefined,
});
export const useAuth = () => useContext(AuthCtx);
export function AuthProvider({ children }: { children: ReactNode }) {
const [loading, setLoading] = useState(true);
const [authRequired, setAuthRequired] = useState(false);
const [emailEnabled, setEmailEnabled] = useState(false);
const [user, setUser] = useState<AuthUserT | null>(null);
useEffect(() => {
setOnUnauthorized(() => {
setUser(null);
setAuthRequired(true);
});
return () => setOnUnauthorized(null);
}, []);
useEffect(() => {
(async () => {
try {
const status = await api.authStatus();
setAuthRequired(status.auth_required);
setEmailEnabled(status.email_enabled ?? false);
if (status.auth_required && getToken()) {
try {
const me = await api.me();
if (me.authenticated && me.username) {
setUser({ username: me.username, display_name: me.display_name ?? me.username,
is_admin: me.is_admin ?? false });
}
} catch {
clearToken();
}
}
} catch {
// Backend unreachable — leave the app open; queries will surface the real error.
} finally {
setLoading(false);
}
})();
}, []);
const value = useMemo<AuthState>(() => ({
loading,
authRequired,
emailEnabled,
user,
login: async (username: string, password: string) => {
const res = await api.login(username, password);
setToken(res.token);
setUser(res.user);
},
logout: () => {
clearToken();
setUser(null);
},
}), [loading, authRequired, emailEnabled, user]);
return <AuthCtx.Provider value={value}>{children}</AuthCtx.Provider>;
}

124
ar-aging-app/frontend/src/components/BankReceipts.tsx
View File

@ -1,7 +1,7 @@
import { useEffect, useRef, useState } from "react";
import { useEffect, useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Banknote, CheckCircle2, Clock, RefreshCw, Save, Upload, X } from "lucide-react";
import { api, PayoutImportT, PayoutT } from "../api/client";
import { Banknote, CheckCircle2, Clock, RefreshCw, Save } from "lucide-react";
import { api, PayoutT } from "../api/client";
import { acct, date as fmtDate } from "../lib/format";
import { InfoTip, Section, Spinner, useDefinitions } from "./ui";
@ -56,36 +56,6 @@ export default function BankReceipts({ id, marketplace }: { id: number; marketpl
onSuccess: () => qc.invalidateQueries({ queryKey: ["session", id] }),
});
// Bank-file import: upload -> server proposes matches -> user applies via the normal PUT.
const fileRef = useRef<HTMLInputElement>(null);
const [imported, setImported] = useState<PayoutImportT | null>(null);
const [picked, setPicked] = useState<Record<number, boolean>>({}); // bank_row -> apply?
const importFile = useMutation({
mutationFn: (file: File) => api.importPayoutReceipts(id, file),
onSuccess: (res) => {
setImported(res);
// Pre-select fresh matches; leave payouts that already have this receipt unticked.
setPicked(Object.fromEntries(res.matched.map((m) => [
m.bank_row, !m.already_had_receipt || m.existing_bank_date !== m.bank_date,
])));
},
});
const applyImport = useMutation({
mutationFn: () => {
const items = (imported?.matched ?? [])
.filter((m) => picked[m.bank_row])
.map((m) => ({
marketplace: m.marketplace, account_type: m.account_type,
settlement_id: m.settlement_id, bank_date: m.bank_date,
bank_amount: m.amount_checked ? m.bank_amount : null,
note: m.note,
}));
return api.putPayoutReceipts(id, items);
},
onSuccess: () => { setImported(null); setPicked({}); invalidate(); },
});
const pickedCount = (imported?.matched ?? []).filter((m) => picked[m.bank_row]).length;
if (isLoading) return null;
if (!data?.payouts?.length) return null;
const manual = data.payout_mode === "manual";
@ -105,16 +75,6 @@ export default function BankReceipts({ id, marketplace }: { id: number; marketpl
onChange={(e) => setMode.mutate(e.target.checked ? "manual" : "auto")} />
Bank dates only (no clearing-lag)
</label>
<input ref={fileRef} type="file" accept=".xlsx,.xls" className="hidden"
onChange={(e) => {
const f = e.target.files?.[0];
if (f) importFile.mutate(f);
e.target.value = "";
}} />
<button className="btn-ghost" disabled={importFile.isPending}
onClick={() => fileRef.current?.click()}>
{importFile.isPending ? <Spinner /> : <Upload size={14} />} Import from Excel
</button>
</div>
}
>
@ -132,84 +92,6 @@ export default function BankReceipts({ id, marketplace }: { id: number; marketpl
</div>
)}
{importFile.isError && (
<div className="mx-4 mt-3 rounded-lg border border-bad/30 bg-badbg/40 px-3 py-2 text-sm text-bad">
Import failed: {(importFile.error as Error).message}
</div>
)}
{imported && (
<div className="mx-4 mt-3 rounded-lg border border-line bg-canvas/40 text-sm">
<div className="px-3 py-2 flex flex-wrap items-center gap-3 border-b border-line">
<Upload size={15} className="text-primary shrink-0" />
<span className="flex-1 min-w-[240px]">
<b>{imported.matched.length} matched</b>
{" · "}{imported.ambiguous.length} ambiguous
{" · "}{imported.unmatched_bank_rows.length} unmatched
{imported.unknown_party.length > 0 && <>{" · "}{imported.unknown_party.length} unknown party</>}
{" · "}{imported.out_of_scope} outside this month
<span className="text-subink"> ({imported.total_rows} deposit rows read)</span>
</span>
<button className="btn-ghost" onClick={() => { setImported(null); setPicked({}); }}>
<X size={14} /> Dismiss
</button>
<button className="btn-primary" disabled={applyImport.isPending || pickedCount === 0}
onClick={() => applyImport.mutate()}>
<Save size={15} /> {applyImport.isPending ? "Applying…" : `Apply ${pickedCount} receipt(s)`}
</button>
</div>
{imported.matched.length > 0 && (
<ul className="px-3 py-2 space-y-1 max-h-56 overflow-y-auto">
{imported.matched.map((m) => (
<li key={m.bank_row} className="flex items-center gap-2">
<input type="checkbox" checked={!!picked[m.bank_row]}
onChange={(e) => setPicked((s) => ({ ...s, [m.bank_row]: e.target.checked }))} />
<span className="num text-xs">{m.marketplace} · {m.settlement_id}</span>
<span className="flex-1 text-xs text-subink">
bank {fmtDate(m.bank_date)} · {m.currency} {m.bank_amount.toLocaleString()}
{m.amount_checked
? (m.delta ? ` · Δ ${m.delta}` : "")
: " · amount not compared (currency differs)"}
{m.already_had_receipt && ` · already had ${fmtDate(m.existing_bank_date)}`}
{" · file row "}{m.bank_row}
</span>
</li>
))}
</ul>
)}
{(imported.ambiguous.length > 0 || imported.unmatched_bank_rows.length > 0
|| imported.unknown_party.length > 0 || imported.problems.length > 0) && (
<details className="px-3 py-2 border-t border-line">
<summary className="cursor-pointer text-xs text-subink select-none">
Rows needing a manual look
</summary>
<ul className="mt-1.5 space-y-1 text-xs text-subink max-h-40 overflow-y-auto">
{imported.ambiguous.map((r) => (
<li key={`a${r.bank_row}`}>
row {r.bank_row} · {r.party} · {fmtDate(r.bank_date)} · {r.amount.toLocaleString()} — {r.reason}
{r.candidates?.length ? ` (candidates: ${r.candidates.map((c) => c.settlement_id).join(", ")})` : ""}
</li>
))}
{imported.unmatched_bank_rows.map((r) => (
<li key={`u${r.bank_row}`}>
row {r.bank_row} · {r.party} · {fmtDate(r.bank_date)} · {r.amount.toLocaleString()} — {r.reason}
</li>
))}
{imported.unknown_party.map((r) => (
<li key={`p${r.bank_row}`}>row {r.bank_row} · unrecognized party “{r.party}”</li>
))}
{imported.problems.map((p, i) => <li key={`q${i}`}>{p}</li>)}
</ul>
</details>
)}
{applyImport.isError && (
<div className="px-3 py-2 border-t border-line text-bad">
{(applyImport.error as Error).message}
</div>
)}
</div>
)}
<div className="overflow-x-auto">
<table className="w-full">
<thead><tr>

33
ar-aging-app/frontend/src/components/MonthSwitcher.tsx
View File

@ -1,33 +0,0 @@
import { useNavigate } from "react-router-dom";
import { useQuery } from "@tanstack/react-query";
import { CalendarDays } from "lucide-react";
import { api } from "../api/client";
/**
* Jump between month-end closings from inside any closing screen. Every previous month
* stays saved and selectable here — uploading a new month never replaces an old one.
*/
export default function MonthSwitcher({ currentId }: { currentId: number }) {
const nav = useNavigate();
const { data: sessions } = useQuery({ queryKey: ["sessions"], queryFn: api.listSessions });
if (!sessions || sessions.length < 2) return null;
return (
<label className="flex items-center gap-1.5 text-sm text-subink">
<CalendarDays size={15} className="text-primary shrink-0" />
<span className="sr-only">Switch closing</span>
<select
className="input py-1.5 pr-7 text-sm max-w-[16rem]"
value={currentId}
onChange={(e) => nav(`/closing/${e.target.value}`)}
>
{sessions.map((s) => (
<option key={s.id} value={s.id}>
{s.reporting_month ?? "no month"} · {s.name}
{s.status === "completed" ? " ✓" : s.journal_approved ? " (published)" : ""}
</option>
))}
</select>
</label>
);
}

3
ar-aging-app/frontend/src/main.tsx
View File

@ -3,7 +3,6 @@ import ReactDOM from "react-dom/client";
import { BrowserRouter } from "react-router-dom";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import App from "./App";
import { AuthProvider } from "./auth";
import "./index.css";
const queryClient = new QueryClient({
@ -13,11 +12,9 @@ const queryClient = new QueryClient({
ReactDOM.createRoot(document.getElementById("root")!).render(
<React.StrictMode>
<QueryClientProvider client={queryClient}>
<AuthProvider>
<BrowserRouter>
<App />
</BrowserRouter>
</AuthProvider>
</QueryClientProvider>
</React.StrictMode>
);

35
ar-aging-app/frontend/src/pages/AccountsSummary.tsx
View File

@ -1,8 +1,8 @@
import { useState } from "react";
import { Link } from "react-router-dom";
import { useQuery } from "@tanstack/react-query";
import { BadgeCheck, CalendarClock, Table2 } from "lucide-react";
import { api, AccountsSummaryT } from "../api/client";
import { BadgeCheck, Table2 } from "lucide-react";
import { api } from "../api/client";
import { acct, money } from "../lib/format";
import { EmptyState, Section, Spinner } from "../components/ui";
@ -26,11 +26,10 @@ export default function AccountsSummary() {
return <div className="p-6 flex items-center gap-2 text-subink"><Spinner /> Loading…</div>;
if (!data?.available)
return (
<div className="p-6 max-w-7xl mx-auto space-y-6">
<div className="p-6 max-w-7xl mx-auto">
<Header />
<EmptyState title="No approved months yet"
hint="Open a closing's Journal Entry tab, have it reviewed and approved — approval publishes that month here, for every marketplace." />
<PendingMonths pending={data?.pending} />
</div>
);
@ -127,8 +126,6 @@ export default function AccountsSummary() {
</div>
</Section>
<PendingMonths pending={data.pending} />
<p className="text-xs text-subink">
{showAll
? "USD figures convert each marketplace at its own closing's confirmed FX rate."
@ -140,32 +137,6 @@ export default function AccountsSummary() {
);
}
/** Months with results that are NOT published — listed with the reason instead of just
* vanishing from the grid (the classic "where did January go?" confusion). */
function PendingMonths({ pending }: { pending: AccountsSummaryT["pending"] }) {
if (!pending?.length) return null;
return (
<div className="card border-warn/30 bg-warnbg/30 p-4">
<p className="text-sm font-semibold text-ink flex items-center gap-2 mb-2">
<CalendarClock size={15} className="text-warn" />
{pending.length} month(s) processed but not shown here
</p>
<ul className="space-y-1.5">
{pending.map((p) => (
<li key={p.session_id} className="text-sm text-subink flex items-baseline gap-2 flex-wrap">
<span className="num font-semibold text-ink">{p.month}</span>
<span className="flex-1 min-w-[200px]">{p.reason}</span>
<Link to={`/closing/${p.session_id}/journal`}
className="text-primary font-medium hover:underline shrink-0">
open journal →
</Link>
</li>
))}
</ul>
</div>
);
}
function Header() {
return (
<header>

156
ar-aging-app/frontend/src/pages/AuditLog.tsx
View File

@ -1,156 +0,0 @@
import { useState } from "react";
import { useQuery } from "@tanstack/react-query";
import { ChevronLeft, ChevronRight, ScrollText } from "lucide-react";
import { api } from "../api/client";
import { EmptyState, Section, Spinner } from "../components/ui";
import { useAuth } from "../auth";
const PAGE = 50;
/** Human labels for audit actions; unknown actions fall back to the raw key. */
const ACTION_LABELS: Record<string, string> = {
login: "Signed in",
session_create: "Created closing",
session_delete: "Deleted closing",
session_reopen: "Reopened closing",
file_upload: "Uploaded file",
file_delete: "Deleted file",
process_run: "Ran processing",
export_generate: "Generated export",
export_download: "Downloaded export",
};
/** Backend timestamps are naive UTC — pin them to UTC before rendering local time. */
function fmtWhen(at: string | null): string {
if (!at) return "—";
const d = new Date(/[Z+]/.test(at.slice(-6)) ? at : at + "Z");
return d.toLocaleString(undefined, {
year: "numeric", month: "short", day: "numeric", hour: "2-digit", minute: "2-digit",
});
}
export default function AuditLog() {
const { user } = useAuth();
const [offset, setOffset] = useState(0);
const [action, setAction] = useState("");
const { data, isLoading, error } = useQuery({
queryKey: ["audit", offset, action],
queryFn: () => api.auditLog({ limit: PAGE, offset, action: action || undefined }),
enabled: !!user?.is_admin,
});
if (user && !user.is_admin)
return (
<div className="p-6">
<EmptyState title="Admin access required"
hint="The audit log is visible to administrators only." />
</div>
);
const total = data?.total ?? 0;
const entries = data?.entries ?? [];
return (
<div className="p-6 max-w-5xl mx-auto space-y-6">
<header className="flex items-center gap-3">
<span className="inline-flex items-center justify-center w-9 h-9 rounded-xl bg-primary-soft text-primary">
<ScrollText size={18} />
</span>
<div>
<h1 className="text-xl font-semibold text-ink">Audit Log</h1>
<p className="text-sm text-subink">
Who signed in, uploaded, processed, exported, and deleted — newest first.
</p>
</div>
</header>
<Section
title={`Activity${total ? ` · ${total.toLocaleString()} entries` : ""}`}
actions={
<select
className="text-sm rounded-xl border border-line bg-panel px-3 py-1.5 text-ink"
value={action}
onChange={(e) => { setAction(e.target.value); setOffset(0); }}
>
<option value="">All actions</option>
{Object.entries(ACTION_LABELS).map(([k, v]) => (
<option key={k} value={k}>{v}</option>
))}
</select>
}
>
{isLoading && (
<div className="p-8 flex items-center justify-center gap-2 text-subink">
<Spinner /> Loading…
</div>
)}
{error instanceof Error && (
<div className="p-6 text-sm text-bad">{error.message}</div>
)}
{!isLoading && !error && entries.length === 0 && (
<EmptyState title="No activity recorded yet"
hint="Entries appear here as people sign in, upload files, run processing, and export." />
)}
{entries.length > 0 && (
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="text-left text-xs uppercase tracking-wide text-muted border-b border-line">
<th className="px-4 py-2.5 whitespace-nowrap">When</th>
<th className="px-4 py-2.5 whitespace-nowrap">Who</th>
<th className="px-4 py-2.5 whitespace-nowrap">Action</th>
<th className="px-4 py-2.5 whitespace-nowrap">Closing</th>
<th className="px-4 py-2.5">Detail</th>
</tr>
</thead>
<tbody className="divide-y divide-line">
{entries.map((e) => (
<tr key={e.id} className="align-top">
<td className="px-4 py-2.5 whitespace-nowrap text-subink">{fmtWhen(e.at)}</td>
<td className="px-4 py-2.5 whitespace-nowrap">
<div className="font-medium text-ink">{e.display_name || "(no login)"}</div>
{e.username && <div className="text-[11px] text-muted">{e.username}</div>}
</td>
<td className="px-4 py-2.5 whitespace-nowrap text-ink">
{ACTION_LABELS[e.action] ?? e.action}
</td>
<td className="px-4 py-2.5 whitespace-nowrap text-subink">
{e.session_name || (e.session_id ? `#${e.session_id}` : "—")}
</td>
<td className="px-4 py-2.5 text-subink break-words max-w-md">{e.detail || "—"}</td>
</tr>
))}
</tbody>
</table>
</div>
)}
{total > PAGE && (
<div className="flex items-center justify-between px-4 py-3 border-t border-line text-sm text-subink">
<span>
{offset + 1}–{Math.min(offset + PAGE, total)} of {total.toLocaleString()}
</span>
<div className="flex gap-2">
<button
className="inline-flex items-center gap-1 px-3 py-1.5 rounded-xl border border-line
bg-panel text-ink disabled:opacity-40"
disabled={offset === 0}
onClick={() => setOffset(Math.max(0, offset - PAGE))}
>
<ChevronLeft size={15} /> Newer
</button>
<button
className="inline-flex items-center gap-1 px-3 py-1.5 rounded-xl border border-line
bg-panel text-ink disabled:opacity-40"
disabled={offset + PAGE >= total}
onClick={() => setOffset(offset + PAGE)}
>
Older <ChevronRight size={15} />
</button>
</div>
</div>
)}
</Section>
</div>
);
}

88
ar-aging-app/frontend/src/pages/Closing.tsx
View File

@ -1,9 +1,8 @@
import { NavLink, Outlet, Route, Routes, useParams, useOutletContext } from "react-router-dom";
import { BadgeInfo, Clock, Lock, RefreshCw, ShieldAlert } from "lucide-react";
import { Clock, RefreshCw, ShieldAlert } from "lucide-react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { api, SessionT } from "../api/client";
import { StatusBadge, ProgressStages, Spinner } from "../components/ui";
import MonthSwitcher from "../components/MonthSwitcher";
import { date } from "../lib/format";
import Overview from "./closing/Overview";
import Controls from "./closing/Controls";
@ -19,13 +18,7 @@ import FinanceSummary from "./closing/FinanceSummary";
import JournalEntry from "./closing/JournalEntry";
import ExportPage from "./closing/ExportPage";
export interface ClosingCtx {
id: number;
session: SessionT;
processed: boolean;
/** Completed closings are read-only until explicitly reopened. */
locked: boolean;
}
export interface ClosingCtx { id: number; session: SessionT; processed: boolean }
export const useClosing = () => useOutletContext<ClosingCtx>();
const TABS = [
@ -50,23 +43,6 @@ function ReprocessButton({ id }: { id: number }) {
);
}
function ReopenButton({ id }: { id: number }) {
const qc = useQueryClient();
const reopen = useMutation({
mutationFn: () => api.reopenSession(id),
onSuccess: () => {
qc.invalidateQueries({ queryKey: ["session", id] });
qc.invalidateQueries({ queryKey: ["sessions"] });
},
});
return (
<button className="btn-ghost shrink-0" disabled={reopen.isPending}
onClick={() => reopen.mutate()}>
{reopen.isPending ? <Spinner /> : null} Reopen for corrections
</button>
);
}
export default function Closing() {
const { id } = useParams();
const sid = Number(id);
@ -79,31 +55,18 @@ export default function Closing() {
refetchIntervalInBackground: true, // keep progress updating if the tab isn't focused
});
// A blocked closing is fully processed — its tabs stay open for diagnosis, but every
// endpoint that publishes a receivable figure withholds it until the control is resolved.
const processed = session
? session.status === "processed" || session.status === "completed"
|| session.status === "blocked"
: false;
// Publish state: the same query key the Journal tab uses, so the cache is shared.
const { data: journal } = useQuery({
queryKey: ["journal", sid, ""],
queryFn: () => api.journal(sid),
enabled: processed,
});
if (isLoading || !session)
return <div className="p-6 flex items-center gap-2 text-subink"><Spinner /> Loading closing…</div>;
const locked = session.status === "completed";
const unpublished = processed && !session.blocked
&& journal?.available === true && !journal.approved_by;
// A blocked closing is fully processed — its tabs stay open for diagnosis, but every
// endpoint that publishes a receivable figure withholds it until the control is resolved.
const processed = session.status === "processed" || session.status === "completed"
|| session.status === "blocked";
return (
<div>
<header className="bg-panel border-b border-line px-6 pt-4">
<div className="flex items-center justify-between gap-4 flex-wrap">
<div className="flex items-center justify-between">
<div>
<h1 className="text-lg font-semibold text-ink">{session.name}</h1>
<p className="text-sm text-subink">
@ -111,11 +74,8 @@ export default function Closing() {
clearing-lag {session.clearing_lag_days}d
</p>
</div>
<div className="flex items-center gap-3">
<MonthSwitcher currentId={sid} />
<StatusBadge status={session.status} />
</div>
</div>
<nav className="flex gap-1 mt-4 -mb-px overflow-x-auto">
{TABS.map(([to, label]) => (
<NavLink key={to} to={to} end={to === ""}
@ -141,25 +101,13 @@ export default function Closing() {
<div className="card border-bad/40 bg-badbg/40 p-4 text-sm text-bad whitespace-pre-wrap">{session.error}</div>
</div>
)}
{locked && (
<div className="px-6 pt-4">
<div className="card border-line bg-neutralbg/50 p-3 flex items-center gap-3 text-sm">
<Lock size={16} className="text-subink shrink-0" />
<span className="flex-1">
This closing is <b>completed and locked</b> — its figures are read-only so
published history cannot drift. Reopen it only if a correction is genuinely needed.
</span>
<ReopenButton id={sid} />
</div>
</div>
)}
{session.needs_reprocess && session.status !== "processing" && !locked && (
{session.needs_reprocess && session.status !== "processing" && (
<div className="px-6 pt-4">
<div className="card border-warn/30 bg-warnbg/40 p-3 flex items-center gap-3 text-sm">
<Clock size={16} className="text-warn shrink-0" />
<span className="flex-1">
Inputs changed after the last run (files, bank receipts, or the payout mode) —
the figures on screen don't reflect them yet. <b>Re-process to apply.</b>
Bank receipts or the payout mode changed after the last run — the figures on
screen don't reflect them yet. <b>Re-process to apply.</b>
</span>
<ReprocessButton id={sid} />
</div>
@ -179,24 +127,10 @@ export default function Closing() {
</div>
</div>
)}
{unpublished && !locked && (
<div className="px-6 pt-4">
<div className="card border-line bg-primary-soft/30 p-3 flex items-center gap-3 text-sm">
<BadgeInfo size={16} className="text-primary shrink-0" />
<span className="flex-1">
This month is <b>not on the Accounts Summary yet</b> — approving the journal
is what publishes it{journal?.entry_no
? " (a re-process withdraws any earlier sign-off, so it may need re-approval)"
: ""}.
</span>
<NavLink to="journal" className="btn-ghost shrink-0">Open Journal Entry</NavLink>
</div>
</div>
)}
<div className="p-6">
<Routes>
<Route element={<Outlet context={{ id: sid, session, processed, locked } satisfies ClosingCtx} />}>
<Route element={<Outlet context={{ id: sid, session, processed } satisfies ClosingCtx} />}>
<Route index element={<Overview />} />
<Route path="controls" element={<Controls />} />
<Route path="opening" element={<OpeningBalances />} />

27
ar-aging-app/frontend/src/pages/Dashboard.tsx
View File

@ -1,7 +1,7 @@
import { Link, useNavigate } from "react-router-dom";
import { useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { AlertTriangle, BadgeCheck, FilePlus2, Trash2, ChevronRight } from "lucide-react";
import { FilePlus2, Trash2, ChevronRight } from "lucide-react";
import { api, SessionT } from "../api/client";
import { usd, date } from "../lib/format";
import { ConfirmDialog, EmptyState, Kpi, Spinner, StatusBadge } from "../components/ui";
@ -59,36 +59,15 @@ export default function Dashboard() {
<table className="w-full">
<thead><tr>
<th className="th">Name</th><th className="th">Month</th><th className="th">Month-end</th>
<th className="th">Status</th><th className="th">Published</th>
<th className="th text-right">Receivable (USD)</th><th className="th"></th>
<th className="th">Status</th><th className="th text-right">Receivable (USD)</th><th className="th"></th>
</tr></thead>
<tbody>
{sessions.map((s) => (
<tr key={s.id} className="hover:bg-canvas/60 cursor-pointer" onClick={() => nav(`/closing/${s.id}`)}>
<td className="td font-medium">{s.name}</td>
<td className="td num">
{s.reporting_month ?? "—"}
{s.duplicate_month && (
<span className="ml-1.5 inline-flex align-middle" title="Another closing exists for this month">
<AlertTriangle size={13} className="text-warn" />
</span>
)}
</td>
<td className="td num">{s.reporting_month ?? "—"}</td>
<td className="td num">{date(s.month_end_date)}</td>
<td className="td"><StatusBadge status={s.status} /></td>
<td className="td text-xs">
{s.journal_approved ? (
<span className="inline-flex items-center gap-1 text-ok font-medium">
<BadgeCheck size={13} /> published
</span>
) : s.status === "processed" || s.status === "completed" ? (
<span className="text-subink" title="Approve the journal to publish this month to the Accounts Summary">
not published
</span>
) : (
<span className="text-muted">—</span>
)}
</td>
<td className="td text-right num">{s.status === "processed" ? <LatestReceivable id={s.id} /> : "—"}</td>
<td className="td text-right">
<div className="flex items-center justify-end gap-1">

369
ar-aging-app/frontend/src/pages/Login.tsx
View File

@ -1,369 +0,0 @@
import { FormEvent, useState } from "react";
import { BookOpenCheck, Eye, EyeOff, FileSpreadsheet, Globe, Landmark, Lock, LogIn, Scale, ShieldCheck, User } from "lucide-react";
import { api } from "../api/client";
import { Spinner } from "../components/ui";
import { useAuth } from "../auth";
/**
* Sign-in screen: navy brand panel (desktop) + form on the app canvas.
* Uses the app's "ledgr" tokens — periwinkle primary, navy, lavender canvas — so the
* login feels like the first screen of the dashboard, not a bolt-on.
*/
export default function Login() {
const { login, emailEnabled } = useAuth();
const [username, setUsername] = useState("");
const [password, setPassword] = useState("");
const [showPw, setShowPw] = useState(false);
const [error, setError] = useState<string | null>(null);
const [busy, setBusy] = useState(false);
const [forgot, setForgot] = useState(false);
const submit = async (e: FormEvent) => {
e.preventDefault();
setError(null);
setBusy(true);
try {
await login(username.trim(), password);
} catch (err) {
setError((err as Error).message);
} finally {
setBusy(false);
}
};
return (
<div className="min-h-full flex bg-canvas">
{/* ---------------- brand panel (desktop) ---------------- */}
<aside
className="hidden lg:flex flex-col justify-between w-[44%] max-w-xl p-12 text-white relative overflow-hidden bg-navy"
style={{
backgroundImage: `
radial-gradient(52rem 30rem at -10% -20%, rgba(109,93,232,0.45), transparent 60%),
radial-gradient(40rem 26rem at 110% 115%, rgba(201,195,245,0.22), transparent 60%),
linear-gradient(rgba(255,255,255,0.045) 1px, transparent 1px),
linear-gradient(90deg, rgba(255,255,255,0.045) 1px, transparent 1px)`,
backgroundSize: "auto, auto, 100% 3.25rem, 3.25rem 100%",
}}
>
<div className="flex items-center gap-3">
<span className="inline-flex items-center justify-center w-11 h-11 rounded-2xl bg-primary text-white shadow-pop">
<Landmark size={22} />
</span>
<div>
<div className="text-base font-semibold leading-tight">Amazon A/R Aging</div>
<div className="text-xs text-lavender">Month-End Closing · Utopia Brands Finance</div>
</div>
</div>
<div className="max-w-md">
<h1 className="text-3xl xl:text-4xl font-semibold leading-tight [text-wrap:balance]">
Every month closed, checked, and accounted for.
</h1>
<p className="mt-4 text-sm leading-relaxed text-lavender-soft/90">
Upload the month's Amazon transaction files and get the full receivable
position — settlements, aging, journal entry, and the audit trail behind
every figure.
</p>
<ul className="mt-8 space-y-3 text-sm">
{[
{ Icon: FileSpreadsheet,
text: "Parses the month's Amazon transaction files — 13 marketplaces, millions of rows" },
{ Icon: Scale,
text: "Classifies every settlement paid vs receivable and computes the closing position" },
{ Icon: BookOpenCheck,
text: "AR roll-forward and month-end journal entry, reconciled to the cent" },
{ Icon: Globe,
text: "Each marketplace converted at confirmed month-end ECB rates (local → USD)" },
{ Icon: ShieldCheck,
text: "Six month-end controls and a SHA-256 audit trail gate every published figure" },
].map(({ Icon, text }) => (
<li key={text} className="flex items-start gap-3">
<span className="mt-0.5 inline-flex items-center justify-center w-6 h-6 rounded-lg bg-white/10 text-lavender shrink-0">
<Icon size={14} />
</span>
<span className="text-lavender-soft/90 leading-snug">{text}</span>
</li>
))}
</ul>
</div>
<p className="text-[11px] text-lavender/60">
Amazon-only · 13 marketplaces · reconciles to the cent
</p>
</aside>
{/* ---------------- sign-in form ---------------- */}
<main
className="flex-1 flex items-center justify-center p-6 sm:p-10"
style={{
backgroundImage:
"radial-gradient(38rem 24rem at 85% -10%, rgba(109,93,232,0.10), transparent 65%)",
}}
>
<div className="w-full max-w-sm motion-safe:animate-[login-in_.45s_ease-out]">
{/* compact brand header for mobile, where the panel is hidden */}
<div className="lg:hidden flex items-center gap-2.5 mb-8">
<span className="inline-flex items-center justify-center w-9 h-9 rounded-xl bg-primary text-white shadow-card">
<Landmark size={18} />
</span>
<div>
<div className="text-sm font-semibold leading-tight text-ink">Amazon A/R Aging</div>
<div className="text-[11px] text-muted">Month-End Closing</div>
</div>
</div>
{forgot ? (
<ForgotPassword
initialUsername={username}
onDone={() => setForgot(false)}
/>
) : (
<>
<h2 className="text-2xl font-semibold text-ink">Welcome back</h2>
<p className="mt-1 text-sm text-subink">Sign in to continue to this month's closing.</p>
<form onSubmit={submit} className="mt-8 space-y-4">
<div>
<label className="label" htmlFor="login-user">Username</label>
<div className="relative">
<User size={16} className="absolute left-3.5 top-1/2 -translate-y-1/2 text-faint pointer-events-none" />
<input
id="login-user"
className="input pl-10 py-2.5"
placeholder="you@utopiabrands.com"
autoComplete="username"
autoFocus
value={username}
onChange={(e) => setUsername(e.target.value)}
/>
</div>
</div>
<div>
<label className="label" htmlFor="login-pw">Password</label>
<div className="relative">
<Lock size={16} className="absolute left-3.5 top-1/2 -translate-y-1/2 text-faint pointer-events-none" />
<input
id="login-pw"
className="input pl-10 pr-11 py-2.5"
type={showPw ? "text" : "password"}
autoComplete="current-password"
value={password}
onChange={(e) => setPassword(e.target.value)}
/>
<button
type="button"
aria-label={showPw ? "Hide password" : "Show password"}
className="absolute right-2 top-1/2 -translate-y-1/2 p-1.5 rounded-lg text-faint hover:text-subink hover:bg-neutralbg transition-colors"
onClick={() => setShowPw((v) => !v)}
tabIndex={-1}
>
{showPw ? <EyeOff size={16} /> : <Eye size={16} />}
</button>
</div>
</div>
{error && (
<div className="rounded-xl border border-bad/30 bg-badbg/60 px-3.5 py-2.5 text-sm text-bad">
{error}
</div>
)}
<button
className="btn-primary w-full justify-center py-2.5 text-[15px]"
type="submit"
disabled={busy || !username.trim() || !password}
>
{busy ? <Spinner /> : <LogIn size={16} />} Sign in
</button>
{emailEnabled && (
<div className="text-right">
<button type="button"
className="text-xs font-medium text-primary hover:underline"
onClick={() => { setError(null); setForgot(true); }}>
Forgot password?
</button>
</div>
)}
</form>
<p className="mt-8 text-xs text-muted leading-relaxed">
{emailEnabled
? "No account? Ask the administrator — accounts are created on the server."
: "No account or forgot your password? Ask the administrator — accounts are created and reset on the server."}
</p>
</>
)}
</div>
</main>
{/* one-time entrance; browsers honouring reduced motion skip it via motion-safe */}
<style>{`@keyframes login-in { from { opacity: 0; transform: translateY(10px); } to { opacity: 1; transform: none; } }`}</style>
</div>
);
}
/** Forgot-password, one step at a time:
* 1. email → send the code 2. enter + verify the code 3. set the new password. */
function ForgotPassword({ initialUsername, onDone }: {
initialUsername: string; onDone: () => void;
}) {
const [step, setStep] = useState<"email" | "code" | "password" | "done">("email");
const [email, setEmail] = useState(initialUsername);
const [code, setCode] = useState("");
const [next, setNext] = useState("");
const [repeat, setRepeat] = useState("");
const [error, setError] = useState<string | null>(null);
const [busy, setBusy] = useState(false);
const run = async (fn: () => Promise<void>) => {
setError(null); setBusy(true);
try {
await fn();
} catch (e) {
setError((e as Error).message);
} finally {
setBusy(false);
}
};
const sendCode = () => run(async () => {
await api.requestPasswordCode(email.trim());
setCode("");
setStep("code");
});
const verifyCode = (e: FormEvent) => {
e.preventDefault();
run(async () => {
await api.verifyPasswordCode(code.trim(), email.trim());
setStep("password");
});
};
const reset = (e: FormEvent) => {
e.preventDefault();
run(async () => {
await api.resetPassword(code.trim(), next, email.trim());
setStep("done");
});
};
const Err = () => error && (
<div className="rounded-xl border border-bad/30 bg-badbg/60 px-3.5 py-2.5 text-sm text-bad">
{error}
</div>
);
const Back = () => (
<div className="text-center">
<button type="button" className="text-xs font-medium text-subink hover:text-ink"
onClick={onDone}>
← Back to sign in
</button>
</div>
);
if (step === "done")
return (
<div>
<h2 className="text-2xl font-semibold text-ink">Password updated</h2>
<p className="mt-2 text-sm text-subink">Sign in with your new password.</p>
<button className="btn-primary w-full justify-center py-2.5 mt-6" onClick={onDone}>
<LogIn size={16} /> Back to sign in
</button>
</div>
);
if (step === "email")
return (
<div>
<h2 className="text-2xl font-semibold text-ink">Reset password</h2>
<p className="mt-1 text-sm text-subink">
Step 1 of 3 — we'll email a 6-digit code to your account's address.
</p>
<form className="mt-8 space-y-4" onSubmit={(e) => { e.preventDefault(); sendCode(); }}>
<div>
<label className="label" htmlFor="fp-user">Username (email)</label>
<div className="relative">
<User size={16} className="absolute left-3.5 top-1/2 -translate-y-1/2 text-faint pointer-events-none" />
<input id="fp-user" className="input pl-10 py-2.5" autoComplete="username"
autoFocus placeholder="you@utopiabrands.com"
value={email} onChange={(e) => setEmail(e.target.value)} />
</div>
<p className="mt-1.5 text-xs text-muted">
Your <b>@utopiabrands.com</b> account address — codes only go to registered accounts.
</p>
</div>
<Err />
<button className="btn-primary w-full justify-center py-2.5" type="submit"
disabled={busy || !email.trim()}>
{busy ? <Spinner /> : null} Email me a code
</button>
<Back />
</form>
</div>
);
if (step === "code")
return (
<div>
<h2 className="text-2xl font-semibold text-ink">Enter the code</h2>
<p className="mt-1 text-sm text-subink">
Step 2 of 3 — sent to <b className="text-ink">{email.trim()}</b>, valid 10 minutes.
</p>
<form className="mt-8 space-y-4" onSubmit={verifyCode}>
<div>
<label className="label" htmlFor="fp-code">6-digit code from the email</label>
<input id="fp-code" className="input py-2.5 num tracking-[0.35em] text-center"
inputMode="numeric" maxLength={6} autoFocus placeholder="••••••"
value={code} onChange={(e) => setCode(e.target.value.replace(/\D/g, ""))} />
</div>
<Err />
<button className="btn-primary w-full justify-center py-2.5" type="submit"
disabled={busy || code.length !== 6}>
{busy ? <Spinner /> : null} Verify code
</button>
<div className="flex items-center justify-between text-xs">
<button type="button" className="font-medium text-subink hover:text-ink"
onClick={() => setStep("email")}>
← Different email
</button>
<button type="button" className="font-medium text-primary hover:underline"
disabled={busy} onClick={sendCode}>
Resend code
</button>
</div>
<Back />
</form>
</div>
);
return (
<div>
<h2 className="text-2xl font-semibold text-ink">Choose a new password</h2>
<p className="mt-1 text-sm text-subink">Step 3 of 3 — code verified ✓</p>
<form className="mt-8 space-y-4" onSubmit={reset}>
<div>
<label className="label" htmlFor="fp-new">New password (min 8)</label>
<input id="fp-new" className="input py-2.5" type="password" autoFocus
autoComplete="new-password"
value={next} onChange={(e) => setNext(e.target.value)} />
</div>
<div>
<label className="label" htmlFor="fp-rep">Repeat new password</label>
<input id="fp-rep" className="input py-2.5" type="password"
autoComplete="new-password"
value={repeat} onChange={(e) => setRepeat(e.target.value)} />
{repeat.length > 0 && next !== repeat &&
<p className="mt-1.5 text-xs text-bad">Passwords don't match.</p>}
</div>
<Err />
<button className="btn-primary w-full justify-center py-2.5" type="submit"
disabled={busy || next.length < 8 || next !== repeat}>
{busy ? <Spinner /> : null} Set new password
</button>
<Back />
</form>
</div>
);
}

88
ar-aging-app/frontend/src/pages/NewClosing.tsx
View File

@ -1,7 +1,6 @@
import { useEffect, useRef, useState } from "react";
import { useState } from "react";
import { useNavigate } from "react-router-dom";
import { useMutation, useQuery } from "@tanstack/react-query";
import { AlertTriangle } from "lucide-react";
import { api } from "../api/client";
type OpeningMode = "zero" | "carry_forward" | "manual";
@ -11,50 +10,22 @@ function lastDayOfMonth(ym: string): string {
return new Date(y, m, 0).toISOString().slice(0, 10);
}
function nextMonth(ym: string): string {
const [y, m] = ym.split("-").map(Number);
const d = new Date(y, m, 1); // month is 0-based, so this is the month AFTER ym
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
}
function currentMonth(): string {
const d = new Date();
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
}
export default function NewClosing() {
const nav = useNavigate();
const [month, setMonth] = useState(currentMonth());
const [month, setMonth] = useState("2026-01");
const [name, setName] = useState("");
const [lag, setLag] = useState(2);
const [allowance, setAllowance] = useState(0);
const [openingMode, setOpeningMode] = useState<OpeningMode>("zero");
const [sourceId, setSourceId] = useState<number | undefined>();
const [forceDuplicate, setForceDuplicate] = useState(false);
// Once the user touches month/opening themselves, stop auto-defaulting over their choice.
const touched = useRef({ month: false, opening: false });
const { data: sessions } = useQuery({ queryKey: ["sessions"], queryFn: api.listSessions });
// Prior closings whose closing balance can be carried into this one.
const { data: sessions } = useQuery({ queryKey: ["sessions"], queryFn: api.listSessions });
const priors = (sessions ?? []).filter(
(s) => s.status === "processed" || s.status === "completed",
);
const effectiveSource = sourceId ?? priors[0]?.id;
// Smart defaults once the sessions load: the month AFTER the latest closing, opening
// carried forward from it — the normal month-to-month flow needs zero clicks.
useEffect(() => {
if (!sessions?.length) return;
const latest = sessions.find((s) => s.reporting_month)?.reporting_month; // list is newest-first
if (latest && !touched.current.month) setMonth(nextMonth(latest));
if (priors.length > 0 && !touched.current.opening) setOpeningMode("carry_forward");
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [sessions]);
const existing = (sessions ?? []).filter(
(s) => s.reporting_month === month && s.status !== "error");
const isDuplicate = existing.length > 0;
const create = useMutation({
mutationFn: () =>
api.createSession({
@ -65,7 +36,6 @@ export default function NewClosing() {
reporting_currency: "USD",
opening_mode: openingMode,
opening_source_session_id: openingMode === "carry_forward" ? effectiveSource : null,
allow_duplicate: forceDuplicate,
}),
onSuccess: (s) => nav(`/closing/${s.id}/upload`),
});
@ -80,8 +50,7 @@ export default function NewClosing() {
}`}>
<div className="flex items-start gap-2.5">
<input type="radio" name="opening" className="mt-1 accent-[#6D5DE8]"
checked={active}
onChange={() => { touched.current.opening = true; setOpeningMode(value); }} />
checked={active} onChange={() => setOpeningMode(value)} />
<div className="flex-1">
<div className="text-sm font-medium text-ink">{title}</div>
<div className="text-xs text-subink mt-0.5">{desc}</div>
@ -96,48 +65,15 @@ export default function NewClosing() {
<div className="p-6 max-w-2xl mx-auto space-y-6">
<header>
<h1 className="text-xl font-semibold text-ink">New Month-End Closing</h1>
<p className="text-sm text-subink">Set the reporting period, then upload the Amazon transaction files. Every month stays saved as its own closing — new months never overwrite previous ones.</p>
<p className="text-sm text-subink">Set the reporting period, then upload the Amazon transaction files.</p>
</header>
<div className="card p-5 space-y-4">
<div>
<label className="label">Reporting month</label>
<input type="month" className="input" value={month}
onChange={(e) => {
touched.current.month = true;
setForceDuplicate(false);
setMonth(e.target.value);
}} />
<input type="month" className="input" value={month} onChange={(e) => setMonth(e.target.value)} />
<p className="text-xs text-subink mt-1">Month-end date will be {month ? lastDayOfMonth(month) : "—"} (auto).</p>
</div>
{isDuplicate && (
<div className="rounded-xl border border-warn/40 bg-warnbg/40 p-3 space-y-2">
<p className="text-sm text-ink flex items-start gap-2">
<AlertTriangle size={16} className="text-warn shrink-0 mt-0.5" />
<span>
A closing for <b>{month}</b> already exists:{" "}
<b>{existing[0].name}</b>. Two closings for one month means two competing
datasets for the same period.
</span>
</p>
<div className="flex items-center gap-2 pl-6">
<button className="btn-primary" onClick={() => nav(`/closing/${existing[0].id}`)}>
Open the existing closing
</button>
{forceDuplicate ? (
<span className="text-xs text-warn font-medium">
Creating a second closing for {month} — deliberate.
</span>
) : (
<button className="btn-ghost text-xs" onClick={() => setForceDuplicate(true)}>
I need another one anyway
</button>
)}
</div>
</div>
)}
<div>
<label className="label">Closing name</label>
<input className="input" placeholder={`Amazon A/R Aging — ${month}`} value={name} onChange={(e) => setName(e.target.value)} />
@ -159,14 +95,12 @@ export default function NewClosing() {
<div className="pt-1">
<label className="label">Opening AR balance</label>
<div className="space-y-2">
<Option value="zero" title="Start at zero"
<Option value="zero" title="Start at zero (default)"
desc="Every marketplace opens at 0. Use this for your first-ever closing." />
<Option value="carry_forward"
title={priors.length > 0
? "Carry forward from a previous closing (default)"
: "Carry forward from a previous closing"}
desc="Copies each marketplace's closing receivable into this month's opening balance — the normal month-to-month flow.">
title="Carry forward from a previous closing"
desc="Copies each marketplace's closing receivable into this month's opening balance.">
{priors.length === 0 ? (
<p className="text-xs text-warn mt-2">
No processed closing available yet — this will fall back to zero.
@ -195,9 +129,7 @@ export default function NewClosing() {
{create.isError && <p className="text-sm text-bad">{(create.error as Error).message}</p>}
<div className="flex justify-end gap-2 pt-2">
<button className="btn-ghost" onClick={() => nav("/")}>Cancel</button>
<button className="btn-primary"
disabled={create.isPending || !month || (isDuplicate && !forceDuplicate)}
onClick={() => create.mutate()}>
<button className="btn-primary" disabled={create.isPending || !month} onClick={() => create.mutate()}>
{create.isPending ? "Creating…" : "Create & upload files"}
</button>
</div>

75
ar-aging-app/frontend/src/pages/Settings.tsx
View File

@ -1,13 +1,10 @@
import { FormEvent, useState } from "react";
import { useMutation, useQuery } from "@tanstack/react-query";
import { KeyRound, ShieldCheck } from "lucide-react";
import { useQuery } from "@tanstack/react-query";
import { ShieldCheck } from "lucide-react";
import { api } from "../api/client";
import { Section, Spinner } from "../components/ui";
import { useAuth } from "../auth";
import { Section } from "../components/ui";
export default function Settings() {
const { data: health } = useQuery({ queryKey: ["health"], queryFn: api.health });
const { user } = useAuth();
return (
<div className="p-6 max-w-3xl mx-auto space-y-6">
@ -16,8 +13,6 @@ export default function Settings() {
<p className="text-sm text-subink">Application defaults and security posture.</p>
</header>
{user && <ChangePassword username={user.username} />}
<Section title="Processing defaults">
<dl className="divide-y divide-line">
{[
@ -39,17 +34,10 @@ export default function Settings() {
<div className="p-4 flex items-start gap-3">
<ShieldCheck className="text-ok shrink-0" size={22} />
<ul className="text-sm text-subink space-y-1.5">
<li>Files are processed locally on the company server — no third-party or AI services.
The one exception: exchange-rate lookups send currency codes and dates to the
configured FX provider (Frankfurter/ECB by default). No financial data ever leaves.</li>
<li>Login is per-user; journal review/approval and FX confirmations record the
signed-in person's verified name. Accounts are created by the administrator.</li>
<li>Uploads are session-scoped; filenames are sanitized; no public file URLs.
Re-uploads replace their file — a month can never count a file twice.</li>
<li>Full financial transaction rows are not logged. Generated exports are purged
after the retention window; uploaded source files are kept as the audit source.</li>
<li>Files are processed locally on the company server — no third-party or AI services.</li>
<li>Uploads are session-scoped; filenames are sanitized; no public file URLs.</li>
<li>Full financial transaction rows are not logged; temporary files follow a retention policy.</li>
<li>Every generated workbook includes a Processing Audit Trail with SHA-256 file hashes.</li>
<li>Completed closings are locked read-only; corrections require an explicit reopen.</li>
</ul>
</div>
</Section>
@ -60,54 +48,3 @@ export default function Settings() {
</div>
);
}
function ChangePassword({ username }: { username: string }) {
const [current, setCurrent] = useState("");
const [next, setNext] = useState("");
const [repeat, setRepeat] = useState("");
const change = useMutation({
mutationFn: () => api.changePassword(current, next),
onSuccess: () => { setCurrent(""); setNext(""); setRepeat(""); },
});
const mismatch = repeat.length > 0 && next !== repeat;
const tooShort = next.length > 0 && next.length < 8;
const ready = current && next.length >= 8 && next === repeat;
const submit = (e: FormEvent) => {
e.preventDefault();
if (ready) change.mutate();
};
return (
<Section title="Change my password"
subtitle={`Signed in as ${username}. Forgot the current one? Sign out and use "Forgot password?" on the login screen — a code is emailed to you.`}>
<form onSubmit={submit} className="p-4 grid grid-cols-1 sm:grid-cols-3 gap-3 items-end">
<label className="text-sm">
<span className="block text-xs font-medium text-subink mb-1">Current password</span>
<input className="input" type="password" autoComplete="current-password"
value={current} onChange={(e) => setCurrent(e.target.value)} />
</label>
<label className="text-sm">
<span className="block text-xs font-medium text-subink mb-1">New password (min 8)</span>
<input className="input" type="password" autoComplete="new-password"
value={next} onChange={(e) => setNext(e.target.value)} />
</label>
<label className="text-sm">
<span className="block text-xs font-medium text-subink mb-1">Repeat new password</span>
<input className="input" type="password" autoComplete="new-password"
value={repeat} onChange={(e) => setRepeat(e.target.value)} />
</label>
<div className="sm:col-span-3 flex items-center gap-3 flex-wrap">
<button className="btn-primary" type="submit" disabled={!ready || change.isPending}>
{change.isPending ? <Spinner /> : <KeyRound size={15} />} Update password
</button>
{tooShort && <span className="text-xs text-warn">At least 8 characters.</span>}
{mismatch && <span className="text-xs text-bad">Passwords don't match.</span>}
{change.isSuccess && <span className="text-xs text-ok">Password updated — use it from your next sign-in.</span>}
{change.isError && <span className="text-xs text-bad">{(change.error as Error).message}</span>}
</div>
</form>
</Section>
);
}

34
ar-aging-app/frontend/src/pages/closing/Aging.tsx
View File

@ -1,28 +1,14 @@
import { useState } from "react";
import { useQuery } from "@tanstack/react-query";
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from "recharts";
import { AgingSchemeT, api } from "../../api/client";
import { api } from "../../api/client";
import { usd } from "../../lib/format";
import { BlockedNotice, InfoTip, Section, EmptyState, useDefinitions } from "../../components/ui";
import { useClosing } from "../Closing";
const SCHEMES: { key: AgingSchemeT; label: string }[] = [
{ key: "weekly", label: "Weekly" },
{ key: "monthly", label: "Monthly" },
{ key: "half_year", label: "6 months" },
{ key: "yearly", label: "Yearly" },
];
export default function Aging() {
const { id, processed } = useClosing();
const defs = useDefinitions();
const [scheme, setScheme] = useState<AgingSchemeT>("monthly");
const { data } = useQuery({
queryKey: ["aging", id, scheme],
queryFn: () => api.aging(id, scheme),
enabled: processed,
placeholderData: (prev) => prev, // keep the table while the new bands load
});
const { data } = useQuery({ queryKey: ["aging", id], queryFn: () => api.aging(id), enabled: processed });
if (!processed) return <EmptyState title="Process the closing to see the A/R aging." />;
if (data?.blocked) return <BlockedNotice reason={data.blocked_reason} />;
@ -34,21 +20,7 @@ export default function Aging() {
<div className="space-y-6">
<Section title="Accounts Receivable Aging"
subtitle="Banded by days past due at month-end — a settlement is due 14 days after its last activity plus the clearing lag."
actions={
<div className="flex items-center gap-2">
<div className="flex rounded-lg border border-line overflow-hidden text-xs">
{SCHEMES.map((s) => (
<button key={s.key}
className={`px-2.5 py-1.5 transition-colors ${
scheme === s.key ? "bg-primary text-white" : "bg-panel text-subink hover:text-ink"}`}
onClick={() => setScheme(s.key)}>
{s.label}
</button>
))}
</div>
<InfoTip def={defs.aging_basis} label="How the aging bands work" />
</div>
}>
actions={<InfoTip def={defs.aging_basis} label="How the aging bands work" />}>
<div className="overflow-auto">
<table className="w-full">
<thead><tr>

84
ar-aging-app/frontend/src/pages/closing/ArLedger.tsx
View File

@ -1,7 +1,7 @@
import { ReactNode, useEffect, useState } from "react";
import { useMutation, useQuery, useQueryClient, keepPreviousData } from "@tanstack/react-query";
import { ArrowDownRight, ArrowUpRight, CloudDownload, Pencil, Save, CalendarRange,
RotateCcw, CornerDownRight, Loader2 } from "lucide-react";
import { ArrowDownRight, ArrowUpRight, Pencil, Save, CalendarRange,
RotateCcw, CornerDownRight } from "lucide-react";
import { api, OpeningBalanceT } from "../../api/client";
import { money, acct, num } from "../../lib/format";
import { InfoTip, Section, EmptyState, StatusBadge, useDefinitions } from "../../components/ui";
@ -54,13 +54,6 @@ export default function ArLedger() {
const mkt = mv.marketplace ?? "USA";
const cur = mv.currency ?? "USD";
const m = (v: number | null | undefined, dp = 0) => money(v, cur, dp);
// USD-reporting marketplaces would just repeat every figure — show the pair only when
// the local currency actually differs.
const dual = cur !== "USD";
const inUsd = (v: number | null | undefined, dp = 2) =>
dual && v != null ? (
<div className="text-[11px] leading-tight text-subink">{money(v, "USD", dp)}</div>
) : null;
const opening = openings?.find((o) => o.marketplace === mkt);
const diff = mv.difference_vs_settlement ?? 0;
const reconciled = Math.abs(diff) < 1;
@ -107,14 +100,6 @@ export default function ArLedger() {
<span className="font-semibold text-primary">= Closing receivable</span>
<span className="num text-lg font-semibold text-primary">{m(mv.closing)}</span>
</div>
{dual && detail?.month_rate != null && (
<div className="flex items-center justify-between text-xs text-subink">
<span>in USD @ month rate {num(detail.month_rate, 6)}</span>
<span className="num font-medium">
{money((mv.closing ?? 0) * detail.month_rate, "USD", 2)}
</span>
</div>
)}
<p className="text-xs text-subink pt-2">
In-transit payouts of <span className="num">{m(mv.in_transit_payouts)}</span> remain
in receivable (not yet cleared).
@ -180,8 +165,7 @@ export default function ArLedger() {
{/* ---------------- date-filtered movement ---------------- */}
<Section title="Movement by date"
subtitle={`Daily, weekly or monthly view of the same ledger. Unfiltered, the running balance ends at the closing receivable.${
dual ? " USD figures (grey) are converted at each transaction date's exchange rate." : ""}`}>
subtitle="Daily, weekly or monthly view of the same ledger. Unfiltered, the running balance ends at the closing receivable.">
<div className="p-4 flex flex-wrap items-end gap-3 border-b border-line">
<div className="flex gap-1 p-1 rounded-xl bg-neutralbg">
{(["day", "week", "month"] as Gran[]).map((g) => (
@ -229,40 +213,26 @@ export default function ArLedger() {
<tr className="bg-neutralbg/50 font-medium">
<td className="td">Opening</td>
<td className="td" /><td className="td" /><td className="td" /><td className="td" />
<td className="td text-right num">
{m(detail?.opening)}
{inUsd(detail?.opening_usd)}
</td>
<td className="td text-right num">{m(detail?.opening)}</td>
</tr>
{(detail?.periods ?? []).map((p) => (
<tr key={p.key}>
<td className="td">{p.label}</td>
<td className="td text-right num text-xs text-subink">{p.rows.toLocaleString()}</td>
<td className="td text-right num">
{acct(p.revenue)}
{inUsd(p.revenue_usd)}
</td>
<td className="td text-right num">{acct(p.revenue)}</td>
<td className="td text-right num text-bad">
{p.payouts_received ? acct(p.payouts_received) : ""}
{p.payouts_received ? inUsd(p.payouts_received_usd) : null}
</td>
<td className="td text-right num text-warn">
{p.payouts_in_transit ? acct(p.payouts_in_transit) : ""}
{p.payouts_in_transit ? inUsd(p.payouts_in_transit_usd) : null}
</td>
<td className="td text-right num font-medium">
{m(p.balance)}
{inUsd(p.balance_usd)}
</td>
<td className="td text-right num font-medium">{m(p.balance)}</td>
</tr>
))}
<tr className="bg-primary-soft/50 font-semibold">
<td className="td text-primary">Closing</td>
<td className="td" /><td className="td" /><td className="td" /><td className="td" />
<td className="td text-right num text-primary">
{m(detail?.closing)}
{inUsd(detail?.closing_usd)}
</td>
<td className="td text-right num text-primary">{m(detail?.closing)}</td>
</tr>
</tbody>
</table>
@ -277,8 +247,7 @@ export default function ArLedger() {
{/* ---------------- daily FX ---------------- */}
<Section title={`Daily exchange rates — ${mkt}`}
subtitle="Local value per day, the USD rate applied, and the USD equivalent."
actions={cur !== "USD" ? <FetchDailyRates id={id} mkt={mkt} /> : undefined}>
subtitle="Local value per day, the USD rate applied, and the USD equivalent.">
{cur === "USD" && (
<div className="px-4 pt-3 text-xs text-subink">
{mkt} reports in USD — no conversion applied (rate 1.000000).
@ -314,11 +283,8 @@ export default function ArLedger() {
</table>
</div>
<p className="px-4 py-3 text-xs text-subink border-t border-line">
Daily rates are fetched from the FX provider automatically when the closing is
processed; each movement converts at the rate effective on its transaction date — a
date without a fixing (weekend or holiday) uses the previous banking day&apos;s rate.
The month rate ({num(fx?.month_rate, 6)}) applies to the opening balance and any date
with no fetched rate. Every rate used is shown here so the conversion is auditable.
Rates default to the marketplace month rate ({num(fx?.month_rate, 6)}). Every rate used is
shown here so the conversion is auditable.
</p>
</Section>
@ -332,36 +298,6 @@ export default function ArLedger() {
);
}
/** Re-fetch the official (ECB via Frankfurter) daily rates for the closing's transaction
* dates. Processing already fetches them automatically — this button retries after an
* outage or replaces hand-entered overrides with official fixings. */
function FetchDailyRates({ id, mkt }: { id: number; mkt: string }) {
const qc = useQueryClient();
const { locked } = useClosing();
const fetchDaily = useMutation({
mutationFn: () => api.fetchFxDaily(id, { marketplace: mkt }),
onSuccess: () => qc.invalidateQueries({ queryKey: ["fx-daily", id] }),
});
return (
<div className="flex items-center gap-2">
{fetchDaily.isSuccess && (
<span className="text-xs text-ok">
{fetchDaily.data.saved} daily rate(s) loaded ({fetchDaily.data.provider}).
</span>
)}
{fetchDaily.isError && (
<span className="text-xs text-bad">{(fetchDaily.error as Error).message}</span>
)}
<button className="btn-ghost" disabled={fetchDaily.isPending || locked}
title="Re-fetch the official daily rates for the closing's transaction dates. Hand-entered overrides are replaced for the fetched dates."
onClick={() => fetchDaily.mutate()}>
{fetchDaily.isPending ? <Loader2 size={15} className="animate-spin" /> : <CloudDownload size={15} />}
Fetch daily rates
</button>
</div>
);
}
function Row({ label, v, cur, bold, pos }: {
label: ReactNode; v: number; cur: string; bold?: boolean; pos?: boolean;
}) {

106
ar-aging-app/frontend/src/pages/closing/Controls.tsx
View File

@ -1,9 +1,8 @@
import { useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { CheckCircle2, XCircle, MinusCircle, AlertTriangle, CloudDownload, RefreshCw, ShieldCheck, UserCircle2 } from "lucide-react";
import { CheckCircle2, XCircle, MinusCircle, AlertTriangle, RefreshCw, ShieldCheck } from "lucide-react";
import { api, MonthEndControlT } from "../../api/client";
import { Section, EmptyState, Spinner } from "../../components/ui";
import { useAuth } from "../../auth";
import { useClosing } from "../Closing";
const ICON = {
@ -20,45 +19,23 @@ function tone(r: MonthEndControlT) {
}
export default function Controls() {
const { id, session, locked } = useClosing();
const { user } = useAuth();
const { id, session } = useClosing();
const qc = useQueryClient();
const [who, setWho] = useState("");
// Signed in -> the verified identity confirms; the free-text box only exists without auth.
const confirmer = user?.display_name ?? who;
const [edits, setEdits] = useState<Record<string, string>>({});
const { data, isLoading } = useQuery({
queryKey: ["controls", id], queryFn: () => api.controls(id),
});
const fxFailing = (data?.controls ?? []).some((r) => r.key === "C5" && r.status === "fail");
const { data: fx } = useQuery({
queryKey: ["fx", id], queryFn: () => api.getFx(id), enabled: fxFailing,
});
const invalidate = () => {
qc.invalidateQueries({ queryKey: ["controls", id] });
qc.invalidateQueries({ queryKey: ["session", id] });
qc.invalidateQueries({ queryKey: ["summary", id] });
qc.invalidateQueries({ queryKey: ["sessions"] });
qc.invalidateQueries({ queryKey: ["fx", id] });
};
const rerun = useMutation({ mutationFn: () => api.runControls(id), onSuccess: invalidate });
const confirmFx = useMutation({
mutationFn: () => api.confirmAllFx(id, confirmer.trim()), onSuccess: invalidate,
});
const fetchRates = useMutation({
mutationFn: () => api.fetchFx(id), onSuccess: invalidate,
});
// Saving marks the rates source=manual and (by design) withdraws any prior confirmation
// for a changed rate — the person then confirms the corrected value below.
const saveFx = useMutation({
mutationFn: () => api.putFx(id, (fx ?? []).map((r) => ({
marketplace: r.marketplace, currency: r.currency,
rate: Number(edits[r.marketplace] ?? r.rate),
}))),
onSuccess: () => { setEdits({}); invalidate(); },
mutationFn: () => api.confirmAllFx(id, who.trim()), onSuccess: invalidate,
});
if (isLoading) return <div className="p-6 flex items-center gap-2 text-subink"><Spinner /> Loading controls…</div>;
@ -66,9 +43,7 @@ export default function Controls() {
return <EmptyState title="No controls have run yet."
hint="Process the closing — the month-end controls run automatically at the end of processing." />;
const dirty = Object.entries(edits).some(
([m, v]) => Number(v) !== (fx ?? []).find((r) => r.marketplace === m)?.rate);
const invalid = Object.values(edits).some((v) => !Number.isFinite(Number(v)) || Number(v) <= 0);
const fxFailing = data.controls.some((r) => r.key === "C5" && r.status === "fail");
return (
<div className="space-y-6">
@ -100,87 +75,24 @@ export default function Controls() {
{fxFailing && (
<Section title="Confirm exchange rates"
subtitle={`Control C5 requires a rate confirmed for ${session.reporting_month ?? "this month"}. Seeded defaults are a January-2026 snapshot and are treated as missing.`}
actions={
<button className="btn-ghost" disabled={fetchRates.isPending || locked}
title="Fetch official month-end rates (ECB via Frankfurter). Fetched rates still need your confirmation below."
onClick={() => fetchRates.mutate()}>
{fetchRates.isPending ? <Spinner /> : <CloudDownload size={15} />}
Fetch month-end rates
</button>
}>
{fetchRates.isSuccess && (
<p className="px-4 pt-3 text-xs text-ok">
Fetched {fetchRates.data.updated.length} rate(s) from {fetchRates.data.source}.
{fetchRates.data.missing.length > 0 &&
` No rate available for: ${fetchRates.data.missing.join(", ")} — enter those manually.`}{" "}
Review the rates, then confirm them below.
</p>
)}
{fetchRates.isError && (
<p className="px-4 pt-3 text-xs text-bad">{(fetchRates.error as Error).message}</p>
)}
<div className="overflow-x-auto border-b border-line">
<table className="w-full">
<thead><tr>
<th className="th">Marketplace</th><th className="th">Currency</th>
<th className="th text-right">Rate → USD</th><th className="th">Source</th>
</tr></thead>
<tbody>
{(fx ?? []).map((r) => (
<tr key={r.marketplace}>
<td className="td font-medium">{r.marketplace}</td>
<td className="td">{r.currency}</td>
<td className="td text-right">
<input className="input num w-36 py-1 text-right"
value={edits[r.marketplace] ?? String(r.rate)}
onChange={(e) =>
setEdits((p) => ({ ...p, [r.marketplace]: e.target.value }))} />
</td>
<td className="td text-xs text-subink">{r.source}</td>
</tr>
))}
{!fx?.length && (
<tr><td className="td text-subink" colSpan={4}>No rates yet — process the closing first.</td></tr>
)}
</tbody>
</table>
</div>
subtitle={`Control C5 requires a rate confirmed for ${session.reporting_month ?? "this month"}. Seeded defaults are a January-2026 snapshot and are treated as missing.`}>
<div className="p-4 flex flex-wrap items-end gap-3">
{dirty && (
<button className="btn-ghost" disabled={invalid || saveFx.isPending}
onClick={() => saveFx.mutate()}>
{saveFx.isPending ? <Spinner /> : null} Save corrected rates
</button>
)}
{user ? (
<p className="text-sm text-subink flex items-center gap-1.5">
<UserCircle2 size={16} className="text-primary" />
Confirming as <b className="text-ink">{user.display_name}</b>
</p>
) : (
<label className="text-sm">
<span className="block text-xs font-medium text-subink mb-1">Confirmed by</span>
<input className="input" placeholder="Your name" value={who}
onChange={(e) => setWho(e.target.value)} />
</label>
)}
<button className="btn-primary"
disabled={!confirmer.trim() || dirty || confirmFx.isPending || locked}
<button className="btn-primary" disabled={!who.trim() || confirmFx.isPending}
onClick={() => confirmFx.mutate()}>
{confirmFx.isPending ? <Spinner /> : <CheckCircle2 size={15} />}
Confirm all rates for {session.reporting_month ?? "this month"}
</button>
<p className="text-xs text-subink flex-1 min-w-[220px]">
{dirty
? "Save the corrected rates first, then confirm them."
: "Correct any rate that changed, then confirm — confirming records who accepted these rates and when."}
Review the rates on the Settings tab first — confirming records who accepted them and when.
</p>
</div>
{(confirmFx.isError || saveFx.isError) && (
<p className="px-4 pb-4 text-sm text-bad">
{((confirmFx.error || saveFx.error) as Error).message}
</p>
{confirmFx.isError && (
<p className="px-4 pb-4 text-sm text-bad">{(confirmFx.error as Error).message}</p>
)}
</Section>
)}

31
ar-aging-app/frontend/src/pages/closing/JournalEntry.tsx
View File

@ -1,11 +1,10 @@
import { useEffect, useState } from "react";
import { useMutation, useQueries, useQuery, useQueryClient } from "@tanstack/react-query";
import { BadgeCheck, CheckCircle2, FileCheck2, RotateCcw, UserCircle2 } from "lucide-react";
import { BadgeCheck, CheckCircle2, FileCheck2, RotateCcw } from "lucide-react";
import { api, JournalLineT, JournalT } from "../../api/client";
import { acct, date as fmtDate } from "../../lib/format";
import { EmptyState, InfoTip, Section, Spinner, useDefinitions } from "../../components/ui";
import { ALL_MARKETS, MarketTabs, isAll, useMarket } from "../../components/market";
import { useAuth } from "../../auth";
import { useClosing } from "../Closing";
/**
@ -226,37 +225,25 @@ function AllMarketsJournal({ id, markets }: { id: number; markets: string[] }) {
/* ------------------------------------------------------- review & approval */
function SignOff({ id, j }: { id: number; j: JournalT }) {
const { user } = useAuth();
const qc = useQueryClient();
const [reviewer, setReviewer] = useState("");
const [approver, setApprover] = useState("");
// Signed in -> the verified identity signs; free-text boxes only exist without auth.
const reviewerName = user?.display_name ?? reviewer;
const approverName = user?.display_name ?? approver;
const invalidate = () => {
qc.invalidateQueries({ queryKey: ["journal", id] });
qc.invalidateQueries({ queryKey: ["journal", id, ""] });
qc.invalidateQueries({ queryKey: ["accounts-summary"] });
qc.invalidateQueries({ queryKey: ["sessions"] });
};
const review = useMutation({
mutationFn: () => api.reviewJournal(id, reviewerName.trim()),
mutationFn: () => api.reviewJournal(id, reviewer.trim()),
onSuccess: () => { setReviewer(""); invalidate(); },
});
const approve = useMutation({
mutationFn: () => api.approveJournal(id, approverName.trim()),
mutationFn: () => api.approveJournal(id, approver.trim()),
onSuccess: () => { setApprover(""); invalidate(); },
});
const reset = useMutation({ mutationFn: () => api.resetJournalSignoff(id), onSuccess: invalidate });
const reviewed = !!j.reviewed_by;
const approved = !!j.approved_by;
const Identity = () => (
<p className="text-sm text-subink flex items-center gap-1.5 flex-1">
<UserCircle2 size={16} className="text-primary" />
as <b className="text-ink">{user?.display_name}</b>
</p>
);
return (
<Section title="Review & approval"
@ -273,12 +260,10 @@ function SignOff({ id, j }: { id: number; j: JournalT }) {
<span className="text-xs text-subink ml-2">{fmtDate(j.reviewed_at)}</span>
</p>
) : (
<div className="flex gap-2 mt-2 items-center">
{user ? <Identity /> : (
<div className="flex gap-2 mt-2">
<input className="input flex-1" placeholder="Reviewer's name" value={reviewer}
onChange={(e) => setReviewer(e.target.value)} />
)}
<button className="btn-primary" disabled={!reviewerName.trim() || review.isPending}
<button className="btn-primary" disabled={!reviewer.trim() || review.isPending}
onClick={() => review.mutate()}>
{review.isPending ? <Spinner /> : <FileCheck2 size={15} />} Mark reviewed
</button>
@ -299,13 +284,11 @@ function SignOff({ id, j }: { id: number; j: JournalT }) {
<span className="badge bg-okbg text-ok ml-2"><CheckCircle2 size={12} /> published to Accounts Summary</span>
</p>
) : (
<div className="flex gap-2 mt-2 items-center">
{user ? <Identity /> : (
<div className="flex gap-2 mt-2">
<input className="input flex-1" placeholder="Approver's name" value={approver}
onChange={(e) => setApprover(e.target.value)}
disabled={!reviewed} />
)}
<button className="btn-primary" disabled={!reviewed || !approverName.trim() || approve.isPending}
<button className="btn-primary" disabled={!reviewed || !approver.trim() || approve.isPending}
onClick={() => approve.mutate()}>
{approve.isPending ? <Spinner /> : <BadgeCheck size={15} />} Approve
</button>

34
ar-aging-app/frontend/src/pages/closing/Upload.tsx
View File

@ -1,6 +1,6 @@
import { useNavigate } from "react-router-dom";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Trash2, Play, FileSpreadsheet, CopyX } from "lucide-react";
import { Trash2, Play, FileSpreadsheet } from "lucide-react";
import { api } from "../../api/client";
import { bytes, date, int } from "../../lib/format";
import { FileDrop, Section, StatusBadge, Spinner } from "../../components/ui";
@ -8,16 +8,13 @@ import HeaderMapping from "../../components/HeaderMapping";
import { useClosing } from "../Closing";
export default function Upload() {
const { id, session, locked } = useClosing();
const { id, session } = useClosing();
const qc = useQueryClient();
const nav = useNavigate();
const busy = session.status === "processing" || session.status === "exporting";
const { data: files } = useQuery({ queryKey: ["files", id], queryFn: () => api.listFiles(id) });
const invalidate = () => {
qc.invalidateQueries({ queryKey: ["files", id] });
qc.invalidateQueries({ queryKey: ["session", id] });
};
const invalidate = () => qc.invalidateQueries({ queryKey: ["files", id] });
const upload = useMutation({ mutationFn: (fs: File[]) => api.uploadFiles(id, fs), onSuccess: invalidate });
const remove = useMutation({ mutationFn: (fid: number) => api.deleteFile(id, fid), onSuccess: invalidate });
@ -26,30 +23,15 @@ export default function Upload() {
onSuccess: () => { qc.invalidateQueries({ queryKey: ["session", id] }); nav(`/closing/${id}`); },
});
const skipped = upload.data?.skipped ?? [];
const hasInvalid = files?.some((f) => f.status === "invalid");
const dates = (files ?? []).flatMap((f) => [f.min_date, f.max_date]).filter(Boolean) as string[];
const cover = dates.length ? `${dates.reduce((a, b) => (a < b ? a : b))} → ${dates.reduce((a, b) => (a > b ? a : b))}` : "—";
return (
<div className="space-y-6">
<FileDrop disabled={busy || locked || upload.isPending} onFiles={(fs) => upload.mutate(fs)} />
<FileDrop disabled={busy || upload.isPending} onFiles={(fs) => upload.mutate(fs)} />
{upload.isPending && <p className="text-sm text-subink flex items-center gap-2"><Spinner /> Uploading & validating…</p>}
{upload.isError && <p className="text-sm text-bad">{(upload.error as Error).message}</p>}
{skipped.length > 0 && (
<div className="card border-warn/30 bg-warnbg/40 p-3 text-sm space-y-1">
<p className="font-semibold text-ink flex items-center gap-2">
<CopyX size={15} className="text-warn" />
{skipped.length} file(s) skipped as duplicates — nothing was double-counted
</p>
<ul className="text-xs text-subink pl-6 list-disc">
{skipped.map((s) => (
<li key={s.filename}><b>{s.filename}</b> — {s.reason}</li>
))}
</ul>
</div>
)}
<Section title="Uploaded files" subtitle={`Detected date coverage: ${cover}`}>
{!files?.length ? (
@ -73,7 +55,7 @@ export default function Upload() {
<td className="td text-xs">{f.marketplace ?? "—"}</td>
<td className="td"><StatusBadge status={f.status} /></td>
<td className="td text-right">
<button className="p-1.5 rounded hover:bg-badbg text-subink hover:text-bad" disabled={busy || locked}
<button className="p-1.5 rounded hover:bg-badbg text-subink hover:text-bad" disabled={busy}
onClick={() => remove.mutate(f.id)}><Trash2 size={15} /></button>
</td>
</tr>
@ -84,17 +66,15 @@ export default function Upload() {
</Section>
<div className="card p-4 text-xs text-subink">
<p className="font-semibold text-ink mb-1">Column mapping & duplicates</p>
<p className="font-semibold text-ink mb-1">Column mapping</p>
Headers are auto-matched to the internal schema by normalized name (not position): the raw
<span className="num"> date/time · settlement id · type · account type · total </span> columns are
required. The pivot sheet in each file is ignored automatically. Files failing validation are flagged above.
Re-uploading a file with the same name <b>replaces</b> it; a file whose content is already
uploaded (even under another name) is skipped — a month can never count a file twice.
</div>
<div className="flex items-center justify-between">
<p className="text-sm text-subink">{files?.length ?? 0} file(s) · {hasInvalid ? <span className="text-bad">fix invalid files before processing</span> : "ready"}</p>
<button className="btn-primary" disabled={!files?.length || hasInvalid || busy || locked || run.isPending}
<button className="btn-primary" disabled={!files?.length || hasInvalid || busy || run.isPending}
onClick={() => run.mutate()}>
<Play size={16} /> {run.isPending ? "Starting…" : "Run processing"}
</button>

3
ar-aging-app/scripts/start.bat
View File

@ -1,3 +0,0 @@
@echo off
rem Double-click launcher: starts backend (8010) + frontend (5174) and opens the app.
powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0start.ps1"

18
ar-aging-app/scripts/start.ps1
View File

@ -1,18 +0,0 @@
# Launch the A/R Aging app on Windows.
# Ports 8010/5174 because the defaults (8000/5173) are used by the ATS project.
# Run with: powershell -ExecutionPolicy Bypass -File scripts\start.ps1 (or double-click start.bat)
$app = Split-Path $PSScriptRoot -Parent # scripts/ lives inside ar-aging-app/
$python = "$env:LOCALAPPDATA\anaconda3\envs\Talha\python.exe"
Start-Process powershell -ArgumentList @(
"-NoExit", "-Command",
"cd '$app\backend'; & '$python' -m uvicorn app.api.main:app --host 127.0.0.1 --port 8010 --reload"
)
Start-Process powershell -ArgumentList @(
"-NoExit", "-Command",
"cd '$app\frontend'; `$env:VITE_API_PROXY = 'http://localhost:8010'; npx vite --port 5174 --strictPort"
)
Start-Sleep -Seconds 4
Start-Process "http://localhost:5174"

162
plan.md
View File

@ -1,162 +0,0 @@
# AR Aging App — Production-Ready Plan
**Date:** 19 Aug 2026
**Goal:** Upload Jan file → Jan data is saved and shown. Upload Feb later → Feb becomes a NEW month dataset; all previous months stay saved and selectable (never merged). Hosted on AWS with backups, automatic exchange rates, and a 5-user login.
**Bottom line:**
| | |
|---|---|
| AWS hosting cost | **≈ $50–55 / month** |
| Exchange-rate API | **$0 / month** (Frankfurter, free) |
| Build effort | **~12–17 dev days**, 5 phases, each shippable on its own |
---
## 1. What is wrong today (why months "merge" or disappear)
The app already stores each month as its own closing session in the database — the foundation is right. Four defects break it:
1. **Double-count bug (verified in code):** re-uploading a file with the same name overwrites it on disk (`ar-aging-app/backend/app/api/routes/files.py:38`) but inserts a **second** database row (`files.py:74`). Processing then parses the same file twice and **doubles every number**.
2. **Nothing prevents two sessions for the same month** — creating "January" twice by accident produces two competing datasets, no warning.
3. **New months start from zero** — default `opening_mode="zero"` even though carry-forward logic already exists in `routes/ar.py`, so months lose continuity unless someone remembers to switch it.
4. **Previous months silently vanish from Accounts Summary** — re-processing clears journal approval (correct audit control), but the summary grid only shows approved months, so the old month disappears with no explanation.
## 2. Confirmed decisions
- **Exchange rates:** free **Frankfurter API** — $0, no API key, central-bank (ECB) rates, historical month-end dates. The human confirmation gate (Control C5) stays.
- **Login:** simple per-user login, **5 users**. Approver/reviewer names come from the logged-in user.
- **File sizes:** uploads are **300–500 MB** each → server sized with **8 GB RAM** for Excel parsing.
---
## 3. AWS architecture & monthly cost
### Recommended: one production server + S3 backups (≈ $50–55/mo)
**Lightsail 8 GB instance** ($44/mo: 2 vCPU, 8 GB RAM, 160 GB SSD, static IP included) — or equivalent EC2 `t4g.large` (≈ $61/mo with EBS + IPv4). Runs everything via a new `docker-compose.prod.yml`:
| Container | Role |
|---|---|
| **nginx** | Serves built React app, proxies `/api` to backend, HTTPS (Let's Encrypt), `client_max_body_size 2g` + long timeouts for big uploads |
| **backend** | FastAPI/uvicorn, **single worker** (background jobs are in-process — documented constraint, fine for 5 users) |
| **MySQL 8** | Data on instance disk. Code already supports MySQL (`AR_DB_BACKEND=mysql`) and ships `migrate_sqlite_to_mysql.py` |
| **backup cron** | Nightly `mysqldump` + `aws s3 sync` of uploads/exports → versioned S3 bucket (lifecycle → cold storage after 90 days), via IAM role (no keys on disk) |
### Cost breakdown
| Item | Monthly cost |
|---|---|
| Lightsail 8 GB (EC2 t4g.large route ≈ $61) | $44 |
| S3 backups (~60 GB year one, versioned) | $1.50–3 |
| Weekly instance snapshots | $2–4 |
| Route 53 hosted zone (optional domain) | $0.50 |
| Frankfurter FX API | $0 |
| **Total** | **≈ $50–55/mo** |
**Alternative (managed DB):** same server + **RDS MySQL db.t4g.small** ≈ **$85–100/mo** — buys automated patching + point-in-time restore. Not needed at this scale; upgrading later is a one-line config change (`MYSQL_HOST`).
**Storage decisions:**
- Uploads/exports stay on the **instance disk** — the parsing pipeline needs local file paths; round-tripping 500 MB files through S3 adds complexity for no benefit. S3 = durable **backup**, not primary storage.
- Data growth ~1 GB/month (~3.4M transaction rows) — well within sizing.
- Firewall inbound restricted to office IPs/VPN as defense-in-depth on top of login.
### Exchange-rate API pricing (researched)
| Provider | Free tier | Paid | Verdict |
|---|---|---|---|
| **Frankfurter** (chosen) | Unlimited, **no API key**, ECB rates, historical dates, self-hostable | — | ✅ $0, ideal for month-end closes |
| ExchangeRate-API | 1,500 req/mo, daily updates | Pro $10/mo (30k req, hourly) | fallback provider stub |
| Open Exchange Rates | 1,000 req/mo, USD base only | $12/mo | not needed |
| Fixer | 100 req/mo, no HTTPS on free | higher | ruled out |
App usage: ~13 marketplaces × a few fetches/month → even free tiers would never be exceeded.
---
## 4. Implementation phases
### Phase 1 — Month management & data separation (~3–4 days) ← HIGHEST PRIORITY
**1.1 Fix double-count bug** — `backend/app/api/routes/files.py`:
- Stream uploads to a temp `.part` name; move into place after hashing (also stops a failed upload corrupting an existing good file).
- **Content dedup:** SHA-256 match with an existing file in the session → skip, report "identical content already uploaded as X".
- **Same filename → UPDATE the existing row** instead of inserting a second one; if month already processed, flag `needs_reprocess` (existing banner handles messaging).
- Batch-friendly response `{files, skipped}` so one duplicate doesn't fail a 13-file upload; UI shows skipped/replaced notices.
- One-time `cli.py dedupe-files` cleanup for existing bad rows, then a unique index on (session_id, filename).
**1.2 One session per month (guide, don't hard-block)** — `routes/sessions.py`: 409 on duplicate `reporting_month` unless `allow_duplicate: true`; `NewClosing.tsx` shows "A closing for 2026-02 already exists — Open it | Create anyway".
**1.3 Default carry-forward** — `NewClosing.tsx`: default `openingMode="carry_forward"` when prior processed months exist (backend logic already exists); default month = month after latest close.
**1.4 Month selector UX ("show previous options"):**
- Sessions list ordered as a month timeline with a `journal_approved` flag per row.
- New `MonthSwitcher` dropdown in the closing header (month · name · status) — jump between months from any screen.
- Dashboard shows "published / processed-but-unpublished" and flags duplicate months.
**1.5 Stop previous months vanishing from Accounts Summary:**
- `accounts_summary.py` also returns pending months (processed/blocked but unapproved, incl. "approval cleared by re-processing").
- Grid renders them as greyed columns linking to the journal tab ("2026-01 processed but unpublished — re-approve the journal"). Audit control untouched.
**1.6 Read-only lock after completion** — new `ensure_editable()` guard on all mutating endpoints (409 on completed months); new `POST /sessions/{id}/reopen`; frontend disables edit controls on locked months.
**Tests:** filename replace, sha256 skip, duplicate-month 409, completed-session 409s.
### Phase 2 — Login, 5 users (~2 days)
- New `User` table (username, display name, bcrypt password hash); users created via CLI (`add-user` / `set-password`) — no self-signup.
- `POST /api/auth/login` → signed bearer token (12 h expiry, new `AR_SECRET_KEY` env); every route requires login except `/api/health` + login.
- **Real identity in sign-offs:** journal review/approve, FX confirm, control verify, payout entry all record the logged-in user — free-text "your name" boxes removed.
- Frontend: `Login.tsx`, token auto-attached, 401 → redirect to login, user + logout in sidebar.
- New deps: `passlib[bcrypt]`, `itsdangerous`.
### Phase 3 — AWS deployment (~3–5 days incl. migration dry-run)
1. Production backend image (no `--reload`, `--proxy-headers`, single worker documented).
2. Production frontend image: multi-stage `node:20` build → `nginx` serving static + API proxy.
3. New `docker-compose.prod.yml` + `example.env.production` (`AR_DB_BACKEND=mysql`, `AR_CORS_ORIGINS=https://<domain>`, `AR_SECRET_KEY`, FX vars); HTTPS via certbot; DNS → static IP.
4. **Data migration:** MySQL up → schema auto-created → run existing `migrate_sqlite_to_mysql.py` → verify per-table row counts + to-the-cent reconciliation spot check → cut over; archive SQLite file to S3. **Timed dry run first** (3.4M rows/month).
5. Backups: nightly dump + S3 sync, weekly snapshots, S3 versioning + lifecycle.
6. Deploy runbook: `git pull && docker compose -f docker-compose.prod.yml up -d --build`.
### Phase 4 — Exchange-rate service (~2 days)
- New `backend/app/services/fx_service.py`: provider abstraction — **Frankfurter** default (`api.frankfurter.dev/v1/{date}?base=USD&symbols=CAD,GBP,AUD,EUR,PLN,SEK,TRY`), paid-provider stub via `FX_PROVIDER` env.
- Rates **inverted to USD-per-local** (the app's convention) — documented + locked with a test; one inversion mistake would mis-state every non-USD receivable.
- Month-end fetch seeds `fx_rates` per session **unconfirmed** → Control C5 still blocks the close until a human confirms — workflow unchanged, just pre-filled. Daily fetch fills `fx_rates_daily`.
- Cache table makes re-fetch idempotent + works offline after first fetch; provider failure → clear 502 "enter rates manually", never a silent default.
- UI: "Fetch month-end rates" button in the C5 panel (`Controls.tsx`); "Fetch daily rates" in `ArLedger.tsx`. Update "no third-party egress" copy (only currency codes are sent, never financial data).
- ⚠️ Verify Frankfurter publishes TRY; if not, Turkey stays manual (C5 still enforces) or paid stub takes over.
### Phase 5 — Hardening (~1–2 days)
1. **Stale-job recovery:** months stuck "processing" after a restart → marked error with a re-run message (otherwise a deploy mid-job blocks the month forever).
2. **Retention:** auto-purge old generated **exports only** (`AR_RETENTION_DAYS`, currently dead config) — uploaded source files never auto-deleted (audit source).
3. Health check: DB ping + disk-writable.
4. Logging: request timing + job start/finish/fail, shipped via Docker logs.
Deferred nice-to-haves: CI pipeline, Sentry, login rate-limiting, Alembic, S3-primary storage, audit-log table, MFA.
---
## 5. Verification (acceptance checks)
- Upload same CSV twice → one file row, totals unchanged; renamed-identical file → skipped with message.
- Close Jan → create Feb → warns nothing, carries forward Jan balances; Jan stays selectable, read-only, unchanged; both months on Accounts Summary.
- Fetch month-end rates → unconfirmed → C5 fails → Confirm all → C5 passes; EUR rate checked against ECB published figure.
- Unauthenticated API call → 401; all 5 users can log in; approvals record the real approver name.
- Prod compose rehearsed locally with a 500 MB file (memory watched) **before buying the instance**; restart mid-processing recovers; nightly backup object appears in S3; restore drill succeeds.
- All existing tests pass, incl. Jan-2026 reconciliation integration test **to the cent** after MySQL migration.
## 6. Key risks
| Risk | Mitigation |
|---|---|
| SQLite→MySQL migration (~3.4M rows/month) | Timed dry run, per-table counts, cent-level reconciliation before cutover |
| FX rate orientation (USD-per-local vs local-per-USD) | Fixture test against a known EUR rate |
| Read-only lock missing an endpoint | Sweep all mutating routes during implementation |
| 8 GB RAM sizing assumption | Validate with the real 283 MB test workbook in local rehearsal before instance purchase |
---
*Pricing sources: [ExchangeRate-API](https://www.exchangerate-api.com/#pricing) · [Open Exchange Rates](https://openexchangerates.org/signup) · [Frankfurter](https://frankfurter.dev/) · AWS Lightsail/EC2/RDS public pricing, Aug 2026. Costs are USD estimates, on-demand.*

199
ar-aging-app/scripts/start.command → start.command
View File

@ -15,10 +15,9 @@ set -u -o pipefail
HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
cd -- "$HERE" || exit 1
APP_DIR="$(dirname -- "$HERE")" # scripts/ lives inside ar-aging-app/
APP_DIR="$HERE/ar-aging-app"
BACKEND_DIR="$APP_DIR/backend"
FRONTEND_DIR="$APP_DIR/frontend"
VENV_DIR="$APP_DIR/.venv"
LOG_DIR="$APP_DIR/backend/data/logs"
BACKEND_LOG="$LOG_DIR/backend.log"
FRONTEND_LOG="$LOG_DIR/frontend.log"
@ -32,23 +31,6 @@ DASHBOARD_URL="http://localhost:$FRONTEND_PORT"
# ~/.local/bin or Homebrew would otherwise be "command not found".
export PATH="$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:$PATH"
# Auto-answer prompts when stdin is not a Terminal (CI / Cursor / piped runs).
INTERACTIVE=0
[ -t 0 ] && INTERACTIVE=1
ask_yes() {
local prompt="$1"
if [ "$INTERACTIVE" -eq 0 ]; then
printf ' %s Y (non-interactive)\n' "$prompt"
return 0
fi
printf ' %s [Y/n] ' "$prompt"
read -r reply
case "${reply:-Y}" in
[Nn]*) return 1 ;;
*) return 0 ;;
esac
}
# --- pretty output ------------------------------------------------------------------
if [ -t 1 ]; then
B=$'\033[1m'; DIM=$'\033[2m'; R=$'\033[0m'
@ -66,10 +48,8 @@ die() {
printf '\n%s%sCould not start the dashboard.%s\n\n' "$ERR" "$B" "$R"
printf ' %s\n\n' "$1"
[ $# -gt 1 ] && printf ' Try: %s%s%s\n\n' "$B" "$2" "$R"
if [ "$INTERACTIVE" -eq 1 ]; then
printf '%sPress Return to close this window.%s\n' "$DIM" "$R"
read -r _
fi
exit 1
}
@ -81,133 +61,40 @@ say ""
# --- 1. prerequisites ---------------------------------------------------------------
say "${B}1. Checking prerequisites${R}"
[ -d "$BACKEND_DIR" ] || die "Backend folder not found at: $BACKEND_DIR" \
"keep start.command inside ar-aging-app/scripts/"
"keep start.command in the same folder as ar-aging-app/"
command -v python3 >/dev/null 2>&1 || die "python3 was not found." \
"install Python 3.11+ from python.org"
command -v npm >/dev/null 2>&1 || die "npm was not found." \
"install Node.js 20+ from nodejs.org"
good "python $(python3 -V 2>&1 | awk '{print $2}') · node $(node -v 2>/dev/null) · npm $(npm -v 2>/dev/null)"
# Isolated venv — required. Global site-packages (e.g. streamlit's Starlette 1.x) break
# FastAPI 0.115's Router(on_startup=...) and make "import fastapi" look fine while the app dies.
ensure_venv() {
if [ ! -x "$VENV_DIR/bin/python" ]; then
step "creating project virtualenv at $VENV_DIR…"
python3 -m venv "$VENV_DIR" \
|| die "Could not create the virtualenv." "python3 -m venv '$VENV_DIR'"
good "virtualenv created"
else
good "virtualenv present"
fi
PYTHON="$VENV_DIR/bin/python"
PIP="$VENV_DIR/bin/pip"
}
install_python_deps() {
step "installing Python packages into the project venv…"
"$PIP" install -q --upgrade pip \
|| die "pip upgrade failed." "'$PIP' install --upgrade pip"
"$PIP" install -q -r "$BACKEND_DIR/requirements.txt" \
# Python packages — actually IMPORT the app rather than checking a few package names.
# "Installed" is not the same as "compatible": an unpinned starlette upgrade once satisfied
# every import check while breaking the app at load time. Importing proves it will boot.
import_error=""
if ! import_error="$(cd -- "$BACKEND_DIR" && python3 -c "from app.api.main import app" 2>&1)"; then
warn "the backend could not be loaded:"
printf '%s\n' "$import_error" | tail -n 3 | sed 's/^/ /'
printf ' Install/repair Python packages now? [Y/n] '
read -r reply
case "${reply:-Y}" in
[Nn]*) die "The backend cannot start with the current Python packages." \
"python3 -m pip install -r '$BACKEND_DIR/requirements.txt'" ;;
esac
step "installing Python packages (this can take a minute)…"
python3 -m pip install -q -r "$BACKEND_DIR/requirements.txt" \
|| die "pip install failed — see the messages above." \
"'$PIP' install -r '$BACKEND_DIR/requirements.txt'"
}
backend_imports_ok() {
"$PYTHON" -c "import fastapi, uvicorn, sqlalchemy, openpyxl, pymysql, dotenv" >/dev/null 2>&1
}
# Prove the pin that Streamlit commonly breaks: Starlette must stay <0.42 for this FastAPI.
backend_versions_ok() {
"$PYTHON" -c "
import fastapi, starlette
from packaging.version import Version
assert Version(fastapi.__version__) >= Version('0.115.0')
assert Version(starlette.__version__) < Version('0.42.0'), starlette.__version__
" >/dev/null 2>&1
}
backend_app_loads() {
( cd -- "$BACKEND_DIR" && "$PYTHON" -c "from app.api.main import app" ) >/dev/null 2>&1
}
ensure_venv
need_install=0
if ! backend_imports_ok; then
warn "Python packages are missing or incomplete in the project venv."
need_install=1
elif ! backend_versions_ok; then
warn "Wrong Starlette/FastAPI versions in the venv (often after a global pip upgrade)."
need_install=1
elif ! backend_app_loads; then
warn "Backend failed to import — reinstalling pinned dependencies."
need_install=1
"python3 -m pip install -r '$BACKEND_DIR/requirements.txt'"
if ! import_error="$(cd -- "$BACKEND_DIR" && python3 -c "from app.api.main import app" 2>&1)"; then
printf '%s\n' "$import_error" | tail -n 5 | sed 's/^/ /'
die "The backend still cannot be loaded after installing packages." \
"python3 -m pip check"
fi
if [ "$need_install" -eq 1 ]; then
ask_yes "Install / repair Python packages now?" \
|| die "Python dependencies are not installed." \
"'$PIP' install -r '$BACKEND_DIR/requirements.txt'"
install_python_deps
# packaging is used only for the version check; requirements may not list it.
"$PIP" install -q packaging >/dev/null 2>&1 || true
backend_imports_ok \
|| die "Python packages are still incomplete after installing." "'$PIP' check"
backend_versions_ok \
|| die "Starlette is still too new for this FastAPI pin." \
"'$PIP' install 'starlette==0.41.3'"
good "Python packages installed"
good "Python packages repaired"
else
good "Python packages present (pinned versions)"
good "Python packages present and compatible"
fi
# 1b. Which database? SQLite needs nothing; MySQL needs a server and credentials. The app
# picks MySQL only when ar-aging-app/.env names a real host, so a machine with no
# database installed still runs off the local file instead of refusing to start.
ENV_FILE="$APP_DIR/.env"
db_backend="$(cd -- "$BACKEND_DIR" && "$PYTHON" -c "
from app.config import DB_BACKEND; print(DB_BACKEND)" 2>/dev/null)"
db_label="$(cd -- "$BACKEND_DIR" && "$PYTHON" -c "
from app.config import database_label; print(database_label())" 2>/dev/null)"
if [ "$db_backend" = "mysql" ]; then
good "database: $db_label"
db_err="$(cd -- "$BACKEND_DIR" && "$PYTHON" -c "
import sys, pymysql
from app.config import MYSQL_HOST, MYSQL_PORT, MYSQL_USER, MYSQL_PASSWORD
try:
pymysql.connect(host=MYSQL_HOST, port=MYSQL_PORT, user=MYSQL_USER,
password=MYSQL_PASSWORD, connect_timeout=6).close()
except Exception as e:
sys.stderr.write(str(e)); sys.exit(1)
" 2>&1)" || {
fail "Cannot reach the MySQL server named in $ENV_FILE"
say ""
printf ' %s\n' "$(printf '%s' "$db_err" | tail -n 2)"
say ""
say " Start the server and run this again, or fall back to the local file by setting"
say " ${B}AR_DB_BACKEND=sqlite${R} in $ENV_FILE."
say ""
if [ "$INTERACTIVE" -eq 1 ]; then
printf '%sPress Return to close this window.%s\n' "$DIM" "$R"
read -r _
fi
exit 1
}
good "MySQL reachable"
else
good "database: ${db_label:-SQLite (local file)}"
[ -f "$ENV_FILE" ] || warn "no .env — using the local file (fine for a demo or one user)"
fi
# 1d. Everything above is fine — now prove the app itself loads.
if ! import_error="$(cd -- "$BACKEND_DIR" && "$PYTHON" -c "from app.api.main import app" 2>&1)"; then
fail "The backend failed to load even though its packages and database are fine:"
printf '%s\n' "$import_error" | tail -n 8 | sed 's/^/ /'
die "This looks like a code or dependency-version problem." "'$PIP' check"
fi
good "backend loads cleanly"
# Frontend packages — safe to install unattended, they're local to the project.
if [ ! -d "$FRONTEND_DIR/node_modules" ]; then
step "installing frontend packages (first run only, ~1 minute)…"
@ -224,7 +111,7 @@ fi
# The Vite proxy points at a fixed localhost:8000, so we can't just pick another port.
free_port() {
local port="$1" label="$2" pids
pids="$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null || true)"
pids="$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null)"
[ -z "$pids" ] && { good "port $port free ($label)"; return 0; }
warn "port $port is already in use ($label):"
@ -232,20 +119,24 @@ free_port() {
for pid in $pids; do
printf ' pid %-7s %s\n' "$pid" "$(ps -p "$pid" -o command= 2>/dev/null | cut -c1-88)"
done
ask_yes "Stop it and continue?" \
|| die "Port $port is in use, so the dashboard cannot start." \
"quit the other program, or close the old dashboard window"
for pid in $pids; do kill "$pid" 2>/dev/null || true; done
printf ' Stop it and continue? [Y/n] '
read -r reply
case "${reply:-Y}" in
[Nn]*) die "Port $port is in use, so the dashboard cannot start." \
"quit the other program, or close the old dashboard window" ;;
esac
for pid in $pids; do kill "$pid" 2>/dev/null; done
for _ in 1 2 3 4 5 6 7 8 9 10; do
sleep 0.3
[ -z "$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null || true)" ] && break
[ -z "$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null)" ] && break
done
pids="$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null || true)"
# Still holding on? Escalate once.
pids="$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null)"
if [ -n "$pids" ]; then
for pid in $pids; do kill -9 "$pid" 2>/dev/null || true; done
for pid in $pids; do kill -9 "$pid" 2>/dev/null; done
sleep 1
fi
[ -n "$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null || true)" ] \
[ -n "$(lsof -ti tcp:"$port" -sTCP:LISTEN 2>/dev/null)" ] \
&& die "Port $port is still in use after trying to stop it." \
"restart your Mac, or find the process with: lsof -i :$port"
good "port $port freed"
@ -264,11 +155,11 @@ FRONTEND_PID=""
shutdown() {
printf '\n%sStopping…%s\n' "$DIM" "$R"
# Kill the whole process group of each server: uvicorn --reload and vite both fork.
[ -n "$FRONTEND_PID" ] && kill -- "-$FRONTEND_PID" 2>/dev/null || true
[ -n "$BACKEND_PID" ] && kill -- "-$BACKEND_PID" 2>/dev/null || true
[ -n "$FRONTEND_PID" ] && kill -- "-$FRONTEND_PID" 2>/dev/null
[ -n "$BACKEND_PID" ] && kill -- "-$BACKEND_PID" 2>/dev/null
sleep 0.5
[ -n "$FRONTEND_PID" ] && kill -9 -- "-$FRONTEND_PID" 2>/dev/null || true
[ -n "$BACKEND_PID" ] && kill -9 -- "-$BACKEND_PID" 2>/dev/null || true
[ -n "$FRONTEND_PID" ] && kill -9 -- "-$FRONTEND_PID" 2>/dev/null
[ -n "$BACKEND_PID" ] && kill -9 -- "-$BACKEND_PID" 2>/dev/null
printf '%sBoth servers stopped.%s\n\n' "$OK" "$R"
exit 0
}
@ -278,15 +169,16 @@ say ""
say "${B}3. Starting servers${R}"
step "backend (FastAPI on :$BACKEND_PORT)"
# Own process group so shutdown() can take down the reloader children too.
# setsid-style: run in its own process group so shutdown() can take down the reloader too.
set -m
"$PYTHON" -m uvicorn app.api.main:app \
python3 -m uvicorn app.api.main:app \
--app-dir "$BACKEND_DIR" \
--host 127.0.0.1 --port "$BACKEND_PORT" \
>"$BACKEND_LOG" 2>&1 &
BACKEND_PID=$!
set +m
# Wait for it to actually answer — "process started" is not the same as "server ready".
backend_ready=""
for _ in $(seq 1 60); do
if ! kill -0 "$BACKEND_PID" 2>/dev/null; then
@ -325,6 +217,7 @@ done
die "The frontend did not respond within 30 seconds." "full log: $FRONTEND_LOG"; }
good "frontend ready"
# End-to-end check: the browser reaches the API *through* the Vite proxy, not directly.
if curl -fsS -o /dev/null "$DASHBOARD_URL/api/sessions" 2>/dev/null; then
good "dashboard is talking to the API"
else
@ -343,12 +236,12 @@ say ""
say " Dashboard ${B}$DASHBOARD_URL${R}"
say " API docs ${DIM}http://localhost:$BACKEND_PORT/docs${R}"
say " Logs ${DIM}$LOG_DIR${R}"
say " Python ${DIM}$PYTHON${R}"
say ""
say "${DIM} Keep this window open while you work.${R}"
say "${DIM} Press Ctrl-C to stop both servers.${R}"
say ""
# Stay alive until a server dies or the user interrupts.
while kill -0 "$BACKEND_PID" 2>/dev/null && kill -0 "$FRONTEND_PID" 2>/dev/null; do
sleep 1
done