Production readiness: month separation, auth, FX service, AWS deployment
Month management & data separation: - Fix double-count bug: re-uploading a filename updates the existing session_files row in place; identical content (sha256) is skipped — a closing can never parse the same file twice - Guard against duplicate closings per reporting month (409 unless explicitly overridden); dashboard flags duplicates - Default new closings to carry-forward openings; month switcher in the closing header; publish state visible everywhere; Accounts Summary lists unpublished months with the reason instead of dropping them - Completed closings are locked read-only with an explicit reopen Authentication (stdlib only, no new deps): - Per-user login (scrypt + HMAC tokens), AR_AUTH=auto turns on with the first user; manage.py add-user/set-password/deactivate-user - Verified identity feeds reviewed_by/approved_by/confirmed_by Exchange rates: - fx_service with provider abstraction: Frankfurter (free, keyless, ECB) default, exchangerate-api stub; month-end + daily fetch endpoints and UI buttons; rates arrive unconfirmed so Control C5 still gates the close; cache table; certifi CA bundle Deployment & hardening: - Production Docker stack: caddy (auto-HTTPS) + nginx + single-worker backend + mysql:8.4; per-context .dockerignore (images carry no financial data); .env.example with local+production sections - deploy/DEPLOY.md runbook + nightly S3 backup script - Stale-job recovery on startup; export retention (AR_RETENTION_DAYS); deep /api/health; request/job logging; Gitea Actions CI - Repo reorganized: launchers in scripts/, dated lowercase docs, root README, .gitattributes for deterministic line endings Tests: 152 passed (25+ new: dedup, month locking, auth, FX orientation) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>main
parent
c322437599
commit
d823a45cb2
|
|
@ -0,0 +1,28 @@
|
|||
# Deterministic line endings regardless of each developer's core.autocrlf.
|
||||
# Shell scripts and Docker/compose files MUST be LF — CRLF breaks bash and can break
|
||||
# Docker builds when the repo is cloned on the Linux server.
|
||||
* text=auto
|
||||
|
||||
*.sh text eol=lf
|
||||
*.command text eol=lf
|
||||
Makefile text eol=lf
|
||||
Dockerfile text eol=lf
|
||||
*.yml text eol=lf
|
||||
*.yaml text eol=lf
|
||||
*.py text eol=lf
|
||||
*.ts text eol=lf
|
||||
*.tsx text eol=lf
|
||||
*.md text eol=lf
|
||||
*.json text eol=lf
|
||||
*.html text eol=lf
|
||||
*.css text eol=lf
|
||||
*.conf text eol=lf
|
||||
|
||||
# Windows launchers keep CRLF
|
||||
*.bat text eol=crlf
|
||||
*.ps1 text eol=crlf
|
||||
|
||||
# Never mangle binaries
|
||||
*.png binary
|
||||
*.ico binary
|
||||
*.woff2 binary
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
# CI for Gitea Actions (git.utopiadeals.com). GitHub-compatible syntax — if the repo ever
|
||||
# moves to GitHub, copy this file to .github/workflows/ci.yml unchanged.
|
||||
#
|
||||
# Requires a Gitea Actions runner to be registered on the server
|
||||
# (Site Administration -> Actions -> Runners; docker label recommended).
|
||||
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
backend-tests:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ar-aging-app/backend
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: ar-aging-app/backend/requirements.txt
|
||||
- name: Install dependencies
|
||||
run: pip install -r requirements.txt
|
||||
- name: Run tests (isolated temp DB — never touches real data)
|
||||
run: python -m pytest -q
|
||||
|
||||
frontend-build:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ar-aging-app/frontend
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
- name: Install dependencies
|
||||
run: npm ci || npm install
|
||||
- name: Type-check + production build
|
||||
run: npm run build
|
||||
|
||||
docker-images:
|
||||
# Prove both production images actually build — the exact images the server will run.
|
||||
runs-on: ubuntu-latest
|
||||
needs: [backend-tests, frontend-build]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Build backend image
|
||||
run: docker build ar-aging-app/backend -t ar-backend:ci
|
||||
- name: Build frontend (nginx) image
|
||||
run: docker build ar-aging-app/frontend --target prod -t ar-web:ci
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
# Finance-Accounts
|
||||
|
||||
Finance team tooling for **Utopia Brands**. The main (currently only) application is the
|
||||
**Amazon Accounts Receivable Aging Dashboard** — it turns the month's Amazon *Custom
|
||||
Unified Transaction* exports into the month-end AR workbook, with per-user login,
|
||||
month-end controls, exchange-rate fetching, and a full audit trail.
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
ar-aging-app/ The application (FastAPI backend · React frontend)
|
||||
│
|
||||
├── backend/ Python API + calculation engine + tests
|
||||
├── frontend/ React + TypeScript dashboard
|
||||
├── scripts/ Launchers: start.ps1 / start.bat (Windows) · start.command (macOS)
|
||||
├── deploy/ Production runbook (DEPLOY.md) + backup script
|
||||
├── docs/ System guide, AR logic, audit reports
|
||||
├── docker-compose.yml local Docker stack (dev)
|
||||
├── docker-compose.prod.yml production stack (AWS: HTTPS + MySQL + backups)
|
||||
└── .env.example every setting, local + production sections
|
||||
|
||||
plan.md Production-readiness plan (architecture, AWS costs, phases)
|
||||
sample data (local) "Test Files/" — real Amazon exports; gitignored, never committed
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
| I want to… | Do this |
|
||||
|---|---|
|
||||
| Run the app on this PC | double-click `ar-aging-app/scripts/start.bat` → http://localhost:5174 |
|
||||
| Understand the app | [ar-aging-app/README.md](ar-aging-app/README.md) |
|
||||
| Deploy to AWS | [ar-aging-app/deploy/DEPLOY.md](ar-aging-app/deploy/DEPLOY.md) (~$50–55/month) |
|
||||
| See how figures are calculated | [ar-aging-app/docs/system-guide.md](ar-aging-app/docs/system-guide.md) |
|
||||
| Read the production plan | [plan.md](plan.md) |
|
||||
| Manage users / fix data | `python ar-aging-app/backend/manage.py --help` |
|
||||
|
||||
Financial data never enters git: spreadsheets, databases, uploads, and `.env*` secrets are
|
||||
all ignored (see `.gitignore`). The only template committed is `ar-aging-app/.env.example`.
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
# ==============================================================================
|
||||
# AR Aging — environment template (this file IS committed; real copies are NOT)
|
||||
#
|
||||
# Local development: cp .env.example .env -> fill the LOCAL section
|
||||
# Production (AWS): cp .env.example .env.production -> fill the PRODUCTION section
|
||||
#
|
||||
# .env and .env.production are gitignored — secrets never enter git.
|
||||
# ==============================================================================
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ LOCAL (dev)
|
||||
# Database: leave MYSQL_* unset and the app uses a local SQLite file —
|
||||
# backend/data/ar_aging.db (zero setup; this is the local database "name")
|
||||
# Force it explicitly if you like:
|
||||
AR_DB_BACKEND=sqlite
|
||||
|
||||
# Signs login tokens (sessions survive backend restarts). Generate:
|
||||
# python -c "import secrets; print(secrets.token_hex(32))"
|
||||
AR_SECRET_KEY=
|
||||
|
||||
# Login: auto = required as soon as users exist (create with: python manage.py add-user)
|
||||
AR_AUTH=auto
|
||||
|
||||
# Vite dev server origins
|
||||
AR_CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173,http://localhost:5174,http://127.0.0.1:5174
|
||||
|
||||
# Exchange rates: frankfurter = free, keyless, central-bank rates
|
||||
AR_FX_PROVIDER=frankfurter
|
||||
|
||||
# Generated exports older than this are purged (uploads are NEVER auto-deleted). 0 = keep.
|
||||
AR_RETENTION_DAYS=90
|
||||
|
||||
|
||||
# ------------------------------------------------------------- PRODUCTION (AWS)
|
||||
# Used by docker-compose.prod.yml. Fill these in .env.production on the server.
|
||||
|
||||
# Domain — DNS A record must point at the server; HTTPS certificate is automatic.
|
||||
#AR_DOMAIN=ar.utopiabrands.com
|
||||
|
||||
# MySQL (the database is created automatically on first start).
|
||||
# MYSQL_HOST is set to the compose service name by docker-compose.prod.yml.
|
||||
#MYSQL_PORT=3306
|
||||
#MYSQL_DATABASE=account_finance
|
||||
#MYSQL_USER=ar_app
|
||||
#MYSQL_PASSWORD= <- strong generated password
|
||||
#MYSQL_ROOT_PASSWORD= <- different strong generated password
|
||||
#MYSQL_SLOW_QUERY_MS=500
|
||||
#MYSQL_POOL_SIZE=10
|
||||
#MYSQL_POOL_RECYCLE=3600
|
||||
|
||||
# Auth — REQUIRED in production. Different key than local!
|
||||
#AR_SECRET_KEY= <- openssl rand -hex 32
|
||||
#AR_AUTH=on
|
||||
#AR_AUTH_TOKEN_HOURS=12
|
||||
|
||||
# Same-origin behind nginx/caddy; still set exactly.
|
||||
#AR_CORS_ORIGINS=https://ar.utopiabrands.com
|
||||
|
||||
#AR_FX_PROVIDER=frankfurter
|
||||
# AR_FX_PROVIDER=exchangerate-api # paid fallback ($10/mo) — then set:
|
||||
# AR_FX_API_KEY=
|
||||
|
||||
#AR_RETENTION_DAYS=90
|
||||
# AR_MAX_UPLOAD_BYTES=2147483648 # 2 GB default
|
||||
|
||||
# Nightly backups (deploy/backup.sh) — S3 bucket; instance IAM role grants access.
|
||||
#AR_BACKUP_S3_BUCKET=s3://utopia-ar-backups
|
||||
|
|
@ -13,17 +13,51 @@ See [docs/accounts-receivable-logic.md](docs/accounts-receivable-logic.md).
|
|||
```
|
||||
backend/
|
||||
app/core/ # streaming parser + receivable engine (stdlib + openpyxl only)
|
||||
app/api/ # FastAPI app (uploads, jobs, endpoints) — Phase 3
|
||||
tests/ # unit + Jan-2026 reconciliation integration test
|
||||
app/api/ # FastAPI app: routes, auth (login), deps
|
||||
app/services/ # jobs, persistence, FX-rate fetch, controls, retention
|
||||
app/db/ # SQLAlchemy models (20 tables) + engine (SQLite / MySQL)
|
||||
tests/ # 150+ tests: engine, API, auth, FX, dedup, Jan-2026 integration
|
||||
cli.py # process files from the command line
|
||||
frontend/ # React + TS + Vite dashboard — Phase 4
|
||||
docs/
|
||||
manage.py # admin: add-user / set-password / dedupe-files / …
|
||||
migrate_sqlite_to_mysql.py # one-time data migration for the AWS cutover
|
||||
frontend/ # React + TS + Vite dashboard (nginx-served in production)
|
||||
scripts/ # launchers: start.ps1 / start.bat (Windows) · start.command (macOS)
|
||||
deploy/ # DEPLOY.md runbook + backup.sh (nightly mysqldump + S3 sync)
|
||||
docs/ # system guide · AR logic · audit reports
|
||||
docker-compose.yml # dev stack docker-compose.prod.yml # production
|
||||
.env.example # every setting, LOCAL + PRODUCTION sections
|
||||
```
|
||||
|
||||
## Run the app
|
||||
## Production deployment (AWS)
|
||||
|
||||
One 8 GB server runs the whole stack with automatic HTTPS, MySQL, per-user login, and
|
||||
nightly S3 backups — see **[deploy/DEPLOY.md](deploy/DEPLOY.md)** for the full runbook
|
||||
(provisioning, user creation, SQLite→MySQL migration, backups, updates). ≈ $50–55/month.
|
||||
|
||||
```bash
|
||||
# 1. Configure MySQL (hosted RDS) + data paths
|
||||
cp example.env .env # then fill in MYSQL_* credentials
|
||||
cp .env.example .env.production # fill the PRODUCTION section (domain, passwords, AR_SECRET_KEY)
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
|
||||
python manage.py add-user <user> --name "Full Name"
|
||||
```
|
||||
|
||||
Key production behaviors:
|
||||
- **One closing per month** — every month stays saved and selectable (month switcher in the
|
||||
closing header); creating a second closing for an existing month requires an explicit override.
|
||||
- **Duplicate-proof uploads** — re-uploading a filename *replaces* it; identical content
|
||||
under another name is skipped. A month can never count a file twice.
|
||||
- **Login** (`AR_AUTH`) — per-user accounts via `manage.py add-user`; journal review/approval
|
||||
and FX confirmations record the signed-in user's verified name.
|
||||
- **Exchange rates** — "Fetch month-end rates" pulls central-bank rates (Frankfurter, free,
|
||||
keyless; `AR_FX_PROVIDER`); fetched rates still require human confirmation (Control C5).
|
||||
- **Completed closings are locked** read-only; corrections need an explicit Reopen.
|
||||
- **Crash-safe jobs** — a restart mid-processing marks the closing as interrupted instead of
|
||||
leaving it stuck; generated exports are purged after `AR_RETENTION_DAYS` (uploads never are).
|
||||
|
||||
## Run the app (development)
|
||||
```bash
|
||||
# 1. Configure the environment (SQLite by default — no database setup needed)
|
||||
cp .env.example .env # then fill the LOCAL section (AR_SECRET_KEY at minimum)
|
||||
|
||||
# Option A — Docker (backend + Vite hot reload)
|
||||
docker compose up --build
|
||||
|
|
@ -33,6 +67,11 @@ docker compose up --build
|
|||
make install # backend deps + npm install
|
||||
make backend # terminal 1 → FastAPI on :8000
|
||||
make frontend # terminal 2 → dashboard on http://localhost:5173
|
||||
|
||||
# Option C — Windows one-click (ports 8010/5174)
|
||||
scripts\start.bat # or: powershell -ExecutionPolicy Bypass -File scripts\start.ps1
|
||||
# macOS one-click:
|
||||
scripts/start.command
|
||||
```
|
||||
Then open http://localhost:5173 → **New Closing** → pick the month → drag in the three Amazon
|
||||
files → **Run processing** → review → **Download Full A/R Aging Excel**.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,18 @@
|
|||
# Docker only reads the .dockerignore INSIDE the build context (this folder) — the one at
|
||||
# the app root does not apply to `build: ./backend`. Without this file the image would
|
||||
# bake in backend/data: the live SQLite database and uploaded financial files.
|
||||
data
|
||||
.pytest_cache
|
||||
__pycache__
|
||||
**/__pycache__
|
||||
*.pyc
|
||||
.venv
|
||||
venv
|
||||
*.db
|
||||
*.db-shm
|
||||
*.db-wal
|
||||
*.xlsx
|
||||
*.xls
|
||||
*.csv
|
||||
.env
|
||||
.env.*
|
||||
|
|
@ -4,6 +4,7 @@ WORKDIR /app
|
|||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
gcc \
|
||||
default-mysql-client \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements.txt .
|
||||
|
|
@ -13,4 +14,10 @@ COPY . .
|
|||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["uvicorn", "app.api.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
|
||||
# Production default: single worker, proxy-aware, no reload.
|
||||
# SINGLE WORKER IS A CONSTRAINT, NOT A TUNABLE: processing/export jobs run in-process
|
||||
# (FastAPI BackgroundTasks) and progress state lives in that process. More workers or
|
||||
# replicas would split jobs from their status polling. Fine for a ~5-user finance team.
|
||||
# The dev compose file overrides this command with --reload.
|
||||
CMD ["uvicorn", "app.api.main:app", "--host", "0.0.0.0", "--port", "8000", \
|
||||
"--workers", "1", "--proxy-headers", "--forwarded-allow-ips", "*"]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,238 @@
|
|||
"""
|
||||
Authentication: per-user login with signed bearer tokens. Stdlib only — no new deps.
|
||||
|
||||
Design (deliberately minimal for a ~5-user internal finance tool):
|
||||
* Passwords: hashlib.scrypt (OpenSSL), per-user random salt, constant-time compare.
|
||||
* Tokens: HMAC-SHA256-signed JSON (user id, username, display name, expiry) — the
|
||||
same shape as a JWT but without the dependency. Signed with AR_SECRET_KEY;
|
||||
when unset, an ephemeral key is generated and a warning logged (every
|
||||
restart then logs everyone out — fine on a laptop, wrong on a server).
|
||||
* Enforcement: an HTTP middleware guards every /api/* route except the open set below.
|
||||
AR_AUTH=auto (default) requires login as soon as at least one user exists,
|
||||
so a fresh dev checkout and the test suite run without ceremony while
|
||||
creating the first real user turns authentication on by itself.
|
||||
* Identity: the verified display name feeds reviewed_by / approved_by / confirmed_by
|
||||
via actor_name(), replacing free-text name fields.
|
||||
|
||||
Users are created with `python manage.py add-user` — there is no self-signup endpoint.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ..config import AUTH_MODE, AUTH_TOKEN_HOURS, SECRET_KEY
|
||||
from ..db import models
|
||||
from ..db.database import SessionLocal
|
||||
from .deps import db_dep
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/auth", tags=["auth"])
|
||||
|
||||
# Paths reachable without a token: health probes, login itself, and the "is auth on?"
|
||||
# check the frontend makes before deciding whether to show the login screen.
|
||||
OPEN_PATHS = {"/api/health", "/api/auth/login", "/api/auth/status"}
|
||||
|
||||
if SECRET_KEY:
|
||||
_SECRET = SECRET_KEY.encode()
|
||||
else:
|
||||
_SECRET = secrets.token_bytes(32)
|
||||
logger.warning(
|
||||
"AR_SECRET_KEY is not set — using an ephemeral signing key. Login sessions will "
|
||||
"not survive a restart. Set AR_SECRET_KEY in production."
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- password hashing
|
||||
_SCRYPT_N, _SCRYPT_R, _SCRYPT_P = 16384, 8, 1
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
salt = secrets.token_bytes(16)
|
||||
digest = hashlib.scrypt(password.encode(), salt=salt,
|
||||
n=_SCRYPT_N, r=_SCRYPT_R, p=_SCRYPT_P, dklen=32)
|
||||
return (f"scrypt${_SCRYPT_N}${_SCRYPT_R}${_SCRYPT_P}"
|
||||
f"${salt.hex()}${digest.hex()}")
|
||||
|
||||
|
||||
def verify_password(password: str, stored: str) -> bool:
|
||||
try:
|
||||
algo, n, r, p, salt_hex, hash_hex = stored.split("$")
|
||||
if algo != "scrypt":
|
||||
return False
|
||||
digest = hashlib.scrypt(password.encode(), salt=bytes.fromhex(salt_hex),
|
||||
n=int(n), r=int(r), p=int(p),
|
||||
dklen=len(bytes.fromhex(hash_hex)))
|
||||
return hmac.compare_digest(digest, bytes.fromhex(hash_hex))
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------- tokens
|
||||
def _b64(data: bytes) -> str:
|
||||
return base64.urlsafe_b64encode(data).rstrip(b"=").decode()
|
||||
|
||||
|
||||
def _unb64(data: str) -> bytes:
|
||||
return base64.urlsafe_b64decode(data + "=" * (-len(data) % 4))
|
||||
|
||||
|
||||
def create_token(user: models.User) -> str:
|
||||
payload = json.dumps({
|
||||
"uid": user.id, "u": user.username, "dn": user.display_name,
|
||||
"exp": int(time.time()) + AUTH_TOKEN_HOURS * 3600,
|
||||
}, separators=(",", ":")).encode()
|
||||
sig = hmac.new(_SECRET, payload, hashlib.sha256).digest()
|
||||
return f"{_b64(payload)}.{_b64(sig)}"
|
||||
|
||||
|
||||
def parse_token(token: str) -> dict | None:
|
||||
"""The signed payload, or None if the token is malformed, forged, or expired."""
|
||||
try:
|
||||
payload_b64, sig_b64 = token.split(".")
|
||||
payload = _unb64(payload_b64)
|
||||
expected = hmac.new(_SECRET, payload, hashlib.sha256).digest()
|
||||
if not hmac.compare_digest(expected, _unb64(sig_b64)):
|
||||
return None
|
||||
data = json.loads(payload)
|
||||
if data.get("exp", 0) < time.time():
|
||||
return None
|
||||
return data
|
||||
except (ValueError, TypeError, json.JSONDecodeError):
|
||||
return None
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------- enforcement
|
||||
@dataclass
|
||||
class AuthUser:
|
||||
id: int
|
||||
username: str
|
||||
display_name: str
|
||||
|
||||
|
||||
# auto mode asks "do any users exist?" — cached briefly so it isn't a query per request.
|
||||
_users_exist_cache: tuple[float, bool] = (0.0, False)
|
||||
_USERS_CACHE_TTL_S = 10.0
|
||||
|
||||
|
||||
def _users_exist() -> bool:
|
||||
global _users_exist_cache
|
||||
ts, val = _users_exist_cache
|
||||
now = time.time()
|
||||
if now - ts < _USERS_CACHE_TTL_S:
|
||||
return val
|
||||
db = SessionLocal()
|
||||
try:
|
||||
val = db.query(models.User.id).filter(
|
||||
models.User.is_active == True).first() is not None # noqa: E712
|
||||
except Exception: # noqa: BLE001 — table may not exist mid-migration; fail open once
|
||||
val = False
|
||||
finally:
|
||||
db.close()
|
||||
_users_exist_cache = (now, val)
|
||||
return val
|
||||
|
||||
|
||||
def invalidate_users_cache() -> None:
|
||||
global _users_exist_cache
|
||||
_users_exist_cache = (0.0, False)
|
||||
|
||||
|
||||
def auth_required() -> bool:
|
||||
if AUTH_MODE == "off":
|
||||
return False
|
||||
if AUTH_MODE == "on":
|
||||
return True
|
||||
return _users_exist() # auto
|
||||
|
||||
|
||||
def _user_from_request(request: Request) -> AuthUser | None:
|
||||
header = request.headers.get("Authorization", "")
|
||||
if not header.startswith("Bearer "):
|
||||
return None
|
||||
data = parse_token(header[7:].strip())
|
||||
if data is None:
|
||||
return None
|
||||
return AuthUser(id=data["uid"], username=data["u"], display_name=data["dn"])
|
||||
|
||||
|
||||
async def auth_middleware(request: Request, call_next):
|
||||
"""Guards every /api/* route except OPEN_PATHS. Registered in api/main.py."""
|
||||
path = request.url.path.rstrip("/") or "/"
|
||||
if path.startswith("/api") and path not in OPEN_PATHS:
|
||||
user = _user_from_request(request)
|
||||
if user is not None:
|
||||
request.state.user = user
|
||||
elif auth_required():
|
||||
return JSONResponse({"detail": "Not signed in (or the session expired). "
|
||||
"Sign in to continue."}, status_code=401)
|
||||
else:
|
||||
request.state.user = None
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
def current_user(request: Request) -> AuthUser | None:
|
||||
"""The signed-in user, or None when auth is off/auto-without-users (dev, tests)."""
|
||||
return getattr(request.state, "user", None)
|
||||
|
||||
|
||||
def actor_name(request: Request, provided: str = "") -> str:
|
||||
"""The name that lands in accountability fields (reviewed_by / approved_by / …).
|
||||
|
||||
The verified identity always wins; the body-provided name is only honoured when no
|
||||
one is signed in (auth off / auto without users), which keeps dev and tests working."""
|
||||
user = current_user(request)
|
||||
if user is not None and user.display_name:
|
||||
return user.display_name
|
||||
return (provided or "").strip()
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------- routes
|
||||
class LoginIn(BaseModel):
|
||||
username: str
|
||||
password: str
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
def auth_status() -> dict:
|
||||
"""Whether the frontend must show a login screen."""
|
||||
return {"auth_required": auth_required()}
|
||||
|
||||
|
||||
@router.post("/login")
|
||||
def login(body: LoginIn, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
user = db.query(models.User).filter(
|
||||
models.User.username == body.username.strip().lower()).first()
|
||||
if (user is None or not user.is_active
|
||||
or not verify_password(body.password, user.password_hash)):
|
||||
# One message for both wrong-user and wrong-password: don't confirm usernames.
|
||||
raise HTTPException(401, "Wrong username or password.")
|
||||
logger.info("login: %s", user.username)
|
||||
return {
|
||||
"token": create_token(user),
|
||||
"user": {"username": user.username, "display_name": user.display_name},
|
||||
"expires_in_hours": AUTH_TOKEN_HOURS,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/me")
|
||||
def me(request: Request) -> dict:
|
||||
user = current_user(request)
|
||||
if user is None:
|
||||
if auth_required():
|
||||
raise HTTPException(401, "Not signed in.")
|
||||
return {"authenticated": False, "auth_required": False}
|
||||
return {"authenticated": True, "auth_required": True,
|
||||
"username": user.username, "display_name": user.display_name}
|
||||
|
|
@ -49,6 +49,20 @@ def ensure_not_blocked(s: models.Session) -> None:
|
|||
)
|
||||
|
||||
|
||||
def ensure_editable(s: models.Session) -> None:
|
||||
"""Guard every mutating endpoint: a completed closing is locked history.
|
||||
|
||||
Its figures were signed off and possibly booked — silently editing them would make the
|
||||
record disagree with what was published. Corrections go through an explicit reopen
|
||||
(POST /sessions/{id}/reopen), which is visible and deliberate."""
|
||||
if s.status == "completed":
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="This closing is completed and locked (read-only). "
|
||||
"Reopen it first if a correction is genuinely needed.",
|
||||
)
|
||||
|
||||
|
||||
_SAFE = re.compile(r"[^A-Za-z0-9 ._,()\-]+")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,28 +1,66 @@
|
|||
"""FastAPI application entrypoint."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from sqlalchemy import text
|
||||
|
||||
from .. import APP_NAME, APP_VERSION
|
||||
from ..config import CORS_ORIGINS
|
||||
from ..db.database import init_db
|
||||
from ..config import CORS_ORIGINS, DATA_DIR
|
||||
from ..db.database import ENGINE, init_db
|
||||
from . import auth
|
||||
from .routes import (
|
||||
sessions, files, processing, results, settings as settings_routes, export, ar, control,
|
||||
analytics, controls, payouts, accounts_summary,
|
||||
analytics, controls, payouts, accounts_summary, fx,
|
||||
)
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app: FastAPI):
|
||||
init_db()
|
||||
# A restart mid-job leaves a closing stuck on "processing" forever — recover it.
|
||||
from ..services.jobs import recover_stale_jobs
|
||||
recover_stale_jobs()
|
||||
# Daily retention sweep (exports only; uploads are the audit source and are kept).
|
||||
from ..services.retention import retention_loop
|
||||
sweeper = asyncio.create_task(retention_loop())
|
||||
yield
|
||||
sweeper.cancel()
|
||||
|
||||
|
||||
app = FastAPI(title=APP_NAME, version=APP_VERSION, lifespan=lifespan)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def _request_log(request: Request, call_next):
|
||||
"""One line per API request: method, path, status, duration."""
|
||||
if not request.url.path.startswith("/api"):
|
||||
return await call_next(request)
|
||||
t0 = time.perf_counter()
|
||||
response = await call_next(request)
|
||||
ms = (time.perf_counter() - t0) * 1000
|
||||
# /status is polled sub-second during processing; logging it would drown everything.
|
||||
if not request.url.path.endswith("/status"):
|
||||
logger.info("%s %s -> %d (%.0f ms)",
|
||||
request.method, request.url.path, response.status_code, ms)
|
||||
return response
|
||||
|
||||
|
||||
# Registered BEFORE CORSMiddleware so CORS stays outermost (Starlette applies middleware
|
||||
# in reverse registration order) and 401 responses still carry CORS headers.
|
||||
app.middleware("http")(auth.auth_middleware)
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=CORS_ORIGINS,
|
||||
|
|
@ -34,9 +72,27 @@ app.add_middleware(
|
|||
|
||||
@app.get("/api/health")
|
||||
def health() -> dict:
|
||||
return {"status": "ok", "app": APP_NAME, "version": APP_VERSION}
|
||||
"""Liveness + readiness: DB reachable and the data dir writable — deep enough for a
|
||||
load balancer / monitor probe, fast enough to hit every few seconds."""
|
||||
checks = {"db": "ok", "data_dir": "ok"}
|
||||
status = "ok"
|
||||
try:
|
||||
with ENGINE.connect() as conn:
|
||||
conn.execute(text("SELECT 1"))
|
||||
except Exception as e: # noqa: BLE001
|
||||
checks["db"] = f"error: {type(e).__name__}"
|
||||
status = "degraded"
|
||||
try:
|
||||
probe = DATA_DIR / ".health-probe"
|
||||
probe.write_text("ok")
|
||||
probe.unlink()
|
||||
except OSError as e:
|
||||
checks["data_dir"] = f"error: {type(e).__name__}"
|
||||
status = "degraded"
|
||||
return {"status": status, "app": APP_NAME, "version": APP_VERSION, "checks": checks}
|
||||
|
||||
|
||||
app.include_router(auth.router)
|
||||
app.include_router(sessions.router)
|
||||
app.include_router(files.router)
|
||||
app.include_router(processing.router)
|
||||
|
|
@ -51,3 +107,4 @@ app.include_router(analytics.router)
|
|||
app.include_router(controls.router)
|
||||
app.include_router(payouts.router)
|
||||
app.include_router(accounts_summary.router)
|
||||
app.include_router(fx.router)
|
||||
|
|
|
|||
|
|
@ -77,6 +77,42 @@ def accounts_summary(db: OrmSession = Depends(db_dep)) -> dict:
|
|||
"receivable": accrual_total, # Dr A/R (net revenue accrued)
|
||||
})
|
||||
|
||||
# Months that HAVE results but are not published, so they don't just vanish from this
|
||||
# view without a word (the #1 "my previous month disappeared" confusion): processed or
|
||||
# blocked closings whose journal is not approved — including sign-offs cleared by a
|
||||
# re-process — are listed with the reason and a link target.
|
||||
published_ids = {m["session_id"] for m in months}
|
||||
pending: list[dict] = []
|
||||
candidates = db.query(models.Session).filter(
|
||||
models.Session.status.in_(("processed", "blocked", "completed"))).all()
|
||||
journals = {j.session_id: j for j in db.query(models.JournalEntry).filter(
|
||||
models.JournalEntry.session_id.in_([s.id for s in candidates]))} if candidates else {}
|
||||
for s in candidates:
|
||||
if s.id in published_ids:
|
||||
continue
|
||||
j = journals.get(s.id)
|
||||
if is_blocked(s):
|
||||
reason = f"blocked by a failed month-end control — {s.blocked_reason}"
|
||||
elif j is None or not j.data:
|
||||
reason = "no journal entry yet — re-process the closing"
|
||||
elif j.approved_by:
|
||||
reason = "approved, but the closing is blocked or has no journal data"
|
||||
elif j.entry_no and not j.reviewed_by:
|
||||
# An entry number exists but both sign-offs are empty: the usual cause is a
|
||||
# re-process, which deliberately withdraws review/approval.
|
||||
reason = ("sign-off was cleared (typically by re-processing) — "
|
||||
"review and approve the journal again to re-publish")
|
||||
else:
|
||||
reason = "journal not approved yet — approval is what publishes a month here"
|
||||
pending.append({
|
||||
"month": s.reporting_month or (s.month_end_date.isoformat()[:7]
|
||||
if s.month_end_date else f"session-{s.id}"),
|
||||
"session_id": s.id,
|
||||
"session_name": s.name,
|
||||
"reason": reason,
|
||||
})
|
||||
pending.sort(key=lambda p: p["month"])
|
||||
|
||||
return {
|
||||
"available": bool(months),
|
||||
"line_keys": line_keys,
|
||||
|
|
@ -84,4 +120,5 @@ def accounts_summary(db: OrmSession = Depends(db_dep)) -> dict:
|
|||
"months": months,
|
||||
"marketplaces": sorted(marketplaces),
|
||||
"cells": cells,
|
||||
"pending": pending,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ from sqlalchemy.orm import Session as OrmSession
|
|||
|
||||
from ...core.i18n import currency_for_region, default_fx_for_region
|
||||
from ...db import models
|
||||
from ..deps import db_dep, get_session_or_404
|
||||
from ..deps import db_dep, ensure_editable, get_session_or_404
|
||||
from .ar import _market_list, _movement_for, _payouts_for, fx_for
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["analytics"])
|
||||
|
|
@ -332,7 +332,7 @@ def fx_daily(session_id: int, marketplace: str | None = None,
|
|||
@router.put("/{session_id}/fx-daily")
|
||||
def put_fx_daily(session_id: int, items: list[DailyFxIn],
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
get_session_or_404(session_id, db)
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
for it in items:
|
||||
d = _parse_date(it.rate_date, "rate_date")
|
||||
row = db.query(models.FxRateDaily).filter(
|
||||
|
|
|
|||
|
|
@ -10,7 +10,8 @@ from sqlalchemy.orm import Session as OrmSession
|
|||
from ...core.i18n import currency_for_region, default_fx_for_region
|
||||
from ...core.movement import compute_movement
|
||||
from ...db import models
|
||||
from ..deps import blocked_payload, db_dep, get_session_or_404, is_blocked, to_dict
|
||||
from ..deps import (blocked_payload, db_dep, ensure_editable, get_session_or_404, is_blocked,
|
||||
to_dict)
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["ar"])
|
||||
|
||||
|
|
@ -50,6 +51,7 @@ def put_openings(session_id: int, items: list[OpeningIn],
|
|||
db: OrmSession = Depends(db_dep)) -> list[dict]:
|
||||
"""Set one or more marketplaces' opening balances (only the ones sent are touched)."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
existing = {o.marketplace: o for o in db.query(models.OpeningBalance).filter(
|
||||
models.OpeningBalance.session_id == session_id)}
|
||||
for it in items:
|
||||
|
|
@ -365,6 +367,7 @@ def carry_forward(session_id: int, body: CarryForwardIn | None = None,
|
|||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Copy a prior closing's per-marketplace closing balance into this closing's opening."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
src_id = (body.from_session_id if body else None) or s.opening_source_session_id
|
||||
if src_id is None:
|
||||
cands = opening_candidates(session_id, db)["candidates"]
|
||||
|
|
@ -402,6 +405,7 @@ def carry_forward(session_id: int, body: CarryForwardIn | None = None,
|
|||
def reset_openings(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Set every opening balance to zero (the default for a first-ever closing)."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
for o in db.query(models.OpeningBalance).filter(
|
||||
models.OpeningBalance.session_id == session_id):
|
||||
o.amount = 0.0
|
||||
|
|
|
|||
|
|
@ -4,13 +4,14 @@ from __future__ import annotations
|
|||
import datetime as dt
|
||||
import json
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ...core.money import USD, Total, to_usd
|
||||
from ...db import models
|
||||
from ..deps import db_dep, ensure_not_blocked, get_session_or_404
|
||||
from ..auth import actor_name
|
||||
from ..deps import db_dep, ensure_editable, ensure_not_blocked, get_session_or_404
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["control"])
|
||||
|
||||
|
|
@ -123,7 +124,7 @@ def _get_or_create(db: OrmSession, session_id: int) -> models.FinanceControl:
|
|||
|
||||
@router.put("/{session_id}/reconciliation-control")
|
||||
def put_control(session_id: int, body: ControlIn, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
get_session_or_404(session_id, db)
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
fc = _get_or_create(db, session_id)
|
||||
data = body.model_dump(exclude_unset=True)
|
||||
# A sign-off attests to specific numbers. If any control figure or the tolerance changes,
|
||||
|
|
@ -143,15 +144,19 @@ def put_control(session_id: int, body: ControlIn, db: OrmSession = Depends(db_de
|
|||
|
||||
|
||||
class VerifyIn(BaseModel):
|
||||
verified_by: str
|
||||
verified_by: str = "" # ignored when signed in — the verified identity wins
|
||||
comment: str = ""
|
||||
|
||||
|
||||
@router.post("/{session_id}/reconciliation-control/verify")
|
||||
def verify_control(session_id: int, body: VerifyIn, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
get_session_or_404(session_id, db)
|
||||
def verify_control(session_id: int, body: VerifyIn, request: Request,
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
who = actor_name(request, body.verified_by)
|
||||
if not who:
|
||||
raise HTTPException(400, "verified_by is required — the control is verified by a person.")
|
||||
fc = _get_or_create(db, session_id)
|
||||
fc.verified_by = body.verified_by
|
||||
fc.verified_by = who
|
||||
fc.verified_at = dt.datetime.utcnow()
|
||||
if body.comment:
|
||||
fc.comment = body.comment
|
||||
|
|
|
|||
|
|
@ -3,13 +3,14 @@ from __future__ import annotations
|
|||
|
||||
import datetime as dt
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ...db import models
|
||||
from ...services.controls_run import payload, run_and_persist
|
||||
from ..deps import db_dep, get_session_or_404
|
||||
from ..auth import actor_name
|
||||
from ..deps import db_dep, ensure_editable, get_session_or_404
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["controls"])
|
||||
|
||||
|
|
@ -38,11 +39,12 @@ class FxConfirmIn(BaseModel):
|
|||
marketplace: str
|
||||
rate: float | None = None # optionally correct the rate while confirming it
|
||||
currency: str | None = None
|
||||
confirmed_by: str
|
||||
confirmed_by: str = "" # ignored when signed in — the verified identity wins
|
||||
|
||||
|
||||
@router.post("/{session_id}/fx/confirm")
|
||||
def confirm_fx(session_id: int, body: FxConfirmIn, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
def confirm_fx(session_id: int, body: FxConfirmIn, request: Request,
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""
|
||||
Record that a human confirmed this marketplace's rate FOR THIS REPORTING MONTH.
|
||||
|
||||
|
|
@ -50,7 +52,9 @@ def confirm_fx(session_id: int, body: FxConfirmIn, db: OrmSession = Depends(db_d
|
|||
snapshot and would otherwise value any later month at January's rates in silence.
|
||||
"""
|
||||
s = get_session_or_404(session_id, db)
|
||||
if not body.confirmed_by.strip():
|
||||
ensure_editable(s)
|
||||
who = actor_name(request, body.confirmed_by)
|
||||
if not who:
|
||||
raise HTTPException(400, "confirmed_by is required — a rate is confirmed by a person.")
|
||||
row = db.query(models.FxRate).filter(
|
||||
models.FxRate.session_id == session_id,
|
||||
|
|
@ -62,7 +66,7 @@ def confirm_fx(session_id: int, body: FxConfirmIn, db: OrmSession = Depends(db_d
|
|||
row.rate = body.rate
|
||||
if body.currency:
|
||||
row.currency = body.currency
|
||||
row.confirmed_by = body.confirmed_by.strip()
|
||||
row.confirmed_by = who
|
||||
row.confirmed_at = dt.datetime.utcnow()
|
||||
row.confirmed_month = s.reporting_month or ""
|
||||
row.source = f"confirmed by {row.confirmed_by}"
|
||||
|
|
@ -71,15 +75,16 @@ def confirm_fx(session_id: int, body: FxConfirmIn, db: OrmSession = Depends(db_d
|
|||
|
||||
|
||||
class FxConfirmAllIn(BaseModel):
|
||||
confirmed_by: str
|
||||
confirmed_by: str = "" # ignored when signed in — the verified identity wins
|
||||
|
||||
|
||||
@router.post("/{session_id}/fx/confirm-all")
|
||||
def confirm_all_fx(session_id: int, body: FxConfirmAllIn,
|
||||
def confirm_all_fx(session_id: int, body: FxConfirmAllIn, request: Request,
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Confirm every rate on the closing as-is (after reviewing them on the Settings tab)."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
who = body.confirmed_by.strip()
|
||||
ensure_editable(s)
|
||||
who = actor_name(request, body.confirmed_by)
|
||||
if not who:
|
||||
raise HTTPException(400, "confirmed_by is required — a rate is confirmed by a person.")
|
||||
now = dt.datetime.utcnow()
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ from ...config import ALLOWED_EXTENSIONS, MAX_UPLOAD_BYTES, UPLOAD_DIR
|
|||
from ...core.readers import make_reader
|
||||
from ...core.xlsx_reader import ParseError
|
||||
from ...db import models
|
||||
from ..deps import db_dep, file_dict, get_session_or_404, sanitize_filename
|
||||
from ..deps import db_dep, ensure_editable, file_dict, get_session_or_404, sanitize_filename
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["files"])
|
||||
|
||||
|
|
@ -23,45 +23,15 @@ def list_files(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict]:
|
|||
return [file_dict(f) for f in rows]
|
||||
|
||||
|
||||
@router.post("/{session_id}/files")
|
||||
async def upload_files(session_id: int, files: list[UploadFile] = File(...),
|
||||
db: OrmSession = Depends(db_dep)) -> list[dict]:
|
||||
session = get_session_or_404(session_id, db)
|
||||
dest_dir = UPLOAD_DIR / f"session_{session_id}"
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
out = []
|
||||
for uf in files:
|
||||
safe = sanitize_filename(uf.filename or "upload.xlsx")
|
||||
ext = os.path.splitext(safe)[1].lower()
|
||||
if ext not in ALLOWED_EXTENSIONS:
|
||||
raise HTTPException(400, f"Unsupported file type: {safe} ({ext})")
|
||||
path = dest_dir / safe
|
||||
h = hashlib.sha256()
|
||||
size = 0
|
||||
with open(path, "wb") as fh:
|
||||
while True:
|
||||
chunk = await uf.read(1 << 20)
|
||||
if not chunk:
|
||||
break
|
||||
size += len(chunk)
|
||||
if size > MAX_UPLOAD_BYTES:
|
||||
fh.close()
|
||||
os.remove(path)
|
||||
raise HTTPException(413, f"File too large: {safe}")
|
||||
h.update(chunk)
|
||||
fh.write(chunk)
|
||||
rec = models.SessionFile(
|
||||
session_id=session_id, filename=safe, stored_path=str(path),
|
||||
size_bytes=size, sha256=h.hexdigest(), status="uploaded",
|
||||
)
|
||||
# light validation: detect sheet/header + required columns (no full row scan)
|
||||
def _validate(rec: models.SessionFile, path: str) -> None:
|
||||
"""Light validation: detect sheet/header + required columns (no full row scan)."""
|
||||
try:
|
||||
reader = make_reader(str(path))
|
||||
reader = make_reader(path)
|
||||
reader.detect()
|
||||
rec.data_sheet = reader.sheet_name
|
||||
rec.status = "invalid" if reader.column_mapping.missing_required else "parsed"
|
||||
if reader.column_mapping.missing_required:
|
||||
rec.message = f"missing required columns: {reader.column_mapping.missing_required}"
|
||||
rec.message = (f"missing required columns: {reader.column_mapping.missing_required}"
|
||||
if reader.column_mapping.missing_required else "")
|
||||
# Surface marketplace / date span when cheap (CSV already has rows in memory;
|
||||
# for xlsx this stays blank until processing).
|
||||
meta = reader.file_meta
|
||||
|
|
@ -71,15 +41,119 @@ async def upload_files(session_id: int, files: list[UploadFile] = File(...),
|
|||
except ParseError as e:
|
||||
rec.status = "invalid"
|
||||
rec.message = str(e)
|
||||
|
||||
|
||||
@router.post("/{session_id}/files")
|
||||
async def upload_files(session_id: int, files: list[UploadFile] = File(...),
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""
|
||||
Upload one or more source files into a closing.
|
||||
|
||||
Duplicate protection (both were real double-count bugs):
|
||||
* same filename again -> the existing row is UPDATED in place (the file is replaced),
|
||||
never a second row pointing at the same path — two rows would make the pipeline
|
||||
parse and sum the file twice.
|
||||
* same content under a different name -> skipped and reported, for the same reason.
|
||||
|
||||
Returns {"files": [...saved/replaced...], "skipped": [{"filename", "reason"}]} so one
|
||||
duplicate in a 13-file batch doesn't fail the other twelve.
|
||||
"""
|
||||
session = get_session_or_404(session_id, db)
|
||||
ensure_editable(session)
|
||||
dest_dir = UPLOAD_DIR / f"session_{session_id}"
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
existing = db.query(models.SessionFile).filter(
|
||||
models.SessionFile.session_id == session_id).all()
|
||||
by_name = {f.filename: f for f in existing}
|
||||
by_sha = {f.sha256: f for f in existing if f.sha256}
|
||||
|
||||
was_processed = session.status in ("processed", "blocked")
|
||||
out: list[models.SessionFile] = []
|
||||
skipped: list[dict] = []
|
||||
changed = False
|
||||
|
||||
for uf in files:
|
||||
safe = sanitize_filename(uf.filename or "upload.xlsx")
|
||||
ext = os.path.splitext(safe)[1].lower()
|
||||
if ext not in ALLOWED_EXTENSIONS:
|
||||
raise HTTPException(400, f"Unsupported file type: {safe} ({ext})")
|
||||
path = dest_dir / safe
|
||||
# Stream to a temp name first: the hash decides whether this upload is kept, and a
|
||||
# failed/oversized upload must never clobber a good file already on disk.
|
||||
tmp = dest_dir / (safe + ".part")
|
||||
h = hashlib.sha256()
|
||||
size = 0
|
||||
with open(tmp, "wb") as fh:
|
||||
while True:
|
||||
chunk = await uf.read(1 << 20)
|
||||
if not chunk:
|
||||
break
|
||||
size += len(chunk)
|
||||
if size > MAX_UPLOAD_BYTES:
|
||||
fh.close()
|
||||
os.remove(tmp)
|
||||
raise HTTPException(413, f"File too large: {safe}")
|
||||
h.update(chunk)
|
||||
fh.write(chunk)
|
||||
sha = h.hexdigest()
|
||||
|
||||
same_name = by_name.get(safe)
|
||||
same_content = by_sha.get(sha)
|
||||
|
||||
if same_name is not None and same_name.sha256 == sha:
|
||||
os.remove(tmp)
|
||||
skipped.append({"filename": safe,
|
||||
"reason": "identical file already uploaded — unchanged"})
|
||||
continue
|
||||
if same_content is not None and (same_name is None or same_content.id != same_name.id):
|
||||
os.remove(tmp)
|
||||
skipped.append({"filename": safe,
|
||||
"reason": f"identical content already uploaded as "
|
||||
f"'{same_content.filename}'"})
|
||||
continue
|
||||
|
||||
os.replace(tmp, path)
|
||||
changed = True
|
||||
if same_name is not None:
|
||||
# Replace in place: update the existing row rather than adding a second one.
|
||||
rec = same_name
|
||||
rec.stored_path = str(path)
|
||||
rec.size_bytes = size
|
||||
rec.sha256 = sha
|
||||
rec.status = "uploaded"
|
||||
rec.message = ""
|
||||
rec.imported_rows = 0
|
||||
rec.min_date = None
|
||||
rec.max_date = None
|
||||
rec.marketplace = None
|
||||
rec.sheet_last_row = 0
|
||||
rec.blank_rows_skipped = 0
|
||||
rec.helper_rows_skipped = 0
|
||||
else:
|
||||
rec = models.SessionFile(
|
||||
session_id=session_id, filename=safe, stored_path=str(path),
|
||||
size_bytes=size, sha256=sha, status="uploaded",
|
||||
)
|
||||
db.add(rec)
|
||||
_validate(rec, str(path))
|
||||
by_name[safe] = rec
|
||||
by_sha[sha] = rec
|
||||
out.append(rec)
|
||||
|
||||
if changed:
|
||||
session.status = "draft"
|
||||
if was_processed:
|
||||
# The stored results no longer reflect the files on disk.
|
||||
session.needs_reprocess = True
|
||||
db.commit()
|
||||
return [file_dict(f) for f in out]
|
||||
return {"files": [file_dict(f) for f in out], "skipped": skipped}
|
||||
|
||||
|
||||
@router.delete("/{session_id}/files/{file_id}")
|
||||
def delete_file(session_id: int, file_id: int, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
f = db.get(models.SessionFile, file_id)
|
||||
if not f or f.session_id != session_id:
|
||||
raise HTTPException(404, "File not found")
|
||||
|
|
@ -89,5 +163,7 @@ def delete_file(session_id: int, file_id: int, db: OrmSession = Depends(db_dep))
|
|||
except OSError:
|
||||
pass
|
||||
db.delete(f)
|
||||
if s.status in ("processed", "blocked"):
|
||||
s.needs_reprocess = True
|
||||
db.commit()
|
||||
return {"deleted": file_id}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,47 @@
|
|||
"""Fetch exchange rates from the configured provider (Frankfurter by default).
|
||||
|
||||
Fetched rates arrive UNCONFIRMED: Control C5 still blocks the close until a person
|
||||
confirms them for the reporting month — this endpoint only replaces typing rates by hand."""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ...services.fx_service import FxProviderError, seed_daily_fx, seed_session_fx
|
||||
from ..deps import db_dep, ensure_editable, get_session_or_404
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["fx"])
|
||||
|
||||
|
||||
@router.post("/{session_id}/fx/fetch")
|
||||
def fetch_month_end_rates(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Pre-fill this closing's FX table with the provider's month-end rates."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
try:
|
||||
return seed_session_fx(db, s)
|
||||
except FxProviderError as e:
|
||||
raise HTTPException(502, f"{e} — enter the rates manually on the Controls tab.")
|
||||
|
||||
|
||||
class DailyFetchIn(BaseModel):
|
||||
marketplace: str | None = None # default: every non-USD marketplace in the closing
|
||||
date_from: dt.date | None = None # default: first day of the reporting month
|
||||
date_to: dt.date | None = None # default: month-end
|
||||
|
||||
|
||||
@router.post("/{session_id}/fx/fetch-daily")
|
||||
def fetch_daily_rates(session_id: int, body: DailyFetchIn | None = None,
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Fill the per-date FX override table from the provider for a date range."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
body = body or DailyFetchIn()
|
||||
try:
|
||||
return seed_daily_fx(db, s, marketplace=body.marketplace,
|
||||
date_from=body.date_from, date_to=body.date_to)
|
||||
except FxProviderError as e:
|
||||
raise HTTPException(502, f"{e} — enter daily rates manually on the AR Ledger tab.")
|
||||
|
|
@ -19,13 +19,14 @@ from __future__ import annotations
|
|||
|
||||
import datetime as dt
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ...db import models
|
||||
from ..deps import db_dep, get_session_or_404
|
||||
from ..auth import actor_name
|
||||
from ..deps import db_dep, ensure_editable, get_session_or_404
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["payouts"])
|
||||
|
||||
|
|
@ -118,11 +119,12 @@ class ReceiptIn(BaseModel):
|
|||
|
||||
|
||||
@router.put("/{session_id}/payouts/receipts")
|
||||
def put_receipts(session_id: int, items: list[ReceiptIn],
|
||||
def put_receipts(session_id: int, items: list[ReceiptIn], request: Request,
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Batch upsert bank receipts. Only the payouts sent are touched; a null bank_date
|
||||
deletes that payout's receipt (it reverts to the mode's default rule)."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
if s.status == "processing":
|
||||
raise HTTPException(409, "This closing is still processing — wait for it to finish.")
|
||||
existing = {(r.marketplace, r.account_type, r.settlement_id): r
|
||||
|
|
@ -149,7 +151,8 @@ def put_receipts(session_id: int, items: list[ReceiptIn],
|
|||
row.bank_date = bank_date
|
||||
row.bank_amount = it.bank_amount
|
||||
row.note = it.note or ""
|
||||
row.entered_by = it.entered_by or ""
|
||||
# The signed-in user's name wins; the free-text field only counts without auth.
|
||||
row.entered_by = actor_name(request, it.entered_by)
|
||||
saved += 1
|
||||
if saved or removed:
|
||||
# The stored classification no longer reflects the receipts until a re-process.
|
||||
|
|
@ -166,6 +169,7 @@ class ModeIn(BaseModel):
|
|||
def put_mode(session_id: int, body: ModeIn, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""auto = bank date wins, clearing-lag fallback · manual = bank dates only, no heuristic."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
if body.mode not in ("auto", "manual"):
|
||||
raise HTTPException(400, "mode must be 'auto' or 'manual'.")
|
||||
if s.status == "processing":
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ from sqlalchemy.orm import Session as OrmSession
|
|||
|
||||
from ...db import models
|
||||
from ...services.jobs import run_processing
|
||||
from ..deps import db_dep, get_session_or_404, session_dict
|
||||
from ..deps import db_dep, ensure_editable, get_session_or_404, session_dict
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["processing"])
|
||||
|
||||
|
|
@ -15,6 +15,7 @@ router = APIRouter(prefix="/api/sessions", tags=["processing"])
|
|||
def start_processing(session_id: int, background: BackgroundTasks,
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
if s.month_end_date is None:
|
||||
raise HTTPException(400, "Set the month-end date before processing.")
|
||||
valid_files = db.query(models.SessionFile).filter(
|
||||
|
|
|
|||
|
|
@ -4,14 +4,16 @@ from __future__ import annotations
|
|||
import datetime as dt
|
||||
import json
|
||||
|
||||
from fastapi import APIRouter, Body, Depends, HTTPException, Query
|
||||
from fastapi import APIRouter, Body, Depends, HTTPException, Query, Request
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ...core.receivable import AGING_BANDS, classify_aging
|
||||
from ...core.settlements import RECEIVABLE_ACCOUNT_TYPES
|
||||
from ...db import models
|
||||
from ..deps import blocked_payload, db_dep, get_session_or_404, is_blocked, to_dict
|
||||
from ..auth import actor_name
|
||||
from ..deps import (blocked_payload, db_dep, ensure_editable, get_session_or_404, is_blocked,
|
||||
to_dict)
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["results"])
|
||||
|
||||
|
|
@ -176,7 +178,7 @@ def journal(session_id: int, marketplace: str | None = None,
|
|||
@router.put("/{session_id}/journal/entry-no")
|
||||
def set_journal_entry_no(session_id: int, entry_no: str = Body(..., embed=True),
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
get_session_or_404(session_id, db)
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
j = db.query(models.JournalEntry).filter(
|
||||
models.JournalEntry.session_id == session_id).first()
|
||||
if j:
|
||||
|
|
@ -194,28 +196,35 @@ def _journal_row_or_400(session_id: int, db: OrmSession) -> models.JournalEntry:
|
|||
|
||||
|
||||
@router.post("/{session_id}/journal/review")
|
||||
def review_journal(session_id: int, name: str = Body(..., embed=True),
|
||||
def review_journal(session_id: int, request: Request, name: str = Body("", embed=True),
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Step 1 of the sign-off: a person confirms they reviewed this month's journal."""
|
||||
get_session_or_404(session_id, db)
|
||||
if not name.strip():
|
||||
"""Step 1 of the sign-off: a person confirms they reviewed this month's journal.
|
||||
|
||||
The signed-in user's display name is recorded; the body `name` only counts when no
|
||||
one is signed in (auth off — dev and tests)."""
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
who = actor_name(request, name)
|
||||
if not who:
|
||||
raise HTTPException(400, "A reviewer name is required.")
|
||||
j = _journal_row_or_400(session_id, db)
|
||||
j.reviewed_by = name.strip()
|
||||
j.reviewed_by = who
|
||||
j.reviewed_at = dt.datetime.utcnow()
|
||||
db.commit()
|
||||
return journal(session_id, None, db)
|
||||
|
||||
|
||||
@router.post("/{session_id}/journal/approve")
|
||||
def approve_journal(session_id: int, name: str = Body(..., embed=True),
|
||||
def approve_journal(session_id: int, request: Request, name: str = Body("", embed=True),
|
||||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Step 2: approval — this is what publishes the month to the Accounts Summary.
|
||||
|
||||
Requires a prior review, and a closing that isn't blocked by a month-end control:
|
||||
an unverified number must never become part of the cross-month accounts view."""
|
||||
an unverified number must never become part of the cross-month accounts view.
|
||||
The signed-in user's display name is recorded (body `name` only without auth)."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
if not name.strip():
|
||||
ensure_editable(s)
|
||||
who = actor_name(request, name)
|
||||
if not who:
|
||||
raise HTTPException(400, "An approver name is required.")
|
||||
if is_blocked(s):
|
||||
raise HTTPException(409, f"This closing is blocked by a failed month-end control — "
|
||||
|
|
@ -223,7 +232,7 @@ def approve_journal(session_id: int, name: str = Body(..., embed=True),
|
|||
j = _journal_row_or_400(session_id, db)
|
||||
if not j.reviewed_by:
|
||||
raise HTTPException(400, "The journal must be reviewed before it can be approved.")
|
||||
j.approved_by = name.strip()
|
||||
j.approved_by = who
|
||||
j.approved_at = dt.datetime.utcnow()
|
||||
db.commit()
|
||||
return journal(session_id, None, db)
|
||||
|
|
@ -232,7 +241,7 @@ def approve_journal(session_id: int, name: str = Body(..., embed=True),
|
|||
@router.post("/{session_id}/journal/reset-signoff")
|
||||
def reset_journal_signoff(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Withdraw the sign-off (removes the month from the Accounts Summary)."""
|
||||
get_session_or_404(session_id, db)
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
j = _journal_row_or_400(session_id, db)
|
||||
j.reviewed_by = ""
|
||||
j.reviewed_at = None
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
"""Session (month-end closing) CRUD and parameters."""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime
|
||||
import logging
|
||||
from datetime import date
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
|
@ -13,6 +14,8 @@ from ..deps import db_dep, get_session_or_404, session_dict
|
|||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["sessions"])
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SessionCreate(BaseModel):
|
||||
name: str
|
||||
|
|
@ -24,6 +27,9 @@ class SessionCreate(BaseModel):
|
|||
# zero (default) | carry_forward | manual
|
||||
opening_mode: str = "zero"
|
||||
opening_source_session_id: int | None = None
|
||||
# Two closings for one month is almost always an accident (two competing datasets for
|
||||
# the same period); creating a second one requires this explicit flag.
|
||||
allow_duplicate: bool = False
|
||||
|
||||
|
||||
class SessionUpdate(BaseModel):
|
||||
|
|
@ -39,19 +45,56 @@ class SessionUpdate(BaseModel):
|
|||
opening_source_session_id: int | None = None
|
||||
|
||||
|
||||
def _approved_session_ids(db: OrmSession) -> set[int]:
|
||||
rows = db.query(models.JournalEntry.session_id).filter(
|
||||
models.JournalEntry.approved_by != "").all()
|
||||
return {r[0] for r in rows}
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_sessions(db: OrmSession = Depends(db_dep)) -> list[dict]:
|
||||
rows = db.query(models.Session).order_by(models.Session.created_at.desc()).all()
|
||||
return [session_dict(s) for s in rows]
|
||||
"""Every closing, newest month first — the dashboard reads as a month timeline."""
|
||||
rows = db.query(models.Session).all()
|
||||
# reporting_month is "YYYY-MM" so string sort == chronological; sessions without a
|
||||
# month (never given a month-end date) sort last, newest created first.
|
||||
rows.sort(key=lambda s: (s.reporting_month or "",
|
||||
s.created_at.isoformat() if s.created_at else ""), reverse=True)
|
||||
approved = _approved_session_ids(db)
|
||||
months_seen: dict[str, int] = {}
|
||||
for s in rows:
|
||||
if s.reporting_month:
|
||||
months_seen[s.reporting_month] = months_seen.get(s.reporting_month, 0) + 1
|
||||
out = []
|
||||
for s in rows:
|
||||
d = session_dict(s)
|
||||
# "Published" = the journal is approved, which is what puts the month on the
|
||||
# cross-month Accounts Summary.
|
||||
d["journal_approved"] = s.id in approved
|
||||
d["duplicate_month"] = bool(s.reporting_month
|
||||
and months_seen.get(s.reporting_month, 0) > 1)
|
||||
out.append(d)
|
||||
return out
|
||||
|
||||
|
||||
@router.post("")
|
||||
def create_session(body: SessionCreate, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
me = body.month_end_date
|
||||
month = me.strftime("%Y-%m") if me else None
|
||||
if month and not body.allow_duplicate:
|
||||
clash = db.query(models.Session).filter(
|
||||
models.Session.reporting_month == month,
|
||||
models.Session.status != "error").first()
|
||||
if clash is not None:
|
||||
raise HTTPException(
|
||||
409,
|
||||
f"A closing for {month} already exists ('{clash.name}', id {clash.id}). "
|
||||
f"Open that closing instead — or pass allow_duplicate to deliberately "
|
||||
f"create a second one.",
|
||||
)
|
||||
s = models.Session(
|
||||
name=body.name,
|
||||
month_end_date=me,
|
||||
reporting_month=me.strftime("%Y-%m") if me else None,
|
||||
reporting_month=month,
|
||||
reporting_currency=body.reporting_currency,
|
||||
clearing_lag_days=body.clearing_lag_days,
|
||||
rounding_tolerance=body.rounding_tolerance,
|
||||
|
|
@ -77,6 +120,9 @@ def update_session(session_id: int, body: SessionUpdate,
|
|||
db: OrmSession = Depends(db_dep)) -> dict:
|
||||
s = get_session_or_404(session_id, db)
|
||||
data = body.model_dump(exclude_unset=True)
|
||||
if s.status == "completed" and set(data) - {"name"}:
|
||||
raise HTTPException(409, "This closing is completed and locked — only the name can "
|
||||
"be changed. Reopen it first for anything else.")
|
||||
for k, v in data.items():
|
||||
setattr(s, k, v)
|
||||
if "month_end_date" in data and s.month_end_date:
|
||||
|
|
@ -85,6 +131,20 @@ def update_session(session_id: int, body: SessionUpdate,
|
|||
return session_dict(s)
|
||||
|
||||
|
||||
@router.post("/{session_id}/reopen")
|
||||
def reopen_session(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Unlock a completed closing for corrections. Deliberate and logged — the opposite of
|
||||
silently editing published history."""
|
||||
s = get_session_or_404(session_id, db)
|
||||
if s.status != "completed":
|
||||
raise HTTPException(409, "Only a completed closing can be reopened.")
|
||||
s.status = "blocked" if s.blocked_reason else "processed"
|
||||
db.commit()
|
||||
logger.warning("closing %s (%s, %s) reopened for corrections",
|
||||
s.id, s.name, s.reporting_month or "no month")
|
||||
return session_dict(s)
|
||||
|
||||
|
||||
@router.delete("/{session_id}")
|
||||
def delete_session(session_id: int, db: OrmSession = Depends(db_dep)) -> dict:
|
||||
"""Delete a closing and every row/file that belongs to it."""
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ from sqlalchemy.orm import Session as OrmSession
|
|||
|
||||
from ...core.column_map import FIELD_ORDER, normalize_header
|
||||
from ...db import models
|
||||
from ..deps import db_dep, get_session_or_404, to_dict
|
||||
from ..deps import db_dep, ensure_editable, get_session_or_404, to_dict
|
||||
|
||||
router = APIRouter(prefix="/api/sessions", tags=["settings"])
|
||||
rules_router = APIRouter(prefix="/api/mapping-rules", tags=["mapping"])
|
||||
|
|
@ -91,7 +91,7 @@ def get_reserves(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict
|
|||
@router.put("/{session_id}/reserves")
|
||||
def put_reserves(session_id: int, items: list[ReserveIn],
|
||||
db: OrmSession = Depends(db_dep)) -> list[dict]:
|
||||
get_session_or_404(session_id, db)
|
||||
ensure_editable(get_session_or_404(session_id, db))
|
||||
db.query(models.Reserve).filter(models.Reserve.session_id == session_id).delete()
|
||||
for it in items:
|
||||
db.add(models.Reserve(session_id=session_id, marketplace=it.marketplace,
|
||||
|
|
@ -110,6 +110,7 @@ def get_fx(session_id: int, db: OrmSession = Depends(db_dep)) -> list[dict]:
|
|||
@router.put("/{session_id}/fx")
|
||||
def put_fx(session_id: int, items: list[FxIn], db: OrmSession = Depends(db_dep)) -> list[dict]:
|
||||
s = get_session_or_404(session_id, db)
|
||||
ensure_editable(s)
|
||||
# Upsert per marketplace — NOT delete-all-then-insert. This used to wipe every rate not
|
||||
# named in the body, so a partial PUT silently removed the other marketplaces' rates and
|
||||
# the close fell back to the hardcoded Jan-26 defaults without a word.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,9 @@
|
|||
"""Application configuration (env-overridable). No third-party data egress."""
|
||||
"""Application configuration (env-overridable).
|
||||
|
||||
Data egress: none, with ONE deliberate exception — the exchange-rate fetch
|
||||
(services/fx_service.py) calls the configured FX provider (Frankfurter by default) with
|
||||
currency codes and dates only. No financial figures, filenames, or transaction data ever
|
||||
leave the server."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
|
@ -51,7 +56,7 @@ def mysql_url() -> str:
|
|||
if not _mysql_configured():
|
||||
raise RuntimeError(
|
||||
"MYSQL_HOST, MYSQL_USER, and MYSQL_DATABASE are required for the mysql "
|
||||
"backend. Copy example.env to .env and fill in real credentials, or set "
|
||||
"backend. Copy .env.example to .env and fill in real credentials, or set "
|
||||
"AR_DB_BACKEND=sqlite to use a local file."
|
||||
)
|
||||
user = quote_plus(MYSQL_USER)
|
||||
|
|
@ -92,6 +97,33 @@ CORS_ORIGINS = os.environ.get(
|
|||
"AR_CORS_ORIGINS", "http://localhost:5173,http://127.0.0.1:5173"
|
||||
).split(",")
|
||||
|
||||
# --------------------------------------------------------------------------- auth
|
||||
# AR_AUTH: on | off | auto (default).
|
||||
# auto — login is required as soon as at least one user exists (create users with
|
||||
# `python manage.py add-user`), and the API is open while there are none.
|
||||
# A fresh dev checkout and the test suite therefore run without ceremony,
|
||||
# while creating the first real user turns authentication on by itself.
|
||||
# on — login is always required (production; set it in .env.production).
|
||||
# off — never required (explicit opt-out; never use on a reachable server).
|
||||
AUTH_MODE = os.environ.get("AR_AUTH", "auto").strip().lower()
|
||||
if AUTH_MODE not in ("on", "off", "auto"):
|
||||
raise RuntimeError(f"AR_AUTH must be 'on', 'off' or 'auto' (got {AUTH_MODE!r}).")
|
||||
|
||||
# Signs login tokens. REQUIRED in production — without it a random ephemeral key is used
|
||||
# and every restart logs everyone out (fine for a laptop, wrong for a server).
|
||||
SECRET_KEY = os.environ.get("AR_SECRET_KEY", "")
|
||||
|
||||
# Token lifetime (hours).
|
||||
AUTH_TOKEN_HOURS = int(os.environ.get("AR_AUTH_TOKEN_HOURS", "12"))
|
||||
|
||||
# --------------------------------------------------------------------------- FX provider
|
||||
# frankfurter (default; free, keyless, central-bank rates) | exchangerate-api (paid, needs
|
||||
# FX_API_KEY). Rates fetched are suggestions: Control C5 still requires a human to confirm
|
||||
# them for the reporting month before the close can publish.
|
||||
FX_PROVIDER = os.environ.get("AR_FX_PROVIDER", "frankfurter").strip().lower()
|
||||
FX_API_KEY = os.environ.get("AR_FX_API_KEY", "")
|
||||
FX_TIMEOUT_S = float(os.environ.get("AR_FX_TIMEOUT_S", "15"))
|
||||
|
||||
|
||||
def ensure_dirs() -> None:
|
||||
for d in (DATA_DIR, UPLOAD_DIR, EXPORT_DIR):
|
||||
|
|
|
|||
|
|
@ -15,6 +15,19 @@ def _now() -> dt.datetime:
|
|||
return dt.datetime.utcnow()
|
||||
|
||||
|
||||
class User(Base):
|
||||
"""A named person who can sign in. Created via `python manage.py add-user` — there is no
|
||||
self-signup. The display name is what lands in reviewed_by / approved_by / confirmed_by,
|
||||
so accountability fields carry a verified identity instead of free text."""
|
||||
__tablename__ = "users"
|
||||
id = Column(Integer, primary_key=True)
|
||||
username = Column(String(64), unique=True, nullable=False)
|
||||
display_name = Column(String(255), nullable=False)
|
||||
password_hash = Column(String(512), nullable=False)
|
||||
is_active = Column(Boolean, default=True)
|
||||
created_at = Column(DateTime, default=_now)
|
||||
|
||||
|
||||
class Session(Base):
|
||||
__tablename__ = "sessions"
|
||||
id = Column(Integer, primary_key=True)
|
||||
|
|
@ -254,6 +267,24 @@ class ReconciliationRow(Base):
|
|||
all_payouts = Column(Float, default=0.0) # Σ all transfers (negative)
|
||||
|
||||
|
||||
class FxProviderRate(Base):
|
||||
"""Cache of rates fetched from the FX provider (Frankfurter by default).
|
||||
|
||||
One row per (provider, date, currency); `rate` is USD per 1 unit of local currency —
|
||||
the same orientation as FxRate.rate, i.e. usd = local * rate. Caching makes a re-fetch
|
||||
idempotent and keeps month-end seeding working offline once fetched."""
|
||||
__tablename__ = "fx_provider_rates"
|
||||
id = Column(Integer, primary_key=True)
|
||||
provider = Column(String(32), nullable=False, default="frankfurter")
|
||||
rate_date = Column(Date, nullable=False)
|
||||
currency = Column(String(16), nullable=False)
|
||||
rate = Column(Float, nullable=False)
|
||||
fetched_at = Column(DateTime, default=_now)
|
||||
__table_args__ = (
|
||||
Index("ix_fx_provider_key", "provider", "rate_date", "currency", unique=True),
|
||||
)
|
||||
|
||||
|
||||
class FxRateDaily(Base):
|
||||
"""Optional per-date FX override. Falls back to the marketplace's month rate."""
|
||||
__tablename__ = "fx_rates_daily"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,308 @@
|
|||
"""
|
||||
Exchange-rate fetching (the ONE deliberate network egress in the app — currency codes and
|
||||
dates only, never financial data).
|
||||
|
||||
Providers
|
||||
frankfurter (default) free, keyless, central-bank (ECB) reference rates, historical
|
||||
dates supported. Weekend/holiday dates snap to the previous
|
||||
banking day — exactly the month-end convention Finance uses.
|
||||
exchangerate-api paid fallback (AR_FX_PROVIDER=exchangerate-api + AR_FX_API_KEY).
|
||||
|
||||
ORIENTATION — the #1 way to corrupt every non-USD receivable:
|
||||
The app stores USD per 1 unit of LOCAL currency (usd = local * FxRate.rate; see
|
||||
core/money.to_usd and store.py). Providers return the opposite (local per 1 USD when
|
||||
base=USD), so every provider here INVERTS before returning. test_fx_service.py pins
|
||||
this with a known EUR fixture.
|
||||
|
||||
Fetched rates are SUGGESTIONS: seeding writes them unconfirmed, so Control C5 still blocks
|
||||
the close until a person reviews and confirms them for the reporting month — identical to
|
||||
the manual-entry workflow, just pre-filled with a real rate instead of the Jan-26 snapshot.
|
||||
|
||||
Failure policy: a provider error raises FxProviderError (the route answers 502 "enter rates
|
||||
manually"). DEFAULT_FX_USD is never written silently — the existing merge in jobs.py is
|
||||
already the fallback and C5 already flags unconfirmed defaults.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
import json
|
||||
import logging
|
||||
import ssl
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from sqlalchemy.orm import Session as OrmSession
|
||||
|
||||
from ..config import FX_API_KEY, FX_PROVIDER, FX_TIMEOUT_S
|
||||
from ..core.i18n import currency_for_region
|
||||
from ..db import models
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FxProviderError(RuntimeError):
|
||||
"""The provider could not supply rates (network, quota, unknown currency...)."""
|
||||
|
||||
|
||||
def _ssl_context() -> ssl.SSLContext | None:
|
||||
"""Prefer certifi's CA bundle: on some Windows machines loading the OS certificate
|
||||
store fails outright (ssl [ASN1: NOT_ENOUGH_DATA]), which would break every fetch.
|
||||
Fall back to the default context when certifi isn't installed (Linux containers)."""
|
||||
try:
|
||||
import certifi
|
||||
return ssl.create_default_context(cafile=certifi.where())
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
|
||||
def _http_get_json(url: str) -> dict:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "ar-aging-app/1.0"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=FX_TIMEOUT_S,
|
||||
context=_ssl_context()) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as e:
|
||||
raise FxProviderError(f"FX provider answered HTTP {e.code} for {url.split('?')[0]}") from e
|
||||
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError, OSError, ssl.SSLError) as e:
|
||||
raise FxProviderError(f"Could not reach the FX provider: {e}") from e
|
||||
|
||||
|
||||
class FrankfurterProvider:
|
||||
"""https://frankfurter.dev — GET /v1/{date}?base=USD&symbols=EUR,GBP,..."""
|
||||
|
||||
name = "frankfurter"
|
||||
_BASE = "https://api.frankfurter.dev/v1"
|
||||
|
||||
def rates_on(self, on: dt.date, currencies: set[str]) -> tuple[dict[str, float], dt.date]:
|
||||
"""{currency: USD-per-local}, plus the banking day the provider actually used."""
|
||||
symbols = sorted(c for c in currencies if c and c != "USD")
|
||||
if not symbols:
|
||||
return {}, on
|
||||
url = (f"{self._BASE}/{on.isoformat()}"
|
||||
f"?base=USD&symbols={urllib.parse.quote(','.join(symbols))}")
|
||||
data = _http_get_json(url)
|
||||
raw = data.get("rates") or {}
|
||||
# base=USD → provider returns LOCAL per USD; the app stores USD per LOCAL. Invert.
|
||||
out = {ccy: 1.0 / v for ccy, v in raw.items() if v}
|
||||
actual = dt.date.fromisoformat(data["date"]) if data.get("date") else on
|
||||
return out, actual
|
||||
|
||||
def rates_series(self, date_from: dt.date, date_to: dt.date,
|
||||
currencies: set[str]) -> dict[dt.date, dict[str, float]]:
|
||||
"""{date: {currency: USD-per-local}} for every banking day in the range."""
|
||||
symbols = sorted(c for c in currencies if c and c != "USD")
|
||||
if not symbols:
|
||||
return {}
|
||||
url = (f"{self._BASE}/{date_from.isoformat()}..{date_to.isoformat()}"
|
||||
f"?base=USD&symbols={urllib.parse.quote(','.join(symbols))}")
|
||||
data = _http_get_json(url)
|
||||
out: dict[dt.date, dict[str, float]] = {}
|
||||
for day, raw in (data.get("rates") or {}).items():
|
||||
out[dt.date.fromisoformat(day)] = {c: 1.0 / v for c, v in raw.items() if v}
|
||||
return out
|
||||
|
||||
|
||||
class ExchangeRateApiProvider:
|
||||
"""https://www.exchangerate-api.com — paid fallback. Needs AR_FX_API_KEY."""
|
||||
|
||||
name = "exchangerate-api"
|
||||
_BASE = "https://v6.exchangerate-api.com/v6"
|
||||
|
||||
def __init__(self) -> None:
|
||||
if not FX_API_KEY:
|
||||
raise FxProviderError(
|
||||
"AR_FX_PROVIDER=exchangerate-api requires AR_FX_API_KEY.")
|
||||
|
||||
def rates_on(self, on: dt.date, currencies: set[str]) -> tuple[dict[str, float], dt.date]:
|
||||
symbols = {c for c in currencies if c and c != "USD"}
|
||||
if not symbols:
|
||||
return {}, on
|
||||
# History endpoint (paid plans); falls back to latest when the date is today.
|
||||
if on >= dt.date.today():
|
||||
url = f"{self._BASE}/{FX_API_KEY}/latest/USD"
|
||||
else:
|
||||
url = f"{self._BASE}/{FX_API_KEY}/history/USD/{on.year}/{on.month}/{on.day}"
|
||||
data = _http_get_json(url)
|
||||
if data.get("result") != "success":
|
||||
raise FxProviderError(f"exchangerate-api: {data.get('error-type', 'error')}")
|
||||
raw = data.get("conversion_rates") or {}
|
||||
return {c: 1.0 / raw[c] for c in symbols if raw.get(c)}, on
|
||||
|
||||
def rates_series(self, date_from: dt.date, date_to: dt.date,
|
||||
currencies: set[str]) -> dict[dt.date, dict[str, float]]:
|
||||
out: dict[dt.date, dict[str, float]] = {}
|
||||
day = date_from
|
||||
while day <= date_to:
|
||||
try:
|
||||
rates, actual = self.rates_on(day, currencies)
|
||||
out[actual] = rates
|
||||
except FxProviderError:
|
||||
pass # weekends/holidays have no fixing
|
||||
day += dt.timedelta(days=1)
|
||||
return out
|
||||
|
||||
|
||||
def get_provider():
|
||||
if FX_PROVIDER == "exchangerate-api":
|
||||
return ExchangeRateApiProvider()
|
||||
if FX_PROVIDER == "frankfurter":
|
||||
return FrankfurterProvider()
|
||||
raise FxProviderError(f"Unknown AR_FX_PROVIDER {FX_PROVIDER!r} "
|
||||
f"(use 'frankfurter' or 'exchangerate-api').")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- caching
|
||||
def _cached_rates(db: OrmSession, provider_name: str, on: dt.date,
|
||||
currencies: set[str]) -> dict[str, float] | None:
|
||||
"""All requested currencies from the cache, or None on any miss."""
|
||||
want = {c for c in currencies if c != "USD"}
|
||||
if not want:
|
||||
return {}
|
||||
rows = db.query(models.FxProviderRate).filter(
|
||||
models.FxProviderRate.provider == provider_name,
|
||||
models.FxProviderRate.rate_date == on,
|
||||
models.FxProviderRate.currency.in_(want)).all()
|
||||
got = {r.currency: r.rate for r in rows}
|
||||
return got if set(got) >= want else None
|
||||
|
||||
|
||||
def _cache_rates(db: OrmSession, provider_name: str, on: dt.date,
|
||||
rates: dict[str, float]) -> None:
|
||||
existing = {r.currency for r in db.query(models.FxProviderRate).filter(
|
||||
models.FxProviderRate.provider == provider_name,
|
||||
models.FxProviderRate.rate_date == on)}
|
||||
for ccy, rate in rates.items():
|
||||
if ccy not in existing:
|
||||
db.add(models.FxProviderRate(provider=provider_name, rate_date=on,
|
||||
currency=ccy, rate=rate))
|
||||
db.commit()
|
||||
|
||||
|
||||
def rates_for_date(db: OrmSession, on: dt.date,
|
||||
currencies: set[str]) -> tuple[dict[str, float], str]:
|
||||
"""{currency: USD-per-local} for a date — cache first, provider on miss.
|
||||
|
||||
Returns (rates, source_label). The label names the provider and the banking day the
|
||||
rates are actually for, so an FxRate row's `source` explains itself."""
|
||||
provider = get_provider()
|
||||
cached = _cached_rates(db, provider.name, on, currencies)
|
||||
if cached is not None:
|
||||
return cached, f"{provider.name} {on.isoformat()} (cached)"
|
||||
rates, actual = provider.rates_on(on, currencies)
|
||||
# Cache under both the requested date and the provider's actual banking day, so a
|
||||
# weekend month-end (snapped to Friday) is served from cache next time as well.
|
||||
_cache_rates(db, provider.name, actual, rates)
|
||||
if actual != on:
|
||||
_cache_rates(db, provider.name, on, rates)
|
||||
return rates, f"{provider.name} {actual.isoformat()}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- seeding
|
||||
def _session_fx_targets(db: OrmSession, session: models.Session) -> list[models.FxRate]:
|
||||
"""The session's existing FX rows — the marketplaces this close actually involves.
|
||||
|
||||
Rows are created during processing for every marketplace that appears in the files
|
||||
(jobs.py), so 'process first' is the natural precondition; seeding rates for
|
||||
marketplaces the close doesn't contain would only widen what C5 asks Finance to
|
||||
confirm."""
|
||||
return db.query(models.FxRate).filter(
|
||||
models.FxRate.session_id == session.id).all()
|
||||
|
||||
|
||||
def seed_session_fx(db: OrmSession, session: models.Session) -> dict:
|
||||
"""Fetch month-end rates and pre-fill the session's FX table (UNCONFIRMED).
|
||||
|
||||
Existing confirmations are cleared — same withdrawal semantics as editing a rate by
|
||||
hand (settings.put_fx): a confirmation attests to a specific number."""
|
||||
if session.month_end_date is None:
|
||||
raise FxProviderError("Set the month-end date first.")
|
||||
rows = _session_fx_targets(db, session)
|
||||
if not rows:
|
||||
raise FxProviderError(
|
||||
"No FX rows exist yet for this closing — process it first so its "
|
||||
"marketplaces are known.")
|
||||
|
||||
currencies = {(r.currency or currency_for_region(r.marketplace)) for r in rows}
|
||||
fetched, source = rates_for_date(db, session.month_end_date, currencies)
|
||||
|
||||
updated, missing = [], []
|
||||
for r in rows:
|
||||
ccy = r.currency or currency_for_region(r.marketplace)
|
||||
if ccy == "USD":
|
||||
new_rate = 1.0
|
||||
elif ccy in fetched:
|
||||
new_rate = round(fetched[ccy], 6)
|
||||
else:
|
||||
missing.append(f"{r.marketplace} ({ccy})")
|
||||
continue
|
||||
r.rate = new_rate
|
||||
r.currency = ccy
|
||||
r.rate_date = session.month_end_date
|
||||
r.source = source
|
||||
# A fetched rate is a suggestion — it must be confirmed for THIS month (C5).
|
||||
r.confirmed_by = ""
|
||||
r.confirmed_at = None
|
||||
r.confirmed_month = ""
|
||||
updated.append({"marketplace": r.marketplace, "currency": ccy, "rate": new_rate})
|
||||
db.commit()
|
||||
|
||||
if session.status in ("processed", "blocked", "completed"):
|
||||
from .controls_run import run_and_persist
|
||||
run_and_persist(db, session.id)
|
||||
|
||||
logger.info("fx seed: session %s, %d rate(s) from %s, %d missing",
|
||||
session.id, len(updated), source, len(missing))
|
||||
return {"updated": updated, "missing": missing, "source": source,
|
||||
"rate_date": session.month_end_date.isoformat()}
|
||||
|
||||
|
||||
def seed_daily_fx(db: OrmSession, session: models.Session, marketplace: str | None = None,
|
||||
date_from: dt.date | None = None, date_to: dt.date | None = None) -> dict:
|
||||
"""Fill fx_rates_daily from the provider for a date range (defaults: the whole month).
|
||||
|
||||
Daily rows are optional per-date OVERRIDES of the month rate (analytics fx-daily),
|
||||
marked source=provider so hand-entered rows are distinguishable."""
|
||||
if session.month_end_date is None:
|
||||
raise FxProviderError("Set the month-end date first.")
|
||||
date_to = date_to or session.month_end_date
|
||||
date_from = date_from or session.month_end_date.replace(day=1)
|
||||
if date_from > date_to:
|
||||
raise FxProviderError("date_from is after date_to.")
|
||||
|
||||
rows = _session_fx_targets(db, session)
|
||||
targets = [(r.marketplace, r.currency or currency_for_region(r.marketplace))
|
||||
for r in rows
|
||||
if (marketplace is None or r.marketplace == marketplace)]
|
||||
targets = [(m, c) for m, c in targets if c != "USD"]
|
||||
if not targets:
|
||||
raise FxProviderError(
|
||||
"No non-USD marketplace to fetch daily rates for — process the closing "
|
||||
"first (or this closing is USD-only).")
|
||||
|
||||
provider = get_provider()
|
||||
series = provider.rates_series(date_from, date_to, {c for _, c in targets})
|
||||
|
||||
existing = {(r.marketplace, r.rate_date): r for r in db.query(models.FxRateDaily).filter(
|
||||
models.FxRateDaily.session_id == session.id)}
|
||||
saved = 0
|
||||
for day, per_ccy in sorted(series.items()):
|
||||
for mkt, ccy in targets:
|
||||
rate = per_ccy.get(ccy)
|
||||
if not rate:
|
||||
continue
|
||||
row = existing.get((mkt, day))
|
||||
if row is None:
|
||||
row = models.FxRateDaily(session_id=session.id, marketplace=mkt,
|
||||
rate_date=day)
|
||||
db.add(row)
|
||||
existing[(mkt, day)] = row
|
||||
row.rate = round(rate, 6)
|
||||
row.source = provider.name
|
||||
saved += 1
|
||||
db.commit()
|
||||
logger.info("fx daily seed: session %s, %d row(s) %s..%s",
|
||||
session.id, saved, date_from, date_to)
|
||||
return {"saved": saved, "date_from": date_from.isoformat(),
|
||||
"date_to": date_to.isoformat(), "provider": provider.name,
|
||||
"marketplaces": sorted({m for m, _ in targets})}
|
||||
|
|
@ -1,6 +1,7 @@
|
|||
"""Background processing job: run the engine over a session's files and persist results."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
import traceback
|
||||
|
||||
|
|
@ -10,6 +11,33 @@ from ..db import models
|
|||
from ..db.database import SessionLocal
|
||||
from .store import TransactionSink, persist_aggregates, clear_session_results
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def recover_stale_jobs() -> None:
|
||||
"""Called once at startup. Jobs run in-process (single worker), so a session still
|
||||
marked processing/exporting at boot was killed mid-run by a restart or deploy — without
|
||||
this it stays stuck forever and the 409 "already processing" guard blocks every re-run."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
stale = db.query(models.Session).filter(
|
||||
models.Session.status.in_(("processing", "exporting"))).all()
|
||||
for s in stale:
|
||||
was = s.status
|
||||
s.status = "error" if was == "processing" else "processed"
|
||||
s.error = ("Interrupted by a server restart before it finished — run it again."
|
||||
if was == "processing"
|
||||
else "Export was interrupted by a server restart — export again.")
|
||||
s.progress_stage = "Interrupted"
|
||||
logger.warning("recovered stale job: session %s (%s) was '%s'", s.id, s.name, was)
|
||||
if stale:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — recovery must never prevent startup
|
||||
db.rollback()
|
||||
logger.exception("stale-job recovery failed")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def load_mapping_rules(db) -> dict[str, str]:
|
||||
"""Admin-saved header rules (normalized header -> canonical field)."""
|
||||
|
|
|
|||
|
|
@ -0,0 +1,62 @@
|
|||
"""
|
||||
Retention: purge generated export workbooks older than AR_RETENTION_DAYS.
|
||||
|
||||
EXPORTS ONLY, by design. Uploaded source files are never auto-deleted — they are the audit
|
||||
source for every published figure, and re-generating the full workbook re-parses them.
|
||||
Export files are pure derivatives: anything purged can be regenerated with one click, and
|
||||
the DB row is kept so the Exports list still shows what was generated and when
|
||||
(`available: false` once the file is gone).
|
||||
|
||||
AR_RETENTION_DAYS=0 keeps everything forever.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import datetime as dt
|
||||
import logging
|
||||
import os
|
||||
|
||||
from ..config import RETENTION_DAYS
|
||||
from ..db import models
|
||||
from ..db.database import SessionLocal
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_SWEEP_INTERVAL_S = 24 * 3600
|
||||
|
||||
|
||||
def purge_old_exports(retention_days: int | None = None) -> int:
|
||||
"""Delete export files older than the retention window. Returns files removed."""
|
||||
days = RETENTION_DAYS if retention_days is None else retention_days
|
||||
if days <= 0:
|
||||
return 0
|
||||
cutoff = dt.datetime.utcnow() - dt.timedelta(days=days)
|
||||
removed = 0
|
||||
db = SessionLocal()
|
||||
try:
|
||||
rows = db.query(models.ExportRecord).filter(
|
||||
models.ExportRecord.generated_at < cutoff).all()
|
||||
for r in rows:
|
||||
if not r.path:
|
||||
continue
|
||||
try:
|
||||
if os.path.exists(r.path):
|
||||
os.remove(r.path)
|
||||
removed += 1
|
||||
except OSError:
|
||||
logger.warning("retention: could not remove %s", r.path)
|
||||
if removed:
|
||||
logger.info("retention: removed %d export file(s) older than %d days",
|
||||
removed, days)
|
||||
except Exception: # noqa: BLE001 — a failed sweep must never take the app down
|
||||
logger.exception("retention sweep failed")
|
||||
finally:
|
||||
db.close()
|
||||
return removed
|
||||
|
||||
|
||||
async def retention_loop() -> None:
|
||||
"""Daily sweep, started from the app lifespan. Cancelled cleanly on shutdown."""
|
||||
while True:
|
||||
await asyncio.to_thread(purge_old_exports)
|
||||
await asyncio.sleep(_SWEEP_INTERVAL_S)
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
"""
|
||||
Admin commands (run on the server, next to the app):
|
||||
|
||||
python manage.py add-user <username> --name "Display Name" # prompts for password
|
||||
python manage.py set-password <username> # prompts for password
|
||||
python manage.py list-users
|
||||
python manage.py deactivate-user <username>
|
||||
python manage.py dedupe-files [--apply] # fix double-counted uploads
|
||||
|
||||
There is deliberately no self-signup: the 5-or-so finance users are created here.
|
||||
`dedupe-files` is the one-time cleanup for the historical upload bug where re-uploading a
|
||||
file created a second session_files row pointing at the same stored file — which made the
|
||||
pipeline parse and sum that file twice. Run it once after deploying the fix; affected
|
||||
closings are flagged needs_reprocess so the corrected totals are one click away.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import getpass
|
||||
import sys
|
||||
|
||||
from app.db.database import SessionLocal, init_db
|
||||
from app.db import models
|
||||
|
||||
|
||||
def _prompt_password() -> str:
|
||||
pw = getpass.getpass("Password: ")
|
||||
if len(pw) < 8:
|
||||
sys.exit("Password must be at least 8 characters.")
|
||||
if pw != getpass.getpass("Repeat password: "):
|
||||
sys.exit("Passwords do not match.")
|
||||
return pw
|
||||
|
||||
|
||||
def cmd_add_user(args) -> int:
|
||||
from app.api.auth import hash_password
|
||||
db = SessionLocal()
|
||||
try:
|
||||
username = args.username.strip().lower()
|
||||
if db.query(models.User).filter(models.User.username == username).first():
|
||||
print(f"User '{username}' already exists — use set-password to change it.")
|
||||
return 1
|
||||
pw = _prompt_password()
|
||||
db.add(models.User(username=username,
|
||||
display_name=(args.name or username).strip(),
|
||||
password_hash=hash_password(pw), is_active=True))
|
||||
db.commit()
|
||||
print(f"Created user '{username}' ({args.name or username}). "
|
||||
f"Login is now required (AR_AUTH=auto turns on with the first user).")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def cmd_set_password(args) -> int:
|
||||
from app.api.auth import hash_password
|
||||
db = SessionLocal()
|
||||
try:
|
||||
user = db.query(models.User).filter(
|
||||
models.User.username == args.username.strip().lower()).first()
|
||||
if user is None:
|
||||
print(f"No user '{args.username}'.")
|
||||
return 1
|
||||
user.password_hash = hash_password(_prompt_password())
|
||||
user.is_active = True
|
||||
db.commit()
|
||||
print(f"Password updated for '{user.username}'.")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def cmd_list_users(_args) -> int:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
rows = db.query(models.User).order_by(models.User.username).all()
|
||||
if not rows:
|
||||
print("No users yet — the API is open until the first one is created "
|
||||
"(AR_AUTH=auto).")
|
||||
return 0
|
||||
for u in rows:
|
||||
flag = "" if u.is_active else " [DEACTIVATED]"
|
||||
print(f" {u.username:<20} {u.display_name}{flag}")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def cmd_deactivate_user(args) -> int:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
user = db.query(models.User).filter(
|
||||
models.User.username == args.username.strip().lower()).first()
|
||||
if user is None:
|
||||
print(f"No user '{args.username}'.")
|
||||
return 1
|
||||
user.is_active = False
|
||||
db.commit()
|
||||
print(f"Deactivated '{user.username}' — existing tokens stop working within "
|
||||
f"their normal expiry; new logins are refused immediately.")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def cmd_dedupe_files(args) -> int:
|
||||
"""Collapse session_files rows that point at the same stored file (or share a name)
|
||||
within one closing. Keeps the NEWEST row (it matches the bytes on disk — uploads
|
||||
overwrote the file), deletes the rest, flags the closing for re-processing."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
rows = db.query(models.SessionFile).order_by(
|
||||
models.SessionFile.session_id, models.SessionFile.id).all()
|
||||
by_key: dict[tuple, list[models.SessionFile]] = {}
|
||||
for f in rows:
|
||||
by_key.setdefault((f.session_id, f.stored_path or f.filename), []).append(f)
|
||||
dupes = {k: v for k, v in by_key.items() if len(v) > 1}
|
||||
if not dupes:
|
||||
print("No duplicated file rows found — nothing to do.")
|
||||
return 0
|
||||
|
||||
affected_sessions: set[int] = set()
|
||||
removed = 0
|
||||
for (session_id, path), group in sorted(dupes.items()):
|
||||
keep = group[-1] # newest row matches the bytes on disk
|
||||
print(f"session {session_id}: '{keep.filename}' has {len(group)} rows "
|
||||
f"-> keeping id {keep.id}, "
|
||||
f"removing {[g.id for g in group if g.id != keep.id]}")
|
||||
for g in group:
|
||||
if g.id == keep.id:
|
||||
continue
|
||||
if args.apply:
|
||||
db.delete(g)
|
||||
removed += 1
|
||||
affected_sessions.add(session_id)
|
||||
|
||||
if not args.apply:
|
||||
print(f"\nDRY RUN: would remove {removed} duplicate row(s) across "
|
||||
f"{len(affected_sessions)} closing(s). Re-run with --apply to fix.")
|
||||
return 0
|
||||
|
||||
for sid in affected_sessions:
|
||||
s = db.get(models.Session, sid)
|
||||
if s is not None and s.status in ("processed", "blocked", "completed"):
|
||||
s.needs_reprocess = True
|
||||
db.commit()
|
||||
print(f"\nRemoved {removed} duplicate row(s). {len(affected_sessions)} closing(s) "
|
||||
f"flagged needs_reprocess — re-process them so the corrected totals persist.")
|
||||
return 0
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
ap = argparse.ArgumentParser(description="AR Aging admin commands")
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
p = sub.add_parser("add-user", help="create a user (prompts for password)")
|
||||
p.add_argument("username")
|
||||
p.add_argument("--name", default="", help="display name (lands in sign-off fields)")
|
||||
p.set_defaults(fn=cmd_add_user)
|
||||
|
||||
p = sub.add_parser("set-password", help="reset a user's password")
|
||||
p.add_argument("username")
|
||||
p.set_defaults(fn=cmd_set_password)
|
||||
|
||||
p = sub.add_parser("list-users", help="list users")
|
||||
p.set_defaults(fn=cmd_list_users)
|
||||
|
||||
p = sub.add_parser("deactivate-user", help="disable a user's login")
|
||||
p.add_argument("username")
|
||||
p.set_defaults(fn=cmd_deactivate_user)
|
||||
|
||||
p = sub.add_parser("dedupe-files", help="fix double-counted duplicate upload rows")
|
||||
p.add_argument("--apply", action="store_true", help="actually delete (default: dry run)")
|
||||
p.set_defaults(fn=cmd_dedupe_files)
|
||||
|
||||
args = ap.parse_args(argv)
|
||||
init_db()
|
||||
return args.fn(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv[1:]))
|
||||
|
|
@ -319,7 +319,8 @@ def migrate() -> int:
|
|||
|
||||
log("")
|
||||
if ok:
|
||||
log(" Migration complete. Start the dashboard with start.command.")
|
||||
log(" Migration complete. Start the dashboard (scripts/start.ps1 on Windows, "
|
||||
"scripts/start.command on macOS, or docker compose in production).")
|
||||
return 0
|
||||
log(" Migration finished with MISMATCHES — do not rely on the MySQL data until")
|
||||
log(" they are explained. The SQLite file is untouched.")
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ SQLAlchemy==2.0.36
|
|||
PyMySQL==1.1.1
|
||||
cryptography>=42.0.0
|
||||
python-dotenv==1.0.1
|
||||
# CA bundle for the FX-rate fetch — the Windows OS cert store is unreliable on some machines
|
||||
certifi>=2024.2.2
|
||||
|
||||
# Data helpers (optional / analysis)
|
||||
pandas>=2.2
|
||||
|
|
|
|||
|
|
@ -24,12 +24,14 @@ def test_full_api_flow():
|
|||
|
||||
sid = c.post("/api/sessions", json={
|
||||
"name": "test", "month_end_date": "2026-01-31", "clearing_lag_days": 2,
|
||||
"allow_duplicate": True, # suite shares one DB; the guard has its own test
|
||||
}).json()["id"]
|
||||
|
||||
with open(synth, "rb") as fh:
|
||||
up = c.post(f"/api/sessions/{sid}/files", files={"files": ("USA synthetic.xlsx", fh)})
|
||||
assert up.status_code == 200
|
||||
assert up.json()[0]["status"] == "parsed"
|
||||
assert up.json()["files"][0]["status"] == "parsed"
|
||||
assert up.json()["skipped"] == []
|
||||
|
||||
c.put(f"/api/sessions/{sid}/reserves",
|
||||
json=[{"marketplace": "USA", "account_type": "Standard Orders", "amount": 0.0}])
|
||||
|
|
@ -67,7 +69,8 @@ def test_invalid_file_rejected():
|
|||
with open(bad, "w") as f:
|
||||
f.write("not a spreadsheet")
|
||||
with TestClient(app) as c:
|
||||
sid = c.post("/api/sessions", json={"name": "bad", "month_end_date": "2026-01-31"}).json()["id"]
|
||||
sid = c.post("/api/sessions", json={"name": "bad", "month_end_date": "2026-01-31",
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
with open(bad, "rb") as fh:
|
||||
r = c.post(f"/api/sessions/{sid}/files", files={"files": ("bad.txt", fh)})
|
||||
assert r.status_code == 400 # unsupported extension
|
||||
|
|
|
|||
|
|
@ -0,0 +1,125 @@
|
|||
"""Login: AR_AUTH=auto turns on with the first user; identity feeds sign-off fields."""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api import auth as auth_mod
|
||||
from app.api.main import app
|
||||
from app.db import models
|
||||
from app.db.database import SessionLocal, init_db
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def clean_users():
|
||||
"""Users flip AR_AUTH=auto to 'required' for the WHOLE shared test DB — always remove
|
||||
them again so the rest of the suite keeps running unauthenticated."""
|
||||
init_db()
|
||||
yield
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.User).delete()
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
auth_mod.invalidate_users_cache()
|
||||
|
||||
|
||||
def _add_user(username: str, display: str, password: str) -> None:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.add(models.User(username=username, display_name=display,
|
||||
password_hash=auth_mod.hash_password(password), is_active=True))
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
auth_mod.invalidate_users_cache()
|
||||
|
||||
|
||||
def test_password_hash_roundtrip():
|
||||
h = auth_mod.hash_password("s3cret-pw!")
|
||||
assert h.startswith("scrypt$")
|
||||
assert auth_mod.verify_password("s3cret-pw!", h)
|
||||
assert not auth_mod.verify_password("wrong", h)
|
||||
assert not auth_mod.verify_password("s3cret-pw!", "garbage")
|
||||
|
||||
|
||||
def test_token_roundtrip_and_tamper():
|
||||
user = models.User(id=7, username="jane", display_name="Jane D",
|
||||
password_hash="x", is_active=True)
|
||||
tok = auth_mod.create_token(user)
|
||||
data = auth_mod.parse_token(tok)
|
||||
assert data and data["uid"] == 7 and data["dn"] == "Jane D"
|
||||
payload, sig = tok.split(".")
|
||||
assert auth_mod.parse_token(f"{payload}x.{sig}") is None # tampered payload
|
||||
assert auth_mod.parse_token(f"{payload}.{sig[:-2]}aa") is None # tampered signature
|
||||
|
||||
|
||||
def test_api_open_with_no_users(clean_users):
|
||||
auth_mod.invalidate_users_cache()
|
||||
with TestClient(app) as c:
|
||||
assert c.get("/api/auth/status").json()["auth_required"] is False
|
||||
assert c.get("/api/sessions").status_code == 200
|
||||
assert c.get("/api/auth/me").json()["authenticated"] is False
|
||||
|
||||
|
||||
def test_first_user_turns_auth_on_and_login_works(clean_users):
|
||||
_add_user("talha", "Talha Ahmed", "correct-horse-9")
|
||||
with TestClient(app) as c:
|
||||
assert c.get("/api/auth/status").json()["auth_required"] is True
|
||||
# Locked out without a token; health stays open for probes.
|
||||
assert c.get("/api/sessions").status_code == 401
|
||||
assert c.get("/api/health").status_code == 200
|
||||
|
||||
assert c.post("/api/auth/login", json={
|
||||
"username": "talha", "password": "nope"}).status_code == 401
|
||||
assert c.post("/api/auth/login", json={
|
||||
"username": "ghost", "password": "correct-horse-9"}).status_code == 401
|
||||
|
||||
r = c.post("/api/auth/login", json={"username": "TALHA", # case-insensitive
|
||||
"password": "correct-horse-9"})
|
||||
assert r.status_code == 200
|
||||
token = r.json()["token"]
|
||||
assert r.json()["user"]["display_name"] == "Talha Ahmed"
|
||||
|
||||
hdr = {"Authorization": f"Bearer {token}"}
|
||||
assert c.get("/api/sessions", headers=hdr).status_code == 200
|
||||
me = c.get("/api/auth/me", headers=hdr).json()
|
||||
assert me["authenticated"] and me["display_name"] == "Talha Ahmed"
|
||||
|
||||
|
||||
def test_signed_in_identity_overrides_body_name(clean_users):
|
||||
"""Accountability fields record the VERIFIED identity, not whatever the body claims."""
|
||||
_add_user("ayesha", "Ayesha K", "another-pw-123")
|
||||
with TestClient(app) as c:
|
||||
token = c.post("/api/auth/login", json={
|
||||
"username": "ayesha", "password": "another-pw-123"}).json()["token"]
|
||||
hdr = {"Authorization": f"Bearer {token}"}
|
||||
|
||||
sid = c.post("/api/sessions", json={"name": "identity", "month_end_date": "2029-01-31",
|
||||
"allow_duplicate": True}, headers=hdr).json()["id"]
|
||||
r = c.post(f"/api/sessions/{sid}/reconciliation-control/verify",
|
||||
json={"verified_by": "Somebody Else", "comment": "spoof attempt"},
|
||||
headers=hdr)
|
||||
assert r.status_code == 200
|
||||
db = SessionLocal()
|
||||
try:
|
||||
fc = db.query(models.FinanceControl).filter_by(session_id=sid).first()
|
||||
assert fc.verified_by == "Ayesha K" # not "Somebody Else"
|
||||
finally:
|
||||
db.close()
|
||||
c.delete(f"/api/sessions/{sid}", headers=hdr)
|
||||
|
||||
|
||||
def test_inactive_user_cannot_login(clean_users):
|
||||
_add_user("gone", "Gone Person", "some-pw-12345")
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.query(models.User).filter_by(username="gone").first().is_active = False
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
auth_mod.invalidate_users_cache()
|
||||
with TestClient(app) as c:
|
||||
assert c.post("/api/auth/login", json={
|
||||
"username": "gone", "password": "some-pw-12345"}).status_code == 401
|
||||
|
|
@ -26,7 +26,7 @@ def _sources() -> list[Path]:
|
|||
def test_no_native_browser_dialogs():
|
||||
offenders: list[str] = []
|
||||
for path in _sources():
|
||||
for i, line in enumerate(path.read_text().splitlines(), 1):
|
||||
for i, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("//") or stripped.startswith("*"):
|
||||
continue
|
||||
|
|
@ -40,14 +40,15 @@ def test_no_native_browser_dialogs():
|
|||
|
||||
|
||||
def test_confirm_dialog_component_exists():
|
||||
ui = (SRC / "components" / "ui.tsx").read_text()
|
||||
# encoding pinned: sources are UTF-8; Windows' default cp1252 chokes on curly quotes
|
||||
ui = (SRC / "components" / "ui.tsx").read_text(encoding="utf-8")
|
||||
assert "export function ConfirmDialog" in ui
|
||||
|
||||
|
||||
def test_destructive_actions_use_confirm_dialog():
|
||||
"""Anything calling deleteSession must route through the in-app dialog."""
|
||||
for path in _sources():
|
||||
text = path.read_text()
|
||||
text = path.read_text(encoding="utf-8")
|
||||
if "deleteSession" in text and "api.ts" not in path.name and "client.ts" not in path.name:
|
||||
assert "ConfirmDialog" in text, (
|
||||
f"{path.relative_to(SRC)} deletes a closing without <ConfirmDialog>"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,139 @@
|
|||
"""FX rate service: orientation (the #1 risk), confirmation withdrawal, caching, failure.
|
||||
|
||||
The provider is mocked — no network in tests. Frankfurter with base=USD returns LOCAL per
|
||||
USD; the app stores USD per LOCAL (usd = local * rate), so the service must invert."""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api.main import app
|
||||
from app.db import models
|
||||
from app.db.database import SessionLocal, init_db
|
||||
from app.services import fx_service
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def fake_frankfurter(monkeypatch):
|
||||
"""Replace the HTTP layer with a fixture: 1 USD = 0.85 EUR on 2029-06-29 (Friday)."""
|
||||
calls = {"n": 0}
|
||||
|
||||
def fake_get(url: str) -> dict:
|
||||
calls["n"] += 1
|
||||
if ".." in url: # time-series request
|
||||
return {"base": "USD", "rates": {
|
||||
"2029-06-28": {"EUR": 0.86},
|
||||
"2029-06-29": {"EUR": 0.85},
|
||||
}}
|
||||
return {"base": "USD", "date": "2029-06-29", "rates": {"EUR": 0.85}}
|
||||
|
||||
monkeypatch.setattr(fx_service, "_http_get_json", fake_get)
|
||||
return calls
|
||||
|
||||
|
||||
def _session_with_fx(c, name: str, month_end: str) -> int:
|
||||
sid = c.post("/api/sessions", json={"name": name, "month_end_date": month_end,
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.add(models.FxRate(session_id=sid, marketplace="Germany", currency="EUR",
|
||||
rate=1.185665, source="default (Jan-26 workbook)",
|
||||
confirmed_by="Old Confirmer", confirmed_month="2026-01"))
|
||||
db.add(models.FxRate(session_id=sid, marketplace="USA", currency="USD", rate=1.0,
|
||||
source="default"))
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
return sid
|
||||
|
||||
|
||||
def test_fetch_inverts_to_usd_per_local_and_clears_confirmation(fake_frankfurter):
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
sid = _session_with_fx(c, "fx orient", "2029-06-30")
|
||||
r = c.post(f"/api/sessions/{sid}/fx/fetch")
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
de = next(u for u in body["updated"] if u["marketplace"] == "Germany")
|
||||
# 1 USD = 0.85 EUR -> 1 EUR = 1/0.85 USD. The inverse (0.85) would mis-state
|
||||
# every EUR receivable — this assertion pins the orientation.
|
||||
assert de["rate"] == pytest.approx(1 / 0.85, abs=1e-6)
|
||||
usa = next(u for u in body["updated"] if u["marketplace"] == "USA")
|
||||
assert usa["rate"] == 1.0
|
||||
assert "frankfurter" in body["source"]
|
||||
assert "2029-06-29" in body["source"] # the provider's banking day
|
||||
|
||||
rows = c.get(f"/api/sessions/{sid}/fx").json()
|
||||
de_row = next(x for x in rows if x["marketplace"] == "Germany")
|
||||
assert "frankfurter" in de_row["source"]
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
fx = db.query(models.FxRate).filter_by(session_id=sid,
|
||||
marketplace="Germany").first()
|
||||
# A fetched rate is a suggestion: the old confirmation no longer applies (C5).
|
||||
assert fx.confirmed_by == "" and fx.confirmed_month == ""
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_second_fetch_is_served_from_cache(fake_frankfurter):
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
sid = _session_with_fx(c, "fx cache", "2029-06-30")
|
||||
assert c.post(f"/api/sessions/{sid}/fx/fetch").status_code == 200
|
||||
n_after_first = fake_frankfurter["n"]
|
||||
r = c.post(f"/api/sessions/{sid}/fx/fetch")
|
||||
assert r.status_code == 200
|
||||
assert fake_frankfurter["n"] == n_after_first # no second HTTP call
|
||||
assert "(cached)" in r.json()["source"]
|
||||
|
||||
|
||||
def test_provider_failure_is_a_502_never_a_silent_default(monkeypatch):
|
||||
init_db()
|
||||
|
||||
def boom(url: str) -> dict:
|
||||
raise fx_service.FxProviderError("provider down")
|
||||
|
||||
monkeypatch.setattr(fx_service, "_http_get_json", boom)
|
||||
with TestClient(app) as c:
|
||||
sid = _session_with_fx(c, "fx down", "2029-08-31")
|
||||
r = c.post(f"/api/sessions/{sid}/fx/fetch")
|
||||
assert r.status_code == 502
|
||||
assert "manually" in r.json()["detail"]
|
||||
# The stored rate is untouched — not overwritten with anything.
|
||||
de = next(x for x in c.get(f"/api/sessions/{sid}/fx").json()
|
||||
if x["marketplace"] == "Germany")
|
||||
assert de["rate"] == 1.185665
|
||||
|
||||
|
||||
def test_fetch_without_processing_explains_the_precondition():
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
sid = c.post("/api/sessions", json={"name": "fx bare", "month_end_date": "2029-09-30",
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
r = c.post(f"/api/sessions/{sid}/fx/fetch")
|
||||
assert r.status_code == 502
|
||||
assert "process" in r.json()["detail"].lower()
|
||||
|
||||
|
||||
def test_daily_fetch_fills_fx_rates_daily(fake_frankfurter):
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
sid = _session_with_fx(c, "fx daily", "2029-06-30")
|
||||
r = c.post(f"/api/sessions/{sid}/fx/fetch-daily",
|
||||
json={"date_from": "2029-06-28", "date_to": "2029-06-29"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["saved"] == 2 # two banking days, EUR only
|
||||
db = SessionLocal()
|
||||
try:
|
||||
rows = db.query(models.FxRateDaily).filter_by(
|
||||
session_id=sid, marketplace="Germany").all()
|
||||
by_date = {row.rate_date: row for row in rows}
|
||||
assert by_date[dt.date(2029, 6, 29)].rate == pytest.approx(1 / 0.85, abs=1e-6)
|
||||
assert by_date[dt.date(2029, 6, 28)].rate == pytest.approx(1 / 0.86, abs=1e-6)
|
||||
assert all(row.source == "frankfurter" for row in rows)
|
||||
finally:
|
||||
db.close()
|
||||
|
|
@ -0,0 +1,117 @@
|
|||
"""One closing per month (guarded), completed closings are read-only, reopen unlocks."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api.main import app
|
||||
from app.db import models
|
||||
from app.db.database import SessionLocal, init_db
|
||||
from tests.test_excel_export import make_amazon_xlsx
|
||||
|
||||
_TMP = tempfile.mkdtemp(prefix="ar_lock_test_")
|
||||
|
||||
|
||||
def _processed(c, name: str, month_end: str) -> int:
|
||||
sid = c.post("/api/sessions", json={"name": name, "month_end_date": month_end,
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
path = os.path.join(_TMP, f"USA {name}.xlsx")
|
||||
make_amazon_xlsx(path, order_rows=5)
|
||||
with open(path, "rb") as fh:
|
||||
assert c.post(f"/api/sessions/{sid}/files",
|
||||
files={"files": (os.path.basename(path), fh)}).status_code == 200
|
||||
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
|
||||
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
|
||||
return sid
|
||||
|
||||
|
||||
def _force_complete(sid: int) -> None:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.get(models.Session, sid).status = "completed"
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_duplicate_month_is_refused_unless_explicit():
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
first = c.post("/api/sessions", json={"name": "dup guard A",
|
||||
"month_end_date": "2028-01-31"})
|
||||
assert first.status_code == 200, first.text
|
||||
|
||||
again = c.post("/api/sessions", json={"name": "dup guard B",
|
||||
"month_end_date": "2028-01-31"})
|
||||
assert again.status_code == 409
|
||||
assert "2028-01" in again.json()["detail"]
|
||||
assert "dup guard A" in again.json()["detail"] # names the existing closing
|
||||
|
||||
forced = c.post("/api/sessions", json={"name": "dup guard C",
|
||||
"month_end_date": "2028-01-31",
|
||||
"allow_duplicate": True})
|
||||
assert forced.status_code == 200
|
||||
|
||||
# The listing flags both sessions as sharing a month.
|
||||
rows = c.get("/api/sessions").json()
|
||||
flagged = [s for s in rows if s["reporting_month"] == "2028-01"]
|
||||
assert len(flagged) == 2 and all(s["duplicate_month"] for s in flagged)
|
||||
|
||||
|
||||
def test_listing_is_month_ordered_with_publish_flag():
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
c.post("/api/sessions", json={"name": "older", "month_end_date": "2028-02-29",
|
||||
"allow_duplicate": True})
|
||||
c.post("/api/sessions", json={"name": "newer", "month_end_date": "2028-03-31",
|
||||
"allow_duplicate": True})
|
||||
rows = c.get("/api/sessions").json()
|
||||
months = [s["reporting_month"] for s in rows if s["reporting_month"]]
|
||||
assert months == sorted(months, reverse=True)
|
||||
assert all("journal_approved" in s for s in rows)
|
||||
|
||||
|
||||
def test_completed_closing_is_locked_and_reopen_unlocks():
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
sid = _processed(c, "lock me", "2028-04-30")
|
||||
_force_complete(sid)
|
||||
|
||||
# Every mutating surface answers 409 while completed.
|
||||
path = os.path.join(_TMP, "USA lock me.xlsx")
|
||||
with open(path, "rb") as fh:
|
||||
up = c.post(f"/api/sessions/{sid}/files",
|
||||
files={"files": ("late file.xlsx", fh)})
|
||||
assert up.status_code == 409
|
||||
assert c.post(f"/api/sessions/{sid}/process").status_code == 409
|
||||
assert c.put(f"/api/sessions/{sid}/fx",
|
||||
json=[{"marketplace": "USA", "currency": "USD", "rate": 1.0}]).status_code == 409
|
||||
assert c.put(f"/api/sessions/{sid}/opening-balances",
|
||||
json=[{"marketplace": "USA", "amount": 1.0}]).status_code == 409
|
||||
assert c.post(f"/api/sessions/{sid}/journal/reset-signoff").status_code == 409
|
||||
assert c.patch(f"/api/sessions/{sid}",
|
||||
json={"clearing_lag_days": 5}).status_code == 409
|
||||
# …but a rename stays allowed, and reads still work.
|
||||
assert c.patch(f"/api/sessions/{sid}", json={"name": "renamed"}).status_code == 200
|
||||
assert c.get(f"/api/sessions/{sid}/summary").status_code == 200
|
||||
|
||||
# Reopen restores editability.
|
||||
assert c.post(f"/api/sessions/{sid}/reopen").status_code == 200
|
||||
assert c.get(f"/api/sessions/{sid}").json()["status"] == "processed"
|
||||
assert c.put(f"/api/sessions/{sid}/opening-balances",
|
||||
json=[{"marketplace": "USA", "amount": 1.0}]).status_code == 200
|
||||
# Reopen on a non-completed closing is refused.
|
||||
assert c.post(f"/api/sessions/{sid}/reopen").status_code == 409
|
||||
|
||||
|
||||
def test_accounts_summary_lists_unpublished_months_as_pending():
|
||||
init_db()
|
||||
with TestClient(app) as c:
|
||||
sid = _processed(c, "pending month", "2028-05-31")
|
||||
summ = c.get("/api/accounts-summary").json()
|
||||
mine = [p for p in summ["pending"] if p["session_id"] == sid]
|
||||
assert len(mine) == 1
|
||||
assert mine[0]["month"] == "2028-05"
|
||||
assert "approv" in mine[0]["reason"] # explains HOW to publish it
|
||||
|
|
@ -32,6 +32,7 @@ def _fresh(c, name: str) -> int:
|
|||
sid = c.post("/api/sessions", json={
|
||||
"name": name, "reporting_month": "2026-01",
|
||||
"month_end_date": "2026-01-31", "clearing_lag_days": 2,
|
||||
"allow_duplicate": True, # suite shares one DB; the guard has its own test
|
||||
}).json()["id"]
|
||||
path = os.path.join(_TMP, f"USA {name}.xlsx")
|
||||
make_amazon_xlsx(path, order_rows=4)
|
||||
|
|
|
|||
|
|
@ -102,6 +102,7 @@ def test_per_market_endpoints():
|
|||
sid = c.post("/api/sessions", json={
|
||||
"name": "multi", "reporting_month": "2026-01",
|
||||
"month_end_date": "2026-01-31", "clearing_lag_days": 2,
|
||||
"allow_duplicate": True, # suite shares one DB; the guard has its own test
|
||||
}).json()["id"]
|
||||
for path in (usa, nl):
|
||||
with open(path, "rb") as fh:
|
||||
|
|
|
|||
|
|
@ -23,7 +23,10 @@ from tests.test_excel_export import make_amazon_xlsx # noqa: E402
|
|||
|
||||
|
||||
def _new(c, name: str, month_end: str, **kw) -> int:
|
||||
body = {"name": name, "month_end_date": month_end, "clearing_lag_days": 2, **kw}
|
||||
# allow_duplicate: the suite reuses months across tests on one shared DB; the
|
||||
# duplicate-month guard itself is covered in test_month_locking.py.
|
||||
body = {"name": name, "month_end_date": month_end, "clearing_lag_days": 2,
|
||||
"allow_duplicate": True, **kw}
|
||||
r = c.post("/api/sessions", json=body)
|
||||
assert r.status_code == 200, r.text
|
||||
return r.json()["id"]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,104 @@
|
|||
"""Upload duplicate protection — the historical double-count bug.
|
||||
|
||||
Re-uploading a file with the same name used to overwrite it on disk but insert a SECOND
|
||||
session_files row pointing at the same path, so processing parsed and summed the file
|
||||
twice. Same content under a different name was equally unguarded."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api.main import app
|
||||
from app.db import models
|
||||
from app.db.database import SessionLocal, init_db
|
||||
from tests.test_excel_export import make_amazon_xlsx
|
||||
|
||||
_TMP = tempfile.mkdtemp(prefix="ar_dedup_test_")
|
||||
|
||||
|
||||
def _upload(c, sid: int, path: str, as_name: str | None = None):
|
||||
with open(path, "rb") as fh:
|
||||
return c.post(f"/api/sessions/{sid}/files",
|
||||
files={"files": (as_name or os.path.basename(path), fh)})
|
||||
|
||||
|
||||
def _file_rows(sid: int) -> list[models.SessionFile]:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
return db.query(models.SessionFile).filter_by(session_id=sid).all()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def test_same_filename_reupload_updates_row_not_duplicates():
|
||||
init_db()
|
||||
a = os.path.join(_TMP, "USA jan.xlsx")
|
||||
make_amazon_xlsx(a, order_rows=6)
|
||||
with TestClient(app) as c:
|
||||
sid = c.post("/api/sessions", json={"name": "dedup-name", "month_end_date": "2027-01-31",
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
first = _upload(c, sid, a).json()
|
||||
assert len(first["files"]) == 1 and first["skipped"] == []
|
||||
first_id = first["files"][0]["id"]
|
||||
|
||||
# Different bytes, same filename -> the existing row is replaced, never doubled.
|
||||
b = os.path.join(_TMP, "USA jan v2.xlsx")
|
||||
make_amazon_xlsx(b, order_rows=9)
|
||||
second = _upload(c, sid, b, as_name="USA jan.xlsx").json()
|
||||
assert len(second["files"]) == 1 and second["skipped"] == []
|
||||
assert second["files"][0]["id"] == first_id # updated in place
|
||||
|
||||
rows = _file_rows(sid)
|
||||
assert len(rows) == 1
|
||||
assert rows[0].sha256 == second["files"][0]["sha256"]
|
||||
|
||||
|
||||
def test_identical_bytes_same_name_is_skipped():
|
||||
init_db()
|
||||
a = os.path.join(_TMP, "USA feb.xlsx")
|
||||
make_amazon_xlsx(a, order_rows=6)
|
||||
with TestClient(app) as c:
|
||||
sid = c.post("/api/sessions", json={"name": "dedup-same", "month_end_date": "2027-02-28",
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
assert _upload(c, sid, a).status_code == 200
|
||||
again = _upload(c, sid, a).json()
|
||||
assert again["files"] == []
|
||||
assert len(again["skipped"]) == 1
|
||||
assert "unchanged" in again["skipped"][0]["reason"]
|
||||
assert len(_file_rows(sid)) == 1
|
||||
|
||||
|
||||
def test_identical_content_under_new_name_is_skipped():
|
||||
init_db()
|
||||
a = os.path.join(_TMP, "USA mar.xlsx")
|
||||
make_amazon_xlsx(a, order_rows=6)
|
||||
with TestClient(app) as c:
|
||||
sid = c.post("/api/sessions", json={"name": "dedup-bytes", "month_end_date": "2027-03-31",
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
assert _upload(c, sid, a).status_code == 200
|
||||
renamed = _upload(c, sid, a, as_name="USA mar COPY.xlsx").json()
|
||||
assert renamed["files"] == []
|
||||
assert "already uploaded as" in renamed["skipped"][0]["reason"]
|
||||
assert len(_file_rows(sid)) == 1
|
||||
|
||||
|
||||
def test_double_upload_no_longer_doubles_the_totals():
|
||||
"""End to end: upload, process, re-upload the SAME file, re-process — totals unchanged."""
|
||||
init_db()
|
||||
a = os.path.join(_TMP, "USA apr.xlsx")
|
||||
make_amazon_xlsx(a, order_rows=8)
|
||||
with TestClient(app) as c:
|
||||
sid = c.post("/api/sessions", json={"name": "dedup-e2e", "month_end_date": "2027-04-30",
|
||||
"allow_duplicate": True}).json()["id"]
|
||||
assert _upload(c, sid, a).status_code == 200
|
||||
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
|
||||
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
|
||||
before = c.get(f"/api/sessions/{sid}/summary").json()["closing_receivable_usd"]
|
||||
|
||||
assert _upload(c, sid, a).status_code == 200 # skipped as unchanged
|
||||
assert c.post(f"/api/sessions/{sid}/process").status_code == 200
|
||||
assert c.get(f"/api/sessions/{sid}/status").json()["status"] == "processed"
|
||||
after = c.get(f"/api/sessions/{sid}/summary").json()["closing_receivable_usd"]
|
||||
assert after == before
|
||||
|
|
@ -0,0 +1,129 @@
|
|||
# Production deployment (AWS, one server)
|
||||
|
||||
One 8 GB server runs everything via `docker-compose.prod.yml`:
|
||||
**caddy** (automatic HTTPS) → **web** (nginx: React build + `/api` proxy) → **backend**
|
||||
(FastAPI, single worker) + **mysql** (data on the instance disk), with nightly backups to S3.
|
||||
|
||||
8 GB RAM is not optional: uploaded Amazon exports are 300–500 MB and expand to multi-GB
|
||||
while parsing. Validate with your largest real file before buying anything smaller.
|
||||
|
||||
Monthly cost: **≈ $50–55** — Lightsail 8 GB $44 (or EC2 `t4g.large` ≈ $61 with EBS + IPv4),
|
||||
S3 backups $1.50–3, weekly snapshots $2–4, Route 53 $0.50, Frankfurter FX API $0.
|
||||
|
||||
---
|
||||
|
||||
## 1. Provision
|
||||
|
||||
1. **Lightsail**: 8 GB / 2 vCPU / 160 GB SSD instance, Ubuntu 24.04. Attach the included
|
||||
static IP. (EC2 route: `t4g.large` + 100 GB gp3 EBS + Elastic IP.)
|
||||
2. Firewall: allow 22 (your office IPs only), 80, 443. Everything else closed.
|
||||
3. Install Docker + AWS CLI:
|
||||
```bash
|
||||
curl -fsSL https://get.docker.com | sh
|
||||
sudo usermod -aG docker $USER # re-login after this
|
||||
sudo apt-get install -y awscli # or the AWS CLI v2 bundle
|
||||
```
|
||||
4. **S3 bucket** for backups: create `company-ar-backups`, enable **versioning**, add a
|
||||
lifecycle rule (transition to Glacier/IA after 90 days). Attach an IAM **role** to the
|
||||
instance allowing `s3:PutObject`, `s3:GetObject`, `s3:ListBucket` on that bucket —
|
||||
no access keys on disk.
|
||||
5. **DNS**: A record `ar.<company>.com` → the static IP. Caddy then issues and renews the
|
||||
TLS certificate automatically — there is no certbot step.
|
||||
|
||||
## 2. Configure & start
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/ar-aging && sudo chown $USER /opt/ar-aging
|
||||
cd /opt/ar-aging && git clone <repo-url> . && cd ar-aging-app
|
||||
|
||||
cp .env.example .env.production
|
||||
nano .env.production # fill the PRODUCTION section: AR_DOMAIN, passwords,
|
||||
# AR_SECRET_KEY (openssl rand -hex 32), backup bucket
|
||||
# (a pre-filled .env.production with generated credentials already exists on the
|
||||
# dev machine — copy it to the server instead of re-generating)
|
||||
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
|
||||
curl -s https://ar.<company>.com/api/health # {"status":"ok",...}
|
||||
```
|
||||
|
||||
> Every `docker compose ... -f docker-compose.prod.yml` command below also needs
|
||||
> `--env-file .env.production` — set an alias once and forget it:
|
||||
> `alias dcp='docker compose --env-file .env.production -f docker-compose.prod.yml'`
|
||||
|
||||
## 3. Create the users (5 logins)
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
|
||||
python manage.py add-user talha --name "Talha Ahmed"
|
||||
# repeat per user; passwords are prompted, never stored in shell history
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend python manage.py list-users
|
||||
```
|
||||
|
||||
`AR_AUTH=on` means the API refuses everything except login/health until users exist.
|
||||
Password resets: `manage.py set-password <username>`. Leavers: `manage.py deactivate-user`.
|
||||
|
||||
## 4. Migrate the existing SQLite data (one-time)
|
||||
|
||||
The current data lives in `backend/data/ar_aging.db` on the dev machine. **Do a timed dry
|
||||
run first** — January alone is ~3.4M transaction rows.
|
||||
|
||||
```bash
|
||||
# copy the SQLite file to the server first (scp), then from ar-aging-app/:
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml cp ./ar_aging.db backend:/tmp/ar_aging.db
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
|
||||
python migrate_sqlite_to_mysql.py --sqlite /tmp/ar_aging.db --dry-run
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend \
|
||||
python migrate_sqlite_to_mysql.py --sqlite /tmp/ar_aging.db
|
||||
```
|
||||
|
||||
Verify before anyone uses it: per-table row counts printed by the script must match, and a
|
||||
spot check to the cent — open the January closing and compare `/api/sessions/{id}/reconciliation`
|
||||
`final_receivable_usd` against the dev machine. Copy `backend/data/uploads/` into the
|
||||
`ar_data` volume the same way (`compose cp ./uploads backend:/data/`), then archive the
|
||||
SQLite file to S3 and retire the dev copy.
|
||||
|
||||
**One-time cleanup for the historical double-count bug** (duplicate upload rows):
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend python manage.py dedupe-files # dry run
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml exec backend python manage.py dedupe-files --apply
|
||||
# then re-process the closings it flagged
|
||||
```
|
||||
|
||||
## 5. Backups
|
||||
|
||||
```bash
|
||||
chmod +x deploy/backup.sh
|
||||
crontab -e
|
||||
# 30 2 * * * /opt/ar-aging/ar-aging-app/deploy/backup.sh >> /var/log/ar-backup.log 2>&1
|
||||
```
|
||||
|
||||
Three layers: nightly `mysqldump` + uploads/exports → versioned S3 (the script), weekly
|
||||
instance snapshots (Lightsail console → enable automatic snapshots), and MySQL's own volume
|
||||
on the instance disk. **Run the restore drill quarterly** — commands are at the bottom of
|
||||
`backup.sh`.
|
||||
|
||||
## 6. Deploying updates
|
||||
|
||||
```bash
|
||||
cd /opt/ar-aging/ar-aging-app
|
||||
git pull
|
||||
docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
|
||||
```
|
||||
|
||||
Deploy outside a processing run when possible. If a restart does land mid-run, the closing
|
||||
is auto-marked as interrupted at startup (never stuck on "processing") — just re-run it.
|
||||
|
||||
## 7. Operating notes
|
||||
|
||||
- **Single backend worker, single instance — by design.** Jobs and their progress live
|
||||
in-process. Do not add `--workers` or replicas.
|
||||
- Logs: `docker compose --env-file .env.production -f docker-compose.prod.yml logs -f backend` (requests, jobs, FX
|
||||
fetches, logins). Add the CloudWatch agent if you want them off-box.
|
||||
- Health: `GET /api/health` checks the DB and data-dir and is unauthenticated — point
|
||||
Lightsail/CloudWatch monitoring at it.
|
||||
- Exchange rates: Frankfurter (free, keyless). The only outbound call the app makes;
|
||||
currency codes and dates only. Fetched rates still require in-app confirmation (C5).
|
||||
- Upgrade path (not needed at this scale): move MySQL to RDS `db.t4g.small` (+~$30/mo,
|
||||
point-in-time restore) by setting `MYSQL_HOST` to the RDS endpoint and removing the
|
||||
mysql service; move exports to S3-primary with presigned URLs if the disk ever tightens.
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
#!/usr/bin/env bash
|
||||
# Nightly backup: MySQL dump + uploads/exports -> S3 (versioned bucket).
|
||||
#
|
||||
# Install on the server (as the user that runs docker):
|
||||
# crontab -e
|
||||
# 30 2 * * * /opt/ar-aging/ar-aging-app/deploy/backup.sh >> /var/log/ar-backup.log 2>&1
|
||||
#
|
||||
# Requires: aws cli v2 on the host, an instance IAM role with s3:PutObject/ListBucket on
|
||||
# the bucket (no access keys on disk), and .env.production next to docker-compose.prod.yml.
|
||||
set -euo pipefail
|
||||
|
||||
APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$APP_DIR"
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
set -a; source .env.production; set +a
|
||||
BUCKET="${AR_BACKUP_S3_BUCKET:?AR_BACKUP_S3_BUCKET not set in .env.production}"
|
||||
STAMP="$(date +%Y-%m-%d_%H%M)"
|
||||
COMPOSE="docker compose --env-file .env.production -f docker-compose.prod.yml"
|
||||
|
||||
echo "[$STAMP] backup starting"
|
||||
|
||||
# 1) MySQL dump (single transaction: consistent snapshot without locking the app out).
|
||||
$COMPOSE exec -T mysql sh -c \
|
||||
'exec mysqldump --single-transaction --quick --routines \
|
||||
-u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' \
|
||||
| gzip > "/tmp/ar_aging_${STAMP}.sql.gz"
|
||||
aws s3 cp "/tmp/ar_aging_${STAMP}.sql.gz" "$BUCKET/mysql/ar_aging_${STAMP}.sql.gz"
|
||||
rm -f "/tmp/ar_aging_${STAMP}.sql.gz"
|
||||
|
||||
# 2) Uploaded source files + generated exports (the audit trail).
|
||||
# The ar_data volume is mounted by the backend container; sync straight from it.
|
||||
DATA_MOUNT="$(docker volume inspect -f '{{ .Mountpoint }}' \
|
||||
"$(basename "$APP_DIR" | tr '[:upper:]' '[:lower:]' | tr -cd 'a-z0-9')_ar_data" 2>/dev/null \
|
||||
|| docker volume inspect -f '{{ .Mountpoint }}' ar-aging-app_ar_data)"
|
||||
aws s3 sync "$DATA_MOUNT/uploads" "$BUCKET/data/uploads" --only-show-errors
|
||||
aws s3 sync "$DATA_MOUNT/exports" "$BUCKET/data/exports" --only-show-errors
|
||||
|
||||
echo "[$STAMP] backup finished"
|
||||
|
||||
# Restore drill (run quarterly — a backup you never restored is a hope, not a backup):
|
||||
# aws s3 cp "$BUCKET/mysql/<latest>.sql.gz" - | gunzip | \
|
||||
# docker compose --env-file .env.production -f docker-compose.prod.yml exec -T mysql \
|
||||
# sh -c 'exec mysql -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"'
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
# Production stack: caddy (auto-HTTPS) -> web (nginx: SPA + /api proxy) -> backend + mysql.
|
||||
#
|
||||
# cp .env.example .env.production # fill the PRODUCTION section first
|
||||
# docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build
|
||||
#
|
||||
# --env-file is REQUIRED: the ${AR_DOMAIN} / ${MYSQL_ROOT_PASSWORD} references below are
|
||||
# resolved from it (env_file: alone only feeds the containers, not this YAML).
|
||||
#
|
||||
# Sized for one 8 GB server (300-500 MB Excel parsing needs the RAM). Backend runs ONE
|
||||
# worker by design — jobs and their progress live in-process. See deploy/DEPLOY.md.
|
||||
|
||||
services:
|
||||
mysql:
|
||||
image: mysql:8.4
|
||||
restart: unless-stopped
|
||||
env_file: .env.production # uses MYSQL_PASSWORD / MYSQL_DATABASE / MYSQL_USER
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?set MYSQL_ROOT_PASSWORD in .env.production}
|
||||
command:
|
||||
- --innodb-buffer-pool-size=1G
|
||||
- --max-allowed-packet=256M
|
||||
volumes:
|
||||
- mysql_data:/var/lib/mysql
|
||||
healthcheck:
|
||||
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-p${MYSQL_ROOT_PASSWORD}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 12
|
||||
# Not exposed to the host network — only the backend reaches it.
|
||||
|
||||
backend:
|
||||
build: ./backend
|
||||
restart: unless-stopped
|
||||
env_file: .env.production
|
||||
environment:
|
||||
AR_DATA_DIR: /data
|
||||
MYSQL_HOST: mysql
|
||||
AR_DB_BACKEND: mysql
|
||||
volumes:
|
||||
- ar_data:/data
|
||||
depends_on:
|
||||
mysql:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c",
|
||||
"import urllib.request;urllib.request.urlopen('http://localhost:8000/api/health', timeout=5)"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
web:
|
||||
build:
|
||||
context: ./frontend
|
||||
target: prod
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- backend
|
||||
# Not exposed directly — caddy fronts it with TLS.
|
||||
|
||||
# TLS terminator: automatic Let's Encrypt certificates for AR_DOMAIN, renewed by itself.
|
||||
# No certbot cron, no cert plumbing. Set AR_DOMAIN (and a DNS A record) and it works.
|
||||
caddy:
|
||||
image: caddy:2-alpine
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
environment:
|
||||
AR_DOMAIN: ${AR_DOMAIN:?set AR_DOMAIN in .env.production}
|
||||
command: caddy reverse-proxy --from "https://${AR_DOMAIN}" --to web:80
|
||||
volumes:
|
||||
- caddy_data:/data
|
||||
- caddy_config:/config
|
||||
|
||||
volumes:
|
||||
mysql_data:
|
||||
ar_data:
|
||||
caddy_data:
|
||||
caddy_config:
|
||||
|
|
@ -12,7 +12,9 @@ services:
|
|||
command: uvicorn app.api.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
|
||||
frontend:
|
||||
build: ./frontend
|
||||
build:
|
||||
context: ./frontend
|
||||
target: dev
|
||||
ports:
|
||||
- "5173:5173"
|
||||
volumes:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,55 @@
|
|||
# Local audit — 19 Aug 2026
|
||||
|
||||
Full audit of the local database, files, and the running app before the team enters the
|
||||
first production month. **Verdict: system healthy and ready; three data-cleanup items for
|
||||
the team below.**
|
||||
|
||||
## What was checked
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Database integrity (`PRAGMA integrity_check`) | ✅ ok — `ar_aging.db`, 295 MB, 20 tables |
|
||||
| Foreign keys / orphaned rows | ✅ zero violations, zero orphans |
|
||||
| Transactions | ✅ 883,930 rows across 4 processed closings, none missing currency or marketplace |
|
||||
| Uploaded files vs database | ✅ all 4 files present on disk, sizes match, SHA-256 recorded |
|
||||
| Duplicate upload rows (historical double-count bug) | ✅ none found — no closing ever double-counted a file |
|
||||
| Orphan files on disk | ✅ none |
|
||||
| Users | ✅ 4 active accounts (login verified for each) |
|
||||
| Full test suite | ✅ 152 passed, 0 failed (12 skipped — large sample files) |
|
||||
| Live API (running app, port 8010) | ✅ health deep-check ok; no token → 401; wrong password → 401; login ok; month-ordered listing; pending-months explanations; controls 5/6 on Jan |
|
||||
| Frontend (port 5174) | ✅ serving, proxying to the API |
|
||||
| Exchange-rate provider (live call) | ✅ Frankfurter reachable; EUR→USD 2026-06-30 = **1.139406**, 2026-01-30 = 1.191895 |
|
||||
|
||||
One environment fix made during the audit: this Windows machine's OS certificate store is
|
||||
corrupted (Python `ssl [ASN1: NOT_ENOUGH_DATA]`), which blocked HTTPS calls. The FX service
|
||||
now uses the `certifi` CA bundle instead (added to requirements) — affects nothing else.
|
||||
|
||||
## Findings for the team (data, not code)
|
||||
|
||||
### 1. ⚠️ June closings are valued at January's exchange rate — ≈ $50k overstated
|
||||
Closings **#3, #4, #5** (all 2026-06) carry EUR→USD = **1.185665**, the January-2026
|
||||
workbook snapshot, and it was *confirmed* at that value. The actual ECB rate on
|
||||
2026-06-30 was **1.139406** — the June receivable of $1,290,921 is overstated by roughly
|
||||
**$50,000**. Fix on whichever June closing is kept: Controls tab → **Fetch month-end
|
||||
rates** → review → Confirm → re-run controls. This is precisely the failure mode the new
|
||||
FX fetch exists to prevent.
|
||||
|
||||
### 2. ⚠️ Three identical June closings + three empty drafts
|
||||
Closings #3 ("July finance report"), #4 ("june"), #5 ("Test Case - Germany Jun-2026") are
|
||||
the **same June file processed three times** — identical 214,166 transactions and identical
|
||||
receivable. Keep one, delete the other two. Drafts #2, #6 (2026-06) and #7 (2026-01) are
|
||||
empty and can be deleted. The dashboard now flags all of these with a duplicate-month ⚠.
|
||||
Going forward the app blocks accidental month duplicates at creation.
|
||||
|
||||
### 3. ℹ️ No month is published yet
|
||||
No journal has been approved, so the Accounts Summary is empty — the summary page now
|
||||
lists each processed month with the reason ("journal not approved yet") and a link. When
|
||||
January is final: Journal Entry tab → Mark reviewed → Approve (records the signed-in
|
||||
user's name).
|
||||
|
||||
## State after cleanup (recommended target)
|
||||
|
||||
- One closing per month: `2026-01` (#1) and one `2026-06`, both with fetched + confirmed
|
||||
June/January rates, journals approved, then **Complete** to lock them read-only.
|
||||
- First production month gets entered by the team on the deployed server per
|
||||
`deploy/DEPLOY.md`; this local database migrates there as-is.
|
||||
|
|
@ -215,7 +215,7 @@ the ledger, a wrong one can.
|
|||
Resolve a block on the **Controls** tab (e.g. confirm the FX rates for the month), then re-run.
|
||||
Every control result travels with the workbook on its own *Month-End Controls* sheet.
|
||||
|
||||
See [`AUDIT-REPORT.md`](AUDIT-REPORT.md) for the audit these controls came out of.
|
||||
See [`audit-2026-07-31-code-review.md`](audit-2026-07-31-code-review.md) for the audit these controls came out of.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
MYSQL_HOST=your-mysql-host.example.com
|
||||
MYSQL_PORT=3306
|
||||
MYSQL_USER=your_mysql_user
|
||||
MYSQL_PASSWORD=your_mysql_password
|
||||
MYSQL_DATABASE=account_finance
|
||||
MYSQL_SLOW_QUERY_MS=500
|
||||
MYSQL_POOL_SIZE=10
|
||||
MYSQL_POOL_RECYCLE=3600
|
||||
|
||||
# Uploads/exports. Docker Compose sets AR_DATA_DIR=/data (named volume).
|
||||
# Leave unset locally to use backend/data.
|
||||
# AR_DATA_DIR=/data
|
||||
AR_CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# Docker only reads the .dockerignore INSIDE the build context (this folder). Without it,
|
||||
# `COPY . .` would overwrite the image's freshly installed node_modules with the host's
|
||||
# (built for a different OS) and drag in stale dist output.
|
||||
node_modules
|
||||
dist
|
||||
.env
|
||||
.env.*
|
||||
|
|
@ -1,12 +1,22 @@
|
|||
FROM node:20-alpine
|
||||
# Multi-stage: `dev` target = Vite dev server (docker-compose.yml),
|
||||
# default/`prod` target = static build served by nginx (docker-compose.prod.yml).
|
||||
|
||||
FROM node:20-alpine AS deps
|
||||
WORKDIR /app
|
||||
|
||||
COPY package.json package-lock.json* ./
|
||||
RUN npm install
|
||||
|
||||
COPY . .
|
||||
|
||||
# ---- dev: hot-reload server (dev compose bind-mounts the source over /app) ----
|
||||
FROM deps AS dev
|
||||
EXPOSE 5173
|
||||
|
||||
CMD ["npm", "run", "dev", "--", "--host", "0.0.0.0", "--port", "5173"]
|
||||
|
||||
# ---- prod: type-check + build, then serve the static bundle with nginx ----
|
||||
FROM deps AS build
|
||||
RUN npm run build
|
||||
|
||||
FROM nginx:1.27-alpine AS prod
|
||||
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
||||
COPY --from=build /app/dist /usr/share/nginx/html
|
||||
EXPOSE 80
|
||||
|
|
|
|||
|
|
@ -0,0 +1,40 @@
|
|||
# Production frontend: serve the built SPA, proxy /api to the backend container.
|
||||
# TLS is terminated in front of this (caddy service in docker-compose.prod.yml).
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name _;
|
||||
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
# Uploads are 300-500 MB Amazon exports; the app enforces its own 2 GB cap.
|
||||
client_max_body_size 2g;
|
||||
|
||||
gzip on;
|
||||
gzip_types text/plain text/css application/json application/javascript image/svg+xml;
|
||||
|
||||
location /api {
|
||||
proxy_pass http://backend:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
|
||||
# Big uploads and long-running processing/status calls.
|
||||
proxy_request_buffering off;
|
||||
proxy_read_timeout 600s;
|
||||
proxy_send_timeout 600s;
|
||||
}
|
||||
|
||||
# SPA routing: every non-file path renders index.html.
|
||||
location / {
|
||||
try_files $uri /index.html;
|
||||
}
|
||||
|
||||
# Hashed assets can cache forever; index.html must not.
|
||||
location /assets/ {
|
||||
add_header Cache-Control "public, max-age=31536000, immutable";
|
||||
}
|
||||
location = /index.html {
|
||||
add_header Cache-Control "no-cache";
|
||||
}
|
||||
}
|
||||
|
|
@ -1,10 +1,13 @@
|
|||
import { NavLink, Route, Routes } from "react-router-dom";
|
||||
import { LayoutDashboard, FilePlus2, Settings as SettingsIcon, Landmark, Table2 } from "lucide-react";
|
||||
import { LayoutDashboard, FilePlus2, LogOut, Settings as SettingsIcon, Landmark, Table2, UserCircle2 } from "lucide-react";
|
||||
import Dashboard from "./pages/Dashboard";
|
||||
import AccountsSummary from "./pages/AccountsSummary";
|
||||
import NewClosing from "./pages/NewClosing";
|
||||
import Closing from "./pages/Closing";
|
||||
import Settings from "./pages/Settings";
|
||||
import Login from "./pages/Login";
|
||||
import { Spinner } from "./components/ui";
|
||||
import { useAuth } from "./auth";
|
||||
|
||||
function SideLink({ to, icon: Icon, children, end }: {
|
||||
to: string; icon: typeof LayoutDashboard; children: string; end?: boolean;
|
||||
|
|
@ -28,6 +31,16 @@ function SideLink({ to, icon: Icon, children, end }: {
|
|||
}
|
||||
|
||||
export default function App() {
|
||||
const { loading, authRequired, user, logout } = useAuth();
|
||||
|
||||
if (loading)
|
||||
return (
|
||||
<div className="h-full bg-canvas flex items-center justify-center gap-2 text-subink">
|
||||
<Spinner /> Loading…
|
||||
</div>
|
||||
);
|
||||
if (authRequired && !user) return <Login />;
|
||||
|
||||
return (
|
||||
<div className="h-full bg-canvas p-3 sm:p-4">
|
||||
<div className="flex h-full min-h-0 bg-panel rounded-3xl shadow-pop overflow-hidden border border-line">
|
||||
|
|
@ -47,8 +60,22 @@ export default function App() {
|
|||
<SideLink to="/new" icon={FilePlus2}>New Closing</SideLink>
|
||||
<SideLink to="/settings" icon={SettingsIcon}>Settings</SideLink>
|
||||
</nav>
|
||||
<div className="m-3 p-3.5 rounded-2xl bg-canvas/70 text-[11px] text-muted leading-relaxed">
|
||||
Amazon‑only · processed locally on your server · no third‑party egress.
|
||||
{user && (
|
||||
<div className="mx-3 mb-1 p-3 rounded-2xl bg-canvas/70 flex items-center gap-2.5">
|
||||
<UserCircle2 size={20} className="text-primary shrink-0" />
|
||||
<div className="flex-1 min-w-0">
|
||||
<div className="text-xs font-semibold text-ink truncate">{user.display_name}</div>
|
||||
<div className="text-[11px] text-muted truncate">{user.username}</div>
|
||||
</div>
|
||||
<button className="p-1.5 rounded-lg text-subink hover:text-bad hover:bg-badbg"
|
||||
title="Sign out" onClick={logout}>
|
||||
<LogOut size={15} />
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
<div className="m-3 mt-1 p-3.5 rounded-2xl bg-canvas/70 text-[11px] text-muted leading-relaxed">
|
||||
Amazon‑only · processed on your server. No third‑party egress except the FX‑rate
|
||||
lookup (currency codes only — never financial data).
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,20 @@
|
|||
const BASE = "/api";
|
||||
|
||||
const TOKEN_KEY = "ar_token";
|
||||
export const getToken = () => localStorage.getItem(TOKEN_KEY);
|
||||
export const setToken = (t: string) => localStorage.setItem(TOKEN_KEY, t);
|
||||
export const clearToken = () => localStorage.removeItem(TOKEN_KEY);
|
||||
|
||||
/** Set by the auth provider: called on a 401 so the app can drop to the login screen. */
|
||||
export let onUnauthorized: (() => void) | null = null;
|
||||
export const setOnUnauthorized = (fn: (() => void) | null) => { onUnauthorized = fn; };
|
||||
|
||||
async function req<T>(path: string, opts: RequestInit = {}): Promise<T> {
|
||||
const res = await fetch(`${BASE}${path}`, {
|
||||
headers: opts.body && !(opts.body instanceof FormData)
|
||||
? { "Content-Type": "application/json" }
|
||||
: undefined,
|
||||
...opts,
|
||||
});
|
||||
const headers: Record<string, string> = {};
|
||||
if (opts.body && !(opts.body instanceof FormData)) headers["Content-Type"] = "application/json";
|
||||
const token = getToken();
|
||||
if (token) headers["Authorization"] = `Bearer ${token}`;
|
||||
const res = await fetch(`${BASE}${path}`, { headers, ...opts });
|
||||
if (!res.ok) {
|
||||
let detail = res.statusText;
|
||||
try {
|
||||
|
|
@ -14,6 +22,10 @@ async function req<T>(path: string, opts: RequestInit = {}): Promise<T> {
|
|||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
if (res.status === 401 && !path.startsWith("/auth/")) {
|
||||
clearToken();
|
||||
onUnauthorized?.();
|
||||
}
|
||||
throw new Error(detail);
|
||||
}
|
||||
const ct = res.headers.get("content-type") ?? "";
|
||||
|
|
@ -49,6 +61,20 @@ export interface SessionT {
|
|||
payout_mode?: string;
|
||||
/** Bank receipts / payout mode changed after the last run — re-process to apply. */
|
||||
needs_reprocess?: boolean;
|
||||
/** Journal approved = published to the Accounts Summary (list endpoint only). */
|
||||
journal_approved?: boolean;
|
||||
/** Another closing exists for the same reporting month (list endpoint only). */
|
||||
duplicate_month?: boolean;
|
||||
}
|
||||
|
||||
export interface UploadResultT {
|
||||
files: FileT[];
|
||||
skipped: { filename: string; reason: string }[];
|
||||
}
|
||||
|
||||
export interface AuthUserT {
|
||||
username: string;
|
||||
display_name: string;
|
||||
}
|
||||
|
||||
export interface PayoutT {
|
||||
|
|
@ -243,6 +269,9 @@ export interface AccountsSummaryT {
|
|||
month: string; session_id: number; marketplace: string; currency: string; fx_rate: number;
|
||||
values: Record<string, number>; receivable: number;
|
||||
}[];
|
||||
/** Months with results that are NOT published (unapproved / blocked), with the reason —
|
||||
* so a month never silently vanishes from this view. */
|
||||
pending?: { month: string; session_id: number; session_name: string; reason: string }[];
|
||||
}
|
||||
|
||||
export interface ComponentT {
|
||||
|
|
@ -460,19 +489,27 @@ const q = (o: Record<string, string | undefined>) =>
|
|||
export const api = {
|
||||
health: () => req<{ status: string; version: string }>("/health"),
|
||||
|
||||
authStatus: () => req<{ auth_required: boolean }>("/auth/status"),
|
||||
login: (username: string, password: string) =>
|
||||
req<{ token: string; user: AuthUserT }>("/auth/login", {
|
||||
method: "POST", body: JSON.stringify({ username, password }),
|
||||
}),
|
||||
me: () => req<{ authenticated: boolean; username?: string; display_name?: string }>("/auth/me"),
|
||||
|
||||
listSessions: () => req<SessionT[]>("/sessions"),
|
||||
createSession: (body: Partial<SessionT>) =>
|
||||
createSession: (body: Partial<SessionT> & { allow_duplicate?: boolean }) =>
|
||||
req<SessionT>("/sessions", { method: "POST", body: JSON.stringify(body) }),
|
||||
getSession: (id: number) => req<SessionT>(`/sessions/${id}`),
|
||||
updateSession: (id: number, body: Partial<SessionT>) =>
|
||||
req<SessionT>(`/sessions/${id}`, { method: "PATCH", body: JSON.stringify(body) }),
|
||||
deleteSession: (id: number) => req<void>(`/sessions/${id}`, { method: "DELETE" }),
|
||||
reopenSession: (id: number) => req<SessionT>(`/sessions/${id}/reopen`, { method: "POST" }),
|
||||
|
||||
listFiles: (id: number) => req<FileT[]>(`/sessions/${id}/files`),
|
||||
uploadFiles: (id: number, files: File[]) => {
|
||||
const fd = new FormData();
|
||||
files.forEach((f) => fd.append("files", f));
|
||||
return req<FileT[]>(`/sessions/${id}/files`, { method: "POST", body: fd });
|
||||
return req<UploadResultT>(`/sessions/${id}/files`, { method: "POST", body: fd });
|
||||
},
|
||||
deleteFile: (id: number, fileId: number) =>
|
||||
req<void>(`/sessions/${id}/files/${fileId}`, { method: "DELETE" }),
|
||||
|
|
@ -577,6 +614,14 @@ export const api = {
|
|||
source: string; rate_date: string | null }[]>(`/sessions/${id}/fx`),
|
||||
putFx: (id: number, items: { marketplace: string; currency: string; rate: number }[]) =>
|
||||
req(`/sessions/${id}/fx`, { method: "PUT", body: JSON.stringify(items) }),
|
||||
fetchFx: (id: number) =>
|
||||
req<{ updated: { marketplace: string; currency: string; rate: number }[];
|
||||
missing: string[]; source: string; rate_date: string }>(
|
||||
`/sessions/${id}/fx/fetch`, { method: "POST" }),
|
||||
fetchFxDaily: (id: number, body: { marketplace?: string; date_from?: string; date_to?: string } = {}) =>
|
||||
req<{ saved: number; date_from: string; date_to: string; provider: string;
|
||||
marketplaces: string[] }>(
|
||||
`/sessions/${id}/fx/fetch-daily`, { method: "POST", body: JSON.stringify(body) }),
|
||||
|
||||
startExport: (id: number, kind: "full" | "summary" = "full") =>
|
||||
req<{ started: boolean; kind: string }>(`/sessions/${id}/export?kind=${kind}`, { method: "POST" }),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,79 @@
|
|||
import { ReactNode, createContext, useContext, useEffect, useMemo, useState } from "react";
|
||||
import { api, AuthUserT, clearToken, getToken, setOnUnauthorized, setToken } from "./api/client";
|
||||
|
||||
/**
|
||||
* Login state for the whole app.
|
||||
*
|
||||
* The backend decides whether login is required (/auth/status): with AR_AUTH=auto it turns
|
||||
* on as soon as the first user is created, so a dev checkout keeps working with no ceremony
|
||||
* while production requires sign-in. The signed-in display name is also what the backend
|
||||
* records in review/approve/confirm fields — the UI shows it instead of a free-text box.
|
||||
*/
|
||||
interface AuthState {
|
||||
loading: boolean;
|
||||
authRequired: boolean;
|
||||
user: AuthUserT | null;
|
||||
login: (username: string, password: string) => Promise<void>;
|
||||
logout: () => void;
|
||||
}
|
||||
|
||||
const AuthCtx = createContext<AuthState>({
|
||||
loading: true, authRequired: false, user: null,
|
||||
login: async () => undefined, logout: () => undefined,
|
||||
});
|
||||
|
||||
export const useAuth = () => useContext(AuthCtx);
|
||||
|
||||
export function AuthProvider({ children }: { children: ReactNode }) {
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [authRequired, setAuthRequired] = useState(false);
|
||||
const [user, setUser] = useState<AuthUserT | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
setOnUnauthorized(() => {
|
||||
setUser(null);
|
||||
setAuthRequired(true);
|
||||
});
|
||||
return () => setOnUnauthorized(null);
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
(async () => {
|
||||
try {
|
||||
const status = await api.authStatus();
|
||||
setAuthRequired(status.auth_required);
|
||||
if (status.auth_required && getToken()) {
|
||||
try {
|
||||
const me = await api.me();
|
||||
if (me.authenticated && me.username) {
|
||||
setUser({ username: me.username, display_name: me.display_name ?? me.username });
|
||||
}
|
||||
} catch {
|
||||
clearToken();
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Backend unreachable — leave the app open; queries will surface the real error.
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
})();
|
||||
}, []);
|
||||
|
||||
const value = useMemo<AuthState>(() => ({
|
||||
loading,
|
||||
authRequired,
|
||||
user,
|
||||
login: async (username: string, password: string) => {
|
||||
const res = await api.login(username, password);
|
||||
setToken(res.token);
|
||||
setUser(res.user);
|
||||
},
|
||||
logout: () => {
|
||||
clearToken();
|
||||
setUser(null);
|
||||
},
|
||||
}), [loading, authRequired, user]);
|
||||
|
||||
return <AuthCtx.Provider value={value}>{children}</AuthCtx.Provider>;
|
||||
}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
import { useNavigate } from "react-router-dom";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { CalendarDays } from "lucide-react";
|
||||
import { api } from "../api/client";
|
||||
|
||||
/**
|
||||
* Jump between month-end closings from inside any closing screen. Every previous month
|
||||
* stays saved and selectable here — uploading a new month never replaces an old one.
|
||||
*/
|
||||
export default function MonthSwitcher({ currentId }: { currentId: number }) {
|
||||
const nav = useNavigate();
|
||||
const { data: sessions } = useQuery({ queryKey: ["sessions"], queryFn: api.listSessions });
|
||||
if (!sessions || sessions.length < 2) return null;
|
||||
|
||||
return (
|
||||
<label className="flex items-center gap-1.5 text-sm text-subink">
|
||||
<CalendarDays size={15} className="text-primary shrink-0" />
|
||||
<span className="sr-only">Switch closing</span>
|
||||
<select
|
||||
className="input py-1.5 pr-7 text-sm max-w-[16rem]"
|
||||
value={currentId}
|
||||
onChange={(e) => nav(`/closing/${e.target.value}`)}
|
||||
>
|
||||
{sessions.map((s) => (
|
||||
<option key={s.id} value={s.id}>
|
||||
{s.reporting_month ?? "no month"} · {s.name}
|
||||
{s.status === "completed" ? " ✓" : s.journal_approved ? " (published)" : ""}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
);
|
||||
}
|
||||
|
|
@ -3,6 +3,7 @@ import ReactDOM from "react-dom/client";
|
|||
import { BrowserRouter } from "react-router-dom";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import App from "./App";
|
||||
import { AuthProvider } from "./auth";
|
||||
import "./index.css";
|
||||
|
||||
const queryClient = new QueryClient({
|
||||
|
|
@ -12,9 +13,11 @@ const queryClient = new QueryClient({
|
|||
ReactDOM.createRoot(document.getElementById("root")!).render(
|
||||
<React.StrictMode>
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<AuthProvider>
|
||||
<BrowserRouter>
|
||||
<App />
|
||||
</BrowserRouter>
|
||||
</AuthProvider>
|
||||
</QueryClientProvider>
|
||||
</React.StrictMode>
|
||||
);
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
import { useState } from "react";
|
||||
import { Link } from "react-router-dom";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { BadgeCheck, Table2 } from "lucide-react";
|
||||
import { api } from "../api/client";
|
||||
import { BadgeCheck, CalendarClock, Table2 } from "lucide-react";
|
||||
import { api, AccountsSummaryT } from "../api/client";
|
||||
import { acct, money } from "../lib/format";
|
||||
import { EmptyState, Section, Spinner } from "../components/ui";
|
||||
|
||||
|
|
@ -26,10 +26,11 @@ export default function AccountsSummary() {
|
|||
return <div className="p-6 flex items-center gap-2 text-subink"><Spinner /> Loading…</div>;
|
||||
if (!data?.available)
|
||||
return (
|
||||
<div className="p-6 max-w-7xl mx-auto">
|
||||
<div className="p-6 max-w-7xl mx-auto space-y-6">
|
||||
<Header />
|
||||
<EmptyState title="No approved months yet"
|
||||
hint="Open a closing's Journal Entry tab, have it reviewed and approved — approval publishes that month here, for every marketplace." />
|
||||
<PendingMonths pending={data?.pending} />
|
||||
</div>
|
||||
);
|
||||
|
||||
|
|
@ -126,6 +127,8 @@ export default function AccountsSummary() {
|
|||
</div>
|
||||
</Section>
|
||||
|
||||
<PendingMonths pending={data.pending} />
|
||||
|
||||
<p className="text-xs text-subink">
|
||||
{showAll
|
||||
? "USD figures convert each marketplace at its own closing's confirmed FX rate."
|
||||
|
|
@ -137,6 +140,32 @@ export default function AccountsSummary() {
|
|||
);
|
||||
}
|
||||
|
||||
/** Months with results that are NOT published — listed with the reason instead of just
|
||||
* vanishing from the grid (the classic "where did January go?" confusion). */
|
||||
function PendingMonths({ pending }: { pending: AccountsSummaryT["pending"] }) {
|
||||
if (!pending?.length) return null;
|
||||
return (
|
||||
<div className="card border-warn/30 bg-warnbg/30 p-4">
|
||||
<p className="text-sm font-semibold text-ink flex items-center gap-2 mb-2">
|
||||
<CalendarClock size={15} className="text-warn" />
|
||||
{pending.length} month(s) processed but not shown here
|
||||
</p>
|
||||
<ul className="space-y-1.5">
|
||||
{pending.map((p) => (
|
||||
<li key={p.session_id} className="text-sm text-subink flex items-baseline gap-2 flex-wrap">
|
||||
<span className="num font-semibold text-ink">{p.month}</span>
|
||||
<span className="flex-1 min-w-[200px]">{p.reason}</span>
|
||||
<Link to={`/closing/${p.session_id}/journal`}
|
||||
className="text-primary font-medium hover:underline shrink-0">
|
||||
open journal →
|
||||
</Link>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function Header() {
|
||||
return (
|
||||
<header>
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
import { NavLink, Outlet, Route, Routes, useParams, useOutletContext } from "react-router-dom";
|
||||
import { Clock, RefreshCw, ShieldAlert } from "lucide-react";
|
||||
import { BadgeInfo, Clock, Lock, RefreshCw, ShieldAlert } from "lucide-react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { api, SessionT } from "../api/client";
|
||||
import { StatusBadge, ProgressStages, Spinner } from "../components/ui";
|
||||
import MonthSwitcher from "../components/MonthSwitcher";
|
||||
import { date } from "../lib/format";
|
||||
import Overview from "./closing/Overview";
|
||||
import Controls from "./closing/Controls";
|
||||
|
|
@ -18,7 +19,13 @@ import FinanceSummary from "./closing/FinanceSummary";
|
|||
import JournalEntry from "./closing/JournalEntry";
|
||||
import ExportPage from "./closing/ExportPage";
|
||||
|
||||
export interface ClosingCtx { id: number; session: SessionT; processed: boolean }
|
||||
export interface ClosingCtx {
|
||||
id: number;
|
||||
session: SessionT;
|
||||
processed: boolean;
|
||||
/** Completed closings are read-only until explicitly reopened. */
|
||||
locked: boolean;
|
||||
}
|
||||
export const useClosing = () => useOutletContext<ClosingCtx>();
|
||||
|
||||
const TABS = [
|
||||
|
|
@ -43,6 +50,23 @@ function ReprocessButton({ id }: { id: number }) {
|
|||
);
|
||||
}
|
||||
|
||||
function ReopenButton({ id }: { id: number }) {
|
||||
const qc = useQueryClient();
|
||||
const reopen = useMutation({
|
||||
mutationFn: () => api.reopenSession(id),
|
||||
onSuccess: () => {
|
||||
qc.invalidateQueries({ queryKey: ["session", id] });
|
||||
qc.invalidateQueries({ queryKey: ["sessions"] });
|
||||
},
|
||||
});
|
||||
return (
|
||||
<button className="btn-ghost shrink-0" disabled={reopen.isPending}
|
||||
onClick={() => reopen.mutate()}>
|
||||
{reopen.isPending ? <Spinner /> : null} Reopen for corrections
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
export default function Closing() {
|
||||
const { id } = useParams();
|
||||
const sid = Number(id);
|
||||
|
|
@ -55,18 +79,31 @@ export default function Closing() {
|
|||
refetchIntervalInBackground: true, // keep progress updating if the tab isn't focused
|
||||
});
|
||||
|
||||
// A blocked closing is fully processed — its tabs stay open for diagnosis, but every
|
||||
// endpoint that publishes a receivable figure withholds it until the control is resolved.
|
||||
const processed = session
|
||||
? session.status === "processed" || session.status === "completed"
|
||||
|| session.status === "blocked"
|
||||
: false;
|
||||
|
||||
// Publish state: the same query key the Journal tab uses, so the cache is shared.
|
||||
const { data: journal } = useQuery({
|
||||
queryKey: ["journal", sid, ""],
|
||||
queryFn: () => api.journal(sid),
|
||||
enabled: processed,
|
||||
});
|
||||
|
||||
if (isLoading || !session)
|
||||
return <div className="p-6 flex items-center gap-2 text-subink"><Spinner /> Loading closing…</div>;
|
||||
|
||||
// A blocked closing is fully processed — its tabs stay open for diagnosis, but every
|
||||
// endpoint that publishes a receivable figure withholds it until the control is resolved.
|
||||
const processed = session.status === "processed" || session.status === "completed"
|
||||
|| session.status === "blocked";
|
||||
const locked = session.status === "completed";
|
||||
const unpublished = processed && !session.blocked
|
||||
&& journal?.available === true && !journal.approved_by;
|
||||
|
||||
return (
|
||||
<div>
|
||||
<header className="bg-panel border-b border-line px-6 pt-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center justify-between gap-4 flex-wrap">
|
||||
<div>
|
||||
<h1 className="text-lg font-semibold text-ink">{session.name}</h1>
|
||||
<p className="text-sm text-subink">
|
||||
|
|
@ -74,8 +111,11 @@ export default function Closing() {
|
|||
clearing-lag {session.clearing_lag_days}d
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-3">
|
||||
<MonthSwitcher currentId={sid} />
|
||||
<StatusBadge status={session.status} />
|
||||
</div>
|
||||
</div>
|
||||
<nav className="flex gap-1 mt-4 -mb-px overflow-x-auto">
|
||||
{TABS.map(([to, label]) => (
|
||||
<NavLink key={to} to={to} end={to === ""}
|
||||
|
|
@ -101,13 +141,25 @@ export default function Closing() {
|
|||
<div className="card border-bad/40 bg-badbg/40 p-4 text-sm text-bad whitespace-pre-wrap">{session.error}</div>
|
||||
</div>
|
||||
)}
|
||||
{session.needs_reprocess && session.status !== "processing" && (
|
||||
{locked && (
|
||||
<div className="px-6 pt-4">
|
||||
<div className="card border-line bg-neutralbg/50 p-3 flex items-center gap-3 text-sm">
|
||||
<Lock size={16} className="text-subink shrink-0" />
|
||||
<span className="flex-1">
|
||||
This closing is <b>completed and locked</b> — its figures are read-only so
|
||||
published history cannot drift. Reopen it only if a correction is genuinely needed.
|
||||
</span>
|
||||
<ReopenButton id={sid} />
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{session.needs_reprocess && session.status !== "processing" && !locked && (
|
||||
<div className="px-6 pt-4">
|
||||
<div className="card border-warn/30 bg-warnbg/40 p-3 flex items-center gap-3 text-sm">
|
||||
<Clock size={16} className="text-warn shrink-0" />
|
||||
<span className="flex-1">
|
||||
Bank receipts or the payout mode changed after the last run — the figures on
|
||||
screen don't reflect them yet. <b>Re-process to apply.</b>
|
||||
Inputs changed after the last run (files, bank receipts, or the payout mode) —
|
||||
the figures on screen don't reflect them yet. <b>Re-process to apply.</b>
|
||||
</span>
|
||||
<ReprocessButton id={sid} />
|
||||
</div>
|
||||
|
|
@ -127,10 +179,24 @@ export default function Closing() {
|
|||
</div>
|
||||
</div>
|
||||
)}
|
||||
{unpublished && !locked && (
|
||||
<div className="px-6 pt-4">
|
||||
<div className="card border-line bg-primary-soft/30 p-3 flex items-center gap-3 text-sm">
|
||||
<BadgeInfo size={16} className="text-primary shrink-0" />
|
||||
<span className="flex-1">
|
||||
This month is <b>not on the Accounts Summary yet</b> — approving the journal
|
||||
is what publishes it{journal?.entry_no
|
||||
? " (a re-process withdraws any earlier sign-off, so it may need re-approval)"
|
||||
: ""}.
|
||||
</span>
|
||||
<NavLink to="journal" className="btn-ghost shrink-0">Open Journal Entry</NavLink>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="p-6">
|
||||
<Routes>
|
||||
<Route element={<Outlet context={{ id: sid, session, processed } satisfies ClosingCtx} />}>
|
||||
<Route element={<Outlet context={{ id: sid, session, processed, locked } satisfies ClosingCtx} />}>
|
||||
<Route index element={<Overview />} />
|
||||
<Route path="controls" element={<Controls />} />
|
||||
<Route path="opening" element={<OpeningBalances />} />
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import { Link, useNavigate } from "react-router-dom";
|
||||
import { useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { FilePlus2, Trash2, ChevronRight } from "lucide-react";
|
||||
import { AlertTriangle, BadgeCheck, FilePlus2, Trash2, ChevronRight } from "lucide-react";
|
||||
import { api, SessionT } from "../api/client";
|
||||
import { usd, date } from "../lib/format";
|
||||
import { ConfirmDialog, EmptyState, Kpi, Spinner, StatusBadge } from "../components/ui";
|
||||
|
|
@ -59,15 +59,36 @@ export default function Dashboard() {
|
|||
<table className="w-full">
|
||||
<thead><tr>
|
||||
<th className="th">Name</th><th className="th">Month</th><th className="th">Month-end</th>
|
||||
<th className="th">Status</th><th className="th text-right">Receivable (USD)</th><th className="th"></th>
|
||||
<th className="th">Status</th><th className="th">Published</th>
|
||||
<th className="th text-right">Receivable (USD)</th><th className="th"></th>
|
||||
</tr></thead>
|
||||
<tbody>
|
||||
{sessions.map((s) => (
|
||||
<tr key={s.id} className="hover:bg-canvas/60 cursor-pointer" onClick={() => nav(`/closing/${s.id}`)}>
|
||||
<td className="td font-medium">{s.name}</td>
|
||||
<td className="td num">{s.reporting_month ?? "—"}</td>
|
||||
<td className="td num">
|
||||
{s.reporting_month ?? "—"}
|
||||
{s.duplicate_month && (
|
||||
<span className="ml-1.5 inline-flex align-middle" title="Another closing exists for this month">
|
||||
<AlertTriangle size={13} className="text-warn" />
|
||||
</span>
|
||||
)}
|
||||
</td>
|
||||
<td className="td num">{date(s.month_end_date)}</td>
|
||||
<td className="td"><StatusBadge status={s.status} /></td>
|
||||
<td className="td text-xs">
|
||||
{s.journal_approved ? (
|
||||
<span className="inline-flex items-center gap-1 text-ok font-medium">
|
||||
<BadgeCheck size={13} /> published
|
||||
</span>
|
||||
) : s.status === "processed" || s.status === "completed" ? (
|
||||
<span className="text-subink" title="Approve the journal to publish this month to the Accounts Summary">
|
||||
not published
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-muted">—</span>
|
||||
)}
|
||||
</td>
|
||||
<td className="td text-right num">{s.status === "processed" ? <LatestReceivable id={s.id} /> : "—"}</td>
|
||||
<td className="td text-right">
|
||||
<div className="flex items-center justify-end gap-1">
|
||||
|
|
|
|||
|
|
@ -0,0 +1,64 @@
|
|||
import { FormEvent, useState } from "react";
|
||||
import { Landmark, LogIn } from "lucide-react";
|
||||
import { Spinner } from "../components/ui";
|
||||
import { useAuth } from "../auth";
|
||||
|
||||
export default function Login() {
|
||||
const { login } = useAuth();
|
||||
const [username, setUsername] = useState("");
|
||||
const [password, setPassword] = useState("");
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [busy, setBusy] = useState(false);
|
||||
|
||||
const submit = async (e: FormEvent) => {
|
||||
e.preventDefault();
|
||||
setError(null);
|
||||
setBusy(true);
|
||||
try {
|
||||
await login(username.trim(), password);
|
||||
} catch (err) {
|
||||
setError((err as Error).message);
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="h-full bg-canvas flex items-center justify-center p-4">
|
||||
<div className="card w-full max-w-sm p-6 shadow-pop">
|
||||
<div className="flex items-center gap-2.5 mb-5">
|
||||
<span className="inline-flex items-center justify-center w-9 h-9 rounded-xl bg-primary text-white">
|
||||
<Landmark size={18} />
|
||||
</span>
|
||||
<div>
|
||||
<div className="text-sm font-semibold leading-tight text-ink">Amazon A/R Aging</div>
|
||||
<div className="text-[11px] text-muted">Month-End Closing · sign in</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<form onSubmit={submit} className="space-y-3">
|
||||
<div>
|
||||
<label className="label" htmlFor="login-user">Username</label>
|
||||
<input id="login-user" className="input" autoComplete="username" autoFocus
|
||||
value={username} onChange={(e) => setUsername(e.target.value)} />
|
||||
</div>
|
||||
<div>
|
||||
<label className="label" htmlFor="login-pw">Password</label>
|
||||
<input id="login-pw" className="input" type="password" autoComplete="current-password"
|
||||
value={password} onChange={(e) => setPassword(e.target.value)} />
|
||||
</div>
|
||||
{error && <p className="text-sm text-bad">{error}</p>}
|
||||
<button className="btn-primary w-full justify-center" type="submit"
|
||||
disabled={busy || !username.trim() || !password}>
|
||||
{busy ? <Spinner /> : <LogIn size={16} />} Sign in
|
||||
</button>
|
||||
</form>
|
||||
|
||||
<p className="text-[11px] text-muted mt-4">
|
||||
Accounts are created by the administrator on the server
|
||||
(<span className="num">manage.py add-user</span>) — there is no self-signup.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
|
@ -1,6 +1,7 @@
|
|||
import { useState } from "react";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import { useNavigate } from "react-router-dom";
|
||||
import { useMutation, useQuery } from "@tanstack/react-query";
|
||||
import { AlertTriangle } from "lucide-react";
|
||||
import { api } from "../api/client";
|
||||
|
||||
type OpeningMode = "zero" | "carry_forward" | "manual";
|
||||
|
|
@ -10,22 +11,50 @@ function lastDayOfMonth(ym: string): string {
|
|||
return new Date(y, m, 0).toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
function nextMonth(ym: string): string {
|
||||
const [y, m] = ym.split("-").map(Number);
|
||||
const d = new Date(y, m, 1); // month is 0-based, so this is the month AFTER ym
|
||||
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
function currentMonth(): string {
|
||||
const d = new Date();
|
||||
return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
export default function NewClosing() {
|
||||
const nav = useNavigate();
|
||||
const [month, setMonth] = useState("2026-01");
|
||||
const [month, setMonth] = useState(currentMonth());
|
||||
const [name, setName] = useState("");
|
||||
const [lag, setLag] = useState(2);
|
||||
const [allowance, setAllowance] = useState(0);
|
||||
const [openingMode, setOpeningMode] = useState<OpeningMode>("zero");
|
||||
const [sourceId, setSourceId] = useState<number | undefined>();
|
||||
const [forceDuplicate, setForceDuplicate] = useState(false);
|
||||
// Once the user touches month/opening themselves, stop auto-defaulting over their choice.
|
||||
const touched = useRef({ month: false, opening: false });
|
||||
|
||||
// Prior closings whose closing balance can be carried into this one.
|
||||
const { data: sessions } = useQuery({ queryKey: ["sessions"], queryFn: api.listSessions });
|
||||
// Prior closings whose closing balance can be carried into this one.
|
||||
const priors = (sessions ?? []).filter(
|
||||
(s) => s.status === "processed" || s.status === "completed",
|
||||
);
|
||||
const effectiveSource = sourceId ?? priors[0]?.id;
|
||||
|
||||
// Smart defaults once the sessions load: the month AFTER the latest closing, opening
|
||||
// carried forward from it — the normal month-to-month flow needs zero clicks.
|
||||
useEffect(() => {
|
||||
if (!sessions?.length) return;
|
||||
const latest = sessions.find((s) => s.reporting_month)?.reporting_month; // list is newest-first
|
||||
if (latest && !touched.current.month) setMonth(nextMonth(latest));
|
||||
if (priors.length > 0 && !touched.current.opening) setOpeningMode("carry_forward");
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [sessions]);
|
||||
|
||||
const existing = (sessions ?? []).filter(
|
||||
(s) => s.reporting_month === month && s.status !== "error");
|
||||
const isDuplicate = existing.length > 0;
|
||||
|
||||
const create = useMutation({
|
||||
mutationFn: () =>
|
||||
api.createSession({
|
||||
|
|
@ -36,6 +65,7 @@ export default function NewClosing() {
|
|||
reporting_currency: "USD",
|
||||
opening_mode: openingMode,
|
||||
opening_source_session_id: openingMode === "carry_forward" ? effectiveSource : null,
|
||||
allow_duplicate: forceDuplicate,
|
||||
}),
|
||||
onSuccess: (s) => nav(`/closing/${s.id}/upload`),
|
||||
});
|
||||
|
|
@ -50,7 +80,8 @@ export default function NewClosing() {
|
|||
}`}>
|
||||
<div className="flex items-start gap-2.5">
|
||||
<input type="radio" name="opening" className="mt-1 accent-[#6D5DE8]"
|
||||
checked={active} onChange={() => setOpeningMode(value)} />
|
||||
checked={active}
|
||||
onChange={() => { touched.current.opening = true; setOpeningMode(value); }} />
|
||||
<div className="flex-1">
|
||||
<div className="text-sm font-medium text-ink">{title}</div>
|
||||
<div className="text-xs text-subink mt-0.5">{desc}</div>
|
||||
|
|
@ -65,15 +96,48 @@ export default function NewClosing() {
|
|||
<div className="p-6 max-w-2xl mx-auto space-y-6">
|
||||
<header>
|
||||
<h1 className="text-xl font-semibold text-ink">New Month-End Closing</h1>
|
||||
<p className="text-sm text-subink">Set the reporting period, then upload the Amazon transaction files.</p>
|
||||
<p className="text-sm text-subink">Set the reporting period, then upload the Amazon transaction files. Every month stays saved as its own closing — new months never overwrite previous ones.</p>
|
||||
</header>
|
||||
|
||||
<div className="card p-5 space-y-4">
|
||||
<div>
|
||||
<label className="label">Reporting month</label>
|
||||
<input type="month" className="input" value={month} onChange={(e) => setMonth(e.target.value)} />
|
||||
<input type="month" className="input" value={month}
|
||||
onChange={(e) => {
|
||||
touched.current.month = true;
|
||||
setForceDuplicate(false);
|
||||
setMonth(e.target.value);
|
||||
}} />
|
||||
<p className="text-xs text-subink mt-1">Month-end date will be {month ? lastDayOfMonth(month) : "—"} (auto).</p>
|
||||
</div>
|
||||
|
||||
{isDuplicate && (
|
||||
<div className="rounded-xl border border-warn/40 bg-warnbg/40 p-3 space-y-2">
|
||||
<p className="text-sm text-ink flex items-start gap-2">
|
||||
<AlertTriangle size={16} className="text-warn shrink-0 mt-0.5" />
|
||||
<span>
|
||||
A closing for <b>{month}</b> already exists:{" "}
|
||||
<b>{existing[0].name}</b>. Two closings for one month means two competing
|
||||
datasets for the same period.
|
||||
</span>
|
||||
</p>
|
||||
<div className="flex items-center gap-2 pl-6">
|
||||
<button className="btn-primary" onClick={() => nav(`/closing/${existing[0].id}`)}>
|
||||
Open the existing closing
|
||||
</button>
|
||||
{forceDuplicate ? (
|
||||
<span className="text-xs text-warn font-medium">
|
||||
Creating a second closing for {month} — deliberate.
|
||||
</span>
|
||||
) : (
|
||||
<button className="btn-ghost text-xs" onClick={() => setForceDuplicate(true)}>
|
||||
I need another one anyway
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div>
|
||||
<label className="label">Closing name</label>
|
||||
<input className="input" placeholder={`Amazon A/R Aging — ${month}`} value={name} onChange={(e) => setName(e.target.value)} />
|
||||
|
|
@ -95,12 +159,14 @@ export default function NewClosing() {
|
|||
<div className="pt-1">
|
||||
<label className="label">Opening AR balance</label>
|
||||
<div className="space-y-2">
|
||||
<Option value="zero" title="Start at zero (default)"
|
||||
<Option value="zero" title="Start at zero"
|
||||
desc="Every marketplace opens at 0. Use this for your first-ever closing." />
|
||||
|
||||
<Option value="carry_forward"
|
||||
title="Carry forward from a previous closing"
|
||||
desc="Copies each marketplace's closing receivable into this month's opening balance.">
|
||||
title={priors.length > 0
|
||||
? "Carry forward from a previous closing (default)"
|
||||
: "Carry forward from a previous closing"}
|
||||
desc="Copies each marketplace's closing receivable into this month's opening balance — the normal month-to-month flow.">
|
||||
{priors.length === 0 ? (
|
||||
<p className="text-xs text-warn mt-2">
|
||||
No processed closing available yet — this will fall back to zero.
|
||||
|
|
@ -129,7 +195,9 @@ export default function NewClosing() {
|
|||
{create.isError && <p className="text-sm text-bad">{(create.error as Error).message}</p>}
|
||||
<div className="flex justify-end gap-2 pt-2">
|
||||
<button className="btn-ghost" onClick={() => nav("/")}>Cancel</button>
|
||||
<button className="btn-primary" disabled={create.isPending || !month} onClick={() => create.mutate()}>
|
||||
<button className="btn-primary"
|
||||
disabled={create.isPending || !month || (isDuplicate && !forceDuplicate)}
|
||||
onClick={() => create.mutate()}>
|
||||
{create.isPending ? "Creating…" : "Create & upload files"}
|
||||
</button>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -34,10 +34,17 @@ export default function Settings() {
|
|||
<div className="p-4 flex items-start gap-3">
|
||||
<ShieldCheck className="text-ok shrink-0" size={22} />
|
||||
<ul className="text-sm text-subink space-y-1.5">
|
||||
<li>Files are processed locally on the company server — no third-party or AI services.</li>
|
||||
<li>Uploads are session-scoped; filenames are sanitized; no public file URLs.</li>
|
||||
<li>Full financial transaction rows are not logged; temporary files follow a retention policy.</li>
|
||||
<li>Files are processed locally on the company server — no third-party or AI services.
|
||||
The one exception: exchange-rate lookups send currency codes and dates to the
|
||||
configured FX provider (Frankfurter/ECB by default). No financial data ever leaves.</li>
|
||||
<li>Login is per-user; journal review/approval and FX confirmations record the
|
||||
signed-in person's verified name. Accounts are created by the administrator.</li>
|
||||
<li>Uploads are session-scoped; filenames are sanitized; no public file URLs.
|
||||
Re-uploads replace their file — a month can never count a file twice.</li>
|
||||
<li>Full financial transaction rows are not logged. Generated exports are purged
|
||||
after the retention window; uploaded source files are kept as the audit source.</li>
|
||||
<li>Every generated workbook includes a Processing Audit Trail with SHA-256 file hashes.</li>
|
||||
<li>Completed closings are locked read-only; corrections require an explicit reopen.</li>
|
||||
</ul>
|
||||
</div>
|
||||
</Section>
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import { ReactNode, useEffect, useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient, keepPreviousData } from "@tanstack/react-query";
|
||||
import { ArrowDownRight, ArrowUpRight, Pencil, Save, CalendarRange,
|
||||
RotateCcw, CornerDownRight } from "lucide-react";
|
||||
import { ArrowDownRight, ArrowUpRight, CloudDownload, Pencil, Save, CalendarRange,
|
||||
RotateCcw, CornerDownRight, Loader2 } from "lucide-react";
|
||||
import { api, OpeningBalanceT } from "../../api/client";
|
||||
import { money, acct, num } from "../../lib/format";
|
||||
import { InfoTip, Section, EmptyState, StatusBadge, useDefinitions } from "../../components/ui";
|
||||
|
|
@ -277,7 +277,8 @@ export default function ArLedger() {
|
|||
|
||||
{/* ---------------- daily FX ---------------- */}
|
||||
<Section title={`Daily exchange rates — ${mkt}`}
|
||||
subtitle="Local value per day, the USD rate applied, and the USD equivalent.">
|
||||
subtitle="Local value per day, the USD rate applied, and the USD equivalent."
|
||||
actions={cur !== "USD" ? <FetchDailyRates id={id} mkt={mkt} /> : undefined}>
|
||||
{cur === "USD" && (
|
||||
<div className="px-4 pt-3 text-xs text-subink">
|
||||
{mkt} reports in USD — no conversion applied (rate 1.000000).
|
||||
|
|
@ -328,6 +329,34 @@ export default function ArLedger() {
|
|||
);
|
||||
}
|
||||
|
||||
/** Fill the daily override table with official (ECB via Frankfurter) rates for the month. */
|
||||
function FetchDailyRates({ id, mkt }: { id: number; mkt: string }) {
|
||||
const qc = useQueryClient();
|
||||
const { locked } = useClosing();
|
||||
const fetchDaily = useMutation({
|
||||
mutationFn: () => api.fetchFxDaily(id, { marketplace: mkt }),
|
||||
onSuccess: () => qc.invalidateQueries({ queryKey: ["fx-daily", id] }),
|
||||
});
|
||||
return (
|
||||
<div className="flex items-center gap-2">
|
||||
{fetchDaily.isSuccess && (
|
||||
<span className="text-xs text-ok">
|
||||
{fetchDaily.data.saved} daily rate(s) loaded ({fetchDaily.data.provider}).
|
||||
</span>
|
||||
)}
|
||||
{fetchDaily.isError && (
|
||||
<span className="text-xs text-bad">{(fetchDaily.error as Error).message}</span>
|
||||
)}
|
||||
<button className="btn-ghost" disabled={fetchDaily.isPending || locked}
|
||||
title="Fetch the month's official daily rates. Hand-entered overrides are replaced for the fetched dates."
|
||||
onClick={() => fetchDaily.mutate()}>
|
||||
{fetchDaily.isPending ? <Loader2 size={15} className="animate-spin" /> : <CloudDownload size={15} />}
|
||||
Fetch daily rates
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function Row({ label, v, cur, bold, pos }: {
|
||||
label: ReactNode; v: number; cur: string; bold?: boolean; pos?: boolean;
|
||||
}) {
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
import { useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { CheckCircle2, XCircle, MinusCircle, AlertTriangle, RefreshCw, ShieldCheck } from "lucide-react";
|
||||
import { CheckCircle2, XCircle, MinusCircle, AlertTriangle, CloudDownload, RefreshCw, ShieldCheck, UserCircle2 } from "lucide-react";
|
||||
import { api, MonthEndControlT } from "../../api/client";
|
||||
import { Section, EmptyState, Spinner } from "../../components/ui";
|
||||
import { useAuth } from "../../auth";
|
||||
import { useClosing } from "../Closing";
|
||||
|
||||
const ICON = {
|
||||
|
|
@ -19,9 +20,12 @@ function tone(r: MonthEndControlT) {
|
|||
}
|
||||
|
||||
export default function Controls() {
|
||||
const { id, session } = useClosing();
|
||||
const { id, session, locked } = useClosing();
|
||||
const { user } = useAuth();
|
||||
const qc = useQueryClient();
|
||||
const [who, setWho] = useState("");
|
||||
// Signed in -> the verified identity confirms; the free-text box only exists without auth.
|
||||
const confirmer = user?.display_name ?? who;
|
||||
|
||||
const [edits, setEdits] = useState<Record<string, string>>({});
|
||||
|
||||
|
|
@ -42,7 +46,10 @@ export default function Controls() {
|
|||
};
|
||||
const rerun = useMutation({ mutationFn: () => api.runControls(id), onSuccess: invalidate });
|
||||
const confirmFx = useMutation({
|
||||
mutationFn: () => api.confirmAllFx(id, who.trim()), onSuccess: invalidate,
|
||||
mutationFn: () => api.confirmAllFx(id, confirmer.trim()), onSuccess: invalidate,
|
||||
});
|
||||
const fetchRates = useMutation({
|
||||
mutationFn: () => api.fetchFx(id), onSuccess: invalidate,
|
||||
});
|
||||
// Saving marks the rates source=manual and (by design) withdraws any prior confirmation
|
||||
// for a changed rate — the person then confirms the corrected value below.
|
||||
|
|
@ -93,7 +100,26 @@ export default function Controls() {
|
|||
|
||||
{fxFailing && (
|
||||
<Section title="Confirm exchange rates"
|
||||
subtitle={`Control C5 requires a rate confirmed for ${session.reporting_month ?? "this month"}. Seeded defaults are a January-2026 snapshot and are treated as missing.`}>
|
||||
subtitle={`Control C5 requires a rate confirmed for ${session.reporting_month ?? "this month"}. Seeded defaults are a January-2026 snapshot and are treated as missing.`}
|
||||
actions={
|
||||
<button className="btn-ghost" disabled={fetchRates.isPending || locked}
|
||||
title="Fetch official month-end rates (ECB via Frankfurter). Fetched rates still need your confirmation below."
|
||||
onClick={() => fetchRates.mutate()}>
|
||||
{fetchRates.isPending ? <Spinner /> : <CloudDownload size={15} />}
|
||||
Fetch month-end rates
|
||||
</button>
|
||||
}>
|
||||
{fetchRates.isSuccess && (
|
||||
<p className="px-4 pt-3 text-xs text-ok">
|
||||
Fetched {fetchRates.data.updated.length} rate(s) from {fetchRates.data.source}.
|
||||
{fetchRates.data.missing.length > 0 &&
|
||||
` No rate available for: ${fetchRates.data.missing.join(", ")} — enter those manually.`}{" "}
|
||||
Review the rates, then confirm them below.
|
||||
</p>
|
||||
)}
|
||||
{fetchRates.isError && (
|
||||
<p className="px-4 pt-3 text-xs text-bad">{(fetchRates.error as Error).message}</p>
|
||||
)}
|
||||
<div className="overflow-x-auto border-b border-line">
|
||||
<table className="w-full">
|
||||
<thead><tr>
|
||||
|
|
@ -127,13 +153,20 @@ export default function Controls() {
|
|||
{saveFx.isPending ? <Spinner /> : null} Save corrected rates
|
||||
</button>
|
||||
)}
|
||||
{user ? (
|
||||
<p className="text-sm text-subink flex items-center gap-1.5">
|
||||
<UserCircle2 size={16} className="text-primary" />
|
||||
Confirming as <b className="text-ink">{user.display_name}</b>
|
||||
</p>
|
||||
) : (
|
||||
<label className="text-sm">
|
||||
<span className="block text-xs font-medium text-subink mb-1">Confirmed by</span>
|
||||
<input className="input" placeholder="Your name" value={who}
|
||||
onChange={(e) => setWho(e.target.value)} />
|
||||
</label>
|
||||
)}
|
||||
<button className="btn-primary"
|
||||
disabled={!who.trim() || dirty || confirmFx.isPending}
|
||||
disabled={!confirmer.trim() || dirty || confirmFx.isPending || locked}
|
||||
onClick={() => confirmFx.mutate()}>
|
||||
{confirmFx.isPending ? <Spinner /> : <CheckCircle2 size={15} />}
|
||||
Confirm all rates for {session.reporting_month ?? "this month"}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,11 @@
|
|||
import { useEffect, useState } from "react";
|
||||
import { useMutation, useQueries, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { BadgeCheck, CheckCircle2, FileCheck2, RotateCcw } from "lucide-react";
|
||||
import { BadgeCheck, CheckCircle2, FileCheck2, RotateCcw, UserCircle2 } from "lucide-react";
|
||||
import { api, JournalLineT, JournalT } from "../../api/client";
|
||||
import { acct, date as fmtDate } from "../../lib/format";
|
||||
import { EmptyState, InfoTip, Section, Spinner, useDefinitions } from "../../components/ui";
|
||||
import { ALL_MARKETS, MarketTabs, isAll, useMarket } from "../../components/market";
|
||||
import { useAuth } from "../../auth";
|
||||
import { useClosing } from "../Closing";
|
||||
|
||||
/**
|
||||
|
|
@ -225,25 +226,37 @@ function AllMarketsJournal({ id, markets }: { id: number; markets: string[] }) {
|
|||
|
||||
/* ------------------------------------------------------- review & approval */
|
||||
function SignOff({ id, j }: { id: number; j: JournalT }) {
|
||||
const { user } = useAuth();
|
||||
const qc = useQueryClient();
|
||||
const [reviewer, setReviewer] = useState("");
|
||||
const [approver, setApprover] = useState("");
|
||||
// Signed in -> the verified identity signs; free-text boxes only exist without auth.
|
||||
const reviewerName = user?.display_name ?? reviewer;
|
||||
const approverName = user?.display_name ?? approver;
|
||||
const invalidate = () => {
|
||||
qc.invalidateQueries({ queryKey: ["journal", id] });
|
||||
qc.invalidateQueries({ queryKey: ["journal", id, ""] });
|
||||
qc.invalidateQueries({ queryKey: ["accounts-summary"] });
|
||||
qc.invalidateQueries({ queryKey: ["sessions"] });
|
||||
};
|
||||
const review = useMutation({
|
||||
mutationFn: () => api.reviewJournal(id, reviewer.trim()),
|
||||
mutationFn: () => api.reviewJournal(id, reviewerName.trim()),
|
||||
onSuccess: () => { setReviewer(""); invalidate(); },
|
||||
});
|
||||
const approve = useMutation({
|
||||
mutationFn: () => api.approveJournal(id, approver.trim()),
|
||||
mutationFn: () => api.approveJournal(id, approverName.trim()),
|
||||
onSuccess: () => { setApprover(""); invalidate(); },
|
||||
});
|
||||
const reset = useMutation({ mutationFn: () => api.resetJournalSignoff(id), onSuccess: invalidate });
|
||||
|
||||
const reviewed = !!j.reviewed_by;
|
||||
const approved = !!j.approved_by;
|
||||
const Identity = () => (
|
||||
<p className="text-sm text-subink flex items-center gap-1.5 flex-1">
|
||||
<UserCircle2 size={16} className="text-primary" />
|
||||
as <b className="text-ink">{user?.display_name}</b>
|
||||
</p>
|
||||
);
|
||||
|
||||
return (
|
||||
<Section title="Review & approval"
|
||||
|
|
@ -260,10 +273,12 @@ function SignOff({ id, j }: { id: number; j: JournalT }) {
|
|||
<span className="text-xs text-subink ml-2">{fmtDate(j.reviewed_at)}</span>
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex gap-2 mt-2">
|
||||
<div className="flex gap-2 mt-2 items-center">
|
||||
{user ? <Identity /> : (
|
||||
<input className="input flex-1" placeholder="Reviewer's name" value={reviewer}
|
||||
onChange={(e) => setReviewer(e.target.value)} />
|
||||
<button className="btn-primary" disabled={!reviewer.trim() || review.isPending}
|
||||
)}
|
||||
<button className="btn-primary" disabled={!reviewerName.trim() || review.isPending}
|
||||
onClick={() => review.mutate()}>
|
||||
{review.isPending ? <Spinner /> : <FileCheck2 size={15} />} Mark reviewed
|
||||
</button>
|
||||
|
|
@ -284,11 +299,13 @@ function SignOff({ id, j }: { id: number; j: JournalT }) {
|
|||
<span className="badge bg-okbg text-ok ml-2"><CheckCircle2 size={12} /> published to Accounts Summary</span>
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex gap-2 mt-2">
|
||||
<div className="flex gap-2 mt-2 items-center">
|
||||
{user ? <Identity /> : (
|
||||
<input className="input flex-1" placeholder="Approver's name" value={approver}
|
||||
onChange={(e) => setApprover(e.target.value)}
|
||||
disabled={!reviewed} />
|
||||
<button className="btn-primary" disabled={!reviewed || !approver.trim() || approve.isPending}
|
||||
)}
|
||||
<button className="btn-primary" disabled={!reviewed || !approverName.trim() || approve.isPending}
|
||||
onClick={() => approve.mutate()}>
|
||||
{approve.isPending ? <Spinner /> : <BadgeCheck size={15} />} Approve
|
||||
</button>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import { useNavigate } from "react-router-dom";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { Trash2, Play, FileSpreadsheet } from "lucide-react";
|
||||
import { Trash2, Play, FileSpreadsheet, CopyX } from "lucide-react";
|
||||
import { api } from "../../api/client";
|
||||
import { bytes, date, int } from "../../lib/format";
|
||||
import { FileDrop, Section, StatusBadge, Spinner } from "../../components/ui";
|
||||
|
|
@ -8,13 +8,16 @@ import HeaderMapping from "../../components/HeaderMapping";
|
|||
import { useClosing } from "../Closing";
|
||||
|
||||
export default function Upload() {
|
||||
const { id, session } = useClosing();
|
||||
const { id, session, locked } = useClosing();
|
||||
const qc = useQueryClient();
|
||||
const nav = useNavigate();
|
||||
const busy = session.status === "processing" || session.status === "exporting";
|
||||
|
||||
const { data: files } = useQuery({ queryKey: ["files", id], queryFn: () => api.listFiles(id) });
|
||||
const invalidate = () => qc.invalidateQueries({ queryKey: ["files", id] });
|
||||
const invalidate = () => {
|
||||
qc.invalidateQueries({ queryKey: ["files", id] });
|
||||
qc.invalidateQueries({ queryKey: ["session", id] });
|
||||
};
|
||||
|
||||
const upload = useMutation({ mutationFn: (fs: File[]) => api.uploadFiles(id, fs), onSuccess: invalidate });
|
||||
const remove = useMutation({ mutationFn: (fid: number) => api.deleteFile(id, fid), onSuccess: invalidate });
|
||||
|
|
@ -23,15 +26,30 @@ export default function Upload() {
|
|||
onSuccess: () => { qc.invalidateQueries({ queryKey: ["session", id] }); nav(`/closing/${id}`); },
|
||||
});
|
||||
|
||||
const skipped = upload.data?.skipped ?? [];
|
||||
|
||||
const hasInvalid = files?.some((f) => f.status === "invalid");
|
||||
const dates = (files ?? []).flatMap((f) => [f.min_date, f.max_date]).filter(Boolean) as string[];
|
||||
const cover = dates.length ? `${dates.reduce((a, b) => (a < b ? a : b))} → ${dates.reduce((a, b) => (a > b ? a : b))}` : "—";
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<FileDrop disabled={busy || upload.isPending} onFiles={(fs) => upload.mutate(fs)} />
|
||||
<FileDrop disabled={busy || locked || upload.isPending} onFiles={(fs) => upload.mutate(fs)} />
|
||||
{upload.isPending && <p className="text-sm text-subink flex items-center gap-2"><Spinner /> Uploading & validating…</p>}
|
||||
{upload.isError && <p className="text-sm text-bad">{(upload.error as Error).message}</p>}
|
||||
{skipped.length > 0 && (
|
||||
<div className="card border-warn/30 bg-warnbg/40 p-3 text-sm space-y-1">
|
||||
<p className="font-semibold text-ink flex items-center gap-2">
|
||||
<CopyX size={15} className="text-warn" />
|
||||
{skipped.length} file(s) skipped as duplicates — nothing was double-counted
|
||||
</p>
|
||||
<ul className="text-xs text-subink pl-6 list-disc">
|
||||
{skipped.map((s) => (
|
||||
<li key={s.filename}><b>{s.filename}</b> — {s.reason}</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<Section title="Uploaded files" subtitle={`Detected date coverage: ${cover}`}>
|
||||
{!files?.length ? (
|
||||
|
|
@ -55,7 +73,7 @@ export default function Upload() {
|
|||
<td className="td text-xs">{f.marketplace ?? "—"}</td>
|
||||
<td className="td"><StatusBadge status={f.status} /></td>
|
||||
<td className="td text-right">
|
||||
<button className="p-1.5 rounded hover:bg-badbg text-subink hover:text-bad" disabled={busy}
|
||||
<button className="p-1.5 rounded hover:bg-badbg text-subink hover:text-bad" disabled={busy || locked}
|
||||
onClick={() => remove.mutate(f.id)}><Trash2 size={15} /></button>
|
||||
</td>
|
||||
</tr>
|
||||
|
|
@ -66,15 +84,17 @@ export default function Upload() {
|
|||
</Section>
|
||||
|
||||
<div className="card p-4 text-xs text-subink">
|
||||
<p className="font-semibold text-ink mb-1">Column mapping</p>
|
||||
<p className="font-semibold text-ink mb-1">Column mapping & duplicates</p>
|
||||
Headers are auto-matched to the internal schema by normalized name (not position): the raw
|
||||
<span className="num"> date/time · settlement id · type · account type · total </span> columns are
|
||||
required. The pivot sheet in each file is ignored automatically. Files failing validation are flagged above.
|
||||
Re-uploading a file with the same name <b>replaces</b> it; a file whose content is already
|
||||
uploaded (even under another name) is skipped — a month can never count a file twice.
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-sm text-subink">{files?.length ?? 0} file(s) · {hasInvalid ? <span className="text-bad">fix invalid files before processing</span> : "ready"}</p>
|
||||
<button className="btn-primary" disabled={!files?.length || hasInvalid || busy || run.isPending}
|
||||
<button className="btn-primary" disabled={!files?.length || hasInvalid || busy || locked || run.isPending}
|
||||
onClick={() => run.mutate()}>
|
||||
<Play size={16} /> {run.isPending ? "Starting…" : "Run processing"}
|
||||
</button>
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ set -u -o pipefail
|
|||
HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||
cd -- "$HERE" || exit 1
|
||||
|
||||
APP_DIR="$HERE/ar-aging-app"
|
||||
APP_DIR="$(dirname -- "$HERE")" # scripts/ lives inside ar-aging-app/
|
||||
BACKEND_DIR="$APP_DIR/backend"
|
||||
FRONTEND_DIR="$APP_DIR/frontend"
|
||||
VENV_DIR="$APP_DIR/.venv"
|
||||
|
|
@ -81,7 +81,7 @@ say ""
|
|||
# --- 1. prerequisites ---------------------------------------------------------------
|
||||
say "${B}1. Checking prerequisites${R}"
|
||||
[ -d "$BACKEND_DIR" ] || die "Backend folder not found at: $BACKEND_DIR" \
|
||||
"keep start.command in the same folder as ar-aging-app/"
|
||||
"keep start.command inside ar-aging-app/scripts/"
|
||||
command -v python3 >/dev/null 2>&1 || die "python3 was not found." \
|
||||
"install Python 3.11+ from python.org"
|
||||
command -v npm >/dev/null 2>&1 || die "npm was not found." \
|
||||
|
|
@ -1,8 +1,8 @@
|
|||
# Launch the A/R Aging app on Windows.
|
||||
# Ports 8010/5174 because the defaults (8000/5173) are used by the ATS project.
|
||||
# Run with: powershell -ExecutionPolicy Bypass -File start.ps1
|
||||
# Run with: powershell -ExecutionPolicy Bypass -File scripts\start.ps1 (or double-click start.bat)
|
||||
|
||||
$app = $PSScriptRoot
|
||||
$app = Split-Path $PSScriptRoot -Parent # scripts/ lives inside ar-aging-app/
|
||||
$python = "$env:LOCALAPPDATA\anaconda3\envs\Talha\python.exe"
|
||||
|
||||
Start-Process powershell -ArgumentList @(
|
||||
|
|
@ -0,0 +1,162 @@
|
|||
# AR Aging App — Production-Ready Plan
|
||||
|
||||
**Date:** 19 Aug 2026
|
||||
**Goal:** Upload Jan file → Jan data is saved and shown. Upload Feb later → Feb becomes a NEW month dataset; all previous months stay saved and selectable (never merged). Hosted on AWS with backups, automatic exchange rates, and a 5-user login.
|
||||
|
||||
**Bottom line:**
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| AWS hosting cost | **≈ $50–55 / month** |
|
||||
| Exchange-rate API | **$0 / month** (Frankfurter, free) |
|
||||
| Build effort | **~12–17 dev days**, 5 phases, each shippable on its own |
|
||||
|
||||
---
|
||||
|
||||
## 1. What is wrong today (why months "merge" or disappear)
|
||||
|
||||
The app already stores each month as its own closing session in the database — the foundation is right. Four defects break it:
|
||||
|
||||
1. **Double-count bug (verified in code):** re-uploading a file with the same name overwrites it on disk (`ar-aging-app/backend/app/api/routes/files.py:38`) but inserts a **second** database row (`files.py:74`). Processing then parses the same file twice and **doubles every number**.
|
||||
2. **Nothing prevents two sessions for the same month** — creating "January" twice by accident produces two competing datasets, no warning.
|
||||
3. **New months start from zero** — default `opening_mode="zero"` even though carry-forward logic already exists in `routes/ar.py`, so months lose continuity unless someone remembers to switch it.
|
||||
4. **Previous months silently vanish from Accounts Summary** — re-processing clears journal approval (correct audit control), but the summary grid only shows approved months, so the old month disappears with no explanation.
|
||||
|
||||
## 2. Confirmed decisions
|
||||
|
||||
- **Exchange rates:** free **Frankfurter API** — $0, no API key, central-bank (ECB) rates, historical month-end dates. The human confirmation gate (Control C5) stays.
|
||||
- **Login:** simple per-user login, **5 users**. Approver/reviewer names come from the logged-in user.
|
||||
- **File sizes:** uploads are **300–500 MB** each → server sized with **8 GB RAM** for Excel parsing.
|
||||
|
||||
---
|
||||
|
||||
## 3. AWS architecture & monthly cost
|
||||
|
||||
### Recommended: one production server + S3 backups (≈ $50–55/mo)
|
||||
|
||||
**Lightsail 8 GB instance** ($44/mo: 2 vCPU, 8 GB RAM, 160 GB SSD, static IP included) — or equivalent EC2 `t4g.large` (≈ $61/mo with EBS + IPv4). Runs everything via a new `docker-compose.prod.yml`:
|
||||
|
||||
| Container | Role |
|
||||
|---|---|
|
||||
| **nginx** | Serves built React app, proxies `/api` to backend, HTTPS (Let's Encrypt), `client_max_body_size 2g` + long timeouts for big uploads |
|
||||
| **backend** | FastAPI/uvicorn, **single worker** (background jobs are in-process — documented constraint, fine for 5 users) |
|
||||
| **MySQL 8** | Data on instance disk. Code already supports MySQL (`AR_DB_BACKEND=mysql`) and ships `migrate_sqlite_to_mysql.py` |
|
||||
| **backup cron** | Nightly `mysqldump` + `aws s3 sync` of uploads/exports → versioned S3 bucket (lifecycle → cold storage after 90 days), via IAM role (no keys on disk) |
|
||||
|
||||
### Cost breakdown
|
||||
|
||||
| Item | Monthly cost |
|
||||
|---|---|
|
||||
| Lightsail 8 GB (EC2 t4g.large route ≈ $61) | $44 |
|
||||
| S3 backups (~60 GB year one, versioned) | $1.50–3 |
|
||||
| Weekly instance snapshots | $2–4 |
|
||||
| Route 53 hosted zone (optional domain) | $0.50 |
|
||||
| Frankfurter FX API | $0 |
|
||||
| **Total** | **≈ $50–55/mo** |
|
||||
|
||||
**Alternative (managed DB):** same server + **RDS MySQL db.t4g.small** ≈ **$85–100/mo** — buys automated patching + point-in-time restore. Not needed at this scale; upgrading later is a one-line config change (`MYSQL_HOST`).
|
||||
|
||||
**Storage decisions:**
|
||||
- Uploads/exports stay on the **instance disk** — the parsing pipeline needs local file paths; round-tripping 500 MB files through S3 adds complexity for no benefit. S3 = durable **backup**, not primary storage.
|
||||
- Data growth ~1 GB/month (~3.4M transaction rows) — well within sizing.
|
||||
- Firewall inbound restricted to office IPs/VPN as defense-in-depth on top of login.
|
||||
|
||||
### Exchange-rate API pricing (researched)
|
||||
|
||||
| Provider | Free tier | Paid | Verdict |
|
||||
|---|---|---|---|
|
||||
| **Frankfurter** (chosen) | Unlimited, **no API key**, ECB rates, historical dates, self-hostable | — | ✅ $0, ideal for month-end closes |
|
||||
| ExchangeRate-API | 1,500 req/mo, daily updates | Pro $10/mo (30k req, hourly) | fallback provider stub |
|
||||
| Open Exchange Rates | 1,000 req/mo, USD base only | $12/mo | not needed |
|
||||
| Fixer | 100 req/mo, no HTTPS on free | higher | ruled out |
|
||||
|
||||
App usage: ~13 marketplaces × a few fetches/month → even free tiers would never be exceeded.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation phases
|
||||
|
||||
### Phase 1 — Month management & data separation (~3–4 days) ← HIGHEST PRIORITY
|
||||
|
||||
**1.1 Fix double-count bug** — `backend/app/api/routes/files.py`:
|
||||
- Stream uploads to a temp `.part` name; move into place after hashing (also stops a failed upload corrupting an existing good file).
|
||||
- **Content dedup:** SHA-256 match with an existing file in the session → skip, report "identical content already uploaded as X".
|
||||
- **Same filename → UPDATE the existing row** instead of inserting a second one; if month already processed, flag `needs_reprocess` (existing banner handles messaging).
|
||||
- Batch-friendly response `{files, skipped}` so one duplicate doesn't fail a 13-file upload; UI shows skipped/replaced notices.
|
||||
- One-time `cli.py dedupe-files` cleanup for existing bad rows, then a unique index on (session_id, filename).
|
||||
|
||||
**1.2 One session per month (guide, don't hard-block)** — `routes/sessions.py`: 409 on duplicate `reporting_month` unless `allow_duplicate: true`; `NewClosing.tsx` shows "A closing for 2026-02 already exists — Open it | Create anyway".
|
||||
|
||||
**1.3 Default carry-forward** — `NewClosing.tsx`: default `openingMode="carry_forward"` when prior processed months exist (backend logic already exists); default month = month after latest close.
|
||||
|
||||
**1.4 Month selector UX ("show previous options"):**
|
||||
- Sessions list ordered as a month timeline with a `journal_approved` flag per row.
|
||||
- New `MonthSwitcher` dropdown in the closing header (month · name · status) — jump between months from any screen.
|
||||
- Dashboard shows "published / processed-but-unpublished" and flags duplicate months.
|
||||
|
||||
**1.5 Stop previous months vanishing from Accounts Summary:**
|
||||
- `accounts_summary.py` also returns pending months (processed/blocked but unapproved, incl. "approval cleared by re-processing").
|
||||
- Grid renders them as greyed columns linking to the journal tab ("2026-01 processed but unpublished — re-approve the journal"). Audit control untouched.
|
||||
|
||||
**1.6 Read-only lock after completion** — new `ensure_editable()` guard on all mutating endpoints (409 on completed months); new `POST /sessions/{id}/reopen`; frontend disables edit controls on locked months.
|
||||
|
||||
**Tests:** filename replace, sha256 skip, duplicate-month 409, completed-session 409s.
|
||||
|
||||
### Phase 2 — Login, 5 users (~2 days)
|
||||
|
||||
- New `User` table (username, display name, bcrypt password hash); users created via CLI (`add-user` / `set-password`) — no self-signup.
|
||||
- `POST /api/auth/login` → signed bearer token (12 h expiry, new `AR_SECRET_KEY` env); every route requires login except `/api/health` + login.
|
||||
- **Real identity in sign-offs:** journal review/approve, FX confirm, control verify, payout entry all record the logged-in user — free-text "your name" boxes removed.
|
||||
- Frontend: `Login.tsx`, token auto-attached, 401 → redirect to login, user + logout in sidebar.
|
||||
- New deps: `passlib[bcrypt]`, `itsdangerous`.
|
||||
|
||||
### Phase 3 — AWS deployment (~3–5 days incl. migration dry-run)
|
||||
|
||||
1. Production backend image (no `--reload`, `--proxy-headers`, single worker documented).
|
||||
2. Production frontend image: multi-stage `node:20` build → `nginx` serving static + API proxy.
|
||||
3. New `docker-compose.prod.yml` + `example.env.production` (`AR_DB_BACKEND=mysql`, `AR_CORS_ORIGINS=https://<domain>`, `AR_SECRET_KEY`, FX vars); HTTPS via certbot; DNS → static IP.
|
||||
4. **Data migration:** MySQL up → schema auto-created → run existing `migrate_sqlite_to_mysql.py` → verify per-table row counts + to-the-cent reconciliation spot check → cut over; archive SQLite file to S3. **Timed dry run first** (3.4M rows/month).
|
||||
5. Backups: nightly dump + S3 sync, weekly snapshots, S3 versioning + lifecycle.
|
||||
6. Deploy runbook: `git pull && docker compose -f docker-compose.prod.yml up -d --build`.
|
||||
|
||||
### Phase 4 — Exchange-rate service (~2 days)
|
||||
|
||||
- New `backend/app/services/fx_service.py`: provider abstraction — **Frankfurter** default (`api.frankfurter.dev/v1/{date}?base=USD&symbols=CAD,GBP,AUD,EUR,PLN,SEK,TRY`), paid-provider stub via `FX_PROVIDER` env.
|
||||
- Rates **inverted to USD-per-local** (the app's convention) — documented + locked with a test; one inversion mistake would mis-state every non-USD receivable.
|
||||
- Month-end fetch seeds `fx_rates` per session **unconfirmed** → Control C5 still blocks the close until a human confirms — workflow unchanged, just pre-filled. Daily fetch fills `fx_rates_daily`.
|
||||
- Cache table makes re-fetch idempotent + works offline after first fetch; provider failure → clear 502 "enter rates manually", never a silent default.
|
||||
- UI: "Fetch month-end rates" button in the C5 panel (`Controls.tsx`); "Fetch daily rates" in `ArLedger.tsx`. Update "no third-party egress" copy (only currency codes are sent, never financial data).
|
||||
- ⚠️ Verify Frankfurter publishes TRY; if not, Turkey stays manual (C5 still enforces) or paid stub takes over.
|
||||
|
||||
### Phase 5 — Hardening (~1–2 days)
|
||||
|
||||
1. **Stale-job recovery:** months stuck "processing" after a restart → marked error with a re-run message (otherwise a deploy mid-job blocks the month forever).
|
||||
2. **Retention:** auto-purge old generated **exports only** (`AR_RETENTION_DAYS`, currently dead config) — uploaded source files never auto-deleted (audit source).
|
||||
3. Health check: DB ping + disk-writable.
|
||||
4. Logging: request timing + job start/finish/fail, shipped via Docker logs.
|
||||
|
||||
Deferred nice-to-haves: CI pipeline, Sentry, login rate-limiting, Alembic, S3-primary storage, audit-log table, MFA.
|
||||
|
||||
---
|
||||
|
||||
## 5. Verification (acceptance checks)
|
||||
|
||||
- Upload same CSV twice → one file row, totals unchanged; renamed-identical file → skipped with message.
|
||||
- Close Jan → create Feb → warns nothing, carries forward Jan balances; Jan stays selectable, read-only, unchanged; both months on Accounts Summary.
|
||||
- Fetch month-end rates → unconfirmed → C5 fails → Confirm all → C5 passes; EUR rate checked against ECB published figure.
|
||||
- Unauthenticated API call → 401; all 5 users can log in; approvals record the real approver name.
|
||||
- Prod compose rehearsed locally with a 500 MB file (memory watched) **before buying the instance**; restart mid-processing recovers; nightly backup object appears in S3; restore drill succeeds.
|
||||
- All existing tests pass, incl. Jan-2026 reconciliation integration test **to the cent** after MySQL migration.
|
||||
|
||||
## 6. Key risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| SQLite→MySQL migration (~3.4M rows/month) | Timed dry run, per-table counts, cent-level reconciliation before cutover |
|
||||
| FX rate orientation (USD-per-local vs local-per-USD) | Fixture test against a known EUR rate |
|
||||
| Read-only lock missing an endpoint | Sweep all mutating routes during implementation |
|
||||
| 8 GB RAM sizing assumption | Validate with the real 283 MB test workbook in local rehearsal before instance purchase |
|
||||
|
||||
---
|
||||
|
||||
*Pricing sources: [ExchangeRate-API](https://www.exchangerate-api.com/#pricing) · [Open Exchange Rates](https://openexchangerates.org/signup) · [Frankfurter](https://frankfurter.dev/) · AWS Lightsail/EC2/RDS public pricing, Aug 2026. Costs are USD estimates, on-demand.*
|
||||
Loading…
Reference in New Issue