From b7cc1bac710a61c5bd44882a04775ef2eaeb155e Mon Sep 17 00:00:00 2001 From: "bahawal.baloch" Date: Wed, 12 Aug 2026 13:42:00 +0500 Subject: [PATCH] Add hosted multi-user mode: accounts, MySQL, Docker, brand identity Converts the single-user localhost tool into a hosted application while leaving local mode intact. The pipeline moves out of serve.py into app/services/{analysis,exporters}.py, which import only the standard library and ppcbudget. serve.py becomes a thin shim over them, so it still runs offline with no dependency beyond openpyxl and there is one copy of the analysis rather than two. ppcbudget and run_report.py are otherwise untouched. app/ is a FastAPI application serving the same dashboard behind sign-in: - Open signup with email confirmation, forgot/reset, argon2id hashing, and opaque DB-backed session cookies (HttpOnly, SameSite=Lax, Secure derived from APP_BASE_URL so TLS is later a config change, not a rewrite). - Signup and forgot-password answer identically whether or not an address exists; login gives one generic message for unknown, wrong, locked and disabled alike. Email tokens are HMAC'd at rest and spent by a single atomic UPDATE, and /verify and /reset are inert pages that POST the token -- a mail scanner following the link cannot burn it. - Routes are def, not async def: the pipeline is CPU-bound and would otherwise block the event loop. A semaphore bounds concurrent analyses. - The process-wide SESSION global becomes a workspace per account, with per-workspace locks, size caps and idle eviction. The old code held one lock for the whole analysis; per-user locks fix that by construction. - Accounts live in three oob_-prefixed tables. The connection URL is built with URL.create, since the password contains ? and # and a hand-built DSN truncates it there. - Minimal /admin page for enabling, disabling and signing accounts out. Hardening: server-side upload extension and size limits, an Origin guard on writes, correlation-id 500s instead of echoed exceptions, and a CSP. Every page script is external because script-src 'self' blocks inline blocks. Restyled to the Utopia Brands guidelines. Brand swatches sit verbatim in the --u-* properties and everything derives from them, including the Excel report. --red and --amber are deliberately not from the guide: it covers identity, not function, and has no warning colour, but this dashboard exists to show campaigns going dark. The four data colours were checked for colour-vision separation across every pair on both surfaces, and the day heatmap is now one hue getting darker rather than a rainbow. The claim that nothing is uploaded anywhere stays true only for local mode, so the dashboard copy and the README now say so. Tests: tests/test_auth_smoke.py covers the account lifecycle plus upload, analyse and export against SQLite with mail captured, so it needs no MySQL and no network. tests/synthetic.py builds a small stand-in export. Co-Authored-By: Claude Opus 5 (1M context) --- .dockerignore | 41 +++++ .env.example | 54 ++++++ Dockerfile | 47 +++++ README.md | 131 ++++++++++++-- app/__init__.py | 11 ++ app/bootstrap.py | 66 +++++++ app/config.py | 133 ++++++++++++++ app/db.py | 124 +++++++++++++ app/deps.py | 73 ++++++++ app/errors.py | 80 +++++++++ app/main.py | 161 +++++++++++++++++ app/models.py | 106 +++++++++++ app/routers/__init__.py | 1 + app/routers/admin.py | 96 ++++++++++ app/routers/analysis.py | 113 ++++++++++++ app/routers/auth.py | 310 ++++++++++++++++++++++++++++++++ app/routers/pages.py | 102 +++++++++++ app/schemas.py | 84 +++++++++ app/security.py | 94 ++++++++++ app/services/__init__.py | 6 + app/services/analysis.py | 95 ++++++++++ app/services/email_service.py | 149 +++++++++++++++ app/services/exporters.py | 85 +++++++++ app/services/ratelimit.py | 59 ++++++ app/services/store.py | 226 +++++++++++++++++++++++ docker-compose.yml | 36 ++++ ppcbudget/excelout.py | 78 ++++---- ppcbudget/payload.py | 14 +- requirements.txt | 25 +++ serve.py | 139 ++------------ tests/synthetic.py | 72 ++++++++ tests/test_auth_smoke.py | 328 ++++++++++++++++++++++++++++++++++ web/admin.html | 52 ++++++ web/app.js | 85 ++++++--- web/auth.js | 83 +++++++++ web/fonts/README.md | 35 ++++ web/forgot.html | 42 +++++ web/index.html | 20 ++- web/login.html | 53 ++++++ web/page-admin.js | 80 +++++++++ web/page-forgot.js | 11 ++ web/page-login.js | 39 ++++ web/page-reset.js | 22 +++ web/page-signup.js | 19 ++ web/page-verify.js | 40 +++++ web/reset.html | 52 ++++++ web/signup.html | 59 ++++++ web/styles.css | 208 +++++++++++++++------ web/verify.html | 53 ++++++ 49 files changed, 3830 insertions(+), 262 deletions(-) create mode 100644 .dockerignore create mode 100644 .env.example create mode 100644 Dockerfile create mode 100644 app/__init__.py create mode 100644 app/bootstrap.py create mode 100644 app/config.py create mode 100644 app/db.py create mode 100644 app/deps.py create mode 100644 app/errors.py create mode 100644 app/main.py create mode 100644 app/models.py create mode 100644 app/routers/__init__.py create mode 100644 app/routers/admin.py create mode 100644 app/routers/analysis.py create mode 100644 app/routers/auth.py create mode 100644 app/routers/pages.py create mode 100644 app/schemas.py create mode 100644 app/security.py create mode 100644 app/services/__init__.py create mode 100644 app/services/analysis.py create mode 100644 app/services/email_service.py create mode 100644 app/services/exporters.py create mode 100644 app/services/ratelimit.py create mode 100644 app/services/store.py create mode 100644 docker-compose.yml create mode 100644 requirements.txt create mode 100644 tests/synthetic.py create mode 100644 tests/test_auth_smoke.py create mode 100644 web/admin.html create mode 100644 web/auth.js create mode 100644 web/fonts/README.md create mode 100644 web/forgot.html create mode 100644 web/login.html create mode 100644 web/page-admin.js create mode 100644 web/page-forgot.js create mode 100644 web/page-login.js create mode 100644 web/page-reset.js create mode 100644 web/page-signup.js create mode 100644 web/page-verify.js create mode 100644 web/reset.html create mode 100644 web/signup.html create mode 100644 web/verify.html diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..54f15e0 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,41 @@ +.git +.gitignore + +# Secrets arrive at runtime via env_file. Baking them into a layer would leave +# them extractable from the image forever. +.env +.env.* +!.env.example + +__pycache__/ +*.py[cod] +.venv/ +venv/ +env/ +ENV/ + +# Customer exports; also keeps the build context small. +data/ +reports/ +*.xlsx +*.xlsm +*.xls +*.csv +~$* + +.pytest_cache/ +.mypy_cache/ +.ruff_cache/ +.coverage +htmlcov/ + +.idea/ +.vscode/ +*.swp +.DS_Store +Thumbs.db + +Dockerfile +.dockerignore +docker-compose.yml +README.md diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..4230ff7 --- /dev/null +++ b/.env.example @@ -0,0 +1,54 @@ +# Copy to .env and fill in. .env is gitignored; this file is not. +# Only hosted mode reads any of this -- serve.py and run_report.py ignore it entirely. + +# ─── Application ─────────────────────────────────────────────────── +# The public origin. Verification and reset links in emails are built from this, +# so if it is wrong every emailed link points somewhere unreachable. No trailing slash. +APP_BASE_URL=http://localhost:8000 + +# python -c "import secrets; print(secrets.token_urlsafe(48))" +# Rotating this signs everyone out and invalidates all pending email links. +SECRET_KEY=change-me-to-at-least-32-random-characters + +# Leave false while serving plain HTTP. If this is true over http:// the browser +# silently discards the session cookie: login appears to work and every later +# request 401s. Blank means "derive it from APP_BASE_URL", which is usually right. +COOKIE_SECURE=false + +ENV=prod +LOG_LEVEL=INFO + +# ─── Database - MySQL ────────────────────────────────────────────── +MYSQL_HOST=your-instance.rds.amazonaws.com +MYSQL_PORT=3306 +MYSQL_USER=your_user +MYSQL_PASSWORD='your password; quote it if it has # or ? in it' +MYSQL_DATABASE=your_database +MYSQL_SLOW_QUERY_MS=500 +MYSQL_POOL_SIZE=10 +MYSQL_POOL_RECYCLE=3600 + +# ─── Default admin (seeded on startup) ───────────────────────────── +# Created pre-verified on first boot. The password is NOT overwritten on later +# boots, so rotating it in the app sticks. Set ADMIN_RESET_PASSWORD=true to force it. +ADMIN_EMAIL=admin@example.com +ADMIN_PASSWORD=change-me +ADMIN_RESET_PASSWORD=false + +# ─── Email ───────────────────────────────────────────────────────── +# "api" posts to EMAIL_ENDPOINT. "console" just logs the message and its link, +# which is how you exercise signup/verify/reset locally without sending real mail. +EMAIL_PROVIDER=api +EMAIL_ENDPOINT=http://your-mail-host:8000/mail/send +EMAIL_API_KEY=your-key +EMAIL_FROM_NAME=PPC Dashboard + +# ─── Signup ──────────────────────────────────────────────────────── +SIGNUP_ENABLED=true +# Comma-separated. Blank means any domain may register. +SIGNUP_ALLOWED_DOMAINS= + +# ─── Limits ──────────────────────────────────────────────────────── +# Blank WORKSPACE_ROOT means the system temp dir. The container sets /srv/work. +WORKSPACE_ROOT= +MAX_CONCURRENT_ANALYSES=2 diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..84936cd --- /dev/null +++ b/Dockerfile @@ -0,0 +1,47 @@ +# Hosted mode. The local serve.py is not what runs here -- see the CMD. +FROM python:3.13-slim + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PIP_DISABLE_PIP_VERSION_CHECK=1 + +WORKDIR /srv/app + +# Created before the COPYs so --chown has someone to point at. +RUN groupadd --system app && useradd --system --gid app --uid 10001 --home /srv/app app + +# Dependencies first: this layer stays cached until requirements.txt changes. +# argon2-cffi, cryptography and openpyxl all ship manylinux wheels, so slim +# needs no compiler. If a future pin lacks one, add a builder stage rather than +# putting gcc in the runtime image. +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY --chown=app:app ppcbudget/ ./ppcbudget/ +COPY --chown=app:app app/ ./app/ +COPY --chown=app:app web/ ./web/ +COPY --chown=app:app tests/ ./tests/ +# Carried along only so `docker compose exec app python run_report.py` works for +# debugging. serve.py is never the container's entrypoint, so its localhost bind +# and its webbrowser.open are unreachable here. +COPY --chown=app:app run_report.py serve.py ./ + +# Scratch space for per-user uploads, writable by the non-root user. +RUN mkdir -p /srv/work && chown app:app /srv/work +ENV WORKSPACE_ROOT=/srv/work + +USER app +EXPOSE 8000 + +# No curl in the image: the interpreter is already here and is one fewer thing +# to patch. /healthz deliberately does not touch MySQL -- see app/routers/pages.py. +HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \ + CMD ["python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=3).status == 200 else 1)"] + +# One worker is load-bearing: uploads and the last analysis live in this +# process's memory, so a second worker would answer half the requests without +# them. Add --proxy-headers --forwarded-allow-ips= only once a trusted +# proxy is in front; with the port exposed directly, it would let a client spoof +# X-Forwarded-For and walk past the per-IP rate limits. +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"] diff --git a/README.md b/README.md index 2a725ed..3d5a9e5 100644 --- a/README.md +++ b/README.md @@ -3,11 +3,14 @@ Finds the campaigns that keep running out of budget, how long they were dark, how often it happened, and what it plausibly cost. -Two ways to use it. Both run the same analysis code, so they can never -disagree — the dashboard just makes it explorable and the report makes it -shareable. +Three ways to use it. All of them run the same analysis code — `ppcbudget/` +via `app/services/analysis.py` — so they can never disagree. The dashboard +makes it explorable, the report makes it shareable, and the hosted version +makes it available to a team. -## The dashboard +## Local mode — your files never leave your machine + +### The dashboard ```bash python3 serve.py @@ -18,16 +21,24 @@ Sort and filter by any column, click a campaign for its full timeline and outage list, and download the Excel or CSV version from the header. Add `--preload` to pick up whatever is already sitting in `data/` on startup. -The server binds to localhost only; nothing is uploaded anywhere. +The server binds to localhost only; nothing is uploaded anywhere, and there are +no accounts, no database and nothing to install beyond `openpyxl`. -## The Excel report +### The Excel report ```bash python3 run_report.py ``` -Put your exports in `data/` first. The report lands in `reports/`. Nothing to -install — it uses `openpyxl`, which you already have. +Put your exports in `data/` first. The report lands in `reports/`. Same +dependencies as above: just `openpyxl`. + +## Hosted mode — accounts, on a server + +The hosted version is the same dashboard with sign-in in front of it. **Files +you upload here do go to the server**, where they are analysed and then deleted +when you sign out or go idle; nobody else with an account can see them. Accounts +live in MySQL. See [Running the hosted version](#running-the-hosted-version). ## Loading more than one day @@ -156,16 +167,110 @@ zero would become a fact the moment someone summed the column. budget, so paused minutes are removed from both the loss total and the in-budget denominator that sets the spend rate. +## Running the hosted version + +The hosted app is FastAPI (`app/`), serving the same dashboard behind sign-in. +Accounts, sessions and email tokens live in MySQL in three `oob_`-prefixed +tables. Uploads and the last analysis stay in memory and in a scratch directory, +keyed by user — they are not written to the database and do not survive a +restart. + +### Configuration + +Copy `.env.example` to `.env` and fill it in. Four settings have no sensible +default and matter more than the rest: + +| Setting | Why it matters | +| --- | --- | +| `APP_BASE_URL` | Verification and reset links are built from it. Point it at wherever people actually reach the app, or every emailed link is dead. No trailing slash. | +| `SECRET_KEY` | Keys the session-cookie and email-token fingerprints. Generate with `python -c "import secrets; print(secrets.token_urlsafe(48))"`. Rotating it signs everyone out and voids pending email links. | +| `COOKIE_SECURE` | Leave `false` over plain HTTP. Set it `true` there and the browser silently discards the session cookie, so signing in appears to do nothing. Blank derives it from `APP_BASE_URL`. | +| `EMAIL_PROVIDER` | `api` posts to `EMAIL_ENDPOINT`; `console` just logs the message and its link, which is how to exercise signup and reset without sending real mail. | + +`ADMIN_EMAIL` and `ADMIN_PASSWORD` seed an administrator on first startup, +already confirmed. Later startups repair the account's flags but leave the +password alone, so rotating it in the app sticks — set `ADMIN_RESET_PASSWORD=true` +for one boot if you need to force it back. + +### With Docker + +```bash +docker compose up -d --build +docker compose ps # healthy +curl -s localhost:8000/readyz # {"db":"ok"} proves it can reach MySQL +``` + +The compose file starts the app only; the database is whatever `MYSQL_HOST` +points at. The EC2 instance's security group has to be allowed inbound on 3306 +at RDS. + +### Without Docker + +```bash +python3 -m venv .venv && .venv/bin/pip install -r requirements.txt +.venv/bin/python -m app.db # create the tables, seed the admin +.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 1 +``` + +**One worker is load-bearing.** Uploads and the last analysis live in the +process's memory, so a second worker would answer half the requests without +them. Growing past one box means moving that state out of memory first. + +### Branding + +The dashboard, the auth pages and the Excel report all follow the Utopia Brands +guidelines. The brand swatches live verbatim in the `--u-*` custom properties at +the top of `web/styles.css` and everything else derives from them; the Excel +palette is the matching set of constants at the top of `ppcbudget/excelout.py`. + +Two colours are deliberately **not** from the guide. The guide covers identity, +not function, and has no warning colour — but this dashboard exists to show +campaigns going dark, so out-of-budget has to read as a problem. `--red` and +`--amber` are tuned to sit against the brand greens and are only ever used to +encode state. + +Headings are set in Belleza and body copy in Neue Montreal, per the guide's +typeface hierarchy. Neither font file ships with the repository — see +[web/fonts/README.md](web/fonts/README.md) for how to add them. Until they are +there, both fall back to close system faces and the `@font-face` request for +Belleza 404s harmlessly. + +The mark in the top bar is a placeholder built from the two brand primaries. +Replace it with the real emblem when the SVG is available rather than redrawing +it — `.mark` in `web/styles.css`. + +### Schema changes + +Tables are created with `create_all` on startup. That creates missing tables but +never alters existing ones, so the first time a column is added you either run +the `ALTER` by hand or adopt Alembic then. Three tables did not justify a +migrations tree up front. + +### Before it faces the internet + +The app is built so TLS is a configuration change rather than a rewrite: set +`APP_BASE_URL=https://…` and the session cookie picks up `Secure` on its own. +Until then, passwords and session cookies cross the network in the clear. If you +put a proxy in front, add `--proxy-headers --forwarded-allow-ips=` to +the uvicorn command — but not while port 8000 is exposed directly, where it +would let a client spoof `X-Forwarded-For` and walk past the per-IP rate limits. + ## Tests ```bash -python3 tests/test_golden.py +python3 tests/test_golden.py # the analysis +python3 tests/test_auth_smoke.py # the hosted app ``` -34 checks: frozen totals from the reference export, structural invariants, -overlapping-export handling, action classification, and edge cases. The -important ones are `test_chain_breaks_canary` and -`test_amazon_pacing_rows_are_not_actions`. +`test_golden.py` is 34 checks: frozen totals from the reference export, +structural invariants, overlapping-export handling, action classification, and +edge cases. The important ones are `test_chain_breaks_canary` and +`test_amazon_pacing_rows_are_not_actions`. It needs the reference export in +`data/`; the 15 synthetic-fixture checks run without it. + +`test_auth_smoke.py` is 53 checks covering the whole account lifecycle plus +upload, analyse and export. It runs against in-memory SQLite with mail captured +rather than sent, so it needs no MySQL, no network and no configuration. The export is written newest-first, so rows sharing the same minute are also newest-first and must be reversed before the state machine walks them. Sorting diff --git a/app/__init__.py b/app/__init__.py new file mode 100644 index 0000000..399045f --- /dev/null +++ b/app/__init__.py @@ -0,0 +1,11 @@ +"""Hosted, multi-user front end for the out-of-budget analysis. + +`app.services` holds the analysis and export code, and imports nothing but the +standard library and `ppcbudget` -- that is what lets serve.py keep running +offline with no third-party dependencies beyond openpyxl. + +Everything else in this package is the web application: FastAPI routers, the +MySQL-backed account store, and the per-user workspaces. +""" + +__version__ = "2.0.0" diff --git a/app/bootstrap.py b/app/bootstrap.py new file mode 100644 index 0000000..c561b3c --- /dev/null +++ b/app/bootstrap.py @@ -0,0 +1,66 @@ +"""Seed the administrator account named in the environment.""" + +from __future__ import annotations + +import logging + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.config import Settings +from app.models import User, utcnow +from app.security import hash_password + +log = logging.getLogger(__name__) + + +def seed_admin(db: Session, s: Settings) -> User | None: + """Idempotent. Creates the admin pre-verified on first run; on later runs it + repairs the flags but leaves the password alone -- otherwise every restart + would revert a rotated password and the stale .env value would be a + permanent way in. ADMIN_RESET_PASSWORD=true is the deliberate override.""" + if not (s.ADMIN_EMAIL and s.ADMIN_PASSWORD): + log.info("ADMIN_EMAIL/ADMIN_PASSWORD not set; skipping admin seed") + return None + + email = s.ADMIN_EMAIL.strip().lower() + user = db.scalar(select(User).where(User.email == email)) + + if user is None: + user = User( + email=email, + name="Administrator", + password_hash=hash_password(s.ADMIN_PASSWORD.get_secret_value()), + is_admin=True, + is_active=True, + email_verified_at=utcnow(), + ) + db.add(user) + db.commit() + log.info("seeded admin account %s", email) + return user + + changed = [] + if not user.is_admin: + user.is_admin = True + changed.append("is_admin") + if not user.is_active: + user.is_active = True + changed.append("is_active") + if user.email_verified_at is None: + user.email_verified_at = utcnow() + changed.append("email_verified_at") + if user.locked_until is not None or user.failed_login_count: + user.locked_until = None + user.failed_login_count = 0 + changed.append("lockout cleared") + if s.ADMIN_RESET_PASSWORD: + user.password_hash = hash_password(s.ADMIN_PASSWORD.get_secret_value()) + changed.append("password reset from ADMIN_PASSWORD") + + if changed: + db.commit() + log.info("admin account %s updated: %s", email, ", ".join(changed)) + else: + log.info("admin account %s already present", email) + return user diff --git a/app/config.py b/app/config.py new file mode 100644 index 0000000..78c1e1b --- /dev/null +++ b/app/config.py @@ -0,0 +1,133 @@ +"""Every knob the hosted app has, read once from the environment. + +Secrets are `SecretStr` so they render as `**********` in tracebacks and in any +accidental `print(settings)`. The validator refuses to start on the two +misconfigurations that otherwise fail silently and confusingly at runtime. +""" + +from __future__ import annotations + +from functools import lru_cache +from typing import Literal + +from pydantic import SecretStr, field_validator, model_validator +from pydantic_settings import BaseSettings, SettingsConfigDict + + +def _unquote(value: str) -> str: + """Drop one matched pair of surrounding quotes. + + `docker run --env-file` passes values through literally, so a .env line + reading MYSQL_PASSWORD='secret' arrives with the quotes still attached and + the database answers "Access denied" with nothing to explain why. dotenv and + `docker compose env_file:` both strip them, so this only ever fires on the + raw docker run path -- but that failure is expensive to diagnose and this is + cheap. A value that genuinely begins and ends with the same quote character + would need it doubled. + """ + if len(value) >= 2 and value[0] == value[-1] and value[0] in ("'", '"'): + return value[1:-1] + return value + + +class Settings(BaseSettings): + model_config = SettingsConfigDict( + env_file=".env", env_file_encoding="utf-8", + extra="ignore", case_sensitive=False, + ) + + # ---- application ------------------------------------------------------- + APP_NAME: str = "PPC Out-of-Budget Dashboard" + # Verification and reset links are built from this. If it is wrong, every + # emailed link points somewhere the recipient cannot reach. + APP_BASE_URL: str = "http://localhost:8000" + SECRET_KEY: SecretStr + ENV: Literal["dev", "prod"] = "prod" + LOG_LEVEL: str = "INFO" + + # ---- database ---------------------------------------------------------- + MYSQL_HOST: str + MYSQL_PORT: int = 3306 + MYSQL_USER: str + MYSQL_PASSWORD: SecretStr + MYSQL_DATABASE: str + MYSQL_POOL_SIZE: int = 5 + MYSQL_POOL_RECYCLE: int = 3600 + MYSQL_SLOW_QUERY_MS: int = 500 + + # ---- sessions & auth --------------------------------------------------- + SESSION_COOKIE_NAME: str = "oob_session" + SESSION_TTL_DAYS: int = 14 + # None means "derive from APP_BASE_URL". Setting it True over plain http + # makes the browser silently drop the cookie -- login then does nothing. + COOKIE_SECURE: bool | None = None + VERIFY_TOKEN_TTL_HOURS: int = 24 + RESET_TOKEN_TTL_MINUTES: int = 60 + MAX_FAILED_LOGINS: int = 5 + LOCKOUT_MINUTES: int = 15 + MIN_PASSWORD_LENGTH: int = 10 + MAX_PASSWORD_LENGTH: int = 128 + SIGNUP_ENABLED: bool = True + SIGNUP_ALLOWED_DOMAINS: str = "" # comma separated; blank means any + + # ---- admin bootstrap --------------------------------------------------- + ADMIN_EMAIL: str | None = None + ADMIN_PASSWORD: SecretStr | None = None + # Off by default: otherwise every restart reverts a rotated password and the + # stale .env value becomes a permanent backdoor. + ADMIN_RESET_PASSWORD: bool = False + + # ---- email ------------------------------------------------------------- + EMAIL_PROVIDER: Literal["api", "console"] = "api" + EMAIL_ENDPOINT: str | None = None + EMAIL_API_KEY: SecretStr | None = None + EMAIL_FROM_NAME: str = "PPC Dashboard" + EMAIL_TIMEOUT_S: float = 10.0 + + # ---- workspaces & limits ---------------------------------------------- + WORKSPACE_ROOT: str | None = None # None -> system temp dir + MAX_UPLOAD_BYTES: int = 200 * 1024 * 1024 # one file + MAX_WORKSPACE_BYTES: int = 400 * 1024 * 1024 # per user, cumulative + MAX_TOTAL_WORKSPACE_BYTES: int = 4 * 1024 ** 3 + MAX_ACTIVE_WORKSPACES: int = 20 + WORKSPACE_IDLE_TTL_MIN: int = 30 + MAX_CONCURRENT_ANALYSES: int = 2 + + @field_validator("MYSQL_PASSWORD", "ADMIN_PASSWORD", "EMAIL_API_KEY", + "SECRET_KEY", mode="before") + @classmethod + def _strip_quotes(cls, v): + return _unquote(v) if isinstance(v, str) else v + + @property + def base_url(self) -> str: + return self.APP_BASE_URL.rstrip("/") + + @property + def cookie_secure(self) -> bool: + if self.COOKIE_SECURE is not None: + return self.COOKIE_SECURE + return self.base_url.startswith("https://") + + @property + def allowed_signup_domains(self) -> set[str]: + return {d.strip().lower() for d in self.SIGNUP_ALLOWED_DOMAINS.split(",") if d.strip()} + + @model_validator(mode="after") + def _check(self) -> "Settings": + if len(self.SECRET_KEY.get_secret_value()) < 32: + raise ValueError( + "SECRET_KEY must be at least 32 characters. Generate one with: " + 'python -c "import secrets; print(secrets.token_urlsafe(48))"' + ) + if self.EMAIL_PROVIDER == "api" and not (self.EMAIL_ENDPOINT and self.EMAIL_API_KEY): + raise ValueError( + "EMAIL_PROVIDER=api needs EMAIL_ENDPOINT and EMAIL_API_KEY. " + "Set EMAIL_PROVIDER=console to log emails instead of sending them." + ) + return self + + +@lru_cache(maxsize=1) +def get_settings() -> Settings: + return Settings() diff --git a/app/db.py b/app/db.py new file mode 100644 index 0000000..25e7c6e --- /dev/null +++ b/app/db.py @@ -0,0 +1,124 @@ +"""Engine, session factory, and schema creation. + +The connection URL is built with `URL.create`, never an f-string. The staged +MySQL password contains `?` and `#`, which start the query and fragment +components of a URL -- a hand-built DSN silently truncates the password there +and MySQL answers "Access denied" with nothing to suggest why. +""" + +from __future__ import annotations + +import logging +import time +from collections.abc import Iterator + +from sqlalchemy import create_engine, event, text +from sqlalchemy.engine import URL, Engine +from sqlalchemy.exc import OperationalError, ProgrammingError +from sqlalchemy.orm import Session, sessionmaker + +from app.config import Settings, get_settings +from app.models import Base + +log = logging.getLogger(__name__) +slow_log = logging.getLogger("app.db.slow") + + +def build_url(s: Settings) -> URL: + return URL.create( + drivername="mysql+pymysql", + username=s.MYSQL_USER, + password=s.MYSQL_PASSWORD.get_secret_value(), # raw; URL.create encodes it + host=s.MYSQL_HOST, + port=s.MYSQL_PORT, + database=s.MYSQL_DATABASE, # hyphens are legal in a URL path segment + query={"charset": "utf8mb4"}, + ) + + +def build_engine(s: Settings) -> Engine: + engine = create_engine( + build_url(s), + pool_size=s.MYSQL_POOL_SIZE, + max_overflow=5, + pool_recycle=s.MYSQL_POOL_RECYCLE, # must stay under RDS wait_timeout + pool_pre_ping=True, # survives idle kills and failovers + pool_timeout=10, + future=True, + ) + _install_slow_query_log(engine, s.MYSQL_SLOW_QUERY_MS) + return engine + + +def _install_slow_query_log(engine: Engine, threshold_ms: int) -> None: + """Warn about queries slower than the threshold. RDS is across a network, + so this is worth having. Logs the statement only -- the bind parameters + hold password hashes, token fingerprints and email addresses.""" + + @event.listens_for(engine, "before_cursor_execute") + def _start(conn, cursor, statement, parameters, context, executemany): + conn.info.setdefault("_q_start", []).append(time.perf_counter()) + + @event.listens_for(engine, "after_cursor_execute") + def _end(conn, cursor, statement, parameters, context, executemany): + stack = conn.info.get("_q_start") + if not stack: + return + elapsed_ms = (time.perf_counter() - stack.pop()) * 1000 + if elapsed_ms >= threshold_ms: + slow_log.warning("slow query %.0fms: %s", elapsed_ms, + " ".join(statement.split())[:400]) + + +_settings = get_settings() +engine: Engine = build_engine(_settings) +SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False, + future=True) + + +def get_db() -> Iterator[Session]: + """FastAPI dependency. One session per request, always closed.""" + db = SessionLocal() + try: + yield db + finally: + db.close() + + +def create_schema(bind: Engine | None = None) -> None: + """Create any missing tables. + + Note what this does not do: it never ALTERs an existing table. Adding a + column later means running the ALTER by hand or adopting Alembic then. + """ + target = bind or engine + try: + Base.metadata.create_all(bind=target) + except (OperationalError, ProgrammingError) as exc: + log.critical( + "Could not create tables in %r as %r. Check the database exists, the " + "credentials are right, this host can reach it, and the user holds " + "CREATE privileges. Underlying error: %s", + _settings.MYSQL_DATABASE, _settings.MYSQL_USER, exc, + ) + raise + + +def ping(bind: Engine | None = None) -> None: + """Raise if the database is unreachable. Used by /readyz.""" + with (bind or engine).connect() as conn: + conn.execute(text("SELECT 1")) + + +if __name__ == "__main__": # python -m app.db -- create tables and seed the admin + from app.bootstrap import seed_admin + + logging.basicConfig(level=_settings.LOG_LEVEL, + format="%(levelname)s %(name)s: %(message)s") + log.info("connecting to %s:%s/%s as %s", _settings.MYSQL_HOST, _settings.MYSQL_PORT, + _settings.MYSQL_DATABASE, _settings.MYSQL_USER) + ping() + create_schema() + log.info("tables present: %s", ", ".join(sorted(Base.metadata.tables))) + with SessionLocal() as db: + seed_admin(db, _settings) diff --git a/app/deps.py b/app/deps.py new file mode 100644 index 0000000..6489b97 --- /dev/null +++ b/app/deps.py @@ -0,0 +1,73 @@ +"""Request-scoped dependencies: the database session and the signed-in user.""" + +from __future__ import annotations + +from datetime import timedelta + +from fastapi import Depends, HTTPException, Request, status +from sqlalchemy import delete, select +from sqlalchemy.orm import Session + +from app.config import Settings, get_settings +from app.db import get_db +from app.models import AuthSession, User, utcnow +from app.security import fingerprint, session_expiry + +# Written at most this often, so ordinary asset requests do not each cost a write. +_LAST_SEEN_RESOLUTION = timedelta(minutes=5) + + +def client_ip(request: Request) -> str: + return request.client.host if request.client else "unknown" + + +def current_user(request: Request, + db: Session = Depends(get_db), + s: Settings = Depends(get_settings)) -> User | None: + """Resolve the session cookie, or None. Never raises -- pages that merely + render differently for anonymous visitors depend on that.""" + raw = request.cookies.get(s.SESSION_COOKIE_NAME) + if not raw: + return None + + now = utcnow() + sess = db.scalar(select(AuthSession).where(AuthSession.token_hash == fingerprint(raw, s))) + if sess is None: + return None + if sess.expires_at <= now: + db.execute(delete(AuthSession).where(AuthSession.id == sess.id)) + db.commit() + return None + + user = db.get(User, sess.user_id) + if user is None or not user.is_active: + return None + + dirty = False + if now - sess.last_seen_at > _LAST_SEEN_RESOLUTION: + sess.last_seen_at = now + dirty = True + # Rolling expiry: past the halfway mark, extend rather than sign the user + # out mid-session. The cookie's own max-age is refreshed in the same place + # it was set, so this only needs to move the server-side row. + if sess.expires_at - now < timedelta(days=s.SESSION_TTL_DAYS) / 2: + sess.expires_at = session_expiry(s, now) + dirty = True + if dirty: + db.commit() + + request.state.session_id = sess.id + return user + + +def require_user(user: User | None = Depends(current_user)) -> User: + if user is None: + raise HTTPException(status.HTTP_401_UNAUTHORIZED, "Please sign in.") + return user + + +def require_admin(user: User = Depends(require_user)) -> User: + if not user.is_admin: + raise HTTPException(status.HTTP_403_FORBIDDEN, + "That area is for administrators.") + return user diff --git a/app/errors.py b/app/errors.py new file mode 100644 index 0000000..1989964 --- /dev/null +++ b/app/errors.py @@ -0,0 +1,80 @@ +"""Error rendering. + +Two jobs. First, answer a 401 the way the caller can use: an HTML navigation +gets a redirect to the sign-in page, a fetch gets JSON. Without this, clicking +Excel export on an expired session dumps raw JSON into a new tab. + +Second, stop leaking internals. The local serve.py answers 500s with +`f"{type(exc).__name__}: {exc}"`, which is fine for a tool on your own laptop +and not fine on a shared server. +""" + +from __future__ import annotations + +import logging +import uuid + +from fastapi import FastAPI, HTTPException, Request, status +from fastapi.exceptions import RequestValidationError +from fastapi.responses import JSONResponse, RedirectResponse + +log = logging.getLogger(__name__) + + +def wants_html(request: Request) -> bool: + accept = request.headers.get("accept") or "" + # fetch() sends */* or application/json; a browser navigation asks for HTML + # explicitly and first. + return "text/html" in accept + + +def _login_redirect(request: Request) -> RedirectResponse: + nxt = request.url.path + if request.url.query: + nxt = f"{nxt}?{request.url.query}" + return RedirectResponse(f"/login?next={nxt}", status_code=status.HTTP_303_SEE_OTHER) + + +def install(app: FastAPI) -> None: + + @app.exception_handler(HTTPException) + async def http_exception(request: Request, exc: HTTPException): + if exc.status_code == status.HTTP_401_UNAUTHORIZED and wants_html(request): + return _login_redirect(request) + payload = {"error": exc.detail} + if isinstance(exc.detail, dict): + payload = exc.detail + return JSONResponse(payload, status_code=exc.status_code, + headers=getattr(exc, "headers", None)) + + @app.exception_handler(RequestValidationError) + async def validation_error(request: Request, exc: RequestValidationError): + # Literal 422: the Starlette constant for it was renamed, and pinning to + # either spelling would tie this file to a version range. + return JSONResponse({"error": _readable(exc)}, status_code=422) + + @app.exception_handler(ValueError) + async def value_error(request: Request, exc: ValueError): + # The pipeline raises ValueError with deliberate user-facing copy + # ("No change-history files uploaded yet."). Pass it straight through. + return JSONResponse({"error": str(exc)}, status_code=status.HTTP_400_BAD_REQUEST) + + @app.exception_handler(Exception) + async def unhandled(request: Request, exc: Exception): + ref = uuid.uuid4().hex[:12] + log.exception("unhandled error ref=%s on %s %s", ref, request.method, + request.url.path) + return JSONResponse( + {"error": "Something went wrong on our side.", "ref": ref}, + status_code=status.HTTP_500_INTERNAL_SERVER_ERROR) + + +def _readable(exc: RequestValidationError) -> str: + """Turn pydantic's error list into one sentence a person can act on.""" + for err in exc.errors(): + field = ".".join(str(p) for p in err.get("loc", ()) if p not in ("body", "query")) + msg = err.get("msg", "is not valid") + if field: + return f"{field}: {msg}" + return msg + return "That request was not valid." diff --git a/app/main.py b/app/main.py new file mode 100644 index 0000000..3a76ff4 --- /dev/null +++ b/app/main.py @@ -0,0 +1,161 @@ +"""The hosted application. + + uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 1 + +One worker is load-bearing, not a leftover: uploads and the last analysis live +in this process's memory, so with two workers a user's upload and their analyse +request could land in different processes. +""" + +from __future__ import annotations + +import logging +import tempfile +import threading +import time +from contextlib import asynccontextmanager +from datetime import timedelta +from pathlib import Path +from urllib.parse import urlparse + +from fastapi import FastAPI, Request, status +from fastapi.responses import JSONResponse +from fastapi.staticfiles import StaticFiles +from sqlalchemy import delete + +from app import errors +from app.bootstrap import seed_admin +from app.config import get_settings +from app.db import SessionLocal, create_schema +from app.models import AuthSession, EmailToken, utcnow +from app.routers import admin, analysis, auth, pages +from app.services.ratelimit import limiter +from app.services.store import WorkspaceStore, set_store, store_of + +log = logging.getLogger(__name__) +settings = get_settings() + +WEB = Path(__file__).resolve().parent.parent / "web" +SAFE_METHODS = {"GET", "HEAD", "OPTIONS", "TRACE"} +SWEEP_INTERVAL_S = 60 +TOKEN_RETENTION = timedelta(days=7) +_STARTED = time.monotonic() + + +def _workspace_root() -> Path: + base = settings.WORKSPACE_ROOT or tempfile.gettempdir() + return Path(base) / "oob-workspaces" + + +def _sweeper(store: WorkspaceStore, stop: threading.Event) -> None: + """Housekeeping, in a thread rather than an asyncio task: removing a few + hundred megabytes of uploads is blocking work and would stall the loop.""" + while not stop.wait(SWEEP_INTERVAL_S): + try: + store.sweep() + limiter.prune() + with SessionLocal() as db: + now = utcnow() + db.execute(delete(AuthSession).where(AuthSession.expires_at < now)) + db.execute(delete(EmailToken).where( + EmailToken.expires_at < now - TOKEN_RETENTION)) + db.commit() + except Exception: # noqa: BLE001 - never kill the sweeper + log.exception("sweep failed") + + +@asynccontextmanager +async def lifespan(app: FastAPI): + logging.basicConfig(level=settings.LOG_LEVEL, + format="%(asctime)s %(levelname)s %(name)s: %(message)s") + log.info("%s starting", settings.APP_NAME) + log.info("base url %s, cookie secure=%s", settings.base_url, settings.cookie_secure) + if not settings.cookie_secure and settings.base_url.startswith("https://"): + log.warning("COOKIE_SECURE is off but APP_BASE_URL is https -- " + "session cookies will travel without the Secure flag") + + create_schema() + with SessionLocal() as db: + seed_admin(db, settings) + + store = WorkspaceStore( + _workspace_root(), + idle_ttl_s=settings.WORKSPACE_IDLE_TTL_MIN * 60, + max_workspaces=settings.MAX_ACTIVE_WORKSPACES, + max_total_bytes=settings.MAX_TOTAL_WORKSPACE_BYTES, + ) + store.start() + set_store(store) + + stop = threading.Event() + thread = threading.Thread(target=_sweeper, args=(store, stop), + name="workspace-sweeper", daemon=True) + thread.start() + log.info("ready on %s", settings.base_url) + try: + yield + finally: + stop.set() + thread.join(timeout=5) + store.dispose_all() + log.info("stopped") + + +app = FastAPI(title=settings.APP_NAME, lifespan=lifespan, + docs_url=None, redoc_url=None, openapi_url=None) + + +@app.middleware("http") +async def guard(request: Request, call_next): + """Body-size ceiling, cross-origin write guard, and security headers.""" + if request.method not in SAFE_METHODS: + declared = request.headers.get("content-length") + if declared and declared.isdigit() and int(declared) > settings.MAX_UPLOAD_BYTES: + # Refuse before reading the body. Starlette has no size limit of + # its own, so without this a large POST is fully buffered first. + return JSONResponse( + {"error": "That file is too large."}, + status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE) + + # SameSite=Lax already blocks cross-site cookie-bearing writes; this + # covers the multipart upload route, which a plain HTML form can forge. + origin = request.headers.get("origin") or request.headers.get("referer") + if origin: + host = urlparse(origin).netloc.lower() + allowed = {urlparse(settings.base_url).netloc.lower(), + (request.headers.get("host") or "").lower()} + if host and host not in allowed: + return JSONResponse({"error": "Bad origin."}, + status_code=status.HTTP_403_FORBIDDEN) + + response = await call_next(request) + response.headers.setdefault("X-Content-Type-Options", "nosniff") + response.headers.setdefault("X-Frame-Options", "DENY") + # Reset and verification tokens ride in the query string, so no Referer. + response.headers.setdefault("Referrer-Policy", "no-referrer") + response.headers.setdefault( + "Content-Security-Policy", + "default-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-inline'; " + "script-src 'self'; frame-ancestors 'none'; base-uri 'none'; form-action 'self'") + if request.url.path.startswith("/api/"): + response.headers["Cache-Control"] = "no-store" + return response + + +errors.install(app) +app.include_router(pages.router) +app.include_router(auth.router) +app.include_router(analysis.router) +app.include_router(admin.router) + +# Component-wise containment, ETags and 304s for free. The local serve.py does +# this with a str.startswith prefix check, which a sibling directory would pass. +app.mount("/static", StaticFiles(directory=WEB), name="static") + + +@app.get("/statusz", include_in_schema=False) +def statusz() -> dict: + """Cheap operational counters. No account data, so no auth needed.""" + store = store_of() + return {"workspaces": len(store), "bytes": store.total_bytes(), + "uptime_s": int(time.monotonic() - _STARTED)} diff --git a/app/models.py b/app/models.py new file mode 100644 index 0000000..9e3a911 --- /dev/null +++ b/app/models.py @@ -0,0 +1,106 @@ +"""Account tables. + +Every table is prefixed `oob_`. The target database is shared and `create_all` +silently skips a table that already exists, so an unprefixed `users` could +quietly bind the app to somebody else's table. + +Column types are deliberately portable -- Integer/String/Boolean/DateTime, no +MySQL-specific types -- so the test suite can run the whole auth flow against +in-memory SQLite without a database server. +""" + +from __future__ import annotations + +from datetime import datetime, timezone + +from sqlalchemy import (Boolean, DateTime, ForeignKey, Index, Integer, String, + UniqueConstraint) +from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column + +VERIFY_EMAIL = "verify_email" +PASSWORD_RESET = "password_reset" + + +def utcnow() -> datetime: + """Naive UTC. MySQL DATETIME carries no timezone, so storing an aware value + would round-trip as naive anyway and the comparisons would start raising.""" + return datetime.now(timezone.utc).replace(tzinfo=None) + + +class Base(DeclarativeBase): + pass + + +class User(Base): + __tablename__ = "oob_users" + + id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) + email: Mapped[str] = mapped_column(String(255), nullable=False) # lowercased on write + name: Mapped[str] = mapped_column(String(120), nullable=False, default="") + password_hash: Mapped[str] = mapped_column(String(255), nullable=False) + is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True) + is_admin: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False) + # NULL means unverified. A timestamp is more useful than a bool here. + email_verified_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True) + failed_login_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0) + locked_until: Mapped[datetime | None] = mapped_column(DateTime, nullable=True) + last_login_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True) + created_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, default=utcnow) + updated_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, + default=utcnow, onupdate=utcnow) + + __table_args__ = (UniqueConstraint("email", name="uq_oob_users_email"),) + + @property + def is_verified(self) -> bool: + return self.email_verified_at is not None + + +class AuthSession(Base): + """A signed-in browser. + + Named AuthSession, not Session, so it cannot be confused with + sqlalchemy.orm.Session in a type annotation or an import. + """ + + __tablename__ = "oob_sessions" + + id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) + user_id: Mapped[int] = mapped_column( + Integer, ForeignKey("oob_users.id", ondelete="CASCADE"), nullable=False) + # HMAC of the cookie value, never the value itself: a read-only database + # leak then yields nothing anyone can log in with. + token_hash: Mapped[str] = mapped_column(String(64), nullable=False) + created_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, default=utcnow) + last_seen_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, default=utcnow) + expires_at: Mapped[datetime] = mapped_column(DateTime, nullable=False) + ip: Mapped[str | None] = mapped_column(String(45), nullable=True) # 45 = IPv6 text max + user_agent: Mapped[str | None] = mapped_column(String(255), nullable=True) + + __table_args__ = ( + UniqueConstraint("token_hash", name="uq_oob_sessions_hash"), + Index("ix_oob_sessions_user", "user_id"), + Index("ix_oob_sessions_expires", "expires_at"), + ) + + +class EmailToken(Base): + """One-shot link sent by email: address verification or password reset.""" + + __tablename__ = "oob_email_tokens" + + id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) + user_id: Mapped[int] = mapped_column( + Integer, ForeignKey("oob_users.id", ondelete="CASCADE"), nullable=False) + purpose: Mapped[str] = mapped_column(String(32), nullable=False) + token_hash: Mapped[str] = mapped_column(String(64), nullable=False) + expires_at: Mapped[datetime] = mapped_column(DateTime, nullable=False) + used_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True) + created_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, default=utcnow) + requested_ip: Mapped[str | None] = mapped_column(String(45), nullable=True) + + __table_args__ = ( + UniqueConstraint("token_hash", name="uq_oob_email_tokens_hash"), + Index("ix_oob_email_tokens_user_purpose", "user_id", "purpose"), + Index("ix_oob_email_tokens_expires", "expires_at"), + ) diff --git a/app/routers/__init__.py b/app/routers/__init__.py new file mode 100644 index 0000000..efd524c --- /dev/null +++ b/app/routers/__init__.py @@ -0,0 +1 @@ +"""HTTP routes.""" diff --git a/app/routers/admin.py b/app/routers/admin.py new file mode 100644 index 0000000..0c07ac6 --- /dev/null +++ b/app/routers/admin.py @@ -0,0 +1,96 @@ +"""Administrator endpoints: see who has an account, and turn them off.""" + +from __future__ import annotations + +import logging +from datetime import timedelta + +from fastapi import (APIRouter, BackgroundTasks, Depends, HTTPException, Request, + status) +from sqlalchemy import delete, func, select +from sqlalchemy.orm import Session + +from app.config import Settings, get_settings +from app.db import get_db +from app.deps import client_ip, require_admin +from app.models import VERIFY_EMAIL, AuthSession, User +from app.routers.auth import _mint, _send_verification +from app.schemas import Ok, UserOut +from app.services.store import store_of + +log = logging.getLogger(__name__) +router = APIRouter(prefix="/api/admin", tags=["admin"], + dependencies=[Depends(require_admin)]) + + +@router.get("/users") +def list_users(db: Session = Depends(get_db)) -> dict: + users = db.scalars(select(User).order_by(User.created_at.desc())).all() + live = {uid for (uid,) in db.execute( + select(AuthSession.user_id).group_by(AuthSession.user_id))} + return { + "users": [ + {**UserOut.of(u).model_dump(mode="json"), "signed_in": u.id in live} + for u in users + ], + "sessions": db.scalar(select(func.count()).select_from(AuthSession)) or 0, + "workspaces": len(store_of()), + } + + +def _target(db: Session, user_id: int) -> User: + user = db.get(User, user_id) + if user is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "No such account.") + return user + + +@router.post("/users/{user_id}/deactivate", response_model=Ok) +def deactivate(user_id: int, admin: User = Depends(require_admin), + db: Session = Depends(get_db)) -> Ok: + user = _target(db, user_id) + if user.id == admin.id: + raise HTTPException(status.HTTP_400_BAD_REQUEST, + "You cannot deactivate your own account.") + user.is_active = False + # Signing them out is the point; leaving live sessions would make the + # button look like it worked while they carried on using the dashboard. + db.execute(delete(AuthSession).where(AuthSession.user_id == user.id)) + db.commit() + store_of().drop(user.id) + log.info("admin %s deactivated %s", admin.email, user.email) + return Ok(message=f"{user.email} can no longer sign in.") + + +@router.post("/users/{user_id}/activate", response_model=Ok) +def activate(user_id: int, admin: User = Depends(require_admin), + db: Session = Depends(get_db)) -> Ok: + user = _target(db, user_id) + user.is_active = True + user.failed_login_count = 0 + user.locked_until = None + db.commit() + log.info("admin %s reactivated %s", admin.email, user.email) + return Ok(message=f"{user.email} can sign in again.") + + +@router.post("/users/{user_id}/resend-verification", response_model=Ok) +def resend(user_id: int, request: Request, bg: BackgroundTasks, + db: Session = Depends(get_db), + s: Settings = Depends(get_settings)) -> Ok: + user = _target(db, user_id) + if user.is_verified: + return Ok(message=f"{user.email} is already confirmed.") + raw = _mint(db, user, VERIFY_EMAIL, timedelta(hours=s.VERIFY_TOKEN_TTL_HOURS), + client_ip(request)) + bg.add_task(_send_verification, user, raw, s) + return Ok(message=f"A new confirmation link is on its way to {user.email}.") + + +@router.post("/users/{user_id}/sign-out", response_model=Ok) +def sign_out(user_id: int, db: Session = Depends(get_db)) -> Ok: + user = _target(db, user_id) + db.execute(delete(AuthSession).where(AuthSession.user_id == user.id)) + db.commit() + store_of().drop(user.id) + return Ok(message=f"{user.email} has been signed out everywhere.") diff --git a/app/routers/analysis.py b/app/routers/analysis.py new file mode 100644 index 0000000..c34b741 --- /dev/null +++ b/app/routers/analysis.py @@ -0,0 +1,113 @@ +"""Upload, analyse, export -- the original dashboard API, now per user. + +Every route here is `def`, not `async def`, on purpose. The pipeline is +synchronous and CPU-bound: one analysis holds a core for seconds to tens of +seconds. FastAPI runs `def` handlers in a worker thread, so a long run cannot +stall the event loop and everyone else's requests with it. +""" + +from __future__ import annotations + +import logging +import threading + +from fastapi import (APIRouter, Depends, File, Form, HTTPException, Response, + UploadFile, status) + +from app.config import Settings, get_settings +from app.deps import require_user +from app.models import User +from app.schemas import Ok, SettingsIn +from app.services import exporters +from app.services.analysis import run_analysis +from app.services.ratelimit import limiter +from app.services.store import UploadTooLarge, UserWorkspace, store_of + +log = logging.getLogger(__name__) +router = APIRouter(prefix="/api", tags=["analysis"]) + +# Bounds peak memory. The default worker threadpool is 40 threads; without this +# a burst of uploads could start 40 concurrent pipelines and exhaust the box. +_slots = threading.BoundedSemaphore(get_settings().MAX_CONCURRENT_ANALYSES) + + +def workspace(user: User = Depends(require_user)) -> UserWorkspace: + return store_of().get(user.id) + + +@router.get("/state") +def state(user: User = Depends(require_user), + ws: UserWorkspace = Depends(workspace)) -> dict: + with ws.lock: + payload = ws.names() + payload["user"] = {"email": user.email, "name": user.name, "is_admin": user.is_admin} + return payload + + +@router.post("/upload") +def upload(file: UploadFile = File(...), kind: str = Form("history"), + ws: UserWorkspace = Depends(workspace), + s: Settings = Depends(get_settings)) -> dict: + if kind not in ("history", "perf"): + raise ValueError("Unknown upload kind.") + try: + with ws.lock: + saved = ws.add(file.filename or "upload.xlsx", file.file, kind, + s.MAX_UPLOAD_BYTES, s.MAX_WORKSPACE_BYTES) + except UploadTooLarge as exc: + raise HTTPException(status.HTTP_413_REQUEST_ENTITY_TOO_LARGE, str(exc)) from exc + finally: + file.file.close() + return {"ok": True, "name": saved.name.split("_", 1)[-1]} + + +@router.post("/analyze") +def analyze(body: SettingsIn | None = None, + user: User = Depends(require_user), + ws: UserWorkspace = Depends(workspace)) -> dict: + wait = limiter.check("analyze:user", str(user.id)) + if wait: + raise HTTPException(status.HTTP_429_TOO_MANY_REQUESTS, + "That is a lot of runs in a short time. Give it a minute.", + headers={"Retry-After": str(int(wait))}) + + if not _slots.acquire(blocking=False): + raise HTTPException(status.HTTP_503_SERVICE_UNAVAILABLE, + "The server is analysing other reports right now. " + "Try again in a moment.", + headers={"Retry-After": "30"}) + try: + settings_in = (body or SettingsIn()).model_dump() + with ws.lock: + # Released before the run so the previous result is not held + # alongside the new one at the peak of the build. + ws.last = None + result = run_analysis(ws.history, ws.perf, settings_in) + ws.last = result.artifacts + return result.payload + finally: + _slots.release() + + +@router.post("/clear", response_model=Ok) +def clear(ws: UserWorkspace = Depends(workspace)) -> Ok: + with ws.lock: + ws.clear() + return Ok() + + +@router.get("/export") +def export(format: str = "xlsx", ws: UserWorkspace = Depends(workspace)) -> Response: + with ws.lock: + last = ws.last + if not last: + raise ValueError("Analyse some files first.") + stamp = exporters.export_stamp(last) + if format == "csv": + body, mime, name = (exporters.csv_bytes(last), exporters.CSV_MIME, + f"ppc-budget_{stamp}.csv") + else: + body, mime, name = (exporters.xlsx_bytes(last), exporters.XLSX_MIME, + f"ppc-budget-report_{stamp}.xlsx") + return Response(body, media_type=mime, + headers={"Content-Disposition": f'attachment; filename="{name}"'}) diff --git a/app/routers/auth.py b/app/routers/auth.py new file mode 100644 index 0000000..3cb8ee7 --- /dev/null +++ b/app/routers/auth.py @@ -0,0 +1,310 @@ +"""Sign up, sign in, verify an address, reset a password. + +Two rules run through all of it. + +Nothing here tells a caller whether an address has an account. Signup and +forgot-password always answer the same way; login answers with one generic +message whether the address is unknown, the password wrong, the account locked, +or the account disabled. + +Nothing here consumes a token on GET. Mail scanners -- Outlook Safe Links, +Defender, Proofpoint -- fetch every URL in an inbound message, so a link that +acts on GET is spent before the recipient ever clicks it. The pages are static; +they read the token from the URL and POST it. +""" + +from __future__ import annotations + +import logging +from datetime import timedelta + +from fastapi import (APIRouter, BackgroundTasks, Depends, HTTPException, Request, + Response, status) +from sqlalchemy import delete, select, update +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.config import Settings, get_settings +from app.db import get_db +from app.deps import client_ip, current_user, require_user +from app.models import (PASSWORD_RESET, VERIFY_EMAIL, AuthSession, EmailToken, + User, utcnow) +from app.schemas import EmailIn, LoginIn, Ok, ResetIn, SignupIn, TokenIn, UserOut +from app.security import (burn_cpu, clear_session_cookie, fingerprint, + hash_password, new_token, session_expiry, + set_session_cookie, verify_password) +from app.services import email_service +from app.services.ratelimit import limiter +from app.services.store import store_of + +log = logging.getLogger(__name__) +router = APIRouter(prefix="/api/auth", tags=["auth"]) + +BAD_CREDENTIALS = "Email or password is incorrect." +CHECK_INBOX = "Check your email to finish setting up your account." +RESET_SENT = "If that address has an account, a reset link is on its way." +BAD_LINK = "That link has expired or has already been used." + + +# --------------------------------------------------------------------- helpers + +def _limit(bucket: str, subject: str) -> None: + wait = limiter.check(bucket, subject) + if wait: + raise HTTPException(status.HTTP_429_TOO_MANY_REQUESTS, + "Too many attempts. Try again in a few minutes.", + headers={"Retry-After": str(int(wait))}) + + +def _mint(db: Session, user: User, purpose: str, ttl: timedelta, + ip: str | None) -> str: + """Issue a one-shot token, retiring any earlier unused one of this purpose + so a mailbox cannot accumulate a stack of live links.""" + db.execute( + update(EmailToken) + .where(EmailToken.user_id == user.id, EmailToken.purpose == purpose, + EmailToken.used_at.is_(None)) + .values(used_at=utcnow()) + ) + raw = new_token() + db.add(EmailToken(user_id=user.id, purpose=purpose, token_hash=fingerprint(raw), + expires_at=utcnow() + ttl, requested_ip=ip)) + db.commit() + return raw + + +def _consume(db: Session, raw: str, purpose: str) -> User: + """Spend a token, or refuse. One UPDATE, so two simultaneous clicks cannot + both win: whichever loses sees rowcount 0.""" + now = utcnow() + result = db.execute( + update(EmailToken) + .where(EmailToken.token_hash == fingerprint(raw), + EmailToken.purpose == purpose, + EmailToken.used_at.is_(None), + EmailToken.expires_at > now) + .values(used_at=now) + ) + if result.rowcount != 1: + db.rollback() + raise HTTPException(status.HTTP_400_BAD_REQUEST, BAD_LINK) + + token = db.scalar(select(EmailToken).where(EmailToken.token_hash == fingerprint(raw))) + user = db.get(User, token.user_id) if token else None + if user is None: + db.rollback() + raise HTTPException(status.HTTP_400_BAD_REQUEST, BAD_LINK) + db.commit() + return user + + +def _start_session(db: Session, user: User, request: Request, + response: Response, s: Settings) -> None: + now = utcnow() + raw = new_token() + db.add(AuthSession( + user_id=user.id, token_hash=fingerprint(raw, s), + created_at=now, last_seen_at=now, expires_at=session_expiry(s, now), + ip=client_ip(request)[:45], + user_agent=(request.headers.get("user-agent") or "")[:255] or None, + )) + user.last_login_at = now + user.failed_login_count = 0 + user.locked_until = None + db.commit() + set_session_cookie(response, raw, s) + + +def _send_verification(user: User, raw: str, s: Settings) -> None: + email_service.send_verification_email( + user.email, user.name, f"{s.base_url}/verify?token={raw}") + + +def _domain_allowed(email: str, s: Settings) -> bool: + allowed = s.allowed_signup_domains + return not allowed or email.rsplit("@", 1)[-1].lower() in allowed + + +# ---------------------------------------------------------------------- routes + +@router.post("/signup", response_model=Ok) +def signup(body: SignupIn, request: Request, bg: BackgroundTasks, + db: Session = Depends(get_db), s: Settings = Depends(get_settings)) -> Ok: + if not s.SIGNUP_ENABLED: + raise HTTPException(status.HTTP_403_FORBIDDEN, + "New accounts are not being created right now.") + ip = client_ip(request) + _limit("signup:ip", ip) + + email = body.email.strip().lower() + if not _domain_allowed(email, s): + allowed = ", ".join(sorted(s.allowed_signup_domains)) + raise HTTPException(status.HTTP_403_FORBIDDEN, + f"Accounts are limited to these domains: {allowed}.") + + existing = db.scalar(select(User).where(User.email == email)) + if existing is not None: + # Same answer as a fresh signup. The real owner is told what happened; + # whoever submitted the form learns nothing. + if existing.is_verified: + bg.add_task(email_service.send_account_exists_email, existing.email, + existing.name, f"{s.base_url}/login", f"{s.base_url}/forgot") + else: + raw = _mint(db, existing, VERIFY_EMAIL, + timedelta(hours=s.VERIFY_TOKEN_TTL_HOURS), ip) + bg.add_task(_send_verification, existing, raw, s) + return Ok(message=CHECK_INBOX) + + user = User(email=email, name=body.name, password_hash=hash_password(body.password)) + db.add(user) + try: + db.commit() + except IntegrityError: + # Lost a race against a simultaneous signup for the same address. + db.rollback() + return Ok(message=CHECK_INBOX) + + raw = _mint(db, user, VERIFY_EMAIL, timedelta(hours=s.VERIFY_TOKEN_TTL_HOURS), ip) + bg.add_task(_send_verification, user, raw, s) + log.info("account created: %s", email) + return Ok(message=CHECK_INBOX) + + +@router.post("/login") +def login(body: LoginIn, request: Request, response: Response, + db: Session = Depends(get_db), s: Settings = Depends(get_settings)) -> dict: + email = body.email.strip().lower() + _limit("login:ip", client_ip(request)) + _limit("login:email", email) + + user = db.scalar(select(User).where(User.email == email)) + if user is None: + burn_cpu() # match the timing of a real verify + raise HTTPException(status.HTTP_401_UNAUTHORIZED, BAD_CREDENTIALS) + + now = utcnow() + # Locked accounts get the same message as a wrong password. Saying "locked" + # would confirm the address exists and hand out a way to grief a colleague. + if user.locked_until and user.locked_until > now: + burn_cpu() + raise HTTPException(status.HTTP_401_UNAUTHORIZED, BAD_CREDENTIALS) + + ok, rehashed = verify_password(user.password_hash, body.password) + if not ok: + user.failed_login_count += 1 + if user.failed_login_count >= s.MAX_FAILED_LOGINS: + over = user.failed_login_count - s.MAX_FAILED_LOGINS + minutes = min(60, s.LOCKOUT_MINUTES * (2 ** over)) + user.locked_until = now + timedelta(minutes=minutes) + log.warning("account %s locked for %d minutes", email, minutes) + db.commit() + raise HTTPException(status.HTTP_401_UNAUTHORIZED, BAD_CREDENTIALS) + + if not user.is_active: + raise HTTPException(status.HTTP_401_UNAUTHORIZED, BAD_CREDENTIALS) + + if not user.is_verified: + # The one place the answer is specific. It only reaches someone who + # already has valid credentials, and without it an unverified user has + # no way forward. + user.failed_login_count = 0 + db.commit() + raise HTTPException( + status.HTTP_403_FORBIDDEN, + {"error": "Confirm your email address before signing in.", + "code": "email_not_verified"}) + + if rehashed: + user.password_hash = rehashed + _start_session(db, user, request, response, s) + limiter.reset("login:email", email) + return {"ok": True, "user": UserOut.of(user).model_dump(mode="json")} + + +@router.post("/logout", response_model=Ok) +def logout(request: Request, response: Response, + db: Session = Depends(get_db), s: Settings = Depends(get_settings)) -> Ok: + raw = request.cookies.get(s.SESSION_COOKIE_NAME) + if raw: + db.execute(delete(AuthSession).where(AuthSession.token_hash == fingerprint(raw, s))) + db.commit() + clear_session_cookie(response, s) + return Ok(message="Signed out.") + + +@router.get("/me") +def me(user: User = Depends(require_user)) -> dict: + return {"user": UserOut.of(user).model_dump(mode="json")} + + +@router.post("/verify", response_model=Ok) +def verify(body: TokenIn, db: Session = Depends(get_db)) -> Ok: + user = _consume(db, body.token, VERIFY_EMAIL) + if not user.is_verified: + user.email_verified_at = utcnow() + db.commit() + log.info("email verified: %s", user.email) + return Ok(message="Your email address is confirmed. You can sign in now.") + + +@router.post("/resend-verification", response_model=Ok) +def resend_verification(body: EmailIn, request: Request, bg: BackgroundTasks, + db: Session = Depends(get_db), + s: Settings = Depends(get_settings)) -> Ok: + email = body.email.strip().lower() + _limit("resend:email", email) + user = db.scalar(select(User).where(User.email == email)) + if user is not None and not user.is_verified and user.is_active: + raw = _mint(db, user, VERIFY_EMAIL, timedelta(hours=s.VERIFY_TOKEN_TTL_HOURS), + client_ip(request)) + bg.add_task(_send_verification, user, raw, s) + return Ok(message=CHECK_INBOX) + + +@router.post("/forgot", response_model=Ok) +def forgot(body: EmailIn, request: Request, bg: BackgroundTasks, + db: Session = Depends(get_db), s: Settings = Depends(get_settings)) -> Ok: + email = body.email.strip().lower() + _limit("forgot:ip", client_ip(request)) + _limit("forgot:email", email) + + user = db.scalar(select(User).where(User.email == email)) + if user is None or not user.is_active: + burn_cpu() # keep the timing indistinguishable + return Ok(message=RESET_SENT) + + raw = _mint(db, user, PASSWORD_RESET, + timedelta(minutes=s.RESET_TOKEN_TTL_MINUTES), client_ip(request)) + bg.add_task(email_service.send_password_reset_email, user.email, user.name, + f"{s.base_url}/reset?token={raw}") + return Ok(message=RESET_SENT) + + +@router.post("/reset", response_model=Ok) +def reset(body: ResetIn, bg: BackgroundTasks, db: Session = Depends(get_db)) -> Ok: + user = _consume(db, body.token, PASSWORD_RESET) + + user.password_hash = hash_password(body.password) + # Controlling the mailbox proves the address, so an unverified account + # becomes verified here rather than stranding the user. + if not user.is_verified: + user.email_verified_at = utcnow() + user.failed_login_count = 0 + user.locked_until = None + # Every other browser is signed out. This is the point of server-side + # sessions: a stolen cookie dies with the password that leaked it. + db.execute(delete(AuthSession).where(AuthSession.user_id == user.id)) + db.commit() + + store_of().drop(user.id) + bg.add_task(email_service.send_password_changed_email, user.email, user.name) + log.info("password reset completed: %s", user.email) + return Ok(message="Your password has been changed. Sign in with it now.") + + +@router.get("/session") +def session_state(user: User | None = Depends(current_user)) -> dict: + """Whether this browser is signed in. Never 401s -- the sign-in page itself + uses it to bounce an already-authenticated visitor onward.""" + return {"authenticated": user is not None, + "user": UserOut.of(user).model_dump(mode="json") if user else None} diff --git a/app/routers/pages.py b/app/routers/pages.py new file mode 100644 index 0000000..3e5beaf --- /dev/null +++ b/app/routers/pages.py @@ -0,0 +1,102 @@ +"""The HTML pages, plus the two health endpoints. + +There is no template engine. The only thing a template would inject is the +`next` parameter and a status message, both of which the page reads from +location.search -- so these are static files and the JS does the rest. +""" + +from __future__ import annotations + +import logging +from pathlib import Path + +from fastapi import APIRouter, Depends, Request, status +from fastapi.responses import FileResponse, JSONResponse, RedirectResponse, Response + +from app import db as db_module +from app.deps import current_user +from app.models import User + +log = logging.getLogger(__name__) +router = APIRouter(tags=["pages"]) + +WEB = Path(__file__).resolve().parent.parent.parent / "web" +NO_STORE = {"Cache-Control": "no-store"} + + +def page(name: str) -> FileResponse: + return FileResponse(WEB / name, media_type="text/html; charset=utf-8", + headers=NO_STORE) + + +@router.get("/", include_in_schema=False) +def index(user: User | None = Depends(current_user)) -> Response: + # Redirect on the server. Gating in JS would flash the whole app shell + # before bouncing, and would show nothing at all with scripting disabled. + if user is None: + return RedirectResponse("/login", status_code=status.HTTP_303_SEE_OTHER) + return page("index.html") + + +@router.get("/login", include_in_schema=False) +def login_page(request: Request, user: User | None = Depends(current_user)) -> Response: + if user is not None: + nxt = request.query_params.get("next") or "/" + if not nxt.startswith("/") or nxt.startswith("//"): + nxt = "/" # only ever redirect within this site + return RedirectResponse(nxt, status_code=status.HTTP_303_SEE_OTHER) + return page("login.html") + + +@router.get("/signup", include_in_schema=False) +def signup_page(user: User | None = Depends(current_user)) -> Response: + if user is not None: + return RedirectResponse("/", status_code=status.HTTP_303_SEE_OTHER) + return page("signup.html") + + +@router.get("/forgot", include_in_schema=False) +def forgot_page() -> Response: + return page("forgot.html") + + +# Both of these are deliberately inert: they render, read the token out of the +# query string, and POST it. A mail scanner that GETs every link in a message +# therefore cannot spend the token before the recipient opens it. +@router.get("/reset", include_in_schema=False) +def reset_page() -> Response: + return page("reset.html") + + +@router.get("/verify", include_in_schema=False) +def verify_page() -> Response: + return page("verify.html") + + +@router.get("/admin", include_in_schema=False) +def admin_page(user: User | None = Depends(current_user)) -> Response: + if user is None: + return RedirectResponse("/login?next=/admin", status_code=status.HTTP_303_SEE_OTHER) + if not user.is_admin: + return RedirectResponse("/", status_code=status.HTTP_303_SEE_OTHER) + return page("admin.html") + + +@router.get("/healthz", include_in_schema=False) +def healthz() -> JSONResponse: + """Liveness only. Deliberately does not touch MySQL: if it did, a thirty + second RDS failover would restart the container instead of just failing the + handful of routes that actually need the database.""" + return JSONResponse({"status": "ok"}, headers=NO_STORE) + + +@router.get("/readyz", include_in_schema=False) +def readyz() -> JSONResponse: + try: + db_module.ping() + except Exception as exc: # noqa: BLE001 - report, don't raise + log.warning("readiness check failed: %s", exc) + return JSONResponse({"db": "unreachable"}, + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, + headers=NO_STORE) + return JSONResponse({"db": "ok"}, headers=NO_STORE) diff --git a/app/schemas.py b/app/schemas.py new file mode 100644 index 0000000..96c6e52 --- /dev/null +++ b/app/schemas.py @@ -0,0 +1,84 @@ +"""Request and response bodies. + +Declaring these as pydantic models is also a CSRF control: FastAPI then rejects +form-encoded content types with a 422, and an HTML form can only produce those. +A cross-site form therefore cannot reach any of these endpoints even before the +SameSite cookie attribute is considered. +""" + +from __future__ import annotations + +from datetime import datetime + +from pydantic import BaseModel, EmailStr, Field, field_validator + +from app.config import get_settings + + +def _password_field() -> Field: + s = get_settings() + return Field(min_length=s.MIN_PASSWORD_LENGTH, max_length=s.MAX_PASSWORD_LENGTH) + + +class SignupIn(BaseModel): + email: EmailStr + password: str = _password_field() + name: str = Field(default="", max_length=120) + + @field_validator("name") + @classmethod + def _tidy(cls, v: str) -> str: + return v.strip() + + +class LoginIn(BaseModel): + email: EmailStr + password: str = Field(min_length=1, max_length=256) + + +class EmailIn(BaseModel): + email: EmailStr + + +class TokenIn(BaseModel): + token: str = Field(min_length=8, max_length=256) + + +class ResetIn(TokenIn): + password: str = _password_field() + + +class SettingsIn(BaseModel): + """The modelling assumptions the dashboard sends with /api/analyze. + + `roas` is a string because the settings dialog sends "" for "use the + account average", and the pipeline already treats falsy as absent. + """ + + roas: str | float | None = None + haircut: float = Field(default=0.7, ge=0, le=1) + cap: float = Field(default=3, gt=0, le=100) + merge_gap: int = Field(default=5, ge=0, le=720) + + +class UserOut(BaseModel): + id: int + email: str + name: str + is_admin: bool + is_active: bool + verified: bool + created_at: datetime | None = None + last_login_at: datetime | None = None + + @classmethod + def of(cls, user) -> "UserOut": + return cls(id=user.id, email=user.email, name=user.name, + is_admin=user.is_admin, is_active=user.is_active, + verified=user.is_verified, created_at=user.created_at, + last_login_at=user.last_login_at) + + +class Ok(BaseModel): + ok: bool = True + message: str | None = None diff --git a/app/security.py b/app/security.py new file mode 100644 index 0000000..4548cf4 --- /dev/null +++ b/app/security.py @@ -0,0 +1,94 @@ +"""Password hashing, opaque token minting, and the session cookie.""" + +from __future__ import annotations + +import hashlib +import hmac +import logging +import secrets +from datetime import datetime, timedelta + +from argon2 import PasswordHasher +from argon2.exceptions import (InvalidHashError, VerificationError, + VerifyMismatchError) +from fastapi import Response + +from app.config import Settings, get_settings + +log = logging.getLogger(__name__) + +# OWASP's second recommended argon2id profile: 19 MiB, t=2, p=1. Sized so that +# a handful of concurrent logins cannot exhaust a 2 GB container's memory. +_ph = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1, + hash_len=32, salt_len=16) + +# Verified on the unknown-email path so response time does not distinguish +# "no such account" from "wrong password". +_DUMMY_HASH = _ph.hash("timing-equalisation-placeholder") + + +def hash_password(raw: str) -> str: + return _ph.hash(raw) + + +def verify_password(stored: str, raw: str) -> tuple[bool, str | None]: + """(ok, replacement_hash). The replacement is set when the stored hash was + made with weaker parameters than the current ones.""" + try: + _ph.verify(stored, raw) + except (VerifyMismatchError, VerificationError, InvalidHashError): + return False, None + try: + return True, (_ph.hash(raw) if _ph.check_needs_rehash(stored) else None) + except Exception: # noqa: BLE001 - a rehash failure must not block login + return True, None + + +def burn_cpu() -> None: + """Spend the same time a real verify would, and discard the result.""" + try: + _ph.verify(_DUMMY_HASH, "x") + except Exception: # noqa: BLE001 - always mismatches, by design + pass + + +# --------------------------------------------------------------------- tokens + +def new_token() -> str: + """256 bits of URL-safe randomness. Used for both session cookies and the + one-shot links sent by email.""" + return secrets.token_urlsafe(32) + + +def fingerprint(raw: str, s: Settings | None = None) -> str: + """What gets stored. Keyed with SECRET_KEY so a leaked table of hashes is + not enough to forge one, and so rotating the key invalidates everything.""" + key = (s or get_settings()).SECRET_KEY.get_secret_value().encode() + return hmac.new(key, raw.encode(), hashlib.sha256).hexdigest() + + +# --------------------------------------------------------------------- cookie + +def set_session_cookie(response: Response, raw_token: str, s: Settings) -> None: + response.set_cookie( + key=s.SESSION_COOKIE_NAME, + value=raw_token, + max_age=s.SESSION_TTL_DAYS * 86400, + httponly=True, # app.js builds a lot of markup; XSS must not reach this + samesite="lax", # strict would break links clicked from a webmail tab + secure=s.cookie_secure, + path="/", + # No domain: host-only. A Domain attribute on a bare IP is invalid and + # the browser drops the cookie without saying so. + ) + + +def clear_session_cookie(response: Response, s: Settings) -> None: + response.delete_cookie( + key=s.SESSION_COOKIE_NAME, path="/", + httponly=True, samesite="lax", secure=s.cookie_secure, + ) + + +def session_expiry(s: Settings, now: datetime) -> datetime: + return now + timedelta(days=s.SESSION_TTL_DAYS) diff --git a/app/services/__init__.py b/app/services/__init__.py new file mode 100644 index 0000000..c6b1542 --- /dev/null +++ b/app/services/__init__.py @@ -0,0 +1,6 @@ +"""Framework-free services. + +`analysis` and `exporters` deliberately import nothing beyond the standard +library and `ppcbudget`, so that the local single-user serve.py can use them +without pulling in FastAPI, SQLAlchemy or anything else. +""" diff --git a/app/services/analysis.py b/app/services/analysis.py new file mode 100644 index 0000000..c6e0f14 --- /dev/null +++ b/app/services/analysis.py @@ -0,0 +1,95 @@ +"""The analysis pipeline, lifted out of serve.py so more than one caller can use it. + +This module imports only the standard library and `ppcbudget`. Keep it that way: +serve.py depends on it staying installable with nothing but openpyxl. +""" + +from __future__ import annotations + +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from ppcbudget import actions as actions_mod +from ppcbudget import aggregate, metrics, payload, perfjoin +from ppcbudget.ingest import dedupe_events, load_history +from ppcbudget.scoring import check_invariants, score_all + + +@dataclass +class AnalysisResult: + """What one run produced. + + `payload` is the compact JSON the dashboard renders. `artifacts` is the + scored data itself, kept because the Excel and CSV exports rebuild their + output from it rather than from the payload. + """ + + payload: dict + artifacts: dict + + +def run_analysis(history: Sequence[Path], perf: Path | None, + settings_in: Mapping[str, Any]) -> AnalysisResult: + """Run the pipeline over everything uploaded so far.""" + if not history: + raise ValueError("No change-history files uploaded yet.") + + events, metas, qas, skipped = [], [], [], [] + for path in history: + try: + evs, meta, qa = load_history(path) + except (ValueError, KeyError, OSError) as exc: + skipped.append(f"{path.name}: {exc}") + continue + events.extend(evs) + metas.append(meta) + qas.append(qa) + + if not events: + detail = " ".join(skipped) or "no readable rows" + raise ValueError(f"None of the files could be read as a change-history export. {detail}") + + events, overlap_rows = dedupe_events(events) + days = score_all(events, merge_gap_min=int(settings_in.get("merge_gap", 5))) + if not days: + raise ValueError("No campaigns had budget-state changes, so there is nothing to score.") + + join_report = None + roas_source = "account_average" + if perf: + records, join_report = perfjoin.load_performance(perf) + perfjoin.apply_to(days, records, join_report) + roas_source = "campaign" + + account_roas = next((m.roas for m in metas if m.roas), None) + roas_override = settings_in.get("roas") + settings = metrics.ModelSettings( + roas=float(roas_override) if roas_override else (account_roas or 4.0), + roas_source="override" if roas_override else roas_source, + haircut=float(settings_in.get("haircut", metrics.DEFAULT_ROAS_HAIRCUT)), + cap_multiple=float(settings_in.get("cap", metrics.DEFAULT_CAP_MULTIPLE)), + ) + metrics.apply(days, settings) + totals = metrics.summarize(days) + rollups = aggregate.rollup(days) + date_keys = sorted({d.date_key for d in days}) + + scored_names = {d.campaign for d in days} + acts = actions_mod.build(events, date_keys, scored_names) + act_summary = actions_mod.summarize(acts) + + data = payload.build(days, totals, rollups, qas, metas, settings, date_keys, + join_report, overlap_rows, acts, act_summary) + problems = check_invariants(days) + data["invariants"] = {"checked": len(days), "failed": problems[:5]} + data["skipped"] = skipped + + artifacts = { + "days": days, "totals": totals, "rollups": rollups, "qas": qas, + "metas": metas, "settings": settings, "date_keys": date_keys, + "join_report": join_report, "overlap_rows": overlap_rows, + "actions": acts, + } + return AnalysisResult(payload=data, artifacts=artifacts) diff --git a/app/services/email_service.py b/app/services/email_service.py new file mode 100644 index 0000000..5b1f43c --- /dev/null +++ b/app/services/email_service.py @@ -0,0 +1,149 @@ +"""Outbound email. + +`_send` never raises into a request path. A mail outage should leave the user +with "check your inbox" and a resend button, not a 500 -- and on the +forgot-password route, raising would also reveal whether the address existed. +""" + +from __future__ import annotations + +import logging +import os +from collections.abc import Sequence +from html import escape + +import httpx + +from app.config import get_settings + +log = logging.getLogger(__name__) + + +def _shell(name: str, heading: str, body_html: str) -> str: + """Plain, table-free HTML. Mail clients are not browsers.""" + greeting = f"Hi {escape(name)}," if name else "Hi," + return ( + '
' + f"

{escape(heading)}

" + f"

{greeting}

" + f"{body_html}" + '

' + "You are receiving this because someone used this address on the " + "PPC out-of-budget dashboard. If that was not you, ignore this message." + "

" + ) + + +def _button(url: str, label: str) -> str: + safe = escape(url, quote=True) + return ( + f'

{escape(label)}

' + '

' + "If the button does not work, paste this into your browser:

" + f'

{escape(url)}

' + ) + + +def _send(to: str, subject: str, html: str, + attachment: str | Sequence[str] | None = None) -> bool: + """Deliver one message. Returns True on success, False on any failure. + + The endpoint declares `to` as an array and attachments as `files` (plural). + httpx sends a single string for `to` as one repeated field, which the API + wraps back into a list, so one recipient needs no special handling. + """ + s = get_settings() + + if s.EMAIL_PROVIDER == "console": + log.info("EMAIL (console provider)\n to: %s\n subject: %s\n body:\n%s", + to, subject, html) + return True + + data = {"to": to, "subject": subject, "body": html, "content_type": "html"} + paths = [attachment] if isinstance(attachment, str) else list(attachment or ()) + handles = [] + try: + # The field name has to be "files"; anything else is accepted and then + # silently dropped, and the response comes back with attachments: []. + for path in paths: + handle = open(path, "rb") + handles.append(handle) + files = [("files", (os.path.basename(p), h)) + for p, h in zip(paths, handles)] or None + + resp = httpx.post( + s.EMAIL_ENDPOINT, + headers={"Authorization": f"Bearer {s.EMAIL_API_KEY.get_secret_value()}"}, + data=data, files=files, timeout=s.EMAIL_TIMEOUT_S, + ) + if resp.status_code >= 300: # the API answers 202 on success + log.error("email API returned %s for %s: %s", + resp.status_code, to, resp.text[:300]) + return False + return True + except (httpx.HTTPError, OSError) as exc: + log.error("could not send email to %s: %s", to, exc) + return False + finally: + for handle in handles: + handle.close() + + +def send_email(to: str, subject: str, html: str, + attachment: str | Sequence[str] | None = None) -> bool: + """Public wrapper for arbitrary messages. Prefer the named helpers below.""" + return _send(to, subject, html, attachment) + + +def send_verification_email(to: str, name: str, verify_url: str) -> bool: + hours = get_settings().VERIFY_TOKEN_TTL_HOURS + body = ( + "

Confirm this address to finish setting up your " + "account on the PPC out-of-budget dashboard.

" + + _button(verify_url, "Confirm my email") + + f'

' + f"This link works once and expires in {hours} hours.

" + ) + return _send(to, "Confirm your email address", + _shell(name, "One more step", body)) + + +def send_password_reset_email(to: str, name: str, reset_url: str) -> bool: + minutes = get_settings().RESET_TOKEN_TTL_MINUTES + body = ( + '

Use the button below to choose a new password.

' + + _button(reset_url, "Choose a new password") + + f'

' + f"This link works once and expires in {minutes} minutes. If you did not " + "ask for it, nothing has changed and you can ignore this.

" + ) + return _send(to, "Reset your password", _shell(name, "Password reset", body)) + + +def send_account_exists_email(to: str, name: str, login_url: str, forgot_url: str) -> bool: + """Sent when someone signs up with an address that already has an account. + + Signup answers identically either way, so this is what tells the real owner + what happened without telling the requester whether the address exists. + """ + body = ( + '

Someone just tried to create an account with this ' + "address, but you already have one.

" + + _button(login_url, "Sign in") + + '

Forgotten your password? ' + f'Reset it here.

' + ) + return _send(to, "You already have an account", + _shell(name, "You already have an account", body)) + + +def send_password_changed_email(to: str, name: str) -> bool: + body = ('

Your password has just been changed, and every ' + "browser that was signed in has been signed out.

" + '

If this was not you, reset your password immediately ' + "and tell whoever runs this dashboard.

") + return _send(to, "Your password was changed", + _shell(name, "Your password was changed", body)) diff --git a/app/services/exporters.py b/app/services/exporters.py new file mode 100644 index 0000000..4290d90 --- /dev/null +++ b/app/services/exporters.py @@ -0,0 +1,85 @@ +"""CSV and Excel renderings of an analysis, lifted out of serve.py. + +Same rule as `analysis`: standard library and `ppcbudget` only. +""" + +from __future__ import annotations + +import csv +import io +import shutil +import tempfile +from datetime import date +from pathlib import Path + +from ppcbudget import excelout + +CSV_MIME = "text/csv; charset=utf-8" +XLSX_MIME = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" + + +def export_stamp(artifacts: dict) -> str: + """The suffix both filenames carry: last day analysed, then today.""" + return f"{artifacts['date_keys'][-1]}_{date.today():%Y%m%d}" + + +def csv_bytes(artifacts: dict) -> bytes: + """One row per campaign-day. BOM-prefixed so Excel opens it as UTF-8.""" + return _csv(artifacts["days"], artifacts.get("actions") or {}).encode("utf-8-sig") + + +def xlsx_bytes(artifacts: dict) -> bytes: + """The full formatted workbook, built in a temp dir and read back.""" + out = Path(tempfile.mkdtemp()) / f"ppc-budget-report_{export_stamp(artifacts)}.xlsx" + try: + excelout.write_report(out, artifacts["days"], artifacts["totals"], + artifacts["rollups"], artifacts["qas"], artifacts["metas"], + artifacts["settings"], artifacts["date_keys"], + artifacts["join_report"], artifacts.get("overlap_rows", 0), + artifacts.get("actions")) + return out.read_bytes() + finally: + shutil.rmtree(out.parent, ignore_errors=True) + + +def _csv(days, actions: dict) -> str: + buf = io.StringIO() + w = csv.writer(buf, lineterminator="\r\n") + w.writerow([ + "date", "campaign", "eligible_hours", "in_budget_hours", "out_of_budget_hours", + "paused_hours", "pct_of_active_day", "budget_cap_hits", "distinct_outages", + "first_out", "last_recovery", "ended_out", "daily_budget", "budget_source", + "spend_rate_per_hour", "lost_spend", "lost_sales", "capped", "severity", + "diagnosis", "confidence", "uncertainty_hours", + "last_action", "days_since_action", "what_changed_last", "actions_in_window", + ]) + for d in sorted(days, key=lambda x: (-x.severity, x.campaign)): + lost = d.lost or {} + w.writerow([ + d.date_key, d.campaign, f"{d.eligible_min / 60:.2f}", f"{d.in_hours:.2f}", + f"{d.oob_hours:.2f}", f"{d.paused_hours:.2f}", f"{d.oob_share:.4f}", + d.episodes_raw, d.episodes_merged, + excelout.hhmm(d.first_oob_min), excelout.hhmm(d.last_recovery_min), + "yes" if d.closed_oob else "no", + # Deliberately blank, never 0, when unobserved. + f"{d.budget.time_weighted:.2f}" if d.budget.time_weighted else "", + d.budget.source, + f"{lost['spend_rate_per_hour']:.4f}" if lost.get("spend_rate_per_hour") else "", + f"{lost['lost_spend']:.2f}" if lost.get("lost_spend") is not None else "", + f"{lost['lost_sales']:.2f}" if lost.get("lost_sales") is not None else "", + "yes" if lost.get("capped") else "", + f"{d.severity:.1f}", d.diagnosis, d.confidence, + f"{d.oob_uncertainty_min / 60:.2f}" if d.chain_breaks else "", + *_action_columns(actions.get(d.campaign)), + ]) + return buf.getvalue() + + +def _action_columns(act) -> tuple: + """Last meaningful action, or an explicit statement that there was none.""" + if act is None: + return ("not observed", "", "", "") + return (act.summary, + "" if act.days_since is None else act.days_since, + act.last_label or "", + act.count or "") diff --git a/app/services/ratelimit.py b/app/services/ratelimit.py new file mode 100644 index 0000000..1652aa3 --- /dev/null +++ b/app/services/ratelimit.py @@ -0,0 +1,59 @@ +"""A sliding-window rate limiter that needs no infrastructure. + +The app is pinned to a single worker by its in-memory workspaces, so an +in-process limiter is exactly as effective as Redis would be here, and one +fewer thing to run. +""" + +from __future__ import annotations + +import threading +import time +from collections import defaultdict, deque + +# key -> (limit, window seconds) +BUDGETS: dict[str, tuple[int, float]] = { + "login:ip": (20, 600), + "login:email": (10, 600), + "signup:ip": (5, 3600), + "forgot:ip": (5, 3600), + "forgot:email": (3, 3600), + "resend:email": (3, 3600), + "analyze:user": (10, 300), +} + + +class SlidingWindow: + def __init__(self) -> None: + self._hits: dict[str, deque[float]] = defaultdict(deque) + self._lock = threading.Lock() + + def check(self, bucket: str, subject: str) -> float: + """Seconds to wait, or 0.0 if the call is allowed. Records the hit.""" + limit, window = BUDGETS[bucket] + key = f"{bucket}:{subject}" + now = time.monotonic() + with self._lock: + q = self._hits[key] + while q and now - q[0] > window: + q.popleft() + if len(q) >= limit: + return max(1.0, window - (now - q[0])) + q.append(now) + return 0.0 + + def reset(self, bucket: str, subject: str) -> None: + """Forget a subject's history -- called after a successful login.""" + with self._lock: + self._hits.pop(f"{bucket}:{subject}", None) + + def prune(self) -> None: + """Drop empty queues so the dictionary cannot grow without bound.""" + now = time.monotonic() + with self._lock: + for key in [k for k, q in self._hits.items() + if not q or now - q[-1] > max(w for _, w in BUDGETS.values())]: + self._hits.pop(key, None) + + +limiter = SlidingWindow() diff --git a/app/services/store.py b/app/services/store.py new file mode 100644 index 0000000..2ec11a9 --- /dev/null +++ b/app/services/store.py @@ -0,0 +1,226 @@ +"""Per-user upload workspaces. + +The local serve.py keeps one process-wide Session; hosted mode keeps one of +these per signed-in user. Two lock levels, and the order matters: the store +lock only ever guards the dictionary, never file I/O and never the analysis +itself, so one user's thirty-second run cannot block another user's file list. +""" + +from __future__ import annotations + +import logging +import re +import shutil +import threading +import time +from dataclasses import dataclass, field +from pathlib import Path, PurePosixPath +from typing import BinaryIO + +log = logging.getLogger(__name__) + +ALLOWED_SUFFIXES = (".xlsx", ".xlsm", ".csv", ".tsv") +_UNSAFE = re.compile(r"[^A-Za-z0-9._-]") +_CHUNK = 1 << 20 + + +class UploadTooLarge(Exception): + """Raised when a file, or the workspace total, would exceed its cap.""" + + +def safe_name(raw: str, index: int) -> str: + """A filename that cannot escape the workspace directory. + + serve.py used `Path(name).name`, which is Windows-shaped: on Linux a + backslash is an ordinary filename character, so 'a\\b.xlsx' comes back whole + and '..' survives intact. Strip to an explicit allowlist instead. + """ + base = PurePosixPath(raw.replace("\\", "/")).name + base = _UNSAFE.sub("_", base).lstrip(".")[:100] + if not base: + base = "upload.xlsx" + if not base.lower().endswith(ALLOWED_SUFFIXES): + raise ValueError("Only .xlsx, .xlsm, .csv and .tsv files can be analysed.") + return f"{index}_{base}" + + +@dataclass +class UserWorkspace: + """One user's uploads and their most recent analysis.""" + + user_id: int + dir: Path + history: list[Path] = field(default_factory=list) + perf: Path | None = None + last: dict | None = None # AnalysisResult.artifacts + bytes_used: int = 0 + touched: float = field(default_factory=time.monotonic) + lock: threading.Lock = field(default_factory=threading.Lock) + + def touch(self) -> None: + self.touched = time.monotonic() + + def add(self, filename: str, src: BinaryIO, kind: str, + max_file_bytes: int, max_total_bytes: int) -> Path: + """Stream an upload to disk, enforcing both caps as it goes. + + The size is checked while writing rather than from Content-Length: that + header is absent under chunked encoding, and it is the client's claim + either way. + """ + target = self.dir / safe_name(filename, len(self.history)) + written = 0 + try: + with target.open("wb") as fh: + while chunk := src.read(_CHUNK): + written += len(chunk) + if written > max_file_bytes: + raise UploadTooLarge("That file is too large.") + if self.bytes_used + written > max_total_bytes: + raise UploadTooLarge( + "That would use more working space than one account is " + "allowed. Start over to clear the files you have loaded.") + fh.write(chunk) + except BaseException: + target.unlink(missing_ok=True) + raise + + if not written: + target.unlink(missing_ok=True) + raise ValueError("That file was empty.") + + self.bytes_used += written + if kind == "perf": + # Replacing the performance report leaves the old file behind on + # disk; it is bounded by the workspace cap and cleared on reset. + self.perf = target + else: + self.history.append(target) + return target + + def names(self) -> dict: + """What /api/state reports: original filenames, index prefix removed.""" + return { + "history": [p.name.split("_", 1)[-1] for p in self.history], + "perf": self.perf.name.split("_", 1)[-1] if self.perf else None, + } + + def clear(self) -> None: + shutil.rmtree(self.dir, ignore_errors=True) + self.dir.mkdir(parents=True, exist_ok=True) + self.history.clear() + self.perf = None + self.last = None + self.bytes_used = 0 + + def dispose(self) -> None: + self.last = None + shutil.rmtree(self.dir, ignore_errors=True) + + +class WorkspaceStore: + """Workspaces keyed by user id, with idle and size eviction.""" + + def __init__(self, root: Path, *, idle_ttl_s: float, max_workspaces: int, + max_total_bytes: int) -> None: + self.root = root + self.idle_ttl_s = idle_ttl_s + self.max_workspaces = max_workspaces + self.max_total_bytes = max_total_bytes + self._spaces: dict[int, UserWorkspace] = {} + self._lock = threading.RLock() + + def start(self) -> None: + """Begin from a clean slate. Nothing here is meant to outlive a restart.""" + shutil.rmtree(self.root, ignore_errors=True) + self.root.mkdir(parents=True, exist_ok=True) + + def get(self, user_id: int) -> UserWorkspace: + with self._lock: + ws = self._spaces.get(user_id) + if ws is None: + path = self.root / f"u{user_id}" + shutil.rmtree(path, ignore_errors=True) + path.mkdir(parents=True, exist_ok=True) + try: + path.chmod(0o700) + except OSError: # no-op on Windows + pass + ws = UserWorkspace(user_id=user_id, dir=path) + self._spaces[user_id] = ws + ws.touch() + return ws + + def peek(self, user_id: int) -> UserWorkspace | None: + with self._lock: + return self._spaces.get(user_id) + + def drop(self, user_id: int) -> None: + with self._lock: + ws = self._spaces.pop(user_id, None) + if ws is not None: + ws.dispose() + + def total_bytes(self) -> int: + with self._lock: + return sum(w.bytes_used for w in self._spaces.values()) + + def sweep(self) -> int: + """Evict idle workspaces, then the least recently used ones until the + count and byte budgets are satisfied. Returns how many were dropped.""" + now = time.monotonic() + with self._lock: + doomed = [uid for uid, w in self._spaces.items() + if now - w.touched > self.idle_ttl_s] + + survivors = sorted( + ((uid, w) for uid, w in self._spaces.items() if uid not in doomed), + key=lambda kv: kv[1].touched, + ) + while len(survivors) > self.max_workspaces: + uid, _ = survivors.pop(0) + doomed.append(uid) + + total = sum(w.bytes_used for _, w in survivors) + while total > self.max_total_bytes and survivors: + uid, w = survivors.pop(0) + total -= w.bytes_used + doomed.append(uid) + + evicted = [self._spaces.pop(uid) for uid in doomed if uid in self._spaces] + + # Deliberately outside the lock: rmtree of a few hundred megabytes + # would otherwise stall every other request touching the store. + for ws in evicted: + ws.dispose() + if evicted: + log.info("evicted %d workspace(s)", len(evicted)) + return len(evicted) + + def dispose_all(self) -> None: + with self._lock: + spaces, self._spaces = list(self._spaces.values()), {} + for ws in spaces: + ws.dispose() + shutil.rmtree(self.root, ignore_errors=True) + + def __len__(self) -> int: + with self._lock: + return len(self._spaces) + + +# The process-wide store. Created in the app lifespan, which is also why this +# app must run with a single worker: a second process would have its own store +# and a user's upload and analysis could land in different ones. +_store: WorkspaceStore | None = None + + +def set_store(store: WorkspaceStore) -> None: + global _store + _store = store + + +def store_of() -> WorkspaceStore: + if _store is None: + raise RuntimeError("Workspace store not initialised; the app lifespan sets it.") + return _store diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..7a81901 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,36 @@ +# The database is the existing RDS instance, so there is no mysql service here. +# The EC2 host's security group must be allowed inbound on 3306 at RDS. + +services: + app: + build: . + image: oob-dashboard:latest + container_name: oob-dashboard + env_file: .env + environment: + # Overrides .env: inside the container the workspace lives on the volume. + WORKSPACE_ROOT: /srv/work + ports: + - "8000:8000" + restart: unless-stopped + healthcheck: + test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=3).status == 200 else 1)"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 15s + volumes: + # A named volume, not tmpfs: tmpfs is RAM, and a few hundred megabytes of + # uploads would come straight out of the memory budget that the analysis + # itself needs. The app wipes this directory on startup instead. + - workspaces:/srv/work + # An OOM then kills and restarts this container rather than the host. + mem_limit: 2g + logging: + driver: json-file + options: + max-size: "10m" + max-file: "3" + +volumes: + workspaces: diff --git a/ppcbudget/excelout.py b/ppcbudget/excelout.py index 05d2a67..a09b4ae 100644 --- a/ppcbudget/excelout.py +++ b/ppcbudget/excelout.py @@ -26,23 +26,28 @@ from .scoring import CampaignDay # ------------------------------------------------------------------- palette -NAVY = "1F3864" -SLATE = "44546A" +# Utopia Brands. The report is the artefact that leaves the building, so it +# carries the same identity as the dashboard: deep green chrome on pale mint +# panels. RED and AMBER are functional, not brand -- the guide has no warning +# colour and a report about campaigns going dark needs one. +DEEP = "004D43" # brand primary, deep green +MIST = "EAFFF4" # brand primary, pale mint +SLATE = "4A6B64" WHITE = "FFFFFF" -GREEN = "16A34A" -AMBER = "B45309" -RED = "C0392B" -GREY = "9E9E9E" -PANEL = "F4F6FA" +GREEN = "0F7A5C" +AMBER = "A2600A" +RED = "C9401F" +GREY = "7C9A92" +PANEL = MIST -HDR_FILL = PatternFill("solid", fgColor=NAVY) +HDR_FILL = PatternFill("solid", fgColor=DEEP) HDR_FONT = Font(color=WHITE, bold=True, size=10) -TITLE_FONT = Font(color=NAVY, bold=True, size=20) +TITLE_FONT = Font(color=DEEP, bold=True, size=20) SUB_FONT = Font(color=SLATE, size=10, italic=True) -SECTION = Font(color=NAVY, bold=True, size=12) +SECTION = Font(color=DEEP, bold=True, size=12) TILE_FILL = PatternFill("solid", fgColor=PANEL) KPI_LABEL = Font(color=SLATE, size=9, bold=True) -KPI_VALUE = Font(color=NAVY, size=18, bold=True) +KPI_VALUE = Font(color=DEEP, size=18, bold=True) KPI_ALARM = Font(color=RED, size=18, bold=True) KPI_NOTE = Font(color=GREY, size=8, italic=True) RUN_FONT = Font(color=GREEN, size=10) @@ -50,29 +55,34 @@ LOST_FONT = Font(color=RED, size=10, bold=True) UNPRICED = Font(color=GREY, size=9, italic=True) # Per-date sub-columns run narrow, so they get their own smaller type. DAY_RUN_FONT = Font(color=GREEN, size=9) -DAY_LOST_FONT = Font(color="7F1D1D", size=9, bold=True) +DAY_LOST_FONT = Font(color="8F2810", size=9, bold=True) DAY_PAUSE_FONT = Font(color=GREY, size=9) -EDGE = Side(style="thin", color="C9D2E3") +EDGE = Side(style="thin", color="CFE8DC") BOX = Border(left=EDGE, right=EDGE, top=EDGE, bottom=EDGE) -# Hour-of-day heat, reused so 60k cells share nine fill objects. +# Hour-of-day heat, reused so 60k cells share nine fill objects. The first entry +# is the "no time lost" bucket; the rest are one warm hue getting steadily +# darker, matching heatColor() in web/app.js so the report and the dashboard +# shade the same hour the same way. HEAT = [PatternFill("solid", fgColor=c) for c in - ("E8F5E9", "FFF9C4", "FFECB3", "FFE0B2", "FFCCBC", - "FFAB91", "FF8A65", "EF5350", "C62828")] -PAUSED_FILL = PatternFill("solid", fgColor="E0E0E0") -NA_FILL = PatternFill("solid", fgColor="F5F5F5") -HEAT_FONT = Font(size=7, color="616161") + ("D3F6E8", "FDECE7", "FBD7CD", "F9BFAE", "F7A58C", + "F4886A", "F2542D", "D8431F", "B53617")] +PAUSED_FILL = PatternFill("solid", fgColor="DDE5E3") +NA_FILL = PatternFill("solid", fgColor="F2F8F5") +HEAT_FONT = Font(size=7, color=SLATE) HEAT_FONT_DARK = Font(size=7, color=WHITE) +# Same tints the dashboard's diagnosis pills use, so a reader moving between +# the two sees one scheme. DIAGNOSIS_FILL = { - "Structurally underfunded": PatternFill("solid", fgColor="FFCDD2"), - "Exhausts early": PatternFill("solid", fgColor="FFE0B2"), - "Pacing thrash": PatternFill("solid", fgColor="E1BEE7"), - "Evening cap": PatternFill("solid", fgColor="FFF9C4"), - "Intermittent": PatternFill("solid", fgColor="E3F2FD"), - "Healthy": PatternFill("solid", fgColor="C8E6C9"), - "Mostly paused": PatternFill("solid", fgColor="ECEFF1"), + "Structurally underfunded": PatternFill("solid", fgColor="FFE4DC"), + "Exhausts early": PatternFill("solid", fgColor="FDEFD0"), + "Pacing thrash": PatternFill("solid", fgColor="E8E9FF"), + "Evening cap": PatternFill("solid", fgColor="FEF7E4"), + "Intermittent": PatternFill("solid", fgColor="F1FFD7"), + "Healthy": PatternFill("solid", fgColor="D3F6E8"), + "Mostly paused": PatternFill("solid", fgColor="E4F5EC"), } # A duration is a fraction of a day; [h] lets a total exceed 24 hours. @@ -137,9 +147,9 @@ def _as_table(ws, name: str, last_row: int, last_col: int, first_row: int = 1) - ws.add_table(table) -STALE_FILL = PatternFill("solid", fgColor="FFCDD2") -WARM_FILL = PatternFill("solid", fgColor="FFF3CD") -STALE_FONT = Font(color="7F1D1D", size=10, bold=True) +STALE_FILL = PatternFill("solid", fgColor="FFE4DC") +WARM_FILL = PatternFill("solid", fgColor="FDEFD0") +STALE_FONT = Font(color="8F2810", size=10, bold=True) def _action_cells(ws, row: int, col: int, act) -> None: @@ -373,9 +383,9 @@ def _sheet_campaigns_multi(wb: Workbook, rollups: list[CampaignRollup], for j in range(len(shown_dates)): letter = get_column_letter(base + 2 + j * 3) ws.conditional_formatting.add(f"{letter}2:{letter}{last_row}", ColorScaleRule( - start_type="num", start_value=0, start_color="E8F5E9", - mid_type="num", mid_value=0.5, mid_color="FFCC80", - end_type="num", end_value=1, end_color="C62828")) + start_type="num", start_value=0, start_color="D3F6E8", + mid_type="num", mid_value=0.5, mid_color="F7A58C", + end_type="num", end_value=1, end_color="8F2810")) _as_table(ws, "Campaigns", last_row, last_col) ws.freeze_panes = "B2" @@ -676,7 +686,7 @@ def _sheet_method(wb: Workbook, settings: ModelSettings, metas: list[WorkbookMet ] row = 4 for name, text in entries: - ws.cell(row=row, column=1, value=name).font = Font(bold=True, color=NAVY, size=10) + ws.cell(row=row, column=1, value=name).font = Font(bold=True, color=DEEP, size=10) ws.cell(row=row, column=1).alignment = Alignment(vertical="top") c = ws.cell(row=row, column=2, value=text) c.alignment = Alignment(wrap_text=True, vertical="top") @@ -684,7 +694,7 @@ def _sheet_method(wb: Workbook, settings: ModelSettings, metas: list[WorkbookMet row += 1 row += 1 - ws.cell(row=row, column=1, value="Generated").font = Font(bold=True, color=NAVY, size=10) + ws.cell(row=row, column=1, value="Generated").font = Font(bold=True, color=DEEP, size=10) ws.cell(row=row, column=2, value=f"{datetime.now():%Y-%m-%d %H:%M} from " + ", ".join(m.path.name for m in metas)) diff --git a/ppcbudget/payload.py b/ppcbudget/payload.py index 715e590..e754f23 100644 --- a/ppcbudget/payload.py +++ b/ppcbudget/payload.py @@ -12,11 +12,17 @@ from .ingest import QaReport, WorkbookMeta from .metrics import ModelSettings, Totals, hourly_starvation from .scoring import IN, NA, OOB, PAUSED, CampaignDay +# The timeline strips are one gradient per campaign-day, so these four values +# are baked into the payload and have to hold up on both the light and the dark +# surface. Checked for colour-vision separation across every pair, not just +# neighbours: the tightest is paused against in-budget at dE 9.3 (protan) and +# 15.8 with normal vision. Paused is deliberately a cool neutral -- tinting it +# green to match the brand put it right on top of in-budget. TRACK_COLOR = { - IN: "#16a34a", - OOB: "#dc2626", - PAUSED: "#9ca3af", - NA: "#e5e7eb", + IN: "#0f9a74", # brand deep green lifted to read on a dark surface + OOB: "#f2542d", # functional warning; the brand guide has no such colour + PAUSED: "#9aa6ad", # nothing is happening, so it recedes + NA: "#cbdbd4", # not yet created: near-absent, labelled in the legend } STATE_LABEL = {IN: "In budget", OOB: "Out of budget", PAUSED: "Paused", NA: "Not yet created"} diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..20b30d6 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,25 @@ +# Hosted mode only. serve.py and run_report.py need nothing but openpyxl. + +# --- web --- +fastapi>=0.115,<1.0 +uvicorn[standard]>=0.34,<1.0 +python-multipart>=0.0.20 # Starlette needs it to parse multipart uploads + +# --- data --- +sqlalchemy>=2.0.36,<3.0 +pymysql>=1.1.1 # pure-python driver, so the image needs no compiler +cryptography>=43.0 # PyMySQL needs it for caching_sha2_password (RDS MySQL 8 default) + +# --- config / validation --- +pydantic>=2.9,<3 +pydantic-settings>=2.6,<3 # pulls in python-dotenv +email-validator>=2.2 # required by pydantic EmailStr + +# --- auth --- +argon2-cffi>=23.1 # not passlib: its last release is 2020 and it breaks on bcrypt>=4.1 + +# --- outbound --- +httpx>=0.27 # multipart POST to EMAIL_ENDPOINT + +# --- analysis --- +openpyxl>=3.1.5 diff --git a/serve.py b/serve.py index 39e7f0e..6670211 100644 --- a/serve.py +++ b/serve.py @@ -7,8 +7,9 @@ Opens http://localhost:8765 in your browser. Drag change-history exports onto the page and the dashboard appears. Everything runs on this machine -- the server binds to localhost only and nothing is uploaded anywhere. -The analysis is the same code the Excel report uses, so the two can never -disagree. +This is the single-user local mode: no accounts, no database, no dependencies +beyond openpyxl. The hosted multi-user version is `app.main`, run under uvicorn; +both call the same pipeline in app/services, so the two can never disagree. """ from __future__ import annotations @@ -20,16 +21,13 @@ import tempfile import threading import traceback import webbrowser -from datetime import date from http import HTTPStatus from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path from urllib.parse import parse_qs, urlparse -from ppcbudget import actions as actions_mod -from ppcbudget import aggregate, excelout, metrics, payload, perfjoin -from ppcbudget.ingest import dedupe_events, load_history -from ppcbudget.scoring import check_invariants, score_all +from app.services import exporters +from app.services.analysis import run_analysis HERE = Path(__file__).resolve().parent WEB = HERE / "web" @@ -76,66 +74,9 @@ SESSION = Session() def analyze(settings_in: dict) -> dict: """Run the pipeline over everything uploaded so far.""" - if not SESSION.history: - raise ValueError("No change-history files uploaded yet.") - - events, metas, qas, skipped = [], [], [], [] - for path in SESSION.history: - try: - evs, meta, qa = load_history(path) - except (ValueError, KeyError, OSError) as exc: - skipped.append(f"{path.name}: {exc}") - continue - events.extend(evs) - metas.append(meta) - qas.append(qa) - - if not events: - detail = " ".join(skipped) or "no readable rows" - raise ValueError(f"None of the files could be read as a change-history export. {detail}") - - events, overlap_rows = dedupe_events(events) - days = score_all(events, merge_gap_min=int(settings_in.get("merge_gap", 5))) - if not days: - raise ValueError("No campaigns had budget-state changes, so there is nothing to score.") - - join_report = None - roas_source = "account_average" - if SESSION.perf: - records, join_report = perfjoin.load_performance(SESSION.perf) - perfjoin.apply_to(days, records, join_report) - roas_source = "campaign" - - account_roas = next((m.roas for m in metas if m.roas), None) - roas_override = settings_in.get("roas") - settings = metrics.ModelSettings( - roas=float(roas_override) if roas_override else (account_roas or 4.0), - roas_source="override" if roas_override else roas_source, - haircut=float(settings_in.get("haircut", metrics.DEFAULT_ROAS_HAIRCUT)), - cap_multiple=float(settings_in.get("cap", metrics.DEFAULT_CAP_MULTIPLE)), - ) - metrics.apply(days, settings) - totals = metrics.summarize(days) - rollups = aggregate.rollup(days) - date_keys = sorted({d.date_key for d in days}) - - scored_names = {d.campaign for d in days} - acts = actions_mod.build(events, date_keys, scored_names) - act_summary = actions_mod.summarize(acts) - - data = payload.build(days, totals, rollups, qas, metas, settings, date_keys, - join_report, overlap_rows, acts, act_summary) - problems = check_invariants(days) - data["invariants"] = {"checked": len(days), "failed": problems[:5]} - data["skipped"] = skipped - - SESSION.last = { - "days": days, "totals": totals, "rollups": rollups, "qas": qas, - "metas": metas, "settings": settings, "date_keys": date_keys, - "join_report": join_report, "overlap_rows": overlap_rows, - "actions": acts, - } - return data + result = run_analysis(SESSION.history, SESSION.perf, settings_in) + SESSION.last = result.artifacts + return result.payload class Handler(BaseHTTPRequestHandler): @@ -236,71 +177,17 @@ class Handler(BaseHTTPRequestHandler): if not last: self._error("Analyse some files first.") return - stamp = f"{last['date_keys'][-1]}_{date.today():%Y%m%d}" + stamp = exporters.export_stamp(last) if fmt == "csv": - body = _csv(last["days"], last.get("actions") or {}).encode("utf-8-sig") - self._send(HTTPStatus.OK, body, "text/csv; charset=utf-8", + self._send(HTTPStatus.OK, exporters.csv_bytes(last), exporters.CSV_MIME, {"Content-Disposition": f'attachment; filename="ppc-budget_{stamp}.csv"'}) return - out = Path(tempfile.mkdtemp()) / f"ppc-budget-report_{stamp}.xlsx" - excelout.write_report(out, last["days"], last["totals"], last["rollups"], - last["qas"], last["metas"], last["settings"], - last["date_keys"], last["join_report"], - last.get("overlap_rows", 0), last.get("actions")) - body = out.read_bytes() - shutil.rmtree(out.parent, ignore_errors=True) - self._send(HTTPStatus.OK, body, - "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", - {"Content-Disposition": f'attachment; filename="{out.name}"'}) - - -def _csv(days, actions: dict) -> str: - import csv - import io - - buf = io.StringIO() - w = csv.writer(buf, lineterminator="\r\n") - w.writerow([ - "date", "campaign", "eligible_hours", "in_budget_hours", "out_of_budget_hours", - "paused_hours", "pct_of_active_day", "budget_cap_hits", "distinct_outages", - "first_out", "last_recovery", "ended_out", "daily_budget", "budget_source", - "spend_rate_per_hour", "lost_spend", "lost_sales", "capped", "severity", - "diagnosis", "confidence", "uncertainty_hours", - "last_action", "days_since_action", "what_changed_last", "actions_in_window", - ]) - for d in sorted(days, key=lambda x: (-x.severity, x.campaign)): - lost = d.lost or {} - w.writerow([ - d.date_key, d.campaign, f"{d.eligible_min / 60:.2f}", f"{d.in_hours:.2f}", - f"{d.oob_hours:.2f}", f"{d.paused_hours:.2f}", f"{d.oob_share:.4f}", - d.episodes_raw, d.episodes_merged, - excelout.hhmm(d.first_oob_min), excelout.hhmm(d.last_recovery_min), - "yes" if d.closed_oob else "no", - # Deliberately blank, never 0, when unobserved. - f"{d.budget.time_weighted:.2f}" if d.budget.time_weighted else "", - d.budget.source, - f"{lost['spend_rate_per_hour']:.4f}" if lost.get("spend_rate_per_hour") else "", - f"{lost['lost_spend']:.2f}" if lost.get("lost_spend") is not None else "", - f"{lost['lost_sales']:.2f}" if lost.get("lost_sales") is not None else "", - "yes" if lost.get("capped") else "", - f"{d.severity:.1f}", d.diagnosis, d.confidence, - f"{d.oob_uncertainty_min / 60:.2f}" if d.chain_breaks else "", - *_action_columns(actions.get(d.campaign)), - ]) - return buf.getvalue() - - -def _action_columns(act) -> tuple: - """Last meaningful action, or an explicit statement that there was none.""" - if act is None: - return ("not observed", "", "", "") - return (act.summary, - "" if act.days_since is None else act.days_since, - act.last_label or "", - act.count or "") + self._send(HTTPStatus.OK, exporters.xlsx_bytes(last), exporters.XLSX_MIME, + {"Content-Disposition": + f'attachment; filename="ppc-budget-report_{stamp}.xlsx"'}) def main(argv: list[str] | None = None) -> int: diff --git a/tests/synthetic.py b/tests/synthetic.py new file mode 100644 index 0000000..5ecce46 --- /dev/null +++ b/tests/synthetic.py @@ -0,0 +1,72 @@ +"""A tiny change-history export, shaped like Amazon's. + +The golden tests run against a real export that cannot be committed. This one +is small and synthetic: enough for the hosted app's tests to upload something +and get a real analysis back, without depending on customer data. +""" + +from __future__ import annotations + +from pathlib import Path + +import openpyxl + +HEADER = ["Change level type", "Change level name", "Campaign", "Change type", + "From", "To", "From (numeric)", "To (numeric)", "Date and time (ISO)"] + +DATE = "2026-08-06" + +ROWS = [ + # Out of budget mid-morning, never recovers. + ("Campaign", "Chronic Offender", "Chronic Offender", "Campaign created", "", "", None, None, f"{DATE}T00:00:00"), + ("Campaign", "Chronic Offender", "Chronic Offender", "Campaign daily budget", "$50.00", "$50.00", 50, 50, f"{DATE}T00:01:00"), + ("Campaign", "Chronic Offender", "Chronic Offender", "Campaign status", "Out of budget", "In budget", None, None, f"{DATE}T00:02:00"), + ("Campaign", "Chronic Offender", "Chronic Offender", "Campaign status", "In budget", "Out of budget", None, None, f"{DATE}T09:30:00"), + # Flaps in the evening. + ("Campaign", "Evening Cap", "Evening Cap", "Campaign daily budget", "$120.00", "$120.00", 120, 120, f"{DATE}T00:01:00"), + ("Campaign", "Evening Cap", "Evening Cap", "Campaign status", "Out of budget", "In budget", None, None, f"{DATE}T00:03:00"), + ("Campaign", "Evening Cap", "Evening Cap", "Campaign status", "In budget", "Out of budget", None, None, f"{DATE}T18:00:00"), + ("Campaign", "Evening Cap", "Evening Cap", "Campaign status", "Out of budget", "In budget", None, None, f"{DATE}T19:00:00"), + ("Campaign", "Evening Cap", "Evening Cap", "Campaign status", "In budget", "Out of budget", None, None, f"{DATE}T21:00:00"), + # Healthy, with a human budget edit so "last action" has something to find. + ("Campaign", "Healthy One", "Healthy One", "Campaign status", "Out of budget", "In budget", None, None, f"{DATE}T00:04:00"), + ("Campaign", "Healthy One", "Healthy One", "Campaign daily budget", "$80.00", "$95.00", 80, 95, f"{DATE}T11:15:00"), + # Paused for the afternoon. + ("Campaign", "Half Paused", "Half Paused", "Campaign status", "Out of budget", "In budget", None, None, f"{DATE}T00:05:00"), + ("Campaign", "Half Paused", "Half Paused", "Campaign status", "Delivering", "Paused", None, None, f"{DATE}T12:00:00"), + # Account level, no campaign: a row the reader is meant to drop. + ("Account", "", "", "Campaign status", "In budget", "Out of budget", None, None, f"{DATE}T13:00:00"), +] + +CAMPAIGNS = 4 # campaign-days the pipeline should score + + +def build(out: Path) -> Path: + wb = openpyxl.Workbook() + history = wb.active + history.title = "History" + history.append(HEADER) + for row in ROWS: + history.append(list(row)) + + meta = wb.create_sheet("Extraction Metadata") + for pair in (("Account", "Synthetic Test Account"), + ("Marketplace", "United States"), + ("Date range", f"{DATE} to {DATE}"), + ("Extraction run ID", "synthetic-0001"), + ("Status", "Complete"), + ("Rows expected", len(ROWS)), + ("Rows exported", len(ROWS)), + ("Duplicate rows skipped", 0), + ("Pages processed", 1)): + meta.append(list(pair)) + + summary = wb.create_sheet("Summary Metrics") + summary.append(["Metric", "Value"]) + for pair in (("Spend", 1500.0), ("Sales", 6000.0), ("ROAS", 4.0), + ("Impressions", 250000)): + summary.append(list(pair)) + + out.parent.mkdir(parents=True, exist_ok=True) + wb.save(out) + return out diff --git a/tests/test_auth_smoke.py b/tests/test_auth_smoke.py new file mode 100644 index 0000000..8634935 --- /dev/null +++ b/tests/test_auth_smoke.py @@ -0,0 +1,328 @@ +#!/usr/bin/env python3 +"""End-to-end checks for the hosted app. + + python3 tests/test_auth_smoke.py + +Runs against in-memory SQLite with email routed to a capture list, so it needs +no MySQL, no network, and sends no mail. Every model column is a portable type +precisely so this is possible. + +Plain asserts and a main(), matching tests/test_golden.py -- no pytest. +""" + +from __future__ import annotations + +import os +import sys +import tempfile +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT)) + +WORK = Path(tempfile.mkdtemp(prefix="oob-authtest-")) + +# Must be set before app.config reads the environment. +os.environ["EMAIL_PROVIDER"] = "console" +os.environ["APP_BASE_URL"] = "http://testserver" +os.environ["WORKSPACE_ROOT"] = str(WORK) +os.environ.setdefault("SECRET_KEY", "test-secret-key-at-least-32-characters-long") +os.environ.setdefault("MYSQL_HOST", "unused-in-this-test") +os.environ.setdefault("MYSQL_USER", "unused") +os.environ.setdefault("MYSQL_PASSWORD", "unused") +os.environ.setdefault("MYSQL_DATABASE", "unused") +os.environ.setdefault("ADMIN_EMAIL", "admin@example.com") +os.environ.setdefault("ADMIN_PASSWORD", "admin-password-1") + +from sqlalchemy import create_engine # noqa: E402 +from sqlalchemy.orm import sessionmaker # noqa: E402 +from sqlalchemy.pool import StaticPool # noqa: E402 + +import app.db as db_module # noqa: E402 + +# Point the app at SQLite before anything imports SessionLocal from it. +_sqlite = create_engine("sqlite://", connect_args={"check_same_thread": False}, + poolclass=StaticPool) +db_module.engine = _sqlite +db_module.SessionLocal = sessionmaker(bind=_sqlite, autoflush=False, + expire_on_commit=False, future=True) + +from fastapi.testclient import TestClient # noqa: E402 + +from app.config import get_settings # noqa: E402 +from app.main import app # noqa: E402 +from app.services import email_service # noqa: E402 +from tests import synthetic # noqa: E402 + +SENT: list[tuple[str, str, str]] = [] +email_service._send = lambda to, subject, html, attachment=None: ( + SENT.append((to, subject, html)) or True) + +FIXTURE = synthetic.build(WORK / "amazon-ads-history_synthetic.xlsx") +GENERIC_LOGIN_ERROR = "Email or password is incorrect." + +PASSED: list[str] = [] +FAILED: list[str] = [] + + +def check(label: str, condition: bool, detail: object = "") -> None: + if condition: + PASSED.append(label) + print(f" pass {label}") + else: + FAILED.append(label) + print(f" FAIL {label}" + (f" <- {detail}" if detail else "")) + + +def link_from_mail(prefix: str) -> str: + """Pull the most recent link with this prefix out of the captured HTML.""" + for _to, _subject, html in reversed(SENT): + i = html.find(prefix) + if i >= 0: + return html[i:].split('"')[0].split("<")[0].strip() + raise AssertionError(f"no email contained a {prefix!r} link") + + +def token_from_mail(prefix: str) -> str: + return link_from_mail(prefix).split("token=")[1] + + +def verify_new_user(client: TestClient, email: str, password: str, name: str = "") -> None: + client.post("/api/auth/signup", + json={"email": email, "password": password, "name": name}) + client.post("/api/auth/verify", + json={"token": token_from_mail("http://testserver/verify?token=")}) + + +def main() -> int: + with TestClient(app, base_url="http://testserver") as c: + _anonymous(c) + _signup_and_verify(c) + _login(c) + _dashboard(c) + _csrf_and_admin_gate(c) + _isolation(c) + _logout(c) + _forgot_and_reset(c) + _lockout(c) + _admin(c) + + print(f"\n{len(PASSED)}/{len(PASSED) + len(FAILED)} passed") + if FAILED: + print("failed: " + ", ".join(FAILED)) + return 1 if FAILED else 0 + + +# --------------------------------------------------------------------- checks + +def _anonymous(c: TestClient) -> None: + r = c.get("/", follow_redirects=False) + check("anonymous / redirects to the sign-in page", + r.status_code == 303 and r.headers["location"] == "/login", r.status_code) + check("healthz is up", c.get("/healthz").status_code == 200) + check("readyz reports the database", c.get("/readyz").json().get("db") == "ok") + check("anonymous api call is 401", c.get("/api/state").status_code == 401) + + r = c.get("/api/state", headers={"Accept": "text/html"}, follow_redirects=False) + check("a browser navigation gets redirected, not JSON", + r.status_code == 303 and "/login" in r.headers["location"], r.status_code) + + +def _signup_and_verify(c: TestClient) -> None: + body = {"email": "alice@example.com", "password": "correct-horse-1", "name": "Alice"} + first = c.post("/api/auth/signup", json=body) + check("signup succeeds", first.status_code == 200, first.text) + + repeat = c.post("/api/auth/signup", json=body) + check("signing up an existing address answers identically", + repeat.status_code == first.status_code and repeat.json() == first.json(), + repeat.text) + + r = c.post("/api/auth/signup", json={"email": "x@example.com", "password": "short"}) + check("a short password is refused", r.status_code == 422, r.status_code) + + token = token_from_mail("http://testserver/verify?token=") + r = c.get(f"/verify?token={token}") + check("the verify page is served as html", + r.status_code == 200 and "text/html" in r.headers["content-type"]) + + r = c.post("/api/auth/login", + json={"email": "alice@example.com", "password": "correct-horse-1"}) + check("signing in before confirming is refused with a reason", + r.status_code == 403 and r.json().get("code") == "email_not_verified", r.text) + + r = c.post("/api/auth/verify", json={"token": token}) + check("loading the page did not spend the token; posting it does", + r.status_code == 200, r.text) + r = c.post("/api/auth/verify", json={"token": token}) + check("a verification token works only once", r.status_code == 400, r.status_code) + + +def _login(c: TestClient) -> None: + r = c.post("/api/auth/login", + json={"email": "alice@example.com", "password": "wrong-password"}) + check("a wrong password is refused without detail", + r.status_code == 401 and r.json()["error"] == GENERIC_LOGIN_ERROR, r.text) + r = c.post("/api/auth/login", + json={"email": "nobody@example.com", "password": "wrong-password"}) + check("an unknown address gets the very same message", + r.status_code == 401 and r.json()["error"] == GENERIC_LOGIN_ERROR, r.text) + + r = c.post("/api/auth/login", + json={"email": "alice@example.com", "password": "correct-horse-1"}) + check("signing in works", r.status_code == 200, r.text) + + cookie = r.headers.get("set-cookie", "") + check("the session cookie is HttpOnly", "httponly" in cookie.lower(), cookie) + check("the session cookie is SameSite=Lax", "samesite=lax" in cookie.lower(), cookie) + check("the session cookie is not Secure over plain http", + "secure" not in cookie.lower(), cookie) + + state = c.get("/api/state") + check("the api answers once signed in", state.status_code == 200, state.text) + check("the state carries who is signed in", + state.json()["user"]["email"] == "alice@example.com") + check("the dashboard itself is served", c.get("/").status_code == 200) + + +def _dashboard(c: TestClient) -> None: + with FIXTURE.open("rb") as fh: + r = c.post("/api/upload", files={"file": (FIXTURE.name, fh)}, + data={"kind": "history"}) + check("an export uploads", r.status_code == 200, r.text) + check("the uploaded file is listed", + c.get("/api/state").json()["history"] == [FIXTURE.name]) + + r = c.post("/api/analyze", json={"haircut": 0.7, "cap": 3, "merge_gap": 5}) + check("the analysis runs", r.status_code == 200, r.text[:200]) + payload = r.json() + check("the payload has campaigns in it", bool(payload.get("campaigns")), + sorted(payload)[:6]) + + r = c.get("/api/export?format=csv") + check("the csv export downloads", + r.status_code == 200 and r.content[:3] == b"\xef\xbb\xbf", r.status_code) + check("the csv is sent as an attachment", + "attachment" in r.headers.get("content-disposition", "")) + r = c.get("/api/export?format=xlsx") + check("the excel export downloads", + r.status_code == 200 and r.content[:2] == b"PK", r.status_code) + + r = c.post("/api/upload", files={"file": ("payload.exe", b"x" * 32)}, + data={"kind": "history"}) + check("the file type is enforced on the server, not just in the browser", + r.status_code == 400, r.status_code) + + r = c.post("/api/clear") + check("starting over empties the workspace", + r.status_code == 200 and c.get("/api/state").json()["history"] == []) + + +def _csrf_and_admin_gate(c: TestClient) -> None: + r = c.post("/api/clear", headers={"Origin": "https://evil.example"}) + check("a write from another origin is refused", r.status_code == 403, r.status_code) + check("an ordinary member cannot reach the admin api", + c.get("/api/admin/users").status_code == 403) + r = c.get("/admin", follow_redirects=False) + check("an ordinary member is bounced off the admin page", + r.status_code == 303 and r.headers["location"] == "/", r.status_code) + + +def _isolation(c: TestClient) -> None: + with TestClient(app, base_url="http://testserver") as other: + verify_new_user(other, "bob@example.com", "another-good-1", "Bob") + other.post("/api/auth/login", + json={"email": "bob@example.com", "password": "another-good-1"}) + with FIXTURE.open("rb") as fh: + other.post("/api/upload", files={"file": ("bobs-file.xlsx", fh)}, + data={"kind": "history"}) + check("the second account sees its own upload", + other.get("/api/state").json()["history"] == ["bobs-file.xlsx"]) + check("the first account does not see it", + c.get("/api/state").json()["history"] == []) + check("and has no analysis to export from the other's data", + other.get("/api/export?format=csv").status_code == 400) + + +def _logout(c: TestClient) -> None: + check("signing out works", c.post("/api/auth/logout").status_code == 200) + check("the session is dead afterwards", c.get("/api/state").status_code == 401) + + +def _forgot_and_reset(c: TestClient) -> None: + before = len(SENT) + known = c.post("/api/auth/forgot", json={"email": "alice@example.com"}) + unknown = c.post("/api/auth/forgot", json={"email": "ghost@example.com"}) + check("forgot-password answers the same for a known and unknown address", + known.status_code == unknown.status_code and known.json() == unknown.json()) + check("but only the real address was actually emailed", len(SENT) == before + 1) + + token = token_from_mail("http://testserver/reset?token=") + r = c.post("/api/auth/reset", + json={"token": token, "password": "brand-new-secret-9"}) + check("the password is reset", r.status_code == 200, r.text) + r = c.post("/api/auth/reset", + json={"token": token, "password": "yet-another-one-1"}) + check("a reset token works only once", r.status_code == 400, r.status_code) + + r = c.post("/api/auth/login", + json={"email": "alice@example.com", "password": "correct-horse-1"}) + check("the old password stops working", r.status_code == 401) + r = c.post("/api/auth/login", + json={"email": "alice@example.com", "password": "brand-new-secret-9"}) + check("the new password works", r.status_code == 200, r.text) + c.post("/api/auth/logout") + + +def _lockout(c: TestClient) -> None: + s = get_settings() + for _ in range(s.MAX_FAILED_LOGINS + 1): + r = c.post("/api/auth/login", + json={"email": "bob@example.com", "password": "definitely-wrong"}) + check("a locked account is not announced as locked", + r.status_code == 401 and r.json()["error"] == GENERIC_LOGIN_ERROR, r.text) + r = c.post("/api/auth/login", + json={"email": "bob@example.com", "password": "another-good-1"}) + check("even the right password is refused while locked", + r.status_code == 401, r.status_code) + + +def _admin(c: TestClient) -> None: + from app.bootstrap import seed_admin + + s = get_settings() + with db_module.SessionLocal() as db: + seed_admin(db, s) + + r = c.post("/api/auth/login", + json={"email": s.ADMIN_EMAIL, + "password": s.ADMIN_PASSWORD.get_secret_value()}) + check("the seeded admin can sign in", r.status_code == 200, r.text) + + r = c.get("/api/admin/users") + check("the admin can list accounts", + r.status_code == 200 and len(r.json()["users"]) >= 3, r.text[:200]) + check("the admin page renders", c.get("/admin").status_code == 200) + + users = r.json()["users"] + bob = next(u for u in users if u["email"] == "bob@example.com") + check("the admin can clear a lockout", + c.post(f"/api/admin/users/{bob['id']}/activate").status_code == 200) + check("the admin can disable an account", + c.post(f"/api/admin/users/{bob['id']}/deactivate").status_code == 200) + + r = c.post("/api/auth/login", + json={"email": "bob@example.com", "password": "another-good-1"}) + check("a disabled account cannot sign in", r.status_code == 401, r.status_code) + + me = c.get("/api/auth/me").json()["user"] + check("the admin cannot disable themselves", + c.post(f"/api/admin/users/{me['id']}/deactivate").status_code == 400) + + +if __name__ == "__main__": + try: + raise SystemExit(main()) + finally: + import shutil + shutil.rmtree(WORK, ignore_errors=True) diff --git a/web/admin.html b/web/admin.html new file mode 100644 index 0000000..af9ec63 --- /dev/null +++ b/web/admin.html @@ -0,0 +1,52 @@ + + + + + +Accounts · Out-of-Budget Dashboard + + + + + +
+
+ +
+

Accounts

+

Who can sign in to this dashboard

+
+
+
+ + +
+
+ +
+
+
+
+

Accounts

+

Loading…

+
+
+ +
+ + + + + + + + +
EmailNameStatusRoleLast signed in
+
+
+
+ + + + + diff --git a/web/app.js b/web/app.js index d1955a0..9a9de7a 100644 --- a/web/app.js +++ b/web/app.js @@ -59,12 +59,13 @@ function renderFileList() { } async function upload(file, kind) { - const body = await file.arrayBuffer(); - const res = await fetch('/api/upload', { - method: 'POST', - headers: { 'X-Filename': encodeURIComponent(file.name).replace(/%20/g, ' '), 'X-Kind': kind }, - body, - }); + // FormData rather than a raw body: the filename travels in the part header, + // so no X-Filename escaping, and the server can stream it to disk instead of + // holding the whole thing in memory. + const form = new FormData(); + form.append('file', file); + form.append('kind', kind); + const res = await A.api('/api/upload', { method: 'POST', body: form }); if (!res.ok) throw new Error((await res.json().catch(() => ({}))).error || 'Upload failed'); state.files.push({ name: file.name, kind }); renderFileList(); @@ -92,7 +93,7 @@ async function analyze() { let i = 0; const tick = setInterval(() => { $('loading-text').textContent = steps[++i % steps.length]; }, 900); try { - const res = await fetch('/api/analyze', { + const res = await A.api('/api/analyze', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(state.settings), @@ -160,15 +161,20 @@ function renderAnswer(t, m) { `able to run — and ${hrs(a.out)} shut off because it hit its daily budget.` + (a.paused > 0.05 ? ` A further ${hrs(a.paused)} it was paused, which costs nothing.` : ''); + // Same four values as TRACK_COLOR in ppcbudget/payload.py -- the day bar and + // the timeline strips have to agree. const segs = [ - ['running', a.running, '#16a34a', 'Running'], - ['out', a.out, '#dc2626', 'Out of budget'], - ['paused', a.paused, '#9ca3af', 'Paused'], - ['na', a.na, '#e5e7eb', 'Not yet created'], + ['running', a.running, '#0f9a74', 'Running'], + ['out', a.out, '#f2542d', 'Out of budget'], + ['paused', a.paused, '#9aa6ad', 'Paused'], + ['na', a.na, '#cbdbd4', 'Not yet created'], ].filter(([, v]) => v > 0.01); - $('daybar').innerHTML = segs.map(([, v, color, label]) => - ` + `${(v / 24) > 0.13 ? hrs(v) : ''}`).join(''); $('daykeys').innerHTML = segs.map(([, v, color, label]) => `
${label} ${hrs(v)}
`).join(''); @@ -326,12 +332,15 @@ function setColumns(mode, multi) { wrap.classList.toggle('multi', mode === 'day' && multi); } -/** Colour for a day, on the same green-to-red scale as the hour heatmap. */ +/** Colour for a day: nothing lost is the in-budget green, and everything above + * that rides one warm ramp. A single hue getting steadily darker, rather than + * the yellow-orange-red rainbow it replaced -- with one hue, "worse" is + * readable from the depth of the colour alone. */ function heatColor(lostHours, eligibleHours) { const f = eligibleHours > 0 ? Math.min(1, lostHours / eligibleHours) : 0; - if (f <= 0.005) return '#16a34a'; - const ramp = ['#fff9c4', '#ffecb3', '#ffe0b2', '#ffccbc', '#ffab91', - '#ff8a65', '#ef5350', '#dc2626', '#b71c1c']; + if (f <= 0.005) return '#0f9a74'; + const ramp = ['#fdece7', '#fbd7cd', '#f9bfae', '#f7a58c', '#f4886a', + '#f2542d', '#d8431f', '#b53617', '#8f2810']; return ramp[Math.min(ramp.length - 1, Math.floor(f * ramp.length))]; } @@ -692,8 +701,13 @@ window.addEventListener('drop', (e) => e.preventDefault()); $('btn-analyze').addEventListener('click', analyze); $('btn-error-back').addEventListener('click', () => stage(state.data ? 'dash' : 'upload')); +$('btn-signout').addEventListener('click', async () => { + await fetch('/api/auth/logout', { method: 'POST' }); + location.replace('/login'); +}); + $('btn-reset').addEventListener('click', async () => { - await fetch('/api/clear', { method: 'POST' }); + await A.api('/api/clear', { method: 'POST' }); state.data = null; state.files = []; state.diagnoses.clear(); state.search = ''; $('search').value = ''; state.pricedOnly = false; $('only-priced').checked = false; state.staleOnly = false; $('only-stale').checked = false; @@ -701,8 +715,28 @@ $('btn-reset').addEventListener('click', async () => { stage('upload'); }); -$('btn-csv').addEventListener('click', () => { location.href = '/api/export?format=csv'; }); -$('btn-xlsx').addEventListener('click', () => { location.href = '/api/export?format=xlsx'; }); +// Fetched rather than navigated to, so a refusal renders in the UI instead of +// dumping JSON into a new tab. The session cookie rides along either way. +async function download(format) { + const res = await A.api('/api/export?format=' + format); + if (!res.ok) { + fail((await res.json().catch(() => ({}))).error || 'That export failed.'); + return; + } + const name = /filename="([^"]+)"/.exec(res.headers.get('content-disposition') || ''); + const url = URL.createObjectURL(await res.blob()); + const a = document.createElement('a'); + a.href = url; + a.download = name ? name[1] : 'ppc-budget-report.' + format; + document.body.appendChild(a); + a.click(); + a.remove(); + // Revoked on a later tick: Safari has not finished reading it synchronously. + setTimeout(() => URL.revokeObjectURL(url), 30000); +} + +$('btn-csv').addEventListener('click', () => download('csv')); +$('btn-xlsx').addEventListener('click', () => download('xlsx')); $('search').addEventListener('input', (ev) => { state.search = ev.target.value; applyFilters(); }); $('only-priced').addEventListener('change', (ev) => { state.pricedOnly = ev.target.checked; applyFilters(); }); @@ -775,8 +809,15 @@ $('settings').addEventListener('close', (ev) => { analyze(); }); -// Pick up anything preloaded from data/ on startup. -fetch('/api/state').then((r) => r.json()).then((s) => { +// Pick up whatever this account already has loaded. Routed through A.api so an +// expired session redirects to the sign-in page rather than silently rendering +// an empty dashboard. +A.api('/api/state').then((r) => r.json()).then((s) => { + if (s.user) { + $('whoami').textContent = s.user.name || s.user.email; + $('whoami').title = s.user.email; + $('link-admin').hidden = !s.user.is_admin; + } state.files = s.history.map((n) => ({ name: n, kind: 'history' })); if (s.perf) state.files.push({ name: s.perf, kind: 'perf' }); renderFileList(); diff --git a/web/auth.js b/web/auth.js new file mode 100644 index 0000000..1b870ef --- /dev/null +++ b/web/auth.js @@ -0,0 +1,83 @@ +'use strict'; + +// Shared by the auth pages and by app.js. Plain script, no modules -- same +// style as app.js, and it has to work when loaded before it. + +const A = { + qs(key) { + return new URLSearchParams(location.search).get(key); + }, + + // Redirect a 401 to the sign-in page instead of letting the caller render an + // empty, broken screen. Everything else is handed back for normal handling. + guard(res) { + if (res.status === 401) { + const here = location.pathname + location.search; + location.replace('/login?next=' + encodeURIComponent(here)); + throw new Error('not signed in'); + } + return res; + }, + + async api(url, opts) { + return A.guard(await fetch(url, opts)); + }, + + async postJSON(url, body) { + const res = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body), + }); + let json = {}; + try { json = await res.json(); } catch (_) { /* empty or non-JSON body */ } + if (!res.ok) { + const err = new Error(json.error || 'That did not work. Try again.'); + err.status = res.status; + err.code = json.code; + throw err; + } + return json; + }, + + msg(el, text, kind) { + if (!el) return; + el.textContent = text || ''; + el.className = 'msg' + (kind ? ' ' + kind : ''); + el.hidden = !text; + }, + + // Only ever bounce to a path on this site: '//evil.example' is a protocol + // relative URL and would leave it. + safeNext(raw) { + if (!raw || !raw.startsWith('/') || raw.startsWith('//')) return '/'; + return raw; + }, + + busy(button, on, labelWhenBusy) { + if (!button) return; + if (on) { + button.dataset.label = button.textContent; + button.textContent = labelWhenBusy || 'Working…'; + } else if (button.dataset.label) { + button.textContent = button.dataset.label; + } + button.disabled = on; + }, + + // Every auth page shares one submit shape: disable, call, show the outcome. + wire(form, button, note, handler) { + form.addEventListener('submit', async (ev) => { + ev.preventDefault(); + A.msg(note, ''); + A.busy(button, true); + try { + await handler(); + } catch (err) { + A.msg(note, String(err.message || err), 'err'); + } finally { + A.busy(button, false); + } + }); + }, +}; diff --git a/web/fonts/README.md b/web/fonts/README.md new file mode 100644 index 0000000..89238f0 --- /dev/null +++ b/web/fonts/README.md @@ -0,0 +1,35 @@ +# Brand fonts + +These files are not in the repository. Drop them here and the app picks them up +with no code change — `web/styles.css` already declares the `@font-face` rules +and the fallback stacks. + +| File | Face | Where it is used | Source | +| --- | --- | --- | --- | +| `Belleza-Regular.woff2` | Belleza | headings (`h1`, `h2`) | Google Fonts, SIL Open Font License | +| `NeueMontreal-Regular.woff2` | Neue Montreal | body copy, sub-headings | Pangram Pangram, commercial licence | +| `NeueMontreal-Medium.woff2` | Neue Montreal Medium | emphasis | as above | +| `NeueMontreal-Bold.woff2` | Neue Montreal Bold | strong emphasis | as above | + +Belleza is already wired up. For Neue Montreal, uncomment the `@font-face` +block at the top of `web/styles.css` once the files are here. + +## They have to be self-hosted + +The app sends `Content-Security-Policy: default-src 'self'`, so a stylesheet +that pulls fonts from `fonts.gstatic.com` is blocked by the browser and the +page silently falls back. Converting to `.woff2` and serving them from this +directory is the only route. + +To convert from `.ttf`/`.otf`: + +```bash +pip install fonttools brotli +python -c "from fontTools.ttLib import TTFont; f=TTFont('Belleza-Regular.ttf'); f.flavor='woff2'; f.save('Belleza-Regular.woff2')" +``` + +## Until then + +Headings fall back to Optima → Candara → Gill Sans → Trebuchet MS, and body +copy to the system UI sans. Both are chosen to sit close to the real faces, so +the layout does not shift when the files arrive. diff --git a/web/forgot.html b/web/forgot.html new file mode 100644 index 0000000..a219c19 --- /dev/null +++ b/web/forgot.html @@ -0,0 +1,42 @@ + + + + + +Reset your password · Out-of-Budget Dashboard + + + + + +
+
+ +
+

Out-of-Budget Dashboard

+

Amazon Ads change history → which campaigns keep going dark

+
+
+
+ +
+
+

Reset your password

+

We will email you a link to choose a new one.

+ +
+ + +
+ + +

Back to sign in

+
+
+ + + + + diff --git a/web/index.html b/web/index.html index cee3f10..cf661ce 100644 --- a/web/index.html +++ b/web/index.html @@ -4,7 +4,7 @@ PPC Out-of-Budget Dashboard - + @@ -22,6 +22,10 @@ + + + + @@ -40,7 +44,8 @@

Drop your change-history exports here

One file or a whole week of them. — or drop a folder.

-

Runs entirely on this machine. Nothing is uploaded anywhere.

+

Your files are uploaded to this server, analysed, and deleted + when you sign out or go idle. Nobody else using this dashboard can see them.

@@ -136,10 +141,10 @@

- In budget - Out of budget - Paused - Not yet created + In budget + Out of budget + Paused + Not yet created Timeline runs midnight to midnight, left to right.

@@ -198,6 +203,7 @@ - + + diff --git a/web/login.html b/web/login.html new file mode 100644 index 0000000..032c5f7 --- /dev/null +++ b/web/login.html @@ -0,0 +1,53 @@ + + + + + +Sign in · Out-of-Budget Dashboard + + + + + +
+
+ +
+

Out-of-Budget Dashboard

+

Amazon Ads change history → which campaigns keep going dark

+
+
+
+ +
+
+

Sign in

+

Use the address you signed up with.

+ +
+ + + +
+ + + + + +

+ Forgot your password?  ·  + Create an account +

+
+
+ + + + + diff --git a/web/page-admin.js b/web/page-admin.js new file mode 100644 index 0000000..c380571 --- /dev/null +++ b/web/page-admin.js @@ -0,0 +1,80 @@ +'use strict'; + +const $ = (id) => document.getElementById(id); +let me = null; + +const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, + (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c])); + +function when(iso) { + if (!iso) return 'never'; + // Timestamps are stored as naive UTC, so say so before parsing. + const d = new Date(/[Zz+]|\d-\d\d:\d\d$/.test(iso) ? iso : iso + 'Z'); + return esc(d.toLocaleString()); +} + +function statusPill(u) { + if (!u.is_active) return 'disabled'; + if (!u.verified) return 'unconfirmed'; + return 'active'; +} + +function actions(u) { + const out = []; + if (!u.verified) out.push(``); + if (u.signed_in) out.push(``); + if (u.is_active) { + // Disabling yourself would lock you out of this page immediately. + if (!me || u.id !== me.id) { + out.push(``); + } + } else { + out.push(``); + } + return out.join(''); +} + +async function load() { + const res = await A.api('/api/admin/users'); + if (res.status === 403) { location.replace('/'); return; } + const data = await res.json(); + $('summary').textContent = + `${data.users.length} account${data.users.length === 1 ? '' : 's'}, ` + + `${data.sessions} live session${data.sessions === 1 ? '' : 's'}, ` + + `${data.workspaces} workspace${data.workspaces === 1 ? '' : 's'} in memory`; + + $('rows').innerHTML = data.users.map((u) => ` + + ${esc(u.email)} + ${esc(u.name) || '—'} + ${statusPill(u)} + ${u.is_admin ? 'admin' : 'member'} + ${when(u.last_login_at)} +
${actions(u)}
+ `).join(''); +} + +$('rows').addEventListener('click', async (ev) => { + const btn = ev.target.closest('button[data-act]'); + if (!btn) return; + A.busy(btn, true, '…'); + try { + const r = await A.postJSON(`/api/admin/users/${btn.dataset.id}/${btn.dataset.act}`, {}); + A.msg($('note'), r.message, 'ok'); + await load(); + } catch (err) { + A.msg($('note'), String(err.message || err), 'err'); + A.busy(btn, false); + } +}); + +$('signout').addEventListener('click', async () => { + await fetch('/api/auth/logout', { method: 'POST' }); + location.replace('/login'); +}); + +(async () => { + const res = await A.api('/api/auth/me'); + me = (await res.json()).user; + await load(); +})().catch((err) => A.msg($('note'), String(err.message || err), 'err')); diff --git a/web/page-forgot.js b/web/page-forgot.js new file mode 100644 index 0000000..8b6cb30 --- /dev/null +++ b/web/page-forgot.js @@ -0,0 +1,11 @@ +'use strict'; + +const $ = (id) => document.getElementById(id); + +A.wire($('form'), $('submit'), $('note'), async () => { + const r = await A.postJSON('/api/auth/forgot', { email: $('email').value.trim() }); + // The server answers the same way for an address it has never seen, and so + // does this page: nothing here reveals who has an account. + A.msg($('note'), r.message, 'ok'); + $('form').hidden = true; +}); diff --git a/web/page-login.js b/web/page-login.js new file mode 100644 index 0000000..624e5e9 --- /dev/null +++ b/web/page-login.js @@ -0,0 +1,39 @@ +'use strict'; + +// In its own file, not inline: the Content-Security-Policy is script-src +// 'self', so an inline block would be blocked and the page would do nothing. + +const $ = (id) => document.getElementById(id); +const next = A.safeNext(A.qs('next')); + +A.wire($('form'), $('submit'), $('note'), async () => { + const email = $('email').value.trim(); + try { + await A.postJSON('/api/auth/login', { email, password: $('password').value }); + } catch (err) { + // The one specific answer login gives: right password, unconfirmed address. + if (err.code === 'email_not_verified') { + $('resend-wrap').hidden = false; + $('resend').dataset.email = email; + } + throw err; + } + location.replace(next); +}); + +$('resend').addEventListener('click', async () => { + A.busy($('resend'), true, 'Sending…'); + try { + const r = await A.postJSON('/api/auth/resend-verification', + { email: $('resend').dataset.email }); + A.msg($('note'), r.message, 'ok'); + $('resend-wrap').hidden = true; + } catch (err) { + A.msg($('note'), String(err.message || err), 'err'); + } finally { + A.busy($('resend'), false); + } +}); + +if (A.qs('verified')) A.msg($('note'), 'Your email is confirmed. Sign in below.', 'ok'); +if (A.qs('reset')) A.msg($('note'), 'Your password has been changed. Sign in with it now.', 'ok'); diff --git a/web/page-reset.js b/web/page-reset.js new file mode 100644 index 0000000..c84c641 --- /dev/null +++ b/web/page-reset.js @@ -0,0 +1,22 @@ +'use strict'; + +const $ = (id) => document.getElementById(id); + +// The token is read here rather than acted on by the server during the GET: +// mail scanners fetch every link in an inbound message, and a link that acted +// on GET would be spent before anyone clicked it. +const token = A.qs('token'); + +if (!token) { + $('form-panel').hidden = true; + $('bad-panel').hidden = false; + $('bad-text').textContent = 'That link is missing its token. Ask for a new one.'; +} + +A.wire($('form'), $('submit'), $('note'), async () => { + if ($('password').value !== $('confirm').value) { + throw new Error('Those two passwords are not the same.'); + } + await A.postJSON('/api/auth/reset', { token, password: $('password').value }); + location.replace('/login?reset=1'); +}); diff --git a/web/page-signup.js b/web/page-signup.js new file mode 100644 index 0000000..ecff29e --- /dev/null +++ b/web/page-signup.js @@ -0,0 +1,19 @@ +'use strict'; + +const $ = (id) => document.getElementById(id); + +A.wire($('form'), $('submit'), $('note'), async () => { + if ($('password').value !== $('confirm').value) { + throw new Error('Those two passwords are not the same.'); + } + const r = await A.postJSON('/api/auth/signup', { + email: $('email').value.trim(), + password: $('password').value, + name: $('name').value.trim(), + }); + // Deliberately the same screen whether or not the address was already + // registered -- the server answers identically, so the page must too. + $('done-text').textContent = r.message; + $('form-panel').hidden = true; + $('done-panel').hidden = false; +}); diff --git a/web/page-verify.js b/web/page-verify.js new file mode 100644 index 0000000..ea25452 --- /dev/null +++ b/web/page-verify.js @@ -0,0 +1,40 @@ +'use strict'; + +const $ = (id) => document.getElementById(id); +const token = A.qs('token'); + +function fail(text) { + $('busy-panel').hidden = true; + $('ok-panel').hidden = true; + $('bad-panel').hidden = false; + $('bad-text').textContent = text; +} + +async function confirmEmail() { + try { + const r = await A.postJSON('/api/auth/verify', { token }); + $('busy-panel').hidden = true; + $('ok-text').textContent = r.message; + $('ok-panel').hidden = false; + } catch (err) { + fail(String(err.message || err)); + } +} + +// Confirming happens here, on a POST from script, rather than on the GET that +// loaded this page. A mail scanner following the link does not run this, so the +// token survives until a person actually opens it. +if (!token) { + fail('That link is missing its token.'); +} else { + $('manual').hidden = false; + $('manual').addEventListener('click', confirmEmail); + confirmEmail(); +} + +A.wire($('resend-form'), $('resend'), $('note'), async () => { + const r = await A.postJSON('/api/auth/resend-verification', + { email: $('email').value.trim() }); + A.msg($('note'), r.message, 'ok'); + $('resend-form').hidden = true; +}); diff --git a/web/reset.html b/web/reset.html new file mode 100644 index 0000000..0b73c74 --- /dev/null +++ b/web/reset.html @@ -0,0 +1,52 @@ + + + + + +Choose a new password · Out-of-Budget Dashboard + + + + + +
+
+ +
+

Out-of-Budget Dashboard

+

Amazon Ads change history → which campaigns keep going dark

+
+
+
+ +
+
+

Choose a new password

+

Every browser currently signed in to this account will be signed out.

+ +
+ + + +
+ + +

Back to sign in

+
+ + +
+ + + + + diff --git a/web/signup.html b/web/signup.html new file mode 100644 index 0000000..04082e2 --- /dev/null +++ b/web/signup.html @@ -0,0 +1,59 @@ + + + + + +Create an account · Out-of-Budget Dashboard + + + + + +
+
+ +
+

Out-of-Budget Dashboard

+

Amazon Ads change history → which campaigns keep going dark

+
+
+
+ +
+
+

Create an account

+

You will need to confirm your email address before you can sign in.

+ +
+ + + + + +
+ + + +

Already have an account? Sign in

+
+ + +
+ + + + + diff --git a/web/styles.css b/web/styles.css index 6710c0c..4470061 100644 --- a/web/styles.css +++ b/web/styles.css @@ -1,39 +1,86 @@ +/* ============================================================================ + Utopia Brands identity. + + Brand swatches are held verbatim in the --u-* tokens and never altered; + everything else is derived from them. Two colours are NOT in the brand guide: + --red and --amber. The guide covers identity, not function, and it has no + warning colour -- but this dashboard exists to show campaigns going dark, so + "out of budget" has to read as a problem. They are tuned to sit against the + brand greens and are used only to encode state, never as decoration. + ========================================================================== */ + +/* Belleza carries the headings, per the guide's typeface hierarchy. Drop + Belleza-Regular.woff2 into web/fonts/ and it takes over; until then the + fallbacks below are the nearest humanist faces already on most machines. + Fonts must be self-hosted -- the Content-Security-Policy blocks every + external host, so a Google Fonts would silently fail. */ +@font-face { + font-family: 'Belleza'; + src: url('/static/fonts/Belleza-Regular.woff2') format('woff2'); + font-weight: 400; + font-style: normal; + font-display: swap; +} + +/* Body copy is Neue Montreal in the guide. It is a commercial licence, so the + stack falls through to the closest grotesques until the files are added: + @font-face { font-family:'Neue Montreal'; + src:url('/static/fonts/NeueMontreal-Regular.woff2') format('woff2'); + font-weight:400; font-display:swap; } + Add that block plus the Medium/Bold cuts and the whole app picks it up. */ + :root { - --bg: #f6f7f9; + /* ---- brand, exactly as published ---- */ + --u-deep: #004d43; /* primary deep green */ + --u-lime: #ceff71; /* primary lime */ + --u-ink: #1a3134; /* primary near-black teal */ + --u-mist: #eafff4; /* primary pale mint */ + --u-mint: #25e9a5; /* secondary mint */ + --u-peri: #8e92ff; /* secondary periwinkle */ + + --font-display: 'Belleza', Optima, Candara, 'Gill Sans MT', 'Trebuchet MS', sans-serif; + --font-body: 'Neue Montreal', -apple-system, BlinkMacSystemFont, 'Segoe UI', Inter, Roboto, sans-serif; + + /* ---- light mode ---- */ + --bg: var(--u-mist); --panel: #ffffff; - --ink: #16202e; - --ink-2: #55637a; - --ink-3: #8b97ab; - --line: #e3e7ee; - --line-2: #eef1f6; - --navy: #1f3864; - --accent: #2563eb; - --green: #16a34a; - --amber: #b45309; - --red: #dc2626; - --red-soft: #fee2e2; - --amber-soft: #fef3c7; - --green-soft: #dcfce7; - --shadow: 0 1px 2px rgba(16,32,46,.06), 0 8px 24px rgba(16,32,46,.06); + --ink: var(--u-ink); + --ink-2: #4a6b64; + --ink-3: #7c9a92; + --line: #cfe8dc; + --line-2: #e4f5ec; + --brand: #004d43; + --accent: #5a5fd6; /* periwinkle, darkened to hold up as link text */ + --green: #0f7a5c; + --amber: #a2600a; + --red: #c9401f; + --red-soft: #ffe4dc; + --amber-soft: #fdefd0; + --green-soft: #d3f6e8; + --shadow: 0 1px 2px rgba(0,77,67,.07), 0 8px 24px rgba(0,77,67,.07); --radius: 12px; --row-h: 34px; } @media (prefers-color-scheme: dark) { :root { - --bg: #0f1419; - --panel: #171d26; - --ink: #e6ebf2; - --ink-2: #9aa7ba; - --ink-3: #6b7789; - --line: #263040; - --line-2: #1e2734; - --navy: #7aa2e8; - --accent: #60a5fa; - --red-soft: #3b1d1d; - --amber-soft: #3a2e12; - --green-soft: #12301f; - --shadow: 0 1px 2px rgba(0,0,0,.4), 0 8px 24px rgba(0,0,0,.3); + --bg: #14282a; + --panel: var(--u-ink); + --ink: var(--u-mist); + --ink-2: #a4c6bc; + --ink-3: #789c93; + --line: #2c4a4a; + --line-2: #21393a; + /* Lime with dark text is the brand's own call-to-action treatment. */ + --brand: var(--u-lime); + --accent: var(--u-peri); + --green: var(--u-mint); + --amber: #f0b429; + --red: #ff7a5c; + --red-soft: #40201a; + --amber-soft: #3d2f10; + --green-soft: #0f3a2c; + --shadow: 0 1px 2px rgba(0,0,0,.45), 0 8px 24px rgba(0,0,0,.35); } } @@ -43,13 +90,17 @@ body { margin: 0; background: var(--bg); color: var(--ink); - font: 14px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", Inter, Roboto, sans-serif; + font: 14px/1.5 var(--font-body); -webkit-font-smoothing: antialiased; } +/* Headings in Belleza, body and sub-headings in the body face -- the hierarchy + set out in the brand guide. Belleza runs small and light for its size, so it + is stepped up slightly and given back the letter-spacing a display face wants. */ h1, h2, h3 { margin: 0; font-weight: 650; letter-spacing: -.01em; } -h1 { font-size: 17px; } -h2 { font-size: 15px; } +h1, h2 { font-family: var(--font-display); font-weight: 400; letter-spacing: 0; } +h1 { font-size: 19px; } +h2 { font-size: 17px; } h3 { font-size: 13px; } p { margin: 0; } .sub { color: var(--ink-2); font-size: 12.5px; max-width: 78ch; } @@ -67,9 +118,15 @@ p { margin: 0; } border-bottom: 1px solid var(--line); } .brand { display: flex; align-items: center; gap: 12px; } +/* Placeholder for the Utopia emblem: the brand's two primaries, split on the + same diagonal the guide's livery uses. Swap for the real mark when the SVG + lands -- redrawing a registered emblem by hand is one of the guide's don'ts. */ .mark { width: 30px; height: 30px; border-radius: 8px; flex: none; - background: linear-gradient(135deg, var(--green) 0%, var(--green) 32%, var(--red) 32%, var(--red) 100%); + background: linear-gradient(135deg, var(--u-deep) 0 32%, var(--u-lime) 32% 100%); +} +@media (prefers-color-scheme: dark) { + .mark { background: linear-gradient(135deg, var(--u-mint) 0 32%, var(--u-lime) 32% 100%); } } .brand p { font-size: 12px; color: var(--ink-2); } .topbar-actions { display: flex; gap: 8px; } @@ -84,8 +141,8 @@ button { button:hover { border-color: var(--ink-3); } button:active { transform: translateY(1px); } button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } -.primary { background: var(--navy); border-color: var(--navy); color: #fff; font-weight: 600; } -@media (prefers-color-scheme: dark) { .primary { color: #0f1419; } } +.primary { background: var(--brand); border-color: var(--brand); color: #fff; font-weight: 600; } +@media (prefers-color-scheme: dark) { .primary { color: var(--u-ink); } } .primary:hover { filter: brightness(1.08); } .ghost { background: transparent; } .danger:hover { border-color: var(--red); color: var(--red); } @@ -133,7 +190,7 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } .loading { text-align: center; padding: 90px 20px; color: var(--ink-2); } .spinner { width: 30px; height: 30px; margin: 0 auto 16px; border-radius: 50%; - border: 3px solid var(--line); border-top-color: var(--navy); + border: 3px solid var(--line); border-top-color: var(--brand); animation: spin .8s linear infinite; } @keyframes spin { to { transform: rotate(360deg); } } @@ -163,6 +220,8 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } font-size: 12px; font-weight: 650; color: #fff; white-space: nowrap; overflow: hidden; text-shadow: 0 1px 2px rgba(0,0,0,.3); } +/* Paused and not-yet-created are pale by design, so white would vanish. */ +.daybar span.on-pale { color: var(--u-ink); text-shadow: none; } .daykeys { display: flex; flex-wrap: wrap; gap: 20px; margin-top: 11px; font-size: 12.5px; } .daykeys div { display: flex; align-items: center; gap: 7px; color: var(--ink-2); } .daykeys b { color: var(--ink); font-variant-numeric: tabular-nums; } @@ -186,7 +245,7 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } } .kpi .value { font-size: 27px; font-weight: 680; letter-spacing: -.02em; - margin: 5px 0 3px; font-variant-numeric: tabular-nums; color: var(--navy); + margin: 5px 0 3px; font-variant-numeric: tabular-nums; color: var(--brand); } .kpi .note { font-size: 11.5px; color: var(--ink-3); } .kpi.alarm .value { color: var(--red); } @@ -215,8 +274,8 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } background: transparent; color: var(--ink-2); } .segmented button + button { border-left: 1px solid var(--line); } -.segmented button[aria-pressed="true"] { background: var(--navy); color: #fff; font-weight: 600; } -@media (prefers-color-scheme: dark) { .segmented button[aria-pressed="true"] { color: #0f1419; } } +.segmented button[aria-pressed="true"] { background: var(--brand); color: #fff; font-weight: 600; } +@media (prefers-color-scheme: dark) { .segmented button[aria-pressed="true"] { color: var(--u-ink); } } /* One cell per day: which days were bad, at a glance. */ .dayheat { display: flex; gap: 2px; height: 15px; margin: 0 10px; } @@ -230,7 +289,10 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } align-items: center; height: 100%; gap: 5px; } .curve .fill { width: 100%; border-radius: 4px 4px 0 0; min-height: 2px; - background: linear-gradient(180deg, var(--red), #ef4444); + /* The same coral the timeline strips use for out-of-budget: this chart + measures the identical thing by hour, so it must not read as a new colour. + Two adjacent steps of the warm ramp, not a darkened tint. */ + background: linear-gradient(180deg, #f2542d, #d8431f); transition: filter .12s; } .curve .bar:hover .fill { filter: brightness(1.2); } @@ -251,8 +313,8 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } border: 1px solid var(--line); border-radius: 999px; padding: 4px 12px; font-size: 12px; background: var(--panel); color: var(--ink-2); } -.chip[aria-pressed="true"] { background: var(--navy); border-color: var(--navy); color: #fff; } -@media (prefers-color-scheme: dark) { .chip[aria-pressed="true"] { color: #0f1419; } } +.chip[aria-pressed="true"] { background: var(--brand); border-color: var(--brand); color: #fff; } +@media (prefers-color-scheme: dark) { .chip[aria-pressed="true"] { color: var(--u-ink); } } .chip .n { opacity: .65; margin-left: 5px; font-variant-numeric: tabular-nums; } /* ------------------------------------------------------------------- table */ @@ -331,16 +393,18 @@ button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } font-size: 10.5px; font-weight: 600; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; max-width: 100%; } +/* Diagnoses run from worst to benign. The label text is always present, so the + tint reinforces severity rather than carrying it alone. */ .dx-under { background: var(--red-soft); color: var(--red); } .dx-early { background: var(--amber-soft); color: var(--amber); } -.dx-thrash { background: #ede9fe; color: #6d28d9; } +.dx-thrash { background: #e8e9ff; color: #4b50c8; } /* periwinkle */ .dx-evening { background: var(--amber-soft); color: var(--amber); } -.dx-inter { background: #dbeafe; color: #1d4ed8; } +.dx-inter { background: #f1ffd7; color: #5d7d1f; } /* lime */ .dx-healthy { background: var(--green-soft); color: var(--green); } .dx-paused { background: var(--line-2); color: var(--ink-2); } @media (prefers-color-scheme: dark) { - .dx-thrash { background: #2e1f4d; color: #c4b5fd; } - .dx-inter { background: #17294d; color: #93c5fd; } + .dx-thrash { background: #272b55; color: #babdff; } + .dx-inter { background: #303f18; color: #ceff71; } } .unpriced { color: var(--ink-3); font-style: italic; font-size: 11.5px; } @@ -421,19 +485,21 @@ table.grid th small { display: block; font-size: 9px; font-weight: 500; opacity: table.act-log td { font-size: 11.5px; vertical-align: top; } table.act-log td:last-child { color: var(--ink-2); word-break: break-word; } -.pill.act-budget { background: #dbeafe; color: #1d4ed8; } -.pill.act-placement { background: #ede9fe; color: #6d28d9; } -.pill.act-strategy { background: #fce7f3; color: #be185d; } +/* Eight action categories, more than the palette has distinct hues. Each pill + is labelled, so colour is reinforcement and the two quietest categories can + share the neutral tint without ambiguity. */ +.pill.act-budget { background: #d5f0e7; color: #045f50; } /* deep green */ +.pill.act-placement { background: #e8e9ff; color: #4b50c8; } /* periwinkle */ +.pill.act-strategy { background: var(--red-soft); color: var(--red); } .pill.act-bid { background: var(--green-soft); color: var(--green); } .pill.act-targeting { background: var(--amber-soft); color: var(--amber); } .pill.act-status { background: var(--line-2); color: var(--ink-2); } -.pill.act-structure { background: #e0f2fe; color: #0369a1; } +.pill.act-structure { background: #f1ffd7; color: #5d7d1f; } /* lime */ .pill.act-portfolio { background: var(--line-2); color: var(--ink-2); } @media (prefers-color-scheme: dark) { - .pill.act-budget { background: #17294d; color: #93c5fd; } - .pill.act-placement { background: #2e1f4d; color: #c4b5fd; } - .pill.act-strategy { background: #401027; color: #f9a8d4; } - .pill.act-structure { background: #0c2b3d; color: #7dd3fc; } + .pill.act-budget { background: #0d3f37; color: #6fe0bf; } + .pill.act-placement { background: #272b55; color: #babdff; } + .pill.act-structure { background: #303f18; color: #ceff71; } } /* ---------------------------------------------------------------- settings */ @@ -443,14 +509,44 @@ dialog { background: var(--panel); color: var(--ink); max-width: 440px; box-shadow: var(--shadow); } dialog::backdrop { background: rgba(16,32,46,.4); } -dialog label { display: block; margin: 16px 0; font-size: 12.5px; font-weight: 600; } -dialog input { +dialog label, .auth label { display: block; margin: 16px 0; font-size: 12.5px; font-weight: 600; } +dialog input, .auth input { display: block; width: 100%; margin-top: 5px; font: inherit; padding: 7px 10px; border: 1px solid var(--line); border-radius: 8px; background: var(--bg); color: var(--ink); } -dialog small { display: block; margin-top: 4px; font-weight: 400; color: var(--ink-3); } +dialog small, .auth small { display: block; margin-top: 4px; font-weight: 400; color: var(--ink-3); } dialog menu { display: flex; justify-content: flex-end; gap: 8px; padding: 0; margin: 20px 0 0; } +/* -------------------------------------------------------------- auth pages */ + +.auth { max-width: 400px; margin: 56px auto; padding: 0 20px; } +.auth .panel { padding: 24px; } +.auth h2 { font-size: 17px; margin-bottom: 6px; } +.auth .primary { width: 100%; margin-top: 6px; } +.auth .alt { margin-top: 18px; font-size: 12.5px; color: var(--ink-2); text-align: center; } +.auth .alt a { color: var(--accent); } +.auth .msg { border-radius: 9px; padding: 10px 12px; font-size: 12.5px; margin-top: 14px; } +.auth .msg.err { background: var(--red-soft); color: var(--red); } +.auth .msg.ok { background: var(--green-soft); color: var(--green); } +.auth .centre { text-align: center; padding: 8px 0 4px; } +.auth .centre .spinner { margin: 0 auto 14px; } + +/* ------------------------------------------------------------------- admin */ + +.admin { max-width: 1080px; margin: 28px auto; padding: 0 24px; } +.utable { width: 100%; border-collapse: collapse; font-size: 12.5px; } +.utable th { + text-align: left; font-size: 10.5px; text-transform: uppercase; letter-spacing: .05em; + color: var(--ink-2); font-weight: 600; padding: 8px 10px; border-bottom: 1px solid var(--line); +} +.utable td { padding: 9px 10px; border-bottom: 1px solid var(--line-2); vertical-align: middle; } +.utable tr:last-child td { border-bottom: 0; } +.utable .acts { display: flex; gap: 6px; justify-content: flex-end; } +.utable button { padding: 4px 9px; font-size: 11.5px; } +.tag-on { background: var(--green-soft); color: var(--green); } +.tag-off { background: var(--red-soft); color: var(--red); } +.tag-wait { background: var(--amber-soft); color: var(--amber); } + @media (max-width: 1100px) { .thead, .row { grid-template-columns: minmax(160px, 2fr) 66px 60px 56px 150px 160px; } .thead > div:nth-child(n+7), .row > div:nth-child(n+7) { display: none; } diff --git a/web/verify.html b/web/verify.html new file mode 100644 index 0000000..6b07a13 --- /dev/null +++ b/web/verify.html @@ -0,0 +1,53 @@ + + + + + +Confirming your email · Out-of-Budget Dashboard + + + + + +
+
+ +
+

Out-of-Budget Dashboard

+

Amazon Ads change history → which campaigns keep going dark

+
+
+
+ +
+
+ +

Confirming your email address…

+ +

+
+ + + + +
+ + + + +