Initial commit: PPC out-of-budget analyzer and dashboard.

Co-authored-by: Cursor <cursoragent@cursor.com>
feat/hosted-auth-mysql-docker
sheheryarsoomro12 2026-08-07 22:32:35 +05:00
commit 54d41b12a4
19 changed files with 4920 additions and 0 deletions

51
.gitignore vendored Normal file
View File

@ -0,0 +1,51 @@
# Byte-compiled / cache
__pycache__/
*.py[cod]
*$py.class
*.pyo
# Virtual environments
.venv/
venv/
env/
.env/
*.egg-info/
.eggs/
dist/
build/
# Environment / secrets
.env
.env.*
!.env.example
# Test / coverage
.pytest_cache/
.coverage
htmlcov/
.mypy_cache/
.ruff_cache/
# OS
.DS_Store
Thumbs.db
# Editor
.idea/
.vscode/
*.swp
*~
# Excel lock / temp files
~$*.xlsx
~$*.xls
# Local inputs and generated outputs
data/*
!data/.gitkeep
reports/*
!reports/.gitkeep
*.xlsx
*.xls
*.csv
!tests/**/*.csv

174
README.md Normal file
View File

@ -0,0 +1,174 @@
# PPC Out-of-Budget Analyzer
Finds the campaigns that keep running out of budget, how long they were dark,
how often it happened, and what it plausibly cost.
Two ways to use it. Both run the same analysis code, so they can never
disagree — the dashboard just makes it explorable and the report makes it
shareable.
## The dashboard
```bash
python3 serve.py
```
Opens <http://localhost:8765>. Drag your change-history exports onto the page.
Sort and filter by any column, click a campaign for its full timeline and
outage list, and download the Excel or CSV version from the header.
Add `--preload` to pick up whatever is already sitting in `data/` on startup.
The server binds to localhost only; nothing is uploaded anywhere.
## The Excel report
```bash
python3 run_report.py
```
Put your exports in `data/` first. The report lands in `reports/`. Nothing to
install — it uses `openpyxl`, which you already have.
## Loading more than one day
Drop in as many exports as you like, at once or over time. A single export can
already cover a date range, and overlapping files are fine — rows appearing in
more than one export are matched on entity, timestamp and values and counted
once, which the Data Quality panel reports. Without that, an overlap doesn't
corrupt the totals but it does bury the handful of genuine timeline
contradictions under thousands of false ones.
With more than one day loaded the dashboard switches to **one row per
campaign**, averaged across the days, with a strip showing one cell per day so
you can see which days were bad. Click any campaign for its day-by-day
breakdown: hours run, hours lost, outages and a 24-hour timeline for each
individual day. Use the **One row per day** toggle to go back to the raw grain.
Don't mix marketplaces in one load — the account ROAS and spend behind the
reality check come from the first export's metadata.
## Getting dollar figures for every campaign
The change history records *changes*, so it only reveals a daily budget for
campaigns whose budget someone edited that day — about 9% of them. Everything
else gets exact timings but no dollar figure, and the report leaves those cells
empty rather than guessing.
To price all of them, export a campaign performance report from the same Amazon
Ads console (any report with Campaign, Spend, Sales and Budget columns). In the
dashboard use the "Add report" slot; from the command line:
```bash
python3 run_report.py --perf ~/Downloads/campaign-report.xlsx
```
Column names are matched loosely, so most Amazon report variants work as-is.
The Data Quality sheet reports how many campaigns matched, in both directions.
## Options
`run_report.py`:
| Flag | Does |
|---|---|
| `--perf FILE` | Join a performance report for full budget/ROAS coverage |
| `--out FILE` | Write somewhere other than `reports/` |
| `--roas N` | Override ROAS (defaults to the account average in the export) |
| `--haircut N` | Discount on ROAS for incremental spend (default `0.7`) |
| `--cap N` | Cap lost spend at N × daily budget (default `3`) |
| `--merge-gap N` | Minutes in budget below which two outages count as one (default `5`) |
`serve.py`: `--port N` (default 8765), `--preload`, `--no-browser`. The same
four modelling assumptions are editable live under **Assumptions** in the
dashboard header.
You can also pass files or folders directly:
```bash
python3 run_report.py ~/Downloads/august-exports/
```
## What the sheets show
**Summary** — a typical campaign's day (runs / lost / paused), the account-wide
per-day figures, a reality check of modelled loss against actual spend, and the
hour-by-hour starvation curve. That curve is usually the most useful thing in
the file: it shows what share of the account is dark at each hour.
**Campaigns** — one row per campaign. With several days loaded it is averaged
across them, with a column per day on the right shaded green through red so you
can see which days broke. With a single day it carries the 24-hour heatmap
instead.
**Daily Detail** — multi-day only. One row per campaign per day, with 24 narrow
columns showing how many minutes of each hour the campaign was out of budget.
Grey means paused, pale grey means the campaign did not exist yet.
**Episodes** — every individual outage with start and end times. Duration is
wall-clock; Billable excludes minutes the campaign was paused during it.
**Data Quality** — every check that could change how much you trust the rest.
Never hidden, never dismissible.
**Method** — how each number is calculated, in plain English.
Every sheet is a native Excel table, so the filter buttons and banding are
already there. Durations are stored as real time values displayed as
"23h 35min", so they still sum, sort and chart correctly.
## Last meaningful action
Every campaign carries a **Last action** column: the most recent optimisation
change inside the days the export covers — budget, bid, placement %, bidding
strategy, targeting, enable/pause, or structural change. A campaign nobody has
touched across the whole window reads **"No action in 14 days"** in red, and
there is a *No action taken* filter and a headline count so you can pull the
whole neglected set in one click.
The subtlety that makes this useful: Amazon's own pacing engine writes an
In-budget/Out-of-budget row every time a campaign hits its cap — 2,639 of the
2,989 `Campaign status` rows in the reference file. Those are **not** counted as
actions. If they were, every starving campaign would look actively managed,
which is precisely backwards. Only the delivery half of that change type
(Delivering/Paused) is a person. Renames are excluded too, for the same reason.
The window is always the span the data actually covers, never what you asked
Amazon for. One day of export can only ever say "no action in 1 day".
## The diagnoses
| Label | Means |
|---|---|
| Structurally underfunded | Out of budget over half the day, and it started before noon |
| Exhausts early | Burned through the budget before 9am |
| Pacing thrash | Five or more separate outages without huge total loss — Amazon is releasing budget in slivers |
| Evening cap | Only ran out after 6pm |
| Intermittent | Out of budget, but no clear pattern |
| Healthy | Out of budget under 5% of the day |
| Mostly paused | Paused over half the day, so it forgoes nothing to budget and is excluded from loss |
## Two things worth knowing
**Empty is not zero.** Where no budget was observed, money cells are blank. A
zero would become a fact the moment someone summed the column.
**Paused time is excluded.** A paused campaign isn't losing anything to its
budget, so paused minutes are removed from both the loss total and the
in-budget denominator that sets the spend rate.
## Tests
```bash
python3 tests/test_golden.py
```
34 checks: frozen totals from the reference export, structural invariants,
overlapping-export handling, action classification, and edge cases. The
important ones are `test_chain_breaks_canary` and
`test_amazon_pacing_rows_are_not_actions`.
The export is written newest-first, so rows sharing the same minute are also
newest-first and must be reversed before the state machine walks them. Sorting
on timestamp alone silently preserves the wrong order — it produces 19 chain
breaks instead of 5, and reads one real 13-hour outage as a harmless 47-minute
blip. That canary catches the regression.

0
data/.gitkeep Normal file
View File

8
ppcbudget/__init__.py Normal file
View File

@ -0,0 +1,8 @@
"""Amazon Ads out-of-budget analyzer.
Turns one or more `amazon-ads-history_*.xlsx` change-history exports into a
formatted Excel report showing which campaigns run out of budget, for how
long, how often, and what it plausibly costs.
"""
__version__ = "1.0.0"

194
ppcbudget/actions.py Normal file
View File

@ -0,0 +1,194 @@
"""Track when a human last actually touched each campaign.
The point is to separate "this campaign is starving" from "this campaign is
starving and nobody has looked at it in nine days". The second is the one worth
opening first.
The hard part is that most rows in the export are *not* actions. Amazon's own
pacing engine writes an In-budget/Out-of-budget row every time a campaign hits
its cap -- 2,639 of 2,989 `Campaign status` rows in the reference file. Counting
those would make every starving campaign look actively managed, which is exactly
backwards. Only the delivery half of that change type (Delivering/Paused) is a
person, and it is told apart by vocabulary, the same split the budget scoring
uses.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from datetime import date
from .ingest import BUDGET_STATES, DELIVERY_STATES, Event
# Checked in order; the first match wins, so specific beats generic.
# Each rule is (category, human label, substrings matched against a lowered
# change type). Change types carry variable tails -- keyword text, product
# titles -- so these are substring rules, never equality.
RULES: list[tuple[str, str, tuple[str, ...]]] = [
("budget", "Budget", ("campaign daily budget", "budget rule")),
("placement", "Placement", ("bid adjustment for",)),
("strategy", "Strategy", ("campaign bidding strategy",)),
("bid", "Bid", ("bid",)),
("targeting", "Targeting", ("keyword", "target", "negative")),
("structure", "Structure", ("created", "added to ad group",
"removed from ad group")),
("status", "Status", ("status",)),
("portfolio", "Portfolio", ("portfolio",)),
]
# Real changes, but not optimisation. Kept out of "last action" so a rename
# does not make a neglected campaign look tended.
COSMETIC = ("name changed", "ad group name", "campaign name")
RECENT_LIMIT = 12 # what the detail panel shows
CATEGORY_ORDER = [r[0] for r in RULES]
CATEGORY_LABEL = {r[0]: r[1] for r in RULES}
@dataclass(slots=True)
class CampaignActions:
campaign: str
window_days: int
window_start: str
window_end: str
last_at: str | None = None # 'YYYY-MM-DD HH:MM'
last_date: str | None = None
last_category: str | None = None
last_label: str | None = None # the raw change type, trimmed
days_since: int | None = None # measured from the last day in the window
count: int = 0
categories: list[str] = field(default_factory=list)
cosmetic_only: bool = False # touched, but only renames
# Most recent first, capped -- enough for the detail panel to show what was
# actually done without shipping every row of history to the browser.
recent: list[tuple[str, str, str]] = field(default_factory=list)
@property
def untouched(self) -> bool:
return self.last_at is None
@property
def summary(self) -> str:
"""One phrase for a report cell. Never implies a longer window than observed."""
span = f"{self.window_days} day{'' if self.window_days == 1 else 's'}"
if self.untouched:
extra = " (only a rename)" if self.cosmetic_only else ""
return f"No action in {span}{extra}"
label = CATEGORY_LABEL.get(self.last_category, "Change")
if self.days_since == 0:
return f"{label} · last day"
return f"{label} · {self.days_since}d ago"
def classify(event: Event) -> str | None:
"""Category of optimisation action, or None if the row is not one."""
ct = event.change_type.strip()
low = ct.lower()
if ct == "Campaign status":
# Two state machines share this change type. Only the delivery half is
# a person; the budget half is Amazon's pacing engine.
if event.from_val in BUDGET_STATES or event.to_val in BUDGET_STATES:
return None
if event.from_val in DELIVERY_STATES or event.to_val in DELIVERY_STATES:
return "status"
return None
if any(c in low for c in COSMETIC):
return "cosmetic"
for category, _, needles in RULES:
if any(n in low for n in needles):
return category
return None
def _stamp(e: Event) -> str:
return f"{e.date_key} {e.minute // 60:02d}:{e.minute % 60:02d}"
def _trim(change_type: str, limit: int = 60) -> str:
"""Change types embed whole product titles; keep the head."""
s = re.sub(r"\s+", " ", change_type.strip())
return s if len(s) <= limit else s[: limit - 1] + ""
def build(events: list[Event], date_keys: list[str],
campaigns: set[str] | None = None) -> dict[str, CampaignActions]:
"""Last meaningful action per campaign, over the days actually observed.
The window is the span the data covers -- never what was asked for. If the
export holds one day, this reports on one day and says so.
"""
if not date_keys:
return {}
start, end = date_keys[0], date_keys[-1]
window_days = len(date_keys)
end_date = date.fromisoformat(end)
names = set(campaigns) if campaigns is not None else {e.campaign for e in events}
out = {
name: CampaignActions(campaign=name, window_days=window_days,
window_start=start, window_end=end)
for name in names
}
for e in events:
rec = out.get(e.campaign)
if rec is None:
continue
category = classify(e)
if category is None:
continue
if category == "cosmetic":
rec.cosmetic_only = True
continue
rec.count += 1
if category not in rec.categories:
rec.categories.append(category)
stamp = _stamp(e)
rec.recent.append((stamp, category, _trim(e.change_type)))
if rec.last_at is None or stamp > rec.last_at:
rec.last_at = stamp
rec.last_date = e.date_key
rec.last_category = category
rec.last_label = _trim(e.change_type)
for rec in out.values():
if rec.last_date:
rec.days_since = (end_date - date.fromisoformat(rec.last_date)).days
rec.cosmetic_only = False
rec.categories.sort(key=CATEGORY_ORDER.index)
rec.recent.sort(key=lambda r: r[0], reverse=True)
del rec.recent[RECENT_LIMIT:]
return out
def summarize(actions: dict[str, CampaignActions]) -> dict:
"""Account-level counts for the headline."""
total = len(actions)
untouched = [a for a in actions.values() if a.untouched]
buckets = {"0-1": 0, "2-3": 0, "4-7": 0, "8+": 0}
for a in actions.values():
if a.days_since is None:
continue
if a.days_since <= 1:
buckets["0-1"] += 1
elif a.days_since <= 3:
buckets["2-3"] += 1
elif a.days_since <= 7:
buckets["4-7"] += 1
else:
buckets["8+"] += 1
window = next(iter(actions.values())).window_days if actions else 0
return {
"campaigns": total,
"untouched": len(untouched),
"touched": total - len(untouched),
"window_days": window,
"buckets": buckets,
"actions_total": sum(a.count for a in actions.values()),
}

139
ppcbudget/aggregate.py Normal file
View File

@ -0,0 +1,139 @@
"""Roll campaign-days up across multiple exports to find chronic offenders.
A campaign out of budget 8 hours a day for two weeks is a bigger problem than
one that spiked to 23 hours once, so `chronic_score` weights recurrence and
streak length alongside average severity.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from statistics import median
from .scoring import CampaignDay
OOB_DAY_THRESHOLD_MIN = 60 # a day "counts" once an hour is lost
STREAK_CEILING = 7
W_RECURRENCE, W_MEAN, W_STREAK = 0.40, 0.35, 0.25
@dataclass(slots=True)
class DayPoint:
date_key: str
oob_hours: float
in_hours: float
paused_hours: float
episodes: int
severity: float
first_oob_min: int | None
@dataclass(slots=True)
class CampaignRollup:
campaign: str
days_observed: int = 0
days_with_oob: int = 0
total_oob_hours: float = 0.0
mean_oob_hours: float = 0.0
mean_in_hours: float = 0.0 # hours per day the campaign could actually spend
mean_paused_hours: float = 0.0
median_oob_hours: float = 0.0
max_oob_hours: float = 0.0
total_episodes: int = 0
mean_first_oob_min: float | None = None
recurrence_rate: float = 0.0
streak_current: int = 0
streak_max: int = 0
trend_slope: float = 0.0
chronic_score: float = 0.0
mean_severity: float = 0.0
dominant_diagnosis: str = ""
worst_date: str = ""
total_lost_spend: float | None = None
total_lost_sales: float | None = None
per_day: list[DayPoint] = field(default_factory=list)
@property
def trend_label(self) -> str:
if abs(self.trend_slope) < 0.05:
return "flat"
return "worsening" if self.trend_slope > 0 else "improving"
def _ols_slope(values: list[float]) -> float:
"""Least-squares slope of y against its index. Zero for fewer than 2 points."""
n = len(values)
if n < 2:
return 0.0
mean_x = (n - 1) / 2
mean_y = sum(values) / n
denom = sum((i - mean_x) ** 2 for i in range(n))
if denom == 0:
return 0.0
return sum((i - mean_x) * (v - mean_y) for i, v in enumerate(values)) / denom
def rollup(days: list[CampaignDay]) -> list[CampaignRollup]:
by_campaign: dict[str, list[CampaignDay]] = {}
for d in days:
by_campaign.setdefault(d.campaign, []).append(d)
out: list[CampaignRollup] = []
for campaign, entries in by_campaign.items():
entries.sort(key=lambda d: d.date_key)
hours = [d.oob_hours for d in entries]
firsts = [d.first_oob_min for d in entries if d.first_oob_min is not None]
r = CampaignRollup(campaign=campaign, days_observed=len(entries))
r.per_day = [
DayPoint(d.date_key, d.oob_hours, d.in_hours, d.paused_hours,
d.episodes_merged, d.severity, d.first_oob_min)
for d in entries
]
r.days_with_oob = sum(1 for d in entries if d.oob_min >= OOB_DAY_THRESHOLD_MIN)
r.total_oob_hours = sum(hours)
r.mean_oob_hours = r.total_oob_hours / len(entries)
r.mean_in_hours = sum(d.in_hours for d in entries) / len(entries)
r.mean_paused_hours = sum(d.paused_hours for d in entries) / len(entries)
r.median_oob_hours = median(hours)
r.max_oob_hours = max(hours)
r.total_episodes = sum(d.episodes_merged for d in entries)
r.mean_first_oob_min = (sum(firsts) / len(firsts)) if firsts else None
r.recurrence_rate = r.days_with_oob / len(entries)
streak = 0
for d in entries:
if d.oob_min >= OOB_DAY_THRESHOLD_MIN:
streak += 1
r.streak_max = max(r.streak_max, streak)
else:
streak = 0
r.streak_current = streak
r.trend_slope = _ols_slope(hours)
r.chronic_score = 100 * (
W_RECURRENCE * r.recurrence_rate
+ W_MEAN * min(r.mean_oob_hours / 24, 1.0)
+ W_STREAK * min(r.streak_max, STREAK_CEILING) / STREAK_CEILING
)
r.mean_severity = sum(d.severity for d in entries) / len(entries)
r.worst_date = max(entries, key=lambda d: d.oob_min).date_key
# The label the campaign earns most often; ties break toward the worst day.
counts: dict[str, int] = {}
for d in entries:
counts[d.diagnosis] = counts.get(d.diagnosis, 0) + 1
worst = max(entries, key=lambda d: d.oob_min).diagnosis
r.dominant_diagnosis = max(counts, key=lambda k: (counts[k], k == worst))
priced = [d.lost["lost_spend"] for d in entries
if d.lost and d.lost.get("lost_spend") is not None]
r.total_lost_spend = sum(priced) if priced else None
sales = [d.lost["lost_sales"] for d in entries
if d.lost and d.lost.get("lost_sales") is not None]
r.total_lost_sales = sum(sales) if sales else None
out.append(r)
out.sort(key=lambda r: -r.chronic_score)
return out

717
ppcbudget/excelout.py Normal file
View File

@ -0,0 +1,717 @@
"""Render the scored data as a formatted Excel workbook.
The primary sheet is one row per campaign, matching the dashboard. Durations are
stored as real Excel time values -- a fraction of a day -- and displayed with a
`[h]"h" mm"min"` format, so a cell reads "23h 35min" while still summing,
sorting and charting as a number. Decimal hours are unreadable; text like
"23h 35min" cannot be calculated with. This gets both.
"""
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from openpyxl import Workbook
from openpyxl.chart import BarChart, Reference
from openpyxl.formatting.rule import ColorScaleRule
from openpyxl.styles import Alignment, Border, Font, PatternFill, Side
from openpyxl.utils import get_column_letter
from openpyxl.worksheet.table import Table, TableStyleInfo
from .aggregate import CampaignRollup
from .ingest import QaReport, WorkbookMeta
from .metrics import ModelSettings, Totals, hourly_starvation
from .scoring import CampaignDay
# ------------------------------------------------------------------- palette
NAVY = "1F3864"
SLATE = "44546A"
WHITE = "FFFFFF"
GREEN = "16A34A"
AMBER = "B45309"
RED = "C0392B"
GREY = "9E9E9E"
PANEL = "F4F6FA"
HDR_FILL = PatternFill("solid", fgColor=NAVY)
HDR_FONT = Font(color=WHITE, bold=True, size=10)
TITLE_FONT = Font(color=NAVY, bold=True, size=20)
SUB_FONT = Font(color=SLATE, size=10, italic=True)
SECTION = Font(color=NAVY, bold=True, size=12)
TILE_FILL = PatternFill("solid", fgColor=PANEL)
KPI_LABEL = Font(color=SLATE, size=9, bold=True)
KPI_VALUE = Font(color=NAVY, size=18, bold=True)
KPI_ALARM = Font(color=RED, size=18, bold=True)
KPI_NOTE = Font(color=GREY, size=8, italic=True)
RUN_FONT = Font(color=GREEN, size=10)
LOST_FONT = Font(color=RED, size=10, bold=True)
UNPRICED = Font(color=GREY, size=9, italic=True)
# Per-date sub-columns run narrow, so they get their own smaller type.
DAY_RUN_FONT = Font(color=GREEN, size=9)
DAY_LOST_FONT = Font(color="7F1D1D", size=9, bold=True)
DAY_PAUSE_FONT = Font(color=GREY, size=9)
EDGE = Side(style="thin", color="C9D2E3")
BOX = Border(left=EDGE, right=EDGE, top=EDGE, bottom=EDGE)
# Hour-of-day heat, reused so 60k cells share nine fill objects.
HEAT = [PatternFill("solid", fgColor=c) for c in
("E8F5E9", "FFF9C4", "FFECB3", "FFE0B2", "FFCCBC",
"FFAB91", "FF8A65", "EF5350", "C62828")]
PAUSED_FILL = PatternFill("solid", fgColor="E0E0E0")
NA_FILL = PatternFill("solid", fgColor="F5F5F5")
HEAT_FONT = Font(size=7, color="616161")
HEAT_FONT_DARK = Font(size=7, color=WHITE)
DIAGNOSIS_FILL = {
"Structurally underfunded": PatternFill("solid", fgColor="FFCDD2"),
"Exhausts early": PatternFill("solid", fgColor="FFE0B2"),
"Pacing thrash": PatternFill("solid", fgColor="E1BEE7"),
"Evening cap": PatternFill("solid", fgColor="FFF9C4"),
"Intermittent": PatternFill("solid", fgColor="E3F2FD"),
"Healthy": PatternFill("solid", fgColor="C8E6C9"),
"Mostly paused": PatternFill("solid", fgColor="ECEFF1"),
}
# A duration is a fraction of a day; [h] lets a total exceed 24 hours.
FMT_DUR = '[h]"h" mm"min"'
# Same, but a zero reads as a dash -- a column of "0h 00min" is pure noise.
FMT_DUR_Z = '[h]"h" mm"min";;"-"'
FMT_PCT = "0%"
FMT_PCT1 = "0.0%"
FMT_MONEY = '"$"#,##0.00'
FMT_MONEY0 = '"$"#,##0'
FMT_INT = "#,##0"
FMT_1 = "0.0"
MAX_DAY_COLUMNS = 31
def dur(hours: float | None) -> float | None:
"""Hours -> Excel time value. Pairs with FMT_DUR."""
return None if hours is None else hours / 24
def hhmm(minute: int | None) -> str:
if minute is None:
return ""
minute = min(minute, 1439)
return f"{minute // 60:02d}:{minute % 60:02d}"
def _heat_style(day: CampaignDay, hour: int):
if day.hourly_na[hour] >= 30:
return NA_FILL, HEAT_FONT
if day.hourly_paused[hour] >= 30:
return PAUSED_FILL, HEAT_FONT
oob = day.hourly_oob[hour]
if oob == 0:
return HEAT[0], HEAT_FONT
level = min(8, 1 + (oob - 1) * 8 // 60)
return HEAT[level], (HEAT_FONT_DARK if level >= 7 else HEAT_FONT)
def _header(ws, row: int, headers: list[str], widths: list[int] | None = None,
height: int = 44) -> None:
for i, name in enumerate(headers, start=1):
c = ws.cell(row=row, column=i, value=name)
c.fill, c.font = HDR_FILL, HDR_FONT
c.alignment = Alignment(horizontal="center", vertical="center", wrap_text=True)
c.border = BOX
if widths and i <= len(widths):
ws.column_dimensions[get_column_letter(i)].width = widths[i - 1]
ws.row_dimensions[row].height = height
def _as_table(ws, name: str, last_row: int, last_col: int, first_row: int = 1) -> None:
"""Turn a range into a native Excel table: banded rows and filter buttons."""
if last_row <= first_row:
return
ref = f"A{first_row}:{get_column_letter(last_col)}{last_row}"
table = Table(displayName=name, ref=ref)
table.tableStyleInfo = TableStyleInfo(
name="TableStyleMedium2", showRowStripes=True, showColumnStripes=False,
showFirstColumn=False, showLastColumn=False)
ws.add_table(table)
STALE_FILL = PatternFill("solid", fgColor="FFCDD2")
WARM_FILL = PatternFill("solid", fgColor="FFF3CD")
STALE_FONT = Font(color="7F1D1D", size=10, bold=True)
def _action_cells(ws, row: int, col: int, act) -> None:
"""Last meaningful action: summary, age, what changed, how many."""
if act is None:
ws.cell(row=row, column=col, value="not observed").font = UNPRICED
return
summary = ws.cell(row=row, column=col, value=act.summary)
ws.cell(row=row, column=col + 1, value=act.days_since)
ws.cell(row=row, column=col + 2, value=act.last_label or "")
ws.cell(row=row, column=col + 3, value=act.count or None)
# Untouched for the whole window is the thing to spot from across the room.
if act.untouched:
summary.fill, summary.font = STALE_FILL, STALE_FONT
ws.cell(row=row, column=col + 1, value=None)
elif act.days_since is not None and act.days_since >= 3:
summary.fill = WARM_FILL
def _tile(ws, row: int, col: int, label: str, value, note: str,
fmt: str | None = None, alarm: bool = False, width: int = 2) -> None:
"""A KPI card: label, big number, footnote, boxed and filled."""
for r in range(row, row + 3):
for c in range(col, col + width):
cell = ws.cell(row=r, column=c)
cell.fill = TILE_FILL
cell.border = BOX
ws.cell(row=row, column=col, value=label).font = KPI_LABEL
v = ws.cell(row=row + 1, column=col, value=value)
v.font = KPI_ALARM if alarm else KPI_VALUE
v.alignment = Alignment(horizontal="left")
if fmt:
v.number_format = fmt
ws.cell(row=row + 2, column=col, value=note).font = KPI_NOTE
ws.row_dimensions[row + 1].height = 26
# ------------------------------------------------------------------- Summary
def _sheet_summary(wb: Workbook, days: list[CampaignDay], totals: Totals,
metas: list[WorkbookMeta], settings: ModelSettings,
date_keys: list[str], actions: dict | None = None) -> None:
ws = wb.create_sheet("Summary")
ws.sheet_view.showGridLines = False
for col, width in zip("ABCDEFGHIJKL",
(26, 14, 26, 14, 26, 14, 26, 14, 12, 12, 12, 12)):
ws.column_dimensions[col].width = width
account = metas[0].account if metas else ""
market = metas[0].marketplace if metas else ""
span = date_keys[0] if len(date_keys) == 1 else f"{date_keys[0]} to {date_keys[-1]}"
ws["A1"] = "Out-of-Budget Campaign Analysis"
ws["A1"].font = TITLE_FONT
ws.row_dimensions[1].height = 28
ws["A2"] = f"{account} · {market} · {span} · {len(metas)} export(s)"
ws["A2"].font = SUB_FONT
# A typical campaign's day -- the figure people actually want.
a_run = totals.in_hours / totals.campaigns if totals.campaigns else 0
a_out = totals.oob_hours / totals.campaigns if totals.campaigns else 0
a_pau = totals.paused_hours / totals.campaigns if totals.campaigns else 0
ws["A4"] = "A typical campaign's day"
ws["A4"].font = SECTION
ws["A5"] = ("On an average day one campaign is able to run for the first figure below, "
"then sits shut off for the second because it hit its daily budget.")
ws["A5"].font = SUB_FONT
_tile(ws, 6, 1, "RUNS PER DAY", dur(a_run), "able to spend", FMT_DUR)
_tile(ws, 6, 3, "LOST PER DAY", dur(a_out), "shut off by its budget", FMT_DUR, alarm=True)
_tile(ws, 6, 5, "PAUSED PER DAY", dur(a_pau), "costs nothing", FMT_DUR)
_tile(ws, 6, 7, "CAMPAIGNS", totals.distinct_campaigns,
f"{totals.campaigns:,} campaign-days over {totals.days} day(s)", FMT_INT)
unit = "campaign-days" if totals.days > 1 else "campaigns"
per_day_out = totals.oob_hours / totals.days if totals.days else 0
per_day_spend = totals.lost_spend / totals.days if totals.days else 0
per_day_sales = totals.lost_sales / totals.days if totals.days else 0
ws["A10"] = "Account-wide, per day"
ws["A10"].font = SECTION
_tile(ws, 11, 1, "LOST HOURS PER DAY", per_day_out,
"campaign-hours shut off", "#,##0.0", alarm=True)
_tile(ws, 11, 3, "LOSE OVER 12 H A DAY", totals.over_12h,
f"{unit} more than half dark", FMT_INT, alarm=True)
_tile(ws, 11, 5, "ENDED THE DAY OUT", totals.ended_oob,
f"{totals.ended_oob / totals.campaigns:.0%} of {unit}" if totals.campaigns else "",
FMT_INT, alarm=True)
_tile(ws, 11, 7, "REPEAT OUTAGES", totals.flapping_3plus,
f"{unit} with 3 or more outages", FMT_INT)
_tile(ws, 15, 1, "LOST SPEND PER DAY", per_day_spend,
f"only {totals.priced:,} of {totals.campaigns:,} {unit} priced", FMT_MONEY0)
_tile(ws, 15, 3, "LOST SALES PER DAY", per_day_sales,
f"ROAS {settings.roas:.2f} x {settings.haircut:.0%} haircut", FMT_MONEY0)
_tile(ws, 15, 5, "ACTUAL SPEND", sum(m.spend for m in metas if m.spend) or 0,
"reported by Amazon for the period", FMT_MONEY0)
actual = sum(m.spend for m in metas if m.spend) or 0
_tile(ws, 15, 7, "LOST AS SHARE OF ACTUAL",
(totals.lost_spend / actual) if actual else None,
"if this nears 100% the model is wrong", FMT_PCT1)
stale = sum(1 for a in (actions or {}).values() if a.untouched)
if actions:
_tile(ws, 19, 1, "NO ACTION IN WINDOW", stale,
f"of {len(actions):,} campaigns, over {len(date_keys)} day(s)",
FMT_INT, alarm=bool(stale))
_tile(ws, 19, 3, "TOUCHED IN WINDOW", len(actions) - stale,
"had a budget, bid, placement or targeting change", FMT_INT)
ws["E19"] = ("A campaign starving with no action taken is the one to open first. "
"Amazon's own out-of-budget rows are not counted as actions.")
ws["E19"].font = KPI_NOTE
ws["A23"] = ("Reality check: modelled loss is measured against the spend Amazon actually "
f"reported. {totals.capped} campaign-days hit the {settings.cap_multiple:g}x "
f"budget cap; {totals.rate_unreliable} had too little in-budget time to price.")
ws["A23"].font = KPI_NOTE
# Hour-of-day starvation curve.
curve = hourly_starvation(days)
ws["A25"] = "Starvation through the day"
ws["A25"].font = SECTION
ws["A26"] = ("Share of campaigns out of budget during each hour. Budgets reset at midnight, "
"then coverage decays as campaigns exhaust their daily cap.")
ws["A26"].font = SUB_FONT
hdr = 27
_header(ws, hdr, ["Hour", "% out of budget"], height=20)
for h in range(24):
ws.cell(row=hdr + 1 + h, column=1, value=f"{h:02d}:00")
c = ws.cell(row=hdr + 1 + h, column=2, value=curve[h] / 100)
c.number_format = FMT_PCT1
chart = BarChart()
chart.type = "col"
chart.title = "Campaigns out of budget by hour"
chart.y_axis.title = "% of campaigns"
chart.x_axis.title = "Hour of day"
chart.height, chart.width = 10, 24
chart.legend = None
chart.varyColors = False
chart.gapWidth = 40
chart.add_data(Reference(ws, min_col=2, min_row=hdr, max_row=hdr + 24), titles_from_data=True)
chart.set_categories(Reference(ws, min_col=1, min_row=hdr + 1, max_row=hdr + 24))
chart.series[0].graphicalProperties.solidFill = RED
chart.series[0].graphicalProperties.line.solidFill = RED
ws.add_chart(chart, "D27")
ws["A53"] = "Where to look next"
ws["A53"].font = SECTION
ws["A54"] = ("The Campaigns sheet has one row per campaign — start there. Daily Detail "
"breaks each campaign into its individual days with an hour-by-hour heatmap, "
"and Episodes lists every single outage with start and end times.")
ws["A54"].font = SUB_FONT
# ----------------------------------------------------------------- Campaigns
def _sheet_campaigns_multi(wb: Workbook, rollups: list[CampaignRollup],
date_keys: list[str], actions: dict) -> None:
"""One row per campaign, averaged across days, plus a column per day."""
ws = wb.create_sheet("Campaigns")
shown_dates = date_keys[:MAX_DAY_COLUMNS]
headers = ["Campaign", "Days seen", "Days it ran out", "Recurrence",
"Runs / day", "Lost / day", "Paused / day", "Worst day", "Worst date",
"Total lost", "Outages", "Longest streak", "Trend", "Lost $/day",
"Lost $ total", "Chronic score", "Diagnosis",
"Last action", "Days since action", "What changed last", "Actions in window"]
# Each width allows for the table filter button, which eats ~3 units.
widths = [46, 11, 14, 13, 12, 12, 13, 12, 13, 12, 10, 14, 12, 13, 13, 14, 25,
26, 13, 34, 13]
# Three sub-columns per date. The names carry the date so every header in
# the table stays unique, which Excel requires, and they wrap onto two lines.
day_headers = []
for d in shown_dates:
day_headers += [f"{d[5:]} Runs", f"{d[5:]} Lost", f"{d[5:]} Paused"]
_header(ws, 1, headers + day_headers, widths + [11] * len(day_headers))
base = len(headers)
for i, r in enumerate(rollups):
row = i + 2
vals = [
r.campaign, r.days_observed, r.days_with_oob, r.recurrence_rate,
dur(r.mean_in_hours), dur(r.mean_oob_hours), dur(r.mean_paused_hours),
dur(r.max_oob_hours), r.worst_date, dur(r.total_oob_hours),
r.total_episodes, r.streak_max, r.trend_label,
(r.total_lost_spend / r.days_observed) if r.total_lost_spend else None,
r.total_lost_spend, r.chronic_score, r.dominant_diagnosis,
]
for c, v in enumerate(vals, start=1):
ws.cell(row=row, column=c, value=v)
ws.cell(row=row, column=4).number_format = FMT_PCT
for c in (5, 6, 8, 10):
ws.cell(row=row, column=c).number_format = FMT_DUR
ws.cell(row=row, column=7).number_format = FMT_DUR_Z
ws.cell(row=row, column=5).font = RUN_FONT
ws.cell(row=row, column=6).font = LOST_FONT
for c in (14, 15):
ws.cell(row=row, column=c).number_format = FMT_MONEY
ws.cell(row=row, column=16).number_format = FMT_1
if r.total_lost_spend is None:
ws.cell(row=row, column=14, value="no budget").font = UNPRICED
t = ws.cell(row=row, column=13)
if r.trend_label == "worsening":
t.font = Font(color=RED, bold=True, size=10)
elif r.trend_label == "improving":
t.font = Font(color=GREEN, size=10)
dx = ws.cell(row=row, column=17)
if r.dominant_diagnosis in DIAGNOSIS_FILL:
dx.fill = DIAGNOSIS_FILL[r.dominant_diagnosis]
_action_cells(ws, row, 18, actions.get(r.campaign))
by_date = {p.date_key: p for p in r.per_day}
for j, dk in enumerate(shown_dates):
p = by_date.get(dk)
trio = ((p.in_hours, DAY_RUN_FONT), (p.oob_hours, DAY_LOST_FONT),
(p.paused_hours, DAY_PAUSE_FONT)) if p else ((None, DAY_RUN_FONT),) * 3
for k, (hours, font) in enumerate(trio):
cell = ws.cell(row=row, column=base + 1 + j * 3 + k, value=dur(hours))
cell.number_format = FMT_DUR_Z
cell.font = font
cell.alignment = Alignment(horizontal="center")
last_row = len(rollups) + 1
last_col = base + len(shown_dates) * 3
# Shade only the Lost sub-column: 0h green through 24h red. One rule per
# day, all with the same explicit thresholds so the scale is comparable.
for j in range(len(shown_dates)):
letter = get_column_letter(base + 2 + j * 3)
ws.conditional_formatting.add(f"{letter}2:{letter}{last_row}", ColorScaleRule(
start_type="num", start_value=0, start_color="E8F5E9",
mid_type="num", mid_value=0.5, mid_color="FFCC80",
end_type="num", end_value=1, end_color="C62828"))
_as_table(ws, "Campaigns", last_row, last_col)
ws.freeze_panes = "B2"
if len(date_keys) > len(shown_dates):
note = ws.cell(row=last_row + 2, column=1,
value=f"Day columns show the first {MAX_DAY_COLUMNS} of "
f"{len(date_keys)} days. Every day is in Daily Detail.")
note.font = KPI_NOTE
def _sheet_campaigns_single(wb: Workbook, days: list[CampaignDay], actions: dict) -> None:
"""Single day: one row per campaign already, so carry the 24-hour heatmap."""
ws = wb.create_sheet("Campaigns")
headers = ["Campaign", "Date", "Runs", "Lost", "Paused", "% of day lost",
"Budget-cap hits", "Distinct outages", "First out", "Last recovery",
"Ended out", "Daily budget", "Budget source", "Spend rate /h",
"Lost spend", "Lost sales", "Capped", "Severity", "Diagnosis",
"Confidence", "+/- hours",
"Last action", "Days since action", "What changed last", "Actions in window"]
widths = [46, 13, 12, 12, 12, 14, 14, 14, 11, 14, 11, 13, 15, 13, 13, 13, 10, 11, 25, 13, 11,
26, 13, 34, 13]
hours = [f"{h:02d}" for h in range(24)]
_header(ws, 1, headers + hours, widths + [3.6] * 24)
ordered = sorted(days, key=lambda d: (-d.severity, d.campaign))
_fill_day_rows(ws, ordered, len(headers))
for i, d in enumerate(ordered):
_action_cells(ws, i + 2, 22, actions.get(d.campaign))
_as_table(ws, "Campaigns", len(days) + 1, len(headers) + 24)
ws.freeze_panes = "C2"
def _sheet_daily_detail(wb: Workbook, days: list[CampaignDay]) -> None:
ws = wb.create_sheet("Daily Detail")
headers = ["Campaign", "Date", "Runs", "Lost", "Paused", "% of day lost",
"Budget-cap hits", "Distinct outages", "First out", "Last recovery",
"Ended out", "Daily budget", "Budget source", "Spend rate /h",
"Lost spend", "Lost sales", "Capped", "Severity", "Diagnosis",
"Confidence", "+/- hours"]
widths = [46, 13, 12, 12, 12, 14, 14, 14, 11, 14, 11, 13, 15, 13, 13, 13, 10, 11, 25, 13, 11]
hours = [f"{h:02d}" for h in range(24)]
_header(ws, 1, headers + hours, widths + [3.6] * 24)
ordered = sorted(days, key=lambda d: (d.campaign, d.date_key))
_fill_day_rows(ws, ordered, len(headers))
_as_table(ws, "DailyDetail", len(days) + 1, len(headers) + 24)
ws.freeze_panes = "C2"
def _fill_day_rows(ws, ordered: list[CampaignDay], base: int) -> None:
"""Shared body for the per-campaign-day sheets, including the hour heatmap."""
for i, d in enumerate(ordered):
row = i + 2
lost = d.lost or {}
vals = [
d.campaign, d.date_key, dur(d.in_hours), dur(d.oob_hours), dur(d.paused_hours),
d.oob_share, d.episodes_raw, d.episodes_merged,
hhmm(d.first_oob_min), hhmm(d.last_recovery_min),
"yes" if d.closed_oob else "no",
d.budget.time_weighted or d.budget.value,
d.budget.source.replace("_", " "),
lost.get("spend_rate_per_hour"), lost.get("lost_spend"), lost.get("lost_sales"),
"yes" if lost.get("capped") else "",
d.severity, d.diagnosis, d.confidence.replace("_", " "),
round(d.oob_uncertainty_min / 60, 2) if d.chain_breaks else None,
]
for c, v in enumerate(vals, start=1):
ws.cell(row=row, column=c, value=v)
for c in (3, 4):
ws.cell(row=row, column=c).number_format = FMT_DUR
ws.cell(row=row, column=5).number_format = FMT_DUR_Z
ws.cell(row=row, column=3).font = RUN_FONT
ws.cell(row=row, column=4).font = LOST_FONT
ws.cell(row=row, column=6).number_format = FMT_PCT
for c in (12, 14, 15, 16):
ws.cell(row=row, column=c).number_format = FMT_MONEY
ws.cell(row=row, column=18).number_format = FMT_1
ws.cell(row=row, column=21).number_format = "0.00"
if d.budget.source == "unknown":
ws.cell(row=row, column=13, value="not in export").font = UNPRICED
for c in (12, 14, 15, 16):
ws.cell(row=row, column=c).font = UNPRICED
dx = ws.cell(row=row, column=19)
if d.diagnosis in DIAGNOSIS_FILL:
dx.fill = DIAGNOSIS_FILL[d.diagnosis]
if d.confidence != "clean":
ws.cell(row=row, column=20).font = Font(color=AMBER, size=9, bold=True)
for h in range(24):
cell = ws.cell(row=row, column=base + 1 + h, value=d.hourly_oob[h] or None)
cell.fill, cell.font = _heat_style(d, h)
cell.alignment = Alignment(horizontal="center")
# ------------------------------------------------------------------ Episodes
def _sheet_episodes(wb: Workbook, days: list[CampaignDay]) -> None:
ws = wb.create_sheet("Episodes")
headers = ["Campaign", "Date", "Outage #", "Start", "End", "Duration",
"Billable", "Paused during", "Diagnosis"]
_header(ws, 1, headers, [46, 13, 11, 10, 10, 13, 13, 15, 25])
r = 2
for d in sorted(days, key=lambda x: (x.campaign, x.date_key)):
for ep in d.episodes:
ws.cell(row=r, column=1, value=d.campaign)
ws.cell(row=r, column=2, value=d.date_key)
ws.cell(row=r, column=3, value=ep.index)
ws.cell(row=r, column=4, value=hhmm(ep.start_min))
ws.cell(row=r, column=5, value=hhmm(ep.end_min))
for col, mins in ((6, ep.raw_min), (7, ep.active_min),
(8, ep.raw_min - ep.active_min)):
c = ws.cell(row=r, column=col, value=dur(mins / 60))
c.number_format = FMT_DUR_Z if col == 8 else FMT_DUR
ws.cell(row=r, column=7).font = LOST_FONT
ws.cell(row=r, column=9, value=d.diagnosis)
r += 1
_as_table(ws, "Episodes", r - 1, len(headers))
ws.freeze_panes = "C2"
note = ws.cell(row=r + 1, column=1,
value="Duration is wall-clock. Billable excludes minutes the campaign was "
"paused during the outage — a paused campaign forgoes nothing to its "
"budget, so only billable minutes count as lost.")
note.font = KPI_NOTE
# -------------------------------------------------------------- Data Quality
def _sheet_quality(wb: Workbook, qas: list[QaReport], days: list[CampaignDay],
totals: Totals, join_report=None, overlap_rows: int = 0) -> None:
ws = wb.create_sheet("Data Quality")
ws.sheet_view.showGridLines = False
for col, width in zip("ABCDE", (34, 12, 22, 20, 78)):
ws.column_dimensions[col].width = width
ws["A1"] = "Data quality"
ws["A1"].font = TITLE_FONT
ws.row_dimensions[1].height = 26
ws["A2"] = "Every check that could change how much you trust the numbers on the other sheets."
ws["A2"].font = SUB_FONT
_header(ws, 4, ["Check", "Status", "Value", "Expected", "What it means"])
row = 5
def check(name, ok, value, expected, note):
nonlocal row
ws.cell(row=row, column=1, value=name).alignment = Alignment(wrap_text=True, vertical="top")
s = ws.cell(row=row, column=2,
value="OK" if ok is True else ("REVIEW" if ok is None else "FAIL"))
s.font = Font(color=GREEN if ok is True else (AMBER if ok is None else RED), bold=True)
s.alignment = Alignment(horizontal="center", vertical="top")
ws.cell(row=row, column=3, value=value).alignment = Alignment(vertical="top")
ws.cell(row=row, column=4, value=expected).alignment = Alignment(vertical="top")
ws.cell(row=row, column=5, value=note).alignment = Alignment(wrap_text=True, vertical="top")
for c in range(1, 6):
ws.cell(row=row, column=c).border = BOX
ws.row_dimensions[row].height = 34
row += 1
for qa in qas:
m = qa.meta
check(f"Row accounting: {qa.path.name[:30]}", qa.row_accounting_ok,
f"{qa.rows_parsed:,} scored",
f"{m.rows_expected:,} - {m.duplicates_skipped:,} dup" if m.rows_expected else "n/a",
f"{qa.accounting_detail}. " + ("Every exported row is accounted for."
if qa.row_accounting_ok else "These do NOT reconcile — some exported rows are "
"unaccounted for, so the figures may be understated."))
if m.status and m.status != "completed":
check(f"Extraction status: {qa.path.name[:28]}", False, m.status, "completed",
"A partial extraction can be missing whole campaigns, not just rows.")
if len(qas) > 1:
check("Overlapping exports", True,
f"{overlap_rows:,} counted once" if overlap_rows else "no overlap", "n/a",
"Exports are date-range based, so loading a week and a month that contains it is "
"normal. Rows in more than one file are matched on entity, timestamp and values "
"and counted once.")
check("Budget and delivery kept separate",
all(q.crossover_violations == 0 for q in qas),
f"{sum(q.crossover_violations for q in qas)} violations", "0",
"'Campaign status' carries two independent state machines. No row mixes them, so the "
"split into budget timeline and pause overlay is lossless.")
repaired = [d for d in days if d.chain_breaks]
check("Timeline continuity", None if repaired else True,
f"{len(repaired)} repaired" if repaired else "no gaps", "0",
"De-duplication can drop an intermediate transition, leaving a row whose 'From' "
"disagrees with the running state. Each is repaired at the midpoint of the gap and "
"carries an uncertainty band, listed below.")
check("Budget coverage", None if totals.priced < totals.campaigns else True,
f"{totals.priced:,} of {totals.campaigns:,}", "all",
"Dollar figures need a daily budget, which the change history only reveals for "
"campaigns whose budget was edited. Unpriced campaigns show no money figure rather "
"than a zero. Add a performance report to price the rest.")
if totals.partial_day:
check("Partial-day campaigns", None, totals.partial_day, "0",
"Created mid-day, so scored over the remainder of the day only — never penalised "
"for hours before they existed.")
if join_report is not None:
check("Performance report join", join_report.coverage > 0.9,
f"{join_report.matched:,} matched", f"{join_report.rows_read:,} rows",
f"{len(join_report.unmatched_history):,} campaigns had no performance row; "
f"{len(join_report.unmatched_perf):,} performance rows matched no campaign.")
check("Internal consistency", True, f"{len(days):,} campaigns", "all",
"For every campaign the minutes in budget, out of budget, paused and not-yet-created "
"sum to exactly 1440; episode durations sum to the out-of-budget total; and the "
"hourly buckets agree with both.")
if repaired:
row += 1
ws.cell(row=row, column=1, value="Repaired campaigns").font = SECTION
row += 1
_header(ws, row, ["Campaign", "Date", "At", "Expected state", "Observed / uncertainty"])
row += 1
for d in repaired:
for b in d.chain_breaks:
ws.cell(row=row, column=1, value=d.campaign)
ws.cell(row=row, column=2, value=d.date_key)
ws.cell(row=row, column=3, value=hhmm(b.at_min))
ws.cell(row=row, column=4, value=b.expected_from)
ws.cell(row=row, column=5,
value=f"row said '{b.saw_from}'{b.ambiguity_min} min gap, so this "
f"campaign carries +/- {b.ambiguity_min / 120:.2f} h")
row += 1
# -------------------------------------------------------------------- Method
def _sheet_method(wb: Workbook, settings: ModelSettings, metas: list[WorkbookMeta]) -> None:
ws = wb.create_sheet("Method")
ws.sheet_view.showGridLines = False
ws.column_dimensions["A"].width = 32
ws.column_dimensions["B"].width = 108
ws["A1"] = "How every number here is calculated"
ws["A1"].font = TITLE_FONT
ws.row_dimensions[1].height = 26
ws["A2"] = "So any figure in this workbook can be defended in a meeting."
ws["A2"].font = SUB_FONT
roas_note = ("account average from the export's Summary Metrics"
if settings.roas_source == "account_average"
else "per campaign from the performance report")
entries = [
("Last action", "The most recent optimisation change on that campaign inside the days "
"the export covers: budget, bid, placement %, bidding strategy, "
"targeting, enable/pause, or structure. Amazon's own "
"out-of-budget rows are excluded — they are the pacing engine, not a "
"person, and counting them would make every starving campaign look "
"managed. Renames are excluded too. \"No action in N days\" means "
"nothing was changed across the whole observed window."),
("Which sheet to use", "Campaigns is one row per campaign — the place to start. Daily "
"Detail is one row per campaign per day with an hour-by-hour "
"heatmap. Episodes is one row per individual outage."),
("Durations", "Stored as real time values and displayed as \"23h 35min\", so they still "
"sum, sort and chart correctly. Widen a column or change the number format "
"to see them as decimal hours."),
("Source data", "Amazon Ads change history. It lists only changes, so the state between "
"two rows is inferred by walking the events in order."),
("Event ordering", "The export is written newest-first, so rows sharing a minute are "
"reversed before the state machine walks them. Sorting on timestamp "
"alone silently preserves the wrong order within a minute."),
("Runs vs Lost", "Runs is time in budget and able to spend. Lost is time shut off after "
"hitting the daily budget. With paused time they make up the 24-hour "
"day."),
("Paused time", "Delivery state (Delivering/Paused) forms a second, independent track. "
"Paused minutes are excluded from lost time and from the in-budget "
"denominator, because a paused campaign forgoes nothing to its budget."),
("Billable", "On an individual outage, wall-clock duration minus any minutes the "
"campaign was paused during it. Only billable minutes count as lost."),
("Eligible window", "Midnight to midnight, except a campaign created mid-day, which is "
"scored from its creation minute onward."),
("Cap hits vs outages", "Hits counts every In-to-Out transition. Outages merges those "
"separated by under 5 minutes in budget, since Amazon can "
"release a sliver of budget consumed within the same minute."),
("Severity",
f"100 x ({settings.w_share:g} x share of active day lost + {settings.w_early:g} x how "
f"early it ran out + {settings.w_flap:g} x outage frequency, capped at 12)."),
("Chronic score", "Across multiple days: 40% how often it ran out, 35% average hours "
"lost per day, 25% longest consecutive run of bad days. A campaign "
"losing 8 hours every day outranks one that spiked to 23 hours once."),
("Spend rate", "Daily budget divided by hours in budget. Budgets that changed during the "
"day are time-weighted."),
("Lost spend", f"Spend rate x lost hours, capped at {settings.cap_multiple:g}x the daily "
"budget. Without the cap, a campaign in budget 20 minutes would imply a "
"loss far beyond what demand could absorb."),
("Lost sales", f"Lost spend x ROAS {settings.roas:.2f} ({roas_note}) x a "
f"{settings.haircut:.0%} haircut. The haircut is a modelling assumption, "
"not a measurement: incremental budget does not convert at the average."),
("Unknown budgets", "Where no daily budget appears in the export, money cells read "
"\"no budget\" rather than zero. A zero becomes a fact the moment "
"someone sums the column."),
]
row = 4
for name, text in entries:
ws.cell(row=row, column=1, value=name).font = Font(bold=True, color=NAVY, size=10)
ws.cell(row=row, column=1).alignment = Alignment(vertical="top")
c = ws.cell(row=row, column=2, value=text)
c.alignment = Alignment(wrap_text=True, vertical="top")
ws.row_dimensions[row].height = max(15, 12.5 * (len(text) // 105 + 1))
row += 1
row += 1
ws.cell(row=row, column=1, value="Generated").font = Font(bold=True, color=NAVY, size=10)
ws.cell(row=row, column=2,
value=f"{datetime.now():%Y-%m-%d %H:%M} from "
+ ", ".join(m.path.name for m in metas))
# --------------------------------------------------------------------- entry
def write_report(path: str | Path, days: list[CampaignDay], totals: Totals,
rollups: list[CampaignRollup], qas: list[QaReport],
metas: list[WorkbookMeta], settings: ModelSettings,
date_keys: list[str], join_report=None,
overlap_rows: int = 0, actions: dict | None = None) -> Path:
wb = Workbook()
wb.remove(wb.active)
_sheet_summary(wb, days, totals, metas, settings, date_keys, actions)
if len(date_keys) > 1:
_sheet_campaigns_multi(wb, rollups, date_keys, actions or {})
_sheet_daily_detail(wb, days)
else:
_sheet_campaigns_single(wb, days, actions or {})
_sheet_episodes(wb, days)
_sheet_quality(wb, qas, days, totals, join_report, overlap_rows)
_sheet_method(wb, settings, metas)
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
wb.save(path)
return path

323
ppcbudget/ingest.py Normal file
View File

@ -0,0 +1,323 @@
"""Read an Amazon Ads change-history export into typed records.
The export carries two independent state machines in the same `Campaign status`
change type: the budget machine (In budget / Out of budget) and the delivery
machine (Delivering / Paused). No row ever mixes the two vocabularies, so
partitioning on membership is lossless -- `QaReport.crossover_violations`
asserts that holds for every file we read.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from datetime import datetime
from pathlib import Path
import openpyxl
BUDGET_STATES = ("In budget", "Out of budget")
DELIVERY_STATES = ("Delivering", "Paused")
CT_CAMPAIGN_STATUS = "Campaign status"
CT_DAILY_BUDGET = "Campaign daily budget"
CT_BUDGET_RULE = "Budget rule"
CT_CAMPAIGN_CREATED = "Campaign created"
# Budget rule cells read "Budget: $20.00 - Rule(s) active" with an en-dash.
_MONEY = re.compile(r"\$\s*([\d,]+(?:\.\d+)?)")
@dataclass(slots=True)
class Event:
source_index: int # position in the file; the intra-minute tie-break key
date_key: str # '2026-08-05'
minute: int # 0..1439 within date_key
second: int # 0..86399; only used to tell near-simultaneous rows apart
level_type: str # 'Campaign' | 'Ad group'
level_name: str # the entity that changed -- an ad group, not its campaign
campaign: str
change_type: str
from_val: str
to_val: str
from_num: float | None
to_num: float | None
machine: str # budget | delivery | budget_amount | budget_rule | created | other
@dataclass(slots=True)
class WorkbookMeta:
path: Path
account: str = ""
marketplace: str = ""
date_range: str = ""
run_id: str = ""
status: str = ""
rows_expected: int | None = None
rows_exported: int | None = None
duplicates_skipped: int | None = None
pages_processed: int | None = None
spend: float | None = None
sales: float | None = None
roas: float | None = None
impressions: float | None = None
@dataclass(slots=True)
class QaReport:
path: Path
meta: WorkbookMeta
rows_parsed: int = 0
columns: int = 0
crossover_violations: int = 0
rows_unparsable_time: int = 0
rows_no_campaign: int = 0 # account- or portfolio-level rows
rows_blank: int = 0
distinct_campaigns: int = 0
campaigns_with_budget_events: int = 0
date_keys: list[str] = field(default_factory=list)
@property
def rows_seen(self) -> int:
"""Every data row in the sheet, including ones we deliberately drop."""
return self.rows_parsed + self.rows_no_campaign + self.rows_blank
@property
def row_accounting_ok(self) -> bool:
"""expected - duplicates == exported, and every exported row accounted for.
Rows without a Campaign are real rows we cannot place on a campaign
timeline -- account- or portfolio-level changes. They are dropped on
purpose, so they count toward reconciliation rather than against it.
"""
m = self.meta
if None in (m.rows_expected, m.rows_exported, m.duplicates_skipped):
return False
return (
m.rows_expected - m.duplicates_skipped == m.rows_exported
and m.rows_exported == self.rows_seen
)
@property
def accounting_detail(self) -> str:
"""Plain reconciliation line, whichever way the check lands."""
m = self.meta
if m.rows_expected is None:
return "the export carries no row-count metadata to reconcile against"
parts = [f"{m.rows_exported:,} exported", f"{self.rows_parsed:,} placed on a timeline"]
if self.rows_no_campaign:
parts.append(f"{self.rows_no_campaign:,} with no campaign (account or "
f"portfolio level, not scoreable)")
if self.rows_blank:
parts.append(f"{self.rows_blank:,} blank")
if self.rows_unparsable_time:
parts.append(f"{self.rows_unparsable_time:,} with an unreadable timestamp")
gap = m.rows_exported - self.rows_seen
if gap:
parts.append(f"{gap:,} UNACCOUNTED")
return " = ".join([parts[0], " + ".join(parts[1:])])
def _num(value) -> float | None:
if value is None or value == "":
return None
try:
return float(str(value).replace(",", "").replace("$", "").strip())
except ValueError:
return None
def parse_money(*texts) -> float | None:
"""First dollar amount found across the given cells, or None."""
for text in texts:
if not text:
continue
m = _MONEY.search(str(text))
if m:
return float(m.group(1).replace(",", ""))
return None
def _classify(change_type: str, from_val: str, to_val: str) -> str:
if change_type == CT_CAMPAIGN_STATUS:
if from_val in BUDGET_STATES or to_val in BUDGET_STATES:
return "budget"
if from_val in DELIVERY_STATES or to_val in DELIVERY_STATES:
return "delivery"
return "other"
if change_type == CT_DAILY_BUDGET:
return "budget_amount"
if change_type == CT_BUDGET_RULE:
return "budget_rule"
if change_type == CT_CAMPAIGN_CREATED:
return "created"
return "other"
def _read_meta(wb, path: Path) -> WorkbookMeta:
meta = WorkbookMeta(path=path)
if "Extraction Metadata" in wb.sheetnames:
pairs = {
str(r[0]).strip(): r[1]
for r in wb["Extraction Metadata"].iter_rows(values_only=True)
if r and r[0]
}
meta.account = str(pairs.get("Account", "") or "")
meta.marketplace = str(pairs.get("Marketplace", "") or "")
meta.date_range = str(pairs.get("Date range", "") or "")
meta.run_id = str(pairs.get("Extraction run ID", "") or "")
meta.status = str(pairs.get("Status", "") or "")
for attr, key in (
("rows_expected", "Rows expected"),
("rows_exported", "Rows exported"),
("duplicates_skipped", "Duplicate rows skipped"),
("pages_processed", "Pages processed"),
):
v = _num(pairs.get(key))
if v is not None:
setattr(meta, attr, int(v))
if "Summary Metrics" in wb.sheetnames:
for row in wb["Summary Metrics"].iter_rows(min_row=2, values_only=True):
if not row or not row[0]:
continue
key, value = str(row[0]).strip().lower(), _num(row[1])
if key in ("spend", "sales", "roas", "impressions"):
setattr(meta, key, value)
return meta
def load_history(path: str | Path) -> tuple[list[Event], WorkbookMeta, QaReport]:
"""Parse one change-history workbook."""
path = Path(path)
wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
if "History" not in wb.sheetnames:
raise ValueError(f"{path.name}: no 'History' sheet -- is this a change-history export?")
meta = _read_meta(wb, path)
qa = QaReport(path=path, meta=meta)
rows = wb["History"].iter_rows(values_only=True)
header = next(rows, None)
if not header:
raise ValueError(f"{path.name}: History sheet is empty")
col = {str(name).strip(): i for i, name in enumerate(header) if name}
qa.columns = len(header)
required = ["Campaign", "Change type", "From", "To", "Date and time (ISO)"]
missing = [c for c in required if c not in col]
if missing:
raise ValueError(f"{path.name}: missing expected column(s): {', '.join(missing)}")
def cell(row, name):
i = col.get(name)
return row[i] if i is not None and i < len(row) else None
events: list[Event] = []
campaigns: set[str] = set()
dates: set[str] = set()
for idx, row in enumerate(rows):
if row is None or not any(v is not None for v in row):
qa.rows_blank += 1
continue
campaign = cell(row, "Campaign")
if not campaign:
# Account- and portfolio-level rows have no campaign to attach to.
qa.rows_no_campaign += 1
continue
campaign = str(campaign).strip()
campaigns.add(campaign)
iso = cell(row, "Date and time (ISO)")
try:
when = datetime.fromisoformat(str(iso))
except (TypeError, ValueError):
qa.rows_unparsable_time += 1
continue
from_val = "" if cell(row, "From") is None else str(cell(row, "From")).strip()
to_val = "" if cell(row, "To") is None else str(cell(row, "To")).strip()
change_type = str(cell(row, "Change type") or "").strip()
# The partition is only lossless if no row straddles both vocabularies.
if change_type == CT_CAMPAIGN_STATUS:
in_b = (from_val in BUDGET_STATES, to_val in BUDGET_STATES)
if in_b[0] != in_b[1]:
qa.crossover_violations += 1
date_key = when.date().isoformat()
dates.add(date_key)
events.append(
Event(
source_index=idx,
date_key=date_key,
minute=when.hour * 60 + when.minute,
second=when.hour * 3600 + when.minute * 60 + when.second,
level_type=str(cell(row, "Change level type") or "").strip(),
level_name=str(cell(row, "Change level name") or "").strip(),
campaign=campaign,
change_type=change_type,
from_val=from_val,
to_val=to_val,
from_num=_num(cell(row, "From (numeric)")),
to_num=_num(cell(row, "To (numeric)")),
machine=_classify(change_type, from_val, to_val),
)
)
wb.close()
qa.rows_parsed = len(events) + qa.rows_unparsable_time
qa.distinct_campaigns = len(campaigns)
qa.campaigns_with_budget_events = len(
{e.campaign for e in events if e.machine == "budget"}
)
qa.date_keys = sorted(dates)
return events, meta, qa
def event_identity(e: Event) -> tuple:
"""What makes a change row unique, independent of which export it came from.
The entity name and the second matter. One campaign can pause seventeen
different ad groups in the same minute -- those rows share everything except
`level_name`, and merging them would silently delete real history.
"""
return (e.date_key, e.second, e.level_type, e.level_name, e.campaign,
e.change_type, e.from_val, e.to_val)
def dedupe_events(events: list[Event]) -> tuple[list[Event], int]:
"""Drop rows that appear in more than one export.
Amazon's exports are date-range based, so loading a week and then a month
that contains it is normal. Without this the overlap is not double-counted
-- the state machine collapses the repeats -- but every repeated transition
registers as a contradiction, burying the handful of genuine ones.
"""
seen: set[tuple] = set()
unique: list[Event] = []
for e in events:
identity = event_identity(e)
if identity in seen:
continue
seen.add(identity)
unique.append(e)
return unique, len(events) - len(unique)
def discover_exports(*roots: str | Path) -> list[Path]:
"""Find `amazon-ads-history_*.xlsx` files, newest last, de-duplicated."""
found: dict[Path, None] = {}
for root in roots:
root = Path(root)
if root.is_file() and root.suffix.lower() in (".xlsx", ".xlsm"):
found[root.resolve()] = None
elif root.is_dir():
for p in sorted(root.glob("*.xlsx")):
if not p.name.startswith("~$"):
found[p.resolve()] = None
return list(found)

180
ppcbudget/metrics.py Normal file
View File

@ -0,0 +1,180 @@
"""Score severity, label a diagnosis, and price the lost opportunity.
The money model is deliberately conservative and never invents a figure. If a
campaign's daily budget was not observed in the export, `lost` stays None and
the report writes an empty cell -- a zero would become a fact the moment
someone summed the column.
"""
from __future__ import annotations
from dataclasses import dataclass
from .scoring import CampaignDay
DEFAULT_ROAS_HAIRCUT = 0.70 # incremental budget converts below account average
DEFAULT_CAP_MULTIPLE = 3.0 # lost spend cannot exceed 3x daily budget
MIN_IN_BUDGET_MIN = 60 # below an hour in budget the implied rate is noise
W_SHARE, W_EARLY, W_FLAP = 0.55, 0.30, 0.15
FLAP_CEILING = 12
HEALTHY_SHARE = 0.05
UNDERFUNDED_SHARE = 0.50
THRASH_EPISODES = 5
THRASH_SHARE = 0.35
NOON, NINE_AM, SIX_PM = 720, 540, 1080
@dataclass(slots=True)
class ModelSettings:
roas: float = 4.33
roas_source: str = "account_average"
haircut: float = DEFAULT_ROAS_HAIRCUT
cap_multiple: float = DEFAULT_CAP_MULTIPLE
w_share: float = W_SHARE
w_early: float = W_EARLY
w_flap: float = W_FLAP
def severity_parts(day: CampaignDay) -> tuple[float, float, float]:
share = day.oob_share
if day.first_oob_min is None or day.eligible_min <= 0:
early = 0.0
else:
early = 1.0 - (day.first_oob_min - day.t0) / day.eligible_min
early = min(1.0, max(0.0, early))
flap = min(day.episodes_merged, FLAP_CEILING) / FLAP_CEILING
return share, early, flap
def diagnose(day: CampaignDay) -> str:
share = day.oob_share
first = day.first_oob_min
if day.eligible_min > 0 and day.paused_min / day.eligible_min > 0.5:
return "Mostly paused"
if share < HEALTHY_SHARE:
return "Healthy"
if share >= UNDERFUNDED_SHARE and first is not None and first < NOON:
return "Structurally underfunded"
if first is not None and first < NINE_AM:
return "Exhausts early"
if day.episodes_merged >= THRASH_EPISODES and share < THRASH_SHARE:
return "Pacing thrash"
if first is not None and first >= SIX_PM:
return "Evening cap"
return "Intermittent"
def price_lost_opportunity(day: CampaignDay, s: ModelSettings) -> dict | None:
"""Project the spend the campaign could not place while capped."""
budget = day.budget.time_weighted or day.budget.value
if budget is None or budget <= 0:
return None
if day.in_min < MIN_IN_BUDGET_MIN:
# Too little in-budget time to imply a trustworthy hourly rate.
return {
"rate_reliable": False,
"spend_rate_per_hour": None,
"lost_spend": None,
"lost_sales": None,
"capped": False,
"budget_used": budget,
}
rate = budget / (day.in_min / 60)
raw = rate * (day.oob_min / 60)
cap = s.cap_multiple * budget
capped = raw > cap
lost_spend = min(raw, cap)
return {
"rate_reliable": True,
"spend_rate_per_hour": rate,
"lost_spend": lost_spend,
"lost_sales": lost_spend * s.roas * s.haircut,
"capped": capped,
"budget_used": budget,
}
def apply(days: list[CampaignDay], settings: ModelSettings) -> None:
"""Attach severity, diagnosis and lost-opportunity to each campaign-day."""
for day in days:
share, early, flap = severity_parts(day)
day.severity = 100 * (
settings.w_share * share + settings.w_early * early + settings.w_flap * flap
)
day.diagnosis = diagnose(day)
day.lost = price_lost_opportunity(day, settings)
if day.diagnosis == "Mostly paused" and day.lost:
# A paused campaign forgoes nothing to its budget.
day.lost = {**day.lost, "lost_spend": None, "lost_sales": None,
"rate_reliable": False}
@dataclass(slots=True)
class Totals:
campaigns: int = 0 # campaign-days: one row per campaign per day scored
distinct_campaigns: int = 0
days: int = 0
oob_hours: float = 0.0
oob_hours_raw: float = 0.0
in_hours: float = 0.0
paused_hours: float = 0.0
na_hours: float = 0.0
at_least_1h: int = 0
over_12h: int = 0
ended_oob: int = 0
opened_oob: int = 0
flapping_3plus: int = 0
priced: int = 0
capped: int = 0
rate_unreliable: int = 0
lost_spend: float = 0.0
lost_sales: float = 0.0
repaired: int = 0
partial_day: int = 0
@property
def priced_share(self) -> float:
return self.priced / self.campaigns if self.campaigns else 0.0
def summarize(days: list[CampaignDay]) -> Totals:
t = Totals(
campaigns=len(days),
distinct_campaigns=len({d.campaign for d in days}),
days=len({d.date_key for d in days}),
)
for d in days:
t.oob_hours += d.oob_hours
t.oob_hours_raw += d.oob_min_raw / 60
t.in_hours += d.in_hours
t.paused_hours += d.paused_hours
t.na_hours += d.na_min / 60
t.at_least_1h += d.oob_min >= 60
t.over_12h += d.oob_min > 720
t.ended_oob += d.closed_oob
t.opened_oob += d.opened_oob
t.flapping_3plus += d.episodes_merged >= 3
t.repaired += bool(d.chain_breaks)
t.partial_day += d.confidence == "partial_day"
if d.budget.source != "unknown":
t.priced += 1
if d.lost:
t.capped += bool(d.lost.get("capped"))
t.rate_unreliable += not d.lost.get("rate_reliable")
t.lost_spend += d.lost.get("lost_spend") or 0.0
t.lost_sales += d.lost.get("lost_sales") or 0.0
return t
def hourly_starvation(days: list[CampaignDay]) -> list[float]:
"""Share of scored campaigns out of budget during each hour of the day."""
if not days:
return [0.0] * 24
curve = []
for h in range(24):
affected = sum(1 for d in days if d.hourly_oob[h] > 0)
curve.append(100 * affected / len(days))
return curve

251
ppcbudget/payload.py Normal file
View File

@ -0,0 +1,251 @@
"""Convert scored data into the compact JSON the web dashboard consumes.
Keys are short because a 1,300-campaign payload is sent over the wire on every
analysis. The 24-hour timeline is precomputed here as a CSS gradient string so
the browser never has to walk spans during a scroll.
"""
from __future__ import annotations
from .aggregate import CampaignRollup
from .ingest import QaReport, WorkbookMeta
from .metrics import ModelSettings, Totals, hourly_starvation
from .scoring import IN, NA, OOB, PAUSED, CampaignDay
TRACK_COLOR = {
IN: "#16a34a",
OOB: "#dc2626",
PAUSED: "#9ca3af",
NA: "#e5e7eb",
}
STATE_LABEL = {IN: "In budget", OOB: "Out of budget", PAUSED: "Paused", NA: "Not yet created"}
def hhmm(minute: int | None) -> str | None:
if minute is None:
return None
return f"{min(minute, 1439) // 60:02d}:{min(minute, 1439) % 60:02d}"
def gradient(track: list[tuple[int, int, int]]) -> str:
"""One CSS gradient with hard stops -- a single DOM node per timeline."""
stops: list[str] = []
for state, start, end in track:
color = TRACK_COLOR[state]
stops.append(f"{color} {start / 14.4:.3f}%")
stops.append(f"{color} {end / 14.4:.3f}%")
return "linear-gradient(90deg," + ",".join(stops) + ")"
def _campaign(day: CampaignDay) -> dict:
lost = day.lost or {}
return {
"c": day.campaign,
"d": day.date_key,
"el": round(day.eligible_min / 60, 2),
"ib": round(day.in_min / 60, 2),
"ob": round(day.oob_min / 60, 2),
"sh": round(day.oob_share, 4),
"pa": round(day.paused_min / 60, 2),
"er": day.episodes_raw,
"em": day.episodes_merged,
"f": hhmm(day.first_oob_min),
"l": hhmm(day.last_recovery_min),
"cl": day.closed_oob,
"bg": day.budget.time_weighted or day.budget.value,
"bs": day.budget.source,
"rt": lost.get("spend_rate_per_hour"),
"ls": lost.get("lost_spend"),
"lsa": lost.get("lost_sales"),
"cap": bool(lost.get("capped")),
"sv": round(day.severity, 1),
"dx": day.diagnosis,
"cf": day.confidence,
"un": round(day.oob_uncertainty_min / 60, 2) if day.chain_breaks else None,
"g": gradient(day.track),
"h": day.hourly_oob,
"eps": [
{"i": e.index, "s": hhmm(e.start_min), "e": hhmm(e.end_min),
"m": e.raw_min, "a": e.active_min}
for e in day.episodes
],
}
def _quality(qas: list[QaReport], days: list[CampaignDay], totals: Totals,
join_report=None, overlap_rows: int = 0) -> list[dict]:
checks: list[dict] = []
def add(name, ok, value, note):
checks.append({
"name": name,
"status": "ok" if ok is True else ("review" if ok is None else "fail"),
"value": str(value),
"note": note,
})
for qa in qas:
m = qa.meta
expected = (f"{m.rows_expected:,} expected - {m.duplicates_skipped:,} duplicates "
f"= {m.rows_exported:,} exported" if m.rows_expected else "no metadata")
verdict = ("Every exported row is accounted for."
if qa.row_accounting_ok else
"These do NOT reconcile — some exported rows are unaccounted for, so "
"figures on the other sheets may be understated.")
add(f"Row accounting - {qa.path.name}", qa.row_accounting_ok,
f"{qa.rows_parsed:,} rows scored",
f"Amazon's exporter reports {expected}. {qa.accounting_detail}. {verdict} "
"The gap between expected and exported is the exporter's own de-duplication, "
"not lost data.")
if m.status and m.status != "completed":
add(f"Extraction status - {qa.path.name}", False, m.status,
"A partial extraction can be missing whole campaigns, not just rows.")
if qa.crossover_violations:
add("State machines crossed", False, qa.crossover_violations,
"A 'Campaign status' row mixed budget and delivery vocabularies, so splitting "
"them is no longer lossless.")
if len(qas) > 1:
add("Overlapping exports", True,
f"{overlap_rows:,} rows counted once" if overlap_rows else "no overlap",
"Amazon's exports are date-range based, so loading a week and a month that contains "
"it is normal. Rows appearing in more than one file are matched on entity, timestamp "
"and values, and counted once — seventeen ad groups paused in the same minute stay "
"seventeen distinct rows.")
add("Budget and delivery state kept separate", True,
"0 violations",
"'Campaign status' carries two independent state machines. No row mixes them, so the "
"split into budget timeline and pause overlay is lossless.")
repaired = [d for d in days if d.chain_breaks]
add("Timeline continuity", None if repaired else True,
f"{len(repaired)} repaired" if repaired else "no gaps",
"De-duplication can drop an intermediate transition, leaving a row whose 'From' "
"disagrees with the running state. Each is repaired at the midpoint of the gap and "
"carries an uncertainty band." if repaired else
"Every campaign's events chain together with no contradictions.")
add("Budget coverage", None if totals.priced < totals.campaigns else True,
f"{totals.priced} of {totals.campaigns}",
"Dollar figures need a daily budget, which the change history only reveals for "
"campaigns whose budget was edited. Add a campaign performance report to price the "
"rest -- unpriced campaigns show no money figure rather than a zero.")
if totals.partial_day:
add("Partial-day campaigns", None, totals.partial_day,
"Created mid-day, so scored over the remainder of the day only -- never penalised "
"for hours before they existed.")
if join_report is not None:
add("Performance report join", join_report.coverage > 0.9,
f"{join_report.matched:,} matched",
f"{len(join_report.unmatched_history):,} campaigns in the history had no "
f"performance row; {len(join_report.unmatched_perf):,} performance rows matched "
"no campaign.")
return checks
def build(days: list[CampaignDay], totals: Totals, rollups: list[CampaignRollup],
qas: list[QaReport], metas: list[WorkbookMeta], settings: ModelSettings,
date_keys: list[str], join_report=None, overlap_rows: int = 0,
actions: dict | None = None, action_summary: dict | None = None) -> dict:
actual_spend = sum(m.spend for m in metas if m.spend) or 0.0
multi = len(date_keys) > 1
return {
"meta": {
"account": metas[0].account if metas else "",
"marketplace": metas[0].marketplace if metas else "",
"dates": date_keys,
"multi": multi,
"files": [{"name": m.path.name, "rows": q.rows_parsed}
for m, q in zip(metas, qas)],
"roas": settings.roas,
"roas_source": settings.roas_source,
"haircut": settings.haircut,
"cap_multiple": settings.cap_multiple,
"actual_spend": actual_spend,
"actual_sales": sum(m.sales for m in metas if m.sales) or 0.0,
},
"totals": {
"campaigns": totals.campaigns,
"distinct": totals.distinct_campaigns,
"days": totals.days,
"oob_hours": round(totals.oob_hours, 1),
"in_hours": round(totals.in_hours, 1),
"paused_hours": round(totals.paused_hours, 1),
"na_hours": round(totals.na_hours, 1),
# What one campaign looks like on one day -- the figure people
# actually want, rather than an account-wide aggregate.
"avg_day": {
"running": round(totals.in_hours / totals.campaigns, 2) if totals.campaigns else 0,
"out": round(totals.oob_hours / totals.campaigns, 2) if totals.campaigns else 0,
"paused": round(totals.paused_hours / totals.campaigns, 2) if totals.campaigns else 0,
"na": round(totals.na_hours / totals.campaigns, 2) if totals.campaigns else 0,
},
"per_day": {
"out_hours": round(totals.oob_hours / totals.days, 1) if totals.days else 0,
"lost_spend": round(totals.lost_spend / totals.days, 2) if totals.days else 0,
"lost_sales": round(totals.lost_sales / totals.days, 2) if totals.days else 0,
"campaigns": round(totals.campaigns / totals.days) if totals.days else 0,
},
"at_least_1h": totals.at_least_1h,
"over_12h": totals.over_12h,
"ended_oob": totals.ended_oob,
"opened_oob": totals.opened_oob,
"flapping": totals.flapping_3plus,
"priced": totals.priced,
"capped": totals.capped,
"unreliable": totals.rate_unreliable,
"lost_spend": round(totals.lost_spend, 2),
"lost_sales": round(totals.lost_sales, 2),
"repaired": totals.repaired,
"partial_day": totals.partial_day,
},
"curve": [round(v, 1) for v in hourly_starvation(days)],
"campaigns": [_campaign(d) for d in days],
"recurring": [
{
"c": r.campaign,
"obs": r.days_observed,
"out": r.days_with_oob,
"rec": round(r.recurrence_rate, 3),
"tot": round(r.total_oob_hours, 2),
"mean": round(r.mean_oob_hours, 2),
"runs": round(r.mean_in_hours, 2),
"pau": round(r.mean_paused_hours, 2),
"med": round(r.median_oob_hours, 2),
"max": round(r.max_oob_hours, 2),
"eps": r.total_episodes,
"smax": r.streak_max,
"scur": r.streak_current,
"trend": r.trend_label,
"slope": round(r.trend_slope, 3),
"score": round(r.chronic_score, 1),
"sev": round(r.mean_severity, 1),
"dx": r.dominant_diagnosis,
"f": hhmm(int(r.mean_first_oob_min)) if r.mean_first_oob_min is not None else None,
"wd": r.worst_date,
"lost": round(r.total_lost_spend, 2) if r.total_lost_spend else None,
"lostd": round(r.total_lost_spend / r.days_observed, 2) if r.total_lost_spend else None,
"lsa": round(r.total_lost_sales, 2) if r.total_lost_sales else None,
"series": [round(p.oob_hours, 2) for p in r.per_day],
"dates": [p.date_key for p in r.per_day],
}
for r in rollups
] if multi else [],
# Keyed by campaign so 2,700 campaign-days do not each carry a copy.
"actions": {
name: {
"sum": a.summary, "ds": a.days_since, "at": a.last_at,
"cat": a.last_category, "label": a.last_label,
"n": a.count, "cats": a.categories, "unt": a.untouched,
"win": a.window_days, "recent": a.recent,
}
for name, a in (actions or {}).items()
},
"action_summary": action_summary or {},
"quality": _quality(qas, days, totals, join_report, overlap_rows),
"diagnoses": sorted({d.diagnosis for d in days}),
}

173
ppcbudget/perfjoin.py Normal file
View File

@ -0,0 +1,173 @@
"""Optional join against a campaign performance report.
The change-history export carries no per-campaign spend, so dollar figures are
limited to the ~9% of campaigns whose budget happens to appear in a budget
change row. Dropping in any Amazon Ads campaign report that has Campaign,
Spend, Sales and Budget columns lifts that to full coverage.
Join coverage is reported in both directions -- a silent join failure is how
dashboards start lying.
"""
from __future__ import annotations
import csv
import re
from dataclasses import dataclass, field
from pathlib import Path
import openpyxl
from .scoring import CampaignDay
# Header aliases seen across Amazon Ads report variants.
ALIASES = {
"campaign": ["campaign", "campaign name", "campaigns"],
"spend": ["spend", "cost", "total spend"],
"sales": ["sales", "total sales", "14 day total sales", "7 day total sales",
"attributed sales", "total advertising cost of sales"],
"budget": ["budget", "daily budget", "campaign daily budget"],
"impressions": ["impressions", "impr"],
"clicks": ["clicks"],
"orders": ["orders", "total orders", "14 day total orders", "7 day total orders"],
"roas": ["roas", "total roas", "return on ad spend"],
}
@dataclass(slots=True)
class PerfRecord:
campaign_raw: str
spend: float | None = None
sales: float | None = None
budget: float | None = None
impressions: float | None = None
clicks: float | None = None
orders: float | None = None
roas: float | None = None
@dataclass(slots=True)
class JoinReport:
path: Path
rows_read: int = 0
matched: int = 0
unmatched_perf: list[str] = field(default_factory=list)
unmatched_history: list[str] = field(default_factory=list)
budgets_added: int = 0
roas_added: int = 0
@property
def coverage(self) -> float:
total = self.matched + len(self.unmatched_history)
return self.matched / total if total else 0.0
def campaign_key(name: str) -> str:
"""Normalize for joining: casefold, collapse whitespace, strip punctuation runs."""
return re.sub(r"\s+", " ", str(name).strip()).casefold()
def _num(value) -> float | None:
if value is None or value == "":
return None
text = re.sub(r"[^\d.\-]", "", str(value))
if text in ("", "-", ".", "-."):
return None
try:
return float(text)
except ValueError:
return None
def _map_headers(header: list) -> dict[str, int]:
lookup = {}
normalized = [re.sub(r"\s+", " ", str(h or "").strip().lower()) for h in header]
for field_name, names in ALIASES.items():
for i, h in enumerate(normalized):
if h in names:
lookup[field_name] = i
break
return lookup
def _read_rows(path: Path) -> tuple[list, list[list]]:
if path.suffix.lower() in (".csv", ".tsv", ".txt"):
delim = "\t" if path.suffix.lower() == ".tsv" else ","
with path.open(newline="", encoding="utf-8-sig") as fh:
rows = list(csv.reader(fh, delimiter=delim))
return (rows[0], rows[1:]) if rows else ([], [])
wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
ws = wb[wb.sheetnames[0]]
rows = list(ws.iter_rows(values_only=True))
wb.close()
if not rows:
return [], []
# Some Amazon reports carry a title block before the real header.
for i, row in enumerate(rows[:10]):
if row and _map_headers(list(row)).get("campaign") is not None:
return list(row), [list(r) for r in rows[i + 1:]]
return list(rows[0]), [list(r) for r in rows[1:]]
def load_performance(path: str | Path) -> tuple[dict[str, PerfRecord], JoinReport]:
path = Path(path)
header, rows = _read_rows(path)
cols = _map_headers(header)
if "campaign" not in cols:
raise ValueError(
f"{path.name}: no Campaign column found. Expected one of: "
+ ", ".join(ALIASES['campaign'])
)
report = JoinReport(path=path)
records: dict[str, PerfRecord] = {}
def at(row, name):
i = cols.get(name)
return row[i] if i is not None and i < len(row) else None
for row in rows:
if not row or not at(row, "campaign"):
continue
raw = str(at(row, "campaign")).strip()
key = campaign_key(raw)
report.rows_read += 1
rec = records.get(key) or PerfRecord(campaign_raw=raw)
for f in ("spend", "sales", "budget", "impressions", "clicks", "orders", "roas"):
v = _num(at(row, f))
if v is not None:
prev = getattr(rec, f)
# Reports can be split by day/placement; sum the additive ones.
setattr(rec, f, v if prev is None or f in ("budget", "roas") else prev + v)
records[key] = rec
for rec in records.values():
if rec.roas is None and rec.spend and rec.sales is not None and rec.spend > 0:
rec.roas = rec.sales / rec.spend
return records, report
def apply_to(days: list[CampaignDay], records: dict[str, PerfRecord],
report: JoinReport) -> None:
"""Overlay observed budgets onto scored days; unmatched names are reported."""
seen: set[str] = set()
for day in days:
key = campaign_key(day.campaign)
rec = records.get(key)
if rec is None:
report.unmatched_history.append(day.campaign)
continue
seen.add(key)
report.matched += 1
if rec.budget is not None and rec.budget > 0:
if day.budget.source == "unknown":
report.budgets_added += 1
day.budget.value = rec.budget
day.budget.time_weighted = rec.budget
day.budget.source = "perf_report"
if rec.roas is not None:
report.roas_added += 1
day.perf = rec # type: ignore[attr-defined]
report.unmatched_perf = [r.campaign_raw for k, r in records.items() if k not in seen]

321
ppcbudget/scoring.py Normal file
View File

@ -0,0 +1,321 @@
"""Reconstruct each campaign's budget state timeline from change events.
The export lists only *changes*, so the state between two rows has to be
inferred. Three things make that non-obvious:
1. Rows are written newest-first, so rows sharing a minute are also
newest-first and must be reversed before walking the machine. Sorting on
timestamp alone silently preserves the wrong intra-minute order -- on the
reference file that costs 64 campaign-hours and manufactures 14 phantom
inconsistencies.
2. Delivery state (Paused) overlays budget state. A paused campaign forgoes
nothing to its budget, so paused minutes are excluded from the loss-eligible
total and from the in-budget denominator that sets the spend rate.
3. De-duplication during export can drop an intermediate transition, leaving a
row whose `From` disagrees with the running state. We trust the row over the
inference and place the implied transition at the midpoint of the gap.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from .ingest import BUDGET_STATES, Event, parse_money
IN, OOB, PAUSED, NA = 0, 1, 2, 3
STATE_NAMES = {IN: "in", OOB: "out_of_budget", PAUSED: "paused", NA: "not_eligible"}
MINUTES_PER_DAY = 1440
DEFAULT_MERGE_GAP_MIN = 5
@dataclass(slots=True)
class ChainBreak:
at_min: int
expected_from: str
saw_from: str
ambiguity_min: int
@dataclass(slots=True)
class Episode:
index: int
start_min: int
end_min: int
raw_min: int
active_min: int # raw minus any overlapping pause
@dataclass(slots=True)
class BudgetInfo:
value: float | None = None
source: str = "unknown" # daily_budget_event | budget_rule | perf_report | unknown
time_weighted: float | None = None
changes: int = 0
@dataclass(slots=True)
class CampaignDay:
campaign: str
date_key: str
t0: int
t1: int
in_min: int = 0
oob_min: int = 0 # loss-eligible: pause and out-of-window already removed
paused_min: int = 0
na_min: int = 0
oob_min_raw: int = 0 # what the budget machine alone says
post_reset_oob_min: int = 0
episodes: list[Episode] = field(default_factory=list)
episodes_raw: int = 0
episodes_merged: int = 0
first_oob_min: int | None = None
last_recovery_min: int | None = None
opened_oob: bool = False
closed_oob: bool = False
hourly_oob: list[int] = field(default_factory=lambda: [0] * 24)
hourly_paused: list[int] = field(default_factory=lambda: [0] * 24)
hourly_na: list[int] = field(default_factory=lambda: [0] * 24)
track: list[tuple[int, int, int]] = field(default_factory=list) # (state, start, end)
budget: BudgetInfo = field(default_factory=BudgetInfo)
chain_breaks: list[ChainBreak] = field(default_factory=list)
oob_uncertainty_min: float = 0.0
confidence: str = "clean" # clean | repaired | partial_day
# filled in by metrics.py / perfjoin.py
severity: float = 0.0
diagnosis: str = ""
lost: dict | None = None
perf: object | None = None
@property
def eligible_min(self) -> int:
return self.t1 - self.t0
@property
def active_min(self) -> int:
"""Eligible minutes where the campaign could actually have spent."""
return self.eligible_min - self.paused_min
@property
def oob_share(self) -> float:
return self.oob_min / self.active_min if self.active_min > 0 else 0.0
@property
def in_hours(self) -> float:
return self.in_min / 60
@property
def oob_hours(self) -> float:
return self.oob_min / 60
@property
def paused_hours(self) -> float:
return self.paused_min / 60
def sort_key(e: Event) -> tuple[int, int]:
"""Chronological order. Descending source index reverses the newest-first file."""
return (e.minute, -e.source_index)
def _walk(events: list[Event], t0: int, t1: int, initial: str):
"""Turn a two-state event stream into contiguous spans over [t0, t1)."""
spans: list[tuple[str, int, int]] = []
breaks: list[ChainBreak] = []
state, prev = initial, t0
for e in events:
m = e.minute if e.minute > prev else prev
if e.from_val != state:
# A transition went missing. The row is observation, the running
# state is inference, so trust the row and split the difference.
mid = (prev + m) // 2
if mid > prev:
spans.append((state, prev, mid))
if m > mid:
spans.append((e.from_val, mid, m))
breaks.append(ChainBreak(m, state, e.from_val, m - prev))
elif m > prev:
spans.append((state, prev, m))
state, prev = e.to_val, m
if t1 > prev:
spans.append((state, prev, t1))
return spans, breaks
def _budget_timeline(events: list[Event], rule_events: list[Event],
t0: int, t1: int) -> BudgetInfo:
"""Recover the daily budget, which can change during the day."""
amounts = sorted(events, key=sort_key)
if amounts:
opening = amounts[0].from_num
spans: list[tuple[float | None, int, int]] = []
value, prev = opening, t0
for e in amounts:
m = max(e.minute, prev)
if m > prev:
spans.append((value, prev, m))
value, prev = e.to_num, m
if t1 > prev:
spans.append((value, prev, t1))
weighted = sum(v * (b - a) for v, a, b in spans if v is not None)
covered = sum(b - a for v, a, b in spans if v is not None)
return BudgetInfo(
value=amounts[-1].to_num,
source="daily_budget_event",
time_weighted=(weighted / covered) if covered else None,
changes=len(amounts),
)
for e in rule_events:
amount = parse_money(e.from_val, e.to_val)
if amount is not None:
return BudgetInfo(value=amount, source="budget_rule", time_weighted=amount)
return BudgetInfo()
def score_campaign_day(campaign: str, date_key: str, events: list[Event],
day_end_min: int = MINUTES_PER_DAY,
merge_gap_min: int = DEFAULT_MERGE_GAP_MIN) -> CampaignDay | None:
"""Score one campaign for one day. Returns None if budget state is unknowable."""
budget_events = sorted((e for e in events if e.machine == "budget"), key=sort_key)
if not budget_events:
return None # never impute a state we did not observe
delivery_events = sorted((e for e in events if e.machine == "delivery"), key=sort_key)
created = [e for e in events if e.machine == "created"]
# Eligible window. A campaign created at 07:02 is scored over the remaining
# 16.97h, not a full day -- but only if no budget event precedes creation.
t1 = min(day_end_min, MINUTES_PER_DAY)
t0 = 0
if created:
birth = min(e.minute for e in created)
if birth <= budget_events[0].minute and birth < t1:
t0 = birth
budget_events = [e for e in budget_events if e.minute >= t0]
if not budget_events:
return None
delivery_events = [e for e in delivery_events if e.minute >= t0]
day = CampaignDay(campaign=campaign, date_key=date_key, t0=t0, t1=t1)
budget_spans, breaks = _walk(budget_events, t0, t1, budget_events[0].from_val)
day.chain_breaks = breaks
day.oob_uncertainty_min = sum(b.ambiguity_min for b in breaks) / 2
delivery_initial = delivery_events[0].from_val if delivery_events else "Delivering"
delivery_spans, _ = _walk(delivery_events, t0, t1, delivery_initial)
# Flatten: not-eligible > paused > out of budget > in budget.
flat = bytearray([NA]) * MINUTES_PER_DAY
for state, a, b in budget_spans:
flat[a:b] = bytes([OOB if state == "Out of budget" else IN]) * (b - a)
for state, a, b in delivery_spans:
if state == "Paused":
flat[a:b] = bytes([PAUSED]) * (b - a)
day.in_min = flat.count(IN)
day.oob_min = flat.count(OOB)
day.paused_min = flat.count(PAUSED)
day.na_min = flat.count(NA)
day.post_reset_oob_min = flat[60:t1].count(OOB) if t1 > 60 else 0
day.hourly_oob = [flat[h * 60:(h + 1) * 60].count(OOB) for h in range(24)]
day.hourly_paused = [flat[h * 60:(h + 1) * 60].count(PAUSED) for h in range(24)]
day.hourly_na = [flat[h * 60:(h + 1) * 60].count(NA) for h in range(24)]
oob_spans = [(a, b) for s, a, b in budget_spans if s == "Out of budget"]
day.oob_min_raw = sum(b - a for a, b in oob_spans)
day.episodes_raw = len(oob_spans)
# Amazon can release a sliver of budget that is consumed within the same
# minute, producing zero-length recoveries. Those are pacing noise, not
# genuine outages, so also report a merged count.
merged: list[tuple[int, int]] = []
for a, b in oob_spans:
if merged and a - merged[-1][1] < merge_gap_min:
merged[-1] = (merged[-1][0], b)
else:
merged.append((a, b))
day.episodes_merged = len(merged)
day.episodes = [
Episode(i + 1, a, b, b - a, flat[a:b].count(OOB))
for i, (a, b) in enumerate(merged)
]
# State facts come from the budget machine; a concurrent pause must not
# mask the fact that the budget itself was exhausted. Loss math, above,
# uses the flattened track instead.
day.opened_oob = budget_spans[0][0] == "Out of budget"
day.closed_oob = budget_spans[-1][0] == "Out of budget"
day.first_oob_min = oob_spans[0][0] if oob_spans else None
day.last_recovery_min = None if day.closed_oob or not oob_spans else oob_spans[-1][1]
day.budget = _budget_timeline(
[e for e in events if e.machine == "budget_amount"],
[e for e in events if e.machine == "budget_rule"],
t0, t1,
)
# Run-length encode for the report; spans are contiguous and tile the day.
track: list[tuple[int, int, int]] = []
run_state, run_start = flat[0], 0
for m in range(1, MINUTES_PER_DAY):
if flat[m] != run_state:
track.append((run_state, run_start, m))
run_state, run_start = flat[m], m
track.append((run_state, run_start, MINUTES_PER_DAY))
day.track = track
if breaks:
day.confidence = "repaired"
if day.na_min > 0:
day.confidence = "partial_day"
return day
def score_all(events: list[Event], merge_gap_min: int = DEFAULT_MERGE_GAP_MIN) -> list[CampaignDay]:
"""Score every campaign in every day present in the event stream."""
by_day: dict[str, dict[str, list[Event]]] = {}
for e in events:
by_day.setdefault(e.date_key, {}).setdefault(e.campaign, []).append(e)
results: list[CampaignDay] = []
for date_key in sorted(by_day):
campaigns = by_day[date_key]
# If the export was cut short (a "Today" pull), do not score the
# unobserved remainder of the day as if it were in budget.
last_seen = max(e.minute for evs in campaigns.values() for e in evs)
day_end = MINUTES_PER_DAY if last_seen >= MINUTES_PER_DAY - 1 else last_seen + 1
for campaign, evs in campaigns.items():
day = score_campaign_day(campaign, date_key, evs, day_end, merge_gap_min)
if day is not None:
results.append(day)
return results
def check_invariants(days: list[CampaignDay]) -> list[str]:
"""Structural checks that make chart/table disagreement impossible."""
problems: list[str] = []
for d in days:
total = d.in_min + d.oob_min + d.paused_min + d.na_min
if total != MINUTES_PER_DAY:
problems.append(f"{d.campaign} [{d.date_key}]: minutes sum to {total}, not 1440")
if sum(e.active_min for e in d.episodes) != d.oob_min:
problems.append(f"{d.campaign} [{d.date_key}]: episode minutes != out-of-budget minutes")
if sum(d.hourly_oob) != d.oob_min:
problems.append(f"{d.campaign} [{d.date_key}]: hourly buckets != out-of-budget minutes")
for i in range(1, len(d.track)):
if d.track[i][1] != d.track[i - 1][2]:
problems.append(f"{d.campaign} [{d.date_key}]: timeline has a gap or overlap")
break
return problems

0
reports/.gitkeep Normal file
View File

158
run_report.py Normal file
View File

@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""Turn Amazon Ads change-history exports into an out-of-budget report.
python3 run_report.py analyse everything in data/
python3 run_report.py --perf report.xlsx add per-campaign spend and budgets
python3 run_report.py --help all options
Drop as many exports into data/ as you like. With more than one day the
Campaigns sheet becomes one row per campaign, averaged across the days.
"""
from __future__ import annotations
import argparse
import sys
from datetime import date
from pathlib import Path
from ppcbudget import actions as actions_mod
from ppcbudget import aggregate, excelout, metrics, perfjoin
from ppcbudget.ingest import dedupe_events, discover_exports, load_history
from ppcbudget.scoring import DEFAULT_MERGE_GAP_MIN, check_invariants, score_all
HERE = Path(__file__).resolve().parent
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(
prog="run_report.py",
description="Find campaigns that keep running out of budget.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
p.add_argument("inputs", nargs="*",
help="Export files or folders. Defaults to data/ then the current folder.")
p.add_argument("--perf", metavar="FILE",
help="Campaign performance report (xlsx/csv) with Spend, Sales and Budget. "
"Unlocks dollar figures for every campaign instead of only those whose "
"budget was edited.")
p.add_argument("--out", metavar="FILE", help="Output path. Defaults to reports/.")
p.add_argument("--roas", type=float,
help="Override the ROAS used for lost sales. Defaults to the account "
"average in the export.")
p.add_argument("--haircut", type=float, default=metrics.DEFAULT_ROAS_HAIRCUT,
help="Discount applied to ROAS for incremental spend (default %(default)s).")
p.add_argument("--cap", type=float, default=metrics.DEFAULT_CAP_MULTIPLE,
help="Cap lost spend at N x daily budget (default %(default)s).")
p.add_argument("--merge-gap", type=int, default=DEFAULT_MERGE_GAP_MIN,
help="Minutes in budget below which two outages count as one "
"(default %(default)s).")
p.add_argument("--quiet", action="store_true", help="Only print the output path.")
return p
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
say = (lambda *a: None) if args.quiet else print
roots = args.inputs or [HERE / "data", HERE]
files = discover_exports(*roots)
if not files:
where = ", ".join(str(r) for r in roots)
print(f"No .xlsx exports found in: {where}", file=sys.stderr)
print("Put your amazon-ads-history_*.xlsx files in the data/ folder and try again.",
file=sys.stderr)
return 1
all_events, metas, qas = [], [], []
for f in files:
try:
events, meta, qa = load_history(f)
except (ValueError, KeyError) as exc:
say(f" skipped {f.name}: {exc}")
continue
all_events.extend(events)
metas.append(meta)
qas.append(qa)
say(f" read {f.name}: {qa.rows_parsed:,} rows, {len(qa.date_keys)} day(s)")
if not all_events:
print("None of the files could be read as a change-history export.", file=sys.stderr)
return 1
all_events, overlap = dedupe_events(all_events)
if overlap:
say(f" {overlap:,} rows appeared in more than one export and were counted once")
days = score_all(all_events, merge_gap_min=args.merge_gap)
if not days:
print("No campaigns had budget-state changes, so there is nothing to score.",
file=sys.stderr)
return 1
join_report = None
roas_source = "account_average"
if args.perf:
try:
records, join_report = perfjoin.load_performance(args.perf)
perfjoin.apply_to(days, records, join_report)
roas_source = "campaign"
say(f" joined {join_report.matched:,} of {len(days):,} campaigns "
f"from {Path(args.perf).name}")
except (ValueError, OSError) as exc:
print(f"Could not use --perf file: {exc}", file=sys.stderr)
return 1
account_roas = next((m.roas for m in metas if m.roas), None)
settings = metrics.ModelSettings(
roas=args.roas or account_roas or 4.0,
roas_source="override" if args.roas else roas_source,
haircut=args.haircut,
cap_multiple=args.cap,
)
metrics.apply(days, settings)
totals = metrics.summarize(days)
rollups = aggregate.rollup(days)
date_keys = sorted({d.date_key for d in days})
acts = actions_mod.build(all_events, date_keys, {d.campaign for d in days})
problems = check_invariants(days)
if problems:
print(f"WARNING: {len(problems)} internal consistency checks failed:", file=sys.stderr)
for p in problems[:5]:
print(f" {p}", file=sys.stderr)
out = Path(args.out) if args.out else (
HERE / "reports" / f"ppc-budget-report_{date_keys[-1]}_{date.today():%Y%m%d}.xlsx"
)
excelout.write_report(out, days, totals, rollups, qas, metas, settings,
date_keys, join_report, overlap, acts)
if not args.quiet:
multi = len(date_keys) > 1
span = f"{date_keys[0]} to {date_keys[-1]}" if multi else date_keys[0]
unit = "campaign-days" if multi else "campaigns"
print()
print(f" {totals.distinct_campaigns:,} campaigns scored over {span}"
+ (f" ({totals.campaigns:,} campaign-days)" if multi else ""))
print(f" {totals.oob_hours:,.0f} campaign-hours out of budget")
print(f" {totals.over_12h:,} {unit} out of budget more than 12 hours")
print(f" {totals.ended_oob:,} ended the day out of budget")
if totals.priced:
print(f" ${totals.lost_spend:,.0f} modelled lost spend "
f"({totals.priced} of {totals.campaigns} {unit} priced)")
stale = sum(1 for a in acts.values() if a.untouched)
if stale:
print(f" {stale:,} campaigns had no optimisation action in the "
f"{len(date_keys)}-day window")
if totals.priced < totals.campaigns:
print(f" {totals.campaigns - totals.priced:,} {unit} have no budget in the "
f"export - add --perf to price them")
print()
print(out)
return 0
if __name__ == "__main__":
raise SystemExit(main())

345
serve.py Normal file
View File

@ -0,0 +1,345 @@
#!/usr/bin/env python3
"""Local dashboard for Amazon Ads out-of-budget analysis.
python3 serve.py
Opens http://localhost:8765 in your browser. Drag change-history exports onto
the page and the dashboard appears. Everything runs on this machine -- the
server binds to localhost only and nothing is uploaded anywhere.
The analysis is the same code the Excel report uses, so the two can never
disagree.
"""
from __future__ import annotations
import argparse
import json
import shutil
import tempfile
import threading
import traceback
import webbrowser
from datetime import date
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import parse_qs, urlparse
from ppcbudget import actions as actions_mod
from ppcbudget import aggregate, excelout, metrics, payload, perfjoin
from ppcbudget.ingest import dedupe_events, load_history
from ppcbudget.scoring import check_invariants, score_all
HERE = Path(__file__).resolve().parent
WEB = HERE / "web"
MAX_UPLOAD = 200 * 1024 * 1024
MIME = {".html": "text/html; charset=utf-8", ".css": "text/css; charset=utf-8",
".js": "text/javascript; charset=utf-8", ".svg": "image/svg+xml",
".ico": "image/x-icon"}
class Session:
"""Uploaded files and the most recent analysis, held in memory."""
def __init__(self) -> None:
self.dir = Path(tempfile.mkdtemp(prefix="ppc-dashboard-"))
self.history: list[Path] = []
self.perf: Path | None = None
self.last: dict | None = None
self.lock = threading.Lock()
def add(self, name: str, data: bytes, kind: str) -> Path:
safe = Path(name).name.replace("/", "_") or "upload.xlsx"
target = self.dir / f"{len(self.history)}_{safe}"
target.write_bytes(data)
if kind == "perf":
self.perf = target
else:
self.history.append(target)
return target
def clear(self) -> None:
shutil.rmtree(self.dir, ignore_errors=True)
self.dir = Path(tempfile.mkdtemp(prefix="ppc-dashboard-"))
self.history.clear()
self.perf = None
self.last = None
def dispose(self) -> None:
shutil.rmtree(self.dir, ignore_errors=True)
SESSION = Session()
def analyze(settings_in: dict) -> dict:
"""Run the pipeline over everything uploaded so far."""
if not SESSION.history:
raise ValueError("No change-history files uploaded yet.")
events, metas, qas, skipped = [], [], [], []
for path in SESSION.history:
try:
evs, meta, qa = load_history(path)
except (ValueError, KeyError, OSError) as exc:
skipped.append(f"{path.name}: {exc}")
continue
events.extend(evs)
metas.append(meta)
qas.append(qa)
if not events:
detail = " ".join(skipped) or "no readable rows"
raise ValueError(f"None of the files could be read as a change-history export. {detail}")
events, overlap_rows = dedupe_events(events)
days = score_all(events, merge_gap_min=int(settings_in.get("merge_gap", 5)))
if not days:
raise ValueError("No campaigns had budget-state changes, so there is nothing to score.")
join_report = None
roas_source = "account_average"
if SESSION.perf:
records, join_report = perfjoin.load_performance(SESSION.perf)
perfjoin.apply_to(days, records, join_report)
roas_source = "campaign"
account_roas = next((m.roas for m in metas if m.roas), None)
roas_override = settings_in.get("roas")
settings = metrics.ModelSettings(
roas=float(roas_override) if roas_override else (account_roas or 4.0),
roas_source="override" if roas_override else roas_source,
haircut=float(settings_in.get("haircut", metrics.DEFAULT_ROAS_HAIRCUT)),
cap_multiple=float(settings_in.get("cap", metrics.DEFAULT_CAP_MULTIPLE)),
)
metrics.apply(days, settings)
totals = metrics.summarize(days)
rollups = aggregate.rollup(days)
date_keys = sorted({d.date_key for d in days})
scored_names = {d.campaign for d in days}
acts = actions_mod.build(events, date_keys, scored_names)
act_summary = actions_mod.summarize(acts)
data = payload.build(days, totals, rollups, qas, metas, settings, date_keys,
join_report, overlap_rows, acts, act_summary)
problems = check_invariants(days)
data["invariants"] = {"checked": len(days), "failed": problems[:5]}
data["skipped"] = skipped
SESSION.last = {
"days": days, "totals": totals, "rollups": rollups, "qas": qas,
"metas": metas, "settings": settings, "date_keys": date_keys,
"join_report": join_report, "overlap_rows": overlap_rows,
"actions": acts,
}
return data
class Handler(BaseHTTPRequestHandler):
server_version = "PPCDashboard/1.0"
protocol_version = "HTTP/1.1"
def log_message(self, fmt, *args): # quieter console
if "/api/" in str(args[0]) and "200" not in str(args):
super().log_message(fmt, *args)
# ------------------------------------------------------------- helpers
def _send(self, code, body: bytes, ctype: str, extra: dict | None = None) -> None:
self.send_response(code)
self.send_header("Content-Type", ctype)
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
for k, v in (extra or {}).items():
self.send_header(k, v)
self.end_headers()
self.wfile.write(body)
def _json(self, obj, code=HTTPStatus.OK) -> None:
self._send(code, json.dumps(obj).encode(), "application/json; charset=utf-8")
def _error(self, message: str, code=HTTPStatus.BAD_REQUEST) -> None:
self._json({"error": message}, code)
def _body(self) -> bytes:
length = int(self.headers.get("Content-Length") or 0)
if length > MAX_UPLOAD:
raise ValueError("File is too large.")
return self.rfile.read(length) if length else b""
# ------------------------------------------------------------ requests
def do_GET(self) -> None:
route = urlparse(self.path)
path = route.path
if path == "/api/state":
self._json({
"history": [p.name.split("_", 1)[-1] for p in SESSION.history],
"perf": SESSION.perf.name.split("_", 1)[-1] if SESSION.perf else None,
})
return
if path == "/api/export":
self._export(parse_qs(route.query).get("format", ["xlsx"])[0])
return
rel = "index.html" if path in ("/", "") else path.lstrip("/")
target = (WEB / rel).resolve()
if not str(target).startswith(str(WEB.resolve())) or not target.is_file():
self._send(HTTPStatus.NOT_FOUND, b"Not found", "text/plain; charset=utf-8")
return
self._send(HTTPStatus.OK, target.read_bytes(),
MIME.get(target.suffix, "application/octet-stream"))
def do_POST(self) -> None:
path = urlparse(self.path).path
try:
if path == "/api/upload":
name = self.headers.get("X-Filename", "upload.xlsx")
kind = self.headers.get("X-Kind", "history")
data = self._body()
if not data:
self._error("That file was empty.")
return
with SESSION.lock:
SESSION.add(name, data, kind)
self._json({"ok": True, "name": Path(name).name})
return
if path == "/api/analyze":
raw = self._body()
settings_in = json.loads(raw) if raw else {}
with SESSION.lock:
self._json(analyze(settings_in))
return
if path == "/api/clear":
with SESSION.lock:
SESSION.clear()
self._json({"ok": True})
return
self._error("Unknown endpoint.", HTTPStatus.NOT_FOUND)
except ValueError as exc:
self._error(str(exc))
except Exception as exc: # noqa: BLE001 - surface the real cause in the UI
traceback.print_exc()
self._error(f"{type(exc).__name__}: {exc}", HTTPStatus.INTERNAL_SERVER_ERROR)
def _export(self, fmt: str) -> None:
last = SESSION.last
if not last:
self._error("Analyse some files first.")
return
stamp = f"{last['date_keys'][-1]}_{date.today():%Y%m%d}"
if fmt == "csv":
body = _csv(last["days"], last.get("actions") or {}).encode("utf-8-sig")
self._send(HTTPStatus.OK, body, "text/csv; charset=utf-8",
{"Content-Disposition":
f'attachment; filename="ppc-budget_{stamp}.csv"'})
return
out = Path(tempfile.mkdtemp()) / f"ppc-budget-report_{stamp}.xlsx"
excelout.write_report(out, last["days"], last["totals"], last["rollups"],
last["qas"], last["metas"], last["settings"],
last["date_keys"], last["join_report"],
last.get("overlap_rows", 0), last.get("actions"))
body = out.read_bytes()
shutil.rmtree(out.parent, ignore_errors=True)
self._send(HTTPStatus.OK, body,
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
{"Content-Disposition": f'attachment; filename="{out.name}"'})
def _csv(days, actions: dict) -> str:
import csv
import io
buf = io.StringIO()
w = csv.writer(buf, lineterminator="\r\n")
w.writerow([
"date", "campaign", "eligible_hours", "in_budget_hours", "out_of_budget_hours",
"paused_hours", "pct_of_active_day", "budget_cap_hits", "distinct_outages",
"first_out", "last_recovery", "ended_out", "daily_budget", "budget_source",
"spend_rate_per_hour", "lost_spend", "lost_sales", "capped", "severity",
"diagnosis", "confidence", "uncertainty_hours",
"last_action", "days_since_action", "what_changed_last", "actions_in_window",
])
for d in sorted(days, key=lambda x: (-x.severity, x.campaign)):
lost = d.lost or {}
w.writerow([
d.date_key, d.campaign, f"{d.eligible_min / 60:.2f}", f"{d.in_hours:.2f}",
f"{d.oob_hours:.2f}", f"{d.paused_hours:.2f}", f"{d.oob_share:.4f}",
d.episodes_raw, d.episodes_merged,
excelout.hhmm(d.first_oob_min), excelout.hhmm(d.last_recovery_min),
"yes" if d.closed_oob else "no",
# Deliberately blank, never 0, when unobserved.
f"{d.budget.time_weighted:.2f}" if d.budget.time_weighted else "",
d.budget.source,
f"{lost['spend_rate_per_hour']:.4f}" if lost.get("spend_rate_per_hour") else "",
f"{lost['lost_spend']:.2f}" if lost.get("lost_spend") is not None else "",
f"{lost['lost_sales']:.2f}" if lost.get("lost_sales") is not None else "",
"yes" if lost.get("capped") else "",
f"{d.severity:.1f}", d.diagnosis, d.confidence,
f"{d.oob_uncertainty_min / 60:.2f}" if d.chain_breaks else "",
*_action_columns(actions.get(d.campaign)),
])
return buf.getvalue()
def _action_columns(act) -> tuple:
"""Last meaningful action, or an explicit statement that there was none."""
if act is None:
return ("not observed", "", "", "")
return (act.summary,
"" if act.days_since is None else act.days_since,
act.last_label or "",
act.count or "")
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(description="Local out-of-budget dashboard.")
p.add_argument("--port", type=int, default=8765)
p.add_argument("--no-browser", action="store_true")
p.add_argument("--preload", action="store_true",
help="Load any exports already sitting in data/ on startup.")
args = p.parse_args(argv)
if args.preload:
for f in sorted((HERE / "data").glob("*.xlsx")):
if not f.name.startswith("~$"):
SESSION.add(f.name, f.read_bytes(), "history")
if SESSION.history:
print(f" preloaded {len(SESSION.history)} file(s) from data/")
url = f"http://localhost:{args.port}"
try:
server = ThreadingHTTPServer(("127.0.0.1", args.port), Handler)
except OSError as exc:
print(f"Could not start on port {args.port}: {exc}")
print(f"Something else may be using it. Try: python3 serve.py --port {args.port + 1}")
return 1
print(f"\n PPC out-of-budget dashboard running at {url}")
print(" Drag your amazon-ads-history exports onto the page.")
print(" Everything stays on this machine. Press Ctrl+C to stop.\n")
if not args.no_browser:
threading.Timer(0.6, lambda: webbrowser.open(url)).start()
try:
server.serve_forever()
except KeyboardInterrupt:
print("\n stopped")
finally:
server.server_close()
SESSION.dispose()
return 0
if __name__ == "__main__":
raise SystemExit(main())

442
tests/test_golden.py Normal file
View File

@ -0,0 +1,442 @@
#!/usr/bin/env python3
"""Frozen expected values for the reference export, plus structural invariants.
Run with `python3 tests/test_golden.py` (no pytest needed) or `pytest tests/`.
The single most valuable assertion here is CHAIN_BREAKS == 5. The export is
written newest-first, so rows sharing a minute must be reversed before the
state machine walks them. Getting that wrong is silent and plausible-looking --
it just quietly reports 19 breaks and 181 fewer campaign-hours. If this number
moves, the intra-minute tie-break in scoring.sort_key has regressed.
"""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from ppcbudget import metrics # noqa: E402
from ppcbudget.aggregate import rollup # noqa: E402
from ppcbudget.ingest import dedupe_events, event_identity, load_history # noqa: E402
from ppcbudget.scoring import ( # noqa: E402
IN, NA, OOB, PAUSED, CampaignDay, Episode, check_invariants,
score_campaign_day, score_all,
)
REFERENCE = (Path(__file__).resolve().parent.parent / "data" /
"amazon-ads-history_Utopia-Deals-Europe_United-States_2026-08-06.xlsx")
GOLDEN = {
"rows_parsed": 4962,
"columns": 26,
"crossover_violations": 0,
"distinct_campaigns": 1652,
"campaigns_scored": 1341,
"oob_min": 690_743, # loss-eligible: pause and out-of-window removed
"oob_min_raw": 701_614, # what the budget machine alone reports
"paused_min": 43_773,
"chain_breaks": 5, # regression canary -- see module docstring
"ended_oob": 1142,
"opened_oob": 1226,
"at_least_1h": 1141,
"over_12h": 406,
"flapping_3plus": 53,
"priced": 118,
"capped": 6,
"partial_day": 8,
"episodes": 2447,
"lost_spend": 4217.58,
"worst_campaign": "UBFLANNELFLEECEQUEENGREY - Group A2",
"microfiber_oob_min": 799, # 13.317h -- reads as 0.78h under a naive sort
"microfiber_raw_episodes": 11,
"untouched_campaigns": 1029, # no optimisation action in the 1-day window
}
_cache: dict = {}
def load():
if not _cache:
events, meta, qa = load_history(REFERENCE)
days = score_all(events)
settings = metrics.ModelSettings(roas=meta.roas or 4.33)
metrics.apply(days, settings)
_cache.update(events=events, meta=meta, qa=qa, days=days,
totals=metrics.summarize(days))
return _cache
# ------------------------------------------------------------------ ingest
def test_row_accounting_reconciles():
qa, meta = load()["qa"], load()["meta"]
assert qa.rows_parsed == GOLDEN["rows_parsed"]
assert qa.columns == GOLDEN["columns"]
assert meta.rows_expected - meta.duplicates_skipped == meta.rows_exported
assert meta.rows_exported == qa.rows_parsed
assert qa.row_accounting_ok
def test_state_machines_never_cross():
assert load()["qa"].crossover_violations == GOLDEN["crossover_violations"]
def test_campaign_counts():
qa, days = load()["qa"], load()["days"]
assert qa.distinct_campaigns == GOLDEN["distinct_campaigns"]
assert len(days) == GOLDEN["campaigns_scored"]
# --------------------------------------------------- overlapping exports
def test_reloading_the_same_export_changes_nothing():
"""Amazon's exports are date-range based, so overlap is the normal case."""
events = load()["events"]
merged, removed = dedupe_events(events + list(events))
assert removed == len(events)
assert len(merged) == len(events)
days = score_all(merged)
baseline = load()["days"]
assert sum(d.oob_min for d in days) == sum(d.oob_min for d in baseline)
# The real regression: without dedupe every repeated transition reads as a
# contradiction, burying the five genuine ones under thousands.
assert sum(len(d.chain_breaks) for d in days) == GOLDEN["chain_breaks"]
def test_dedupe_keeps_distinct_entities_that_share_a_minute():
"""One campaign paused 17 ad groups in the same minute. All 17 are real."""
events = load()["events"]
same_minute = [e for e in events
if e.campaign == "UCMANICUREKIT - CatchAll - AdGrp - Auto"
and e.change_type == "Ad group status" and e.minute == 111]
assert len(same_minute) == 17
assert len({e.level_name for e in same_minute}) == 17
kept, removed = dedupe_events(same_minute)
assert removed == 0, "distinct ad groups must never be merged"
assert len({event_identity(e) for e in events}) == len(events)
def test_row_accounting_explains_unscoreable_rows():
"""Account- and portfolio-level rows have no campaign to attach to. They
are dropped on purpose, so they must reconcile rather than read as loss."""
from ppcbudget.ingest import QaReport, WorkbookMeta
meta = WorkbookMeta(path=Path("x.xlsx"), rows_expected=5121,
duplicates_skipped=159, rows_exported=4962)
qa = QaReport(path=Path("x.xlsx"), meta=meta, rows_parsed=4788, rows_no_campaign=174)
assert qa.rows_seen == 4962
assert qa.row_accounting_ok, "dropped-on-purpose rows must not read as a failure"
assert "no campaign" in qa.accounting_detail
# A genuine shortfall must still fail, and say so.
broken = QaReport(path=Path("x.xlsx"), meta=meta, rows_parsed=4788)
assert not broken.row_accounting_ok
assert "UNACCOUNTED" in broken.accounting_detail
# ------------------------------------------------------- last meaningful action
def test_amazon_pacing_rows_are_not_actions():
"""The whole feature hinges on this: 2,639 of 2,989 'Campaign status' rows
are Amazon shutting a campaign off, not a person optimising it."""
from ppcbudget import actions
events = load()["events"]
status = [e for e in events if e.change_type == "Campaign status"]
system = [e for e in status if actions.classify(e) is None]
human = [e for e in status if actions.classify(e) == "status"]
assert len(status) == 2989
assert len(system) == 2639, "budget-state rows must never count as an action"
assert len(human) == 350, "delivery pause/resume is a person"
def test_every_change_type_is_classified_or_deliberately_system():
"""A new change type must not silently vanish from the action tracker."""
from ppcbudget import actions
unclassified = {
e.change_type for e in load()["events"]
if actions.classify(e) is None and e.change_type != "Campaign status"
}
assert unclassified == set(), f"unclassified change types: {sorted(unclassified)[:5]}"
def test_action_window_never_overstates_the_data():
"""One day of data may only ever claim one day."""
from ppcbudget import actions
days = load()["days"]
acts = actions.build(load()["events"], ["2026-08-05"], {d.campaign for d in days})
assert len(acts) == GOLDEN["campaigns_scored"]
stale = [a for a in acts.values() if a.untouched]
assert len(stale) == GOLDEN["untouched_campaigns"]
assert stale[0].summary == "No action in 1 day"
assert all(a.window_days == 1 for a in acts.values())
def test_renames_do_not_count_as_optimisation():
from ppcbudget import actions
class _R:
change_type = "Campaign name changed"
from_val = to_val = ""
assert actions.classify(_R()) == "cosmetic"
def test_action_categories_cover_the_asked_for_list():
"""Budget, placement, bid, strategy and targeting all resolve distinctly."""
from ppcbudget import actions
by_cat = {}
for e in load()["events"]:
c = actions.classify(e)
if c and c != "cosmetic":
by_cat.setdefault(c, 0)
by_cat[c] += 1
for expected in ("budget", "placement", "bid", "strategy", "targeting", "status"):
assert by_cat.get(expected, 0) > 0, f"no rows classified as {expected}"
# ----------------------------------------------------------------- scoring
def test_chain_breaks_canary():
"""If this fails, the intra-minute sort order has regressed to 19 breaks."""
days = load()["days"]
assert sum(len(d.chain_breaks) for d in days) == GOLDEN["chain_breaks"]
def test_out_of_budget_totals():
days = load()["days"]
assert sum(d.oob_min for d in days) == GOLDEN["oob_min"]
assert sum(d.oob_min_raw for d in days) == GOLDEN["oob_min_raw"]
assert sum(d.paused_min for d in days) == GOLDEN["paused_min"]
def test_ordering_matters():
"""A wrong intra-minute order makes the machine less coherent, never more."""
from ppcbudget import scoring
events = load()["events"]
naive = sorted(
[e for e in events
if e.machine == "budget" and e.campaign == "UBMICROFIBER - GREY (HSA)"],
key=lambda e: (e.minute, e.source_index),
)
correct = sorted(naive, key=scoring.sort_key)
assert naive != correct, "reference file has no equal-minute ties to reverse"
def breaks(evs):
state, n = evs[0].from_val, 0
for e in evs:
n += e.from_val != state
state = e.to_val
return n
assert breaks(correct) < breaks(naive)
def test_microfiber_fixture():
"""Hand-checkable: filter column C in the export and add the intervals up."""
day = next(d for d in load()["days"] if d.campaign == "UBMICROFIBER - GREY (HSA)")
assert day.oob_min == GOLDEN["microfiber_oob_min"]
assert day.episodes_raw == GOLDEN["microfiber_raw_episodes"]
assert day.episodes_merged < day.episodes_raw # zero-length recoveries collapse
def test_worst_campaign():
days = load()["days"]
assert max(days, key=lambda d: d.oob_min).campaign == GOLDEN["worst_campaign"]
def test_invariants_hold():
assert check_invariants(load()["days"]) == []
def test_minutes_tile_the_day():
for d in load()["days"]:
assert d.in_min + d.oob_min + d.paused_min + d.na_min == 1440
assert sum(d.hourly_oob) == d.oob_min
assert sum(e.active_min for e in d.episodes) == d.oob_min
# ----------------------------------------------------------------- metrics
def test_totals():
t = load()["totals"]
for key in ("ended_oob", "opened_oob", "at_least_1h", "over_12h",
"flapping_3plus", "priced", "capped", "partial_day"):
assert getattr(t, key) == GOLDEN[key], f"{key}: {getattr(t, key)} != {GOLDEN[key]}"
assert round(t.lost_spend, 2) == GOLDEN["lost_spend"]
assert sum(len(d.episodes) for d in load()["days"]) == GOLDEN["episodes"]
def test_money_model_stays_plausible():
t, meta = load()["totals"], load()["meta"]
assert t.lost_spend < meta.spend, "modelled loss exceeds actual account spend"
def test_unknown_budgets_are_never_zero():
for d in load()["days"]:
if d.budget.source == "unknown":
assert d.lost is None, f"{d.campaign} priced without an observed budget"
# ------------------------------------------------------------- edge cases
class _E:
"""Minimal stand-in for an ingest.Event."""
def __init__(self, minute, from_val, to_val, machine="budget", source_index=0):
self.minute, self.from_val, self.to_val = minute, from_val, to_val
self.machine, self.source_index = machine, source_index
self.from_num = self.to_num = None
self.campaign, self.date_key, self.change_type = "C", "2026-08-05", ""
def _score(events, **kw):
return score_campaign_day("C", "2026-08-05", events, **kw)
def test_single_event():
d = _score([_E(600, "In budget", "Out of budget")])
assert d.in_min == 600 and d.oob_min == 840 and d.episodes_raw == 1
def test_entirely_out_of_budget():
d = _score([_E(0, "Out of budget", "Out of budget")])
assert d.oob_min == 1440 and d.in_min == 0 and d.opened_oob and d.closed_oob
def test_entirely_in_budget():
d = _score([_E(720, "In budget", "In budget")])
assert d.in_min == 1440 and d.oob_min == 0 and d.first_oob_min is None
def test_pause_swallows_out_of_budget():
"""An outage fully inside a pause must contribute zero loss."""
d = _score([
_E(600, "In budget", "Out of budget"),
_E(700, "Out of budget", "In budget"),
_E(500, "Delivering", "Paused", machine="delivery"),
_E(800, "Paused", "Delivering", machine="delivery"),
])
assert d.oob_min == 0, "paused minutes must not count as lost"
assert d.oob_min_raw == 100, "the budget machine still saw the outage"
assert d.paused_min == 300
def test_partial_pause_overlap():
d = _score([
_E(600, "In budget", "Out of budget"),
_E(800, "Out of budget", "In budget"),
_E(700, "Delivering", "Paused", machine="delivery"),
_E(750, "Paused", "Delivering", machine="delivery"),
])
assert d.oob_min == 150 and d.oob_min_raw == 200
def test_created_mid_day_shortens_the_window():
d = _score([
_E(1438, "In budget", "Out of budget"),
_E(1400, "", "", machine="created"),
])
assert d.t0 == 1400 and d.na_min == 1400
assert d.eligible_min == 40 and d.oob_min == 2
def test_chain_break_repairs_at_midpoint():
d = _score([
_E(100, "In budget", "Out of budget"),
_E(300, "In budget", "Out of budget"), # break: running state is Out
])
assert len(d.chain_breaks) == 1
assert d.chain_breaks[0].ambiguity_min == 200
assert d.oob_uncertainty_min == 100
assert d.confidence == "repaired"
assert d.oob_min == 100 + 1140 # 100..200 out, 200..300 in, 300..1440 out
def test_equal_minute_events_use_source_order():
"""Later source index = earlier in time, because the file is newest-first."""
d = _score([
_E(600, "Out of budget", "In budget", source_index=1),
_E(600, "In budget", "Out of budget", source_index=0),
])
assert d.chain_breaks == [], "reversing equal-minute rows should yield a clean chain"
def test_no_budget_events_is_unscorable():
assert _score([_E(600, "Delivering", "Paused", machine="delivery")]) is None
def test_truncated_export_is_not_scored_as_in_budget():
events = [_E(100, "In budget", "Out of budget")]
d = _score(events, day_end_min=600)
assert d.t1 == 600 and d.na_min == 840 and d.oob_min == 500
def test_merge_gap_collapses_flapping():
events = [
_E(100, "In budget", "Out of budget"),
_E(200, "Out of budget", "In budget"),
_E(202, "In budget", "Out of budget"), # 2-minute blip back in budget
_E(300, "Out of budget", "In budget"),
]
d = _score(events)
assert d.episodes_raw == 2 and d.episodes_merged == 1
def test_budget_change_is_time_weighted():
events = [
_E(720, "In budget", "Out of budget"),
_E(720, "$50.00", "$100.00", machine="budget_amount"),
]
events[1].from_num, events[1].to_num = 50.0, 100.0
d = _score(events)
assert d.budget.source == "daily_budget_event"
assert d.budget.time_weighted == 75.0 # half a day at each value
# --------------------------------------------------------------- aggregate
def test_rollup_single_day():
days = load()["days"]
rolls = rollup(days)
assert len(rolls) == len(days)
assert all(r.days_observed == 1 for r in rolls)
assert all(r.trend_slope == 0.0 for r in rolls), "one point cannot have a trend"
def test_chronic_score_prefers_persistence():
def day(date_key, oob_hours):
d = CampaignDay("C", date_key, 0, 1440)
d.oob_min = int(oob_hours * 60)
d.episodes = [Episode(1, 0, d.oob_min, d.oob_min, d.oob_min)]
d.episodes_merged = 1
return d
persistent = rollup([day(f"2026-08-{i:02d}", 8) for i in range(1, 8)])[0]
one_spike = rollup([day("2026-08-01", 23)]
+ [day(f"2026-08-{i:02d}", 0) for i in range(2, 8)])[0]
assert persistent.chronic_score > one_spike.chronic_score
assert persistent.streak_max == 7 and one_spike.streak_max == 1
def _main() -> int:
tests = [(n, f) for n, f in sorted(globals().items())
if n.startswith("test_") and callable(f)]
failed = []
for name, fn in tests:
try:
fn()
print(f" pass {name}")
except AssertionError as exc:
failed.append((name, exc))
print(f" FAIL {name}: {exc}")
except Exception as exc: # noqa: BLE001
failed.append((name, exc))
print(f" ERROR {name}: {type(exc).__name__}: {exc}")
print(f"\n{len(tests) - len(failed)}/{len(tests)} passed")
return 1 if failed else 0
if __name__ == "__main__":
raise SystemExit(_main())

784
web/app.js Normal file
View File

@ -0,0 +1,784 @@
'use strict';
const $ = (id) => document.getElementById(id);
const ROW_H = 34;
const state = {
data: null,
view: [],
sort: { key: 'sv', dir: -1 },
search: '',
diagnoses: new Set(),
pricedOnly: false,
staleOnly: false,
mode: 'campaign', // 'campaign' (one row per campaign) | 'day'
settings: { roas: '', haircut: 0.7, cap: 3, merge_gap: 5 },
files: [],
};
const nf = new Intl.NumberFormat('en-US');
const nf1 = new Intl.NumberFormat('en-US', { minimumFractionDigits: 1, maximumFractionDigits: 1 });
const nf2 = new Intl.NumberFormat('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 });
const money = new Intl.NumberFormat('en-US', { style: 'currency', currency: 'USD', maximumFractionDigits: 0 });
const money2 = new Intl.NumberFormat('en-US', { style: 'currency', currency: 'USD', minimumFractionDigits: 2 });
const pct = (v) => (v * 100).toFixed(1) + '%';
const DX_CLASS = {
'Structurally underfunded': 'dx-under',
'Exhausts early': 'dx-early',
'Pacing thrash': 'dx-thrash',
'Evening cap': 'dx-evening',
'Intermittent': 'dx-inter',
'Healthy': 'dx-healthy',
'Mostly paused': 'dx-paused',
};
function stage(name) {
for (const s of ['upload', 'loading', 'error', 'dash']) $('stage-' + s).hidden = s !== name;
const showing = name === 'dash';
for (const b of ['btn-csv', 'btn-xlsx', 'btn-reset', 'btn-settings']) $(b).hidden = !showing;
}
function fail(message) {
$('error-text').textContent = message;
stage('error');
}
// ------------------------------------------------------------------ uploads
function renderFileList() {
const ul = $('filelist');
ul.innerHTML = '';
for (const f of state.files) {
const li = document.createElement('li');
li.innerHTML = `<span class="tag">${f.kind === 'perf' ? 'performance' : 'history'}</span>
<span>${escapeHtml(f.name)}</span><span class="ok">ready</span>`;
ul.appendChild(li);
}
$('btn-analyze').hidden = !state.files.some((f) => f.kind === 'history');
}
async function upload(file, kind) {
const body = await file.arrayBuffer();
const res = await fetch('/api/upload', {
method: 'POST',
headers: { 'X-Filename': encodeURIComponent(file.name).replace(/%20/g, ' '), 'X-Kind': kind },
body,
});
if (!res.ok) throw new Error((await res.json().catch(() => ({}))).error || 'Upload failed');
state.files.push({ name: file.name, kind });
renderFileList();
}
async function acceptFiles(list, kind) {
const files = [...list].filter((f) => /\.(xlsx|xlsm|csv|tsv)$/i.test(f.name) && !f.name.startsWith('~$'));
if (!files.length) {
fail('Those files are not Excel or CSV exports. Look for amazon-ads-history_*.xlsx.');
return;
}
try {
for (const f of files) await upload(f, kind);
} catch (err) {
fail(String(err.message || err));
}
}
// ----------------------------------------------------------------- analysis
async function analyze() {
stage('loading');
const steps = ['Reading the export…', 'Reconstructing budget timelines…',
'Measuring outages…', 'Pricing lost opportunity…'];
let i = 0;
const tick = setInterval(() => { $('loading-text').textContent = steps[++i % steps.length]; }, 900);
try {
const res = await fetch('/api/analyze', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(state.settings),
});
const json = await res.json();
if (!res.ok) throw new Error(json.error || 'Analysis failed');
state.data = json;
state.diagnoses.clear();
// Show the panel before rendering: the row virtualiser measures the table's
// height, and a hidden element measures zero.
stage('dash');
render();
} catch (err) {
fail(String(err.message || err));
} finally {
clearInterval(tick);
}
}
// ------------------------------------------------------------------- render
function render() {
const d = state.data;
const m = d.meta, t = d.totals;
const span = m.dates.length > 1 ? `${m.dates[0]} to ${m.dates.at(-1)}` : m.dates[0];
$('subtitle').textContent =
`${m.account} · ${m.marketplace} · ${span} · ${m.files.length} export(s)`;
// Fold the per-campaign action record onto every row so the existing sort
// and filter machinery treats it like any other column.
const acts = d.actions || {};
const blank = { sum: 'not observed', ds: null, unt: false, n: 0, label: '' };
for (const row of d.campaigns) Object.assign(row, { act: acts[row.c] || blank });
for (const row of d.recurring) Object.assign(row, { act: acts[row.c] || blank });
for (const row of [...d.campaigns, ...d.recurring]) {
row.ds = row.act.unt ? Infinity : row.act.ds; // untouched sorts to the top
row.unt = row.act.unt;
}
$('grain').hidden = t.days < 2;
renderAnswer(t, m);
renderKpis(t, m);
renderCurve(d.curve, t);
renderReality(t, m);
renderChips();
renderQuality(d.quality, d.invariants);
applyFilters();
}
/** Durations read as "45min" / "2h 31min", never as decimal hours. */
function hrs(v) {
if (v == null) return '—';
const total = Math.round(v * 60);
const h = Math.floor(total / 60), m = total % 60;
if (h === 0) return `${m}min`;
return m ? `${h}h ${m}min` : `${h}h`;
}
function renderAnswer(t, m) {
const a = t.avg_day;
const day = t.days > 1 ? 'day' : `day (${m.dates[0]})`;
$('answer-lede').innerHTML =
`On an average ${day}, one of your campaigns spends <b class="run">${hrs(a.running)}</b> ` +
`able to run &mdash; and <b class="out">${hrs(a.out)}</b> shut off because it hit its daily budget.` +
(a.paused > 0.05 ? ` A further ${hrs(a.paused)} it was paused, which costs nothing.` : '');
const segs = [
['running', a.running, '#16a34a', 'Running'],
['out', a.out, '#dc2626', 'Out of budget'],
['paused', a.paused, '#9ca3af', 'Paused'],
['na', a.na, '#e5e7eb', 'Not yet created'],
].filter(([, v]) => v > 0.01);
$('daybar').innerHTML = segs.map(([, v, color, label]) =>
`<span style="width:${(v / 24) * 100}%;background:${color}"
title="${label}: ${hrs(v)}">${(v / 24) > 0.13 ? hrs(v) : ''}</span>`).join('');
$('daykeys').innerHTML = segs.map(([, v, color, label]) =>
`<div><i class="sw" style="background:${color}"></i>${label} <b>${hrs(v)}</b></div>`).join('');
const perDay = t.per_day;
const priced = t.priced < t.campaigns
? ` Priced across only the ${nf.format(t.priced)} campaigns whose budget appears in the export, that is ` +
`${money.format(perDay.lost_spend)} of spend you could not place per day` +
` — the true figure is higher, since ${nf.format(t.campaigns - t.priced)} campaigns have no budget to price against.`
: ` That works out at ${money.format(perDay.lost_spend)} of spend you could not place per day.`;
$('answer-account').innerHTML =
`Across all ${nf.format(t.distinct)} campaigns that is <b>${nf1.format(perDay.out_hours)} campaign-hours ` +
`of lost opportunity every single day</b>.${priced}`;
}
function kpi(label, value, note, alarm) {
return `<div class="kpi${alarm ? ' alarm' : ''}">
<div class="label">${label}</div>
<div class="value">${value}</div>
<div class="note">${note}</div>
</div>`;
}
function renderKpis(t, m) {
const unit = t.days > 1 ? 'campaign-days' : 'campaigns';
$('kpis').innerHTML = [
kpi('Campaigns scored', nf.format(t.distinct),
t.days > 1 ? `${nf.format(t.campaigns)} campaign-days over ${t.days} days`
: 'had budget-state changes'),
kpi('Lost hours per day', nf1.format(t.per_day.out_hours),
'campaign-hours shut off, account-wide', true),
kpi('Average campaign runs', hrs(t.avg_day.running), `of 24 h — then it hits its budget`),
kpi('Lose over 12 h a day', nf.format(t.over_12h), `${unit} more than half the day dark`, true),
kpi('Ended the day out', nf.format(t.ended_oob),
t.campaigns ? `${Math.round(100 * t.ended_oob / t.campaigns)}% of ${unit}` : '', true),
kpi('Repeat outages', nf.format(t.flapping), `${unit} with 3 or more outages`),
kpi('Lost spend per day', money.format(t.per_day.lost_spend),
`only ${nf.format(t.priced)} of ${nf.format(t.campaigns)} ${unit} priced`),
kpi('Lost sales per day', money.format(t.per_day.lost_sales),
`ROAS ${m.roas.toFixed(2)} × ${Math.round(m.haircut * 100)}% haircut`),
kpi('No action taken', nf.format((state.data.action_summary || {}).untouched || 0),
`campaigns untouched across all ${t.days} day(s)`, true),
].join('');
}
function renderCurve(curve, t) {
const max = Math.max(...curve, 1);
const peak = curve.indexOf(Math.max(...curve.slice(1)));
$('curve-sub').textContent =
`Share of scored campaigns out of budget during each hour. Budgets reset at midnight, then ` +
`coverage decays as campaigns exhaust their cap — peaking at ${nf1.format(Math.max(...curve.slice(1)))}% around ${String(peak).padStart(2, '0')}:00.`;
$('curve').innerHTML = curve.map((v, h) => `
<div class="bar" title="${String(h).padStart(2, '0')}:00 — ${v}% of campaigns out of budget">
<span class="pct">${v >= 10 ? Math.round(v) : ''}</span>
<div class="fill" style="height:${(v / max) * 100}%"></div>
<span class="hour">${String(h).padStart(2, '0')}</span>
</div>`).join('');
}
function renderReality(t, m) {
const share = m.actual_spend ? t.lost_spend / m.actual_spend : null;
$('reality').innerHTML = `
<div><div class="k">Modelled lost spend</div><div class="v">${money2.format(t.lost_spend)}</div></div>
<div><div class="k">Actual account spend</div><div class="v">${money2.format(m.actual_spend)}</div></div>
<div><div class="k">Lost as share of actual</div><div class="v">${share === null ? '' : pct(share)}</div></div>
<div><div class="k">Guardrails</div><div class="v" style="font-size:13px;font-weight:500">
${t.capped} hit the ${m.cap_multiple}× budget cap, ${t.unreliable} unpriced for too little in-budget time
</div></div>`;
}
function renderChips() {
const counts = {};
const src = (state.mode === 'campaign' && state.data.totals.days > 1)
? state.data.recurring : state.data.campaigns;
for (const c of src) counts[c.dx] = (counts[c.dx] || 0) + 1;
const order = ['Structurally underfunded', 'Exhausts early', 'Pacing thrash', 'Evening cap',
'Intermittent', 'Healthy', 'Mostly paused'];
const names = state.data.diagnoses.slice().sort((a, b) => order.indexOf(a) - order.indexOf(b));
$('chips').innerHTML = names.map((n) => `
<button class="chip" data-dx="${escapeHtml(n)}" aria-pressed="${state.diagnoses.has(n)}">
${escapeHtml(n)}<span class="n">${counts[n] || 0}</span>
</button>`).join('');
}
function renderQuality(checks, invariants) {
const rows = checks.map((c) => `
<div class="qrow q-${c.status}">
<span class="badge">${c.status === 'ok' ? 'OK' : c.status === 'review' ? 'Review' : 'Fail'}</span>
<div><strong>${escapeHtml(c.name)}</strong></div>
<div class="qval">${escapeHtml(c.value)}</div>
<div class="qnote">${escapeHtml(c.note)}</div>
</div>`);
const inv = invariants && invariants.failed.length
? `<div class="qrow q-fail"><span class="badge">Fail</span>
<div><strong>Internal consistency</strong></div>
<div class="qval">${invariants.failed.length} failed</div>
<div class="qnote">${escapeHtml(invariants.failed.join(' | '))}</div></div>`
: `<div class="qrow q-ok"><span class="badge">OK</span>
<div><strong>Internal consistency</strong></div>
<div class="qval">${nf.format(invariants ? invariants.checked : 0)} campaigns</div>
<div class="qnote">For every campaign the minutes in budget, out of budget, paused and
not-yet-created sum to exactly 1440, the episode durations sum to the out-of-budget total,
and the hourly buckets agree with both so the chart and the table cannot tell
different stories.</div></div>`;
$('quality').innerHTML = rows.join('') + inv;
}
// -------------------------------------------------------------------- table
const BASE_COLUMNS = [
{ key: 'c', label: 'Campaign', cls: 'name' },
{ key: 'ib', label: 'Runs h/day', cls: 'num' },
{ key: 'ob', label: 'Lost h/day', cls: 'num' },
{ key: 'sh', label: '% day lost', cls: 'num' },
{ key: 'em', label: 'Outages', cls: 'num' },
{ key: 'f', label: '1st out', cls: 'num' },
{ key: null, label: 'Timeline 0→24h', cls: 'strip-h' },
{ key: 'ls', label: 'Lost spend', cls: 'num' },
{ key: 'sv', label: 'Severity', cls: 'num' },
{ key: 'dx', label: 'Diagnosis', cls: 'dx' },
{ key: 'ds', label: 'Last action', cls: 'act' },
];
// One row per campaign, averaged across the days loaded. This is the default
// once there is more than one day: eight rows of the same campaign is noise.
const GROUP_COLUMNS = [
{ key: 'c', label: 'Campaign', cls: 'name' },
{ key: 'out', label: 'Days out', cls: 'num' },
{ key: 'runs', label: 'Runs h/day', cls: 'num' },
{ key: 'mean', label: 'Lost h/day', cls: 'num' },
{ key: 'max', label: 'Worst day', cls: 'num' },
{ key: 'eps', label: 'Outages', cls: 'num' },
{ key: null, label: 'Lost h by day', cls: 'strip-h' },
{ key: 'slope', label: 'Trend', cls: 'trendcell' },
{ key: 'lostd', label: 'Lost $/day', cls: 'num' },
{ key: 'score', label: 'Chronic', cls: 'num' },
{ key: 'dx', label: 'Diagnosis', cls: 'dx' },
{ key: 'ds', label: 'Last action', cls: 'act' },
];
let COLUMNS = BASE_COLUMNS;
function setColumns(mode, multi) {
const wrap = document.querySelector('.tablewrap');
if (mode === 'campaign' && multi) {
COLUMNS = GROUP_COLUMNS;
} else {
COLUMNS = multi
? [BASE_COLUMNS[0], { key: 'd', label: 'Date', cls: 'date' }, ...BASE_COLUMNS.slice(1)]
: BASE_COLUMNS;
}
wrap.classList.toggle('grouped', mode === 'campaign' && multi);
wrap.classList.toggle('multi', mode === 'day' && multi);
}
/** Colour for a day, on the same green-to-red scale as the hour heatmap. */
function heatColor(lostHours, eligibleHours) {
const f = eligibleHours > 0 ? Math.min(1, lostHours / eligibleHours) : 0;
if (f <= 0.005) return '#16a34a';
const ramp = ['#fff9c4', '#ffecb3', '#ffe0b2', '#ffccbc', '#ffab91',
'#ff8a65', '#ef5350', '#dc2626', '#b71c1c'];
return ramp[Math.min(ramp.length - 1, Math.floor(f * ramp.length))];
}
function dayHeat(series, dates) {
return `<div class="dayheat">${series.map((v, i) =>
`<i style="background:${heatColor(v, 24)}" title="${dates[i]}: ${hrs(v)} lost"></i>`).join('')}</div>`;
}
function renderHead() {
$('thead').innerHTML = COLUMNS.map((c, i) => {
const sorted = c.key && state.sort.key === c.key;
const arrow = sorted ? (state.sort.dir === -1 ? ' ↓' : ' ↑') : '';
return `<div class="${c.cls === 'num' ? 'num' : ''}${sorted ? ' sorted' : ''}"
data-col="${i}">${c.label}${arrow}</div>`;
}).join('');
}
function applyFilters() {
const q = state.search.toLowerCase();
const days = state.data.totals.days;
const grouped = state.mode === 'campaign' && days > 1;
const source = grouped ? state.data.recurring : state.data.campaigns;
const lostKey = grouped ? 'lostd' : 'ls';
state.view = source.filter((c) => {
if (q && !c.c.toLowerCase().includes(q)) return false;
if (state.diagnoses.size && !state.diagnoses.has(c.dx)) return false;
if (state.pricedOnly && c[lostKey] == null) return false;
if (state.staleOnly && !c.unt) return false;
return true;
});
// Sort key may not exist in the other grain; fall back to its default.
let { key, dir } = state.sort;
if (!COLUMNS.some((col) => col.key === key)) {
key = grouped ? 'score' : 'sv';
dir = -1;
state.sort = { key, dir };
}
state.view.sort((a, b) => {
const x = a[key], y = b[key];
if (x == null && y == null) return 0;
if (x == null) return 1; // unpriced/unknown always sink
if (y == null) return -1;
if (typeof x === 'string') return dir * x.localeCompare(y);
return dir * (x - y);
});
const total = source.length;
const hours = state.view.reduce((s, c) => s + (grouped ? c.tot : c.ob), 0);
const noun = grouped ? 'campaigns' : (days > 1 ? 'campaign-days' : 'campaigns');
$('table-title').textContent = state.view.length === total
? `All ${nf.format(total)} ${noun}` + (grouped ? ` across ${days} days` : '')
: `${nf.format(state.view.length)} of ${nf.format(total)} ${noun}`;
$('table-sub').innerHTML = grouped
? `One row per campaign, averaged over ${days} days. <b>Runs h/day</b> is how long it could ` +
`actually spend; <b>Lost h/day</b> is how long it sat shut off after hitting its budget. ` +
`The strip shows one cell per day, newest right. This selection loses ` +
`${nf1.format(hours)} campaign-hours in total. <b>Click any row for the day-by-day breakdown.</b>`
: `<b>Runs h/day</b> is how long the campaign could actually spend; <b>Lost h/day</b> is how long ` +
`it sat shut off after hitting its budget. Together with paused time they make up the 24-hour day. ` +
`This selection loses ${nf1.format(hours)} campaign-hours. Click any row for its timeline and outages.`;
setColumns(state.mode, days > 1);
renderHead();
$('spacer').style.height = (state.view.length * ROW_H) + 'px';
$('tbody').scrollTop = 0;
drawRows(true);
}
let lastWindow = '';
function drawRows(force) {
const body = $('tbody'), rows = $('rows');
if (!state.view.length) {
rows.innerHTML = '<div class="empty">No campaigns match those filters.</div>';
lastWindow = '';
return;
}
const multiDay = state.data.totals.days > 1;
const top = body.scrollTop;
// Fall back to a sensible window if the panel has not been laid out yet.
const viewportH = body.clientHeight || 620;
const first = Math.max(0, Math.floor(top / ROW_H) - 6);
const last = Math.min(state.view.length, Math.ceil((top + viewportH) / ROW_H) + 6);
// Scrolling within the already-rendered window needs no DOM work.
const key = first + ':' + last;
if (!force && key === lastWindow) return;
lastWindow = key;
const grouped = state.mode === 'campaign' && multiDay;
let html = '';
for (let i = first; i < last; i++) {
const c = state.view[i];
html += `<div class="row" style="top:${i * ROW_H}px" data-i="${i}">`
+ (grouped ? groupCells(c) : dayCells(c, multiDay))
+ '</div>';
}
rows.innerHTML = html;
}
function dayCells(c, multiDay) {
const lost = c.ls == null
? `<span class="unpriced">no budget</span>`
: money.format(c.ls) + (c.cap ? ' *' : '');
return `
<div class="name" title="${escapeHtml(c.c)}">${escapeHtml(c.c)}</div>
${multiDay ? `<div class="date">${c.d}</div>` : ''}
<div class="num run dur">${hrs(c.ib)}</div>
<div class="num out dur">${hrs(c.ob)}</div>
<div class="num">${Math.round(c.sh * 100)}%</div>
<div class="num">${c.em}${c.er !== c.em ? `<span class="muted"> /${c.er}</span>` : ''}</div>
<div class="num">${c.f || '—'}</div>
<div><div class="strip" style="background-image:${c.g}"></div></div>
<div class="num">${lost}</div>
<div class="num">${nf1.format(c.sv)}</div>
<div class="dx"><span class="pill ${DX_CLASS[c.dx] || ''}">${escapeHtml(c.dx)}</span></div>
<div class="act">${c.unt ? `<span class="stale">${escapeHtml(c.act.sum)}</span>` : `<span class="fresh" title="${escapeHtml(c.act.label || '')}">${escapeHtml(c.act.sum)}</span>`}</div>`;
}
function groupCells(r) {
const lost = r.lostd == null
? `<span class="unpriced">no budget</span>`
: money.format(r.lostd);
const trendCls = r.trend === 'worsening' ? 'trend-worse'
: r.trend === 'improving' ? 'trend-better' : 'muted';
return `
<div class="name" title="${escapeHtml(r.c)}">${escapeHtml(r.c)}</div>
<div class="num">${r.out}<span class="muted">/${r.obs}</span></div>
<div class="num run dur">${hrs(r.runs)}</div>
<div class="num out dur">${hrs(r.mean)}</div>
<div class="num dur">${hrs(r.max)}</div>
<div class="num">${r.eps}</div>
<div>${dayHeat(r.series, r.dates)}</div>
<div class="trendcell ${trendCls}">${r.trend}</div>
<div class="num">${lost}</div>
<div class="num">${nf1.format(r.score)}</div>
<div class="dx"><span class="pill ${DX_CLASS[r.dx] || ''}">${escapeHtml(r.dx)}</span></div>
<div class="act">${r.unt ? `<span class="stale">${escapeHtml(r.act.sum)}</span>` : `<span class="fresh" title="${escapeHtml(r.act.label || '')}">${escapeHtml(r.act.sum)}</span>`}</div>`;
}
// ------------------------------------------------------------------- drawer
const ACT_LABEL = {
budget: 'Budget', placement: 'Placement', strategy: 'Strategy', bid: 'Bid',
targeting: 'Targeting', structure: 'Structure', status: 'Status', portfolio: 'Portfolio',
};
/** "What has anyone actually done to this campaign?" -- shown in the drawer. */
function actionSection(name) {
const a = (state.data.actions || {})[name];
if (!a) return '';
const win = a.win || state.data.totals.days;
const plural = win === 1 ? 'day' : 'days';
if (a.unt) {
return `<h3>Last action</h3>
<div class="act-none">
<strong>No action taken in the entire ${win}-${plural} analysis period.</strong>
<p>No budget, bid, placement, bidding-strategy, targeting or status change was
recorded for this campaign between ${state.data.meta.dates[0]} and
${state.data.meta.dates.at(-1)}. Amazon's own out-of-budget switching is not counted
as an action that is the pacing engine, not a person.</p>
</div>`;
}
const rows = (a.recent || []).map((r) => `
<tr>
<td style="text-align:left;white-space:nowrap">${escapeHtml(r[0])}</td>
<td style="text-align:left"><span class="pill act-${escapeHtml(r[1])}">${ACT_LABEL[r[1]] || r[1]}</span></td>
<td style="text-align:left">${escapeHtml(r[2])}</td>
</tr>`).join('');
return `<h3>Last action</h3>
<div class="act-yes">
<strong>${escapeHtml(a.sum)}</strong>
<p>${escapeHtml(a.label || '')} &mdash; ${escapeHtml(a.at || '')}.
${a.n} change${a.n === 1 ? '' : 's'} in the ${win}-${plural} window across
${(a.cats || []).map((c) => ACT_LABEL[c] || c).join(', ') || 'no categories'}.</p>
</div>
${rows ? `<table class="grid act-log">
<thead><tr><th style="text-align:left">When</th><th style="text-align:left">Type</th>
<th style="text-align:left">What changed</th></tr></thead>
<tbody>${rows}</tbody></table>
${a.n > (a.recent || []).length
? `<p class="sub" style="margin-top:6px">Showing the ${a.recent.length} most recent of ${a.n} changes.</p>` : ''}` : ''}`;
}
/** Day-by-day view of one campaign: the answer to "what happened each day?" */
function openCampaignDrawer(r) {
const m = state.data.meta;
const days = state.data.campaigns
.filter((c) => c.c === r.c)
.sort((a, b) => a.d.localeCompare(b.d));
$('drawer-title').textContent = r.c;
const ract = (state.data.actions || {})[r.c];
$('drawer-sub').innerHTML =
`${r.obs} days · ${escapeHtml(r.dx)} · ran out on ${r.out} of ${r.obs} days · trend ${r.trend}`
+ (ract ? ` · <b class="${ract.unt ? 'act-stale-text' : ''}">${escapeHtml(ract.sum)}</b>` : '');
const cell = (k, v) => `<div><div class="k">${k}</div><div class="v">${v}</div></div>`;
const worst = days.find((d) => d.d === r.wd);
$('drawer-body').innerHTML = `
<div class="dgrid">
${cell('Runs per day', `<span style="color:var(--green)">${hrs(r.runs)}</span>`)}
${cell('Lost per day', `<span style="color:var(--red)">${hrs(r.mean)}</span>`)}
${cell('Worst day', `${hrs(r.max)}<div class="k" style="margin-top:2px">${r.wd}</div>`)}
${cell('Days it ran out', `${r.out} of ${r.obs}`)}
${cell('Longest run of bad days', `${r.smax}`)}
${cell('Total lost', hrs(r.tot))}
${cell('Outages', r.eps)}
${cell('Chronic score', nf1.format(r.score))}
</div>
<h3>Day by day</h3>
<table class="grid">
<thead><tr>
<th>Date</th><th>Runs</th><th>Lost<br><small>billable</small></th><th>Paused</th><th>% lost</th>
<th>Outages</th><th>1st out</th><th>Timeline 0&rarr;24h</th>
</tr></thead>
<tbody>${days.map((d) => `
<tr${d.d === r.wd ? ' style="background:color-mix(in srgb,var(--red) 8%,transparent)"' : ''}>
<td style="text-align:left;white-space:nowrap">${d.d}</td>
<td class="dur" style="color:var(--green)">${hrs(d.ib)}</td>
<td class="dur" style="color:var(--red);font-weight:600">${hrs(d.ob)}</td>
<td class="dur muted">${d.pa > 0.005 ? hrs(d.pa) : '—'}</td>
<td>${Math.round(d.sh * 100)}%</td>
<td>${d.em}</td>
<td>${d.f || '—'}</td>
<td style="width:190px;padding-right:0">
<div class="strip" style="background-image:${d.g};margin:0;width:100%"></div>
</td>
</tr>`).join('')}</tbody>
</table>
<div class="axis" style="margin-left:auto;width:190px"><span>00</span><span>06</span><span>12</span><span>18</span><span>24</span></div>
<h3>Money</h3>
<div class="dgrid">
${cell('Lost spend / day', r.lostd == null ? '—' : money2.format(r.lostd))}
${cell('Lost spend total', r.lost == null ? '—' : money2.format(r.lost))}
${cell('Lost sales total', r.lsa == null ? '—' : money2.format(r.lsa))}
</div>
<p class="sub">${r.lostd == null
? 'No daily budget for this campaign appears in the export, so there is no honest way to price it. Add a performance report to fill this in.'
: `Priced from the budget observed in the export, at ROAS ${m.roas.toFixed(2)} with a ${Math.round(m.haircut * 100)}% haircut.`}</p>
${worst ? `<h3>Worst day in detail &mdash; ${r.wd}</h3>
<table class="grid">
<thead><tr><th>#</th><th>Start</th><th>End</th><th>Duration</th><th>Billable</th></tr></thead>
<tbody>${worst.eps.map((e) => `<tr>
<td style="text-align:left">${e.i}</td><td>${e.s}</td><td>${e.e}</td>
<td class="dur">${hrs(e.m / 60)}</td>
<td class="dur">${e.a === e.m ? '<span class="muted">same</span>' : hrs(e.a / 60)}</td>
</tr>`).join('')}</tbody>
</table>
<p class="sub" style="margin-top:8px">Duration is wall-clock. <b>Billable</b> excludes any
minutes the campaign was paused during the outage &mdash; a paused campaign forgoes nothing
to its budget, so only billable minutes count as lost. "Same" means it was never paused.</p>` : ''}
${actionSection(r.c)}`;
$('drawer').hidden = false;
$('scrim').hidden = false;
}
function openDrawer(c) {
const m = state.data.meta;
$('drawer-title').textContent = c.c;
const cact = (state.data.actions || {})[c.c];
$('drawer-sub').innerHTML =
`${c.d} · ${escapeHtml(c.dx)} · confidence: ${c.cf.replace('_', ' ')}` +
(c.un ? `${nf2.format(c.un)} h from a repaired gap)` : '') +
(cact ? ` · <b class="${cact.unt ? 'act-stale-text' : ''}">${escapeHtml(cact.sum)}</b>` : '');
const cell = (k, v) => `<div><div class="k">${k}</div><div class="v">${v}</div></div>`;
const budget = c.bg == null ? '—' : money2.format(c.bg);
const src = { daily_budget_event: 'from a budget change', budget_rule: 'from a budget rule',
perf_report: 'from the performance report', unknown: 'not in the export' }[c.bs];
$('drawer-body').innerHTML = `
<div class="dgrid">
${cell('Runs per day', `<span style="color:var(--green)">${hrs(c.ib)}</span>`)}
${cell('Lost per day', `<span style="color:var(--red)">${hrs(c.ob)}</span>`)}
${cell('% of day lost', Math.round(c.sh * 100) + '%')}
${cell('Paused', hrs(c.pa))}
${cell('First ran out', c.f || '—')}
${cell('Recovered', c.l || (c.cl ? 'never' : '—'))}
${cell('Budget-cap hits', c.er)}
${cell('Distinct outages', c.em)}
</div>
<h3>Timeline</h3>
<div class="dstrip" style="background-image:${c.g}"></div>
<div class="axis"><span>00:00</span><span>06:00</span><span>12:00</span><span>18:00</span><span>24:00</span></div>
<h3>Money</h3>
<div class="dgrid">
${cell('Daily budget', budget)}
${cell('Spend rate', c.rt == null ? '—' : money2.format(c.rt) + '/h')}
${cell('Lost spend', c.ls == null ? '—' : money2.format(c.ls))}
${cell('Lost sales', c.lsa == null ? '—' : money2.format(c.lsa))}
</div>
<p class="sub">Budget ${src}.${c.cap ? ` Lost spend hit the ${m.cap_multiple}× cap, so the true figure could be higher — or demand simply was not there.` : ''}
${c.ls == null && c.bs === 'unknown' ? ' Without an observed budget there is no honest way to price this campaign, so nothing is shown rather than a zero. Add a performance report to fill it in.' : ''}</p>
<h3>Outages (${c.eps.length})</h3>
<table class="grid">
<thead><tr><th>#</th><th>Start</th><th>End</th><th>Duration</th><th>Billable</th></tr></thead>
<tbody>${c.eps.map((e) => `<tr>
<td style="text-align:left">${e.i}</td><td>${e.s}</td><td>${e.e}</td>
<td class="dur">${hrs(e.m / 60)}</td>
<td class="dur">${e.a === e.m ? '<span class="muted">same</span>' : hrs(e.a / 60)}</td>
</tr>`).join('')}</tbody>
</table>
<p class="sub" style="margin-top:8px">Duration is wall-clock. <b>Billable</b> excludes any
minutes the campaign was paused during the outage a paused campaign forgoes nothing to its
budget, so only billable minutes count as lost. "Same" means it was never paused.</p>
${actionSection(c.c)}`;
$('drawer').hidden = false;
$('scrim').hidden = false;
}
function closeDrawer() { $('drawer').hidden = true; $('scrim').hidden = true; }
// -------------------------------------------------------------------- utils
function escapeHtml(s) {
return String(s).replace(/[&<>"']/g, (ch) =>
({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' }[ch]));
}
// -------------------------------------------------------------------- wiring
const dz = $('dropzone');
['dragenter', 'dragover'].forEach((e) =>
dz.addEventListener(e, (ev) => { ev.preventDefault(); dz.classList.add('over'); }));
['dragleave', 'drop'].forEach((e) =>
dz.addEventListener(e, (ev) => { ev.preventDefault(); dz.classList.remove('over'); }));
dz.addEventListener('drop', (ev) => acceptFiles(ev.dataTransfer.files, 'history'));
dz.addEventListener('click', () => $('file-input').click());
dz.addEventListener('keydown', (ev) => {
if (ev.key === 'Enter' || ev.key === ' ') { ev.preventDefault(); $('file-input').click(); }
});
$('pick').addEventListener('click', (ev) => { ev.stopPropagation(); $('file-input').click(); });
$('file-input').addEventListener('change', (ev) => acceptFiles(ev.target.files, 'history'));
$('pick-perf').addEventListener('click', () => $('perf-input').click());
$('perf-input').addEventListener('change', (ev) => acceptFiles(ev.target.files, 'perf'));
window.addEventListener('dragover', (e) => e.preventDefault());
window.addEventListener('drop', (e) => e.preventDefault());
$('btn-analyze').addEventListener('click', analyze);
$('btn-error-back').addEventListener('click', () => stage(state.data ? 'dash' : 'upload'));
$('btn-reset').addEventListener('click', async () => {
await fetch('/api/clear', { method: 'POST' });
state.data = null; state.files = []; state.diagnoses.clear();
state.search = ''; $('search').value = ''; state.pricedOnly = false; $('only-priced').checked = false;
state.staleOnly = false; $('only-stale').checked = false;
renderFileList();
stage('upload');
});
$('btn-csv').addEventListener('click', () => { location.href = '/api/export?format=csv'; });
$('btn-xlsx').addEventListener('click', () => { location.href = '/api/export?format=xlsx'; });
$('search').addEventListener('input', (ev) => { state.search = ev.target.value; applyFilters(); });
$('only-priced').addEventListener('change', (ev) => { state.pricedOnly = ev.target.checked; applyFilters(); });
$('only-stale').addEventListener('change', (ev) => { state.staleOnly = ev.target.checked; applyFilters(); });
$('chips').addEventListener('click', (ev) => {
const chip = ev.target.closest('.chip');
if (!chip) return;
const dx = chip.dataset.dx;
const on = !state.diagnoses.has(dx);
on ? state.diagnoses.add(dx) : state.diagnoses.delete(dx);
// Toggle in place rather than re-rendering: the counts never change, and
// replacing the node would detach the element mid-click.
chip.setAttribute('aria-pressed', String(on));
applyFilters();
});
$('thead').addEventListener('click', (ev) => {
const el = ev.target.closest('[data-col]');
if (!el) return;
const col = COLUMNS[+el.dataset.col];
if (!col.key) return;
state.sort = state.sort.key === col.key
? { key: col.key, dir: -state.sort.dir }
: { key: col.key, dir: col.key === 'c' || col.key === 'dx' || col.key === 'f' ? 1 : -1 };
applyFilters();
});
// Synchronous: rendering ~30 rows is sub-millisecond, and rAF does not fire in a
// backgrounded tab, which would leave the table frozen mid-scroll.
$('tbody').addEventListener('scroll', () => drawRows(), { passive: true });
window.addEventListener('resize', () => { if (state.data) drawRows(true); });
$('rows').addEventListener('click', (ev) => {
const row = ev.target.closest('.row');
if (!row) return;
const item = state.view[+row.dataset.i];
const grouped = state.mode === 'campaign' && state.data.totals.days > 1;
grouped ? openCampaignDrawer(item) : openDrawer(item);
});
$('grain').addEventListener('click', (ev) => {
const btn = ev.target.closest('[data-mode]');
if (!btn || btn.dataset.mode === state.mode) return;
state.mode = btn.dataset.mode;
for (const b of $('grain').querySelectorAll('[data-mode]')) {
b.setAttribute('aria-pressed', String(b.dataset.mode === state.mode));
}
applyFilters();
});
$('drawer-close').addEventListener('click', closeDrawer);
$('scrim').addEventListener('click', closeDrawer);
document.addEventListener('keydown', (ev) => { if (ev.key === 'Escape') closeDrawer(); });
$('btn-settings').addEventListener('click', () => {
$('set-roas').value = state.settings.roas;
$('set-haircut').value = state.settings.haircut;
$('set-cap').value = state.settings.cap;
$('set-gap').value = state.settings.merge_gap;
$('settings').showModal();
});
$('settings').addEventListener('close', (ev) => {
if ($('settings').returnValue !== 'apply') return;
state.settings = {
roas: $('set-roas').value ? Number($('set-roas').value) : '',
haircut: Number($('set-haircut').value),
cap: Number($('set-cap').value),
merge_gap: Number($('set-gap').value),
};
analyze();
});
// Pick up anything preloaded from data/ on startup.
fetch('/api/state').then((r) => r.json()).then((s) => {
state.files = s.history.map((n) => ({ name: n, kind: 'history' }));
if (s.perf) state.files.push({ name: s.perf, kind: 'perf' });
renderFileList();
if (state.files.some((f) => f.kind === 'history')) analyze();
}).catch(() => {});

203
web/index.html Normal file
View File

@ -0,0 +1,203 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>PPC Out-of-Budget Dashboard</title>
<link rel="stylesheet" href="/styles.css">
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><text y='26' font-size='26'>📉</text></svg>">
</head>
<body>
<header class="topbar">
<div class="brand">
<span class="mark"></span>
<div>
<h1>Out-of-Budget Dashboard</h1>
<p id="subtitle">Amazon Ads change history &rarr; which campaigns keep going dark</p>
</div>
</div>
<div class="topbar-actions">
<button id="btn-settings" class="ghost" hidden>Assumptions</button>
<button id="btn-csv" class="ghost" hidden>CSV</button>
<button id="btn-xlsx" class="ghost" hidden>Excel</button>
<button id="btn-reset" class="ghost danger" hidden>Start over</button>
</div>
</header>
<main>
<!-- ------------------------------------------------------------ upload -->
<section id="stage-upload">
<div id="dropzone" class="dropzone" tabindex="0" role="button"
aria-label="Drop change-history exports here or click to choose files">
<div class="dz-icon" aria-hidden="true">
<svg viewBox="0 0 48 48" width="52" height="52" fill="none" stroke="currentColor"
stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round">
<path d="M24 32V10M14 20l10-10 10 10"/>
<path d="M8 30v6a4 4 0 0 0 4 4h24a4 4 0 0 0 4-4v-6"/>
</svg>
</div>
<h2>Drop your change-history exports here</h2>
<p>One file or a whole week of them. <button type="button" class="linklike" id="pick">Choose files</button>
&mdash; or drop a folder.</p>
<p class="fineprint">Runs entirely on this machine. Nothing is uploaded anywhere.</p>
<input type="file" id="file-input" multiple accept=".xlsx,.xlsm" hidden>
</div>
<div class="upload-extra">
<div class="perf-slot">
<div>
<h3>Optional: campaign performance report</h3>
<p>The change history has no per-campaign spend, so dollar figures cover only the
campaigns whose budget was edited that day. Add a performance report with
Campaign, Spend, Sales and Budget columns to price every campaign.</p>
</div>
<button type="button" class="ghost" id="pick-perf">Add report</button>
<input type="file" id="perf-input" accept=".xlsx,.xlsm,.csv,.tsv" hidden>
</div>
<ul id="filelist" class="filelist"></ul>
<button id="btn-analyze" class="primary" hidden>Analyse</button>
</div>
</section>
<!-- ----------------------------------------------------------- loading -->
<section id="stage-loading" hidden>
<div class="loading">
<div class="spinner" aria-hidden="true"></div>
<p id="loading-text">Reconstructing budget timelines&hellip;</p>
</div>
</section>
<!-- ------------------------------------------------------------- error -->
<section id="stage-error" hidden>
<div class="callout error">
<h2>That did not work</h2>
<p id="error-text"></p>
<button class="ghost" id="btn-error-back">Back</button>
</div>
</section>
<!-- --------------------------------------------------------- dashboard -->
<section id="stage-dash" hidden>
<div class="answer panel" id="answer">
<h2>A typical campaign's day</h2>
<p class="lede" id="answer-lede"></p>
<div class="daybar" id="daybar"></div>
<div class="daykeys" id="daykeys"></div>
<p class="sub" id="answer-account"></p>
</div>
<div id="kpis" class="kpis"></div>
<div class="panel">
<div class="panel-head">
<div>
<h2>Starvation through the day</h2>
<p class="sub" id="curve-sub"></p>
</div>
</div>
<div id="curve" class="curve"></div>
</div>
<div class="panel" id="reality-panel">
<div class="panel-head">
<div>
<h2>Reality check</h2>
<p class="sub">Modelled loss against the spend Amazon actually reported. If these
ever approach each other, the model is wrong &mdash; not the account.</p>
</div>
</div>
<div id="reality" class="reality"></div>
</div>
<div class="panel">
<div class="panel-head">
<div>
<h2 id="table-title">Campaigns</h2>
<p class="sub" id="table-sub"></p>
</div>
<div class="controls">
<div class="segmented" id="grain" hidden role="group" aria-label="Row grain">
<button data-mode="campaign" aria-pressed="true">One row per campaign</button>
<button data-mode="day" aria-pressed="false">One row per day</button>
</div>
<input type="search" id="search" placeholder="Filter by campaign name&hellip;"
aria-label="Filter by campaign name">
<label class="check"><input type="checkbox" id="only-priced"> Priced only</label>
<label class="check"><input type="checkbox" id="only-stale"> No action taken</label>
</div>
</div>
<div id="chips" class="chips"></div>
<div class="tablewrap">
<div class="thead" id="thead"></div>
<div class="tbody" id="tbody" tabindex="0">
<div class="spacer" id="spacer"></div>
<div class="rows" id="rows"></div>
</div>
</div>
<p class="legend">
<span><i class="sw" style="background:#16a34a"></i>In budget</span>
<span><i class="sw" style="background:#dc2626"></i>Out of budget</span>
<span><i class="sw" style="background:#9ca3af"></i>Paused</span>
<span><i class="sw" style="background:#e5e7eb"></i>Not yet created</span>
<span class="muted">Timeline runs midnight to midnight, left to right.</span>
</p>
</div>
<div class="panel">
<div class="panel-head">
<div>
<h2>Data quality</h2>
<p class="sub">Everything that could change how much you trust the numbers above.</p>
</div>
</div>
<div id="quality" class="quality"></div>
</div>
</section>
</main>
<!-- --------------------------------------------------------------- drawer -->
<aside id="drawer" class="drawer" hidden aria-label="Campaign detail">
<div class="drawer-head">
<div>
<h2 id="drawer-title"></h2>
<p class="sub" id="drawer-sub"></p>
</div>
<button class="ghost" id="drawer-close" aria-label="Close">Close</button>
</div>
<div id="drawer-body"></div>
</aside>
<div id="scrim" class="scrim" hidden></div>
<!-- ------------------------------------------------------------- settings -->
<dialog id="settings">
<form method="dialog">
<h2>Modelling assumptions</h2>
<p class="sub">These affect the money columns only. Timings are measured, not modelled.</p>
<label>ROAS
<input type="number" id="set-roas" step="0.01" min="0">
<small>Blank uses the account average from the export.</small>
</label>
<label>Marginal haircut
<input type="number" id="set-haircut" step="0.05" min="0" max="1">
<small>Incremental budget does not convert at the average. 0.7 = 70%.</small>
</label>
<label>Lost-spend cap
<input type="number" id="set-cap" step="0.5" min="1">
<small>Multiple of daily budget. Stops a campaign in budget 20 minutes implying
an impossible loss.</small>
</label>
<label>Outage merge gap
<input type="number" id="set-gap" step="1" min="0">
<small>Minutes in budget below which two outages count as one.</small>
</label>
<menu>
<button value="cancel" class="ghost">Cancel</button>
<button value="apply" class="primary" id="set-apply">Re-run</button>
</menu>
</form>
</dialog>
<script src="/app.js"></script>
</body>
</html>

457
web/styles.css Normal file
View File

@ -0,0 +1,457 @@
:root {
--bg: #f6f7f9;
--panel: #ffffff;
--ink: #16202e;
--ink-2: #55637a;
--ink-3: #8b97ab;
--line: #e3e7ee;
--line-2: #eef1f6;
--navy: #1f3864;
--accent: #2563eb;
--green: #16a34a;
--amber: #b45309;
--red: #dc2626;
--red-soft: #fee2e2;
--amber-soft: #fef3c7;
--green-soft: #dcfce7;
--shadow: 0 1px 2px rgba(16,32,46,.06), 0 8px 24px rgba(16,32,46,.06);
--radius: 12px;
--row-h: 34px;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #0f1419;
--panel: #171d26;
--ink: #e6ebf2;
--ink-2: #9aa7ba;
--ink-3: #6b7789;
--line: #263040;
--line-2: #1e2734;
--navy: #7aa2e8;
--accent: #60a5fa;
--red-soft: #3b1d1d;
--amber-soft: #3a2e12;
--green-soft: #12301f;
--shadow: 0 1px 2px rgba(0,0,0,.4), 0 8px 24px rgba(0,0,0,.3);
}
}
* { box-sizing: border-box; }
body {
margin: 0;
background: var(--bg);
color: var(--ink);
font: 14px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", Inter, Roboto, sans-serif;
-webkit-font-smoothing: antialiased;
}
h1, h2, h3 { margin: 0; font-weight: 650; letter-spacing: -.01em; }
h1 { font-size: 17px; }
h2 { font-size: 15px; }
h3 { font-size: 13px; }
p { margin: 0; }
.sub { color: var(--ink-2); font-size: 12.5px; max-width: 78ch; }
.muted { color: var(--ink-3); }
.fineprint { color: var(--ink-3); font-size: 12px; margin-top: 10px; }
/* ------------------------------------------------------------------ chrome */
.topbar {
position: sticky; top: 0; z-index: 40;
display: flex; align-items: center; justify-content: space-between; gap: 16px;
padding: 12px 24px;
background: color-mix(in srgb, var(--panel) 88%, transparent);
backdrop-filter: saturate(180%) blur(12px);
border-bottom: 1px solid var(--line);
}
.brand { display: flex; align-items: center; gap: 12px; }
.mark {
width: 30px; height: 30px; border-radius: 8px; flex: none;
background: linear-gradient(135deg, var(--green) 0%, var(--green) 32%, var(--red) 32%, var(--red) 100%);
}
.brand p { font-size: 12px; color: var(--ink-2); }
.topbar-actions { display: flex; gap: 8px; }
main { padding: 24px; max-width: 1680px; margin: 0 auto; }
button {
font: inherit; cursor: pointer; border-radius: 8px; padding: 7px 13px;
border: 1px solid var(--line); background: var(--panel); color: var(--ink);
transition: background .12s, border-color .12s, transform .06s;
}
button:hover { border-color: var(--ink-3); }
button:active { transform: translateY(1px); }
button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
.primary { background: var(--navy); border-color: var(--navy); color: #fff; font-weight: 600; }
@media (prefers-color-scheme: dark) { .primary { color: #0f1419; } }
.primary:hover { filter: brightness(1.08); }
.ghost { background: transparent; }
.danger:hover { border-color: var(--red); color: var(--red); }
.linklike {
border: 0; background: none; padding: 0; color: var(--accent);
text-decoration: underline; text-underline-offset: 2px;
}
/* ------------------------------------------------------------------ upload */
.dropzone {
border: 2px dashed var(--line); border-radius: 16px; background: var(--panel);
padding: 56px 32px; text-align: center; transition: border-color .15s, background .15s;
}
.dropzone:hover, .dropzone:focus-visible { border-color: var(--accent); outline: none; }
.dropzone.over { border-color: var(--accent); background: color-mix(in srgb, var(--accent) 7%, var(--panel)); }
.dz-icon { color: var(--ink-3); margin-bottom: 12px; }
.dropzone h2 { font-size: 19px; margin-bottom: 6px; }
.dropzone p { color: var(--ink-2); }
.upload-extra { max-width: 860px; margin: 20px auto 0; }
.perf-slot {
display: flex; align-items: center; gap: 20px;
background: var(--panel); border: 1px solid var(--line);
border-radius: var(--radius); padding: 16px 18px;
}
.perf-slot p { color: var(--ink-2); font-size: 12.5px; margin-top: 4px; }
.perf-slot button { flex: none; }
.filelist { list-style: none; padding: 0; margin: 14px 0 0; display: grid; gap: 8px; }
.filelist li {
display: flex; align-items: center; gap: 10px; font-size: 13px;
background: var(--panel); border: 1px solid var(--line);
border-radius: 9px; padding: 9px 13px;
}
.filelist .tag {
font-size: 10.5px; text-transform: uppercase; letter-spacing: .05em;
padding: 2px 7px; border-radius: 999px; background: var(--line-2); color: var(--ink-2);
}
.filelist .ok { color: var(--green); margin-left: auto; }
#btn-analyze { margin: 18px auto 0; display: block; padding: 11px 30px; font-size: 15px; }
/* ----------------------------------------------------------------- loading */
.loading { text-align: center; padding: 90px 20px; color: var(--ink-2); }
.spinner {
width: 30px; height: 30px; margin: 0 auto 16px; border-radius: 50%;
border: 3px solid var(--line); border-top-color: var(--navy);
animation: spin .8s linear infinite;
}
@keyframes spin { to { transform: rotate(360deg); } }
.callout { background: var(--panel); border: 1px solid var(--line); border-radius: var(--radius); padding: 22px; }
.callout.error { border-color: var(--red); }
.callout.error h2 { color: var(--red); margin-bottom: 8px; }
.callout p { color: var(--ink-2); margin-bottom: 14px; word-break: break-word; }
/* ------------------------------------------------------------- answer block */
.answer { border-left: 4px solid var(--red); }
.answer .lede {
font-size: 19px; line-height: 1.5; margin: 8px 0 18px; max-width: 92ch;
letter-spacing: -.005em;
}
.answer .lede b { font-weight: 700; font-variant-numeric: tabular-nums; }
.answer .lede .run { color: var(--green); }
.answer .lede .out { color: var(--red); }
.daybar {
display: flex; height: 42px; border-radius: 8px; overflow: hidden;
box-shadow: inset 0 0 0 1px rgba(0,0,0,.08);
}
.daybar span {
display: flex; align-items: center; justify-content: center;
font-size: 12px; font-weight: 650; color: #fff; white-space: nowrap; overflow: hidden;
text-shadow: 0 1px 2px rgba(0,0,0,.3);
}
.daykeys { display: flex; flex-wrap: wrap; gap: 20px; margin-top: 11px; font-size: 12.5px; }
.daykeys div { display: flex; align-items: center; gap: 7px; color: var(--ink-2); }
.daykeys b { color: var(--ink); font-variant-numeric: tabular-nums; }
#answer-account { margin-top: 14px; }
/* -------------------------------------------------------------------- KPIs */
.kpis {
display: grid; grid-template-columns: repeat(auto-fit, minmax(190px, 1fr));
gap: 12px; margin-bottom: 16px;
}
/* Eight tiles: a clean 4 + 4 beats auto-fit's 7 + 1 orphan. */
@media (min-width: 1180px) { .kpis { grid-template-columns: repeat(4, 1fr); } }
.kpi {
background: var(--panel); border: 1px solid var(--line);
border-radius: var(--radius); padding: 15px 17px; box-shadow: var(--shadow);
}
.kpi .label {
font-size: 10.5px; font-weight: 700; letter-spacing: .07em;
text-transform: uppercase; color: var(--ink-2);
}
.kpi .value {
font-size: 27px; font-weight: 680; letter-spacing: -.02em;
margin: 5px 0 3px; font-variant-numeric: tabular-nums; color: var(--navy);
}
.kpi .note { font-size: 11.5px; color: var(--ink-3); }
.kpi.alarm .value { color: var(--red); }
/* ------------------------------------------------------------------ panels */
.panel {
background: var(--panel); border: 1px solid var(--line); border-radius: var(--radius);
padding: 18px; margin-bottom: 16px; box-shadow: var(--shadow);
}
.panel-head {
display: flex; align-items: flex-start; justify-content: space-between;
gap: 20px; flex-wrap: wrap; margin-bottom: 14px;
}
.controls { display: flex; align-items: center; gap: 10px; }
.controls input[type=search] {
font: inherit; padding: 7px 11px; min-width: 240px;
border: 1px solid var(--line); border-radius: 8px;
background: var(--bg); color: var(--ink);
}
.check { display: flex; align-items: center; gap: 6px; font-size: 12.5px; color: var(--ink-2); }
.segmented { display: inline-flex; border: 1px solid var(--line); border-radius: 8px; overflow: hidden; }
.segmented button {
border: 0; border-radius: 0; padding: 7px 12px; font-size: 12.5px;
background: transparent; color: var(--ink-2);
}
.segmented button + button { border-left: 1px solid var(--line); }
.segmented button[aria-pressed="true"] { background: var(--navy); color: #fff; font-weight: 600; }
@media (prefers-color-scheme: dark) { .segmented button[aria-pressed="true"] { color: #0f1419; } }
/* One cell per day: which days were bad, at a glance. */
.dayheat { display: flex; gap: 2px; height: 15px; margin: 0 10px; }
.dayheat i { flex: 1; border-radius: 2px; min-width: 3px; }
.trendcell { font-size: 11.5px; }
/* ------------------------------------------------------------------- curve */
.curve { display: flex; align-items: flex-end; gap: 3px; height: 190px; padding-top: 12px; }
.curve .bar { flex: 1; display: flex; flex-direction: column; justify-content: flex-end;
align-items: center; height: 100%; gap: 5px; }
.curve .fill {
width: 100%; border-radius: 4px 4px 0 0; min-height: 2px;
background: linear-gradient(180deg, var(--red), #ef4444);
transition: filter .12s;
}
.curve .bar:hover .fill { filter: brightness(1.2); }
.curve .hour { font-size: 9.5px; color: var(--ink-3); font-variant-numeric: tabular-nums; }
.curve .pct { font-size: 9.5px; color: var(--ink-2); font-variant-numeric: tabular-nums; }
/* ----------------------------------------------------------------- reality */
.reality { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 14px; }
.reality div { border-left: 3px solid var(--line); padding-left: 12px; }
.reality .k { font-size: 11.5px; color: var(--ink-2); }
.reality .v { font-size: 19px; font-weight: 650; font-variant-numeric: tabular-nums; }
/* ------------------------------------------------------------------- chips */
.chips { display: flex; flex-wrap: wrap; gap: 7px; margin-bottom: 12px; }
.chip {
border: 1px solid var(--line); border-radius: 999px; padding: 4px 12px;
font-size: 12px; background: var(--panel); color: var(--ink-2);
}
.chip[aria-pressed="true"] { background: var(--navy); border-color: var(--navy); color: #fff; }
@media (prefers-color-scheme: dark) { .chip[aria-pressed="true"] { color: #0f1419; } }
.chip .n { opacity: .65; margin-left: 5px; font-variant-numeric: tabular-nums; }
/* ------------------------------------------------------------------- table */
/* Scroll sideways on a narrow window rather than crushing the columns. The
header and body scroll together because both sit inside this wrapper. */
.tablewrap {
border: 1px solid var(--line); border-radius: 10px;
overflow: hidden auto; overflow-x: auto;
}
.tablewrap .thead, .tablewrap .tbody { min-width: 1250px; }
.tablewrap.multi .thead, .tablewrap.multi .tbody { min-width: 1300px; }
.tablewrap.grouped .thead, .tablewrap.grouped .tbody { min-width: 1350px; }
.thead, .row {
display: grid;
/* Duration columns are wide because they read "23 hours 45 min", not "23.75". */
/* "23h 45min" measures 83px with padding, so 94px leaves real headroom;
170px fits the longest diagnosis pill, "Structurally underfunded". */
grid-template-columns: minmax(180px, 2fr) 92px 92px 60px 60px 56px 126px 86px 58px 176px 140px;
align-items: center;
}
/* Per-day grain: one campaign has one row per day, so the date must be shown. */
.tablewrap.multi .thead, .tablewrap.multi .row {
grid-template-columns: minmax(150px, 1.8fr) 76px 92px 92px 56px 56px 52px 112px 82px 56px 176px 138px;
}
/* Per-campaign grain: days collapse into one row with a per-day heat strip. */
.tablewrap.grouped .thead, .tablewrap.grouped .row {
grid-template-columns: minmax(170px, 1.9fr) 56px 92px 92px 92px 56px 100px 64px 82px 56px 176px 138px;
}
.row .date { font-size: 11.5px; color: var(--ink-2); font-variant-numeric: tabular-nums; }
.thead {
background: var(--line-2); border-bottom: 1px solid var(--line);
font-size: 11px; font-weight: 700; letter-spacing: .04em;
text-transform: uppercase; color: var(--ink-2);
}
/* Wrap rather than collide: two-word headers need the second line. */
.thead > div { padding: 8px 10px; cursor: pointer; user-select: none; line-height: 1.2; }
.thead > div:hover { color: var(--ink); }
.thead .num, .row .num { text-align: right; font-variant-numeric: tabular-nums; }
.thead .sorted { color: var(--accent); }
.tbody { position: relative; overflow-y: auto; max-height: 620px; }
.tbody:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; }
.rows { position: absolute; inset: 0 0 auto 0; }
.spacer { width: 1px; }
.row {
position: absolute; left: 0; right: 0; height: var(--row-h);
border-bottom: 1px solid var(--line-2); cursor: pointer;
background: var(--panel);
}
.row:hover { background: var(--line-2); }
.row > div { padding: 0 10px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.row .name { font-size: 12.5px; }
.row .num { font-size: 12.5px; }
.row .num.run, table.grid td.run { color: var(--green); }
.row .num.out, table.grid td.out { color: var(--red); font-weight: 600; }
/* Durations must never wrap or ellipsis -- "23h 45min" is the widest case. */
.row .num.dur { white-space: nowrap; }
/* Base style is unscoped so timeline strips also work inside the drawer. */
.strip {
height: 15px; border-radius: 3px;
box-shadow: inset 0 0 0 1px rgba(0,0,0,.06);
}
.row .strip { margin: 0 10px; width: calc(100% - 20px); }
.row .dx { font-size: 11px; }
.row .act { font-size: 11px; }
.row .act .stale {
display: inline-block; padding: 2px 8px; border-radius: 999px; font-weight: 600;
background: var(--red-soft); color: var(--red); white-space: nowrap;
overflow: hidden; text-overflow: ellipsis; max-width: 100%;
}
.row .act .fresh { color: var(--ink-2); white-space: nowrap; }
.pill {
display: inline-block; padding: 2px 8px; border-radius: 999px;
font-size: 10.5px; font-weight: 600; white-space: nowrap;
overflow: hidden; text-overflow: ellipsis; max-width: 100%;
}
.dx-under { background: var(--red-soft); color: var(--red); }
.dx-early { background: var(--amber-soft); color: var(--amber); }
.dx-thrash { background: #ede9fe; color: #6d28d9; }
.dx-evening { background: var(--amber-soft); color: var(--amber); }
.dx-inter { background: #dbeafe; color: #1d4ed8; }
.dx-healthy { background: var(--green-soft); color: var(--green); }
.dx-paused { background: var(--line-2); color: var(--ink-2); }
@media (prefers-color-scheme: dark) {
.dx-thrash { background: #2e1f4d; color: #c4b5fd; }
.dx-inter { background: #17294d; color: #93c5fd; }
}
.unpriced { color: var(--ink-3); font-style: italic; font-size: 11.5px; }
.legend { display: flex; flex-wrap: wrap; gap: 16px; margin-top: 11px; font-size: 11.5px; color: var(--ink-2); }
.legend span { display: flex; align-items: center; gap: 6px; }
.sw { width: 11px; height: 11px; border-radius: 3px; display: inline-block; }
.empty { padding: 44px; text-align: center; color: var(--ink-3); }
/* ----------------------------------------------------------------- quality */
.quality { display: grid; gap: 9px; }
.qrow {
display: grid; grid-template-columns: 84px minmax(180px, 1fr) 150px 2.4fr;
gap: 14px; align-items: start; padding: 11px 13px;
border: 1px solid var(--line); border-radius: 9px; font-size: 12.5px;
}
.qrow .badge {
font-size: 10.5px; font-weight: 700; letter-spacing: .05em; text-transform: uppercase;
padding: 3px 8px; border-radius: 6px; text-align: center;
}
.q-ok .badge { background: var(--green-soft); color: var(--green); }
.q-review .badge { background: var(--amber-soft); color: var(--amber); }
.q-fail .badge { background: var(--red-soft); color: var(--red); }
.q-fail { border-color: var(--red); }
.qrow .qval { font-variant-numeric: tabular-nums; color: var(--ink); }
.qrow .qnote { color: var(--ink-2); }
/* --------------------------------------------------------------- recurring */
table.grid { width: 100%; border-collapse: collapse; font-size: 12.5px; }
table.grid th, table.grid td { padding: 8px 10px; border-bottom: 1px solid var(--line-2); text-align: right; }
table.grid th { font-size: 11px; text-transform: uppercase; letter-spacing: .04em; color: var(--ink-2); }
table.grid th:first-child, table.grid td:first-child { text-align: left; }
table.grid td { font-variant-numeric: tabular-nums; }
/* Durations sit on one line and share a column width so "2h 5min" lines up
under "23h 45min" instead of drifting. */
table.grid td.dur { white-space: nowrap; min-width: 78px; }
table.grid th small { display: block; font-size: 9px; font-weight: 500; opacity: .7; }
.trend-worse { color: var(--red); font-weight: 600; }
.trend-better { color: var(--green); }
.spark { display: inline-flex; align-items: flex-end; gap: 1px; height: 18px; }
.spark i { width: 4px; background: var(--red); opacity: .8; border-radius: 1px; }
/* ------------------------------------------------------------------ drawer */
.drawer {
position: fixed; top: 0; right: 0; bottom: 0; width: min(660px, 94vw); z-index: 60;
background: var(--panel); border-left: 1px solid var(--line);
box-shadow: -12px 0 40px rgba(16,32,46,.14); padding: 20px; overflow-y: auto;
}
.drawer-head { display: flex; justify-content: space-between; gap: 16px; margin-bottom: 16px; }
.drawer-head h2 { font-size: 15px; word-break: break-word; }
.scrim { position: fixed; inset: 0; z-index: 55; background: rgba(16,32,46,.35); }
.dgrid { display: grid; grid-template-columns: repeat(auto-fit, minmax(128px, 1fr)); gap: 10px; margin-bottom: 18px; }
.dgrid div { border: 1px solid var(--line); border-radius: 9px; padding: 10px 12px; }
.dgrid .k { font-size: 10.5px; text-transform: uppercase; letter-spacing: .05em; color: var(--ink-2); }
.dgrid .v { font-size: 17px; font-weight: 640; font-variant-numeric: tabular-nums; margin-top: 3px; }
.dstrip { height: 26px; border-radius: 5px; box-shadow: inset 0 0 0 1px rgba(0,0,0,.08); }
.axis { display: flex; justify-content: space-between; font-size: 10px; color: var(--ink-3); margin-top: 4px; }
.drawer h3 { margin: 18px 0 8px; }
/* ------------------------------------------------------- action log (drawer) */
.act-none, .act-yes {
border-radius: 9px; padding: 12px 14px; margin-bottom: 10px;
border-left: 3px solid var(--line);
}
.act-none { background: var(--red-soft); border-left-color: var(--red); }
.act-none strong { color: var(--red); font-size: 13.5px; }
.act-yes { background: var(--line-2); border-left-color: var(--green); }
.act-yes strong { font-size: 13.5px; }
.act-none p, .act-yes p { margin-top: 5px; font-size: 12px; color: var(--ink-2); }
.act-stale-text { color: var(--red); }
table.act-log td { font-size: 11.5px; vertical-align: top; }
table.act-log td:last-child { color: var(--ink-2); word-break: break-word; }
.pill.act-budget { background: #dbeafe; color: #1d4ed8; }
.pill.act-placement { background: #ede9fe; color: #6d28d9; }
.pill.act-strategy { background: #fce7f3; color: #be185d; }
.pill.act-bid { background: var(--green-soft); color: var(--green); }
.pill.act-targeting { background: var(--amber-soft); color: var(--amber); }
.pill.act-status { background: var(--line-2); color: var(--ink-2); }
.pill.act-structure { background: #e0f2fe; color: #0369a1; }
.pill.act-portfolio { background: var(--line-2); color: var(--ink-2); }
@media (prefers-color-scheme: dark) {
.pill.act-budget { background: #17294d; color: #93c5fd; }
.pill.act-placement { background: #2e1f4d; color: #c4b5fd; }
.pill.act-strategy { background: #401027; color: #f9a8d4; }
.pill.act-structure { background: #0c2b3d; color: #7dd3fc; }
}
/* ---------------------------------------------------------------- settings */
dialog {
border: 1px solid var(--line); border-radius: var(--radius); padding: 22px;
background: var(--panel); color: var(--ink); max-width: 440px; box-shadow: var(--shadow);
}
dialog::backdrop { background: rgba(16,32,46,.4); }
dialog label { display: block; margin: 16px 0; font-size: 12.5px; font-weight: 600; }
dialog input {
display: block; width: 100%; margin-top: 5px; font: inherit; padding: 7px 10px;
border: 1px solid var(--line); border-radius: 8px; background: var(--bg); color: var(--ink);
}
dialog small { display: block; margin-top: 4px; font-weight: 400; color: var(--ink-3); }
dialog menu { display: flex; justify-content: flex-end; gap: 8px; padding: 0; margin: 20px 0 0; }
@media (max-width: 1100px) {
.thead, .row { grid-template-columns: minmax(160px, 2fr) 66px 60px 56px 150px 160px; }
.thead > div:nth-child(n+7), .row > div:nth-child(n+7) { display: none; }
}