20 KiB
Utopia Pricing Agent — Architecture
A one-page Streamlit dashboard (Utopia/CRAI design system) that turns live COSMOS data into price recommendations a human can Approve / Modify / Reject. Read-only: nothing is written back to COSMOS or Amazon.
1. High-level view
flowchart LR
U["🧑 User<br/>(same-network browser)"] --> APP
subgraph APP["app.py — presentation (Streamlit, CRAI theme)"]
SIDE["Sidebar<br/>Single product / Product line<br/>+ filters, kill switch"]
QUEUE["Recommendation queue<br/>tiles · pills · rows"]
SECT["Per-SKU sections<br/>price · inventory · scenarios<br/>competitors · PPC · costs · AI"]
end
subgraph DASH["dashboard/ package"]
THEME["theme.py<br/>CRAI tokens + plotly template"]
LIVE["live_data.py<br/>adapter + decision engine<br/>+ scenario economics"]
end
subgraph CORE["src/pricing_agent — analysis core"]
AN["analyze.py"]
MARGIN["margin_engine.py"]
ELAST["elasticity.py"]
PERF["performance.py"]
SVC["cosmos/service.py"]
CLIENT["cosmos/client.py"]
end
COSMOS[("COSMOS API")]
APIFY[("Apify — optional")]
APP --> THEME
APP --> LIVE
LIVE --> AN
AN --> MARGIN & ELAST & PERF
AN --> SVC --> CLIENT --> COSMOS
LIVE --> SVC
AN -.optional.-> APIFY
2. Layers
| Layer | Files | Responsibility |
|---|---|---|
| Presentation | app.py |
All rendering, zero pricing logic. Session state (approve/modify/reject, filters, per-SKU section + window), staged progress loader, session-state cache. |
| Design system | dashboard/theme.py, .streamlit/config.toml |
CRAI palette (cream #f4f0e8, teal #0c8276, coral #df4f33, navy #22304e), Inter font, plotly template. |
| Adapter + engine | dashboard/live_data.py |
Builds the per-SKU dict; decides the action; computes scenario economics (elasticity projection → bulk reconciliation → calibration); exposes scenarios_for_window(). |
| Competitive state | src/pricing_agent/competitive_state.py |
The one competitive fact the cascade may read. Adapts either scraper into a typed WON/LOST_PRICE/LOST_ELIGIBILITY/SUPPRESSED state with a source and a timestamp, gates it on age, and logs disagreement between sources. |
| Analysis core | src/pricing_agent/analyze.py |
Orchestrates one SKU: fees → trend → bulk → ad cost → elasticity → actual-profit evidence. |
| Money math | tools/margin_engine.py |
Pure: break-even, MAP, contribution margin, suggested price. |
| Statistics | elasticity.py, performance.py |
Log-log elasticity fit, profit-optimal sweep, actual-profit aggregation. |
| Data access | cosmos/{client,service,models}.py |
Auth + retry client; endpoint calls + response flattening; typed pydantic models. |
3. Data sources — what each COSMOS endpoint feeds
flowchart TB
subgraph COSMOS["COSMOS API"]
TH["/sales-insight/takehome-calculator<br/>nested fees.breakdown · cost.breakdown"]
INVP["/invp-insight<br/>trend + inventory + dateMap PROJECTIONS"]
BULK["/sales-insight/bulk-calculator<br/>storage + total take-home"]
SI["/sales-insight (daily, 6-month)<br/>price · units · revenue · profit · ad spend"]
PROD["/products<br/>brand · marketplace"]
CAMP["/api/campaigns · /adsApi<br/>budget · ACoS · ad sales (not yet wired)"]
end
TH -->|"_flatten_takehome()"| FEES["Fee model<br/>referral% · FBA · landed · returns"]
INVP --> TREND["Velocity + cover days"]
INVP --> INVPROJ["Inventory Outlook tab<br/>real weekly units/value/cover/arrivals"]
BULK --> STORAGE["Storage + take-home (scenarios)"]
SI --> HIST["180-day daily series<br/>(window filter + calibration)"]
SI --> ADS["Ad spend / TACoS (PPC tab)"]
PROD --> META["Brand / marketplace"]
Two response quirks handled:
- Fees come nested (
fees.breakdown["Referral Fee"],"$ 9.28"strings).service._flatten_takehome()normalises them — without it every fee parsed to 0 (the old "$0.99 / break-even $0" bug). - INVP
dateMapholds COSMOS's own forward inventory projection (weekly units, value, cover days, warehouse arrivals). The Inventory tab renders this directly — not a locally-invented forecast.
Genuinely not in this COSMOS integration → shown as "—", never faked: ad-attributed
sales / ACoS / campaign budget (live in /api/campaigns + /adsApi, not yet wired), and
historical competitor prices (Apify gives a current snapshot only).
Competitor data is no longer display-only. COSMOS has no Buy Box, no rival price and no third-party offer anywhere in it — that gap is filled by a scrape, and as of the competitor wiring two of its facts (our Buy Box being suppressed, and a rival materially undercutting us) reach the verdict. They are the only two, they are bounded by the guardrails, and their absence changes nothing. See §6.
4. Per-SKU pipeline (one "Analyze")
sequenceDiagram
participant U as User
participant A as app.py
participant L as live_data.build_live_sku
participant AN as analyze.py
participant S as CosmosService
U->>A: Single product / Product line
A->>L: get_live_data(skus, progress_cb)
Note over A: staged progress bar (3→10→45→82→94→100%)
L->>S: get_current_price
L->>S: get_sales_history (180d, parallel windows)
L->>AN: analyze_price (fees, trend, bulk, elasticity, evidence)
L->>S: get_invp (real inventory projection)
L->>S: bulk_quote (storage)
L->>L: decide action + scenario economics + 30d/6mo calibration
L-->>A: {summary, details, errors} (session-cached)
A-->>U: queue + expandable per-SKU analysis
5. Scenario economics (the heart of the Scenarios tab)
For each candidate price, one consistent chain:
flowchart LR
W["Window filter<br/>7/14/30/90d · 6mo"] --> BASE["Baseline velocity<br/>= avg units/day in window"]
BASE --> DEMAND["Units(p) = units × (p/cur)^elasticity"]
DEMAND --> REV["Revenue = units × p × 30"]
REV --> AD["Ad spend = TACoS × revenue"]
DEMAND --> TH["Take-home (bulk calculator fee model)"]
TH --> GROSS["Gross = take-home − storage − ad"]
GROSS --> CAL["× realization factor<br/>(actual booked ÷ modeled at current)"]
CAL --> NET["Net profit / 30d"]
Key rules:
- Current row = FACT, not a projection: real units, real revenue, real ad spend, real
booked profit. Its price is the average price sold (revenue ÷ units) so
price × units × 30 = revenuereconciles — this is below list when promos ran, and changes with the window because the avg selling price differed period to period. The list price is fixed. - Calibration: raw bulk-calculator profit over-states reality (prices at list, ignores real returns/promos). A realization factor = actual booked profit ÷ modeled profit at the current price scales every projected row, anchoring net profit to what the SKU truly earns.
- Suggested price (teal callout) is computed from the full 6-month window always — stable — independent of the display-window filter.
- ⭐ marks the highest-net-profit price in the current view.
6. Decision engine (deterministic, first match wins)
flowchart TD
S([signals]) --> R0{cost data = 0?}
R0 -- yes --> INV["🔍 INVESTIGATE · NO_COST_DATA"]
R0 -- no --> RB{our Buy Box suppressed?}
RB -- yes --> INVB["🔍 INVESTIGATE · BUYBOX_SUPPRESSED"]
RB -- no --> R1{price < break-even?}
R1 -- yes --> UP1["↑ raise to safe floor · BELOW_BREAK_EVEN"]
R1 -- no --> R2{losing money after ads?}
R2 -- yes --> UP2["↑ raise · LOSING_MONEY"]
R2 -- no --> R3{cover < 35d?}
R3 -- yes --> UP3["↑ +5% · LOW_STOCK"]
R3 -- no --> R4{−30% sales, no cause?}
R4 -- yes --> INV2["🔍 INVESTIGATE · UNEXPLAINED_DROP"]
R4 -- no --> R5{cover > 90d?}
R5 -- yes --> DN["↓ −5% · EXCESS_STOCK"]
R5 -- no --> RC{lost Buy Box AND rival ≥3% below?}
RC -- yes --> DNC["↓ toward rival · COMPETITOR_UNDERCUT"]
RC -- no --> R6{profit-optimal ≠ current?}
R6 -- yes --> MOVE["↑/↓ toward optimal · PROFIT_OPTIMAL"]
R6 -- no --> HOLD["→ MAINTAIN · NO_SIGNALS"]
Guardrails: floor = break-even × 1.05, ceiling = current × 1.25; the recommended move is capped at ±5% (bigger steps need elevated approval); the kill switch pauses all approvals.
Competitor rules — the two that can move a price, and what bounds them
Both branches read a single CompetitiveState
(competitive_state.py), never a raw scrape:
| Rule | Fires when | Effect |
|---|---|---|
BUYBOX_SUPPRESSED |
Amazon is not showing our offer | Investigate, hold. Placed directly under NO_COST_DATA: those are the only two states where the answer is "go and fix something" rather than "set a price". A suppressed variant sells nothing at any price, so its margin and its modelled optimum both describe a listing nobody can buy from. |
COMPETITOR_UNDERCUT |
LOST_PRICE and cheapest rival ≥ competitor_undercut_material_pct below us |
Decrease toward the rival, floored and step-capped like every other branch. |
A third signal, a competitor premium while we hold the Buy Box, is a narrative note only. It never sets a price and never changes an action — the elasticity fit is the thing with evidence behind it, and a premium is not grounds to overrule it.
Ordering is deliberate: inventory risk still outranks competitor position, which outranks
profit-optimal. Chasing a rival down while the shelf is emptying pays margin to sell out
faster. This is visible in the backtest below — two of five real SKUs did not move under a
counterfactual undercut precisely because LOW_STOCK and LOSING_MONEY fired first.
Three properties make this safe to ship:
- Fail-safe. Absent, failed, stale (>
competitor_state_max_age_hours) and "ownership unknown" all collapse to one flag, and the cascade then computes exactly the verdict it computed before competitor data existed. Nothing waits on a scrape; nothing is blocked by one. Competitor data can only ever add a verdict. - Never below break-even. The rival price is a candidate (
comp_match), not a decision. Verified against real SKUs with a counterfactual rival 60% below us: against a $14.88 floor the shipped recommendations were $15.19–$21.84, because the ±5% step cap binds first. Zero violations. - One named reason per verdict. No blended scores — every fired rule is traceable to a
single reason code, and
logger.infonames the SKU, the rule, the state and the source.
Thresholds live in config/pricing_rules.yaml
(competitor_undercut_material_pct: 0.03, competitor_premium_material_pct: 0.10,
competitor_state_max_age_hours: 6.0), not in code. The 3% floor sits above the ~2% band our
own realized price already swings through as coupons toggle.
Inventory cover matches COSMOS Inventory Planning
cover_days is COSMOS's own coverDays, so the dashboard and the INVP grid never quote
two different numbers for one SKU. COSMOS counts inbound stock against a 7-day
velocity, so it reads longer than what is on the shelf — UBMICROFIBERDUVETTWINWHITE is
78 days on (3,999 on hand + 1,030 inbound) ÷ 64/day, against 63 on-hand-only. Both are
reported: the tile leads with the matched figure and appends 63 d on hand, rest inbound.
The on-hand figure remains the fallback, because COSMOS returns coverDays: 0 on some
very low-velocity SKUs that hold months of stock (UBMICROFIBERBS4PCFULLGREY: 167 units,
334 real days, COSMOS said 0). Zero satisfies neither inventory rule, so taken literally it
silences both.
Trade-off, accepted deliberately: stockout risk is now judged partly on stock that has not
landed. Measured over the 56-SKU covered line, matching COSMOS moved 7 verdicts —
LOW_STOCK 7 → 4, EXCESS_STOCK 18 → 22. The one to watch is
UBMICROFIBERBS4PCKINGWHITE: 12 days on the shelf, 84 with inbound, so it no longer
raises. If that shipment slips, nothing protects it.
Display bands are COSMOS's Alpha/Beta scheme (theme.COVER_BANDS, Alpha 20/40/70/100). The
pricing triggers are separate and live in pricing_rules.yaml
(low_cover_days: 35, high_cover_days: 90) — COSMOS's pink at 70 days is a replenishment
warning, while crossing a trigger here spends margin on a 5% move. Adopting the band edges
(40/70) would put six more SKUs on a discount; the measured table is in the config beside the
values.
Coverage: the comparison sheet gates competitor data, one product line at a time
The competitor workbook currently covers one product line, so the engine reads it as the first competitor source and gates on coverage:
| SKU | Competitor state |
|---|---|
| In the sheet | Priced from the sheet — real like-for-like rival prices, basis=like-for-like-sheet |
| Not in the sheet | N/A, naming what the sheet does cover. No rule fires; the verdict is byte-identical to the competitor-blind one |
Coverage is the exact SKU set in the sheet, not a line prefix. Measured against the real
workbook, a prefix gate would be wrong in both directions: the UBMICROFIBERDUVET run contains
49 UBMICROFIBERDUVET* SKUs and 7 UBMICROFIBERBS4PC* ones (variants come off the Amazon
parent twister, and COSMOS maps those ASINs to whatever SKU codes they carry), while the line
has 139 SKUs in COSMOS of which only 56 reached a comparison row. So the sheet's own SKU list is
the authority, and "not in the sheet" is reported as a coverage hole, never as a claim that
the SKU has no competitors.
competitor_sheet_only: true (default while one line is under test) means an uncovered SKU gets
N/A rather than falling through to a per-ASIN Apify scrape — so every verdict either rests on
the sheet or says it has no competitor data. Set it False once coverage is broad enough for
Apify to be a sensible fallback. Config: competitor_sheet_path (blank = auto-discover the
newest Competitor_Price_Comparison_*.xlsx), competitor_sheet_dirs,
competitor_sheet_max_age_hours: 168 (the sheet is a 25–35 min batch run, not a live feed, so
it gets a longer limit than the 6 h single-ASIN one).
Two bases, never conflated. competitor_min can be a price of two different things, and the
basis field records which:
same-asin-buybox(Apify) — another seller's offer on our own listing. OnlyLOST_PRICEfires the undercut rule; a rival holding the Buy Box above us isLOST_ELIGIBILITY, where cutting donates margin.like-for-like-sheet— a rival brand's equivalent variant, matched on size + colour. A cheaper one fires the rule regardless of who owns our Buy Box: we can hold ours perfectly well while a different product undercuts us. This is what the comparison tool exists to report. It is never labelled a Buy Box loss.
Two sheet-driven refinements, both from real rows:
- A rival whose own Buy Box is suppressed is excluded from the band. Their price is not buyable, so undercutting it donates margin for nothing.
- A material undercut now explains a velocity drop. Previously a drop with no own price
change and no ad collapse was filed
UNEXPLAINED_DROPeven when the sheet held the explanation — observed onUBMICROFIBERDUVETKINGPURPLE(rival 15.6% below) andUBMICROFIBERBS4PCFULLGREY(35.2% below). A named cause now converts the Investigate into an actionable verdict, exactly as the existing stockout branch already did. A stockout still explains a drop first; an immaterial rival explains nothing.
Finally, a suppressed listing has no selling price (it sells nothing, so COSMOS records no sales), which used to fail with a bare "no current selling price found in COSMOS". That error now names the cause, so the most actionable rows in the sheet stop looking like a data problem.
Two scrapers, one state
| Source | Authoritative for | Why |
|---|---|---|
Apify (tools/amazon/apify.py) |
Buy Box state read by the engine | The only source carrying a seller id, so the only one that can tell WON from LOST_PRICE from LOST_ELIGIBILITY. Those lead to opposite actions (cut price vs. fix fulfilment eligibility). |
Playwright (../scraper/) |
The workbook: like-for-like size/colour matching, BSR, demand buckets, SKU gaps | Apify cannot produce any of it. Its Buy Box field knows only whether a price rendered, not whose it was. |
The split is by question, not preference, so neither source is redundant. reconcile()
cross-checks the authoritative state against the Playwright run's own cache
(scraper/.scrape_cache.json, keyed ASIN@ZIP) and logs any disagreement rather than
letting a workbook and a recommendation contradict each other in front of a stakeholder. A
disagreement never changes the verdict. Where the authoritative source has nothing usable but
the other has a SUPPRESSED, that fact is promoted and the provenance recorded — discarding
it to preserve a hierarchy would be choosing the hierarchy over the fact.
Backtest (scripts/backtest_competitor_rules.py, 5 real SKUs, real COSMOS data):
| Arm | Verdicts changed |
|---|---|
| Real competitor state (both cached entries stale: 66h / 146h) | 0 / 5 — the fail-safe working |
| Counterfactual 8% undercut | 3 / 5 — the other 2 blocked by LOW_STOCK / LOSING_MONEY |
| Counterfactual suppression | 5 / 5 → Investigate |
| Prices shipped below break-even, any arm | 0 |
7. Key formulas
| Quantity | Formula |
|---|---|
| Take-home / unit | p·(1 − referral% − returns%) − landed − FBA − other |
| Break-even | (landed + FBA + returns + other) / (1 − referral%) |
| Elasticity | OLS on ln(units/day) = a + e·ln(price) over 6 months |
| Scenario demand | units × (p / p₀)^e |
| Realization factor | actual booked profit (window) ÷ modeled net at current price |
| TACoS | ad spend ÷ total revenue (window) |
| Avg sold price | revenue ÷ units (window) — reconciles the Current row |
8. Repository map
pricing_agent/
├── app.py # dashboard (presentation only)
├── legacy_app.py # previous analyst UI (still runnable)
├── dashboard/
│ ├── theme.py # CRAI design tokens + plotly template
│ └── live_data.py # COSMOS adapter, decision + scenario engine
├── src/pricing_agent/
│ ├── analyze.py # per-SKU orchestration → AnalysisResult
│ ├── competitive_state.py # canonical Buy Box state + two-scraper reconciliation
│ ├── elasticity.py # demand model + profit optimizer
│ ├── performance.py # actual-profit evidence
│ ├── tools/margin_engine.py # pure fee/break-even math (golden-tested)
│ └── cosmos/
│ ├── client.py # auth + retry HTTP
│ ├── service.py # endpoints, _flatten_takehome, INVP projections
│ └── models.py # typed COSMOS responses (pydantic)
├── config/ # settings + pricing_rules.yaml (incl. competitor thresholds)
├── scripts/
│ └── backtest_competitor_rules.py # verdict delta, competitor rules blind vs live
├── .streamlit/config.toml # CRAI theme
└── tests/ # incl. margin-engine golden values and
# test_competitor_rules.py (fail-safe + ordering invariants)
9. Principles
- Deterministic core, narrative shell — every number is a formula over COSMOS data; language models only phrase explanations.
- Read-only — the agent proposes; a human approves; nothing writes back.
submit_price_approvalremains a stub with no callers. - Honest gaps — missing upstream data shows "—" or an explicit investigation, never a fabricated number.
- Facts vs projections are labeled — the Current row is real booked history; other prices are clearly modeled.
- Everything reconciles — one averaging window drives units, revenue, ads and profit so
price × units = revenuealways holds.