# Utopia Pricing Agent โ€” Architecture A one-page Streamlit dashboard (Utopia/CRAI design system) that turns **live COSMOS data** into price recommendations a human can Approve / Modify / Reject. Read-only: nothing is written back to COSMOS or Amazon. --- ## 1. High-level view ```mermaid flowchart LR U["๐Ÿง‘ User
(same-network browser)"] --> APP subgraph APP["app.py โ€” presentation (Streamlit, CRAI theme)"] SIDE["Sidebar
Single product / Product line
+ filters, kill switch"] QUEUE["Recommendation queue
tiles ยท pills ยท rows"] SECT["Per-SKU sections
price ยท inventory ยท scenarios
competitors ยท PPC ยท costs ยท AI"] end subgraph DASH["dashboard/ package"] THEME["theme.py
CRAI tokens + plotly template"] LIVE["live_data.py
adapter + decision engine
+ scenario economics"] end subgraph CORE["src/pricing_agent โ€” analysis core"] AN["analyze.py"] MARGIN["margin_engine.py"] ELAST["elasticity.py"] PERF["performance.py"] SVC["cosmos/service.py"] CLIENT["cosmos/client.py"] end COSMOS[("COSMOS API")] APIFY[("Apify โ€” optional")] APP --> THEME APP --> LIVE LIVE --> AN AN --> MARGIN & ELAST & PERF AN --> SVC --> CLIENT --> COSMOS LIVE --> SVC AN -.optional.-> APIFY ``` --- ## 2. Layers | Layer | Files | Responsibility | |---|---|---| | **Presentation** | `app.py` | All rendering, zero pricing logic. Session state (approve/modify/reject, filters, per-SKU section + window), staged progress loader, session-state cache. | | **Design system** | `dashboard/theme.py`, `.streamlit/config.toml` | CRAI palette (cream `#f4f0e8`, teal `#0c8276`, coral `#df4f33`, navy `#22304e`), Inter font, plotly template. | | **Adapter + engine** | `dashboard/live_data.py` | Builds the per-SKU dict; **decides** the action; computes **scenario economics** (elasticity projection โ†’ bulk reconciliation โ†’ calibration); exposes `scenarios_for_window()`. | | **Competitive state** | `src/pricing_agent/competitive_state.py` | The one competitive fact the cascade may read. Adapts either scraper into a typed `WON`/`LOST_PRICE`/`LOST_ELIGIBILITY`/`SUPPRESSED` state with a source and a timestamp, gates it on age, and logs disagreement between sources. | | **Analysis core** | `src/pricing_agent/analyze.py` | Orchestrates one SKU: fees โ†’ trend โ†’ bulk โ†’ ad cost โ†’ elasticity โ†’ actual-profit evidence. | | **Money math** | `tools/margin_engine.py` | Pure: break-even, MAP, contribution margin, suggested price. | | **Statistics** | `elasticity.py`, `performance.py` | Log-log elasticity fit, profit-optimal sweep, actual-profit aggregation. | | **Data access** | `cosmos/{client,service,models}.py` | Auth + retry client; endpoint calls + response flattening; typed pydantic models. | --- ## 3. Data sources โ€” what each COSMOS endpoint feeds ```mermaid flowchart TB subgraph COSMOS["COSMOS API"] TH["/sales-insight/takehome-calculator
nested fees.breakdown ยท cost.breakdown"] INVP["/invp-insight
trend + inventory + dateMap PROJECTIONS"] BULK["/sales-insight/bulk-calculator
storage + total take-home"] SI["/sales-insight (daily, 6-month)
price ยท units ยท revenue ยท profit ยท ad spend"] PROD["/products
brand ยท marketplace"] CAMP["/api/campaigns ยท /adsApi
budget ยท ACoS ยท ad sales (not yet wired)"] end TH -->|"_flatten_takehome()"| FEES["Fee model
referral% ยท FBA ยท landed ยท returns"] INVP --> TREND["Velocity + cover days"] INVP --> INVPROJ["Inventory Outlook tab
real weekly units/value/cover/arrivals"] BULK --> STORAGE["Storage + take-home (scenarios)"] SI --> HIST["180-day daily series
(window filter + calibration)"] SI --> ADS["Ad spend / TACoS (PPC tab)"] PROD --> META["Brand / marketplace"] ``` **Two response quirks handled:** - **Fees come nested** (`fees.breakdown["Referral Fee"]`, `"$ 9.28"` strings). `service._flatten_takehome()` normalises them โ€” without it every fee parsed to 0 (the old "$0.99 / break-even $0" bug). - **INVP `dateMap`** holds COSMOS's own **forward inventory projection** (weekly units, value, cover days, warehouse arrivals). The Inventory tab renders this directly โ€” not a locally-invented forecast. Genuinely **not** in this COSMOS integration โ†’ shown as "โ€”", never faked: ad-attributed sales / ACoS / campaign budget (live in `/api/campaigns` + `/adsApi`, not yet wired), and historical competitor prices (Apify gives a current snapshot only). **Competitor data is no longer display-only.** COSMOS has no Buy Box, no rival price and no third-party offer anywhere in it โ€” that gap is filled by a scrape, and as of the competitor wiring two of its facts (our Buy Box being suppressed, and a rival materially undercutting us) reach the verdict. They are the only two, they are bounded by the guardrails, and their absence changes nothing. See ยง6. --- ## 4. Per-SKU pipeline (one "Analyze") ```mermaid sequenceDiagram participant U as User participant A as app.py participant L as live_data.build_live_sku participant AN as analyze.py participant S as CosmosService U->>A: Single product / Product line A->>L: get_live_data(skus, progress_cb) Note over A: staged progress bar (3โ†’10โ†’45โ†’82โ†’94โ†’100%) L->>S: get_current_price L->>S: get_sales_history (180d, parallel windows) L->>AN: analyze_price (fees, trend, bulk, elasticity, evidence) L->>S: get_invp (real inventory projection) L->>S: bulk_quote (storage) L->>L: decide action + scenario economics + 30d/6mo calibration L-->>A: {summary, details, errors} (session-cached) A-->>U: queue + expandable per-SKU analysis ``` --- ## 5. Scenario economics (the heart of the Scenarios tab) For each candidate price, one consistent chain: ```mermaid flowchart LR W["Window filter
7/14/30/90d ยท 6mo"] --> BASE["Baseline velocity
= avg units/day in window"] BASE --> DEMAND["Units(p) = units ร— (p/cur)^elasticity"] DEMAND --> REV["Revenue = units ร— p ร— 30"] REV --> AD["Ad spend = TACoS ร— revenue"] DEMAND --> TH["Take-home (bulk calculator fee model)"] TH --> GROSS["Gross = take-home โˆ’ storage โˆ’ ad"] GROSS --> CAL["ร— realization factor
(actual booked รท modeled at current)"] CAL --> NET["Net profit / 30d"] ``` Key rules: - **Current row = FACT**, not a projection: real units, real revenue, real ad spend, real booked profit. Its price is the **average price sold** (revenue รท units) so `price ร— units ร— 30 = revenue` reconciles โ€” this is *below* list when promos ran, and changes with the window because the avg selling price differed period to period. The **list price is fixed**. - **Calibration:** raw bulk-calculator profit over-states reality (prices at list, ignores real returns/promos). A **realization factor** = actual booked profit รท modeled profit at the current price scales every projected row, anchoring net profit to what the SKU truly earns. - **Suggested price** (teal callout) is computed from the **full 6-month** window always โ€” stable โ€” independent of the display-window filter. - **โญ** marks the highest-net-profit price in the current view. --- ## 6. Decision engine (deterministic, first match wins) ```mermaid flowchart TD S([signals]) --> R0{cost data = 0?} R0 -- yes --> INV["๐Ÿ” INVESTIGATE ยท NO_COST_DATA"] R0 -- no --> RB{our Buy Box suppressed?} RB -- yes --> INVB["๐Ÿ” INVESTIGATE ยท BUYBOX_SUPPRESSED"] RB -- no --> R1{price < break-even?} R1 -- yes --> UP1["โ†‘ raise to safe floor ยท BELOW_BREAK_EVEN"] R1 -- no --> R2{losing money after ads?} R2 -- yes --> UP2["โ†‘ raise ยท LOSING_MONEY"] R2 -- no --> R3{cover < 35d?} R3 -- yes --> UP3["โ†‘ +5% ยท LOW_STOCK"] R3 -- no --> R4{โˆ’30% sales, no cause?} R4 -- yes --> INV2["๐Ÿ” INVESTIGATE ยท UNEXPLAINED_DROP"] R4 -- no --> R5{cover > 90d?} R5 -- yes --> DN["โ†“ โˆ’5% ยท EXCESS_STOCK"] R5 -- no --> RC{lost Buy Box AND rival โ‰ฅ3% below?} RC -- yes --> DNC["โ†“ toward rival ยท COMPETITOR_UNDERCUT"] RC -- no --> R6{profit-optimal โ‰  current?} R6 -- yes --> MOVE["โ†‘/โ†“ toward optimal ยท PROFIT_OPTIMAL"] R6 -- no --> HOLD["โ†’ MAINTAIN ยท NO_SIGNALS"] ``` Guardrails: floor = break-even ร— 1.05, ceiling = current ร— 1.25; the recommended move is capped at ยฑ5% (bigger steps need elevated approval); the kill switch pauses all approvals. ### Competitor rules โ€” the two that can move a price, and what bounds them Both branches read a single `CompetitiveState` ([competitive_state.py](src/pricing_agent/competitive_state.py)), never a raw scrape: | Rule | Fires when | Effect | |---|---|---| | `BUYBOX_SUPPRESSED` | Amazon is not showing our offer | **Investigate, hold.** Placed directly under `NO_COST_DATA`: those are the only two states where the answer is "go and fix something" rather than "set a price". A suppressed variant sells nothing at any price, so its margin and its modelled optimum both describe a listing nobody can buy from. | | `COMPETITOR_UNDERCUT` | `LOST_PRICE` **and** cheapest rival โ‰ฅ `competitor_undercut_material_pct` below us | **Decrease toward the rival**, floored and step-capped like every other branch. | A third signal, a **competitor premium** while we hold the Buy Box, is a narrative note only. It never sets a price and never changes an action โ€” the elasticity fit is the thing with evidence behind it, and a premium is not grounds to overrule it. Ordering is deliberate: **inventory risk still outranks competitor position, which outranks profit-optimal.** Chasing a rival down while the shelf is emptying pays margin to sell out faster. This is visible in the backtest below โ€” two of five real SKUs did not move under a counterfactual undercut precisely because `LOW_STOCK` and `LOSING_MONEY` fired first. Three properties make this safe to ship: 1. **Fail-safe.** Absent, failed, stale (> `competitor_state_max_age_hours`) and "ownership unknown" all collapse to one flag, and the cascade then computes exactly the verdict it computed before competitor data existed. Nothing waits on a scrape; nothing is blocked by one. Competitor data can only ever *add* a verdict. 2. **Never below break-even.** The rival price is a *candidate* (`comp_match`), not a decision. Verified against real SKUs with a counterfactual rival 60% below us: against a $14.88 floor the shipped recommendations were $15.19โ€“$21.84, because the ยฑ5% step cap binds first. Zero violations. 3. **One named reason per verdict.** No blended scores โ€” every fired rule is traceable to a single reason code, and `logger.info` names the SKU, the rule, the state and the source. Thresholds live in [config/pricing_rules.yaml](config/pricing_rules.yaml) (`competitor_undercut_material_pct: 0.03`, `competitor_premium_material_pct: 0.10`, `competitor_state_max_age_hours: 6.0`), not in code. The 3% floor sits above the ~2% band our own realized price already swings through as coupons toggle. ### Inventory cover matches COSMOS Inventory Planning `cover_days` **is COSMOS's own `coverDays`**, so the dashboard and the INVP grid never quote two different numbers for one SKU. COSMOS counts **inbound** stock against a **7-day** velocity, so it reads longer than what is on the shelf โ€” `UBMICROFIBERDUVETTWINWHITE` is 78 days on (3,999 on hand + 1,030 inbound) รท 64/day, against 63 on-hand-only. Both are reported: the tile leads with the matched figure and appends `63 d on hand, rest inbound`. The on-hand figure remains the **fallback**, because COSMOS returns `coverDays: 0` on some very low-velocity SKUs that hold months of stock (`UBMICROFIBERBS4PCFULLGREY`: 167 units, 334 real days, COSMOS said `0`). Zero satisfies neither inventory rule, so taken literally it silences both. **Trade-off, accepted deliberately:** stockout risk is now judged partly on stock that has not landed. Measured over the 56-SKU covered line, matching COSMOS moved 7 verdicts โ€” `LOW_STOCK` 7 โ†’ 4, `EXCESS_STOCK` 18 โ†’ 22. The one to watch is `UBMICROFIBERBS4PCKINGWHITE`: **12 days on the shelf, 84 with inbound**, so it no longer raises. If that shipment slips, nothing protects it. Display bands are COSMOS's Alpha/Beta scheme (`theme.COVER_BANDS`, Alpha 20/40/70/100). The pricing **triggers** are separate and live in `pricing_rules.yaml` (`low_cover_days: 35`, `high_cover_days: 90`) โ€” COSMOS's pink at 70 days is a *replenishment* warning, while crossing a trigger here spends margin on a 5% move. Adopting the band edges (40/70) would put six more SKUs on a discount; the measured table is in the config beside the values. ### Coverage: the comparison sheet gates competitor data, one product line at a time The competitor workbook currently covers **one product line**, so the engine reads it as the first competitor source and **gates on coverage**: | SKU | Competitor state | |---|---| | In the sheet | Priced from the sheet โ€” real like-for-like rival prices, `basis=like-for-like-sheet` | | Not in the sheet | **`N/A`**, naming what the sheet *does* cover. No rule fires; the verdict is byte-identical to the competitor-blind one | **Coverage is the exact SKU set in the sheet, not a line prefix.** Measured against the real workbook, a prefix gate would be wrong in both directions: the `UBMICROFIBERDUVET` run contains 49 `UBMICROFIBERDUVET*` SKUs **and 7 `UBMICROFIBERBS4PC*`** ones (variants come off the Amazon parent twister, and COSMOS maps those ASINs to whatever SKU codes they carry), while the line has 139 SKUs in COSMOS of which only 56 reached a comparison row. So the sheet's own SKU list is the authority, and "not in the sheet" is reported as a **coverage hole**, never as a claim that the SKU has no competitors. `competitor_sheet_only: true` (default while one line is under test) means an uncovered SKU gets N/A rather than falling through to a per-ASIN Apify scrape โ€” so every verdict either rests on the sheet or says it has no competitor data. Set it `False` once coverage is broad enough for Apify to be a sensible fallback. Config: `competitor_sheet_path` (blank = auto-discover the newest `Competitor_Price_Comparison_*.xlsx`), `competitor_sheet_dirs`, `competitor_sheet_max_age_hours: 168` (the sheet is a 25โ€“35 min batch run, not a live feed, so it gets a longer limit than the 6 h single-ASIN one). **Two bases, never conflated.** `competitor_min` can be a price of two different things, and the `basis` field records which: - `same-asin-buybox` (Apify) โ€” another seller's offer on **our own listing**. Only `LOST_PRICE` fires the undercut rule; a rival holding the Buy Box *above* us is `LOST_ELIGIBILITY`, where cutting donates margin. - `like-for-like-sheet` โ€” a **rival brand's** equivalent variant, matched on size + colour. A cheaper one fires the rule **regardless of who owns our Buy Box**: we can hold ours perfectly well while a different product undercuts us. This is what the comparison tool exists to report. It is never labelled a Buy Box loss. Two sheet-driven refinements, both from real rows: - **A rival whose own Buy Box is suppressed is excluded from the band.** Their price is not buyable, so undercutting it donates margin for nothing. - **A material undercut now explains a velocity drop.** Previously a drop with no *own* price change and no ad collapse was filed `UNEXPLAINED_DROP` even when the sheet held the explanation โ€” observed on `UBMICROFIBERDUVETKINGPURPLE` (rival 15.6% below) and `UBMICROFIBERBS4PCFULLGREY` (35.2% below). A named cause now converts the Investigate into an actionable verdict, exactly as the existing stockout branch already did. A stockout still explains a drop first; an immaterial rival explains nothing. Finally, a **suppressed listing has no selling price** (it sells nothing, so COSMOS records no sales), which used to fail with a bare "no current selling price found in COSMOS". That error now names the cause, so the most actionable rows in the sheet stop looking like a data problem. ### Two scrapers, one state | Source | Authoritative for | Why | |---|---|---| | **Apify** (`tools/amazon/apify.py`) | **Buy Box state read by the engine** | The only source carrying a seller id, so the only one that can tell `WON` from `LOST_PRICE` from `LOST_ELIGIBILITY`. Those lead to opposite actions (cut price vs. fix fulfilment eligibility). | | **Playwright** (`../scraper/`) | The workbook: like-for-like size/colour matching, BSR, demand buckets, SKU gaps | Apify cannot produce any of it. Its Buy Box field knows only whether a price *rendered*, not whose it was. | The split is by **question**, not preference, so neither source is redundant. `reconcile()` cross-checks the authoritative state against the Playwright run's own cache (`scraper/.scrape_cache.json`, keyed `ASIN@ZIP`) and **logs any disagreement** rather than letting a workbook and a recommendation contradict each other in front of a stakeholder. A disagreement never changes the verdict. Where the authoritative source has nothing usable but the other has a `SUPPRESSED`, that fact is promoted and the provenance recorded โ€” discarding it to preserve a hierarchy would be choosing the hierarchy over the fact. Backtest (`scripts/backtest_competitor_rules.py`, 5 real SKUs, real COSMOS data): | Arm | Verdicts changed | |---|---| | Real competitor state (both cached entries stale: 66h / 146h) | **0 / 5** โ€” the fail-safe working | | Counterfactual 8% undercut | 3 / 5 โ€” the other 2 blocked by `LOW_STOCK` / `LOSING_MONEY` | | Counterfactual suppression | 5 / 5 โ†’ Investigate | | Prices shipped below break-even, any arm | **0** | --- ## 7. Key formulas | Quantity | Formula | |---|---| | Take-home / unit | `pยท(1 โˆ’ referral% โˆ’ returns%) โˆ’ landed โˆ’ FBA โˆ’ other` | | Break-even | `(landed + FBA + returns + other) / (1 โˆ’ referral%)` | | Elasticity | OLS on `ln(units/day) = a + eยทln(price)` over 6 months | | Scenario demand | `units ร— (p / pโ‚€)^e` | | Realization factor | `actual booked profit (window) รท modeled net at current price` | | TACoS | `ad spend รท total revenue` (window) | | Avg sold price | `revenue รท units` (window) โ€” reconciles the Current row | --- ## 8. Repository map ``` pricing_agent/ โ”œโ”€โ”€ app.py # dashboard (presentation only) โ”œโ”€โ”€ legacy_app.py # previous analyst UI (still runnable) โ”œโ”€โ”€ dashboard/ โ”‚ โ”œโ”€โ”€ theme.py # CRAI design tokens + plotly template โ”‚ โ””โ”€โ”€ live_data.py # COSMOS adapter, decision + scenario engine โ”œโ”€โ”€ src/pricing_agent/ โ”‚ โ”œโ”€โ”€ analyze.py # per-SKU orchestration โ†’ AnalysisResult โ”‚ โ”œโ”€โ”€ competitive_state.py # canonical Buy Box state + two-scraper reconciliation โ”‚ โ”œโ”€โ”€ elasticity.py # demand model + profit optimizer โ”‚ โ”œโ”€โ”€ performance.py # actual-profit evidence โ”‚ โ”œโ”€โ”€ tools/margin_engine.py # pure fee/break-even math (golden-tested) โ”‚ โ””โ”€โ”€ cosmos/ โ”‚ โ”œโ”€โ”€ client.py # auth + retry HTTP โ”‚ โ”œโ”€โ”€ service.py # endpoints, _flatten_takehome, INVP projections โ”‚ โ””โ”€โ”€ models.py # typed COSMOS responses (pydantic) โ”œโ”€โ”€ config/ # settings + pricing_rules.yaml (incl. competitor thresholds) โ”œโ”€โ”€ scripts/ โ”‚ โ””โ”€โ”€ backtest_competitor_rules.py # verdict delta, competitor rules blind vs live โ”œโ”€โ”€ .streamlit/config.toml # CRAI theme โ””โ”€โ”€ tests/ # incl. margin-engine golden values and # test_competitor_rules.py (fail-safe + ordering invariants) ``` --- ## 9. Principles 1. **Deterministic core, narrative shell** โ€” every number is a formula over COSMOS data; language models only phrase explanations. 2. **Read-only** โ€” the agent proposes; a human approves; nothing writes back. `submit_price_approval` remains a stub with no callers. 3. **Honest gaps** โ€” missing upstream data shows "โ€”" or an explicit investigation, never a fabricated number. 4. **Facts vs projections are labeled** โ€” the Current row is real booked history; other prices are clearly modeled. 5. **Everything reconciles** โ€” one averaging window drives units, revenue, ads and profit so `price ร— units = revenue` always holds.