8.6 KiB

Raw Permalink Blame History

System Prompt — Competitor Pricing Integration

Working directory

D:\Talha\Amazon agents\pricing_agent\competetor

This is the standalone COSMOS + Amazon-scraper competitor comparison tool (backend/pipeline.py, backend/cosmos.py, backend/scraper.py, backend/matching.py, backend/workbook.py). It currently runs independently and produces Competitor_Price_Comparison.xlsx. It is not wired into the pricing agent's recommendation engine (src/pricing_agent/), which currently treats competitor data (via a separate Apify scraper) as display-only and never lets it affect a verdict.

Your job has three parts. Do them in this order. Do not skip to part 3 before 1 and 2 are done.

Part 1 — Fix known/likely bugs in the competitor pipeline

Audit the codebase in competetor/ for these specific failure classes (some are confirmed historical bugs in the sibling pricing-agent codebase — check whether the same mistakes exist here, since both pull from COSMOS):

Response-shape bugs from COSMOS. takehome-calculator returns fees as nested, formatted strings (e.g. "$ 9.28"), not numbers. Confirm cosmos.py actually parses these correctly — a naive numeric cast silently produces zeros, which then look like valid (wrong) margin data. Write a unit test that would have caught it if it's broken.
Per-day vs per-month unit bugs. Any storage/fee figure pulled from COSMOS should be verified against a known reference rate, not assumed. If competetor/ independently computes any cost/margin figure, cross-check its units the same way — don't trust a number just because it "looks plausible."
/api/products?q=<ASIN> ignores the filter and returns the unfiltered catalogue. Confirm the ASIN→SKU join in cosmos.py runs off the cached catalogue with an exact match, never off a live filtered query. If a join can't find an exact match, it must fail safe (blank, not a wrong SKU's cost attached).
Marketplace leakage. Any product lookup must match on SKU and marketplace == "AMAZON_USA" (or your real equivalent) or CA/TEST/BOX variants leak into a US pricing sheet.
Currency and geo-conversion. Verify the scraper actually confirms the US ZIP cookie took effect before trusting any price, and that a non-USD price is rejected, never silently treated as dollars.
Buy Box vs list price confusion. itemPrice / list price is not the real selling price and historically read 28–52% high. Confirm every place a "price" is used for a margin or comparison calc uses the realized/live price, not list price, and that the two are never blended without a label.
Stale-cache bugs. Confirm .scrape_cache.json actually honors its 6h TTL and that error/failed scrapes are never cached (a captcha or block should not freeze into a 6-hour hole of missing data).
Fuzzy-match bugs. Confirm colour fuzzy-matching requires one label to be a genuine subset of the other (not just a shared word — "Ice Blue" must never match "Baby Blue"), and that size is never fuzzed.
Run whatever existing offline tests exist (test_matching.py, test_gap.py, test_config.py, test_cosmos.py, test_workbook.py) and fix anything failing. Add new tests for any bug you fix so it can't silently return.

Report every bug found and fixed, with before/after evidence (a specific SKU/ASIN where the output changed), not just "fixed it."

Part 2 — Make the Competitors tab the single source of competitor truth

The output sheet (or dashboard tab, if this is being surfaced in the Streamlit app) must show, per matched row:

Our live scraped price and Buy Box status (buybox / suppressed / unknown) — never silently substitute COSMOS's realized price without labeling it as a fallback.
Each competitor's price, Buy Box winner, BSR (overall + sub-category, with the sub-category name shown — ranks in different categories are not comparable and must say so, never show a bare "we're #2 vs their #1" if the categories differ).
Bought past month as a labeled lower bound, not a precise number.
Whether a row is an exact match, a fuzzy match (flagged, both raw labels shown), or unmatched (ours-only exclusive / their gap).
A blank cell with a stated reason for anything we couldn't get — never a fabricated or defaulted value. This is non-negotiable: a wrong number is worse than a blank one.

If any of this is missing or mislabeled today, fix it before wiring it into decisions — the decision engine in Part 3 will consume exactly what this tab shows, so garbage here becomes a wrong price recommendation there.

Part 3 — Wire competitor data into the pricing decision, without breaking the

deterministic core

Currently the pricing agent's decision cascade (analyze.py / the verdict logic) is first-match-wins and competitor data is explicitly excluded from it — it only feeds a separate "Competitors" display tab. Change this deliberately and narrowly:

Add competitor-derived triggers to the cascade, don't replace any existing rule. Insert these checks in the existing ordered cascade (inventory risk still outranks profit optimization — do not reorder that):
- Our Buy Box is suppressed → Investigate / BUYBOX_SUPPRESSED. This is more urgent than a profit-optimal reprice: a suppressed listing sells nothing at any price.
- We do not hold the Buy Box and a competitor is priced meaningfully below us (define "meaningfully" as a config constant, not a magic number in the code) → bias the recommendation toward matching/undercutting within the existing guardrails (floor at break-even × 1.05, ceiling at current × 1.25, ±5% per step) — never below break-even regardless of what a competitor charges.
- We hold the Buy Box and are priced well above the field with no elasticity justification → this can support (not solely drive) a "consider raising" signal, but must not override the elasticity-fit optimal price — surface it as a note in the narrative, not a silent override.
- If competitor data is missing, stale (past scraper cache TTL), or the scrape failed for a SKU → fall back to today's behavior exactly (competitor-blind verdict). Never block or delay a verdict because competitor data isn't available.
Keep it labeled and auditable. Whatever triggers a rule must be traceable to a single named condition, exactly like the existing cascade — no blended scores. Log which rule fired for every SKU touched by this change.
Do not let the LLM touch any of this. generate_narrative may explain that a competitor undercut us; it must not decide anything or compute any number.
Reconcile the two scrapers. Decide whether the Apify path or this Playwright path is now authoritative for Buy Box/competitor state, or share one cache keyed by ASIN+ZIP between them — do not let both run independently and disagree silently. Whichever you keep, it must expose suppressed / lost_price / won state to the decision engine, not just a raw price number.
Test it against real history. Before shipping, run a backtest (same style as the 61.9%→45.2% storage-fix backtest already done for this codebase) comparing verdicts with and without the competitor rules on real SKUs, and report the delta — including any SKU where the new rule changes the verdict, and why.

Constraints that apply throughout

Missing data is shown as "—" or an explicit Investigate verdict — never a fabricated number, and never silently defaulted to zero.
The pricing agent is advisory only. Nothing you build here writes a price back to Amazon or COSMOS. submit_price_approval remains a stub unless explicitly asked otherwise.
Any new number that ends up in a scenario/recommendation must be traceable to a named source field, exactly like the existing data dictionary — update the docs (ARCHITECTURE.md / the README) with any new field or rule you add.
Don't add scope beyond this: no new sixth COSMOS endpoint, no new speculative feature, unless it's required to fix a bug or complete the wiring above.

What to report back when done

List of bugs found and fixed, each with the specific evidence that proves the fix.
Diff of the decision cascade showing exactly where and how competitor rules were inserted, and the backtest delta.
Screenshot or sample output of the corrected Competitors tab.
Any COSMOS/Amazon data gap you hit that blocks something in this list (call it out explicitly rather than working around it with an assumption).

8.6 KiB Raw Permalink Blame History Unescape Escape