Case study

A daily price monitor for 25 US online stores

Every price, sale and stock change from 25 US online stores, checked every day and delivered as a clean spreadsheet. Built and run in public by the founder of SteadyData.

  • Portfolio project · public data · not client work
  • By Siddharth Kumar, Founder of SteadyData
  • Published

What problem does it solve?

Competitor prices change every day, and checking stores by hand misses the day a sale starts, how deep it goes and when it ends.

The monitor answers that for 25 US online stores in five categories: audio, apparel, beauty, home and kitchen, and outdoor. All of them run on Shopify.

How does it work?

  • CollectThe public product catalog of each store, read slowly, with each store's rules respected.
  • CheckEvery price must be above zero and a sale price below its "was" price. One product per store is compared with a second source on the same store, and every run proves the store showed US dollars.
  • StoreEvery change is kept with the time it happened, so you can see what any product cost on any day.
  • DeliverThe latest prices as CSV, a daily change log, a monthly history file and a live dashboard.

What are the real numbers?

In the run of 20 November 2026, all 25 stores were updated and 11,767 product options were delivered, and every sampled price matched a second source.

  • 25 / 25stores updated
  • 11,767product options delivered
  • 25 / 25second-source checks matched exactly
  • 25 / 25stores proved US dollars
  • 0rows set aside as doubtful
  • 270free gifts and samples left out on purpose
  • 106page requests in 6.8 minutes
  • 0retries, and no store blocked the run
Changes found in the run of 20 November 2026
ChangeCount
Price drops99
Price rises24
Sales started61
Sales ended23
Sold out44
Back in stock51
New30
Removed59

From 4 to 20 November 2026 the run log has 17 daily runs: 12 with every store updated and 5 partial runs, where up to 5 stores could not be read and kept their last good prices, marked as not updated. No run was held.

See today's numbers on the live data page

What broke and how was it fixed?

Real problems from building and running it. Each one is written up in the public incident log.

  1. INC-001, INC-004, INC-005 · 3 to 4 Nov 2026

    Too fast for a shared host

    Each store was read slowly, but all of them run on Shopify's servers, and together the requests were too fast. Shopify asked the run to slow down, then started blocking it.

    Fix: One pace for the whole run. When Shopify asks to wait, every store pauses together, the pace slows and only eases back after a run of clean answers, and the run stops itself if several stores push back at once.

  2. INC-003 · 4 Nov 2026

    Free gifts looked like errors

    The first full test run rejected 255 rows, every one priced at $0: free gifts and samples. The checks treated them as broken data and held the whole run.

    Fix: Free items are left out on purpose and counted separately (270 in the run of 20 November). A missing or unreadable price is still rejected.

  3. INC-006, INC-007 · 4 to 6 Nov 2026

    The daily schedule started late, then not at all

    On 4 November GitHub started the scheduled run 8 hours 18 minutes late. On 6 November it created no scheduled run at all; a manual run filled the day with 25 of 25 stores.

    Fix: A separate watchdog checks twice a day whether the day's run happened. If not, it starts the run, opens an alert and fails loudly the same day. A late run whose slot is already filled skips, so no store is read twice.

  4. INC-008 · 5 Nov 2026

    Moving a tag re-ran a live check

    Pushing a moved release tag triggered a live check against all 25 stores, which added load to the stores for nothing.

    Fix: Live checks now run on branch pushes only, and a test keeps tags out.

Can SteadyData run this for my competitors?

Yes. The same method runs as a SteadyData monitoring plan for the competitor stores you choose, daily or weekly. Send 1-3 competitor links → get up to 1,000 rows + a sample change report within 48 hours.

What else do readers ask?

Is this client work?

No. It is a portfolio project on public data, built and run in public by the founder of SteadyData. No client or client data is involved.

How fresh is the data?

Every store is read once a day, and each row shows when its current price was first seen. The public run log records every run.

What happens when a store changes its site?

A run that looks wrong is held and an alert goes out, while the last good data stays live. The collector is fixed and the next run fills the gap.