For retailers
ZETO shows people the sale items, in their size, from the online stores they follow. Shoppers add a store by pasting its address; ZETO then reads the store's public terms page before it reads anything else, records what it says, and lists the store only after a person has reviewed it.
Getting listed
Any of these lets ZETO list a store's sale items:
- A product feed — a Google Shopping or affiliate feed ZETO can read on a schedule. Feeds are the fastest path and cost your site no crawl at all.
- A robots.txt allowance for the crawler plus a written yes to the address below:
User-agent: curatebot Allow: /
- A store whose robots.txt allows the crawler needs nothing from you — its public terms are read, recorded and reviewed by a person, and it is listed once that review is done.
None of this guarantees a listing: every store is read and reviewed the same way, and a store ZETO cannot read reliably stays off until it can.
What ZETO reads first, and honours
- The store's public terms page, before any product page and about once a month after that. What it says is recorded and reviewed by a person; robots.txt is what the crawler obeys. A store ZETO stops reading tells the people who follow it that it isn't available.
- An email to the address below saying no. That stops ZETO reading the store, and its items come off ZETO.
- robots.txt, Crawl-delay, 403s and challenges — the crawler page below describes exactly how.
What is and is not stored
- Stored: the product facts a shopper compares — name, brand, price and sale price, currency, sizes and availability, image addresses — and a price history so a follower can see when an item went on sale.
- Not stored: full pages, reviews, reviewer names or ratings, and nothing from behind a login, a cart or a checkout.
- Marketplaces and classifieds are never crawled — those are reached only through their own APIs and data providers.
Contact
Feeds, permissions, questions and opt-outs: tcmprog@gmail.com. The crawler's own identity, pace and opt-out details are at https://shopcur8.com/crawler.