1. The two OpenAI crawlers
GPTBot
Training-time ingestion. Content crawled here may shape future model behaviour. Opt-in per site policy.
OAI-SearchBot
Live retrieval for ChatGPT Search. Content served here appears as inline citations in ChatGPT answers.
ChatGPT-User
The on-demand user-agent when a specific ChatGPT user browses to your URL. Rare, but distinct.
robots.txt authority
All three respect robots.txt. Rules are per-user-agent — no wildcard shortcut.
2. How ChatGPT chooses citations
- Passage-level extraction — a strong self-contained paragraph is easier to cite than an idea spread across five.
- Semantic HTML wins — actual <ul>, <ol>, <table>, <dl>, <blockquote>. Div soup gets deprioritised.
- Fresh content wins for time-sensitive queries; authoritative evergreen content wins for stable ones.
- Explicit product / brand / entity mentions win over pronoun-heavy prose.
- One-sentence answers to common questions win. State the answer, then justify it.
3. Earning product recommendations
When ChatGPT recommends a product, it composes from multiple retrieved pages. Winning that recommendation is a function of: how often your product is co-mentioned with the query intent, how confidently the retrieval pages state your product's positioning, and how many corroborating sources agree.
The tactical play: publish comparison pages, integration pages, use-case pages and feature pages that state your product's positioning in canonical, extractable prose. ChatGPT recommends what the corpus consistently recommends.
4. How WBP configures for ChatGPT
WBP's default robots.txt Allows both GPTBot and OAI-SearchBot. The AI Tuning Score checks passage-extractability, semantic HTML depth and canonical entity mentions per URL. The recommendation-earning play — comparison / integration / use-case / feature templates — is a first-class Bulk Publisher template pack.