The Growth Data Stack: Measure Before You Optimize
A question I hear from founders and marketing leaders almost every week: is a full export from Google Search Console and Bing Webmaster Tools enough to rebuild a website for search and AI engines? It is the right starting point, and it covers about half of what you need.
This chapter is the other half. It lays out the datasets a growth program needs, the one-day measurement setup to do before touching content, the evidence on what earns citations, and a scorecard you can run every month. The companion article, Growth Engineering for the AI Search Era, adds the full Claude Code workflow and the ten prompts that turn these exports into a plan.
Three measurement changes made AI search manageable
Until 2026, AI visibility was mostly a guess. Three first-party reports changed that.
| Date | Report | What it gives you | What it still hides |
|---|---|---|---|
| Feb 2026 | Bing Webmaster Tools AI Performance (public preview) | Citations in Copilot and Bing AI answers, cited pages, grounding queries; intent, topic and citation share since June | Clicks, and every engine outside Microsoft |
| May 13, 2026 | GA4 AI Assistant channel | AI chatbot referrals grouped as their own default channel | Visits with no referrer, which still land in Direct |
| Jun 3, 2026 | Search Console Generative AI reports (all sites since Aug 31) | Impressions in AI Overviews, AI Mode and Discover AI features, by page, country, device and date | Clicks, CTR, queries, and the AI Overviews vs AI Mode split |
In the same month, Google published its guide to optimizing for generative AI features, which says plainly that optimizing for its AI search is still SEO. The reports above are how you check whether that SEO is turning into AI visibility.
What Search Console and Bing answer, and where they go blind
| Question | Search Console + Bing? | What fills the gap |
|---|---|---|
| Which queries show my pages, at what CTR and position? | Yes | Nothing |
| Which URLs are 404, redirected, noindexed or not indexed? | Yes, for URLs Google and Bing know about | A full crawl finds orphan pages and internal links to broken URLs |
| Is my structured data valid? | Partly, for rich result types on crawled pages | A crawl that extracts JSON-LD from every page |
| Am I cited in Copilot? | Yes, through Bing AI Performance | Nothing |
| Am I visible in AI Overviews and AI Mode? | Impressions only | Prompt-level AI visibility tracking |
| Am I mentioned or cited in ChatGPT, Perplexity, Claude or Gemini? | No | A cross-engine AI visibility tool |
| Which pages create demo requests, trials and pipeline? | No | GA4 key events joined to CRM stages |
| Who links to me and to my competitors? | Barely: the Links report is a sample | Ahrefs, Semrush or Majestic |
| Do AI crawlers reach my important pages? | No | Server or CDN logs |
The Search Console interface caps performance exports at 1,000 rows. For real analysis, pull 16 months of data through the API or the BigQuery bulk export, or you will optimize against a truncated sample.
The complete growth data stack
Every dataset here earns its place by changing a decision. If a dataset would not change what you build, fix or delete, leave it out.
| Dataset | Typical source | Decisions it drives | Priority |
|---|---|---|---|
| Search performance, indexing, Generative AI report | Search Console (API or BigQuery) | Retitle, refresh, consolidate, fix indexing | Have it |
| Search performance and AI Performance | Bing Webmaster Tools | Which pages AI already trusts, and for which questions | Have it |
| Behavior and conversions | GA4 joined to your CRM | Where to invest by revenue, not traffic | Add first |
| Full site crawl | Screaming Frog, Sitebulb | Technical fixes Search Console cannot see | Add first |
| Backlinks and brand mentions | Ahrefs, Semrush, Majestic | Digital PR targets and which assets to build | Add first |
| Cross-engine AI visibility | An AI visibility platform | Which prompts to win and which sources to influence | Add first |
| Server or CDN logs | Cloudflare, Fastly, your host | AI crawler access and crawl waste | Recommended |
| ICP and competitor brief | You, written once | How every gap gets scored | Required |
| Voice of customer | Sales notes, tickets, churn reasons | Roadmap and bottom-of-funnel content | Recommended |
If you can add only one dataset this week, add GA4 joined to CRM outcomes. Without it, every other analysis optimizes for traffic, and traffic is not the goal.
A one-day measurement setup
Spend one day on measurement before you change any content. AI traffic is easy to undercount, and every later decision depends on these numbers.
- GA4. Confirm the native AI Assistant channel shows in Traffic acquisition, then add a custom channel group as a backup so new assistants count the day they send traffic. Mark demo requests, trials, contact forms and newsletter signups as key events. Pass the first landing page into your CRM.
- Self-reported attribution. Add "AI assistant (ChatGPT, Perplexity, etc.)" to your "How did you hear about us?" field. It catches the influence analytics misses.
- Search Console. Export 16 months by page and query through the API, plus the page indexing, sitemaps and Generative AI reports.
- Bing Webmaster Tools. Export AI Performance monthly. Treat grounding queries as a free list of questions AI already associates with your pages.
- Logs. Export 30 days of CDN or server logs and filter for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Googlebot and Bingbot. Look for important pages never fetched and errors served to bots.
Think hard before blocking AI crawlers. A Rutgers and Wharton study found news publishers that blocked them saw about 7% fewer weekly visits within six weeks. Decide bot by bot: training crawlers, search crawlers and user-triggered fetchers do different jobs.
What earns citations, according to the evidence
| Finding | Evidence | What to do |
|---|---|---|
| Maintained pages win | 75% of pages cited by AI engines were updated within the past year (Seer Interactive, 2026) | Refresh your best pages on a schedule |
| Earned media feeds AI answers | 84% of citations in 25 million+ links cited by ChatGPT, Claude and Gemini came from earned media (Muck Rack, May 2026) | Get covered where AI engines already look |
| Off-site authority counts | Sites with 32,000+ referring domains were 3.5 times more likely to be cited by ChatGPT than sites with up to 200 (SE Ranking) | Treat digital PR as part of AI visibility |
| Journalists research in AI | 82% of journalists use at least one AI tool (Muck Rack State of Journalism 2026) | If AI does not name you, reporters may not find you |
| AI visitors convert | ChatGPT visitors converted at 15.9% vs 1.76% for Google organic on one site (Seer Interactive) | Measure pipeline by landing page, not sessions |
What to skip
- llms.txt as a lever. Ahrefs found 97% of valid llms.txt files got zero requests in May 2026, and Google Search ignores them.
- Chunking and AI-specific rewrites. Google's guide lists both among the things you can ignore.
- Schema as a switch. Markup alone produced no meaningful citation lift in Ahrefs' matched test.
- Manufactured mentions. Google's guide lists seeking inauthentic mentions as a tactic to avoid.
- Rank as the only KPI. Track citations and pipeline next to rankings.
The scorecard and the cadence
| Metric | Source | Why it matters |
|---|---|---|
| Pipeline and leads by landing page | GA4 + CRM | The outcome every other metric serves |
| AI Assistant sessions and conversion rate | GA4 | Value of AI visibility in revenue terms |
| Share of voice on priority prompts, by engine | AI visibility tool | Whether engines recommend you or a competitor |
| Citations and cited pages | Bing AI Performance, AI visibility tool | Which pages AI systems trust |
| AI Overview and AI Mode impressions | Search Console | Exposure in Google's AI surfaces |
| Non-brand clicks and CTR | Search Console | Classic demand captured |
| New referring domains | Ahrefs or Semrush | Whether citation assets are working |
| Inaccurate AI claims open | AI visibility tool | Brand and sales risk |
Run it weekly for 30 minutes (new indexing errors, citation and share-of-voice changes, inaccurate mentions), monthly for the full refresh (fresh exports, 5 to 10 page refreshes, 2 to 4 gap pages), quarterly for one original research asset and a full crawl and schema review, and yearly for a benchmark report and a content prune.
For the ten Claude Code prompts that run this loop on your exports, see Chapter 10 of The Practical Guide to AI Search Visibility and the full article.
Key takeaways
- Search Console and Bing exports cover about half of a growth program: they show demand and technical health, not pipeline or what ChatGPT, Perplexity, Claude and Gemini say.
- Three 2026 reports made AI search measurable: Bing AI Performance (Copilot citations), GA4's AI Assistant channel (referrals) and Search Console's Generative AI reports (impressions).
- Add GA4 joined to CRM first. Without revenue data, every other analysis optimizes for traffic.
- The evidence favors maintained pages, original data and earned media. It does not favor llms.txt, chunking or schema as levers.
- Run a small scorecard on a weekly, monthly, quarterly and yearly rhythm. Growth is a loop, not an audit.