How do I install AI crawler tracking on my site?

AEOlens supports six integration methods: Cloudflare Worker, Vercel Edge Middleware, Next.js middleware (@aeolens/middleware npm package), Nginx mirror, Apache proxy, and a lightweight JavaScript beacon. Most setups take under 5 minutes.

AI CRAWLER ANALYTICSFREE ON ALL PLANS

See Which AI Bots Visit Your Site

Q: Which AI bots does AEOlens track?

AEOlens tracks 8 AI bot families: ChatGPT/GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended/Gemini, GrokBot (xAI), BingBot/Copilot (Microsoft), Meta-ExternalAgent, and other emerging AI crawlers. Each bot is verified via ASN lookup and reverse-DNS.

Q: Is AI crawler analytics free?

Yes. AEOlens offers AI crawler analytics on every plan. Free users get a live crawler feed, up to 3 tracked bot families, and 7-day history. Starter ($35/mo) unlocks the full dashboard — all 8 bot families, 30-day history, crawl trends, discovery insights, bot verification and managed proxy. Growth and Agency extend the history window (90 / 365 days) and Agency adds multi-domain analytics.

AI crawlers don't run JavaScript — so Google Analytics never sees them. AEOlens tracks GPTBot, ClaudeBot, PerplexityBot, and 5 more bot families server-side, with ASN verification. Free forever.

8 bot families · 25+ user-agent strings · ASN verified

Start Tracking Free Compare Plans →

No credit card · Live feed in under 5 minutes · Free on all plans

The blind spot

Google Analytics misses
100% of AI crawlers

GPTBot, ClaudeBot, and PerplexityBot don't execute JavaScript. They fetch raw HTML, parse it, and leave. No analytics tag fires, no pageview is recorded. Your most important new visitors are completely invisible.

AEOlens captures AI bot visits at the server level — before JavaScript even loads — and verifies each one by ASN and reverse-DNS. You see every real AI crawler, and zero spoofed traffic.

Google Analytics

Requires JavaScript execution

Zero AI crawler visibility

No bot verification

Can't distinguish bot families

AEOlens AI Crawler Analytics

Server-side detection — no JS needed

8 bot families, 25+ user-agents

ASN + rDNS verification

Free on all plans

Bot families

8 AI bot families tracked
and verified

Every major AI search engine has its own crawler. AEOlens identifies each one by user-agent string and verifies authenticity via ASN lookup and reverse-DNS — so you see only real AI traffic.

ChatGPT / GPTBot

OpenAI

GPTBot, OAI-SearchBot, ChatGPT-User

ClaudeBot

Anthropic

ClaudeBot, Claude-Web

PerplexityBot

Perplexity

PerplexityBot

Google / Gemini

Google

Google-Extended, Googlebot

GrokBot

xAI

GrokBot

Copilot / BingBot

Microsoft

BingBot, Copilot

Meta AI

From install to live feed
in under 5 minutes

STEP 01

Install

Add one integration to your site — Cloudflare Worker, Vercel Edge, Next.js middleware, Nginx mirror, Apache proxy, or a JS beacon.

STEP 02

Detect & verify

AEOlens identifies AI bot visits by user-agent, then verifies authenticity via ASN lookup and reverse-DNS. Spoofed traffic is flagged.

STEP 03

Monitor

View a live feed of AI crawler visits — bot family, page crawled, timestamp, and verification status. Track trends over 7 to 365 days.

Integrations

Works with any stack

Six integration methods to match your infrastructure. Most take under 5 minutes.

Cloudflare Worker2 min

Deploy a Worker that mirrors requests to AEOlens. Zero-config, works with any origin.

Vercel Edge3 min

Add edge middleware that detects bots and sends a beacon. One file, no latency impact.

Next.js Middleware2 min

Install @aeolens/middleware from npm. Wrap your middleware and you're done.

Nginx Mirror5 min

Add a mirror directive that forwards AI bot requests. No app code changes.

Apache Proxy5 min

RewriteRule that proxies AI bot requests to the AEOlens beacon.

JS Beacon1 min

Lightweight script tag for quick setup. Catches bots that execute JavaScript.

Plans

Free to start, powerful to scale

AI crawler analytics is included on every plan. Free users get started immediately — upgrade for deeper intelligence.

Feature	Free	Starter	Growth	Agency
Live crawler feed	✓	✓	✓	✓
Bot families tracked	3	All 8	All 8	All 8
Crawl history	7 days	30 days	90 days	365 days
Managed proxy	—	✓	✓	✓
Crawl trends & analytics	—	✓	✓	✓
Discovery insights	—	✓	✓	✓
Bot verification engine	—	✓	✓	✓
Multi-domain analytics	—	—	—	✓
Price	$0	$35/mo	$99/mo	$249/mo

Start Tracking Free

Goes further

From crawl data to AI visibility

AI crawler analytics is one part of the AEOlens platform. Pair it with structural audits and buyer simulations to close the loop.

AEO & GEO Audit

48 structural checks across content, schema, crawler access, and answer-readiness. Score your AI visibility and get a prioritized fix list.

Learn about AEO

AI Citation Tracking

Measure citation rate, position, and sentiment across ChatGPT, Gemini, Claude, Perplexity, and Grok. Know when AI recommends you — or a competitor.

Learn about citation tracking

AI Search Optimization

The complete guide to optimizing for AI search engines. Covers AEO, GEO, and the structural signals that drive AI visibility.

Read the guide

FAQ

AI Crawler Analytics — Questions Answered

What is AI crawler analytics?

AI crawler analytics is the practice of detecting, verifying, and monitoring visits from AI search engine bots — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, GrokBot, Copilot, and Meta-ExternalAgent — to understand how AI engines discover and index your content. It answers questions like: which AI bots visit my site, how often, and which pages do they read?

Why can't Google Analytics detect AI crawlers?

Most AI crawlers do not execute JavaScript. Since Google Analytics and similar tools rely on a JavaScript tag firing in the browser, AI bot visits are completely invisible to them. AEOlens uses server-side detection at the middleware or proxy level — catching every AI bot request regardless of whether it runs JavaScript.

Which AI bots does AEOlens track?

AEOlens tracks 8 AI bot families covering 25+ individual user-agent strings: ChatGPT/GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended/Gemini, GrokBot (xAI), BingBot/Copilot (Microsoft), Meta-ExternalAgent, and other emerging AI crawlers. Each visit is verified via ASN lookup and reverse-DNS to filter spoofed requests.

Is AI crawler analytics really free?

Yes. AEOlens includes AI crawler analytics on all plans, including Free. Free users get a live crawler feed, the top 3 tracked bot families, and 7-day history. Starter ($35/mo) unlocks the full dashboard — all 8 bot families, 30-day history, crawl trends, discovery insights, bot verification and managed proxy. Growth extends the history window to 90 days; Agency goes to 365 days and adds multi-domain analytics.

How do I install AI crawler tracking?

AEOlens supports six integration methods: Cloudflare Worker, Vercel Edge Middleware, Next.js middleware (via the @aeolens/middleware npm package), Nginx mirror, Apache proxy, and a lightweight JavaScript beacon. Most setups take under 5 minutes. Integration instructions are shown in your dashboard after sign-up.

How does bot verification work?

Every detected visit is verified in two steps: ASN (Autonomous System Number) lookup confirms the request originates from the expected network (e.g. OpenAI's AS for GPTBot), and reverse-DNS confirms the hostname matches known bot patterns. Visits that fail verification are flagged as unverified, so you see only genuine AI crawler activity.

What's the difference between AI crawler analytics and AI citation tracking?

AI crawler analytics tells you which AI bots visit your site and what they read — it's about discovery and indexing. AI citation tracking tells you whether AI engines actually cite your brand in their responses — it's about output and visibility. AEOlens offers both: crawler analytics (free) shows the input side, and AI Buyer Simulation (Growth+) shows the output side.

Start tracking AI crawlers today

Free on all plans. Set up in under 5 minutes. See which AI bots visit your site and what they read.

Start Tracking Free Integration Docs →