Operations

AEO allowlisting runbook

Use this runbook when configuring CDN, WAF, or bot-management rules so Pawalink's AI-facing pages and feeds remain reachable to approved answer-engine crawlers.

Allowlisted agents

Allow these user-agent tokens through the CDN, WAF, and bot-management layer: OAI-SearchBot, ChatGPT-User, GPTBot, PerplexityBot, Perplexity-User, ClaudeBot, Claude-SearchBot, Claude-User, and Google-Extended.

Do not rely on robots.txt alone. If the edge blocks or challenges these requests, the crawler policy is declared but not reachable.

Public paths

Keep these routes challenge-free for known AI agents: /api/v1/public/, /store/, /market/, /receipt/, /discover, /access-policy, /llms.txt, /trust-methodology, and sitemap.xml.

If bot-management uses allow-by-path rules, include both HTML surfaces and machine-readable feeds.

Verification

Verify robots.txt, llms.txt, access-policy, the public catalog index, and at least one storefront, product, trust, and market-category page from a clean network path after each edge change.

Verify the integrations manifest and AI-commerce onboarding guide too, so feed consumers and operators are testing the same published contract.

Watch the AEO dashboard for crawler-visit telemetry, sitemap coverage, and structured-data validation results after rollout.

Rollback

If allowlisting causes unwanted crawl volume, remove the edge exceptions first, then tighten robots directives second. Reversing that order leaves declared access wider than actual access.