Fix Cloudflare blocking your crawl
Cloudflare's Bot Fight Mode or Under Attack Mode is blocking Tecsaro AI from reading your website. This is a 2-minute fix — follow the steps below.
What's happening
Cloudflare's Bot Fight Mode or Under Attack Mode is treating our crawler as a threat and blocking it before it can read your pages. Your site content is not accessible to us.
Login to Cloudflare
ACCOUNTGo to dash.cloudflare.com and sign in to your account.
Select your domain
DOMAINFrom the dashboard, click on the website domain that Tecsaro AI is trying to crawl.
Check Bot Fight Mode
BOTSGo to Security → Bots. If 'Bot Fight Mode' is ON, turn it OFF temporarily while we crawl your site. This is the most common cause of crawl failures.
Check Security Level
SECURITYGo to Security → Settings. If Security Level is set to 'Under Attack', change it to 'High' or 'Medium'. Under Attack mode blocks all automated requests.
Re-run the crawl
DONEGo back to your Tecsaro AI dashboard and click 'Re-crawl website'. The crawl should now complete successfully.
Re-enable protection (optional)
OPTIONALOnce crawling is complete, you can re-enable Bot Fight Mode. The crawl data is stored — we only re-crawl when you trigger it manually.
🔒 Prefer to whitelist our IP instead?
If you don't want to turn off Bot Fight Mode, we can send you our crawl server IP address. You can add it to Cloudflare's IP Access Rules to allow only our crawler through, keeping all other bots blocked.
Request our IP → [email protected]Common questions
Why is Cloudflare blocking the crawl?
Tecsaro AI crawls your website to extract page content, headings, and structure for AI visibility analysis. Cloudflare's Bot Fight Mode and Under Attack Mode block automated requests — including legitimate crawlers like ours.
Is it safe to turn off Bot Fight Mode temporarily?
Yes. Bot Fight Mode temporarily disabled does not remove your other security rules, firewall rules, or DDoS protection. Your site remains protected. You can re-enable it after the crawl completes.
What if I don't want to turn off Bot Fight Mode?
You can whitelist our server IP instead. Contact [email protected] and we'll send you the crawl server IP to add to your Cloudflare IP Access Rules.
I don't use Cloudflare — what else could block it?
Other security tools like Sucuri, Imperva, or custom nginx/Apache rules can block crawlers. If you're not on Cloudflare, contact [email protected] and we'll help diagnose the specific block.
Will this affect my other security rules?
Turning off Bot Fight Mode only affects Cloudflare's automated bot scoring. All your other WAF rules, firewall rules, rate limiting, and DDoS protection remain fully active.
Done with the fix?
Go back to your dashboard and retry the crawl — it should complete now.