Can AI bots actually reach your site?
Your robots.txt states a policy. Your edge decides what really happens. This checks both and shows where they disagree.
Your robots.txt states a policy. Your edge decides what really happens. This checks both and shows where they disagree.