robots.txt tester

Fetch and inspect robots.txt for a site.