Robots.txt Generator
Generate crawler rules and download robots.txt.