This might seem a lot for a small site, however it isn’t much for big web-sites. On the other hand, if you have beside no targeted visitors from these search engines like google and yahoo, it might be a great way to save some bandwidth. The Crawl-delay directive is surely https://rankingseotoolz.com/robots-txt-generator