Crawlers Software: Manage and track visits by robots/spiders, Saving bandwidth by fixing IIS etags., The Most Popular SiteMapper in the World !

Crawlers Software

Robot-Manager	ETagFix	CoffeeCup Google SiteMapper	Web Scraper Lite

Robot-Manager - Identify which spiders visited your website, when, and what pages were indexed

ETagFix - Reduce the need for user clients to recache your web pages and saving bandwidth.

CoffeeCup Google SiteMapper - Create Snazzy Google Sitemaps in Seconds!

Web Scraper Lite - Web extract / screen scraping and data mining with web spider / web crawler

	Web Scraper Plus+: Web Spider Edition
	Build a custom web spider / web crawler using web data extraction / screen scraping technology. Use the web extract for web data mining of contact lists, product catalogs, government databases, real estate listings, or build a custom email extractor. web spider web crawler web extract email extractor web grabber website spider screen scraping spidering screen scraper
	Robots.txt
	Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the Robots.txt files, required to instruct search engine spiders, which parts of a Web site are not to be indexed site optimization log file analysis spider visits robots.txt editor spider instructions search engine indexation search engine optimization