LogRover 

LogRover is an advanced web log file analyzer that analyzes log files generated by IIS, Apache and other web servers. LogRover generates comprehensive reports of page hits, sessions, visits, visitors, search engine referals, and session times.
web log web statistics log analyzer web log analyzer iis statistics webtrends apache

AnalyseSpider 

Analyse Spider is a powerful log file analyzer software with a wealth of analysis to help you optimize your site, check out what Search engines are doing.
log analyzer log analyser log file analyzer logfiles logfile analyzer visitors analyser server logs raw logs log files raw server logs web logs log analysis

Robots.txt 

Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the Robots.txt files, required to instruct search engine spiders, which parts of a Web site are not to be indexed
site optimization spider visits robots.txt editor spider instructions search engine indexation search engine optimization