73 - 132 ArchitextSpider 3 Cartographer 1.3 33 Crawler V 0.2.1 admin@crawler.de 7 DIIbot/0.0 1 DIIbot/0.2 4 DIIbot/0.3 7 DIIbot/0.4 2 ESISmartSpider 8 FAST-WebCrawler/0.3 (crawler@fast.no) 1 FAST-WebCrawler/0.4 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html) 3 FAST-WebCrawler/0.5 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html) 1 FAST-WebCrawler/2.0.7 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html) 39 FAST-WebCrawler/2.0.9 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html) 4 gazz/1.0 (gazz@nttrd.com) 7 Googlebot/1.0 (googlebot(at)googlebot.com) 46 Googlebot/1.0 (googlebot@googlebot.com http://googlebot.com/) 31 Gulliver/1.2 27 Gulliver/1.3 12 htdig/3.1.0b4 (HSH@taxon.demon.nl) 3 htdig/3.1.0b4 (webmaster@javawoman.com) 1 htdig/3.1.2 (unconfigured@htdig.searchengine.maintainer) 1 h_RobotUA 29 ia_archiver 1 infoseek 5 InfoSeek Sidewinder/0.9 1 Iron33/1.0.2 1 Java1.1.8 3 Java1.2 118 KIT-Fireball/2.0 3 LavaSpider/0.1 5 LinkAlarm/1.04b 12 LinkAlarm/1.5 4 LinkAlarm/2.0 2 Links2Go Similarity Engine 57 LinkWalker 115 Lycos_Spider_(T-Rex) 1 Mediagent-Crawler-(http://mediagent.mip.nus.edu.sg) 42 Mercator-1.0 7 Mothra/126-Paladium 5 Mozilla 4.0/(compatible; MSIE 5.0; Windows 98) 60 Mozilla/2.0 (compatible; EZResult -- Internet Search Engine) 2 Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E) 5 Mozilla/3.0 (compatible; MuscatFerret/1.7; http://www.webtop.com/) 4 Mozilla/3.0 (compatible; PerMan Surfer 3.0; Win95) 1 Mozilla/3.0 (compatible; PerMan Surfer 3.0; Win95) via proxy gateway AnWeb/1.14 3 Mozilla/3.0 (compatible; Webinator-mandrake.oasis-systems.com/2.53; "I'll be back") 15 Mozilla/3.0 (compatible; Win95) 5 Mozilla/3.01C-PBWF (Win95; I) 3 Mozilla/3.01C-PBWF-ip3000.com-crawler 3 Mozilla/4.0 178 Mozilla/4.0 (compatible; MSIE 4.01; MSIECrawler; Windows 95) 2 Mozilla/4.0 (compatible; MSIE 4.01; MSIECrawler; Windows 95) via proxy gateway CERN-HTTPD/3.0 libwww/2.17 37 Mozilla/4.0 (compatible; MSIE 4.01; MSIECrawler; Windows NT) 2 Mozilla/4.0 (compatible; MSIE 4.01; MSIECrawler; Windows NT) via NetCache version 3.1.2d-Solaris 1 Mozilla/4.0 (compatible; MSIE 4.01; Windows NT) 17 Mozilla/4.0 (compatible; MSIE 4.0; MSIECrawler; Windows 95) 11 Mozilla/4.0 (compatible; MSIE 4.0; Scour.Net Crawler) 1 Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) CGA 1 Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) Level 3 Communications 1 Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) MHD 1 Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) Navisite 1 Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) PERIGIS 1 Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; MSIECrawler) 2 Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; MSIECrawler) 4 Mozilla/4.0 (compatible; MSIE 5.01; Windows NT; MSIECrawler) 1 Mozilla/4.0 (compatible; MSIE 5.0; MSN 2.5; Windows 95) 2 Mozilla/4.0 (compatible; MSIE 5.0; MSN 2.5; Windows 98; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; MSN 2.6; Windows 98; DigExt) 14 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) 2 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; CNETHomeBuild03171999) 22 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt; QXW0330m) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt; SQSW) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; Mozilla/4.0 (compatible; MSIE 4.01; Windows NT); YDP; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; QXW03003; QXW03304; DT) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; QXW0300i; QXW03304) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; QXW0300W; QXW0300Z) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; QXW0300W; QXW03301; QXW0330d; pi 2.1.0.0) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; SnapHomeBuildA03251999) 63 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98) 3 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; AT&T WNS IE4.0; DigExt) 2 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; AT&T WNS5.0) 2 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; AtHome0102; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; Compaq; DigExt) 74 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt) 2 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; AT&T WNS5.0) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; freenet200) 2 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; FREESERVE_IE4) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; MSIECrawler) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; QXW0330k) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; QXW03311) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; TUCOWS) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DT) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; ELO Internet - Sao Luis MA - Brasil - +55(098)2353906; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; KUA.net Customer; DigExt) 7 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; MindSpring Internet Services; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; pi 2.1.6.0) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; QXW03002; TalkNet; QXW0300e; Mozilla/2.0(Compatible; MSIE 4.01; AOL 3.0i; Windows 95); QXW0300i; 1u1ie4profi; QXW03301) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; QXW03302) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; QXW03304; talknet2.0) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; QXW0330d) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; TUCOWS; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; ZDNetSL) 5 Mozilla/4.0 (compatible; MSIE 5.0; Windows NT 5.0) 31 Mozilla/4.0 (compatible; MSIE 5.0; Windows NT) 15 Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt) 1 Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; HD 500) 1 Mozilla/4.0 (compatible; MSIE 5.0b2; MSN 2.5; Windows 98; MSIECrawler) 1 Mozilla/4.0 (compatible; MSIE 5.0b2; Windows 95; MSIECrawler) 1 Mozilla/4.0 (compatible; MSIE 5.5; Windows NT; MSIECrawler) 1 mspider/0.1 21 MSProxy/2.0 1 NetIntellect (3.0; WinNT; I) 3 NetMechanic 51 NinjaBot/1.21 3 nttdirectory_robot/0.9 (super-robot@super.navi.ocn.ne.jp) 2 Openfind Robot/1.1A2 3 Robot@SuperSnooper.Com 1 ryan-spider/0.1 (Under development. Contact spider-devel@altavista.net Not affiliated with AltaVista in any way.) 1 Scanner 2.001 3 Scooter/1.0 157 Scooter/2.0 G.R.A.B. V1.1.0 26 sherlock/1.2 httpget/1.2 213 Slurp.so/1.0 (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 69 Slurp/2.0 (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 2 Slurp/2.0-ash_dev (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 2 Slurp/2.0-DN (david.nadler@inktomi.com; http://www.inktomi.com/slurp.html) 16 Slurp/2.0-KiteHourly (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 43 Slurp/2.0-KiteWeekly (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 1 Slurp/2.0-OwlDaily (spider@aeneid.com; http://www.inktomi.com/slurp.html) 13 Slurp/2.0-OwlWeekly (spider@aeneid.com; http://www.inktomi.com/slurp.html) 2 Slurp/2.0-Pre_WTC (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 1 Slurp/2.0-QA (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 4 Slurp/2.0e (slurp@inktomi.com; http://www.inktomisearch.com/) 1 Slurp/2.0_SR (stanr@inktomi.com; http://www.inktomi.com/slurp.html) 15 Slurp/3.0-si (slurp@inktomi.com; http://www.inktomi.com/slurp.html) 1 SmartSpider 1 StarSweeper 2.0 8 SyncIT/0.96 4 Top10ContactBot/1.0 23 Ultraseek 2 vscooter/2.0 G.R.A.B. V1.1.0 1 vspider/3.51 3 W3CRobot/5.2.9 libwww/5.2.9 2 WebSeer 1 Wget/1.4.5 65 WiseWire-Spider 7 www.WebWombat.com.au