BotWatch Configuration File

This table shows the current BotWatch configuration file. It is based on data taken from Martjin Koster's Web Robots Pages plus additional data gathered from local contributions.

Please mail details of any mistakes in this table to the author.

Robot NameUser AgentHosts
ASpider (Associative Spider)ASpidernova.pvv.unit.no
AURESYSAURESYScrrm.univ-mrs.fr
Ahoy! The Homepage FinderAhoy! The Homepage Finder 
Altavista (Scooter)Scooterscooter.pa-x.dec.com
Altavista (Top 1000)Top1000 altavistawidow.av.pa-x.dec.com
www-relay1.pa-x.dec.com
arachnophilia.alta-vista.net
tarantula.av.pa-x.dec.com
BSpiderBSpider 
BackRubBackRubhuron.stanford.edu
jazz.stanford.edu
sole.stanford.edu
BlackWidowBlackWidow 
CACTVS Chemistry SpiderCACTVS Chemistry Spiderutamaro.organik.uni-erlangen.de
CS-HKUST WISE: WWW Index and Search EngineCS-HKUST-IndexServerdbx.cs.ust.hk
Combine Systemcombineub2.lu.se
Conceptbotconceptbotrouter.sifry.com
DeWeb(c) Katalog/IndexDewebdeweb.orbit.de
EuroFerret (MuscatFerret)MuscatFerretmuscat.co.uk
index1-gui.server.virgin.net
search1-gui.server.virgin.net
EuroSeek (Freecrawl)FreeCrawlfreeside.net
Excite (ArchitextSpider)ArchitextSpideratext.com
Felix IDEFelixIDE/1.0 
FetchRoverESIRover 
Fish searchFish-Search-Robotwww.win.tue.nl
FouineurMozilla/2.0 (compatible fouineur v2.0; fouineur.9bit.qc.ca) 
FunnelWebFunnelWebearth.planets.com.au
HI (HTML Index) SearchAITCSRobot/1.1 
HKU WWW OctopusHKU WWW Robot 
HTDightdig 
HarvestHarvest/ 
HotWired testHotWired-testgossip.hotwired.com
Hotbot (Inktomi Slurp)Slurpinktomi.com
HämähäkkiHämähäkki/0.2www.fi
IBM_PlanetwideIBM_Planetwidewww.ibm.com
www2.ibm.com
ILSE charm.il.ft.hse.nl
ISpi CrawleriSpi 
IncyWincyIncyWincyosiris.sunderland.ac.uk
InfoSeek Robot 1.0InfoSeek Robotinfoseek.com
InfoSpidersInfoSpiders 
InformantInformantinformant.dartmouth.edu
Infoseek SidewinderInfoSeek Sidewinderinfoseek.com
IntelliAgentIAGENTsand.it.bond.edu.au
Internet Archiveria_archiverarchive.org
Iron30Iron30verno.ueda.info.waseda.ac.jp
Israeli-searchIsraeliSearchdylan.ius.cs.cmu.edu
dylan.ius.cs.cmu.edu
Job Robot (Jobot)Jobotsupernova.micrognosis.com
JoeBotJoeBot 
JumpStationjumpstation 
LycosLycoslycos.com
MagpieMagpieblueberry.co.uk
MerzScopeMerzScope 
MotorMotorMichael.cybercon.technopark.gmd.de
NHSE Web ForagerNHSEWalkermcs.anl.gov
Nederland.zoekNederland.zoek 
NetCarta WebMap EngineNetCarta 
NetPlaza (NEC-MeshExplorer)NEC-MeshExplorermeshsv300.tk.mesh.ad.jp
NetScoopNetScoopalpha.is.tokushima-u.ac.jp
beta.is.tokushima-u.ac.jp
Netscape Catalog RobotNetscape-Catalog-Robot 
NomadNomad 
Northern Light GulliverGullivergulliver.northernlight.com
tornado.northernlight.com
OccamOccamgentian.cs.washington.edu
sekiu.cs.washington.edu
saxifrage.cs.washington.edu
Open Textflyopentext.com
opentext.net
Open Text Index RobotOpen Text Site Crawleropentext.com
opentext.net
Open Text Livelink SpiderLiveLink 
PGP Key AgentPGP-KAsalerno.starnet.it
PhantomDuppies 
PioneerPioneerflyer.ncsc.org
Planet Search (Fido) fidoplanetsearch.com
RBSE Spider rbse.jsc.nasa.gov
Resume RobotResume Robot 
Road Runner: The ImageScape RobotRoad Runner: ImageScape Robot (lim@cs.leidenuniv.nl) 
Robbie the RobotRobbielmco.com
Robot FrancorouteRobot du CRIM 1.0azorro.crim.ca
RoverbotRoverbotroverbot.com
SG-ScoutSG-Scoutbeta.xerox.com
SafetyNet RobotSafetyNeturlabs.com
SenriganSenrigananiki.olu.info.waseda.ac.jp
Simmany Robot Ver1.0SimBotsansam.hnc.net
SiteTech-RoverSiteTech-Rover 
Smart SpiderESISmartSpider 
SpannerSpanner 
Spry Wizard Robot wizard.spry.com
tiger.spry.com
ginan.interser.net
TITANTITANnlptitan.isl.ntt.jp
The Jubii Indexing RobotJubiiRobot 
The NWI Robotw3indexnwi.ub2.lu.se
mars.dtv.dk
desire1.ub2.lu.se
The NorthStar RobotNorthStarfrognot.utdallas.edu
utdallas.edu
cnidir.org
The PeregrinatorPeregrinator 
The Web Wombat  qwerty.intercom.com.au
The World Wide Web Worm piper.cs.colorado.edu
TitInTitInbarok.foi.hr
UCSD CrawlUCSD-Crawlernuthaus.mib.org
scilib.ucsd.edu
VWbotVWbot_Kvancouver-webpages.com
ValkyrieValkyriec.u-tokyo.ac.jp
W3M2W3M2 
Web Core / Rootsrootshiva.di.uminho.pt
www.di.uminho.pt
Web21 Web21 
WebBandit Web SpiderWebBanditix.ix.netcom.com
WebCrawlerWebCrawlerspidey.webcrawler.com
deep.webcrawler.com
WhoWhere Robot spica.whowhere.com
Wild Ferret Web Hopper #1, #2, #3Hazel's Ferret Web hopper 
Wise Wire WidowWiseWirewisewire.com
ananziEMC Spiderbilbo.internal.empirical.com
nzexplorerexplorersearchbitz.co.nz
vision-searchvision-searchdylan.ius.cs.cmu.edu
webwalkwebwalk