Please mail details of any mistakes in this table to the author.
| Robot Name | User Agent | Hosts |
|---|---|---|
| ASpider (Associative Spider) | ASpider | nova.pvv.unit.no |
| AURESYS | AURESYS | crrm.univ-mrs.fr |
| Ahoy! The Homepage Finder | Ahoy! The Homepage Finder | |
| Altavista (Scooter) | Scooter | scooter.pa-x.dec.com |
| Altavista (Top 1000) | Top1000 altavista | widow.av.pa-x.dec.com www-relay1.pa-x.dec.com arachnophilia.alta-vista.net tarantula.av.pa-x.dec.com |
| BSpider | BSpider | |
| BackRub | BackRub | huron.stanford.edu jazz.stanford.edu sole.stanford.edu |
| BlackWidow | BlackWidow | |
| CACTVS Chemistry Spider | CACTVS Chemistry Spider | utamaro.organik.uni-erlangen.de |
| CS-HKUST WISE: WWW Index and Search Engine | CS-HKUST-IndexServer | dbx.cs.ust.hk |
| Combine System | combine | ub2.lu.se |
| Conceptbot | conceptbot | router.sifry.com |
| DeWeb(c) Katalog/Index | Deweb | deweb.orbit.de |
| EuroFerret (MuscatFerret) | MuscatFerret | muscat.co.uk index1-gui.server.virgin.net search1-gui.server.virgin.net |
| EuroSeek (Freecrawl) | FreeCrawl | freeside.net |
| Excite (ArchitextSpider) | ArchitextSpider | atext.com |
| Felix IDE | FelixIDE/1.0 | |
| FetchRover | ESIRover | |
| Fish search | Fish-Search-Robot | www.win.tue.nl |
| Fouineur | Mozilla/2.0 (compatible fouineur v2.0; fouineur.9bit.qc.ca) | |
| FunnelWeb | FunnelWeb | earth.planets.com.au |
| HI (HTML Index) Search | AITCSRobot/1.1 | |
| HKU WWW Octopus | HKU WWW Robot | |
| HTDig | htdig | |
| Harvest | Harvest/ | |
| HotWired test | HotWired-test | gossip.hotwired.com |
| Hotbot (Inktomi Slurp) | Slurp | inktomi.com |
| Hämähäkki | Hämähäkki/0.2 | www.fi |
| IBM_Planetwide | IBM_Planetwide | www.ibm.com www2.ibm.com |
| ILSE | charm.il.ft.hse.nl | |
| ISpi Crawler | iSpi | |
| IncyWincy | IncyWincy | osiris.sunderland.ac.uk |
| InfoSeek Robot 1.0 | InfoSeek Robot | infoseek.com |
| InfoSpiders | InfoSpiders | |
| Informant | Informant | informant.dartmouth.edu |
| Infoseek Sidewinder | InfoSeek Sidewinder | infoseek.com |
| IntelliAgent | IAGENT | sand.it.bond.edu.au |
| Internet Archiver | ia_archiver | archive.org |
| Iron30 | Iron30 | verno.ueda.info.waseda.ac.jp |
| Israeli-search | IsraeliSearch | dylan.ius.cs.cmu.edu dylan.ius.cs.cmu.edu |
| Job Robot (Jobot) | Jobot | supernova.micrognosis.com |
| JoeBot | JoeBot | |
| JumpStation | jumpstation | |
| Lycos | Lycos | lycos.com |
| Magpie | Magpie | blueberry.co.uk |
| MerzScope | MerzScope | |
| Motor | Motor | Michael.cybercon.technopark.gmd.de |
| NHSE Web Forager | NHSEWalker | mcs.anl.gov |
| Nederland.zoek | Nederland.zoek | |
| NetCarta WebMap Engine | NetCarta | |
| NetPlaza (NEC-MeshExplorer) | NEC-MeshExplorer | meshsv300.tk.mesh.ad.jp |
| NetScoop | NetScoop | alpha.is.tokushima-u.ac.jp beta.is.tokushima-u.ac.jp |
| Netscape Catalog Robot | Netscape-Catalog-Robot | |
| Nomad | Nomad | |
| Northern Light Gulliver | Gulliver | gulliver.northernlight.com tornado.northernlight.com |
| Occam | Occam | gentian.cs.washington.edu sekiu.cs.washington.edu saxifrage.cs.washington.edu |
| Open Text | fly | opentext.com opentext.net |
| Open Text Index Robot | Open Text Site Crawler | opentext.com opentext.net |
| Open Text Livelink Spider | LiveLink | |
| PGP Key Agent | PGP-KA | salerno.starnet.it |
| Phantom | Duppies | |
| Pioneer | Pioneer | flyer.ncsc.org |
| Planet Search (Fido) | fido | planetsearch.com |
| RBSE Spider | rbse.jsc.nasa.gov | |
| Resume Robot | Resume Robot | |
| Road Runner: The ImageScape Robot | Road Runner: ImageScape Robot (lim@cs.leidenuniv.nl) | |
| Robbie the Robot | Robbie | lmco.com |
| Robot Francoroute | Robot du CRIM 1.0a | zorro.crim.ca |
| Roverbot | Roverbot | roverbot.com |
| SG-Scout | SG-Scout | beta.xerox.com |
| SafetyNet Robot | SafetyNet | urlabs.com |
| Senrigan | Senrigan | aniki.olu.info.waseda.ac.jp |
| Simmany Robot Ver1.0 | SimBot | sansam.hnc.net |
| SiteTech-Rover | SiteTech-Rover | |
| Smart Spider | ESISmartSpider | |
| Spanner | Spanner | |
| Spry Wizard Robot | wizard.spry.com tiger.spry.com ginan.interser.net | |
| TITAN | TITAN | nlptitan.isl.ntt.jp |
| The Jubii Indexing Robot | JubiiRobot | |
| The NWI Robot | w3index | nwi.ub2.lu.se mars.dtv.dk desire1.ub2.lu.se |
| The NorthStar Robot | NorthStar | frognot.utdallas.edu utdallas.edu cnidir.org |
| The Peregrinator | Peregrinator | |
| The Web Wombat | qwerty.intercom.com.au | |
| The World Wide Web Worm | piper.cs.colorado.edu | |
| TitIn | TitIn | barok.foi.hr |
| UCSD Crawl | UCSD-Crawler | nuthaus.mib.org scilib.ucsd.edu |
| VWbot | VWbot_K | vancouver-webpages.com |
| Valkyrie | Valkyrie | c.u-tokyo.ac.jp |
| W3M2 | W3M2 | |
| Web Core / Roots | root | shiva.di.uminho.pt www.di.uminho.pt |
| Web21 | Web21 | |
| WebBandit Web Spider | WebBandit | ix.ix.netcom.com |
| WebCrawler | WebCrawler | spidey.webcrawler.com deep.webcrawler.com |
| WhoWhere Robot | spica.whowhere.com | |
| Wild Ferret Web Hopper #1, #2, #3 | Hazel's Ferret Web hopper | |
| Wise Wire Widow | WiseWire | wisewire.com |
| ananzi | EMC Spider | bilbo.internal.empirical.com |
| nzexplorer | explorersearch | bitz.co.nz |
| vision-search | vision-search | dylan.ius.cs.cmu.edu |
| webwalk | webwalk |