Actually, I don't have too many issues with software robots. In most cases, they associated with building search engines, and I'm a big fan of search engines.
I do, however, have a bad habit of reading useless server statistics, and noticing strange things in them. For now, this section of Websnob is devoted to deconstructing the actions of some lesser-known web robots, for the benefit of anybody else who's obsessing over strange things in their server logs.
Robotwatching: UniverseBot/1.0 is the case study of a site crawl by UniverseBot, a spider unleashed by a software company concerned about its reputation. (Added 11 May 2002)
Robotwatching: NetResearchServer/2.2 looks over the shoulder of the robot formerly known as IncyWincy. I would want a new name, too. (Added 11 May 2002)
Robotwatching: DiaGem/1.1 examines a very orderly, if somewhat mercurial robot working for a Japanese research project. (Added 31 May 2002).
Robotwatching: Gigabot/1.0 spies on the spider belonging to a new kid on the block, Gigablast. (Added 8 June 2002).
Robotwatching: Asterias/2.0 takes a peek at an annoying little spider looking for multimedia files. (Added 27 July 2002.)
It's not officially part of Robot Snob, but HTML Snob's guide to the ROBOTS meta tag, might be useful to you. It explains all the extensions to the venerable robots tag, and lists search engines known to obey it.
[an error occurred while processing this directive]