Trying to suss out information on new web robots is always a little painful, since many webmasters can't (or won't) reveal the raw data they've collected on suspected bots. Robotwatching is a experiment in total disclosure: I'm going to share the entire log from an unknown robot's visit, share any useful observations I can make, and see if that helps further the discussions of mystery web spiders.
63.251.169.236 - - [12/Jul/2002:18:09:40 -0400] "GET /robots.txt HTTP/1.1" 200 36 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:09:41 -0400] "GET / HTTP/1.1" 200 3714 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:09:41 -0400] "GET / HTTP/1.1" 200 3714 "-" "asterias/2.0"
Observations: Asterias® (Yes, the owners have registered a trademark on a bot name) starts by requesting robots.txt, which is a good sign. It then requests the domain's index page twice which is never a good sign.
Although nothing in the user agent string says so, Asterias is the robot for Singingfish, a search engine devoted to locating streaming media files (such as MP3 and RealMedia files) on the Web. (There are no streaming media files on bauser.com, by the way.) According to Asterias's information page, the bot obeys the robots.txt exclusion standard.
63.251.169.236 - - [12/Jul/2002:18:09:55 -0400] "GET /michael/index.html HTTP/1.1" 200 18022 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:01 -0400] "GET /alt.security.keydist HTTP/1.1" 301 330 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:02 -0400] "GET /alt.security.keydist/ HTTP/1.1" 200 6899 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:12 -0400] "GET /beer/ HTTP/1.1" 200 14192 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:22 -0400] "GET /roleplaying/index.html HTTP/1.1" 200 7200 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:31 -0400] "GET /robots.txt HTTP/1.1" 200 36 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:31 -0400] "GET / HTTP/1.1" 200 3714 "-" "asterias/2.0"
Observation: Asterias reads the first few links from the index page without incident, then requests robots.txt and the server root document again. How many times is it planning to request the server root?
For future reference (see the next set of observations), the server's
root document really does link to /alt.security.keydist
with the
trailing slash missing. I put the incomplete link on the root document to
catch malfunctioning user agents.
63.251.169.236 - - [12/Jul/2002:18:10:32 -0400] "GET /lens-filters/ HTTP/1.1" 200 2415 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:42 -0400] "GET /websnob/ HTTP/1.1" 200 9700 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:10:52 -0400] "GET /privacy.html HTTP/1.1" 200 3733 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:02 -0400] "GET /search.pl HTTP/1.1" 200 2841 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:12 -0400] "GET /statistics.html HTTP/1.1" 200 38123 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:23 -0400] "GET /michael HTTP/1.1" 301 317 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:24 -0400] "GET /michael/ HTTP/1.1" 200 18022 "-" "asterias/2.0"
Observation: Something subtle but important happens here. Unlike
/alt.security.keydist
a few lines back, /michael
does
not appear in the HTML of any page indexed so far. Asterias seems to have
derrived that URI
from the /michael/index.html
link (on the server index page) by
chopping off the trailing file name and slash. This doesn't have any
practical effect in this directory (since it eventually results in a
redirect back to the /michael/index.html
, but this "directory
sniffing" has interesting consequences later on in Asterias's visit.
63.251.169.236 - - [12/Jul/2002:18:11:33 -0400] "GET /roleplaying HTTP/1.1" 301 321 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:34 -0400] "GET /roleplaying/ HTTP/1.1" 200 7200 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:44 -0400] "GET /michael/words.html HTTP/1.1" 200 4171 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:11:54 -0400] "GET /michael/statistics.html HTTP/1.1" 200 6755 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:12:04 -0400] "GET /michael/rockstars.html HTTP/1.1" 200 8021 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:12:15 -0400] "GET /websnob/rules.html HTTP/1.1" 200 6409 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:12:24 -0400] "GET / HTTP/1.1" 200 3714 "-" "asterias/2.0"
Observation: That's the fourth request for the server root.
63.251.169.236 - - [12/Jul/2002:18:12:36 -0400] "GET /alt.security.keydist/index.html HTTP/1.1" 200 6899 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:12:45 -0400] "GET /beer/index.html HTTP/1.1" 200 28706 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:12:56 -0400] "GET /roleplaying/Ghostbusters/index.html HTTP/1.1" 200 13877 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:13:07 -0400] "GET /roleplaying/StarFrontiers/index.html HTTP/1.1" 200 8715 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:13:16 -0400] "GET /roleplaying/Spelljammer/ HTTP/1.1" 200 8548 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:13:27 -0400] "GET /websnob/index.html HTTP/1.1" 200 9700 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:13:37 -0400] "GET /websnob/finger/index.html HTTP/1.1" 200 10237 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:13:47 -0400] "GET /michael/privacy.html HTTP/1.1" 200 4941 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:13:57 -0400] "GET /michael/search.pl HTTP/1.1" 200 3124 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:07 -0400] "GET /michael/guestbook.html HTTP/1.1" 200 27096 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:16 -0400] "GET /michael/cgi/finger.sh HTTP/1.1" 200 160 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:26 -0400] "GET /alt.security.keydist/michael/bauser.asc HTTP/1.1" 200 2137 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:37 -0400] "GET /michael/hotlist.html HTTP/1.1" 200 5230 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:46 -0400] "GET /websnob HTTP/1.1" 301 317 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:47 -0400] "GET /websnob/ HTTP/1.1" 200 9669 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:57 -0400] "GET /beer HTTP/1.1" 301 314 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:14:58 -0400] "GET /beer/ HTTP/1.1" 200 14199 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:07 -0400] "GET /roleplaying/Ghostbusters HTTP/1.1" 301 334 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:08 -0400] "GET /roleplaying/Ghostbusters/ HTTP/1.1" 200 13867 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:17 -0400] "GET /roleplaying/StarFrontiers HTTP/1.1" 301 335 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:18 -0400] "GET /roleplaying/StarFrontiers/ HTTP/1.1" 200 8729 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:27 -0400] "GET /websnob/finger HTTP/1.1" 301 324 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:29 -0400] "GET /websnob/finger/ HTTP/1.1" 200 10167 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:38 -0400] "GET /michael/cgi HTTP/1.1" 301 321 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:38 -0400] "GET /michael/cgi/ HTTP/1.1" 200 2254 "-" "asterias/2.0"
Observation: More useless directory-sniffing.
63.251.169.236 - - [12/Jul/2002:18:15:48 -0400] "GET /alt.security.keydist/michael HTTP/1.1" 301 338 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:48 -0400] "GET /alt.security.keydist/michael/ HTTP/1.1" 200 1563 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:15:59 -0400] "GET /alt.security.keydist/FAQ.html HTTP/1.1" 200 10314 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:16:08 -0400] "GET /alt.security.keydist/FAQ.txt HTTP/1.1" 200 7172 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:16:18 -0400] "GET /alt.security.keydist/keyrings/alt-security-keydist.pgp HTTP/1.1" 200 0 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:16:28 -0400] "GET /alt.security.keydist/keyrings/demon-security-keys.pgp HTTP/1.1" 200 4257 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:16:38 -0400] "GET /alt.security.keydist/keyrings/t-netz-pgp-schluessel.pgp HTTP/1.1" 200 0 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:16:48 -0400] "GET /alt.security.keydist/keyrings/z-netz-alt-pgp-schluessel.pgp HTTP/1.1" 200 0 "-" "asterias/2.0"
Observation: Like nearly every robot I critique, Asterias decided to grab some PGP files. PGP files aren't streaming media, and they won't contain any hyperlinks, so they're completely useless to Singingfish.
63.251.169.236 - - [12/Jul/2002:18:16:58 -0400] "GET /alt.security.keydist/cryption.tag HTTP/1.1" 200 2455 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:17:09 -0400] "GET /alt.security.keydist/yarn.html HTTP/1.1" 200 4268 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:17:19 -0400] "GET /alt.security.keydist/newsgroups.html HTTP/1.1" 200 7893 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:17:30 -0400] "GET /alt.security.keydist/search.pl HTTP/1.1" 200 2779 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:17:39 -0400] "GET /alt.security.keydist/statistics.html HTTP/1.1" 200 7793 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:17:49 -0400] "GET /alt.security.keydist/revisions.html HTTP/1.1" 200 5039 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:17:59 -0400] "GET /alt.security.keydist/document.location HTTP/1.1" 404 315 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:18:09 -0400] "GET /alt.security.keydist/keyrings HTTP/1.1" 301 339 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:18:09 -0400] "GET /alt.security.keydist/keyrings/ HTTP/1.1" 200 1952 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:18:20 -0400] "GET /beer/coolers.html HTTP/1.1" 200 16513 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:18:30 -0400] "GET /beer/glossary.html HTTP/1.1" 200 20927 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:18:40 -0400] "GET /beer/index.html?pref=brands HTTP/1.1" 200 28838 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:18:51 -0400] "GET /beer/index.html?pref=breweries HTTP/1.1" 200 20547 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:19:02 -0400] "GET /beer/Australia.html HTTP/1.1" 200 9063 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:19:12 -0400] "GET /beer/Austria.html HTTP/1.1" 200 6427 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:19:23 -0400] "GET /beer/Canada.html HTTP/1.1" 200 14984 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:19:33 -0400] "GET /beer/Czech_Republic.html HTTP/1.1" 200 9422 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:19:43 -0400] "GET /beer/Holland.html HTTP/1.1" 200 10746 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:19:53 -0400] "GET /beer/Finland.html HTTP/1.1" 200 6450 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:20:03 -0400] "GET /beer/Germany.html HTTP/1.1" 200 12916 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:20:15 -0400] "GET /beer/Ireland.html HTTP/1.1" 200 11344 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:20:24 -0400] "GET /beer/Japan.html HTTP/1.1" 200 9473 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:20:34 -0400] "GET /beer/Mexico.html HTTP/1.1" 200 13752 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:20:43 -0400] "GET /beer/UK.html HTTP/1.1" 200 8780 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:20:54 -0400] "GET /beer/California.html HTTP/1.1" 200 14342 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:21:03 -0400] "GET /beer/Colorado.html HTTP/1.1" 200 12241 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:21:14 -0400] "GET /beer/Connecticut.html HTTP/1.1" 200 6457 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:21:24 -0400] "GET /beer/Florida.html HTTP/1.1" 200 7207 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:21:34 -0400] "GET /beer/Illinois.html HTTP/1.1" 200 6820 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:21:44 -0400] "GET /beer/Maine.html HTTP/1.1" 200 6407 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:21:54 -0400] "GET /beer/Massachusetts.html HTTP/1.1" 200 8590 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:22:04 -0400] "GET /beer/Missouri.html HTTP/1.1" 200 8804 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:22:14 -0400] "GET /beer/Michigan.html HTTP/1.1" 200 16323 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:22:24 -0400] "GET /beer/New_Jersey.html HTTP/1.1" 200 6215 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:22:34 -0400] "GET /beer/New_York.html HTTP/1.1" 200 13928 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:22:45 -0400] "GET /beer/Ohio.html HTTP/1.1" 200 10764 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:22:54 -0400] "GET /beer/Oregon.html HTTP/1.1" 200 10296 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:23:05 -0400] "GET /beer/North_Carolina.html HTTP/1.1" 200 6560 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:23:16 -0400] "GET /beer/Pennsylvania.html HTTP/1.1" 200 10577 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:23:25 -0400] "GET /beer/Tennessee.html HTTP/1.1" 200 7658 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:23:36 -0400] "GET /beer/Texas.html HTTP/1.1" 200 11570 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:23:45 -0400] "GET /beer/Vermont.html HTTP/1.1" 200 6610 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:23:55 -0400] "GET /beer/Washington.html HTTP/1.1" 200 9537 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:24:06 -0400] "GET /beer/Wisconsin.html HTTP/1.1" 200 18438 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:24:15 -0400] "GET /beer/index.html?pref=regional HTTP/1.1" 200 14124 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:24:25 -0400] "GET /beer/search.pl HTTP/1.1" 200 5508 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:24:36 -0400] "GET /beer/help.html HTTP/1.1" 200 8800 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:24:46 -0400] "GET /beer/guestbook.html HTTP/1.1" 200 29898 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:24:56 -0400] "GET /beer/revisions.html HTTP/1.1" 200 10908 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:25:05 -0400] "GET /beer/statistics.html HTTP/1.1" 200 19190 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:25:15 -0400] "GET /beer/document.location HTTP/1.1" 404 299 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:25:26 -0400] "GET /roleplaying/ADnD/cantrips.html HTTP/1.1" 200 19186 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:25:36 -0400] "GET /roleplaying/reaper.html HTTP/1.1" 200 9875 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:25:47 -0400] "GET /roleplaying/Spelljammer/index.html HTTP/1.1" 200 8538 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:25:57 -0400] "GET /roleplaying/StarFrontiers/interstellar.html HTTP/1.1" 200 30317 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:26:07 -0400] "GET /roleplaying/freighters.html HTTP/1.1" 200 7069 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:26:17 -0400] "GET /roleplaying/revisions.html HTTP/1.1" 200 7402 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:26:27 -0400] "GET /roleplaying/search.pl HTTP/1.1" 200 2601 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:26:36 -0400] "GET /roleplaying/statistics.html HTTP/1.1" 200 17044 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:26:48 -0400] "GET /roleplaying/newsgroups.html HTTP/1.1" 200 12572 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:26:58 -0400] "GET /roleplaying/linking.html HTTP/1.1" 200 5152 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:07 -0400] "GET /roleplaying/document.location HTTP/1.1" 404 306 "-" "asterias/2.0"
Observation: This URI is completely bogus. Asterias has apparently
misinterpreted some Javascript it found inside an A element.
63.251.169.236 - - [12/Jul/2002:18:27:18 -0400] "GET /alt.security.keydist/ HTTP/1.1" 200 6899 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:29 -0400] "GET /michael/ HTTP/1.1" 200 18022 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:37 -0400] "GET /roleplaying/ADnD HTTP/1.1" 301 326 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:38 -0400] "GET /roleplaying/ADnD/ HTTP/1.1" 200 19210 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:47 -0400] "GET /roleplaying/Spelljammer HTTP/1.1" 301 333 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:48 -0400] "GET /roleplaying/Spelljammer/ HTTP/1.1" 200 8532 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:27:57 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.amazon.com/exec/obido/redirect-home/michaelbauser.com HTTP/1.1" 302 327 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:28:08 -0400] "GET /websnob/meta/wireless.html HTTP/1.1" 200 12538 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:28:18 -0400] "GET /websnob/color/cmyk2rgb.htm HTTP/1.1" 200 5955 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:28:29 -0400] "GET /websnob/meta/ HTTP/1.1" 200 5792 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:28:39 -0400] "GET /websnob/html4/link.html HTTP/1.1" 200 9688 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:28:49 -0400] "GET /websnob/html4/relationships.html HTTP/1.1" 200 56510 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:29:00 -0400] "GET /websnob/engines/ HTTP/1.1" 200 6573 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:29:11 -0400] "GET /websnob/engines/NetInsert.html HTTP/1.1" 200 8708 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:29:20 -0400] "GET /websnob/engines/Gigablast.html HTTP/1.1" 200 6796 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:29:31 -0400] "GET /websnob/engines/Teoma.html HTTP/1.1" 200 11689 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:29:41 -0400] "GET /websnob/engines/SurfGopher.html HTTP/1.1" 200 7941 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:29:50 -0400] "GET /websnob/engines/Quick.html HTTP/1.1" 200 10580 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:30:02 -0400] "GET /websnob/engines/Aesop.html HTTP/1.1" 200 9470 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:30:11 -0400] "GET /websnob/robots/ HTTP/1.1" 200 3583 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:30:21 -0400] "GET /websnob/robots/UniverseBot.html HTTP/1.1" 200 31649 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:30:31 -0400] "GET /websnob/robots/NetResearchServer.html HTTP/1.1" 200 13207 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:30:41 -0400] "GET /websnob/robots/Gigabot.html HTTP/1.1" 200 16537 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:30:51 -0400] "GET /websnob/CSS/scrollbar.html HTTP/1.1" 200 12022 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:31:02 -0400] "GET /websnob/html4/stealth_redirection.html HTTP/1.1" 200 11633 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:31:11 -0400] "GET /websnob/CSS/borders.html HTTP/1.1" 200 5791 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:31:21 -0400] "GET /websnob/CSS/Netscape4.html HTTP/1.1" 200 6160 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:31:32 -0400] "GET /websnob/freeware.html HTTP/1.1" 200 4192 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:31:41 -0400] "GET /websnob/domains/reason.html HTTP/1.1" 200 6461 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:31:51 -0400] "GET /websnob/keydist.html HTTP/1.1" 200 6962 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:32:01 -0400] "GET /websnob/traffic.html HTTP/1.1" 200 5422 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:32:11 -0400] "GET /websnob/cliches.html HTTP/1.1" 200 9232 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:32:22 -0400] "GET /websnob/logs.html HTTP/1.1" 200 9780 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:32:32 -0400] "GET /websnob/html3/BANNER.html HTTP/1.1" 200 4794 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:32:42 -0400] "GET /websnob/html3/BQ.html HTTP/1.1" 200 5517 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:32:52 -0400] "GET /websnob/html3/FN.html HTTP/1.1" 200 7154 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:33:02 -0400] "GET /websnob/html3/NOTE.html HTTP/1.1" 200 4564 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:33:11 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.qksrv.net/click-1025412-1192785 HTTP/1.1" 302 305 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:33:23 -0400] "GET /websnob/newsgroups.html HTTP/1.1" 200 56499 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:33:32 -0400] "GET /websnob/revisions.html HTTP/1.1" 200 8848 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:33:42 -0400] "GET /websnob/search.pl HTTP/1.1" 200 2003 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:33:52 -0400] "GET /websnob/document.location HTTP/1.1" 404 302 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:02 -0400] "GET /websnob/statistics.html HTTP/1.1" 200 7174 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:12 -0400] "GET /websnob/meta HTTP/1.1" 301 322 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:13 -0400] "GET /websnob/meta/ HTTP/1.1" 200 5892 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:22 -0400] "GET /websnob/color HTTP/1.1" 301 323 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:24 -0400] "GET /websnob/color/ HTTP/1.1" 200 2425 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:32 -0400] "GET /websnob/html4 HTTP/1.1" 301 323 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:33 -0400] "GET /websnob/html4/ HTTP/1.1" 200 4080 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:42 -0400] "GET /websnob/engines HTTP/1.1" 301 325 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:44 -0400] "GET /websnob/engines/ HTTP/1.1" 200 6635 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:52 -0400] "GET /websnob/robots HTTP/1.1" 301 324 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:34:54 -0400] "GET /websnob/robots/ HTTP/1.1" 200 3642 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:02 -0400] "GET /websnob/CSS HTTP/1.1" 301 321 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:05 -0400] "GET /websnob/CSS/ HTTP/1.1" 200 3209 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:12 -0400] "GET /websnob/domains HTTP/1.1" 301 325 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:14 -0400] "GET /websnob/domains/ HTTP/1.1" 200 2423 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:23 -0400] "GET /websnob/html3 HTTP/1.1" 301 323 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:23 -0400] "GET /websnob/html3/ HTTP/1.1" 200 4053 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:35:40 -0400] "GET /search.pl?BUILDINDEX=y HTTP/1.1" 200 766373 "-" "asterias/2.0"
Observation: That last URI is the address of the keyword index for
bauser.com. Asterias had to ignore a Robots
META tag to request that URI.
63.251.169.236 - - [12/Jul/2002:18:58:01 -0400] "GET /(/alt.security.keydist) HTTP/1.1" 404 300 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:58:11 -0400] "GET /(/beer) HTTP/1.1" 404 284 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:58:21 -0400] "GET /(/roleplaying) HTTP/1.1" 404 291 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:58:31 -0400] "GET /(/news.groups.reviews) HTTP/1.1" 404 299 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:58:41 -0400] "GET /(/websnob) HTTP/1.1" 404 287 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:58:51 -0400] "GET /(/michael) HTTP/1.1" 404 287 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:59:01 -0400] "GET /( HTTP/1.1" 404 278 "-" "asterias/2.0"
Observation: Here's something I've never seen a robot do before.
Asterias pulled a bunch of not-quite-legal URIs from the OPTION element of bauser.com's search form,
creating a serious of 404 errors. Why is it looking for addresses in
elements that aren't hyperlinks?
63.251.169.236 - - [12/Jul/2002:18:59:11 -0400] "GET /index.html HTTP/1.1" 200 3714 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:59:22 -0400] "GET /beer/?pref=brands HTTP/1.1" 200 28838 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:59:32 -0400] "GET /beer/?pref=breweries HTTP/1.1" 200 20547 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:59:43 -0400] "GET /beer/?pref=regional HTTP/1.1" 200 14152 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:18:59:55 -0400] "GET /roleplaying/Ghostbusters/ HTTP/1.1" 200 13884 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:00:04 -0400] "GET /beer/Miller.html HTTP/1.1" 200 12400 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:00:14 -0400] "GET /roleplaying/StarFrontiers/ HTTP/1.1" 200 8652 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:00:24 -0400] "GET /beer/Stroh.html HTTP/1.1" 200 18380 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:00:34 -0400] "GET /beer/Coors.html HTTP/1.1" 200 11628 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:00:44 -0400] "GET /beer/Anheuser-Busch.html HTTP/1.1" 200 18454 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:00:55 -0400] "GET /roleplaying/Ghostbusters/shopper.html HTTP/1.1" 200 13201 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:01:04 -0400] "GET /roleplaying/collection.html HTTP/1.1" 200 42713 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:01:15 -0400] "GET /roleplaying/ HTTP/1.1" 200 7207 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:01:25 -0400] "GET /roleplaying/StarFrontiers/stone_age.html HTTP/1.1" 200 13915 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:01:35 -0400] "GET /beer/guestbook.html? HTTP/1.1" 200 29866 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:01:45 -0400] "GET /roleplaying/Ghostbusters/lochness.html HTTP/1.1" 200 7181 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:01:55 -0400] "GET /roleplaying/StarFrontiers/equipment.html HTTP/1.1" 200 16564 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:02:06 -0400] "GET /roleplaying/StarFrontiers/UPF_ranks.html HTTP/1.1" 200 9994 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:02:15 -0400] "GET /roleplaying/ADnD/ HTTP/1.1" 200 19210 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:02:25 -0400] "GET /roleplaying/StarFrontiers/origins.html HTTP/1.1" 200 15967 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:02:36 -0400] "GET /roleplaying/Ghostbusters/jersey.html HTTP/1.1" 200 7582 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:02:45 -0400] "GET /roleplaying/Ghostbusters/procare.html HTTP/1.1" 200 7075 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:02:56 -0400] "GET /roleplaying/Ghostbusters/bossie.html HTTP/1.1" 200 9947 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:03:07 -0400] "GET /roleplaying/Ghostbusters/hotlist.html HTTP/1.1" 200 8867 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:03:16 -0400] "GET /roleplaying/Ghostbusters/bigfoot.html HTTP/1.1" 200 8775 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:03:26 -0400] "GET /roleplaying/Spelljammer/find_familiar.html HTTP/1.1" 200 6235 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:03:37 -0400] "GET /roleplaying/Spelljammer/gnome_tech.html HTTP/1.1" 200 15276 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:03:46 -0400] "GET /websnob/finger/ HTTP/1.1" 200 10220 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:03:56 -0400] "GET /roleplaying/Ghostbusters/glossary.html HTTP/1.1" 200 7934 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:04:07 -0400] "GET /roleplaying/Spelljammer/magic_items.html HTTP/1.1" 200 6398 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:04:17 -0400] "GET /roleplaying/Ghostbusters/animals.html HTTP/1.1" 200 16383 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:04:27 -0400] "GET /roleplaying/Ghostbusters/cryptids.html HTTP/1.1" 200 11009 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:04:37 -0400] "GET /roleplaying/StarFrontiers/ACU-optiraptor.html HTTP/1.1" 200 7574 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:04:47 -0400] "GET /michael/10back.html HTTP/1.1" 200 3086 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:04:57 -0400] "GET /roleplaying/Ghostbusters/animal13.html HTTP/1.1" 200 6819 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:05:07 -0400] "GET /michael/wonderbra.html HTTP/1.1" 200 4360 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:05:17 -0400] "GET /roleplaying/StarFrontiers/time-space.html HTTP/1.1" 200 14743 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:05:27 -0400] "GET /roleplaying/Ghostbusters/plumbers.html HTTP/1.1" 200 6532 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:05:37 -0400] "GET /roleplaying/Ghostbusters/smokey.html HTTP/1.1" 200 6140 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:05:48 -0400] "GET /websnob/meta/robots.html HTTP/1.1" 200 13882 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:05:58 -0400] "GET /roleplaying/Spelljammer/new_nevermind.html HTTP/1.1" 200 9681 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:06:09 -0400] "GET /roleplaying/StarFrontiers/ACU-relp.html HTTP/1.1" 200 6945 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:06:18 -0400] "GET /websnob/meta/browsers.html HTTP/1.1" 200 24342 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:06:28 -0400] "GET /roleplaying/Ghostbusters/fhorton.html HTTP/1.1" 200 7022 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:06:38 -0400] "GET /roleplaying/Ghostbusters/eddie.html HTTP/1.1" 200 6953 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:06:48 -0400] "GET /websnob/finger/gateways.html HTTP/1.1" 200 12930 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:06:58 -0400] "GET /roleplaying/Ghostbusters/leadfoot.html HTTP/1.1" 200 6816 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:07:09 -0400] "GET /roleplaying/Ghostbusters/field.html HTTP/1.1" 200 7202 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:07:29 -0400] "GET /websnob/finger/cgi.html HTTP/1.1" 200 12197 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:07:39 -0400] "GET /roleplaying/Ghostbusters/joey.html HTTP/1.1" 200 6928 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:07:49 -0400] "GET /roleplaying/Ghostbusters/yoshima.html HTTP/1.1" 200 8274 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:07:59 -0400] "GET /roleplaying/Ghostbusters/slowpoke.html HTTP/1.1" 200 6418 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:08:09 -0400] "GET /roleplaying/Ghostbusters/chief.html HTTP/1.1" 200 7416 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:08:20 -0400] "GET /roleplaying/Ghostbusters/spectre.html HTTP/1.1" 200 6970 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:08:29 -0400] "GET /roleplaying/Ghostbusters/lurker.html HTTP/1.1" 200 6370 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:08:40 -0400] "GET /roleplaying/Ghostbusters/phil.html HTTP/1.1" 200 6477 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:08:49 -0400] "GET /michael/10dorms.html HTTP/1.1" 200 3370 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:09:00 -0400] "GET /roleplaying/Ghostbusters/spot.html HTTP/1.1" 200 6673 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:09:10 -0400] "GET /roleplaying/StarFrontiers/random-planets.html HTTP/1.1" 200 11421 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:09:20 -0400] "GET /roleplaying/Spelljammer/proficiencies.html HTTP/1.1" 200 11832 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:09:30 -0400] "GET /roleplaying/Spelljammer/colophon.html HTTP/1.1" 200 4987 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:09:40 -0400] "GET /roleplaying/Spelljammer/GK-Oerth.html HTTP/1.1" 200 5527 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:09:50 -0400] "GET /roleplaying/Ghostbusters/fran.html HTTP/1.1" 200 6388 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:10:00 -0400] "GET /websnob/domains/aftercare.html HTTP/1.1" 200 11619 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:10:10 -0400] "GET /roleplaying/Ghostbusters/larry.html HTTP/1.1" 200 6602 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:10:20 -0400] "GET /roleplaying/Ghostbusters/betty.html HTTP/1.1" 200 8034 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:10:31 -0400] "GET /websnob/html4/ HTTP/1.1" 200 4007 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:10:41 -0400] "GET /websnob/meta/proprietary.html HTTP/1.1" 200 27539 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:10:51 -0400] "GET /roleplaying/Ghostbusters/mbauser.html HTTP/1.1" 200 7400 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:11:01 -0400] "GET /michael/copyright.html HTTP/1.1" 200 4234 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:11:11 -0400] "GET /roleplaying/Ghostbusters/willie.html HTTP/1.1" 200 6523 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:11:21 -0400] "GET /roleplaying/StarFrontiers/random-planets-2.html HTTP/1.1" 200 11233 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:11:31 -0400] "GET /michael/dorm.html HTTP/1.1" 200 5565 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:11:41 -0400] "GET /roleplaying/Ghostbusters/buddy.html HTTP/1.1" 200 9171 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:11:51 -0400] "GET /roleplaying/Ghostbusters/melvin.html HTTP/1.1" 200 7534 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:12:02 -0400] "GET /websnob/robots/DiaGem.html HTTP/1.1" 200 25532 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:12:12 -0400] "GET /websnob/meta/engines.html HTTP/1.1" 200 8012 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:12:22 -0400] "GET /websnob/meta/list.html HTTP/1.1" 200 19464 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:12:32 -0400] "GET /michael/friends.html HTTP/1.1" 200 5465 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:12:43 -0400] "GET /websnob/meta/useless.html HTTP/1.1" 200 11900 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:12:53 -0400] "GET /websnob/finger/misc.html HTTP/1.1" 200 7401 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:13:03 -0400] "GET /websnob/meta/syntax.html HTTP/1.1" 200 12727 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:13:13 -0400] "GET /michael/10ksu.html HTTP/1.1" 200 3270 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:13:23 -0400] "GET /websnob/color/ HTTP/1.1" 200 2369 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:13:34 -0400] "GET /websnob/CSS/ HTTP/1.1" 200 3138 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:13:43 -0400] "GET /michael/gender.html HTTP/1.1" 200 4717 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:13:53 -0400] "GET /michael/phantom.html HTTP/1.1" 200 4446 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:14:03 -0400] "GET /michael/confessions.html HTTP/1.1" 200 13881 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:14:13 -0400] "GET /websnob/meta/schemas.html HTTP/1.1" 200 6678 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:14:23 -0400] "GET /websnob/html3/ HTTP/1.1" 200 4112 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:14:34 -0400] "GET /websnob/domains/ HTTP/1.1" 200 2378 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:14:44 -0400] "GET /websnob/finger/bibliography.html HTTP/1.1" 200 3275 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:14:53 -0400] "GET /michael/revisions.html HTTP/1.1" 200 10663 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:03 -0400] "GET /roleplaying/Ghostbusters/bin/header.htm HTTP/1.1" 200 356 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:14 -0400] "GET /beer/bibliography.html HTTP/1.1" 200 8207 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:23 -0400] "GET /roleplaying/Ghostbusters/bin HTTP/1.1" 301 338 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:23 -0400] "GET /roleplaying/Ghostbusters/bin/ HTTP/1.1" 200 1480 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:33 -0400] "GET /michael/ads/guestlist.htm HTTP/1.1" 200 23202 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:43 -0400] "GET /michael/ads HTTP/1.1" 301 321 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:43 -0400] "GET /michael/ads/ HTTP/1.1" 200 804 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:15:53 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.webstandards.org/ HTTP/1.1" 302 291 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:03 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.inudge.com/ HTTP/1.1" 302 285 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:13 -0400] "GET /roleplaying/Ghostbusters/application.html HTTP/1.1" 503 362 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:23 -0400] "GET /roleplaying/StarFrontiers/document.location HTTP/1.1" 404 320 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:33 -0400] "GET /roleplaying/Spelljammer/document.location HTTP/1.1" 404 318 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:43 -0400] "GET /websnob/finger/document.location HTTP/1.1" 404 35 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:53 -0400] "GET /michael/stats.html HTTP/1.1" 301 332 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:16:54 -0400] "GET /michael/statistics.html HTTP/1.1" 200 6755 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:17:04 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.qksrv.net/click-1025412-5032928 HTTP/1.1" 302 305 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:17:13 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.qksrv.net/click-1025412-5924456 HTTP/1.1" 302 305 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:17:23 -0400] "GET /michael/cgi/?N=D HTTP/1.1" 200 2254 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:17:34 -0400] "GET /michael/cgi/?M=A HTTP/1.1" 200 2254 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:17:44 -0400] "GET /michael/cgi/?S=A HTTP/1.1" 200 2254 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:17:54 -0400] "GET /michael/cgi/?D=A HTTP/1.1" 200 2254 "-" "asterias/2.0"
Observation: There are those mystery arguments again.
63.251.169.236 - - [12/Jul/2002:19:18:04 -0400] "GET /michael/cgi/1994.pl HTTP/1.1" 200 1607 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:18:15 -0400] "GET /michael/cgi/1995.pl HTTP/1.1" 200 1607 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:18:24 -0400] "GET /michael/cgi/1996.pl HTTP/1.1" 200 1607 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:18:34 -0400] "GET /michael/cgi/1997.pl HTTP/1.1" 200 1607 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:18:44 -0400] "GET /michael/cgi/1998.pl HTTP/1.1" 200 1607 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:18:55 -0400] "GET /michael/cgi/1999.pl HTTP/1.1" 200 1607 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:19:05 -0400] "GET /michael/cgi/countpages.sh HTTP/1.1" 200 21 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:19:14 -0400] "GET /michael/cgi/days.pl HTTP/1.1" 200 40 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:19:24 -0400] "GET /michael/cgi/guestbook.old HTTP/1.1" 200 11441 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:19:35 -0400] "GET /michael/cgi/guestbook.pl HTTP/1.1" 200 1465 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:19:44 -0400] "GET /michael/cgi/head.pl HTTP/1.1" 200 166 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:19:54 -0400] "GET /michael/cgi/htungeek.pl HTTP/1.1" 200 482 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:20:04 -0400] "GET /michael/cgi/menu.pl HTTP/1.1" 200 1166 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:20:14 -0400] "GET /michael/cgi/my_age.pl HTTP/1.1" 200 15 "-" "asterias/2.0"
Observation: Here's where Asterias's directory-sniffing shows me up, because it sniffs out a subdirectory I set the file permissions incorrectly for, and locates a set of scripts that are only meant to be run as server- side includes. Oops. Fixed that.
63.251.169.236 - - [12/Jul/2002:19:20:24 -0400] "GET /alt.security.keydist/michael/?N=D HTTP/1.1" 200 1563 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:20:34 -0400] "GET /alt.security.keydist/michael/?M=A HTTP/1.1" 200 1563 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:20:45 -0400] "GET /alt.security.keydist/michael/?S=A HTTP/1.1" 200 1563 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:20:55 -0400] "GET /alt.security.keydist/michael/?D=A HTTP/1.1" 200 1563 "-" "asterias/2.0"
Observation: I haven't got a clue what happened here. Asterias found a directory that contains some old public encryption keys, and added some mystery parameters to the URI. (If I didn't know better, I'd think Asterias was probing for security holes.) These might represent some standard convention for multimedia servers, but that's outside my area of expertise.
63.251.169.236 - - [12/Jul/2002:19:21:05 -0400] "GET /alt.security.keydist/michael/bauser.puf HTTP/1.1" 200 212 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:21:15 -0400] "GET /alt.security.keydist/michael/export.tsm HTTP/1.1" 200 152 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:21:25 -0400] "GET /alt.security.keydist/michael/islander.asc HTTP/1.1" 200 1485 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:21:35 -0400] "GET /alt.security.keydist/michael/islander.pgp HTTP/1.1" 200 982 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:21:45 -0400] "GET /alt.security.keydist/michael/mbauser2.luc HTTP/1.1" 200 70 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:21:55 -0400] "GET /alt.security.keydist/michael/mbauser2.lux HTTP/1.1" 200 161 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:22:05 -0400] "GET /alt.security.keydist/michael/myripemkey.html HTTP/1.1" 200 2474 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:22:15 -0400] "GET /alt.security.keydist/yarn/me-pgpe.bat HTTP/1.1" 200 4066 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:22:25 -0400] "GET /alt.security.keydist/yarn/showpgp.bat HTTP/1.1" 200 4040 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:22:35 -0400] "GET /alt.security.keydist/yarn/nsb-pgpe.bat HTTP/1.1" 200 2087 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:22:45 -0400] "GET /alt.security.keydist/yarn/ripemenc.bat HTTP/1.1" 200 933 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:22:55 -0400] "GET /alt.security.keydist/yarn/ripemsig.bat HTTP/1.1" 200 923 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:05 -0400] "GET /alt.security.keydist/yarn/ripemd.bat HTTP/1.1" 200 801 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:15 -0400] "GET /alt.security.keydist/yarn HTTP/1.1" 301 335 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:15 -0400] "GET /alt.security.keydist/yarn/ HTTP/1.1" 200 1660 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:25 -0400] "GET /alt.security.keydist/michael/bauser.pgp HTTP/1.1" 302 324 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:25 -0400] "GET /alt.security.keydist/michael/bauser.asc HTTP/1.1" 200 2137 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:35 -0400] "GET /help/netnews.html HTTP/1.1" 404 294 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:45 -0400] "GET /help HTTP/1.1" 404 281 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:55 -0400] "GET /alt.security.keydist/privacy.html HTTP/1.1" 301 321 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:23:55 -0400] "GET /privacy.html HTTP/1.1" 200 3733 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:24:05 -0400] "GET /alt.security.keydist/default.css HTTP/1.1" 200 1077 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:24:15 -0400] "GET /alt.security.keydist/keyrings/?N=D HTTP/1.1" 200 1952 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:24:25 -0400] "GET /alt.security.keydist/keyrings/?M=A HTTP/1.1" 200 1952 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:24:35 -0400] "GET /alt.security.keydist/keyrings/?S=A HTTP/1.1" 200 1952 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:24:45 -0400] "GET /alt.security.keydist/keyrings/?D=A HTTP/1.1" 200 1952 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:24:56 -0400] "GET /alt.security.keydist/keyrings/alt-security-keydist.bak HTTP/1.1" 200 20868 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:25:06 -0400] "GET /alt.security.keydist/keyrings/demon-security-keys.bak HTTP/1.1" 200 3912 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:25:16 -0400] "GET /alt.security.keydist/keyrings/fidonet-pkey_drop.bak HTTP/1.1" 200 4472 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:25:26 -0400] "GET /alt.security.keydist/keyrings/fidonet-pkey_drop.pgp HTTP/1.1" 200 6063 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:25:36 -0400] "GET /alt.security.keydist/keyrings/t-netz-pgp-schluessel.bak HTTP/1.1" 200 15534 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:25:46 -0400] "GET /alt.security.keydist/keyrings/z-netz-alt-pgp-schluessel.bak HTTP/1.1" 200 110912 "-" "asterias/2.0"
Observation: And then it requests the PGP files again. Thanks for wasting bandwidth guys.
63.251.169.236 - - [12/Jul/2002:19:25:56 -0400] "GET /beer/mail/Labatt-1 HTTP/1.1" 200 1411 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:06 -0400] "GET /beer/mail/schrock-1 HTTP/1.1" 200 1653 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:16 -0400] "GET /beer/mail HTTP/1.1" 301 319 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:16 -0400] "GET /beer/mail/ HTTP/1.1" 200 1258 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:26 -0400] "GET /beer/Massachusetts.html/ HTTP/1.1" 200 5284 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:37 -0400] "GET /beer/? HTTP/1.1" 200 28896 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:47 -0400] "GET /beer/aol HTTP/1.1" 404 285 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:26:57 -0400] "GET /beer/www.HighFalls.com HTTP/1.1" 404 299 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:27:07 -0400] "GET /beer/n/a HTTP/1.1" 404 285 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:27:17 -0400] "GET /beer/n HTTP/1.1" 404 283 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:27:27 -0400] "GET /beer/null HTTP/1.1" 200 80 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:27:37 -0400] "GET /beer/index.hdml HTTP/1.1" 200 636 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:27:47 -0400] "GET /beer/Stroh.hdml HTTP/1.1" 200 235 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:27:57 -0400] "GET /beer/Anheuser-Busch.hdml HTTP/1.1" 200 259 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:28:07 -0400] "GET /roleplaying/ADnD/document.location HTTP/1.1" 404 311 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:28:17 -0400] "GET /roleplaying/(/roleplaying) HTTP/1.1" 404 303 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:28:27 -0400] "GET /roleplaying/(/roleplaying/ADnD) HTTP/1.1" 404 308 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:28:37 -0400] "GET /roleplaying/(/roleplaying/Ghostbusters) HTTP/1.1" 404 316 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:28:47 -0400] "GET /roleplaying/(/roleplaying/Spelljammer) HTTP/1.1" 404 315 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:28:57 -0400] "GET /roleplaying/(/roleplaying/StarFrontiers) HTTP/1.1" 404 317 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:29:07 -0400] "GET /roleplaying/( HTTP/1.1" 404 290 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:29:17 -0400] "GET /roleplaying/(/roleplaying HTTP/1.1" 404 302 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:29:27 -0400] "GET /websnob/meta/document.location HTTP/1.1" 404 307 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:29:37 -0400] "GET /websnob/html4/document.location HTTP/1.1" 404 308 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:29:47 -0400] "GET /websnob/html4/shortcut-icon HTTP/1.1" 404 304 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:29:57 -0400] "GET /websnob/html4/author HTTP/1.1" 404 297 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:30:07 -0400] "GET /websnob/adverts/ad_redirect.pl?http://www.qksrv.net/click-1025412-541523 HTTP/1.1" 302 304 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:30:17 -0400] "GET /websnob/engines/document.location HTTP/1.1" 404 310 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:30:27 -0400] "GET /websnob/robots/document.location HTTP/1.1" 404 309 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:30:37 -0400] "GET /websnob/CSS/document.location HTTP/1.1" 404 306 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:30:47 -0400] "GET /websnob/domains/document.location HTTP/1.1" 403 314 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:30:57 -0400] "GET /michael/cgi/finger HTTP/1.1" 200 160 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:08 -0400] "GET /michael/cgi/htungeek.pl?GSS_d_H++_s+:---_g-_p?_au+_a-_w+_v+_C+++_US++_P+_L_3_E-_N++_K+++_W_M-_!V_-po+_Y+_t+_5+_j-_R++_G''_tv+++_b+_!D_B--_e++_u**_h-_f_r---_n-_y+ HTTP/1.1" 200 3764 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:17 -0400] "GET /stats.html HTTP/1.1" 404 287 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:27 -0400] "GET /websnob/html3/document.location HTTP/1.1" 404 308 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:37 -0400] "GET /websnob/html3/footnote.html HTTP/1.1" 404 304 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:47 -0400] "GET /websnob/html4/robots_meta_tag.html HTTP/1.1" 301 333 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:49 -0400] "GET /websnob/meta/robots.html HTTP/1.1" 200 13933 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:58 -0400] "GET /websnob/domaincare.html HTTP/1.1" 301 339 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:31:59 -0400] "GET /websnob/domains/aftercare.html HTTP/1.1" 200 11633 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:32:08 -0400] "GET /websnob/default.css HTTP/1.1" 200 1510 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:32:18 -0400] "GET /websnob/color/inverse.htm HTTP/1.1" 200 3272 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:32:28 -0400] "GET /websnob/color/document.location HTTP/1.1" 404 308 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:32:38 -0400] "GET /search.pl?KEYWORDS=aaaaaaiiiiiieeeee HTTP/1.1" 200 3001 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:32:48 -0400] "GET /search.pl?KEYWORDS=aangepaste HTTP/1.1" 200 2984 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:32:59 -0400] "GET /search.pl?KEYWORDS=abandon HTTP/1.1" 200 2976 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:33:08 -0400] "GET /search.pl?KEYWORDS=abandoned HTTP/1.1" 200 3376 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:33:19 -0400] "GET /search.pl?KEYWORDS=abbereviated HTTP/1.1" 200 2992 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:33:29 -0400] "GET /search.pl?KEYWORDS=abbreviated HTTP/1.1" 200 3133 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:33:39 -0400] "GET /search.pl?KEYWORDS=abbreviation HTTP/1.1" 200 3410 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:33:49 -0400] "GET /search.pl?KEYWORDS=abbreviations HTTP/1.1" 200 3112 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:33:59 -0400] "GET /search.pl?KEYWORDS=abc HTTP/1.1" 200 3353 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:34:09 -0400] "GET /search.pl?KEYWORDS=abcs HTTP/1.1" 200 3003 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:34:19 -0400] "GET /search.pl?KEYWORDS=abcsearch HTTP/1.1" 200 2972 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:34:29 -0400] "GET /search.pl?KEYWORDS=abductions HTTP/1.1" 200 2994 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:34:39 -0400] "GET /search.pl?KEYWORDS=aberdeen HTTP/1.1" 200 3005 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:34:49 -0400] "GET /search.pl?KEYWORDS=abestweb HTTP/1.1" 200 2992 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:34:59 -0400] "GET /search.pl?KEYWORDS=abilities HTTP/1.1" 200 4932 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:35:10 -0400] "GET /search.pl?KEYWORDS=ability HTTP/1.1" 200 4100 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:35:19 -0400] "GET /search.pl?KEYWORDS=able HTTP/1.1" 200 4763 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:35:30 -0400] "GET /search.pl?KEYWORDS=abnormal HTTP/1.1" 200 3246 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:35:40 -0400] "GET /search.pl?KEYWORDS=aboard HTTP/1.1" 200 2985 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:35:50 -0400] "GET /search.pl?KEYWORDS=aborted HTTP/1.1" 200 2997 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:36:01 -0400] "GET /search.pl?KEYWORDS=aborting HTTP/1.1" 200 3019 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:36:10 -0400] "GET /search.pl?KEYWORDS=abound HTTP/1.1" 200 3118 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:36:20 -0400] "GET /search.pl?KEYWORDS=about HTTP/1.1" 200 8990 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:36:30 -0400] "GET /search.pl?KEYWORDS=above HTTP/1.1" 200 5895 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:36:40 -0400] "GET /search.pl?KEYWORDS=absence HTTP/1.1" 200 2976 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:36:50 -0400] "GET /search.pl?KEYWORDS=absolute HTTP/1.1" 200 3001 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:37:01 -0400] "GET /search.pl?KEYWORDS=absolutely HTTP/1.1" 200 3488 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:37:10 -0400] "GET /search.pl?KEYWORDS=absorb HTTP/1.1" 200 2971 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:37:21 -0400] "GET /search.pl?KEYWORDS=absorbs HTTP/1.1" 200 2988 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:37:31 -0400] "GET /search.pl?KEYWORDS=abstract HTTP/1.1" 200 3616 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:37:41 -0400] "GET /search.pl?KEYWORDS=absurdities HTTP/1.1" 200 2989 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:37:51 -0400] "GET /search.pl?KEYWORDS=abuse HTTP/1.1" 200 3903 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:38:01 -0400] "GET /search.pl?KEYWORDS=abused HTTP/1.1" 200 3442 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:38:11 -0400] "GET /search.pl?KEYWORDS=abusers HTTP/1.1" 200 2985 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:38:21 -0400] "GET /search.pl?KEYWORDS=abuses HTTP/1.1" 200 2990 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:38:31 -0400] "GET /search.pl?KEYWORDS=abusing HTTP/1.1" 200 2988 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:38:41 -0400] "GET /search.pl?KEYWORDS=abusive HTTP/1.1" 200 2982 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:38:51 -0400] "GET /search.pl?KEYWORDS=abwholesaler HTTP/1.1" 200 2975 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:39:01 -0400] "GET /search.pl?KEYWORDS=abz HTTP/1.1" 200 3000 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:39:11 -0400] "GET /search.pl?KEYWORDS=academia HTTP/1.1" 200 3128 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:39:22 -0400] "GET /search.pl?KEYWORDS=academic HTTP/1.1" 200 3514 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:39:32 -0400] "GET /search.pl?KEYWORDS=academics HTTP/1.1" 200 3246 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:39:42 -0400] "GET /search.pl?KEYWORDS=accelerate HTTP/1.1" 200 3014 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:39:52 -0400] "GET /search.pl?KEYWORDS=accelerated HTTP/1.1" 200 3014 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:40:02 -0400] "GET /search.pl?KEYWORDS=accelerating HTTP/1.1" 200 3015 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:40:12 -0400] "GET /search.pl?KEYWORDS=acceleration HTTP/1.1" 200 3152 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:40:22 -0400] "GET /search.pl?KEYWORDS=accent HTTP/1.1" 200 3143 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:40:32 -0400] "GET /search.pl?KEYWORDS=accented HTTP/1.1" 200 2992 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:40:42 -0400] "GET /search.pl?KEYWORDS=accept HTTP/1.1" 200 3854 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:40:52 -0400] "GET /search.pl?KEYWORDS=acceptable HTTP/1.1" 200 3644 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:41:02 -0400] "GET /search.pl?KEYWORDS=acceptance HTTP/1.1" 200 3093 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:41:12 -0400] "GET /search.pl?KEYWORDS=accepted HTTP/1.1" 200 3545 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:41:23 -0400] "GET /search.pl?KEYWORDS=accepting HTTP/1.1" 200 3033 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:41:33 -0400] "GET /search.pl?KEYWORDS=accepts HTTP/1.1" 200 2986 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:41:43 -0400] "GET /search.pl?KEYWORDS=access HTTP/1.1" 200 6754 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:41:53 -0400] "GET /search.pl?KEYWORDS=accessability HTTP/1.1" 200 2987 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:42:03 -0400] "GET /search.pl?KEYWORDS=accessarizona HTTP/1.1" 200 2976 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:42:13 -0400] "GET /search.pl?KEYWORDS=accessed HTTP/1.1" 200 3379 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:42:23 -0400] "GET /search.pl?KEYWORDS=accesses HTTP/1.1" 200 2964 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:42:34 -0400] "GET /search.pl?KEYWORDS=accessibility HTTP/1.1" 200 2987 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:42:43 -0400] "GET /search.pl?KEYWORDS=accessible HTTP/1.1" 200 4180 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:42:53 -0400] "GET /search.pl?KEYWORDS=accessing HTTP/1.1" 200 3108 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:43:03 -0400] "GET /search.pl?KEYWORDS=accessokc HTTP/1.1" 200 3087 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:43:14 -0400] "GET /search.pl?KEYWORDS=accessories HTTP/1.1" 200 3104 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:43:23 -0400] "GET /search.pl?KEYWORDS=accessory HTTP/1.1" 200 2976 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:43:34 -0400] "GET /search.pl?KEYWORDS=accident HTTP/1.1" 200 3073 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:43:44 -0400] "GET /search.pl?KEYWORDS=accidental HTTP/1.1" 200 3354 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:43:54 -0400] "GET /search.pl?KEYWORDS=accidentally HTTP/1.1" 200 3148 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:44:04 -0400] "GET /search.pl?KEYWORDS=accidents HTTP/1.1" 200 3371 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:44:14 -0400] "GET /search.pl?KEYWORDS=acclerations HTTP/1.1" 200 3036 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:44:24 -0400] "GET /search.pl?KEYWORDS=accommodate HTTP/1.1" 200 3035 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:44:34 -0400] "GET /search.pl?KEYWORDS=accompanied HTTP/1.1" 200 3000 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:44:44 -0400] "GET /search.pl?KEYWORDS=accompanying HTTP/1.1" 200 3107 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:44:54 -0400] "GET /search.pl?KEYWORDS=accomplish HTTP/1.1" 200 3107 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:45:04 -0400] "GET /search.pl?KEYWORDS=accomplishment HTTP/1.1" 200 2977 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:45:14 -0400] "GET /search.pl?KEYWORDS=according HTTP/1.1" 200 5503 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:45:24 -0400] "GET /search.pl?KEYWORDS=accordingly HTTP/1.1" 200 2999 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:45:34 -0400] "GET /search.pl?KEYWORDS=accordion HTTP/1.1" 200 3026 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:45:45 -0400] "GET /search.pl?KEYWORDS=accordions HTTP/1.1" 200 3006 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:45:55 -0400] "GET /search.pl?KEYWORDS=account HTTP/1.1" 200 3824 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:46:05 -0400] "GET /search.pl?KEYWORDS=accountability HTTP/1.1" 200 2972 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:46:15 -0400] "GET /search.pl?KEYWORDS=accounts HTTP/1.1" 200 3270 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:46:25 -0400] "GET /search.pl?KEYWORDS=accreditation HTTP/1.1" 200 2993 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:46:35 -0400] "GET /search.pl?KEYWORDS=accredited HTTP/1.1" 200 2989 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:46:45 -0400] "GET /search.pl?KEYWORDS=accumulated HTTP/1.1" 200 2977 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:46:55 -0400] "GET /search.pl?KEYWORDS=accumulating HTTP/1.1" 200 3085 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:47:05 -0400] "GET /search.pl?KEYWORDS=accuracy HTTP/1.1" 200 2941 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:47:15 -0400] "GET /search.pl?KEYWORDS=accurate HTTP/1.1" 200 3510 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:47:25 -0400] "GET /search.pl?KEYWORDS=accurately HTTP/1.1" 200 3317 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:47:35 -0400] "GET /search.pl?KEYWORDS=accusation HTTP/1.1" 200 2999 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:47:45 -0400] "GET /search.pl?KEYWORDS=accused HTTP/1.1" 200 3126 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:47:55 -0400] "GET /search.pl?KEYWORDS=acebo HTTP/1.1" 200 2969 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:48:06 -0400] "GET /search.pl?KEYWORDS=acfaq HTTP/1.1" 200 2988 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:48:16 -0400] "GET /search.pl?KEYWORDS=acff HTTP/1.1" 200 3011 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:48:26 -0400] "GET /search.pl?KEYWORDS=acfffaq HTTP/1.1" 200 2993 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:48:36 -0400] "GET /search.pl?KEYWORDS=acfs HTTP/1.1" 200 2980 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:48:46 -0400] "GET /search.pl?KEYWORDS=ach HTTP/1.1" 200 3008 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:48:56 -0400] "GET /search.pl?KEYWORDS=achan HTTP/1.1" 200 3010 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:49:06 -0400] "GET /search.pl?KEYWORDS=achieve HTTP/1.1" 200 2989 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:49:16 -0400] "GET /search.pl?KEYWORDS=acknowledge HTTP/1.1" 200 3101 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:49:26 -0400] "GET /search.pl?KEYWORDS=acknowledgement HTTP/1.1" 200 2987 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:49:36 -0400] "GET /search.pl?KEYWORDS=acknowledgements HTTP/1.1" 200 2984 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:49:46 -0400] "GET /search.pl?KEYWORDS=acns HTTP/1.1" 200 2974 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:49:56 -0400] "GET /search.pl?KEYWORDS=acorn HTTP/1.1" 200 3001 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:50:06 -0400] "GET /search.pl?KEYWORDS=acorns HTTP/1.1" 200 2984 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:50:16 -0400] "GET /search.pl?KEYWORDS=acpub HTTP/1.1" 200 3106 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:50:27 -0400] "GET /search.pl?KEYWORDS=acquainted HTTP/1.1" 200 3025 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:50:36 -0400] "GET /search.pl?KEYWORDS=acquire HTTP/1.1" 200 3116 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:50:47 -0400] "GET /search.pl?KEYWORDS=acquired HTTP/1.1" 200 3567 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:50:57 -0400] "GET /search.pl?KEYWORDS=acquires HTTP/1.1" 200 3082 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:51:07 -0400] "GET /search.pl?KEYWORDS=acquiring HTTP/1.1" 200 2981 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:51:17 -0400] "GET /search.pl?KEYWORDS=acquisition HTTP/1.1" 200 3482 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:51:27 -0400] "GET /search.pl?KEYWORDS=acquisitions HTTP/1.1" 200 2980 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:51:37 -0400] "GET /search.pl?KEYWORDS=acrobat HTTP/1.1" 200 3001 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:51:47 -0400] "GET /search.pl?KEYWORDS=acronym HTTP/1.1" 200 3099 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:51:57 -0400] "GET /search.pl?KEYWORDS=acronyms HTTP/1.1" 200 2992 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:52:07 -0400] "GET /search.pl?KEYWORDS=across HTTP/1.1" 200 4033 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:52:17 -0400] "GET /search.pl?KEYWORDS=act HTTP/1.1" 200 3299 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:52:28 -0400] "GET /search.pl?KEYWORDS=acting HTTP/1.1" 200 3119 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:52:37 -0400] "GET /search.pl?KEYWORDS=action HTTP/1.1" 200 3976 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:52:48 -0400] "GET /search.pl?KEYWORDS=actions HTTP/1.1" 200 3475 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:52:58 -0400] "GET /search.pl?KEYWORDS=activate HTTP/1.1" 200 2973 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:53:08 -0400] "GET /search.pl?KEYWORDS=activated HTTP/1.1" 200 3103 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:53:18 -0400] "GET /search.pl?KEYWORDS=active HTTP/1.1" 200 4175 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:53:28 -0400] "GET /search.pl?KEYWORDS=actively HTTP/1.1" 200 3231 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:53:38 -0400] "GET /search.pl?KEYWORDS=activeperl HTTP/1.1" 200 2988 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:53:48 -0400] "GET /search.pl?KEYWORDS=activestate HTTP/1.1" 200 2979 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:53:59 -0400] "GET /search.pl?KEYWORDS=activex HTTP/1.1" 200 2981 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:54:09 -0400] "GET /search.pl?KEYWORDS=activists HTTP/1.1" 200 2995 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:54:18 -0400] "GET /search.pl?KEYWORDS=activities HTTP/1.1" 200 3287 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:54:28 -0400] "GET /search.pl?KEYWORDS=activity HTTP/1.1" 200 3697 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:54:38 -0400] "GET /search.pl?KEYWORDS=actkxlr HTTP/1.1" 200 3026 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:54:48 -0400] "GET /search.pl?KEYWORDS=actor HTTP/1.1" 200 2993 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:54:58 -0400] "GET /search.pl?KEYWORDS=actors HTTP/1.1" 200 2995 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:55:09 -0400] "GET /search.pl?KEYWORDS=actress HTTP/1.1" 200 2995 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:55:19 -0400] "GET /search.pl?KEYWORDS=acts HTTP/1.1" 200 3078 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:55:29 -0400] "GET /search.pl?KEYWORDS=actual HTTP/1.1" 200 4643 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:55:39 -0400] "GET /search.pl?KEYWORDS=actually HTTP/1.1" 200 9164 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:55:49 -0400] "GET /search.pl?KEYWORDS=acu HTTP/1.1" 200 3095 "-" "asterias/2.0" 63.251.169.236 - - [12/Jul/2002:19:55:59 -0400] "GET /search.pl?KEYWORDS=acutally HTTP/1.1" 200 2967 "-" "asterias/2.0"
Observations: And it concludes it comedic tour through bauser.com by falling right into the spidertrap of the keyword index. You know what? I really hate this robot.
Asterias made 533 requests in 1 hour, 46 minutes, and 19 seconds. An
astonishing number of those were duplicates, sniffed directories, protected
by META tags, or completely bogus. Singingfish is about to
become the only search engine banned from bauser.com, because it's
generating a hugh amount of requests (and errors) looking for multimedia
files that don't exist here.
Which is a shame, really, because Singingfish isn't a horrible engine. It's just got a horrible, anti-social, rampaging robot working for it.
Searching Google for "asterias" finds pages that about anything but the robot. Searching WebmasterWorld for references to asterias found one useful thread about the bot.
[an error occurred while processing this directive]