Great Circle Associates List-Managers
(March 2001)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: robots.txt
From: Chuq Von Rospach <chuqui @ plaidworks . com>
Date: Thu, 01 Mar 2001 08:43:21 -0800
To: Tim Pierce <twp @ rootsweb . com>
Cc: JC Dill <inet-list @ vo . cnchost . com>, List-Managers <list-managers @ greatcircle . com>
In-reply-to: <20010301110428.K47456@ma-1.rootsweb.com>
User-agent: Microsoft-Entourage/9.0.2509

On 3/1/01 8:04 AM, "Tim Pierce" <twp@rootsweb.com> wrote:

> The difference is that there are a lot of "passwords" (search
> terms) which are likely to yield access.

Except that's not really true. If you search on ".com", ".net" and ".edu"
you solve 95% of the problem, and that's more than good enough for most
harvesters. Those pull up lists of links that can then easily be spidered on
most search engines, and will almost always have exactly what they're
looking for: email addresses.

With some search engines, you just search on "@", although that's not
something that'll work for all of them.



-- 
Chuq Von Rospach, Internet Gnome <http://www.chuqui.com>
[<chuqui@plaidworks.com> = <me@chuqui.com> = <chuq@apple.com>]
Yes, yes, I've finally finished my home page. Lucky you.

When an agnostic dies, does he go to the "great perhaps"?




Follow-Ups:
References:
Indexed By Date Previous: Re: robots.txt
From: Tim Pierce <twp@rootsweb.com>
Next: Re: robots.txt
From: Tim Pierce <twp@rootsweb.com>
Indexed By Thread Previous: Re: robots.txt
From: Tim Pierce <twp@rootsweb.com>
Next: Re: robots.txt
From: Tim Pierce <twp@rootsweb.com>

Google
 
Search Internet Search www.greatcircle.com