Great Circle Associates List-Managers
(March 2001)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: robots.txt
From: Aumont <serge . aumont @ cru . fr>
Organization: Comite Reseaux des Universites
Date: Fri, 02 Mar 2001 08:15:46 +0100
To: JC Dill <inet-list @ vo . cnchost . com>
Cc: List-Managers <list-managers @ GreatCircle . COM>
References: <p04310100b6c2558b8621@[212.198.107.19]> <5.0.0.25.2.20010228115334.02e015e0@pop3.vo.cnchost.com>

JC Dill wrote:
> 
> On 12:21 AM 2/28/01, Chuq Von Rospach wrote:
> 
>  >What do I do? That would be telling. Other than saying my archives are
>  >behind a password, are protected by a robots.txt, and aren't in the global
>  >search engines or anywhere the spambots can get to without a lot of work,
> 
> robots.txt is widely ignored by spammer email harvester robots.

Sympa MLM archives are protected by a cookie. This cookie is sent to anyone
acknoledging a simple form (method post) saying "i'm not a spammer". Currently no
harvester accept cookies nor post forms. 
You can test it with a sample : http://listes.cru.fr/wws/arc/sympa-users

Removing emails from messages headers in archives (using mhonarc nospam option)
is not saqfe enough because there is a lot of emails in the message body
(signature etc).


-- 
-----------------------------------------------------------
Serge Aumont        Comité Réseaux des Universités
                     Campus Beaulieu 
                     35042 Rennes Cedex   +33 2 998 471 47


Indexed By Date Previous: Re: robots.txt
From: Chuq Von Rospach <chuqui@plaidworks.com>
Next: Re: List-Managers-Digest V10 #43
From: Todd Olson <tco2@cornell.edu>
Indexed By Thread Previous: Re: robots.txt
From: "Peter Galbavy" <peter.galbavy@knowledge.com>
Next: Re: List-Managers-Digest V10 #43
From: Todd Olson <tco2@cornell.edu>

Google
 
Search Internet Search www.greatcircle.com