Chuq Von Rospach <chuqui@plaidworks.com> writes:
> At 12:12 PM -0800 2/3/98, Grant Neufeld wrote:
> > At 11:25 PM -0700 1998/1/30, Chuq Von Rospach wrote:
> > >I now deny all my archives via robots.txt,
> >
> > I like regular users being able to find the sometimes useful information in
> > my list archives, so I don't block them from robots.
>
> That's why I'm building my own search engines for my archive -- so the
> archives are useful, but the information isn't globally propogated.
> Because, arguably, there's no reason for the archives to be in
> theglobal search engines.
There's a very satisfactory middle ground. Our archive Attic is
closed to the bots but wide open to everyone else. Doesn't
robots.txt accomplish the mission of blocking the search
engines?
mailto:tcbowden@clovis.nerdnosh.org (Tim Bowden)
Proud member of NERDNOSH (tm)!
mailto:majordomo@story.nerdnosh org
the command: subscribe nerdnosh
Follow-Ups:
References:
|
|
