>Thus, most webmasters _should_ know about
> it; it's an official internet standard, right?
No and no. There is no easy way to find out about it, and robots.txt is not
even vaguely an official standard. Most people find out about robots.txt
the same way you found out about the proposal from archive.org: from
reading mailing lists or books about how to run a server.
In other words, this is still a new field. Your opinions are as good as the
next person's. It doesn't seem like there is consensus on this list about
whether what findmail.com is doing is bad, ok, or good.
--Paul Hoffman
--Internet Mail Consortium
Follow-Ups:
References:
|
|
