Once upon a time, far far away GreyWyvern <spam@greywyvern.com>
muttered
>And lo, Matt Probert didst speak in alt.www.webmaster:
>
>> Once upon a time, far far away GreyWyvern <spam@greywyvern.com>
>> muttered
>>
>>> Yesh. All bots which comply with the de facto robots.txt standard
>>> should read it and obey.
>>
>> No, it's a courtesy only. And one that really only applies to search
>> engine spiders. It certainly doesn't extend to automated surfing
>> systems and research robots which emulate or replicate human surfing.
>
>O_o
>
>Did you even read what I just wrote?
I missed the "which" qualifier. Sorry.
Matt