Wrox Home  
Search P2P Archive for: Go

  Return to Index  

javascript_howto thread: Robots & Spiders.


Message #1 by "aaron" <agflem@y...> on Sun, 8 Dec 2002 09:13:24 -0500
Use a script like the one at the URL below. You can add a list of potential
list of robots (that don't follow the robots.txt standard), and only write
the link if they are good. Why would you want to make something uncrawlable?
Not all robots are bad.

http://www.icehousedesigns.com/scripts/ua_block_asp.zip

I have translated it into jscript, right now its in vbscript.
Email me if you want the JavaScript version.

TurnenT


  Return to Index