I need some advise to see if there is a better way of doing what I want...



On my website I have many subwebs (100 or so) and each subweb may have a

directory called Superintendents.  If that directory exist then I need to

index each asp page in it, or any pages in subdirectories under it.  Once I

have an index of all the pages I need to create an asp page that can search,

sort, and link this information.  The good part is that each asp page has

variables in them defining title, description, and keywords.

My first thought was to use a search engine.  I haven't found any that will

only index pages in certain directories without doing a lot of maintenance

on up keep of the paths.

My second thought was to do this manually with vbs scripts.  Once a day I

would search every subfolder in the web looking for directories with the

name Superintendents.  When it found one it would then search every asp page

in that directory reading each asp page pulling out the title, description,

and keywords. This information would be put into a csv file nightly. Then I

would create an asp page for the users that would use the csv as a recordset

for them to search and sort. 

Any comments appreciated

Rob