Wrox Programmer Forums
|
Classic ASP Basics For beginner programmers starting with "classic" ASP 3, pre-".NET." NOT for ASP.NET 1.0, 1.1, or 2.0
Welcome to the p2p.wrox.com Forums.

You are currently viewing the Classic ASP Basics section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
 
Old December 9th, 2004, 02:46 PM
Authorized User
 
Join Date: Oct 2004
Posts: 12
Thanks: 0
Thanked 0 Times in 0 Posts
Default how to speedily pasre text files

Hi,
I have a program which regularly scans a large number of text files, the files are stored in HTML. My script loads the file- line by line, each line is stripped of HTML and then parsed for the sought phrase or word. the problem is that there are over 6000 such files with size (100kb-150kb). i have timed my operations and right now, it takes 1 second to fully scan a file, therefore to scan the 6000 files will take about 2hrs which will be most unacceptable to my users. is there any way to speed up the search process. i am looking at XML but i don't quite understand the concept and i do not know if it is relevant.

please help

__________________
Everything takes longer than you expect!
 
Old December 10th, 2004, 02:33 AM
Friend of Wrox
 
Join Date: Oct 2003
Posts: 463
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via MSN to madhukp
Default

I think you can use Index server for this purpose. The following link gives an article which will give you the introductory knowledge on index server.

http://askasp.com/articles.asp?ArtID=6
 
Old December 10th, 2004, 06:51 AM
Friend of Wrox
 
Join Date: Jul 2003
Posts: 683
Thanks: 0
Thanked 1 Time in 1 Post
Default

Can you post your current code - we may be able to optimise it.

Cheers,

Chris

 
Old December 16th, 2004, 05:08 PM
Authorized User
 
Join Date: Oct 2004
Posts: 12
Thanks: 0
Thanked 0 Times in 0 Posts
Default

Hi guys,
I lost my internet connection for a while....i hope it is not too late to still get some help. i'll really appreciate it

Thanks - please see the code below


        ' assume sought string is in a variable called session("strsoughtdetail")

        intnumfinds=0

        do

            strcasetextfile= 'get a text file name


            SET filescan = Server.CreateObject("Scripting.FileSystemObject")
            SET scanfile = filescan.OpenTextFile(server.mappath(strcasetextfi le),1)

            while (not(scanfile.AtEndOfStream))

                strscannedline=scanfile.readline

                if instr(1,strscannedline,session("strsoughtdetail"), 1)>0 then

                        intnumfinds=intnumfinds+1

                end if
            wend

            scanfile.close
            SET scanfile=nothing
            SET filescan=nothing

        loop 'until 6000 text files of approx. 100kb are proccessed


 
Old December 17th, 2004, 10:03 AM
Friend of Wrox
 
Join Date: Jul 2003
Posts: 683
Thanks: 0
Thanked 1 Time in 1 Post
Default

I got a marginal increase in performance (about 10%) by Dim'ing all variables before use & creating the filesystem object outside the do loop so that you only create it once.

As madhukp suggests, you may want to use some kind of indexing server to get decent increases in performance.

You could also just load the data into a db and query against that.

Cheers,

Chris

 
Old December 17th, 2004, 10:48 AM
Friend of Wrox
 
Join Date: Oct 2003
Posts: 463
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via MSN to madhukp
Default

I had an experience that the performance of filesystemobject is very bad on files larger than 60 KB. (that which cannot be opened with notepad). Later I used Index Server to solve it.





Similar Threads
Thread Thread Starter Forum Replies Last Post
Compare 2 Text Files mollem VBScript 5 April 16th, 2016 09:35 AM
Reading text files senthilreddy ASP.NET 2.0 Basics 1 May 22nd, 2007 08:32 AM
Text Files Jane27 C# 8 January 13th, 2007 12:43 PM
comparing Two text files rajkumar007 Classic ASP Basics 0 October 15th, 2005 07:22 AM
Regarding " reading text files" spraveens Classic ASP Basics 1 October 14th, 2003 07:46 AM





Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.