how to speedily pasre text files
I have a program which regularly scans a large number of text files, the files are stored in HTML. My script loads the file- line by line, each line is stripped of HTML and then parsed for the sought phrase or word. the problem is that there are over 6000 such files with size (100kb-150kb). i have timed my operations and right now, it takes 1 second to fully scan a file, therefore to scan the 6000 files will take about 2hrs which will be most unacceptable to my users. is there any way to speed up the search process. i am looking at XML but i don't quite understand the concept and i do not know if it is relevant.
Everything takes longer than you expect!