p2p.wrox.com Forums

Need to download code?

View our list of code downloads.

  Return to Index  

sql_language thread: Record Insert Speed Optimizing Between SQL <> VB

Message #1 by Sjef van Leeuwen <svanleeuwen@m...> on Sat, 02 Dec 2000 18:40:41 +0100

I am running search engine spiders which fully abstract HTML pages. With 
the first tests I used ADO and Batch Update statements but it took 30 
seconds to insert 1000 records to the back end DB.

That would be too slow, so I tried another approach, each HTML Element and 
its property values are stored in separate flat files (> like a.dat, 
meta.dat) etc. Then storing 1000 Records just took 90 milliseconds.

To insert all the data in the back end DB i am now using SQL7's BCP through 
a windows API (DTS showed slower tan BCP, mainly the object creation).

However does anyone have sample code where I could just BCP or use DTS 
inserting records from Memory rather than from Flatfiles  (what happened to 
the good old DOS ram-disk in WIN2K?

Sjef van Leeuwen

Administrator NOC
Mediaweb Internet Integrators
Tel.+xx xx xx xxxxx (desk)
Tel. +xx xx xx xxxxx (noc)
Fax. +xx xx xx xxxxx

Try our Domain Name Fetcher
With multi lingual synonyms http://namegizmo.com

:+: Roads? Where we are going we don't need Roads. :+:
					*the doc. 

  Return to Index