Managing large amount of data at a time
Hi,
I am building a report where for each user, I need to bring some of his data from database.Here typical scenario is,I have around 14K users records in the report, so for each user I need to connect to database and run a query and get his data.(I am using connetion pooling here)
This is making the report very slow ,instead i am planning if I could get the data of all users into memory once(may be into some java object) and get data for each user from this object.
Actually The data I get from database will be of the format
username rdate createdate
----------------------------------------------------------
user 1 12//06/2006 11:24:46 12//07/2006 12:24:46
user 2 11//06/2006 09:24:46 12//07/2006 17:14:55
.
.
My initial plan was use scrollable result set,but that will not help me here as I do not the row number for this user,are there any search operations we can perform on the resultset object
My second approach was to put all the data into some global hash table
or hash map , fill it after you get data from database and use this for each user.But I doubt my records may even cross some times 100K.
Or which is the best data structure I can use.
Guys,I am trying to make the report as quick as possible,please suggest me some good ideas,
Thanks,
pandu
|