Wrox Programmer Forums

Need to download code?

View our list of code downloads.

Go Back   Wrox Programmer Forums > .NET > Other .NET > General .NET
Password Reminder
Register
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read
General .NET For general discussion of MICROSOFT .NET topics that don't fall within any of the other .NET forum subcategories or .NET language forums.  If your question is specific to a language (C# or Visual Basic) or type of application (Windows Forms or ASP.Net) try an applicable forum category. ** PLEASE BE SPECIFIC WITH YOUR QUESTION ** When posting here, provide details regarding the Microsoft .NET language you are using and/or what type of application (Windows/Web Forms, etc) you are working in, if applicable to the question. This will help others answer the question without having to ask.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the General .NET section of the Wrox Programmer to Programmer discussions. This is a community of tens of thousands of software programmers and website developers including Wrox book authors and readers. As a guest, you can read any forum posting. By joining today you can post your own programming questions, respond to other developers’ questions, and eliminate the ads that are displayed to guests. Registration is fast, simple and absolutely free .
DRM-free e-books 300x50
Reply
 
Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old July 21st, 2004, 08:31 AM
Authorized User
 
Join Date: Sep 2003
Location: , , .
Posts: 93
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via AIM to flyin
Default Design Question

Hello all. I have a design question for all you developers.

Lets take a simple scenario: I have 1 master table that tells me what Topics an author has posted under and then a bunch of tables representing each Topic.

Needless to say these tables aren't related and i have to query the master table for the TableName Field to see what tables an author has posted under. Well now to the question.

This query can get very intense with the multiple unions i am doing and i do not want to hit the database everytime for this query so i figure hey, i'll cache the resulting datatable and cache it for 1 minute to save the DB. That works fine, but once that expires in the minute the DB gets hit up again and i can just see a ton of overhead there. My next solution was to run a job initially that would do this intense query for the most popular authors and in the process of the query, get the resulting set, store that datatable in a dataset and write that dataset out to an xml file using dataset.WriteXML, so that every subsequent request will simply read from the XML file if exists, else write a fresh one.

My main question is that once the xml file starts to get large like lets say about 10,000 records, does filling and querying that dataset resulting from the xml file start to get slow? Does the indexing in the Dataset start to cause troubles? I can do this test myself, but i was just wondering if you guys have encountered this and if you have a better solution perhaps?

Reply With Quote
  #2 (permalink)  
Old July 21st, 2004, 12:54 PM
planoie's Avatar
Friend of Wrox
Points: 16,481, Level: 55
Points: 16,481, Level: 55 Points: 16,481, Level: 55 Points: 16,481, Level: 55
Activity: 0%
Activity: 0% Activity: 0% Activity: 0%
 
Join Date: Aug 2003
Location: Clifton Park, New York, USA.
Posts: 5,407
Thanks: 0
Thanked 16 Times in 16 Posts
Default

Why can't you change the cache timeout to something longer?

And why do you have tables that sound to be very much alike instead of having a single table?
Reply With Quote
  #3 (permalink)  
Old July 21st, 2004, 01:03 PM
Authorized User
 
Join Date: May 2004
Location: , , USA.
Posts: 24
Thanks: 0
Thanked 0 Times in 0 Posts
Default

Cache the entire 10000 rows. Use ADO (.net ofcourse) to filter and
display 500 rows at a time. Remove the cache when any table related
to this query changes.

SS
Reply With Quote
  #4 (permalink)  
Old July 21st, 2004, 03:53 PM
Authorized User
 
Join Date: Sep 2003
Location: , , .
Posts: 93
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via AIM to flyin
Default

Peter, The table structure has been in use for quite some time now and there is alot of uniqueness to each table, that's why i had to make a compatable table that stores what tables the author has posted within.

to answer you and sundar_revathi question, yes i can make the cache time longer as far as expiring cache when when any of the tables change, can i do this through a query? or when a person uploads a new article will i have to know the cachekey in order to expire it there?




Reply With Quote
  #5 (permalink)  
Old July 21st, 2004, 03:59 PM
planoie's Avatar
Friend of Wrox
Points: 16,481, Level: 55
Points: 16,481, Level: 55 Points: 16,481, Level: 55 Points: 16,481, Level: 55
Activity: 0%
Activity: 0% Activity: 0% Activity: 0%
 
Join Date: Aug 2003
Location: Clifton Park, New York, USA.
Posts: 5,407
Thanks: 0
Thanked 16 Times in 16 Posts
Default

Thats' usually how you would do it. When any code runs that affects the contents of the table, you re-cache it or expire the cached item so it automatically reloads and caches the next time. This way you never have to expire the cache until you actually change/add data.
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
design question androoo ADO.NET 1 November 13th, 2004 09:41 AM
design question androoo Classic ASP Databases 1 November 13th, 2004 09:41 AM
design question androoo General .NET 1 November 13th, 2004 09:40 AM
design question androoo .NET Web Services 1 November 13th, 2004 09:39 AM



All times are GMT -4. The time now is 07:57 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
© 2013 John Wiley & Sons, Inc.