I am developing an e-commerse application where I have a relatively small catalog of a few hundred items. I began development using a sql database but since I wasn't updating the database and was simply downloading all of the data into the application each time the page was accessed I switched to reading an XML file instead which is much faster.

Now I am starting on other pages on the site and beginning to wonder if it wouldn't be getter just to load the entire dataset into an application variable. What are the pro's and con's of this? My knowledge and experience of the different types of state variables is limited at best.

The data will be the same for all users and will not be modified. I am assuming that currently the data is being loaded seperately every time the page is called meaning multiple copies could be in the server memory at the same time. I am assuming that the memory will be released quickly though so it probably won't be a problem.

If I use an application variable it will make it available to all pages in the application but will it keep the data in the server's memory at all times? If it does won't the server move the data into it's cache when it hasn't been accessed in a while so it won't slow other applications? I suspect that it will be faster to load the dataset from memory or its cache than from an XML file but would this be the case? Wouldn't loading the XML file for each page of the application every time it is called use more server resources? These are questions I don't know the complete answer for and so am looking for some insight.

I may eventually get to the point where I want to load only part of the catalog at a time in which case I may have to migrate to a database but I suspect technology will always be at the point where server memory will grow faster than my catalog. I'd rather have a faster response time for the user so he'll be more likely to use my site.

Another possibility I have become aware of from another post is the cache object. From what I read it can be used interchangably with an Application variable. What are the benefits and disadvantages of each approach? I read where a cache object has automatic data locks where they have to be explicitely set with an appliction variable but this can be a disadvantage if they are not needed. My catalog can not be changed by any user and would only be changed when I update a new XML file so data locks wouldn't do me any good and only add to overhead.

With either method, how would I get the application to reload the data when the xml file is updated?

I think I may have just found the answer but would like verification. The cache object can be set to expire when the xml file is replaced and can then have a callback function reload it with the new data. Not sure how it could be done with an application variable. In C#:

Cache.Insert ""myKey", myValue,
new Cache\Dependency(Server.MapPath("mysml.XML")),
null,
Cache.NoSlidingExpiration);TimeSpan.FromSeconds(30 ),
CacheItemPriority.High, onRemove);

If I leave out the onremove callback function will it automatically reload when needed or will I have to add functionality to my code?

I suspect cache may be a little slower with the automated datalocks and memory management but the difference would be likely measured in milliseconds and thus won't make much difference overall compared with bandwidth issues unless the data is being accessed repeatedly in the application (would creating a reference to the cache object and then casting that reference to a dataset get away from accessing the cache for each call to the dataset?).

Either would be better than loading an xml file for each page and infinitely better than loading the data from a database for each view. I am thinking now that the Cache's memory management features would make it the best bet.

Sorry for being so long winded, just trying to work things out in my mind and consider all factors so i can come up with the best option.