Wrox Programmer Forums
Go Back   Wrox Programmer Forums > XML > XML
|
XML General XML discussions.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the XML section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
 
Old February 17th, 2008, 01:47 AM
Friend of Wrox
 
Join Date: Jun 2007
Posts: 477
Thanks: 10
Thanked 19 Times in 18 Posts
Default How big is too big?

I'm using .NET, and by and large all of the light configuration files are XML. It runs much faster than hitting a database. There are even cases when I've seen guys take advantage of XML to do some display or light processing using XSLT. However, anytime someone has a really large chunk of data, it doesn't go into an XML file, it goes into a database.

Why does XML slow down and lose it's edge compared to a database as size increases? And what size gives you roughly the same performance? ie if you've less data than that, better to use XML; but if you've got more than store it in a database?

-------------------------

Whatever you can do or dream you can, begin it. Boldness has genius, power and magic in it. Begin it now.
-Johann von Goethe

When Two Hearts Race... Both Win.
-Dove Chocolate Wrapper

Chroniclemaster1, Founder of www.EarthChronicle.com
A Growing History of our Planet, by our Planet, for our Planet.
__________________
-------------------------

Whatever you can do or dream you can, begin it. Boldness has genius, power and magic in it. Begin it now.
-Johann von Goethe

When Two Hearts Race... Both Win.
-Dove Chocolate Wrapper

Chroniclemaster1, Founder of www.EarthChronicle.com
A Growing History of our Planet, by our Planet, for our Planet.
 
Old February 17th, 2008, 02:59 PM
mhkay's Avatar
Wrox Author
 
Join Date: Apr 2004
Posts: 4,962
Thanks: 0
Thanked 292 Times in 287 Posts
Default

Why does XML slow down and lose it's edge compared to a database as size increases?

XML is stored as a text file, and to extract any information you need to read the whole file and parse it. Most applications will also allocate enough memory to hold a representation of the XML file in memory (and this will often be five to ten times the size of the raw data on disk). Incidentally, this is no different from the way a spreadsheet works. If you've got a hundred thousand employees and you just want to know one person's salary, then this obviously imposes a lot of overhead. A database, by contrast, is designed so that you can answer many queries by only reading those blocks on the disk that contain the information you are actually looking for.

>And what size gives you roughly the same performance?

That's very dependent on the actual use you are making of the data. If your application needs all the data, and if it fits in memory, then there's no reason why the XML solution should be slower.

Michael Kay
http://www.saxonica.com/
Author, XSLT Programmer's Reference and XPath 2.0 Programmer's Reference
 
Old February 18th, 2008, 05:33 AM
Friend of Wrox
 
Join Date: Jun 2007
Posts: 477
Thanks: 10
Thanked 19 Times in 18 Posts
Default

*the lightbulb turns on* So a login system for instance would clearly make a more scalable solution as a database because you are typically logging in one person at a time. But a company directory would make sense as XML because you're showing all the data.

What about this case? Does it make sense to use XML if you have an application where each user request is hitting only a fraction of the data, but "as an application" all the data is being used during the day. ie is the important point whether each user request is using all the data or whether the "application" is using all the data?

ie if you were displaying products for a website in several different product lines. Each visitor would probably be looking only at one product line, but throughout the day different visitors are looking at all the various product lines and using all the data.

In .NET, would this depend on whether you had caching turned on?

If you were serving the xml file at each user request, I would think that pretty much the whole thing would have to be used by each user for it to be preferable to use from a performance standpoint. But if you were using XMLReader in conjunction with caching, it seems like the aggragate of all user requests might be more relevant. Would that be a situation where XML would make more sense than using a database?

-------------------------

Whatever you can do or dream you can, begin it. Boldness has genius, power and magic in it. Begin it now.
-Johann von Goethe

When Two Hearts Race... Both Win.
-Dove Chocolate Wrapper

Chroniclemaster1, Founder of www.EarthChronicle.com
A Growing History of our Planet, by our Planet, for our Planet.
 
Old February 18th, 2008, 05:40 AM
mhkay's Avatar
Wrox Author
 
Join Date: Apr 2004
Posts: 4,962
Thanks: 0
Thanked 292 Times in 287 Posts
Default

>So a login system for instance would clearly make a more scalable solution as a database because you are typically logging in one person at a time.

If you've got a few thousand users (rather than a few million) then it could make sense to load the basic authentication data from XML into shared memory at start of day. But the other problem you have to consider is how to handle updates; one of the advantages of a database is that it provides fine-grained updates, with concurrency control and recovery.

Michael Kay
http://www.saxonica.com/
Author, XSLT Programmer's Reference and XPath 2.0 Programmer's Reference
 
Old February 22nd, 2008, 11:37 PM
Friend of Wrox
 
Join Date: Jun 2007
Posts: 477
Thanks: 10
Thanked 19 Times in 18 Posts
Default

Cool. So from a performance standpoint, a mid-sized XML login system would make sense. You can load it, cache it, and use it all day. But the first time two people forget their password at the same and need to reset them... bad news.

I just got Professional ASP.NET 2.0 Databases, and I'm still working on the earlier chapters. However, I was skimming ahead and one of the chapters is a case study in using the new XML data types in SQL Server 2005. I understand the basic principle of serializing and deserializing XML back and forth between DB and XML file. Do you see this as being the future of XML, something that's useful in certain cases, or a big snore? I'm having a hard time understanding how "cross breeding" XML and databases impacts application development.

And thank you for the insights!

-------------------------

Whatever you can do or dream you can, begin it. Boldness has genius, power and magic in it. Begin it now.
-Johann von Goethe

When Two Hearts Race... Both Win.
-Dove Chocolate Wrapper

Chroniclemaster1, Founder of www.EarthChronicle.com
A Growing History of our Planet, by our Planet, for our Planet.
 
Old September 17th, 2012, 01:07 AM
Registered User
 
Join Date: Sep 2012
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
Talking [How big is too big?] - Always depends

XML files can be very big. There is no performance issue with XML file size along.

If you use DOM to process XML, you have to load the whole XML file into memory, the size of file IS an issue. If you use SAX(XMLReader in C#) to process XML, you go through the XML, so the size of XML doesn't matter with performance.

Quote:
Originally Posted by chroniclemaster1 View Post
Cool. So from a performance standpoint, a mid-sized XML login system would make sense. You can load it, cache it, and use it all day. But the first time two people forget their password at the same and need to reset them... bad news...
Solution for this bad news is another application to update this memory.
For real-time change of password or new userID, you need another web application to update that information in your shared server memory. It's not hard at all. :-)

Hope helps,





Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I upload the big file shan168 VS.NET 2002/2003 1 April 1st, 2005 09:48 AM
Big Query hortoristic SQL Server 2000 3 May 25th, 2004 04:01 PM
Big models question darrinps BOOK: Expert One-on-One J2EE Design and Development 0 June 4th, 2003 02:10 PM





Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.