Wrox Programmer Forums
Go Back   Wrox Programmer Forums > Web Programming > JavaScript > Ajax
Ajax the combination of XHTML, CSS, DOM, XML, XSLT, XMLHttpRequest, and JavaScript
Welcome to the p2p.wrox.com Forums.

You are currently viewing the Ajax section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
Old December 7th, 2005, 09:42 AM
Registered User
Join Date: Dec 2005
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
Default Query for ScreenScraping Tech in .NET

Hi Friends,

This is siva from chennai.
I am going to write a good scripter using C# that can capture data from a web page. The data seems to be coming down in XML format and I cannot find how to access it.
Here's the 30 second rundown on how I get to the data now...
Login to a web page (I have a membership)
Select the search
Input the date range and select search button
Output is displayed in a table with links to the actual data
I must then click on each link, which brings up a screen with all the data I
I must do this for each dataset... there are hundreds.
I'd like to automate this process and capture the data into a database.
Is "screen-scraping" a possibility? If so, generally, how does that work?
Or is there a more effective approach to getting at that data in realtime?

Thanks for any thoughts,


Old December 7th, 2005, 10:15 AM
joefawcett's Avatar
Wrox Author
Join Date: Jun 2003
Posts: 3,074
Thanks: 1
Thanked 38 Times in 37 Posts

Not really an Ajax topic but you have two basic approaches:

1) Automate Internet explorer to duplicate your manual actions. Can be difficult to allow for errors etc.

2) Use a programmatic way to access the pages using something like XmlHttp. For example use the msxml2.xmlhttp.4.0 class to send a GET/POST request with the necessary request data.

The second approach is more reliable if the page itself does not rely too heavily on JavaScript.

Try seacrhing Google groups in Microsoft.public.* for internetexplorer.application and XmlHttp.


Joe (Microsoft MVP - XML)

Similar Threads
Thread Thread Starter Forum Replies Last Post
Pro ASP Tech for webmasters grstad All Other Wrox Books 1 February 21st, 2007 12:04 PM
Query for ScreenScraping Tech in .NET visit2siva ASP.NET 1.x and 2.0 Application Design 1 December 15th, 2005 12:42 PM
Query for ScreenScraping Tech in .NET visit2siva ASP.NET 1.0 and 1.1 Professional 1 December 8th, 2005 01:16 AM
Query for ScreenScraping Tech in .NET visit2siva C# 1 December 7th, 2005 09:44 AM
vb.net 2nd ed ch10 Adv Obj Orien Tech frresh BOOK: Beginning VB.NET 2nd Edition/Beginning VB.NET 2003 3 April 2nd, 2005 06:08 AM

Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.