Wrox Programmer Forums
Go Back   Wrox Programmer Forums > C# and C > C# 2005 > C# 2005
C# 2005 For discussion of Visual C# 2005.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the C# 2005 section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
Old March 20th, 2006, 09:32 AM
Registered User
Join Date: Mar 2006
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
Default screen scraper issue

Hi All,

I had a problem to scrape information from a public web page, which has two options for user selection. Following are the source codes for the two option parts of this web page:

<TABLE class="formBody1u">
        <TD ALIGN=RIGHT class="LightNormalText"><span class="componentNameText">Market Date:</span></TD>

            <SELECT NAME="marketdate" onChange="submit()">

                        <OPTION VALUE="2006-03-26">26-MAR-2006

                        <OPTION VALUE="2006-03-25">25-MAR-2006

                        <OPTION VALUE="2006-03-24">24-MAR-2006

                        <OPTION VALUE="2006-03-23">23-MAR-2006

                        <OPTION VALUE="2006-03-22">22-MAR-2006

                        <OPTION VALUE="2006-03-21">21-MAR-2006

                        <OPTION VALUE="2006-03-20" SELECTED>20-MAR-2006

                        <OPTION VALUE="2006-03-19">19-MAR-2006

                        <OPTION VALUE="2006-03-18">18-MAR-2006

                        <OPTION VALUE="2006-03-17">17-MAR-2006

                        <OPTION VALUE="2006-03-16">16-MAR-2006
<TABLE class="formBody3">
        <TD ALIGN=LEFT COLSPAN=4><span class="componentNameText">Display Size:</span>
          <SELECT NAME="scrollsize" onChange="submit()">

            <OPTION VALUE="2">2 rows
            <OPTION VALUE="3">3 rows
            <OPTION VALUE="4">4 rows
            <OPTION VALUE="5">5 rows
            <OPTION VALUE="6">6 rows
            <OPTION VALUE="7">7 rows
            <OPTION VALUE="8">8 rows
            <OPTION VALUE="9">9 rows
            <OPTION VALUE="10">10 rows
            <OPTION VALUE="11">11 rows
            <OPTION VALUE="12">12 rows
            <OPTION VALUE="13">13 rows
            <OPTION VALUE="14">14 rows
            <OPTION VALUE="15">15 rows
            <OPTION VALUE="16">16 rows
            <OPTION VALUE="17">17 rows
            <OPTION VALUE="18">18 rows
            <OPTION VALUE="19">19 rows
            <OPTION VALUE="20">20 rows
            <OPTION VALUE="21">21 rows
            <OPTION VALUE="22">22 rows
            <OPTION VALUE="23">23 rows
            <OPTION VALUE="24" SELECTED>24 rows
            <OPTION VALUE="25">25 rows
My code is following:

String sUrl = "http://mospublic.xxx.com/xxx/jsp/load_forecast.jsp";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(sUrl);
UTF8Encoding encoding = new UTF8Encoding();
string postData="";
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
HttpWebResponse response;

postData = "marketdate=2006-03-21&scrollsize=19";
byte[] data = encoding.GetBytes(postData);
request.ContentLength = data.Length;
Stream newStream = request.GetRequestStream();
newStream.Write(data, 0, data.Length);

response = (HttpWebResponse)request.GetResponse();


I can change various row numbers to get the right information. However I changed the market date to other date(2006-03-21), but always got the 2006-03-20 information. I think there is a issue for parsing the marketdate "2006-03-21", because there is a special character "-". Do I need to take care of it specially? Have anybody can solve this problem.

Similar Threads
Thread Thread Starter Forum Replies Last Post
how can clear the screen the screen malli_kv2 BOOK: Beginning Java 2, JDK 5 Edition 4 January 17th, 2009 12:43 AM
Screen width day10 Classic ASP Basics 1 March 2nd, 2007 12:58 PM
Screen capture issue RNedel C# 3 September 22nd, 2006 01:51 PM
Draw on the screen without clearing the screen pu132 Visual Basic 2005 Basics 0 August 25th, 2006 09:03 PM
Conformation Screen. rupen Classic ASP Basics 3 July 12th, 2005 11:31 PM

Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.