Hi everyone,

I'm working on a project where I need to be able to grab a web page from a
url, parse it for a small section of information and then display it on my
aspx page.

Is there a .net component that will allow me to fetch a url and return the
contents as a string, or some type of object where I can obtain the html
source?  I tried using the InetCtlsObjects component, but when I tried to
instantiate it, it gives off an error telling me that it is not licensed for
use.  I have Visual Studio.net Architect version installed on the box I am
trying to develp on.

Any help is much appreciated.

Chris Kersey

Chris,

	Yes, the WebRequest and WebResponse classes in the System.Net
namespace allows you easily fetch a URL and get the complete HTML
stream. Then, you can use the RegEx class to screen-scrape your HTML and
search for specific information in your string.

	See
http://samples.gotdotnet.com/quickstart/util/srcview.aspx?path=/quicksta
rt/howto/samples/net/WebRequests/clientGET.src (watch for wrapping)

Happy coding,
Minh.

-----Original Message-----
From: Chris Kersey [mailto:ckersey@m...] 
Sent: Tuesday, May 14, 2002 4:23 PM
To: ASP+
Subject: [aspx] Fetch URL (use my aspx page to retrieve html from
another url)


Hi everyone,

I'm working on a project where I need to be able to grab a web page from
a
url, parse it for a small section of information and then display it on
my
aspx page.

Is there a .net component that will allow me to fetch a url and return
the
contents as a string, or some type of object where I can obtain the html
source?  I tried using the InetCtlsObjects component, but when I tried
to
instantiate it, it gives off an error telling me that it is not licensed
for
use.  I have Visual Studio.net Architect version installed on the box I
am
trying to develp on.

Any help is much appreciated.

Chris Kersey

Excellent.  I knew there must have been a class for this but I didn't know
what it would be called.

Thanks so much!
Chris

----- Original Message -----
From: "Minh T. Nguyen" <nguyentriminh@y...>
To: "ASP+" <aspx@p...>
Sent: Tuesday, May 14, 2002 8:16 PM
Subject: [aspx] RE: Fetch URL (use my aspx page to retrieve html from
another url)


> Chris,
>
> Yes, the WebRequest and WebResponse classes in the System.Net
> namespace allows you easily fetch a URL and get the complete HTML
> stream. Then, you can use the RegEx class to screen-scrape your HTML and
> search for specific information in your string.
>
> See
> http://samples.gotdotnet.com/quickstart/util/srcview.aspx?path=/quicksta
> rt/howto/samples/net/WebRequests/clientGET.src (watch for wrapping)
>
> Happy coding,
> Minh.
>
> -----Original Message-----
> From: Chris Kersey [mailto:ckersey@m...]
> Sent: Tuesday, May 14, 2002 4:23 PM
> To: ASP+
> Subject: [aspx] Fetch URL (use my aspx page to retrieve html from
> another url)
>
>
> Hi everyone,
>
> I'm working on a project where I need to be able to grab a web page from
> a
> url, parse it for a small section of information and then display it on
> my
> aspx page.
>
> Is there a .net component that will allow me to fetch a url and return
> the
> contents as a string, or some type of object where I can obtain the html
> source?  I tried using the InetCtlsObjects component, but when I tried
> to
> instantiate it, it gives off an error telling me that it is not licensed
> for
> use.  I have Visual Studio.net Architect version installed on the box I
> am
> trying to develp on.
>
> Any help is much appreciated.
>
> Chris Kersey
>
>
>
>
>
>

Or, you can create a wsdl item that matches a pattern against the html of a
page, and loads everthing that matches into a dataset.  In this way, you can
take the html from the page and turn it into a form of webservice.

Cheers,
Frank

-----Original Message-----
From: Chris Kersey [mailto:ckersey@m...] 
Sent: Wednesday, May 15, 2002 10:56 AM
To: ASP+
Subject: [aspx] RE: Fetch URL (use my aspx page to retrieve html from
another url)

Excellent.  I knew there must have been a class for this but I didn't know
what it would be called.

Thanks so much!
Chris

----- Original Message -----
From: "Minh T. Nguyen" <nguyentriminh@y...>
To: "ASP+" <aspx@p...>
Sent: Tuesday, May 14, 2002 8:16 PM
Subject: [aspx] RE: Fetch URL (use my aspx page to retrieve html from
another url)


> Chris,
>
> Yes, the WebRequest and WebResponse classes in the System.Net
> namespace allows you easily fetch a URL and get the complete HTML
> stream. Then, you can use the RegEx class to screen-scrape your HTML and
> search for specific information in your string.
>
> See
> http://samples.gotdotnet.com/quickstart/util/srcview.aspx?path=/quicksta
> rt/howto/samples/net/WebRequests/clientGET.src (watch for wrapping)
>
> Happy coding,
> Minh.
>
> -----Original Message-----
> From: Chris Kersey [mailto:ckersey@m...]
> Sent: Tuesday, May 14, 2002 4:23 PM
> To: ASP+
> Subject: [aspx] Fetch URL (use my aspx page to retrieve html from
> another url)
>
>
> Hi everyone,
>
> I'm working on a project where I need to be able to grab a web page from
> a
> url, parse it for a small section of information and then display it on
> my
> aspx page.
>
> Is there a .net component that will allow me to fetch a url and return
> the
> contents as a string, or some type of object where I can obtain the html
> source?  I tried using the InetCtlsObjects component, but when I tried
> to
> instantiate it, it gives off an error telling me that it is not licensed
> for
> use.  I have Visual Studio.net Architect version installed on the box I
> am
> trying to develp on.
>
> Any help is much appreciated.
>
> Chris Kersey
>
>
>
>
>
>

Screen Scrape

<description>Scrape an page off the Internet and returns it as a 
string</description>

<code>
        Dim remoteURL As String = "http://www.tech-engine.com/search.asp"

        ' Create a new WebClient instance.
        Dim myWebClient As New System.Net.WebClient()

        ' DownloadData() method takes a 'UriRemote' and downloads the Web 
resource and saves it into a data buffer.
        Dim myDatabuffer As Byte() = myWebClient.DownloadData(remoteURL)

        Return System.Text.Encoding.ASCII.GetString(myDatabuffer)
</code>


You can put the above code in a function with remoteURL as a String 
paramter.