Reading HTML from internet

Ernie · April 25th, 2007, 03:01 PM

Hi,

I'm trying to write a VB program that reads a table of
exchange rates off a web site. I have tried to use the Inet
control but then I don't get the whole HTML code. I tried

    Dim strX As String
    strX = Inet1.OpenURL("http://www.choicefx.com/prices.html")
    Open "D:\FX Lab\FXTest.txt" For Output As #1
    Print #1, strX
    Close #1

Didn't work, why? I think it have something to do with 'frames' on
web site but I don't know how to handle that.
I have also tried to use a WebBrowser control but then I don't know
how to extract the HTML content. I tried:

    Set WB = CreateObject("internetexplorer.application")
    WB.Navigate2 "http://www.choicefx.com/prices.html"
    WB.Visible = True

How do I continue? Would be great if someone had a solution to this.

Thanks, Ernie

BrianWren · April 26th, 2007, 10:27 AM

First of all, you should not hard-code file numbers. There is a possibility that file #1 is already in use when you run your code. Use FreeFile to obtain the next available number into a variable, then use that:

Code:

    Dim fNum As Integer
    fNum = FreeFile
    Open "D:\FX Lab\FXTest.txt" For Output As #fNum
    Print #fNum, strX
    Close #fNum

Question: You say, âDidnât work, why?â. What did happen when you ran this?
What happened with

Code:

    Set WB = CreateObject("internetexplorer.application")
    WB.Navigate2 "http://www.choicefx.com/prices.html"
    WB.Visible = True

Ernie · April 26th, 2007, 11:31 AM

Hi Brian (and others),

Yes running this code opens the webbrowser. However I am
interested in getting hold of the entire HTML source
code as text. This is what I don't know how to do. How
do you?

Ernie

BrianWren · April 26th, 2007, 02:02 PM

I ask again:

Question: You say, âDidnât work, why?â. What did happen when you ran this?

Ernie · April 26th, 2007, 04:09 PM

What happend was that it opend a browser window.
However the whole point with my exercise is to write
VB code that generates the (complete) SOURCE CODE for the
web page - not the PAGE itself. I'm working on it and
will be back if I find a solution. (Should I use the 'All'
property?)

BrianWren · April 26th, 2007, 05:14 PM

OK. One more question, then I'll find an answer for you.

This Inet1 is an object on your from's face, yes? What is this control type called in the toolbox?
If you have added it as an object only in code, what is your instantiation code?

Ernie · April 26th, 2007, 05:48 PM

OK, I Did this:
Started a new VB project.
Then went to menu and clicked 'project', 'components'
I then choose 'Microsoft Internet Transfer Control'
(It lives in C:\WINDOWS\system32\MSINET.OCX)
It now appears in my toolbox.
I lift it (Inet1) onto the empty form.

Now I add the code to Form1:

Private Sub Form_Load()

    Dim strX As String
    strX = Inet1.OpenURL("http://www.choicefx.com/prices.html")
    Open "D:\FX Lab\FXTest.txt" For Output As #1
    Print #1, strX
    Close #1

End Sub

Run the program and get the result:

<html>
<head>
  <title>Choice FX Live Data</title>
</head>
<frameset rows="10%,*,12%" border="0" cols="*">
  <frame name="prices_top" src="banner-join.html" scrolling="no" frameborder="0" noresize>
  <frame name="prices_bottom" src="cgi-bin/liveprices.pl" scrolling="no" frameborder="0" noresize>

  <frame name="help" src="help.html" scrolling=no frameborder="0" noresize>
</frameset>
<noframes></noframes>
</html>

But this is not the complete HTML source that I get
from IE if I open it by hand. Why? (Note the <noframes> tags)
Could this be fixed?
(PS sorry about the file #1.)

BrianWren · April 27th, 2007, 09:55 AM

This is because of the frames.
If you open this URL in a browser, then select [u]V</u>iew|Sour[u]c</u>e from the menu system, you will get the same HTML as you just posted. If you right-click the document that is in one of the frames (ie, right-click the content), then select the [u]V</u>iew Source selection from the context menu, then you will see the HTML of the document contained in the frame.

To get this HTML in your program you will need to use something like this (I might have the names wrongâyouâll have to look ehtat upâbut the general idea is right):

Code:

    Dim strX As String
    Dim fNum As FreeFile

    strX = Inet1.OpenURL("http://www.choicefx.com/prices.html")
    Open "D:\FX Lab\FXTest.txt" For Output As fNum
    Print fNum, strX

    ' Perhaps the statement below should be: 
    '           .Document.Frames("prices_top").document.outerHtml
    ' but you get the idea.  You need to refer to the contents of the frame(s).
    ' Alternately, you could get the URL of the src property, and open that
    ' in the InetTransferControl.  For the first frame, it would be
    ' http://www.choicefx.com from the main document, a '/' and the document from
    ' the frame: http://www.choicefx.com/banner-join.html .

    strX = Inet1.Document.Frames("prices_top").outerHtml  ' I forget the actual
    Print fNum, strX                                      ' case of the 
                                                          ' letters...
    strx = Inet1.Document.Frames("prices_bottom").outerHtml
    Print fNum, strX

    strx = Inet1.Document.Frames("help").outerHtml
    Print fNum, strX

    Close fNum

Ernie · April 28th, 2007, 10:10 AM

The website that I use as an example seems to be down.
Here is another one with frames that I'm experimenting with......
"http://ssp.co.uk/default.asp?lang=1". It contains a frame
called "right_flik".

BrianWren · April 30th, 2007, 12:29 PM

Well, you know: substitute the names I used as examples with the names that match your reality...