cannot retrieve data because of redirecting
i have a problem with retrieving data from nytimes.com website. find below the code that i have written. nytimes.com requires login to see the content. i send the login information in postData, however i am not redirected to the proper address, but to the login page. however when i try the url with the postdata it actually works in browsers. is there someone overthere to help.
PS: you need a userid and pass to actually run this script
Public Function RetrievePage()
Dim postData As String
postData = "URI=http://www.nytimes.com/2004/12/09/garden/09TURF.html&USERID=heregoesyourid&PASSWORD=heregoe syourpass&is_continue=true&SAVEOPTION=Yes"
Dim request As HttpWebRequest
Dim response As HttpWebResponse
request = CType(WebRequest.Create("http://www.nytimes.com/auth/login"), HttpWebRequest)
request.ContentType = "text/html;iso-8859-1"
request.ContentLength = postData.Length
request.Method = "POST"
request.AllowAutoRedirect = True
Dim requestStream As Stream = request.GetRequestStream()
Dim postBytes As Byte() = Encoding.ASCII.GetBytes(postData)
requestStream.Write(postBytes, 0, postBytes.Length)
response = CType(request.GetResponse(), HttpWebResponse)
Dim r As New StreamReader(response.GetResponseStream())
Dim Page As String = r.ReadToEnd()