| Author |
 Topic  |
|
redbrad0
Advanced Member
USA
3725 Posts |
 Posted - 31 July 2003 : 16:29:16
|
I am using the code below to grab text from a webpage..
Function GetHTML(strURL)
on error resume next
Set objXMLHTTP = Server.CreateObject("Microsoft.XMLHTTP")
objXMLHTTP.Open "GET", strURL, False
objXMLHTTP.Send
strReturn = objXMLHTTP.responseText
Set objXMLHTTP = Nothing
GetHTML = strReturn
End Function
I am looping thru about 200 sites checking some information on the site but the problem is if the site is down then my page just times out. Does anyone know a way around this? Where say if it takes longer then 30 - 45 secs it skips the site and goes to the next line of code? |
Brad
Oklahoma City Online Entertainment Guide
Oklahoma Event Tickets |
|
|
DavidRhodes
Senior Member
United Kingdom
1222 Posts |
Posted - 31 July 2003 : 16:39:11
|
Can you not play around with the script.timeout, ie set it to 30 seconds and put on error resume next in your page.
I did a similar thing last year and found the xmlhttp object to be fairly unstable anyway and use c#(.net) instead, you've then got threading etc to speed things up. |
The UK MkIVs Forum |
 |
|
|
redbrad0
Advanced Member
USA
3725 Posts |
Posted - 31 July 2003 : 16:54:48
|
Well if the script timesout then the page will just not display so no further script can run right?
I have never used .net basically what I have is something simple but does take awhile for the code to run. My server has .net is this something that a .net program can grab the information and then be included into regular asp to run thru the rest of the querys? |
Brad
Oklahoma City Online Entertainment Guide
Oklahoma Event Tickets |
|
|
|
DavidRhodes
Senior Member
United Kingdom
1222 Posts |
Posted - 31 July 2003 : 17:07:00
|
Yeah, I did a .net webservice that looped x sites then looped x search engines within (to find position in seaarch engine), this built up an XML file which was returned to a VB6 application using the XML object, so you could easily do something similar in ASP.
Anyway, I meant something like this origianally....
Server.ScriptTimeout = 30
Function GetHTML(strURL)
on error resume next
Set objXMLHTTP = Server.CreateObject("Microsoft.XMLHTTP")
objXMLHTTP.Open "GET", strURL, False
objXMLHTTP.Send
if err.number <> 0 then
strReturn = objXMLHTTP.responseText
Set objXMLHTTP = Nothing
GetHTML = strReturn
else
Set objXMLHTTP = Nothing
GetHTML = ""
end if
End Function
|
The UK MkIVs Forum |
Edited by - DavidRhodes on 31 July 2003 17:08:49 |
|
|
|
redbrad0
Advanced Member
USA
3725 Posts |
|
|
seven
Senior Member
USA
1037 Posts |
Posted - 31 July 2003 : 18:00:40
|
| sort of off topic, when you grab text from the webpage. Are you only grabbing specific text from the webpage or the whole page? I've been trying to do this but it always seems like you need to pull from specific DIV tags or it doesn't know where to pull from. |
|
|
|
|
redbrad0
Advanced Member
USA
3725 Posts |
|
|
Gremlin
General Help Moderator
New Zealand
7528 Posts |
Posted - 31 July 2003 : 19:42:42
|
| Thats basically the only way to do it, you can't selectively "tear" parts of a page. |
Kiwihosting.Net - The Forum Hosting Specialists
|
|
|
|
dayve
Forum Moderator
USA
5820 Posts |
Posted - 31 July 2003 : 23:40:20
|
quote:
Originally posted by Gremlin
Thats basically the only way to do it, you can't selectively "tear" parts of a page.
sure you can, I've done it before. You can search for certain html tags within a document, especially if they are unique tags. I used to do this to grab specific information from the census world population web site. Let me find an example of the script I used and I will post it shortly. |
 |
|
|
|
dayve
Forum Moderator
USA
5820 Posts |
|
|
Gremlin
General Help Moderator
New Zealand
7528 Posts |
Posted - 01 August 2003 : 00:50:14
|
quote:
Originally posted by dayve
quote:
Originally posted by Gremlin
Thats basically the only way to do it, you can't selectively "tear" parts of a page.
sure you can, I've done it before. You can search for certain html tags within a document, especially if they are unique tags. I used to do this to grab specific information from the census world population web site. Let me find an example of the script I used and I will post it shortly.
I think you missunderstood what I was trying to say, you are selectivly taking stuff from the page AFTER you've "teared" it from the remote website, you can only HTMLHTTP an entire page at once, not part of it ... hmm that make sense ?
quote:
yes i grabbing the entire page and then checking to see if certain data is contained in the html
I was replying to that saying yes thats the only way to do it, you tear the page in it's entirety and then selectively determine which bits you want by using subsequent code etc. |
Kiwihosting.Net - The Forum Hosting Specialists
|
Edited by - Gremlin on 01 August 2003 00:52:15 |
|
|
|
dayve
Forum Moderator
USA
5820 Posts |
Posted - 01 August 2003 : 00:52:27
|
gotcha!  |
|
|
|
|
Gremlin
General Help Moderator
New Zealand
7528 Posts |
Posted - 01 August 2003 : 03:21:30
|
| :) ... I've actually been using XMLHTTP etc A LOT the last few weeks for various tasks, getting very good at writing complex instr and replace statements for pulling various bits of web content hehe. |
Kiwihosting.Net - The Forum Hosting Specialists
|
|
|
|
seahorse
Senior Member
USA
1075 Posts |
Posted - 01 August 2003 : 03:41:40
|
is there anyway to know that someone might be pulling text content from your site using this technique?
|
Ken
===============
Worldwide Partner Group
Microsoft |
|
|
|
Gremlin
General Help Moderator
New Zealand
7528 Posts |
Posted - 01 August 2003 : 05:16:48
|
| it just looks like any other request from what I can tell in the logs. |
Kiwihosting.Net - The Forum Hosting Specialists
|
|
|
|
VodkaFish
Average Member
USA
654 Posts |
Posted - 01 August 2003 : 13:09:38
|
| I'm currently fiddling around with this solution: I'm reading someone else's XML file. They update every 10 minutes. I might have a lot of requests on my page for this, so I'm trying to set up a script to run every 10 minutes, get their info, store it on my own site, and then just read it from there. That way, if they go down, I still have the older info and my page won't have any lag. |
v ø d k â f ï § h |
|
|
|
Topic |
|
Topic Locked
