| Author |
 Topic  |
|
|
Astralis
Senior Member
USA
1218 Posts |
 Posted - 08 November 2003 : 19:27:07
|
I have a site that I would like to grab headlines from other sites and put them on my site. I've been reading that I could create an XML file to do this and parse the data.
Could anyone point me to a resource to learn about this or explain how to do it? XML is still a little baffling to me. 
BTW, before anyone comments, I have permission to grab the data and am currently doing it manually. 
(Admins - could you please move this to the ASP category?) |
Edited by - Astralis on 08 November 2003 19:35:27 |
|
|
dayve
Forum Moderator
USA
5820 Posts |
Posted - 08 November 2003 : 19:37:55
|
Here is a combination of screen scraping and output displaying using the GDI.
 - Scraped from http://www.census.gov/cgi-bin/ipc/popclockw
The main part of the scraping is accomplished this way:
HTMLGet = Server.CreateObject("Msxml2.ServerXMLHTTP.3.0")
HTMLGet.Open("GET", "http://www.census.gov/cgi-bin/ipc/popclockw", false)
HTMLGet.Send
strText = HTMLGet.responseText
HTMLGet = Nothing
' Name
strName = strText : i = 0 : j = 0
i = instr(strName,"<h1>")
j = instr(i,strName,"</h1>")
strName = (mid(strName,i,j-i))
strLine1 = "World Population: "
strLine2 = (replace(strName,"<h1>",""))
'### Concatenate Main String
strOutputText = strLine1 & vbNewLine & _
" " & strLine2
You need to identify something in the page you want to scape. In the example above I used the <h1></h1> heading tags.
Once you have that, you can display the data anyway you want. Here is the entire script for the image shown above.
http://www.burningsoulsforum.com/pdwp.txt
You can also search the net for Screen Scraping. You can use the XML parser or the new .NET parser (which I am using for my signature portion on the far right). |
 |
 |
|
|
Astralis
Senior Member
USA
1218 Posts |
Posted - 08 November 2003 : 19:43:30
|
| Wow! Thanks Dayve. Very useful. |
|
|
|
Astralis
Senior Member
USA
1218 Posts |
Posted - 08 November 2003 : 20:20:06
|
| Has anyone been able to screen scrape in Classic ASP. It seems that all the tutorials and examples on the search pages are for asp.net, as well as Dayve's example. |
|
|
|
Doug G
Support Moderator
USA
6493 Posts |
Posted - 08 November 2003 : 21:00:44
|
Dayve's example looks pretty classic to me :)
|
======
Doug G
======
Computer history and help at www.dougscode.com |
|
|
|
Gremlin
General Help Moderator
New Zealand
7528 Posts |
Posted - 08 November 2003 : 21:30:29
|
Definately a Classic :) |
Kiwihosting.Net - The Forum Hosting Specialists
|
|
|
|
Astralis
Senior Member
USA
1218 Posts |
Posted - 08 November 2003 : 21:40:18
|
What would I need to do to replicate that? I'd like to look at how you did it by running it and deconstructing it on my server. If I copied the example from your link, should it work?
Gremlin and Doug said it's classic ASP but I thought you wrote it in for a .NET server - sorry. |
|
|
|
dayve
Forum Moderator
USA
5820 Posts |
Posted - 08 November 2003 : 21:42:10
|
| I wrote it using .NET because I needed to create an image. You can use the XML object and simply do a Response.Write with the string. |
|
|
|
|
Astralis
Senior Member
USA
1218 Posts |
Posted - 08 November 2003 : 21:45:04
|
| I see you're listening to OMD - cool! |
|
|
|
RebelTech
Average Member
USA
613 Posts |
Posted - 08 November 2003 : 21:52:42
|
| Also, do a google search for grabnews. It is a xml and db solution. It will grab headlines and store them in the database and will only update them every so often. Keeps the bandwidth down. |
|
|
|
Astralis
Senior Member
USA
1218 Posts |
Posted - 08 November 2003 : 23:49:32
|
Grabnews is cool. I'm still learning, though, how to grab headlines from sites without RSS Feeds.  |
|
|
|
Astralis
Senior Member
USA
1218 Posts |
Posted - 09 November 2003 : 00:23:22
|
Thanks for your patience everyone. I'm trying to learn this.
On Planetsourcecode, there is a sample that will do exactly what I'm looking for. The author presumes the users will know the basics to the code so he only included essential codes. Unfortunately, I'm learning the basics right now. Using his code, what would I need to do to be able to create the exact sample that he created (seen in the screen-shot)?
View code on planetsourcecode.com. |
|
|
|
n8pbm
Junior Member
USA
212 Posts |
|
| |
Topic |
|
Topic Locked
