0

Can YQL scrape this site?

Hi,

I am trying to create a Yahoo! Pipe from the site listed below using the fetch page module, however, it only seems to retrieve the HTML from a page with a login box.

logis.korail.go.kr/getcarinfo.do?car_no=8201

If you visit the link above you will first be presented with a page featuring a login box, but upon refreshing the page, you are presented with a different page, complete with a table of data, and it's part of this information which I'm trying to grab in my Pipe.

I know absolutely nothing about YQL, but it was suggested that YQL was possibly the way to go in order to get the information I'm after from this page.

If any of the YQL masters on this forum know whether (and how) this can be done it would be very useful to me.

Any questions please ask.

Thanks..

by
  • HZ
  • Jan 19, 2012
1 Reply
  • It looks like that page sets a cookie on the first visit, which is sent on the second visit to show the page that you are trying to scrape. &nbsp;You need to retrieve and send that cookie. As for whether it&#39;s possible with YQL, sure but you might have to create your own data table (http://developer.yahoo.com/yql/guide/yql_dev_guide.html).<br><br> QUOTE (HZ @ 19 Jan 2012 12:50 PM) Hi,<br><br>I am trying to create a Yahoo! Pipe from the site listed below using the fetch page module, however, it only seems to retrieve the HTML from a page with a login box.<br><br>logis.korail.go.kr/getcarinfo.do?car_no=8201<br><br>If you visit the link above you will first be presented with a page featuring a login box, but upon refreshing the page, you are presented with a different page, complete with a table of data, and it&#39;s part of this information which I&#39;m trying to grab in my Pipe.<br><br>I know absolutely nothing about YQL, but it was suggested that YQL was possibly the way to go in order to get the information I&#39;m after from this page.<br><br>If any of the YQL masters on this forum know whether (and how) this can be done it would be very useful to me.<br><br>Any questions please ask.<br><br>Thanks..
    0

Recent Posts

in YQL