0

I think I may be locked out of a table. Is this possible and if so, why?

Hi,

I've been trying to develop an app that will show users stock option quotes. I intend to download the entire options chain fro yahoo every 30 seconds..

Normally, I would just access the open data table but now I am suddenly getting this response below for queries that used to work. I'm guessing I've been locked out but I don't know why. I am way below the limit of 1000 requests per hour per IP. Does anyone have any ideas of what the problem is and how I can correct it? During testing, I did a bunch of refreshes pretty fast (probably 15 requests in a minute or so) if that means anything.


cbfunc({
"query": {
"count": 0,
"created": "2011-05-12T18:14:08Z",
"lang": "en-US",
"diagnostics": {
"publiclyCallable": "true",
"redirect": {
"from": "http://datatables.org/alltables.env",
"status": "301",
"content": "http://www.datatables.org/alltables.env"
},
"url": [
{
"execution-time": "3",
"proxy": "DEFAULT",
"content": "http://www.datatables.org/yahoo/finance/yahoo.finance.options.xml"
},
{
"error": "Redirected to a robots.txt restricted URL: http://finance.yahoo.com/q/op?s=goog",
"execution-time": "1",
"http-status-code": "403",
"http-status-message": "Forbidden",
"proxy": "DEFAULT",
"content": "http://finance.yahoo.com/q/op?s=goog"
}
],
"javascript": [
"Exception: Got error stream: null, url: http://finance.yahoo.com/q/op?s=goog, status: 0",
{
"execution-time": "3",
"instructions-used": "2000",
"table-name": "mytable"
}
],
"user-time": "34",
"service-time": "4",
"build-version": "17886"
},
"results": null
}

by
3 Replies
  • Can you slap on a query parameter of debug=true and see if the problem persists. It looks like there was an issue fetching the robots.txt from the finance.y.c host.
    0
  • QUOTE (The Josh @ May 12 2011, 01:18 PM) <{POST_SNAPBACK}>
    Can you slap on a query parameter of debug=true and see if the problem persists. It looks like there was an issue fetching the robots.txt from the finance.y.c host.


    Thanks for the response. So it started working again. And then it stopped again. I tried the debug=true when it stopped working and I got 3 different responses in a row for the exact same url.


    cbfunc({"query":{"count":0,"created":"2011-05-13T03:39:53Z","lang":"en-US","diagnostics":{"publiclyCallable":"true","warning":"Cannot load an external table environment file from http://datatables.org/alltables.env","url":[{"execution-time":"4","id":"0723738b-9db8-4131-b37e-c64b1dd9359c","proxy":"DEFAULT","content":"http://www.datatables.org/yahoo/finance/yahoo.finance.options.xml"},{"error":"Connect Failure","execution-time":"5002","proxy":"DEFAULT","content":"http://finance.yahoo.com/robots.txt"},{"error":"Redirected to a robots.txt restricted URL: http://finance.yahoo.com/q/op?s=FMCN&m...=2013-01"}],"javascript":["Exception: Got error stream: null, url: http://finance.yahoo.com/q/op?s=FMCN&m=2013-01, status: 0",{"execution-time":"5004","instructions-used":"2000","table-name":"ops"}],"user-time":"10014","service-time":"10009","build-version":"17886"},"results":null}});


    cbfunc({"query":{"count":0,"created":"2011-05-13T03:39:01Z","lang":"en-US","diagnostics":{"publiclyCallable":"true","warning":"Cannot load an external table environment file from http://datatables.org/alltables.env","url":[{"execution-time":"12","id":"b6e2cb87-f711-4df2-8241-bb31c6ef86fd","proxy":"DEFAULT","content":"http://www.datatables.org/yahoo/finance/yahoo.finance.options.xml"},{"execution-time":"12","id":"19b102af-7d0b-4488-acc2-673889ac4c11","proxy":"DEFAULT","content":"http://finance.yahoo.com/robots.txt"},{"execution-time":"8","id":"ebfddd9a-b49e-40c6-8540-51de586ef13b","proxy":"DEFAULT","content":"http://finance.yahoo.com/robots.txt"},{"error":"Read timed out","execution-time":"8688","http-status-code":"200","http-status-message":"OK","proxy":"DEFAULT","content":"http://finance.yahoo.com/q/op?s=FMCN&m=2013-01"}],"javascript":["Exception: Got error stream: null, url: http://finance.yahoo.com/q/op?s=FMCN&m=2013-01, status: 0",{"execution-time":"8691","instructions-used":"2000","table-name":"ops"}],"user-time":"13707","service-time":"8720","build-version":"17886"},"results":null}});

    And then the last one was a result that actually included the expected data. There are a couple interesting things

    1) I can go to the site http://finance.yahoo.com/q/op?s=FMCN&m=2013-01 if I navigate via my browser
    2) When I try the queries from a different ip (iphone not on wifi), I also get errors. But when it works on my computer, it works on my phone as well. This to me implies that there might be an issue with the service but I don't really know enough about it to be sure.
    3) When it stopped working, it seemed to be down for 30 mins to an hour or so.
    0
  • I am running into similar issues using a yahoo.finance table. The YQL is static and been working for about a year without issue. It seems to be a throttle issue, then an enforced timeout. I am performing 500 queries to get the industries of the SP500. The error error being returned back is an internal error has occurred.
    0

Recent Posts

in YQL