Back to YQL Forum
0

number of results

what is the total number of results that can be obtained through a query on search.web? i know i can get up to 5000 with a single query but when i try to access 5001-10000 in my program it just returns 1-5000. I've played with the query console page and this doesn't happen

e.g. select * from search.web(5000, 5000) where query = "hot dogs" works in console but not in my program (start 5000, count 5000)

select * from search.web(5000, 0) where query = "hot dogs" works in both (start 5000, count 5000)

by
Report Abuse
6 Replies
  • QUOTE (tonytors @ Feb 17 2009, 12:07 PM) <{POST_SNAPBACK}>
    what is the total number of results that can be obtained through a query on search.web? i know i can get up to 5000 with a single query but when i try to access 5001-10000 in my program it just returns 1-5000. I've played with the query console page and this doesn't happen

    e.g. select * from search.web(5000, 5000) where query = "hot dogs" works in console but not in my program (start 5000, count 5000)

    select * from search.web(5000, 0) where query = "hot dogs" works in both (start 5000, count 5000)



    couple of mistakes, the last line
    select * from search.web(5000, 0) where query = "hot dogs" works in both (start 5000, count 5000)
    should read
    select * from search.web(5000, 0) where query = "hot dogs" works in both (start 0, count 5000)


    second

    select * from search.web(5000, 5000) where query = "hot dogs"
    returns the same results as
    select * from search.web(0, 5000) where query = "hot dogs"

    so i am getting the same results in my application.


    is there any way to get more than 5000 results?
    Report Abuse
    0
  • QUOTE (tonytors @ Feb 17 2009, 02:19 PM) <{POST_SNAPBACK}>
    couple of mistakes, the last line
    select * from search.web(5000, 0) where query = "hot dogs" works in both (start 5000, count 5000)
    should read
    select * from search.web(5000, 0) where query = "hot dogs" works in both (start 0, count 5000)


    second

    select * from search.web(5000, 5000) where query = "hot dogs"
    returns the same results as
    select * from search.web(0, 5000) where query = "hot dogs"

    so i am getting the same results in my application.


    is there any way to get more than 5000 results?


    No (there is a limit of 5000 items in any given query), but it looks like you've found a bug when its trying to START at 5000 (that should give you 5000 more).

    Jonathan
    Report Abuse
    0
  • QUOTE (Jonathan @ Feb 18 2009, 02:01 PM) <{POST_SNAPBACK}>
    No (there is a limit of 5000 items in any given query), but it looks like you've found a bug when its trying to START at 5000 (that should give you 5000 more).

    Jonathan


    This should be fixed now.

    Jonathan
    Report Abuse
    0
  • QUOTE (Jonathan @ Feb 24 2009, 09:06 AM) <{POST_SNAPBACK}>
    This should be fixed now.

    Jonathan



    thanks for the help Jonathan

    perhaps im doing something wrong but i still dont see the NEXT 5000 results

    these 2 queries run in the yql console give me the same results (except for the <diagnostics> section)


    http://query.yahooapis.com/v1/yql?q=select...&format=xml START 0

    http://query.yahooapis.com/v1/yql?q=select...&format=xml START 5000
    Report Abuse
    0
  • actually this is working now sort of

    the 0-5000 query brings back pizza hut, dominos and papa johns 123 and

    the last 3 results (4997, 4998 and 4999) are

    <url>http://www.area41pizza.com/</url>
    <url>http://www.supervegan.com/?c=79</url>
    <url>http://franklindominos.com/</url>


    THE 5000-10000 query has these as the first 3 results (different from 0-5000 query)

    http://mandyspizza.com/
    http://www.powells.com/subsection/CookingandFoodPizza.html
    http://www.pizza.it/eng-site/default.htm


    but it has the same final 3 results


    <url>http://www.area41pizza.com/</url>
    <url>http://www.supervegan.com/?c=79</url>
    <url>http://franklindominos.com/</url>



    ive noticed that http://mandyspizza.com/ also appears in the 0-5000 result set
    i will have to investigate this further




    QUOTE (tonytors @ Feb 26 2009, 09:54 AM) <{POST_SNAPBACK}>
    thanks for the help Jonathan

    perhaps im doing something wrong but i still dont see the NEXT 5000 results

    these 2 queries run in the yql console give me the same results (except for the <diagnostics> section)


    http://query.yahooapis.com/v1/yql?q=select...&format=xml START 0

    http://query.yahooapis.com/v1/yql?q=select...&format=xml START 5000
    Report Abuse
    0
  • It appears that BOSS only returns up to 1000 results and there is no way to get beyond the 1000th item (this is how search.yahoo.com works as well). We will be writing a max result size for the search.* tables in the next release however in the mean time, to get valid result sets, please limit your below the 1000th item for a search term.

    i.e.

    select * from search.web(0,1000) where query='pizza'
    select * from search.web(500,500) where query='pizza'

    is okay while

    select * from search.web(1000,1000) where query='pizza'
    select * from search.web(1000,10) where query='pizza'

    will return inconsistent results.
    Report Abuse
    0
Ask a Question

Recent Posts

in YQL