Yahoo Pipes - create a RSS for a multiple pages search


I want to create a pipe from a job search website to have job offers. Results are spread on several pages (page 1, 2, 3, etc.). As the pipe doesn’t run often enough, I must catch the results of the four first pages if I don’t want to miss a job offer.

I build a first Pipe to create an URL with an auto-incrementation of the number of the page. Then, I linked this first pipe to a loop where I put a second pipe that create a RSS from the page. Here is the link of the final pipe.

In my debugger, everything works fine. But when I want to run the pipe, it returns no result. I think this problem comes from the blank field in the loop. But I don’t know what to write instead. I already tried "item.url" but it brings the results of the first page only.

I hope I am clear enough. Tell me if it isn’t the case.

Thank you in advance for the time you’ll take to resolve my problem !

  • Wo
  • Jul 23, 2014
2 Replies
  • Hello,


    Your first pipe builds the urls, your second pipe fetches and formats a page, but you haven't put the built URL in the second pipe input. So you indeed have to put item.url as input.

    problem analysis & resolution

    Now, only first page is fetched is seems, why? Well, first and second+ page do not have the exact same format of their source code. In particular, the [2] of the div[contains(class,section)][2] is what fails: in pages 2+, there is only one div "section". If you take that 2 out, it works.

    debugger mode

    You thought it worked in debug but it actually didn't: in your second pipe, you had a debug value for input. As you didn't filled in the input value, the debugger used the debug value... 4 times. eg, the result you had in the debugger was 4 times the posts of the first page.


    • Why don't you use the provided (feed)[http://stage.frenchweb.fr/feed/?post_type=job_listing]? It's richer than your resulting pipe, and you can filter using pipes to have only relevant offers.
    • Finally, your XPath is probably way too specific: simply finding the table (the ol mark up) is probably more robust.


  • Hi,

    Thanks a lot for your very clear answer ! Now the pipe works great !

    To answer your first comment, I didn’t use the provided feed because I want to filter the job offers by region. The provided feed doesn’t always mention the region. So, I can’t use a keyword filter on it.



Recent Posts

in Pipes