0

Best, fast way to pull mail headers over OAuth

I'm trying to pull emails for people's accounts that they've OAuthed and am having trouble with performance of the Webservice API. I need to get all of the envelope data, so I'm pulling the raw headers with the GetMessageRawHeader endpoint on the JSON-RPC API and finding it to be very slow. It looks like the average response time is around 10s, and I can only pull 10 messages per request. I'm also seeing frequent responses with "Server.MailboxOpenFailed.Busy" that take 30s to complete. Is there another way that I should be doing this? I've also tried to use the ListMessages endpoint, and while that was faster it doesn't return complete header info so won't work for me.

Thanks!

by
12 Replies
  • which headers are you looking to get? GetMessageRawHeader is slow and is not recommended to use. You can try using the yql search table (ymail.search) if ListMessages doesn't give you all the headers.

    0
  • I need all of the envelope data, with complete To/From/CC data. The ListMessages endpoint only returns to and from, and then it will often return names instead of addresses. If I use ymail.search, what should I search on? Will that be faster and return the real headers?

    0
  • Any chance of getting some help here?

    0
  • Can you tell what all queries you tried? You can use "keyword" search. You can do something like this' for messages in Inbox.

    0
  • I meant you can go to the yql console and type select * from ymail.search where keyword=' '

    0
  • Hi Ram, that doesn't seem to solve my problem, since I don't want to search by keyword. I tried your suggestion with an empty keyword and it returns no results.

    What I'm trying to do is pull the complete envelope data (timestamp, and all to/from/cc/bcc addresses) for all of a user's emails in the past three months. I've run into a number of problems in trying to do that, including:

    • On the webservice API, GetMessageRawHeader is very slow, as described in my message above.
    • Other endpoints on the webservice API like ListMessages return MessageInfo objects that don't include the full envelope data. They only contain to and from, and from often contains a name rather than an address.
    • Using the YQL API, I tried pulling the data through ymail.msgcontent, and while that includes the data I need it quickly hits what looks like rate limiting but isn't labeled as such. The documents simply come back with no results. When this happens, I've noticed that the webservice API starts returning HTML indicating a temporary problem on your end.
    0
  • Hi Ram, that doesn't seem to solve my problem, since I don't want to search by keyword. I tried your suggestion with an empty keyword and it returns no results.

    What I'm trying to do is pull the complete envelope data (timestamp, and all to/from/cc/bcc addresses) for all of a user's emails in the past three months. I've run into a number of problems in trying to do that, including:

    • On the webservice API, GetMessageRawHeader is very slow, as described in my message above.
    • Other endpoints on the webservice API like ListMessages return MessageInfo objects that don't include the full envelope data. They only contain to and from, and from often contains a name rather than an address.
    • Using the YQL API, I tried pulling the data through ymail.msgcontent, and while that includes the data I need it quickly hits what looks like rate limiting but isn't labeled as such. The documents simply come back with no results. When this happens, I've noticed that the webservice API starts returning HTML indicating a temporary problem on your end.
    0
  • Looks like the forum has a problem and it striped what i typed in the keyword section and doesn't seem to even help with the linking. Try this and see if it works.

    http://developer.yahoo.com/yql/console/#h=select%20*%20from%20ymail.search%20where%20keyword%3D%27folder%3A1%27

    0
  • Hi Ram,

    Is there a way to request other message headers (not included to the envelope) through YQL?

    Thanks in advance.

    0
  • Hi Ram,

    What does the "folder=1" part of that query do? What I need to do is:

    • Pull the last 30 days of email from all folders
    • Then periodically pull any new emails in all folders that have come in since the last pull

    Is there a way to acheive that with this query?

    Thanks, Aubrey

    0

  • What do you mean by other message headers? If you need all headers the only way is to get it using GetMessageRawHeaders?


    QUOTE(Serhiy @ 8 May 2013 2:10 AM)

    Hi Ram,

    Is there a way to request other message headers (not included to the envelope) through YQL?

    Thanks in advance.

    1
  • Thank you.

    0

Recent Posts

in Yahoo! Mail Web Services API