Raw Data Download

Flurry understands that there are many possible reasons that you might need to retrieve the data that your apps send to Flurry. The Raw Data Download service within Flurry supports the extraction of data on a periodic basis.

Setup

In order to use the Raw Data Download service, you must first create a Programmatic Token. You can read about how to do that here. Upon creation, the ~~YOUR TOKEN~~ will be enabled for Raw Data Download API. You can manage the permissions for Programmatic users on the Admin > Users page by clicking the pencil icon.

Making the Request

Once you have your Programmatic Token, the process for requesting an extract involves doing a POST to an api at flurry. The following is an example call using cURL:

curl -X POST   https://rawdata.flurry.com/pulse/v1/rawData
 -H 'accept: application/vnd.api+json'
       -H 'authorization: Bearer ~~YOUR TOKEN~~
       -H 'cache-control: no-cache'
       -H 'content-type: application/vnd.api+json'
       -d '{"data": {
             "type": "rawData",
             "attributes": {
             "startTime": "1511164800000",
         "endTime": "1511251199000",
         "outputFormat": "JSON",
               "apiKey": "AAAA1111BBBB2222CCCC"
           }
       }
     }'

The following must be replaced with your own information:

  • ~~YOUR TOKEN~~ must be replaced with your Programmatic Token from the Setup section above.
  • startTime/endTime: Epochmilli timestamps communicating the time range for which sessions returned. Please note, this may not be longer than 30 days. In some cases, windows must be adjusted due to data volume. To learn more about this, see the Window Size Computation section below.
  • APIKey: The APIKey for the app for which you want to retrieve data. This can be viewed by Admin users on the Admin > Apps page in Flurry .
  • outputFormat: JSON or CSV

If the request was successfully logged for processing, the response from this call should look like this:

{
   "data":{
      "type":"rawData",
      "id":"26",
      "attributes":{
         "creationDate":1513036531916,
         "endTime":1511251199000,
         "modifiedDate":1513036531916,
         "outputFormat":"JSON",
         "requestStatus":"Acknowledged",
         "s3URI":null,
         "startTime":1511164800000
      },
      "relationships":{
         "project":{
            "data":{
               "type":"project",
               "apiKey":"AAAA1111BBBB2222CCCC"
            }
         }
      }
   }
}

From this response, the id value is the important one at the moment. The requestStatus is also important as if it does not say Acknowledged, something prevented the request from being successful (see below for possible issues and error codes).

Polling for Status

Using the value for the id key from the above, construct and send the following request by placing the id value before the ? (e.g. change 26 in the following rawData/26?fields) and including your token by replacing ~~YOUR TOKEN~~:

curl -g https://rawdata.flurry.com/pulse/v1/rawData/26?fields[rawData]=requestStatus,s3URI
-H ‘accept: application/vnd.api+json;’
-H ‘authorization: Bearer ~~YOUR TOKEN~~’ -H ‘cache-control: no-cache’ -H ‘content-type: application/vnd.api+json;’

While the request is processing, the response will be similar to the response in step 3. Periodically check the status of the request by re-submitting the request from step 3.

PLEASE NOTE: Most requests will be ready and available for download within 30 minutes. However, requests for extremely large apps can take up to a few hours.

When the request has been processed successfully, the s3URI key in the response include a URI for downloading the resulting data. For example:

{
   "data":{
      "type":"rawData",
      "id":"26",
      "attributes":{
         "requestStatus":"Success",
         "s3URI":"https://flurry-rdd.s3.amazonaws.com/downloads/26.JSON.gz?AWSAccessKeyId=AAAA1111BBBB2222CCCC&Expires=1513101235&Signature=h%2FChXRi5QwmvhUrkpwq2nVKf8sc%3D"
      }
   }
}

Within the s3URI value is also the time by which you must download the file. After the time provided, the file will no longer be available and a new request will need to be submitted to retrieve the data.

Retrieving Results

To retrieve the data, create a cURL statement with the s3URI value. For example:

curl -O https://flurry-rdd.s3.amazonaws.com/downloads/26.JSON.gz?AWSAccessKeyId=AAAA1111BBBB2222CCCC&Expires=1513039053&Signature=xbKNnTgpv1odAfVgPRLMyck8UnE%3D

Upon submitting the request, a file should be downloaded containing the data that fit the original request from step 2.

Window size computation

We allow requests that are smaller than 1 month. However if the returned data is too large, we ask that you try with smaller start/End window sizes. For example if the requested data is for 1 month and we find that the data is too large, the error response will indicate an alternate window period (either 1 week or 1 day) which is guaranteed to work.

Data

There are two formats for the data, JSON and CSV. Both formats include the same data fields described here:

  • sessionTimestamp - Represents the epochmilli when the session containing the event started.
  • appVersion - The version of the app.
  • sessionDuration - The duration of the session in milliseconds.
  • eventOffset - The number of milliseconds between the sessionTimestamp and the occurrence of the event.
  • deviceIdentifiers - A collection of device identifiers for the device that recorded the session.
  • deviceModel - The name of the model. Examples include iPhone X and Galaxy S9.
  • deviceSubModel - The device model code. Examples include iPhone10,2 and SM-G892U.
  • countryISO - The ISO country code from which the session originated.
  • eventName - The name of the event as recorded in Flurry.
  • eventParameters - A collection of key:value pairs containing the event parameters that were recorded along with the event.
  • userId - If your instrumentation utilized the Flurry.setUserId method to record an non-PII version of your user’s login id, it will be returned here.
  • gender - If your instrumentation utilized the Flurry.setGender method to record the indicated gender of the user, it will be returned here.
  • birthYear - If your instrumentation utilized the Flurry.setAge method to record the indicated age of the user, it will be returned here as the imputed birth year.
  • latitude - If GPS data was available on the device, the recorded latitude will be returned here in a truncated format.
  • longitude - If GPS data was available on the device, the recorded longitude will be returned here in a truncated format.
  • sessionProperties - If you recorded session properties using the Flurry.sessionProperties method to record information about the context of the session, the key/value pairs recorded will be included here.

JSON Format

The JSON output format includes a row for each event recorded. An example of a few events would be:

...
{
         "sessionTimestamp": "1511237978194",
         "appVersion": "1.1",
         "sessionDuration": "90759",
   "eventOffset": "31430",
   "carrier": "monkey brains",
         "deviceIdentifiers": {
         "idfv": "26B46C28-E0A0-4150-B59A-C8ECC03C6BBD"
   },
   "deviceModel": "iPhone 6",
   "deviceSubModel": "iPhone7,2",
   "countryISO": "US",
   "eventName": "ViewBoard",
   "eventParameters": {
     "boardTitle": "Around the House Nov17",
     "boardUuid": "61ab3320-aa28-46a0-b09f-1754d5fc1b61"
   },
   "userId": "",
   "gender": "Unknown",
   "birthYear": "",
   "latitude": "",
   "longitude": "",
   "sessionProperties": "{}"
       },
       {
   "sessionTimestamp": "1511237978194",
   "appVersion": "1.1",
   "sessionDuration": "90759",
   "eventOffset": "31430",
   "carrier": "monkey brains",
   "deviceIdentifiers": {
      "idfv": "26B46C28-E0A0-4150-B59A-C8ECC03C6BBD"
   },
   "deviceModel": "iPhone 6",
   "deviceSubModel": "iPhone7,2",
   "countryISO": "US",
   "eventName": "ShareBoard",
   "eventParameters": {
      "boardTitle": "Around the House Nov17",
      "boardUuid": "61ab3320-aa28-46a0-b09f-1754d5fc1b61"
   },
   "userId": "",
   "gender": "Unknown",
   "birthYear": "",
   "latitude": "",
   "longitude": "",
   "sessionProperties": "{}"
 },
 {
   "sessionTimestamp": "1511259749336",
   "appVersion": "1.1",
   "sessionDuration": "166793",
   "eventOffset": "16630",
   "carrier": "Verizon",
   "deviceIdentifiers": {
      "idfv": "96C13D79-D1B6-5489-C67D-E9FDF16A7ADF"
   },
   "deviceModel": "iPhone X",
   "deviceSubModel": "iPhone10,3",
   "countryISO": "US",
   "eventName": "DeleteBoard",
   "eventParameters": {
     "boardTitle": "My best board",
     "boardUuid": "94ad1963-ab31-64d4-f19d-1963e5ab6d71"
 },
   "userId": "",
   "gender": "Unknown",
   "birthYear": "",
   "latitude": "",
   "longitude": "",
   "sessionProperties": {
     "pushNotificationId": "08a808f-ad2890"
     }
 }
 ....

Notice that the session information for the first two lines are the same, indicating the came from the same session, while the third line is different, indicating it is from a new session.

CSV Format

The CSV format is very similar to the JSON format except that rather that being organized by kev/value pairs, the data is organized by columns separated by commas. Please note that the eventParameters column is presented as a JSON object as the number of key/value pairs is variable.

Responses

Initial Request Errors

On the initial request, there are a few reasons that a request might not be successful. In each of these cases, a response indicating the issue to be corrected will be returned.

Authorization

If the token provided does not have access to the requested data, the response will be similar to:

{"code":401,"message":"Unauthorized"}

Time Frame

If the time frame requested is not supported, the response will be similar to:

{"errors":["Cannot request report for a period greater than 31 days"]}

Request Status

The requestStatus communicates the current status of a request that was successfully received. The following are the possible states and their meanings:

  • Acknowledged: The request has been received and data is being processed
  • Failure: The request was not successfully processed for some reason. In this case, please try your request again and if it fails a second time, reach out to support@flurry.com
  • Success: The request was successfully processed, and the **s3URI **key should have the link for downloading the file

Errors while downloading the File

Expired Request

If you see a response with the following body, the request needs to be created again using the request api. We normally store contents for upto 30 days:

{"data":{"type":"rawData","id":"26","attributes":{"requestStatus":"Success","s3URI":"Expired URI. Please resubmit request"}}}

Expired URLs

Once you get back a valid S3 URI but encounter the following error, this means that the presigned URL has expired. A simple GET call will refresh the presigned url and will last for 10 mins before it expired:


   AccessDenied
   Request has expired
   2017-12-12T17:53:55Z
   2018-01-31T19:47:36Z
   14CE14DDCDF8E396
   GMG6Z6Xtb2CZV45s9p1dpBCapu6vv/jmPXck89mXbmFsvzKirc/6T/s6SI5R18gIpspVFdsURQo=