Bulk operations

Overview

The Gemini bulk operations API allows you to download and edit all the objects in your advertiser account using a spreadsheet in CSV format. You can generate and download a bulk file using the bulk download service. You can also create and edit objects in bulk using the bulk upload service.

When you use the bulk operations API, you are effectively creating asynchronous jobs that are executed by Gemini. Upon submitting a job you will receive an id that you can use to poll the status of your job. When your job is completed, you will receive a token that will enable you to securely download your results.

Gemini bulk schema

Gemini bulk files are tab-separated CSVs. The encoding is UTF16-LE with BOM. On uploads, both comma and tab delimiters are supported. Note that comma separated files need to be encoded in UTF8 and tab separated files needs to be encoded in UTF16LE with BOM.

It is recommended that the file be compressed for upload. When a compressed file is uploaded, the job results will be compressed as well. Compressed files can be uploaded in either zip or gzip format.

For information on working with the Gemini bulk schema, refer to Get Started with Gemini Bulk File Fields and Object Types.

Note

The last two columns in the bulk schema, “Created Date” and “Last Update Date”, are always populated when using the Gemini APIs, regardless of the API request. These specify the creation timestamp for the entity/object type described on the CSV line, and the last updated timestamp, respectively (as long as there is no error on the line). They are both read-only. The format is yyyy-mm-dd hh:mm:ss. For example: 2015-08-29 00:11:55.

Limits

Bulk file limit is 1GB. For downloads, it is recommended to set compressedFile to “TRUE”. If compressedFile was not set to “TRUE” and the result file exceeded 1GB, an error will be thrown. The compressed results file will still be available for downloading using the token provided in the response.

The CSV generated will expire after 30 days.

Bulk changes in v3 Gemini

The following describes the fields available in Site X Device Targeting:

Name Description Type Add Delete
Site The site name to target. enum Required Required

For more information on this targeting attribute in Bulk, refer to Site X Device Targeting.

For information on this new targeting type in the v3 Gemini API, as well as a new bid modifier, refer to Targeting Attributes.

Bulk schema changes in v2 Gemini

Important

One of the primary goals of the v2 Gemini API has been to provide partners and developers with a unified format for bulk operations and data representation.

Because bulk file upload and download operations behave differently in v2 Gemini than in v1, there are breaking changes you may wish to consider when coding and integrating your apps.

In v1 Gemini, targeting attributes – like age, gender and device – utilized an Object ID to populate the targeting attribute entity ID in the result file. Exceptions to this rule included interest, custom audience and location targeting, all of which overloaded the Object ID to store either the Segment ID or Location WOEID. Because of these exceptions, v1 Gemini did not populate the entity IDs for these targeting attributes in bulk files.

To consolidate all targeting attributes in bulk files to a single, unified standard, as well as expose targeting attribute entity IDs for all objects, the v2 Gemini AP includes the following set of changes, described in the tables below.

Interest Target v1

Name Description Type Add Delete
Object ID The ID of interest. long Required Required

Interest Target v2

Name Description Type Add Delete
Object ID The ID of the targeting attribute. long Read-Only/Optional Read-Only/Optional
Segment ID The ID of interest. long Required Required

Custom Audience v1

Name Description Type Add Delete
Object ID The ID of the custom audience. long Required Required

Custom Audience v2

Name Description Type Add Delete
Object ID The ID of the targeting attribute. long Read-Only/Optional Read-Only/Optional
Segment ID The ID of the custom audience. long Required Required

Location Target v1

Name Description Type Add Delete
Object ID The Object ID of the location. Optional when specifying the WOEID value. Required for creates if the Location field is not used. long Optional Required
Location The name of the geographic location associated with the Object ID. Required for creates if the Object ID field is not used. string Optional Read-Only

Location Target v2

Name Description Type Add Delete
Object ID The ID of the targeting attribute entity. long Read-Only/Optional Read-Only/Optional
Location The name of the geographic location associated with the Location ID. Required for creates if the Location ID field is not used. string Optional Read-Only
Location ID The Location ID of the location. Optional when specifying the WOEID value. Required for creates if the Location field is not used. long Optional Required

Bulk download

Endpoint

https://api.gemini.yahoo.com/v3/rest/bulk/download

Submit a job

Call: To download a bulk file, make a POST call to the bulk/download endpoint and pass the following in the request body:

Name Description Type Required
advertiserId The ID of the advertiser whose data will be downloaded long Yes
objectType

The entity level of the requested data. Supported values:

  • ADVERTISER
  • CAMPAIGN
  • ADGROUP
  • AD
  • KEYWORD
enum Yes
       
downloadAllLevels Indicates whether all descendants of the requested objects will be downloaded as well. This field accepts either “TRUE” or “FALSE”, and will default to “FALSE” if not specified. boolean Optional
compressedFile Indicates whether the downloaded file should be compressed. Can be set to “TRUE” or “FALSE”, and will default to “FALSE” if not specified. boolean Optional
skipExcelSpecificCharFixes Indicates whether the downloaded file should skip special Microsoft Excel fixes for inserting a leading space in front of Excel-specific characters (+,-,=) and inserting a leading single quote ‘ in front of a integer with 11 or more characters. Will default to “FALSE” if not specified. boolean Optional
filters The following optional filters can be used:    
objectIds The list of ids of the objects to be downloaded. If none are specified, then all objects for that level will be downloaded. A list should not exceed 1000 ids. long Optional
objectStatus Download only objects in a certain status. For example, status=[”ACTIVE”] will result in downloading only active objects. Refer to the API objects documentation for more information on supported statuses. This filter accepts a list of statuses. enum Optional
fromDate This filter specifies that only objects that were changed since this timestamp should be downloaded. The format is yyyy-mm-dd hh:mm:ss or yyyy-mm-dd. string Optional
toDate This filter specifies that only objects that were changed before this timestamp should be downloaded. The format is yyyy-mm-dd hh:mm:ss or yyyy-mm-dd. string Optional
includeNegative Indicates whether negative keywords will be downloaded. Can be set to “TRUE” or “FALSE”, and will default to “false” if not specified. boolean Optional
includeTargeting Indicates whether targeting objects will be downloaded. Can be set to “TRUE” or “FALSE”, and will default to “false” if not specified. boolean Optional
includeExtensions Indicates whether ad extensions will be downloaded. Can be set to “TRUE” or “FALSE”, and will default to “FALSE” if not specified. boolean Optional
includeAdSiteSettings Indicates whether ad site settings will be downloaded. Can be set to “TRUE” or “FALSE”, and will default to “FALSE” if not specified. boolean Optional

Here is a sample request body for a download job:

 {
  "advertiserId": 30944,
  "objectType": "CAMPAIGN",
  "downloadAllLevels": true,
  "filters": {
             "objectIds": [123,
                456
             ],
             "objectStatus": [
                "ACTIVE"
             ],
             "fromDate": "2014-07-01 00:00:00",
             "toDate": "2014-07-10 00:00:00",
                         "includeNegative":true,
                         "includeTargeting":true,
                         "includeExtensions":true
  }
}

Response: The following is a sample response when submitting a download job:

{
  "errors": null,
  "response": {
   "jobId": 4196,
   "status": "submitted",
   "jobResponse": null
  }
}

The response includes the following fields:

Name Description
jobId The ID of the submitted job. Use this id in order to poll the status of the job.
status The initial status of a job that was successfully submitted will be “submitted”.
jobResponse When the job is completed, this field will provide the token required to download the requested data.

Once the job has been submitted, you can poll the job’s status until its completion. You should poll the status at reasonable intervals given the estimated size of your requested data.

Note

Date filters will be applied from top level to bottom level objects (campaign -> campaign level targeting including ad schedules -> ad group -> ad group level targeting -> ad). This means, in cases where the date filters are used, the children ad object data will not be fetched if the parent ad object was not changed in the date range specified.

Important

As a best practice, if a data filter is used, to download information at all levels (campaign down to all its children entities), make a bulk call for each level (for example, one for campaign, one for ad group and so on).

v3 API Filter Option For Download

A new filter option includeAdSiteSettings is available in v3 Gemini API. When set to true, this indicated whether ad site settings will be downloaded. Default value is false.

Here is a sample request body for a download job:

{
  "advertiserId": 30944,
  "objectType": "CAMPAIGN",
  "downloadAllLevels": true,
  "filters": {
             "objectIds": [
                123,
                456
             ],
             "objectStatus": [
                "ACTIVE"
             ],
             "fromDate": "2014-07-01 00:00:00",
             "toDate": "2014-07-10 00:00:00",
                         "includeNegative":true,
                         "includeTargeting":true,
                         "includeExtensions":true,
                         "includeAdSiteSettings":true
    }
  }

Bulk upload

Endpoint

https://api.gemini.yahoo.com/v3/rest/bulk/upload

Submit a job

Call: To upload a bulk file, make a POST call to the bulk/upload endpoint, passing in the advertiser id in the URL. Here is an example of a request to upload a bulk file for advertiser id 123: https://api.gemini.yahoo.com/v3/rest/bulk/upload/?advertiserId=123 The CSV file you are uploading should be embedded as form-data within the “file” field. Compressed files are accepted. When uploading a compressed file, the results file will be automatically compressed.

Note

There is a new, optional query parameter skipExcelSpecificCharFixes for bulk upload, in addition to the advertiserId query parameter, that can be passed. The skipExcelSpecificCharFixes parameter indicates whether the uploaded file should skip special Microsoft Excel fixes for inserting a leading space in front of Excel-specific characters (+,-,=) and inserting a leading single quote ‘ in front of a integer with 11 or more characters. It will default to FALSE if not specified.

Response:The following is a sample response when submitting a job:

{
  "errors": null,
  "response": {
      "jobId": 7015,
      "status": "submitted",
      "jobResponse": null
  }
}

The response includes the following fields:

Name Description
jobId The ID of the submitted job. Use this id in order to poll the status of the job.
status The initial status of a job that was successfully submitted will be “submitted”.
jobResponse When the job is completed, this field will have details on the results of the job.

Once the job has been submitted, you can poll the job’s status until its completion. You should poll the status at reasonable intervals given the estimated size of your requested data.

Get job status

Call: To query the status of your request, make a GET call to

/bulk/status/?jobId={jobId}&advertiserId={advertiserId}

For example, in order to check the status of jobId 4196 that requested data for advertiser 30944, make the following GET call:

Response - download: The following is a sample response of a bulk download job:

{
  "errors": null,
  "response": {
      "jobId": 14151,
      "status": "completed",
              "jobResponse": "iY.mAQcBTiZfotBOtMDnYDUsothZsIbpfg0bP58oKXbAA6VJlCwuuV2oNsNhGQ7tVcEfVM2NVsIsUoo8EYOj135iBehdHZAF9mPlTBmq9JfWbGLJ9OUpD2JWSwlROI9Kj1Pjo.4PQ65zGMowH5D0zTzqWumt8mHWP.E8n8NtcD2F"
                 }
 }

Upon successful completion, the response will include the token you will need to securely download your file with the requested data. This file is available to be downloaded within 30 days of the job’s completion. The meaning of the possible statuses are detailed below:

Name Description
submitted The job is in queue but work on it has yet to commence.
running The job is running.
failed The job ran but unexpectedly failed.
killed The job was killed, typically due to failure to complete in a timely manner.
completed The job completed successfully.

Response - upload: the following is a sample response of an upload job upon completion:

{
"errors": null,
"timestamp": "2016-06-13 16:57:41",
"response": {
  "jobId": 26597,
  "status": "completed",
  "jobResponse": {
    "Invalid_Row": {
      "totalCount": 0
    },
    "Ad": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Age_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Site_Exclusion": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Sitelink_Extension": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Call_Extension": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Location_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Device_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Ad_Schedule": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "OS_Version_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Radius_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Product_Group": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Keyword": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Shared_Sitelink": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Custom_Audience": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Shared_Negative_Keyword": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Campaign_Shared_Set": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Campaign": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Gender_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Shared_Set": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Ad_Group": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Product_Filter": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Interest_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Shared_Sitelink_Setting": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "Connection_Target": {
      "totalCount": 0,
      "addedCount": 0,
      "updatedCount": 0,
      "deletedCount": 0,
      "failedCount": 0
    },
    "resultFile": "Q_Fy4sBGlxvzkrj.O_6nfabIu2lwu1AapjNQXhyVZX3wkag3Qdl_xo0ubHHRGG8plok_7rzZCSjVO5jjH647ozkKWbWwXOhDunEedtAMIb_24MpQChnQkOge1PaCLA5.6Ho-~A"
    }
  }
}

When an upload job completes, the jobResponse will include details on the number of added, updated, deleted and failed objects for each object type. The resultFile field will include a token that should be used to download the result file. The result file is a spreadsheet that has one row for every object that was passed and includes error messages for failed objects.

Download job results

Once the job has completed, you will need to either download your requested data (on a download job) or get your results file (on an upload job). You can do this by making a GET call to the https://api.gemini.yahoo.com/v3/rest/bulk/read/ endpoint and passing in the token you received in either the resultFile or the jobResponse using the resource parameter.

Here is an example:

https://api.gemini.yahoo.com/v3/rest/bulk/read/?resource=iY.mAQcBTiZcZfotBOtNMDnYDUsothZsIbVpfg0bP5R8oKXbAA6VJlCwuuV2oNsNhGQ7tVcEfVM2pNVsIsUoo8EYOj135iBehdHZAF9mPlTBmq9JfWbGLM9OUpD2JWSwlROI9Kj1Pjo.4PQ65zGMowH5D0zTzqlxWumt8mHWP.E8n8NtcD2F

Important

Bulk Download does not currently work if you want to Download only Product Group object types. There is a workaround, however, which you can use. If you change the objectType to Campaign and download the entire campaign, it will work.

Caveats

Budgets that are set to less than the minimum value (for example, $3 rather $5) will automatically default to the minimum bid value, i.e., $5. No error will be reported to the user. This is standard Gemini behavior. Note that when working with the Gemini UI, you will be able to identify such an error and take the necessary steps to fix it. When working with the BULK API, this may be more difficult to identify and thus fix, so the standard behavior is to default to the minimum bid value.