Upgrade Notes

Users new to DataSource can skip this section and proceed directly to the Getting Started section. Implementers who are upgrading from previous versions of DataSource should note the following changes as of version 2.5.0:

• usage of a callback object literal rather than a callback function and caller reference,
• access to the entire data response after it as been type-converted but before it has been schema-parsed,
• consistency to the oParsedResponse object's members returned by the DataSource to the callback.

While much care has been taken to support backward compatibility whenever possible, implementers are advised to read through this user guide carefully, and upgrade their implementation code at their earliest convenience.

JSON Schema Restriction

When defining JSON schemas, the value for resultsList must now be of type String.

Callback Integration Changes

Calling myDataSource.sendRequest() no longer requires sending the tandem values oCallback (HTMLFunction) and oCaller (Object) values. Instead, DataSource's sendRequest() method now accepts an oCallback object literal with the following members:

• success {HTMLFunction} Handler function that executes in success case.
• failure {HTMLFunction} Handler function that executes in failure case.
• argument {MIXED} Arbitrary data which will get passed along to handler function.
• scope {Object} Scope in which to execute handler function.

As a result of the change to the DataSource's callback value, the following APIs have been changed:

• handleResponse(oRequest, oRawResponse, oCallback, oCaller, tId)
makeConnection(oRequest, oCallback, oCaller)
sendRequest(oRequest, oCallback, oCaller)
setInterval(nMsec, oRequest, oCallback, oCaller) - In each of these methods, the oCallback argument is now an object literal, and the oCaller argument has been deprecated in favor of oCallback.scope.
• cacheRequestEvent
dataErrorEvent
getCachedResponseEvent
requestEvent
responseCacheEvent
responseEvent
responseParseEvent - In each of these Custom Events, the oArgs.callback argument is now an object literal, and the oArgs.caller argument is now deprecated in favor of oArgs.callback.scope.

Access to oFullResponse

As of version 2.5.0, implementers will now have access to the entire data response after it as been type-converted from oRawResponse, but before it has been schema-parsed into oParsedResponse, and it is referred to as oFullResponse. In order to support this access to oFullResponse, the following APIs have been changed:

• doBeforeCallback(oRequest, oFullResponse, oParsedResponse)
doBeforeParseData(oRequest, oFullResponse)
parseArrayData(oRequest, oFullResponse)
parseHTMLTableData(oRequest, oFullResponse)
parseJsonData(oRequest, oFullResponse)
parseTextData(oRequest, oFullResponse)
parseXMLData(oRequest, oFullResponse) - The second argument has been changed from oRawResponse to oFullResponse.

Consistent oParsedResponse Signature

When DataSource's handleResponse() method returns the schema-parsed response object to the given callback function, it will now have the following members:

• tId {Number} - Unique transaction ID.
• results {Array} - Array of schema-parsed data results.
• error {Boolean} - True if there was an error.
• meta {Object} - A collection of (optional) additional field values.

Getting Started

The DataSource code must be included before any control can use it:

<!-- Dependencies --> <script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/yahoo/yahoo-min.js"></script> <script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/event/event-min.js"></script> <!-- OPTIONAL: JSON Utility --> <script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/json/json-min.js"></script> <!-- OPTIONAL: Connection (enables XHR) --> <script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/connection/connection-min.js"></script> <!-- Source file --> <script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/datasource/datasource-beta-min.js"></script>

Instantiating a DataSource

To create a DataSource instance on your page, pass a pointer to your live data to the constructor and set any appropriate configuration properties. The first mandatory argument to instantiate a DataSource is the location from which it should fetch data. It can be any one of several types:

String

A base URL for a remote server that will provide the data

XML object

A reference to an XML DOM object

HTML table reference

A reference to a <table> HTML element

Array

A native JavaScript array

Function reference

Reference to a function that will provide the data

Object

An object that, at some depth, has an array of data

The second optional argument, if present, should be a configuration object; this object allows you to preconfigure the properties of the DataSource in a single step at instantiation. All properties are accessible here, though they can later be modified by explicit assignments.

When providing a URL for XHR DataSources, it is important that it has a trailing ? or / so that the arguments of the query (first argument of sendRequest; see below) can be appended (in the case of a GET request). If set to do POST requests, such trailing characters will be innocuous, so it is better to have them there just in case you switch methods.

// Data Source from a local array var myDataSource = new YAHOO.util.DataSource( [ {name:"Ashley",breed:"German Shepherd",age:12}, {name:"Dirty Harry",breed:"Norwich Terrier",age:5}, {name:"Abby",breed:"Labrador Retriever",age:9}, {name:"Oscar",breed:"Yorkshire Terrier",age:6}, {name:"Riley",breed:"Golden Retriever",age:6}, {name:"Bob",breed:"Miniature Schnauzer",age:8}, {name:"Shannon",breed:"Greyhound",age:12}, {name:"Brandy",breed:"Boxer",age:8}, {name:"Zoe",breed:"Labrador Retriever",age:3} ], {responseType: YAHOO.util.DataSource.TYPE_JSARRAY} ); // DataSource from a remote source: var myDataSource = new YAHOO.util.DataSource("http://myServer/folder/page?"); myDataSource.responseType = YAHOO.util.DataSource.TYPE_JSON;

Using the DataSource Utility

Once created and configured, the DataSource can be used to retrieve data directly or it can be passed to another component (i.e., the DataTable Control or Charts Control) that will use it to fetch the data it needs.

• Properties
• Fetching data
• Parsers
• Polling
• Caching
• Locating Data in JSON/XML responses
• Interesting Moments: Events in the DataSource Utility
• Fetching Data for the DataTable

Properties

connMethodPost

This boolean property is only meaningful when the data comes from a remote source. By default, it is set to false and the HTTP request will be sent as a GET request. If set to true, the request will be sent as a POST.

responseType

responseType tells the DataSource the type of data that is to be read. It is particularly important for remote connections or when the data is provided by a function, as such sources could return data in any of the supported formats. It should be set to one of the following constants (all of which are static members of YAHOO.util.DataSource):

• TYPE_HTMLTABLE
• TYPE_JSARRAY
• TYPE_JSON
• TYPE_TEXT
• TYPE_XML

These constants, plus others, are internally assigned by DataSource to the dataType property, according to the first argument of the constructor.

responseSchema

The responseSchema property is an object literal containing an assortment of properties describing the incoming data.

resultsList

A dot-separated String of identifiers used to reach the data (valid for TYPE_JSON)

resultNode

The name of the XML nodes containing record data to be read (valid for TYPE_XML)

recordDelim, fieldDelim

Characters or strings used to delimit records and fields (valid for TYPE_TEXT

fields

The fields property is required for all responseTypes. An array of field names (e.g., "Key") or object literals of the format {key:"Key", parser:parseFn} to indicate the name of each field and the parser to be used (if applicable). The parser property is a reference to either a built-in parse function or custom function that receives a single argument (which is the raw field value to be parsed) and returns a parsed value, which is especially useful for converting Strings into Numbers and Dates, etc. When dealing with name/value pairs (i.e., RESPONSE_TYPE = TYPE_JSARRAY), each field key value should map to an identifier in the data, but when dealing with unnamed data (i.e., RESPONSE_TYPE = TYPE_JSARRAY), each field key value will be mapped in the given order to the array data.

metaFields

Object literal containing any number of key:value pairs where the value corresponds to a location in the parsed JSON/XML data, and the key is the name used in the oParsedResponse.meta collection. Use this to identify any additional information that would be beneficial to your processing code. Some widgets may look in this collection for "magic" meta used to propagate state or UI changes automatically. An example of this is DataTable's use of totalRecords, sortKey, sortDir, and some pagination related keys.

// Instantiation, as before var myDataSource = new YAHOO.util.DataSource("http://myServer/folder/page?"); myDataSource.responseType = YAHOO.util.DataSource.TYPE_JSON; myDataSource.responseSchema = { resultsList: "data.dogs", // String representation of where to find the Array of results fields: ["name", "breed", {key: "age", parser: YAHOO.util.DataSource.parseNumber}], metaFields: { totalRecords: 'data.total', location: 'data["convention location"]' } };

Fetching Data

You can retrieve data by calling the sendRequest method. It requires two arguments:

oRequest

If the first argument to the constructor was a string, it is assumed to be a URL and the request can either be a URL-encoded list of param/value pairs to be appended to the base URL (for a GET request) or anything valid to be sent in the document body section of the request for a POST request if connMethodPost = true.

If the first argument was a function reference, then this can be anything that can be passed as a single argument to that function.

Note: this is where the initialRequest value of the DataTable component will go.

oCallback

An object literal with the following properties:

success

The function to call when the data is ready

failure

The function to call upon a response failure condition

scope

The object to serve as the scope for the success and failure handlers

argument

Arbitrary data that will be passed back to the success and failure handlers

When the response returns, the handler function will get called with the following arguments:

oRequest

The same value that was passed in as the first argument to sendRequest

oParsedResponse

An object literal containing the following properties:

tId

Unique transaction ID number

results

Array of schema-parsed data results

error

True if there was an error

meta

Object literal of values extracted from oFullResponse per the locations described in responseSchema.metaFields

oPayload

The same value as was passed in to the argument of the oCallback object literal

var oCallback = { success: function(oRequest,oResponse,oPayload) { alert("Hi " + oPayload + "! There are " + oResponse.results.length + " Greyhounds"); }, failure: function(oRequest,oResponse,oPayload) { alert("Hi " + oPayload + "! There were no Greyhounds found."); }, scope: this, argument: this.login_name } myDataSource.sendRequest("breed=Greyhound", oCallback);

Parsers

Parse functions can be designated for individual fields in the fields array of the responseSchema. The DataSource will call the parser for each record and will pass it the raw value of that field and expects the parsed value in return. You can use the built-in parsers, parseDate which will handle any date that the native JavaScript Date.parse() method handles (notice not all date formats are supported) or parseNumber which will try to parse a number whether an integer or float.

Polling

The setInterval method can be used to repeatedly poll for data. The first argument is the number of milliseconds to wait between requests, and the following three arguments are the same as those of sendRequest. This method returns a numeric identifier that can be used to cancel the polling.

The polling can be cancelled by calling clearInterval and passing it the identifier returned by the call to setInterval. All polling can be cancelled by calling clearAllIntervals.

Caching

The DataSource component can cache data. You can set the maxCacheEntries property to the number of responses you want cached. When this number is exceeded, the oldest cached response will be dropped. The data in the cache is already fully processed, extracted, parsed and arranged ready to be returned to the caller. On a cache hit, those events or overridable methods meant to signal or process the retrieval of live data will not be fired or called.

myDataSource.maxCacheEntries = 4;

The cache will be updated when polling but won't be used for retrieval.

Cache entries are indexed by the oRequest (first) argument of sendRequest. There will be one entry per request up to the total of maxCachedEntries.

Locating data in JSON/XML responses

For JSON and XML response types, the important data is likely to be nested somewhere in the parsed response's data structure. To extract the values from this structure, the responseSchema uses location keys for the following properties:

• resultNode (for XML)
• metaNode (for XML)
• resultsList (for JSON)
• fields[n] or fields[n].key
• metaFields[*]

Location keys are strings that describe where to look for a value in the parsed data structure.

Location keys for XML response types should be valid XML node names or attribute names.

Location keys for JSON response types should be valid JavaScript identifiers, including dot notation or square bracket notation for array indices or quoted strings ("this.is[4]['valid identifier']", but "this-is.not valid"). For convenience, the leading dot can be omitted for dot notation location keys (".foo" is equivalent to "foo" is equivalent to "['foo']"). Use of dot notation is encouraged.

For this example data...

{ "Response" : { "Results" : [ { "id":0, "obj":{ "nested":"foo" }, "arr":[null,{"nested 2": 0}]}, { "id":1, "obj":{ "nested":"bar" }, "arr":[null,{"nested 2": 3}]}, { "id":2, "obj":{ "nested":"baz" }, "arr":[null,{"nested 2": null}]}, ... ], "Total" : 1358, "Important" : { "to": { "me": "pot o' gold" } } } }

...you might use a responseSchema like this:

myDataSource.responseType = YAHOO.util.DataSource.TYPE_JSON; myDataSource.responseSchema = { resultsList : 'Response.Results', // String pointer to result data fields : [ { key: "id" }, // simple location { key: "obj.nested" }, // dot notation works { key: "['arr'][1]['nested 2']" } // bracket notation works ], metaFields : { // oParsedResponse.meta.totalRecords === 1358 totalRecords : 'Response.Total', // oParsedResponse.meta.something === "pot o' gold" something : 'Important.to.me' } };

Interesting Moments: Custom Events in the DataSource Utility

Several events can be used to monitor the progress of the data request or its response. There are also overridable methods and plug-in parsers that are meant to transform the data before it is returned to the requesting application.

responseEvent

Fires when response is returned from the live data.

doBeforeParseData

This overridable abstract method receives two arguments, the same oRequest value as was passed in the first argument to sendRequest and the type-converted data before any schema-parsing, oFullResponse. You may want to use this method to parse out metadata from the response or munge the data if it is formatted in a way that the DataSource would be unable to parse it with the schema. Implementers should be sure to return type-converted data that is ready for schema parsing.

doBeforeCallback

This abstract method is called after the data is successfully schema parsed but before the data is added to the cache or the callback is called. Its first two arguments are the same as for doBeforeParseData. The third is the already parsed response. Whatever this method returns is what the callback function will get.

When overriding these two doBefore methods, care should be taken not to tamper with the data in any manner that leaves it unusable in the next step.

Fetching Data for the DataTable

The DataTable provides three functions that can be used as callbacks on a call to sendRequest:

onDataReturnInitializeTable

Draws a DataTable with the data provided. On an existing DataTable it will delete the current contents and states before drawing the new. It is the method that would be used with polling to refresh the DataTable with new data.

onDataReturnInsertRows

This method will insert new rows at the beginning of the DataTable, preserving the existing data and regardless of any sort order. The insertion index for the added records can be assigned by identifying the metaField "recordInsertIndex" in your responseSchema

onDataReturnAppendRows

Rows will be added to the end of the DataTable, preserving the existing records, regardless of sort order.

onDataReturnSetRows

This method will assign record indices of the DataTable to those in the response. Existing records at these indices will be replaced. The starting index for the record assignment can be assigned by identifying the metaField "recordStartIndex" in your responseSchema

Support & Community

The YUI Library and related topics are discussed on the on the ydn-javascript mailing list.

In addition, please visit the YUIBlog for updates and articles about the YUI Library written by the library's developers.

Filing Bugs & Feature Requests

The YUI Library's public bug tracking and feature request repositories are located on the YUI SourceForge project site. Before filing new feature requests or bug reports, please review our reporting guidelines.

• Guidelines for YUI feature requests and bug reports.
• Review current bug list or file a new bug.
• Review current feature requests or file a new feature request.