Developer Network Home - Help

YUI 2: DataSource

YUI 2: DataSource

The DataSource Utility provides a common configurable interface for other components to fetch tabular data from a variety of local or remote sources, from simple JavaScript arrays to online database servers. It is a required dependency of the DataTable, Charts, and AutoComplete controls.

The DataSource will request data and then return the response to a callback function. It has the capability of going deep into the hierarchy of the source data, selecting specified fields from the raw output, parsing data as indicated and calling the provided callback function when finished.

The DataSource has an optional local cache to store retrieved and parsed data. It can also be set to periodically poll for data.

Note: As of the 2.6.0 release, DataSource has been refactored into a base class and subclasses. While backward compatibility has been maintained whenever possible, implementers who are upgrading from an earlier version are strongly advised to read the Upgrade Notes to smooth the transition.

Upgrade Notes

Users new to DataSource can skip this section and proceed directly to the Getting Started section. Implementers who are upgrading from previous versions of DataSource should note the following changes as of version 2.6.0:

  • The DataSource class has been refactored into a DataSourceBase base class and the subclasses LocalDataSource, FunctionDataSource, XHRDataSource, and ScriptNodeDataSource. While backward compatibility of the YAHOO.util.DataSource constructor has been maintained, implementers should be aware that calling new YAHOO.util.DataSource() now actually returns one of these subclasses. Implementers can alternatively call a subclass constructor directly. The DataSource constructor returns one of the subclasses based on the oLiveData passed to it, or the dataType config value. This class-based architecture no longer meaningfully supports swapping data types on the fly.
  • Parsing of totalRecords is no longer supported as a top-leval schema value. Implementers should access this data as a metaFields value.

Getting Started

The DataSource code must be included before any control can use it:

  1. <!-- Dependencies -->
  2. <script src="https://yui-s.yahooapis.com/2.9.0/build/yahoo/yahoo-min.js"></script>
  3. <script src="https://yui-s.yahooapis.com/2.9.0/build/event/event-min.js"></script>
  4.  
  5. <!-- OPTIONAL: JSON Utility -->
  6. <script src="https://yui-s.yahooapis.com/2.9.0/build/json/json-min.js"></script>
  7.  
  8. <!-- OPTIONAL: Connection (enables XHR) -->
  9. <script src="https://yui-s.yahooapis.com/2.9.0/build/connection/connection-min.js"></script>
  10.  
  11. <!-- Source file -->
  12. <!-- OPTIONAL: Get (enables dynamic script nodes) -->
  13. <script src="https://yui-s.yahooapis.com/2.9.0/build/get/get-min.js"></script>
  14.  
  15. <script src="https://yui-s.yahooapis.com/2.9.0/build/datasource/datasource-min.js"></script>
  16.  
<!-- Dependencies -->
<script src="https://yui-s.yahooapis.com/2.9.0/build/yahoo/yahoo-min.js"></script>
<script src="https://yui-s.yahooapis.com/2.9.0/build/event/event-min.js"></script>
 
<!-- OPTIONAL: JSON Utility -->
<script src="https://yui-s.yahooapis.com/2.9.0/build/json/json-min.js"></script>
 
<!-- OPTIONAL: Connection (enables XHR) -->
<script src="https://yui-s.yahooapis.com/2.9.0/build/connection/connection-min.js"></script>
 
<!-- Source file -->
<!-- OPTIONAL: Get (enables dynamic script nodes) -->
<script src="https://yui-s.yahooapis.com/2.9.0/build/get/get-min.js"></script>
 
<script src="https://yui-s.yahooapis.com/2.9.0/build/datasource/datasource-min.js"></script>
 

YUI dependency configurator.

YUI Dependency Configurator:

Instead of copying and pasting the filepaths above, try letting the YUI dependency Configurator determine the optimal file list for your desired components; the Configurator uses YUI Loader to write out the full HTML for including the precise files you need for your implementation.

Note: If you wish to include this component via the YUI Loader, its module name is datasource. (Click here for the full list of module names for YUI Loader.)

Where these files come from: The files included using the text above will be served from Yahoo! servers; see "Serving YUI Files from Yahoo!" for important information about this service. JavaScript files are minified, meaning that comments and white space have been removed to make them more efficient to download. To use the full, commented versions or the -debug versions of YUI JavaScript files, please download the library distribution and host the files on your own server.

Order matters: As is the case generally with JavaScript and CSS, order matters; these files should be included in the order specified above. If you include files in the wrong order, errors may result.

Overview

A DataSource is an abstract representation of a live set of data that presents a common predictable API for other objects to interact with. The nature of your data, its quantity, its complexity, and the logic for returning query results all play a role in determining your type of DataSource. For small amounts of simple textual data, a JavaScript array is a good choice. If your data has a small footprint but requires a simple computational or transformational filter before being displayed, a JavaScript function may be the right approach. For very large datasets — for example, a robust relational database — or to access a third-party webservice you'll certainly need to leverage the power of a Script Node or XHR DataSource. Whatever the source of your live data, the following diagram describes an overview of the DataSource API:

Visual representation of the data lifecycle in DataSource.  Read the text in the sections below for an accessible explanation of this diagram.

  1. A request for data is made to the DataSource by a widget instance. oRequest may define specific parameters, or it may simply be null. The sendRequest method requires two arguments:
    oRequest
    For remote data, this request may be a param/value string, such as "?id=123&name=foo". For local data, this request maybe a simpler value such as 123. Specifying parameters may be irrelevent, so this value may be simply be null
    oCallback
    An object literal with the following properties:
    oCallback properties
    PropertyTypeDescription
    successFunctionThe function to call when the response is ready.
    failureFunctionThe function to call upon a response failure condition.
    scopeWidget instanceThe object to serve as the scope for the success and failure handlers.
    argumentObjectArbitrary data to be passed back to the success and failure handlers.
  2. The DataSource checks its local cache for a response to the oRequest
  3. Not finding a cached response will cause the DataSource to make a connection to the live data. Depending on the dataType, the connection may make a remote connection via Connection Manager or the Get Utility, or simply access local data in thread.
  4. A response from the live data set is returned to the DataSource in its raw state.
  5. The doBeforeParseData() abstract method allows implementers to access and transform the full response before it is schema-parsed.
  6. The DataSource parses the raw response according to responseType and implementer-defined schema.
  7. The doBeforeCallback() abstract method allows implementers to access and transform the parsed response before it is cached.
  8. The parsed data is returned to the callback with the following arguments:
    oRequest
    The same value that was passed in as the first argument to sendRequest
    oParsedResponse
    An object literal containing the following properties:
    oParsedResponse properties
    PropertyTypeDescription
    tIdNumberUnique transaction ID number.
    resultsArraySchema-parsed data results.
    errorBooleanIndicates data error.
    cachedBooleanIndicates cached response.
    metaObjectSchema-parsed meta data.
    oPayload
    The same value as was passed in to the argument of the oCallback object literal

Instantiating a DataSource

To create a DataSource instance on your page, choose the appropriate DataSource class for the type of data you are using and pass in a reference to the data in the constructor. The simplest subclass available is LocalDataSource, which is meant to support data held in local memory (e.g., a JavaScript array, a JSON struct, a local XML document, a delimited string, or an HTML table element). When data is needs to be accessed from a server, you can instantiate either an XHRDataSource or a ScriptNodeDataSource. A FunctionDataSource allows you to define a function to return data in a highly customizeable manner. As a convenience, the YAHOO.util.DataSource constructor will return the appropriate subclass for you based on the type of live data you pass in. However, if you need a ScriptNodeDataSource, or a LocalDataSource pointing to delimited text, please note that you'll need to specify the dataType in the constructor explicitly.

  1. // DataSources for all kinds of local data
  2. var dsLocalArray = new YAHOO.util.LocalDataSource(["apples", "broccoli", "cherries"]);
  3. var dsLocalJSON = new YAHOO.util.LocalDataSource({
  4. found: 3,
  5. total: 20,
  6. results: [
  7. {name: "apples", type:"fruit", color: "red"},
  8. {name: "broccoli", type:"veg", color: "green"},
  9. {name: "cherries", type:"fruit", color: "red"}
  10. ]
  11. });
  12. var dsLocalTable = new YAHOO.util.LocalDataSource(YAHOO.util.Dom.get("myTable"));
  13.  
  14. // DataSource for a server accessed over XHR
  15. var dsXHR = new YAHOO.util.XHRDataSource("http://local_path_or_path_to_proxy");
  16.  
  17. // DataSource for a remote location with dynamic script nodes
  18. var dsScriptNode = new YAHOO.util.ScriptNodeDataSource("http://path_to_webservice");
  19.  
  20. // DataSource for a JavaScript function that returns data
  21. var dsFunction = new YAHOO.util.FunctionDataSource(function() {
  22. return ["apples", "broccoli", "cherries"];
  23. });
  24.  
// DataSources for all kinds of local data
var dsLocalArray = new YAHOO.util.LocalDataSource(["apples", "broccoli", "cherries"]);
var dsLocalJSON = new YAHOO.util.LocalDataSource({
    found: 3,
    total: 20,
    results: [
        {name: "apples", type:"fruit", color: "red"},
        {name: "broccoli", type:"veg", color: "green"},
        {name: "cherries", type:"fruit", color: "red"}
    ]
});
var dsLocalTable = new YAHOO.util.LocalDataSource(YAHOO.util.Dom.get("myTable"));
 
// DataSource for a server accessed over XHR
var dsXHR = new YAHOO.util.XHRDataSource("http://local_path_or_path_to_proxy");
 
// DataSource for a remote location with dynamic script nodes
var dsScriptNode = new YAHOO.util.ScriptNodeDataSource("http://path_to_webservice");
 
// DataSource for a JavaScript function that returns data
var dsFunction = new YAHOO.util.FunctionDataSource(function() {
    return ["apples", "broccoli", "cherries"];
});
 

Setting responseType and responseSchema

DataSource uses the responseType value to determine which parsing algorithm to use, and the responseSchema values determine what data gets parsed out for use by the calling widget. Most DataSources can sniff the responseType value directly from the response, but in some cases you may have to explicitly set the appropriate responseType from the following static constants:

  • TYPE_JSARRAY
  • TYPE_JSON
  • TYPE_XML
  • TYPE_TEXT
  • TYPE_HTMLTABLE
  1. var xmlDataSource = new YAHOO.util.XHRDataSource("http://myxmlwebservice");
  2. xmlDataSource.responseType = YAHOO.util.XHRDataSource.TYPE_XML;
  3.  
var xmlDataSource = new YAHOO.util.XHRDataSource("http://myxmlwebservice");
xmlDataSource.responseType = YAHOO.util.XHRDataSource.TYPE_XML;
 

Defining a schema for your data allows DataSource to cache and return only the data your widgets will consume, thus minimizing the browser footprint and/or latency of your application. A responseSchema is an object literal of pointers that depend on your data's responseType:

Simple JavaScript Array

The simplest type of DataSource points to an array of strings, such that each result is a string.

Defining a Schema for a Simple JS Array of Strings
PropertyTypeDescription
fieldsArrayArray of strings to name the data fields represented by the data. Because DataSource assumes a tabular (two-dimensional) data structure, a simple array of strings is considered to contain only one field of data.

Sample data

  1. // A simple array of strings is only considered to contain one field of data
  2. ["apples", "broccoli", "cherries"]
  3.  
// A simple array of strings is only considered to contain one field of data
["apples", "broccoli", "cherries"]
 

Sample schema

  1. myDataSource.responseSchema = {
  2. fields: ["name"]
  3. };
  4.  
myDataSource.responseSchema = {
    fields: ["name"]
};
 

Complex JavaScript Array

A more common type of DataSource points to a complex array of nested arrays or JSON objects that represents a tabular (two-dimensional) data structure. Fields are used to assign a name to each field of data.

Defining a Schema for a Complex JS Array
PropertyTypeDescription
fieldsArrayArray of string names for each field of data coming in. For nested arrays, the array index of a field maps it directly to the array index of the result data. For nested objects, the order of the fields is irrelevant since the name of the field maps to result data, and data values that don't have a defined field are ignored.

Sample data

  1. // Nested arrays - field order matters and must be one-to-one to the data
  2. [
  3. [ "apples", "fruit", "red" ],
  4. [ "broccoli", "vegetable", "green" ],
  5. [ "cherries","fruit", "red" ]
  6. ]
  7. myDataSource.responseType = YAHOO.util.XHRDataSource.TYPE_JSARRAY;
  8. myDataSource.responseSchema = {
  9. fields: [ "name", "type", "color" ]
  10. };
  11.  
  12. // Nested objects - field order doesn't matter and not all data is required to have a field
  13. [
  14. { type: "fruit", color: "red", name: "apples" },
  15. { name: "broccoli", color: "green", type: "vegetable" },
  16. { color: "red", name: "cherries", type: "fruit" }
  17. ]
  18.  
// Nested arrays - field order matters and must be one-to-one to the data
[
    [ "apples", "fruit", "red" ],
    [ "broccoli", "vegetable", "green" ],
    [ "cherries","fruit", "red" ]
]
myDataSource.responseType = YAHOO.util.XHRDataSource.TYPE_JSARRAY;
myDataSource.responseSchema = {
    fields: [ "name", "type", "color" ]
};
 
// Nested objects - field order doesn't matter and not all data is required to have a field
[
    { type: "fruit", color: "red", name: "apples" },
    { name: "broccoli", color: "green", type: "vegetable" },
    { color: "red", name: "cherries", type: "fruit" }
]
 

Sample schema

  1. myDataSource.responseSchema = {
  2. fields: [ "name", "color" ]
  3. };
  4.  
myDataSource.responseSchema = {
    fields: [ "name", "color" ]
};
 

JSON

When a DataSource returns JSON data, fields point to the location of the results using dot-notation. Optionally, metaFields may be defined to access additional meta data. Location pointers to JSON data should be valid JavaScript identifiers, including dot notation or square bracket notation for array indices or quoted strings ("this.is[4]['valid identifier']", but "this-is.not valid"). For convenience, the leading dot can be omitted for dot notation location keys (".foo" is equivalent to "foo" is equivalent to "['foo']"). Use of dot notation is encouraged.

Defining a Schema for JSON
PropertyTypeDescription
resultsListStringDot-notation location to results. Not all data is required to have a field.
fieldsArrayString locator for each field of data coming in.
metaFieldsObject(Optional) String locator of additional meta data.

Sample data

  1. {
  2. "Response" : {
  3. "Results" : [
  4. { "id":0, "obj": { "nested":"foo" }, "arr": [null, { "nested 2": 0 } ] },
  5. { "id":1, "obj": { "nested":"bar" }, "arr": [null, { "nested 2": 3 } ] },
  6. { "id":2, "obj": { "nested":"baz" }, "arr": [null, { "nested 2": null } ] },
  7. //...
  8. ],
  9. "Total" : 1358,
  10. "Important" : {
  11. "to": {
  12. "me": "pot o' gold"
  13. }
  14. }
  15. },
  16. "Session" : "12345678"
  17. }
  18.  
{
    "Response" : {
        "Results" : [
            { "id":0, "obj": { "nested":"foo" }, "arr": [null, { "nested 2": 0 } ] },
            { "id":1, "obj": { "nested":"bar" }, "arr": [null, { "nested 2": 3 } ] },
            { "id":2, "obj": { "nested":"baz" }, "arr": [null, { "nested 2": null } ] },
            //...
        ],
        "Total" : 1358,
        "Important" : {
            "to": {
                "me": "pot o' gold"
            }
        }
    },
    "Session" : "12345678"
}
 

Sample schema

  1. myDataSource.responseSchema = {
  2. resultsList : "Response.Results", // String pointer to result data
  3. // Field order doesn't matter and not all data is required to have a field
  4. fields : [
  5. { key: "id" }, // simple location
  6. { key: "obj.nested" }, // dot notation works
  7. { key: "['arr'][1]['nested 2']" } // bracket notation works
  8. ],
  9. metaFields : {
  10. // oParsedResponse.meta.totalRecords === 1358
  11. totalRecords : "Response.Total",
  12.  
  13. // oParsedResponse.meta.something === "pot o' gold"
  14. something : "Important.to.me"
  15. }
  16. };
  17.  
myDataSource.responseSchema = {
    resultsList : "Response.Results", // String pointer to result data
    // Field order doesn't matter and not all data is required to have a field
    fields : [
        { key: "id" },                    // simple location
        { key: "obj.nested" },            // dot notation works
        { key: "['arr'][1]['nested 2']" } // bracket notation works
    ],
    metaFields : {
        // oParsedResponse.meta.totalRecords === 1358
        totalRecords : "Response.Total",
 
         // oParsedResponse.meta.something === "pot o' gold"
        something : "Important.to.me"
    }
};
 

XML

A schema for XML data requires a pointer to each result node, in additional to defining fields of data held in each node. Optionally, a metaNode and metaFields may be defined to access additional meta data. Location pointers to XML data can be valid XML node names, attribute names, or preferably, XPath syntax locators. Set the useXPath property to true for XPath support in your locators.

Defining a Schema for a XML
PropertyTypeDescription
resultNodeStringName of the node assigned to each result.
fieldsArrayString locator for each field of data coming in. Not all data is required to have a field.
metaNodeString(Optional) String name of the node under which to search for meta data.
metaFieldsObject(Optional) String locator of additional meta data.

Sample data

  1. <myroot rootatt='5'>
  2. <top>topvalue</top>
  3. <second nested='nestedsecond' />
  4. <allitems>
  5. <livehere>
  6. <item type='foo'>
  7. <name type='nametype0'>Abc</name>
  8. <rank>0</rank>
  9. <subitem>
  10. <name type='subnametype0'>subABC</name>
  11. <age>10</age>
  12. </subitem>
  13. </item>
  14. <item type='bar'>
  15. <name type='nametype1'>Def</name>
  16. <rank>1</rank>
  17. <subitem>
  18. <name type='subnametype1'>subDEF</name>
  19. <age>11</age>
  20. </subitem>
  21. </item>
  22. </livehere>
  23. </allitems>
  24. </myroot>
  25.  
<myroot rootatt='5'>
    <top>topvalue</top>
    <second nested='nestedsecond' />
    <allitems>
        <livehere>
            <item type='foo'>
                <name type='nametype0'>Abc</name>
                <rank>0</rank>
                <subitem>
                    <name type='subnametype0'>subABC</name>
                    <age>10</age>
                </subitem>
            </item>
            <item type='bar'>
                <name type='nametype1'>Def</name>
                <rank>1</rank>
                <subitem>
                    <name type='subnametype1'>subDEF</name>
                    <age>11</age>
                </subitem>
            </item>
            </livehere>
        </allitems>
    </myroot>
 

Sample schema

  1. myDataSource.useXPath = true;
  2. myDataSource.responseSchema = {
  3. metaFields: {
  4. rootatt: "/myroot/@rootatt",
  5. topnode: "//top",
  6. nestedatt: "//second/@nested"
  7. },
  8. resultNode: "item",
  9. fields: [
  10. { key: "type", locator: "@type" },
  11. { key: "rank", parser: "number" },
  12. "name",
  13. { key: "subnameatt", locator: "subitem/name/@type"} ,
  14. { key: "age", locator: "subitem/age", parser: "number" }
  15. ]
  16. };
  17.  
myDataSource.useXPath = true;
myDataSource.responseSchema = {
    metaFields: {
        rootatt: "/myroot/@rootatt",
        topnode: "//top",
        nestedatt: "//second/@nested"
    },
    resultNode: "item",
    fields: [
        { key: "type", locator: "@type" },
        { key: "rank", parser: "number" },
        "name",
        { key: "subnameatt", locator: "subitem/name/@type"} ,
        { key: "age", locator: "subitem/age", parser: "number" }
    ]
};
 

Delimited Text

When using this relatively simple type of data structure, we must make a small set of assumptions:

  • There is a known and constant string delimiter of records.
  • Within each record, there is a known and constant string delimiter of data fields that is not equal to the record delimiter.
  • There are no escaped delimiter characters in the data
Defining a Schema for Text Data
PropertyTypeDescription
recordDelimStringCharacter(s) that delimits each record.
fieldDelimStringCharacter(s) that delimits each field within each record.

Sample data

  1. apples,fruit,red|broccoli,vegetable,green|cherriesfruit,red
  2.  
apples,fruit,red|broccoli,vegetable,green|cherriesfruit,red
 

Sample schema

  1. myDataSource.responseSchema = {
  2. fieldDelim : ",",
  3. recordDelim : "|"
  4. };
  5.  
myDataSource.responseSchema = {
    fieldDelim : ",",
    recordDelim : "|"
};
 

HTML Table

DataSource supports consuming data out of an HTML table element. By default, DataSource will iterate over all TR elements of all TBODY elements of the given TABLE to parse out data. If there are any non-data elements (like summary rows or the message TBODY in DataTable), they should first be removed from the DOM.

Defining a Schema for Text Data
PropertyTypeDescription
fieldsArrayString name for each field of data coming in. The array index of a field maps it to the cellIndex of the result data.

Sample data

  1. <-- HTML tables - field order matters -->
  2. <table>
  3. <thead>
  4. <tr>
  5. <th>Name</th>
  6. <th>Type</th>
  7. <th>Color</th>
  8. </tr>
  9. </thead>
  10. <tbody>
  11. <tr>
  12. <td>Apples</td>
  13. <td>Fruit</td>
  14. <td>Red</td>
  15. </tr>
  16. <tr>
  17. <td>Broccoli</td>
  18. <td>Vegetable</td>
  19. <td>Green</td>
  20. </tr>
  21. <tr>
  22. <td>Cherries</td>
  23. <td>Fruit</td>
  24. <td>Red</td>
  25. </tr>
  26. </tbody>
  27. </table>
  28.  
<-- HTML tables - field order matters -->
<table>
<thead>
    <tr>
        <th>Name</th>
        <th>Type</th>
        <th>Color</th>
    </tr>
</thead>
<tbody>
    <tr>
        <td>Apples</td>
        <td>Fruit</td>
        <td>Red</td>
    </tr>
    <tr>
        <td>Broccoli</td>
        <td>Vegetable</td>
        <td>Green</td>
    </tr>
    <tr>
        <td>Cherries</td>
        <td>Fruit</td>
        <td>Red</td>
    </tr>
</tbody>
</table>
 

Sample schema

  1. myDataSource.responseSchema = {
  2. fields: [ "name", "type", "color" ]
  3. };
  4.  
myDataSource.responseSchema = {
    fields: [ "name", "type", "color" ]
};        
 

Type Conversion

Data supplied by the DataSource may need to be type-converted before it can be consumed. This is especially true for XHRDataSources or progressive enhancement scenarios where data may be intended to be of type Number, but gets loaded as type String. Parser functions can be assigned to convert data on a per field basis.

Built-in parsers include parseDate which will handle any date that the native JavaScript Date.parse() method handles (notice not all date formats are supported) and parseNumber which will try to parse a number whether an integer or float. A custom function may also be defined by the implementer to override these built-in functions.

As of the 2.6.0 release, the following string shortcuts may be used to point to the built-in parser functions:

  • "date" points to YAHOO.util.DataSource.parseDate
  • "number" points to YAHOO.util.DataSource.parseNumber
  • "string" points to YAHOO.util.DataSource.parseString

Sample data

  1. {
  2. "Inventory" : {
  3. "Produce" : [
  4. { "name": "apples", type: "fruit", color: "red", quantity: "50" },
  5. { "name": "broccoli", type: "fruit", color: "green", quantity: "35" },
  6. { "name": "cherries", type: "fruit", color: "red", quantity: "100" },
  7. ],
  8. "Total" : 20
  9. }
  10. }
  11.  
{
    "Inventory" : {
        "Produce" : [
            { "name": "apples", type: "fruit", color: "red", quantity: "50" },
            { "name": "broccoli", type: "fruit", color: "green", quantity: "35" },
            { "name": "cherries", type: "fruit", color: "red", quantity: "100" },
        ],
        "Total" : 20
    }
}
 

Sample schema

  1. myDataSource.responseSchema = {
  2. resultsList : "Inventory.Produce",
  3. fields : [
  4. "name",
  5. "type",
  6. // This field will convert incoming data using a custom function
  7. {key: "color", parser: myColorToRGB },
  8. // This field will convert incoming data to type Number
  9. {key: "quantity", parser: "number" }
  10. ],
  11. metaFields : {
  12. // oParsedResponse.meta.totalRecords === 1358
  13. totalRecords : "Response.Total",
  14.  
  15. // oParsedResponse.meta.something === "pot o' gold"
  16. something : "Important.to.me"
  17. }
  18. };
  19.  
myDataSource.responseSchema = {
    resultsList : "Inventory.Produce",
    fields : [
        "name",
        "type",
        // This field will convert incoming data using a custom function 
        {key: "color", parser: myColorToRGB },
        // This field will convert incoming data to type Number
        {key: "quantity", parser: "number" }
    ],
    metaFields : {
        // oParsedResponse.meta.totalRecords === 1358
        totalRecords : "Response.Total",
 
         // oParsedResponse.meta.something === "pot o' gold"
        something : "Important.to.me"
    }
};
 

Caching

The DataSource component can cache data in a local JavaScript array, which is especially helpful for reducing network traffic to remote servers. You can set the maxCacheEntries property to the number of responses you want held in the cache. When this number is exceeded, the oldest cached response will be dropped. The data in the cache is already fully processed, extracted, parsed and ready to be returned to the caller. On a cache hit, those events or overridable methods meant to signal a connection to the live data or handling of raw responses will not be fired or called.

  1. myDataSource.maxCacheEntries = 4;
  2.  
myDataSource.maxCacheEntries = 4;
 

The cache will be updated when polling but won't be used for retrieval.

Cache entries are indexed by the oRequest (first) argument of sendRequest. There will be one entry per request up to the total of maxCachedEntries.

Polling

The setInterval method can be used to repeatedly poll for data. The first argument is the number of milliseconds to wait between requests, and the following three arguments are the same as those of sendRequest. This method returns a numeric identifier that can be used to cancel the polling.

Polling can be cancelled by calling clearInterval and passing it the identifier returned by the call to setInterval. All polling can be cancelled by calling clearAllIntervals. Please note that existing transactions will not be canceled.

Custom Events and "doBefore" Abstract Methods

DataSource offers several hooks can be used to monitor the progress of the data request/response cycle or perform transformations on the response along its journey back to the callback.

cacheRequestEvent
Fired when a request is made to the local cache.
cacheResponseEvent
Fired when data is retrieved from the local cache.
requestEvent
Fired when a request is sent to the live data source.
responseEvent
Fired when live data source sends response.
doBeforeParseData
This overridable abstract method gives implementers an opportunity to munge the data before it is schema-parsed. Implementers should be sure to return data in a ready-to-parse state to avoid errors.
responseParseEvent
Fired when response is parsed.
dataErrorEvent
Fired when a data error is encountered.
doBeforeCallback
This overridable abstract method gives implementers an opportunity to access the data before it has been cached or returned to the callback. Implementers should be sure to return data in a ready-to-return state to avoid errors.

Using XHRDataSource

The XHRDataSource class uses the YUI Connection Manager to retrieve data from a server. Implementers should be aware of browser security restrictions that prevent XHR connections across domaina. The following properties are available for configuration:

XHRDataSource properties
PropertyTypeDefaultDescription
connMethodPostBooleanfalseTrue if data is to be sent via POST. By default, data will be sent via GET.
connMgrClassYAHOO.util.ConnectAlias to YUI Connection Manager, to allow implementers to use a custom class.
connTimeoutNumber0How many milliseconds the XHR connection will wait for a server response. Any non-zero value will enable the Connection Manager's Auto-Abort feature.
connXhrModeString"allowAll"Defines request/response management in the following manner:
queueRequests
If a request is already in progress, wait until response is returned before sending the next request.
cancelStaleRequests
If a request is already in progress, cancel it before sending the next request.
ignoreStaleResponses
Send all requests, but handle only the response for the most recently sent request.
allowAll
Send all requests and handle all responses.

Special considerations need to be made when using a DataSource that supports asynchronous request/response cycles, including state validation in the sendRequest() callback's success and failure handlers. In the case of YUI widgets consuming DataSource, these handlers will want to verify that the widget instances themselves are still valid.

Using ScriptNodeDataSource

The ScriptNodeDataSource class uses the YUI Get Utility to retrieve data from a remote server without the need for a proxy. Note: ScriptNodeDataSource requires that the webservice facilitate a callback mechanism in its response. The following properties are available for configuration:

ScriptNodeDataSource properties
PropertyTypeDefaultDescription
asyncModeString"allowAll"Defines request/response management in the following manner:
ignoreStaleResponses
Send all requests, but handle only the response for the most recently sent request.
allowAll
Send all requests and handle all responses.
getUtilityClassAlias to YUI Get Utility, to allow implementers to use a custom class.
scriptCallbackParamString"callback"Callback string parameter name sent to the remote script. By default, requests are sent to <URI>?<scriptCallbackParam>=callbackFunction

Special considerations need to be made when using a DataSource that supports asynchronous request/response cycles, including state validation in the sendRequest() callback's success and failure handlers. In the case of YUI widgets consuming DataSource, these handlers will want to verify that the widget instances themselves are still valid.

YUI on Mobile: Using DataSource Utility with "A-Grade" Mobile Browsers

About this Section: YUI generally works well with mobile browsers that are based on A-Grade browser foundations. For example, Nokia's N-series phones, including the N95, use a browser based on Webkit — the same foundation shared by Apple's Safari browser, which is found on the iPhone. The fundamental challenges in developing for this emerging class of full, A-Grade-derived browsers on handheld devices are:

  • Screen size: You have a much smaller canvas;
  • Input devices: Mobile devices generally do not have mouse input, and therefore are missing some or all mouse events (like mouseover);
  • Processor power: Mobile devices have slower processors that can more easily be saturated by JavaScript and DOM interactions — and processor usage affects things like battery life in ways that don't have analogues in desktop browsers;
  • Latency: Most mobile devices have a much higher latency on the network than do terrestrially networked PCs; this can make pages with many script, css or other types of external files load much more slowly.

There are other considerations, many of them device/browser specific (for example, current versions of the iPhone's Safari browser do not support Flash). The goal of these sections on YUI User's Guides is to provide you some preliminary insights about how specific components perform on this emerging class of mobile devices. Although we have not done exhaustive testing, and although these browsers are revving quickly and present a moving target, our goal is to provide some early, provisional advice to help you get started as you contemplate how your YUI-based application will render in the mobile world.

More Information:

The core functionality of the DataSource utility operates without any major issues on high-end mobile platforms, keeping in mind the following preliminary list of smart phone known issues:

  • XML is not currently supported on Android devices.

Support & Community

The YUI Library and related topics are discussed on the on the YUILibrary.com forums.

Also be sure to check out YUIBlog for updates and articles about the YUI Library written by the library's developers.

Filing Bugs & Feature Requests

The YUI Library's public bug tracking and feature request repositories are located on the YUILibrary.com site. Before filing new feature requests or bug reports, please review our reporting guidelines.

DataSource Utility Cheat Sheet:

Cheat Sheet for the YUI DataSource Utility

Download full set of cheat sheets.

DataSource Utility Examples:

Other YUI Examples That Make Use of the DataSource Utility:

YUI DataSource on del.icio.us:

bookmark on del.icio.us

be the first to bookmark this page!

Copyright © 2016 Yahoo! Inc. All rights reserved.

Privacy Policy - Copyright Policy - Job Openings