YUI 3: DataSchema [beta]

The DataSchema Utility applies a given schema against data of arbitrary formats, normalizing input such as JSON, XML, or delimited text into a JavaScript object with known properties. The value of the DataSchema Utility is in its ability to translate data from a variety of sources into a consistent format for consumption by components in a predictable manner.

Getting Started

Include Dependencies

The easiest way to include the source files for DataSchema and its dependencies is to add the YUI seed file to your page, using the following script tag, and allow the YUI instance to download any additional files which may be required:

  1. <script src="http://yui.yahooapis.com/3.0.0/build/yui/yui-min.js"></script>
<script src="http://yui.yahooapis.com/3.0.0/build/yui/yui-min.js"></script>

The YUI instance will automatically pull down DataSchema's source files and any missing dependencies when the dataschema module is used. This helps you avoid having to manually manage the list of files needed on your page to support multiple components while also optimizing your initial page weight by loading files only when they are required.

If you do want to include file dependencies manually on your page, the YUI Dependency Configurator can be used to determine the list of files you need to include in order to use DataSchema.

The YUI Instance

Once you have the YUI seed file on your page (yui-min.js), you can create a new YUI instance for your application to use and populate it with the modules you need, specified as the first set of arguments to the use method:

  1. // Create new YUI instance, and populate it with the required modules
  2. YUI().use('dataschema', function(Y) {
  3.  
  4. // DataSchema available, and ready for use.
  5.  
  6. });
// Create new YUI instance, and populate it with the required modules
YUI().use('dataschema', function(Y) {
 
    // DataSchema available, and ready for use.
 
});

The last argument passed to use is a callback function. The callback function will be invoked as soon as the YUI instance is done downloading any required files missing from your page. Once those files are loaded, your local YUI instance will be supplemented with the classes which make up the dataschema module and any modules it depends on. A reference to the populated YUI instance (Y) is passed back to your callback function. Within your callback, then, you can start writing your application code based on your own custom instance of YUI.

For more information on creating instances of YUI and the use method, see the YUI Global object documentation.

Using the DataSchema Utility

This section describes how to use the DataSchema Utility in further detail. It contains these subsections:

DataSchema basics

DataSchema classes are standalone static utilities that accept data input and a schema definition, and they output schema-parsed JavaScript objects with the following signature:

Property Description
results An array of tabular or multi-dimensional data.
meta Arbitrary data values filtered from the input data.

Note that the schema you define will depend on which subclass of DataSchema is being used.

DataSchema.Array

Use DataSchema.Array when working with JavaScript arrays. These arrays may contain JavaScript objects, other arrays, or primitive values.

  1. // A sample array of objects
  2. [
  3. {make:"Chevrolet",model:"Bel Air",year:1957},
  4. {make:"Dodge",model:"Dart",year:1964},
  5. {make:"Ford",model:"Mustang",year:1968}
  6. ];
  7.  
  8. // A sample array of arrays
  9. [
  10. ["Chevrolet", "Bel Air", 1957],
  11. ["Dodge", "Dart", 1964],
  12. ["Ford", "Mustang", 1968]
  13. ];
  14.  
  15. // A sample array of primitives
  16. [
  17. "1957 Chevrolet Bel Air", "1964 Dodge Dart", "1968 Ford Mustang"
  18. ];
  19.  
// A sample array of objects
[
    {make:"Chevrolet",model:"Bel Air",year:1957},
    {make:"Dodge",model:"Dart",year:1964},
    {make:"Ford",model:"Mustang",year:1968}
];
 
// A sample array of arrays
[
    ["Chevrolet", "Bel Air", 1957],
    ["Dodge", "Dart", 1964],
    ["Ford", "Mustang", 1968]
];
 
// A sample array of primitives
[
    "1957 Chevrolet Bel Air", "1964 Dodge Dart", "1968 Ford Mustang"
];
 

Define a schema with the following properties for your array data:

Property Type Description
resultsField Array Keys to assign to the values contained in the array.
  1. var mySchema = {
  2. resultFields: [{key:"make"}, {key:"model"}, {key:"year"}]
  3. };
  4.  
  5. // Returns an object with the properties "results" and "meta"
  6. var myOutput = Y.DataSchema.Array.apply(mySchema, myData));
  7.  
var mySchema = {
        resultFields: [{key:"make"}, {key:"model"}, {key:"year"}]
};
 
// Returns an object with the properties "results" and "meta"
var myOutput = Y.DataSchema.Array.apply(mySchema, myData));
 
DataSchema.JSON

Use DataSchema.JSON when working with JavaScript objects or JSON data. Typically, your data will hold meta values as well as an internal array of tabular data.

  1. // Sample JSON data
  2. {
  3. "profile":{
  4. "current":160,
  5. "target":150
  6. },
  7. "program": [
  8. {
  9. "category":"exercise",
  10. "weekly schedule":[
  11. {"day":"sunday", "activity":"swimming"},
  12. {"day":"monday", "activity":"running"},
  13. {"day":"tuesday", "activity":"biking"},
  14. {"day":"wednesday", "activity":"running"},
  15. {"day":"thursday", "activity":"swimming"},
  16. {"day":"friday", "activity":"running"},
  17. {"day":"saturday", "activity":"golf"}
  18. ]
  19. }
  20. ]
  21. };
  22.  
// Sample JSON data
{
    "profile":{
        "current":160,
        "target":150
    },
    "program": [
        {
            "category":"exercise",
            "weekly schedule":[
                {"day":"sunday", "activity":"swimming"},
                {"day":"monday", "activity":"running"},
                {"day":"tuesday", "activity":"biking"},
                {"day":"wednesday", "activity":"running"},
                {"day":"thursday", "activity":"swimming"},
                {"day":"friday", "activity":"running"},
                {"day":"saturday", "activity":"golf"}
            ]
        }
    ]
};
 

Locators are string values in your schema that use dot notation or bracket syntax to point to data values within the object. Define a schema with the following properties for your object data:

Property Type Description
metaFields Object Key/locator pairs that point to arbitrary data values.
resultListLocator String Locator to an internal array of tabular data.
resultFields Array Keys to assign to the values contained in the array.
  1. var mySchema = {
  2. metaFields: {current:"profile.current", target:"profile.target"},
  3. resultListLocator: "program[0]['weekly schedule']",
  4. resultFields: [{key:"day"}, {key:"activity"}]
  5. };
  6.  
  7. // Returns an object with the properties "results" and "meta"
  8. var myOutput = Y.DataSchema.JSON.apply(mySchema, myData));
  9.  
var mySchema = {
    metaFields: {current:"profile.current", target:"profile.target"},
    resultListLocator: "program[0]['weekly schedule']",
    resultFields: [{key:"day"}, {key:"activity"}]
};
 
// Returns an object with the properties "results" and "meta"
var myOutput = Y.DataSchema.JSON.apply(mySchema, myData));
 
DataSchema.XML

Use DataSchema.XML when working with XML data. As with JSON data, your XML data may hold meta values as well as an internal node list of tabular data.

  1. // Sample XML data
  2. <root>
  3. <session>34637542</session>
  4. <category name="music" id="5">
  5. <results>
  6. <song id="59672468">
  7. <title>I Kissed A Girl</title>
  8. <rank>1</rank>
  9. <artist id="30326214">Katy Perry</artist>
  10. </song>
  11. <song id="47973564">
  12. <title>Shake It</title>
  13. <rank>2</rank>
  14. <artist id="45575683">Metro Station</artist>
  15. </song>
  16. <song id="52207363">
  17. <title>Bleeding Love</title>
  18. <rank>3</rank>
  19. <artist id="37956508">Leona Lewis</artist>
  20. </song>
  21. </results>
  22. </category>
  23. </root>
  24.  
// Sample XML data
<root>
    <session>34637542</session>
    <category name="music" id="5">
        <results>
            <song id="59672468">
                <title>I Kissed A Girl</title>
                <rank>1</rank>
                <artist id="30326214">Katy Perry</artist>
            </song>
            <song id="47973564">
                <title>Shake It</title>
                <rank>2</rank>
                <artist id="45575683">Metro Station</artist>
            </song>
            <song id="52207363">
                <title>Bleeding Love</title>
                <rank>3</rank>
                <artist id="37956508">Leona Lewis</artist>
            </song>
        </results>
    </category>
</root>
 

Locators are XPath string values in your schema that point to data values within the XML. Define a schema with the following properties for your XML data:

Property Type Description
metaFields Object Key/locator pairs that point to arbitrary data values.
resultListLocator String Locator to an internal node list of tabular data.
resultFields Array Keys to assign to the values contained in the array. Locators may be defined to point to complex nested values or values held in attributes.
  1. var mySchema = {
  2. metaFields: {session:"//Session", total:"//Tracks/@total"},
  3. resultListLocator: "Track",
  4. resultFields: [{key:"song", locator:"@title"}, {key:"artist", locator:"Artist"}, {key:"rank", locator:"ItemInfo/ChartPosition/@this"}]
  5. };
  6.  
  7. // Returns an object with the properties "results" and "meta"
  8. var myOutput = Y.DataSchema.XML.apply(mySchema, myData));
  9.  
var mySchema = {
    metaFields: {session:"//Session", total:"//Tracks/@total"},
    resultListLocator: "Track",
    resultFields: [{key:"song", locator:"@title"}, {key:"artist", locator:"Artist"}, {key:"rank", locator:"ItemInfo/ChartPosition/@this"}]
};
 
// Returns an object with the properties "results" and "meta"
var myOutput = Y.DataSchema.XML.apply(mySchema, myData));
 
DataSchema.Text

Use DataSchema.Text when working with delimited textual data. Typically, your data will not contain meta values.

  1. // Sample text data
  2. notebooks, 100, spiral-bound
  3. pencils, 300, #2 erasers
  4. pens, 500, blue ink
  5.  
// Sample text data
notebooks, 100, spiral-bound
pencils, 300, #2 erasers
pens, 500, blue ink
 

Define a schema with the following properties for your text data:

Property Type Description
resultDelimiter String Delimiter separating each row of tabular data
fieldDelimiter String Delimiter separating each column of tabular data
resultFields Array Keys to assign to the values contained in each field (column).
  1. var mySchema = {
  2. resultDelimiter: "\n",
  3. fieldDelimiter: ",",
  4. resultFields: [{key:"product"}, {key:"quantity"}, {key:"detail"}];
  5.  
  6. // Returns an object with the properties "results" and "meta"
  7. var myOutput = Y.DataSchema.Text.apply(mySchema, myData));
  8.  
var mySchema = {
            resultDelimiter: "\n",
            fieldDelimiter: ",",
            resultFields: [{key:"product"}, {key:"quantity"}, {key:"detail"}];
 
// Returns an object with the properties "results" and "meta"
var myOutput = Y.DataSchema.Text.apply(mySchema, myData));
 

DataSchema as a DataSource plugin

DataSchema plugins integrate DataSource's retrieval functionality with schema-based normalization of the retrieved data for further consumption by another component. There are currently four available DataSource plugins: DataSourceArraySchema, DataSourceJSONSchema, DataSourceXMLSchema, and DataSourceTextSchema.

  1. myDataSource.plug({fn: Y.Plugin.DataSourceJSONSchema, cfg: {
  2. schema: {
  3. resultListLocator: "ResultSet.Result",
  4. resultFields: ["Title"]
  5. }
  6. }});
  7.  
  8. // myCallback functions will receive the schema-normalized response object
  9. myDataSource.sendRequest({
  10. request: myRequest,
  11. callback: myCallback
  12. });
  13.  
myDataSource.plug({fn: Y.Plugin.DataSourceJSONSchema, cfg: {
    schema: {
        resultListLocator: "ResultSet.Result",
        resultFields: ["Title"]
    }
}});
 
// myCallback functions will receive the schema-normalized response object
myDataSource.sendRequest({
    request: myRequest,
    callback: myCallback
});
 

Support & Community

Forums & Blog

YUI 3 discussion forums are hosted on YUILibrary.com.

In addition, please visit the YUIBlog for updates and articles about the YUI Library written by the library's developers.

Filing Bugs & Feature Requests

The YUI Library's public bug tracking and feature request repositories are located on the YUILibrary.com site. Before filing new feature requests or bug reports, please review our reporting guidelines.

Copyright © 2010 Yahoo! Inc. All rights reserved. Copyright | Privacy Policy | Terms of Use | Job Openings