Video: Eric Ferraiuolo — Web App Development with YUI 3

Video published 2009-10-29.

• Download iPhone/iPod-compatible video file (320x240, .m4v)
• Slides

Transcript:

Eric Ferraiuolo: Today I'm going to be talking about web application development with YUI 3. I'm Eric Ferraiuolo, and I do Frontend engineering and development; that's the stuff I like to do on the web. What I want to do is to show everyone how we've taken YUI 3 and set it as our user interface framework that we use for developing web applications, and how we fit everything in with the larger scope of things, in our project and alongside server side code as well. Almost a year ago now, I started working with YUI 3 around Preview Release 2, and that's pretty much when I'd adopted it as my framework: this is what I'm going to use. So I wanted to share our experiences with it.

I also wanted to talk about the development and deployment goals. This is the first section that I'll look at. This is what I ended up finding that I do when developing rich user interfaces, and for applications that are more complex. This is more for web applications and not really web sites; stuff that has high interactivity with the server, that's where these apply more. I wanted to look at how we can use YUI 3 as our framework for developing user interfaces, and leveraging it. All the new features in YUI 3 really help this. And I wanted to look at Logismo. Logismo is a web application that we've developed internally, and it uses YUI 3 for all of its UI. This is sort of a case study along the way. Lastly, I wanted to look at: how can we integrate YUI 3 with our IDE to make development easier when we're developing these larger web applications? I wanted to see how we can integrate it with our projects as a whole.

We'll start off with the development and deployment goals. I first want to take a minute to look at: what are we doing with web application UIs at a higher level? This is sort of like the implementation phase, and this is how I've begun to see what we're doing with user interfaces. We're taking things and organizing them. We have a set of features that we need to implement, and we really need to organize and break things down into components; that really helps make our life easier. Then what we need to do is link these components together, wire or glue them together, and put them on the page to have them interact. Lastly is seems that we're constructing the user interface, and this is something where we set up the world of the current page and do at runtime. So at a high level, I see this is the pattern I end up following: breaking things down, linking them back together and constructing them. Also you could think of, in the construction phase, wanting to minimize HTTP requests and that sort of thing.

The goals for UI development and deployment; these are the things I wanted to make my life easier. They're not really for trivial apps, so for them this may be slight overkill. But these are things that I wanted to have for better code organization, and there are benefits at each stage along the way. One thing that I wanted was this idea of a component framework — something really comprehensive — and this is something to help me build widgets. I need the tools for this type of thing. I didn't want to worry about what component depends on what, or any of the dependencies on the underlying framework that we were using — in this case, YUI — I wanted managed dependencies. This is more of an IDE thing, but I wanted components to be built automatically. I didn't want to have to tear down the server, start a backup, or have to build modules manually for YUI custom modules, I wanted this to happen more automatically. Lastly, and this is more for the construction phase, I didn't want to have to assemble everything. I wanted the UI to be told by the server what its running environment is, either running, or production, or while I'm debugging something. I wanted this to happen automatically.

When we were first looking at YUI, I thought there were a lot of good parts there: it could be used in a real world setting, and it has a component infrastructure that's very solid. It's used on the YUI homepage right now, so even though it's still in beta, it's still very powerful. Y.Attribute, Y.Base, Y.Plugin and Y.Widget are the defined parts of the component infrastructure in YUI 3. Module system is ingrained in this library. Its core, the YUI library, uses it itself, and it's also very extensible for supporting custom modules. There's a build system. Other projects you may or may not be familiar with, but the YUI Builder is the tool that the YUI developers use to build all of their components: take their source, create the minified version, the debug version, and the raw versions. This is done using the YUI Compressor, also. Then runtime configuration — this is something that I really liked about YUI 3. When you're setting up your YUI instance, you can configure it. This is something I really tried to extend to fit YUI in with the rest of our running application.

I wanted to take a minute to look at this idea of organizing your code and breaking down your problems and features into functional components. YUI comes in very big here; it really helps you, using the component infrastructure, to break things down very nicely. With functional components, you're taking a specific feature and trying to encapsulate its functionality, provide an API to use it, and possibly events as well. Basically, think about widgets. An example could be: I would consider this area here a functional component on the YUI homepage. The search was search assist, and it's something very functional, and it's something self-contained and encapsulated.

Another area that you may be less familiar with — but something that I've found myself starting to do — is taking the strategy of having a top-level component in my web applications, a top-level component per page, or screen, possibly. This is at the very top of the component hierarchy, and its goal is to manage the functional components, to instantiate them, to deal with the communication between them. I've found that YUI provides an easy way to do this with standard coding. I happen to use base for this, usually, and this way I have my managed attributes and all the different stuff that comes along with that, and the event target for my functional components to bubble their events up to this top level components. That then makes decisions about what things should do. What I've really found this doing is taking a big chunk of messy glue code in my YUI use state and really making it nice and tidy, where I'm really just wiring up components.

I think that there are some good advantages with event wiring, and this is where YUI 3 has custom events now that bubble up so your components can describe interesting moments within them. Your top-level component can then listen for those things, and wire them up, and maybe tell a different functional component that it may be interested in something. What that really helps to do is add to how well your application is documented. Someone else may come and start working on the application; they can go to this top-level component on a specific page and see what's going on without all this glue code they have to look through. If you use something like Y.Base, there's this standard code structure that people will then be able to follow if they understand YUI and they see how base works with having managed attributes and events, and it should be easier for other developers to follow as well. This is another point about having centralized control.

One example could be if you had a bunch of functional components on the page that all needed data from the server, and maybe they needed to poll the server periodically, it could be potentially harmful to have five components doing XHR polling on the page. Whereas if they said to the top-level component via an event: hey, I want some data right now, the top-level component could possibly queue up these XHR requests. This also reduces what other dependencies, or what other others tools, these functional components may require. For example, going back to using Y.IO, if my functional components may not require Y.IO and JSON to parse the data coming back, that's all handled centrally on the top level.

Sure, breaking things into components is fine, but really what I wanted to do was create custom modules with them, so I found myself turning all of my components into custom modules. There's sort of a one-to-one parity here. The framework will manage the dependencies of the custom modules, so this is perfect. Dynamic loading — all of your custom modules can be loaded via the loader, through which all of the other components in YUI are loaded. Also, this way I can use the build system; I can create my debug version, my production version that's minified and compressed, and custom modules are what the builder produces.

I've found that YUI modules fall in this scale from general to specific, where the framework is a general set of components to use to build on top of. Then there are extra components which are used maybe across applications. You could think of the YUI gallery fitting in here as well, in this section. Then you have your application components — these are ones shared on various pages of your application, and then your page specific ones, where custom modules are the last three. With YUI 3 having support for custom modules, it's great to be able to extend what the framework gives you.

What I wanted to look at now is Logismo, which is an application we built. It was a utility for us — what we wanted to do was have a nice user interface to logging going on in the servers, and web applications that are doing a lot of things and having a lot of users on them. It would be nice to be able to sometimes hook into the server and say: hey, I want to see logging, I want to see what's going on right now, and we really didn't want to do that in the command line. We wanted a nice user interface, and Logismo is this application. It is a single page application, but it's something that we use, and I want to demo it now.

This is it running. It's polling the server actively — that's what you're seeing going on down here. I could pause logging, and that way I could investigate to see what some of these messages are. We wanted a way to filter the log messages to say OK, I only want to see Warren's [?]. We wanted to add other views into logging. If we wanted to say let's just see things that are JavaScript files being requested, and specify newest on top, I could resume pulling here. Obviously I'm not requesting any more JavaScript files. All these views persist over the life cycle of the page, so if I were to come in and refresh this page, all my views are back how they were before. As you can see, the loader was requested on the page refresh here. So this is our application, and there's a lot of stuff going on here. This log is constantly getting filled up with events that are running on the server, and as you can imagine, if there were other applications running on the server there would be much more messages here. We really wanted a way to filter out messages so that we could look at a specific application on the web server, and see all the logging for that. We also needed to filter by log level as well, to see, for example, if there are any errors happening right now.

So the Logismo application is broken down into a few components here, where the log is a widget, it's rendered on the page, and its job is to receive log messages to render out, and also manage the list, whether there's newest on top, and if they're expanded or not. And then the control panel, the whole top area here: its job is to manage the different views going on and be able to handle setting which view is active via the tabs, and also handle controls for clearing out the log, resuming polling, and locking the scroll bar so that you can investigate what's going on. This is the way I ended up breaking down the application into these functional components. Then there is a top level component which is managing the interaction between them.

I wanted to, again, look at this chart here and see where Logismo's modules fall on this chart. These are the YUI framework modules that it's using. It's not using too much, but we really are adding on here. These are three modules which are utility modules that we wrote — they're custom YUI modules, and we're using these in a variety of applications. Markout is a custom module that we have to write dynamic HTML to the page efficiently with a nice API. Form is a form utility that helps with doing some of the over labels with the labels inside the field, and that's a utility that we use across various applications. IO-poller is an extension to add polling support to IO, and it's something that we use in various applications as well.

The Logismo app has certain components specific to it. In this case, since this application is just a single page application, you could imagine that we don't have any page specific components. Logevent is a simple utility that we have that I wanted to break out the rendering of all of our log messages, rendering out the HTML for each one. Log manages the log messages, and this is the widget which deals with the list. The View is also the widget which contains the form for configuring what the view will look like. Controlpanel manages the views and is also a widget. And Logismo is the top level component which is instantiating, managing, and wiring up all the other components.

I wanted to show you guys the custom events that we're using here. This is the main guts of the Logismo top-level component. As you can see, we have various different custom events here, and some are specifically calling up methods on our other components. So when the control panel fires a pause request event, we tell the event resource which is polling for new events to stop, and when the resume request comes in from the control panel, we tell it to start. When a flush request comes, if you're clicking the flush button or clear button, we tell all the log messages to flush. So what I've found is that having this top-level component, we're really just sitting here doing event wiring, and this becomes a lot nicer and a lot easier to work with than having to write a ton of glue code which would be hard to manage.

The next thing is the YUI use statement. I'm leaving out the configuration here for the YUI instance at the top, but this is what our use statement came down to. We're creating a new top-level component which is the Logismo window, and we're telling it where to poll events. It has other configuration as well, but this is the main thing that the user interface does not know, but the server does know. We wanted to make sure that this gets passed on. Everything else, all the wiring that you saw on the previous slide, happens inside of this component. This configuration happens to be written by the server onto the HTML page. There's no caching going on for the underlying HTML page, and therefore no caching going on here because really, all this is is configuration which may change at any point. It may change when we deploy it somewhere else.

I wanted to look at some other things here, including how do we get from writing our code to deploying our code in the browser, and making the development of custom modules easier for us? Also, how do we get things up onto our production servers without having to rebuild all of our software all the time? Automatic building. Really what we're looking to do here is sync our UI development up with our server side code as well, and to pull it all in to a single cohesive project. We want our UI code to be built alongside our server code. This really enables creating a new feature and testing it out to be much faster. We didn't want to switch from our IDE to go somewhere else — to the command line, say — and then type in commands to build our custom modules, and then go to the browser. We just wanted it to happen more seamlessly; we wanted to go into the browser and hit refresh after we've done editing a source file. Another thing is this really helps people who aren't UI developers but who are also working on your project. We all know that the user interface development is way too complex for any backend engineers, anyway.

We happened to use Eclipse for our IDE. Eclipse is an open source development IDE; it's pretty popular, but it's very extensible. One thing is has is external tool builders, and what they can do is run Ant tasks. YUI Builder, which we use to build our custom modules and which YUI framework uses to build their modules, has defined Ant tasks which we can run in Eclipse, and they have triggers. For example, when you're editing the source file of your custom module and hitting save, we can run the build on that source file to create the custom module. Really, what this enables is that saving a source file will run JSLint, which will show any errors that may be detected before going to your browser, compress your files, and deploy them, also create debug versions, minified versions, and raw versions of your source, or of your custom modules.

More on these derivatives created. The debug files take your source code as is, and Luke was just showing debugging. If you write Y.Log messages, these will log to the console, and this is something very important when you're writing your module that you want to test out. You want to do debugging. We also want to have minified versions — maybe we want to see how much K weight we have going over the wire, how heavy these pages are. Having the minified version also allows us to test something like YSlow without having to do any other processing once you're writing your code to test out your custom modules.

Another thing we found is that a lot of times, the server knows best in certain circumstances about how to configure the user interface. We wanted to write our YUI instance configuration dynamically. It can be written on the server. Really, what you're combining is finding out where we're polling YUI from and where we are getting the framework from. The instance needs to know this information, and we could be polling it from the Yahoo! CDN, or we could be pulling it locally. One example of why you may want to pull locally is if you have HTTPS and you need to serve a page securely the Yahoo! CDN does not support SSL, so therefore you may have to host it locally.

We also need to tell the YUI instance about our custom module's meta-data: what it requires, and where the files exist. This is used so we can dynamically load them using the YUI loader. Also, other information — this could be like we want to run in debug mode to get logging messages to be output to a console. This is something else that we can do on the server side so that we can easily hook in, as I'll show later, with a query string parameter on our applications and say we want to run in debug mode by simply typing in query string in the URL. Really, at the end of the day, it's just taking this information and writing it out as an object literal, or effectively JSON data.

So we have the YUI configuration object. We happen to have abstracted this out into server side code to make this stuff less tedious to do. You have a YUI host, and then you have a set of custom modules. We're defining the YUI host information on the server, and we're also defining the custom module's meta-data on the server as well so that decisions can be made. For example, while we're in production, do these custom module files actually exist? Because it may be different than development. And we sort of push all this stuff into the YUI configuration object, which then is really just written out into the YUI instance.

I can show an example of how we can get into debug mode here. We have a hook in here, for example, with debug equaling what we want to log. We want to log the log custom module, the control panel, and Yahoo! or YUI event. By doing this, the page loads as normal, but if we look down here, we're now getting log messages spit out. All of our YUI modules are loaded in debug mode here, and we're also not combining the files so that way it makes drilling down — if you do have an error — much easier, because you can see the source, it's the full source, it's not minified, and you have debug messages logging out to the console. This is something where generating the YUI configuration object on the server allows the server to understand this query string parameter and change the YUI instance. This is something we found very useful, especially in trying to solve problems.

With YUI 3 as the framework, and automatically building our custom modules and choosing certain parts of the configuration to generate on the server, we really found ourselves in a good place. You, in your own development environment, will probably — definitely — have specifics that are different, but I think the concepts of trying to hook into your IDE to get your custom modules built automatically will really help ease your development. And it really helps your projects fit in to the larger scope of things, to fit in with the server side code, to fit in with other team members, being able to copy the source files of the project and build it easily.

It really helps developing features; it makes it much faster. We took some time to set up something as simple as just adding a query string parameter. Having your files built automatically, and doing this server side YUI instance generation, really helps deployment. You can choose the important things that are different between your development environment and production environment and make them variable, so when your code gets pushed out to production, it is then configured for that environment; your user interface is then configured. Doing this stuff has made me a lot happier.

I wanted to thank the YUI team for having me out here. I'm really glad that everything has opened up recently, in the last year — it makes it much easier to communicate with the developers and work on this stuff. Since it was less than a month ago or whatever that YUI was officially released, to see everything before and be able to use it and get help on it, that's helped out a lot for our developing with YUI 3.

[applause]

Eric: I believe the question is how are we separating our files for debug mode? We're using the YUI Loader to load in all of our dependencies for a particular page dynamically. Normally YUI instance has a combine flag that can be set to true or false, which is true by default. When the server detects that we want to run things in debug mode, it simply switches that flag when it writes out the YUI configuration to not combine things, and also to grab the debug versions of all the files instead of the minified versions.

As it stands now, custom modules are actually loaded with one HTTP request per custom module, but with the filtering — and I believe Luke touched on it some in the last presentation — you can say 'filtering debug' or 'filter raw' or 'filter min', and those also apply to the custom module URLs as well. The loader will actually look at the path to your custom module file and say: replace '-min' with '-debug'. We're definitely leveraging the loader and the YUI instance configuration to help make the code loaded on the page very suitable for debugging.

The question is: is there a way to easily combine some of the files but have others be dynamic? One strategy that we've looked at is using combo handled script tags to set the bases. If our application has five pages, they all require some subset of modules. What we may do is make what you may call a big seed file to load up on the page and help for caching, as well, and then have the loader fill in the specifics of that page that aren't there. So yes, instead of just including the YUI core seed file, you could combo handle, say, YUI Loader, JSON, and IO, if you're using those types of things on all the pages in your application, and then have the loader fill in the blanks where the code isn't part of every page.

Audience member 1: Do you use PHP Loader?

Eric: The question is, did we use the YUI PHP Loader? No, we actually aren't using the PHP Loader. Our server side code is in a Java stack, so for the sake of serving YUI locally, we have implemented a special fast directory server to help serve out our static files, deal with combo handling, and far futures, expires, headers, e-tags, and all the best practices for serving static files.

Audience member 1: Does that take the same configuration as, I think [xx]?

Eric: Yeah, the way that the YUI Loader builds up its combo requests for a set of modules is it just uses query string parameters for each one, so we made sure that our combo handling server could handle that sort of API for requests coming in.

Audience member 1: Are you going to put that on the Gallery?

Eric: It's not JavaScript, so we wouldn't put it on the Gallery. But some of our other components I do plan to put on the Gallery, like our IO Polling utility, and our utility for writing markup as well.

The question is, are the servers serving the library GZipped? Yes, the CDN follows the best practices for GZipping using far futures, expires, headers, and I'm not sure if it uses last modify date or e-tags for conditional get requests, but we also took those best practices on our servers to serve YUI along with other static files locally. So yes, GZipping is being used.

The question is, how are we breaking apart our modules, and when do we decide where the deviation is between application specific and page specific? And when do we break out our top-level component? For the Logismo app that I showed, the first time around I wrote it was over six months ago, and I had 800 lines of glue code in my YUI use statement. It was extremely hard to follow, going back to it a few weeks ago when I rewrote it. In that stage, we already knew what the features were for the application, so we looked at the features and tried to break it apart and say what are the functional pieces here? We made the components — for example the control panel, and then the view, which is contained in the control panel, and the log — we were sort of able to draw boxes around what is what, but it did also grow organically. Keeping in mind that we wanted to have this top-level component helped the API for these individual functional pieces as well. I'm not sure if that helps.

And then for the other part of your question, if we have multiple pages for this application and we found ourselves using the same component over and over, we would consider those more like the application components where, if we have something just specific to a page, it would be a page component. But yeah, we look at the top level component as getting told by the server, maybe, what configuration it may not know about. Like for the polling example, what's the URL to the resource that I'm going to make requests to? Nobody knows besides the server, if we want to make that variable.