Video: Stephen Woods — Creating Beautiful Documentation with YUI Doc

Video published 2009-10-29.

• Download iPhone/iPod-compatible video file (320x240, .m4v)
• Slides

Transcript:

Stephen Woods: Hello everybody, my name is Stephen Woods. Here's my information. I work in a Yahoo! platform group doing JavaScript frameworks that are used internally, which means my entire life is creating APIs. Documenting APIs has become a really tragically big part of my job. I feel like I have a lot of strong opinions on this, and hopefully today I'll share some of that wisdom with you.

If you haven't yet, sometime in your life you will have to write APIs that other developers are expected to consume. If you're anything like me, the last thing you want to do is write docs. It's totally boring, it takes forever, it's not fun, and it's not what you got into writing code in the first place for. But unfortunately — or fortunately — undocumented code is so much worse than writing documentation that you'll be glad you did it.

Now, the common response to this is that I don't need to document my code, right? My code is so easy to read that no one needs to have documentation for it, which is great if you're just writing code. But if you're writing APIs, you should not expect people to open your code up, figure out what it does, and then use it. If you're writing APIs then you have to have good documentation; it's not an option. In fact, if you forget to do it, you'll end up with a situation like I found myself in over the last year of so, where people come and ask you questions that you probably should have put in the documentation.

[referencing slide] You think? Yeah, that's me, I admit it, I said that. I found it in my IM logs. Yeah, I think that might be part of the problem that the documentation is not correct. If the 'it's good' argument isn't enough for you to write documentation, how about self-preservation being a good reason to do it? If you don't want to spend your entire day talking on IM explaining how what you wrote works, you should write good docs.

Let's talk about how to write good documentation. If you ever encounter any API out there — maybe Rails, or maybe you hate yourself and you've been developing Windows software, or maybe you love yourself and you've been writing with YUI, regardless — you've found that there's incredibly complete API documentation. Every method, every property is documented and covered, and you might wonder how exactly they did that. My theory before I got into doing it myself was that people did it using some sort of army of highly trained technical writers. The YUI library does not have this army yet, but I think they're working on it.

Previous to this, this was my system of documentation. It worked incredibly well. I could keep my documentation totally in sync, never had any trouble, people were always very happy to read it, and there was no problem at all. When I had some of the problems that I've outlined here, my solution was a certain policy: OK, I'll write a document that says this is how you write documentation. OK, I'll make a checklist that helps you write documentation. None of that stuff really works, because ultimately everyone's just going to ignore it. What I really want to have — and this was my dream for a long time, and I didn't realize the dream had been answered — was to write code, and have docs. That's it, that's what I want. I don't want to have to make docs. So ideally there's some sort of magic box that will make my code writing turn into doc having.

Today we're going to talk about the magic. The magic exists in a lot of forms; these are all different magical tools that generate documentation for you. Generating documentation makes your life so much easier that once you start doing it, you'll wonder why you didn't do it before. Then you'll find yourself, hopefully, getting in the habit of writing the comments that can be generated from it all the time. What a doc generator does, if you don't know about it, is it takes your code and makes documentation from it. It's not a template, it's not a format, it's not anything — it literally takes your code, makes the documentation, and everything else is done.

Now, there's varying degrees of smartness in these tools. With PHP Documenter, you don't even have to write comments — it will write documentation for you. It won't be very usable, but it will do it. YUI Doc is a little less intelligent, which has its benefits and downsides, but I think the benefits outweigh the downsides, especially when you're using JavaScript. What YUI Doc does is it takes these comment blocks right here and that it's, and makes beautiful documentation from it auto-magically. YUI Doc is a language neutral way to auto-magically generate beautiful documentation. 'Auto-magically' is a word, please, OK?

So what does language neutral mean? Like it said, it only reads the comments, so whatever language you want to write in, YUI Doc you could actually use it for because it doesn't care, it always works. In fact, YUI Doc itself was written in Python, and it's documented using YUI Doc. Now, we all love JavaScript the best because it's the best language, so I'm going to focus on JavaScript in this talk.

What language neutral means is that we generate from special comment blocks. It doesn't read the code — well, it does a little bit; Adam, who wrote the code, wanted me to mention that it does a little bit — but for all intents and purposes it generates it from comment blocks. So it can work with any language, and the most important thing for JavaScript is that you can describe the code in whatever way makes sense for your context. Because with JavaScript we can be very, very creative with how we build our applications and our object models; we don't have to follow an 'interface — class — subclass' approach. And with a language neutral documentation approach, you can describe what you think people should understand it to do, rather than trying to write your code to meet that view.

Now, there is a competitive tool, JS Doc, which does try to read the code and is, in my opinion, limiting because you have to stick to a way that it can read. It's not that painful, but I'd rather not have to worry about that. I'd rather be able to describe what my code does, regardless of how I've decided to lay it out.

So there are some downsides to language neutral. Your code is going to fall out of sync — I guarantee it, it's inevitable, but it's better than it was if you were writing separate documentation. We're going to talk about ways to solve this later. You're going to copy and paste. Maybe people will tell you not to copy and paste your methods into descriptions because you're going to make mistakes copying and pasting, and it will happen. Type mismatches will happen, and I'm going to talk a little more about why type is so important in JavaScript documentation. Unfortunately, this is one place where documentation almost always gets out of sync.

What do I mean when I say 'beautiful documentation'? We all know what beautiful code is, or hopefully we do. We look at it and it's elegant, it's beautiful, it's clear, and its solutions are simple, it just is almost self-apparent that it's beautiful. I think documentation can have this. It's a little less exciting to see beautiful documentation probably, but the most important thing in my mind that makes it beautiful is that it's accurate. When I read the docs, they say what is actually going to happen when I call that method. It really is correct. It's complete. There's no: oh, that method's not in the documentation, but it's really useful. It's complete documentation. And usable — that's really important. If people can't find your documentation or get to the method, they're not going to be able to use it. YUI Doc is particularly well suited to this, mostly because of the magical auto-complete.

And finally, it should be understandable. I didn't have this on the slide before and my colleague, Chris Heilmann, told me I should put this on here. You can have documentation, you can write documentation, but it can still make no sense. Unfortunately, we can't automate writing sensible documentation, but I would hope that you would know enough when you're writing a comment — if you've ever done UI design before, it's the same idea. Think about your users, think about their needs, and try and meet those needs. Your users are the people calling your API documentation, so they need to be able to understand what it does. When you're writing your docs, instead of thinking: what I did here was implement this algorithm, think instead: what does your user need to know about that method? How do I explain that to them?

Now for using YUI Doc, which is the whole point of this talk in the first place. YUI Doc is really pretty simple to use, and it also is very, very powerful. I'm going to walk through a very simple sample project — in fact, you can download it right here if you want to check it out. I have a directory of my project, I have two modules which we'll talk about in a second, and which in this case are directories, compression, and widgets. Don't worry, I don't know if this code actually works, so don't worry too much about that.

What this translates to in the YUI Doc world is a layout something like this. You have your project, you have your modules… Unfortunately, module is the world's most overloaded term. In this sense we think of modules as a piece of our code. In the YUI world it's anim, DOM, event, loader; those kind of overarching pieces, but it can mean whatever you need it mean. Then modules have classes, and I'll talk a little bit more about what I think about those later. Now, like I said before, we describe those using comment blocks. Really simple syntax. There was a description for every comment block, then there were tags, and secondary tags. Each block can only have one tag, but it can have however many number of secondary tags. These are the primary tags. I don't expect you to remember this all now; we'll have to look at the docs for YUI Doc later.

Now, there's a huge list of secondary tags, and obviously I have no time to go into detail here. I'm going to talk a little bit about parameter return because I think those are particularly important, but as you can see there's a lot here that gives you the power to really describe what's going on. The whole goal here is to create API documentation that people can count on to describe exactly what's going on. That's why we have so many options in YUI Doc. You see there's a little magic star here on YUI Attribute. We're going to talk a little bit about that later, but if you saw some of my colleagues' talks about using widgets and attributes in YUI 3, this is a special feature in YUI Doc that allows you to do some automatic stuff with that.

So why do I think these are the most important? Because of data tags. When you're calling an API, you have return values, and params, but type that is expected in return is really, really important, because it can confuse and infuriate you if an API doesn't do what you expect it to do. Fortunately, or unfortunately, JavaScript does not have any kind of type hinting. You can't define that this method requires a type, but you often create a method that does require a certain type: a string, a number, an object, or whatever. So in the documentation you need to tell people what type you're going to give them, and what time you expect. Now, the beauty of JavaScript is that it's quite descriptive language, so type can mean a lot of different things. We don't check that and make sure you follow our list of approved types — whatever you want your types to say, they can be, although I recommend you be consistent. For example, maybe you created a special object interface that you expect to always receive for this method, and config objects are always a really common use case for this.

Let's talk about modules — again, the world's most overloaded term. In this case, a module is sort of an overarching piece of a library that you've created. In my case it's compression, for example, so I have this compression module and I have a sub-module that just does LZW compression. You can see that each block has a tag, has a sub-tag, and then those sub-tags take parameters. Now sub-module, I'll show you what happens with that. This doesn't quite match that code actually, but you can see what happens is that it shows on the documentation where the sub-module is, and shows you what classes are inside of this module.

Now, what do we mean when we say class? We don't mean anything, because in JavaScript there is no such thing as a class. I tend to look at a class as anything that's an object that has methods and properties on it that people need to know about. If I'm writing documentation that's what I refer to as a class, the reason being that in my experience, most users of the documentation see it the same way: they see a class as an object. So when I'm writing documentation, I'm again trying to think of what the user is expecting, where they're coming from. Most users have at least experienced classes and they know what they mean, so say that this is an object that may be a constructor or whatever, but it's an object that has methods it can call on it.

You see I've used a construction tag here, because in this case the class can be instantiated with new, and it extends widget. The result is pretty descriptive. You can see the constructor here, and you see it takes no parameters. You see the methods, you show the return value which is void, and it's very informative about what people need to know. Now people know exactly how to use this time figure widget, which is the whole point of this exercise.

Now, events. We do a lot of events in JavaScript, especially when we're doing client-side applications. APIs have a tendency to fire events, consume events, and in YUI 3 we don't even have to describe those events before we fire them. Oftentimes you will end up documenting something that doesn't even really exist in the code — there may be a line somewhere that says 'fire this', but it isn't instantiated or created at any point. So I often use events in the way that I'll just put in a certain section of my code document, so that when I'm looking at my code to see what events are being handled, and when other people are trying to consume the code, they can see these events. And again, you can see how important it is that type is described here. This turns out looking pretty nice. We understand how to handle the DOM event.

Now, properties. I hopefully don't have to explain too much about what a property is. Obviously you can see the syntax is a little different because property's a major tag, so we had to create a sub-tag for type for properties, but again, the type matters. You get this really beautiful thing.

What I was talking about before, the magic from YUI 3. My time picker uses attribute provider, it does cool stuff with attribute because it's a widget, so I know that there has been automatically generated some events when attributes are changed. So I've documented it this way: I say there's a property called attributes and the attribute time type object has this default value. As a result, event is documented automatically because that event is there — I didn't have to actually tell YUI Doc that event was there. It's useful, but only useful if you're doing YUI, so obviously you won't want to use this otherwise.

The meat of API documentation is methods. When we're talking about APIs we're talking about calling methods on an API — that's the number one interaction — so if anything is the most important part of your documentation, it's that the method documentation is accurate, complete, and reliable. I have this weird method that tells you that the string size takes a string and returns a number. Pretty important people know that this is probably not going to work too great if you try and get the string length of an array or an object. And it comes out beautiful. Now people can look at the documentation, understand what's going on, and use it.

So you've written your code, it's really beautiful, and it's easy to use. How do you build it? It's so easy that I'm just going to point you to the documentation online. If you want to take that down: http://bit.ly/yuidoc

In more detail, what happens: it's a Python script, it runs from the command line, and it looks pretty gnarly if you don't spend a lot of time in the command line. If you want to understand it more, there is a [xx] that will show you all the different options. Most people — and this is the real power for having a command line tool rather than an executable that just runs — is that you integrate it into your build. If you download YUI Doc now you're going to get this example: the build script is just a simple shell script, you set up some of these values, and it's pretty straight forward.

Now, if you use this right now out of the box you're going to have a problem in that the documentation is going to pretend it's documenting YUI, and it'll say YUI and have the links. So there's some more stuff that needs to be listed there. Again, in that package I put up you can check this out. All these values just get plugged in to that script. If you're like me and use something a little more complicated than a shell script, maybe use Ant titles like content. I didn't do that, someone else must have messed that up.

There's a giant bear coming in there. Why is that?

[laughter]

Stephen: I actually don't know how to process that. Wow.

Audience member 1: It's Halloween.

Stephen: Is it Halloween?

Audience member 1: A gnome just walked by too.

Stephen: Oh well, that I'm not surprised. Gnomes I understand, but bears are a little weird.

Anyway, where was I? I'm using Ant because it's easy to integrate YUI Doc into Ant and then put it right into your build.

A couple of little things to watch out for when you're doing this in your build: right now if the temp directory that's being output by YUI Doc contains these files, they will be parsed, which can cause some weird issues. For example, if you copy your CVS directory from one directory to the other, the new directory thinks it's the old directory, and everything gets messed up. Adam is aware of this and it's being worked on, but in the meantime, if everything goes crazy with your source control, this is what's happening.

Audience member 2: Does that include [xx]?

Stephen: Basically it doesn't have a 'don't look at these' list, it doesn't have an ignore list.

Keeping docs relevant — this is the big important thing. If you've written beautiful documentation and it's perfect, like I showed in that slide earlier, three bug fix releases later it might have nothing to do with your code. You can still do this. It's a lot harder to do this if you're documenting inside your code and generating, but it can still happen. There are some simple rules. First of all, generate documentation as part of the build. This is so critical. If you do a build thinking you can rely on someone to remember to generate the documentation later and put it up, they're not going to remember, and the builds are going to be out of sync with the docs. That's a big problem.

Don't just have someone manually check documentation into version control systems. In fact, why would you check your documentation into version control anyway, if the documentation is being generated from the source? Unfortunately at some companies — I won't mention any names, but they start with a Y — we have a documentation system that requires us to check our documentation into source code. The way I deal with this is that my build checks my documentation into source control. I don't physically do it because when you introduce a human into this process, that's when things get out of sync. Like I said, better living through automation.

This is my favorite one: treat documentation like code. If you're writing an API — and this is probably true of any kind of code, but APIs more than anything else — your documentation is part of your code. People are using your tool and they count on the documentation as much as they count on the actual code itself. If your documentation is out of sync, they can't use the API, and they're just as unhappy as they would be if there was a bug. That's why I say if you're doing code reviews, look at those doc comments. If your code is expecting an object and the doc comment says it's expecting a string, have something to flag in a code review.

Comments or source, docs or compile code? I think this is the easier way to think of it. That's why I don't say to check your docs into version control, because you wouldn't check your binaries into source control. Your comments are the source code, and that's what you need to be looking at. You should be treating it like any other source code: you should review it, and you should have it in CVS. One of the cool things about doing it this way is that you've checked in your bug fix, you've updated your documentation, and someone can look at the deflator and they can see the documentation change just as well as they can see the other changes.

And what if it's broken? File a bug. It's code, it has a bug, the documentation is wrong, so file a bug and tell people to fix it. Again, it's just as bad as any other bug, because your code may work perfectly but if someone goes and tries to call it, it's going to fail, so to them it's a bug. To you it's: oh well, I just didn't fix the documentation, but to everyone else it's a bug.

Let's try it out real quick. I don't expect you to do this now, but I just wanted to cover this. I threw up an example on GitHub, and when you download it you're going to get this stuff. It's that same project I've been going through before. You can build it really easily, provided you have installed YUI Doc. In the earlier links there's instructions on how to get it installed. Or if you want to use Ant, I've included sample build files, a shell script, and we'll build XML for Ant. It's pretty cool. I'm using Fing [?] in my build rather than Ant, but that's not by choice. It's the same idea, right? I have a build process, it does YUI compressor, it does all this other stuff, and it generates my documentation. It's just part of the process. I don't have to think too hard, and my docs are always there. I also have a build clean that will remove all of the docs, because when you're done building, if you push your Doc somewhere, you don't really need it anymore in the build directory.

YUI Doc has some other cool features. Let's say you don't like the beautiful styles that we have, and you want to make them really wonderful and brown for some reason. It's really easy. Obviously I'm not much of a designer, but I like brown for some reason. Anyway, I made mine brown and it's easy to do. There's a style sheet there, and I think it's incredibly simply laid out. We use YUI Grids — HTML is designed to use YUI Grids — and customizing your template is generally as simple as adding the style sheet. But some of you are obsessive compulsive and want to have the perfect documentation layout, so we use Cheetah Template, which is a Python templating tool. If you use Django, generally you've probably been exposed to this. Then you can get all fancy and make it look however you want it to look. I'm not going to go into Cheetah, but look at our templates, look at the Cheetah website, and you'll see how powerful this is. You can really do whatever it is that you want to do.

Now how about this: custom everything. You're not happy enough changing your template, you're not happy with your custom template, you want to change the frontend completely. Luckily, remember I talked earlier about those temp directories that we output when we were using the parser, and saw set up? Those contain a JSON representation of the entire documentation. So if you want to write your own custom frontend in whatever language you want… Actually I just found out this morning that someone has written an iPhone app that contains the YUI Docs. There's no reason you can't do this; it's a JSON structure, you can consume it, you know how to do that.

This is Dav Glass' Adobe Air app for the YUI Docs. That output can be parsed by whoever wants to parse it. We don't have to worry about how we do it. A little bit about that, just so we can get it straight — remember back here? There's that parser route, and that contains the complete representation. Everything that you see in the page is in that JSON structure, so you can consume it and do whatever you want with it. I think this is actually one of the most powerful features of YUI Doc, because other documentation tools are really focused on the output and on the fact that it makes HTML. This tool allows you to get in the middle of that and do whatever you want from it. A cool bonus feature is that code highlighting is in there. If you just like code highlighting you can steal that and use that yourself. I think that's cool.

I'm moving really fast for some reason. Let's talk about my parting wisdom. Documentation is part of the build. This is so critical. If you're not doing this, you're going to have pain, and I know this because I was not doing this until fairly recently and my documentation was constantly out of sync. People would say: Stephen, the docs say this and it doesn't work, and I'd be like oh yeah, I haven't generated the documentation, I'm really busy. I'm working on my presentation for YUIConf. So use tasks if you're using a build tool. I recommend you use a real build too, by the way, and not a script, because they give you all these bonus features.

This is another idea that I like. I haven't been able to do it because I can't get the hardware to do it, but having a documentation server's really nice because there's so many that people that it can just point to. You can say the docs are always there, and they can rely on the up to date documentation being in one place that they can always go to. I find that to be the easiest way to handle this, but unfortunately you have to beg someone to give you hardware to do it.

It's 12.15 and I'm sure you're all hungry, but hopefully you'll have some questions too. Yeah?

Audience member 3: Does it work pretty well — I mean, obviously it does — when the language is ambiguous, but do you have any examples for PHP?

Stephen: I don't have an example of PHP.

Audience member 3: Yeah, PHP Documenter uses it, so YUI Doc [inaudible].

Stephen: Yeah. My favorite example of the language neutrality is the documentation for YUI Doc that can be generated out of Python.

Audience member 3: Would you compare it to PHP?

Stephen: The huge difference is that it doesn't read it right. PHP Documenter does read the code, which has some real benefits especially since PHP is a lot more of a faux static language, so you can have type hinting and things like that, and formal classes. But reading the code and scribing it in documentation is a lot easier because the idiom is a lot clearer of what's going to happen. So just reading the code and generating docs works pretty well. But the downside — and this is the upside and downside of this — is that if you're using PHP Documenter, you can get lazy. If you know it's always going to work, that it's there and that you don't have to update your comments… I think there's something to be said for having the have the discipline to make sure that those comments really describe what's happening. If you get in the habit of 'if I change this method, then I need to change the docs', that's a good habit. So it's a bug and a feature: that's the big difference.

Audience member 3: Do you guys update your docs on every release?

Stephen: It's built into the YUI builder tool, so I'm assuming they do. I work in a different group so I don't know. But my group, yes, every single build. The other benefit is that if you have it in source code, you can actually just check out that snapshot and build docs for that snapshot. You know that you can generate docs that are in sync with that moment. When I do a release I do a fork and a subversion, so every time a do a release, if I need to produce docs for that release I know that I can always do it because it's just right there.

Audience member 4: How do you format the C-tag so that it creates a link to whatever other class you want it to go to?

Stephen: The C-tag?

Audience member 4: Isn't there a C or SE tag?

Stephen: Not that I'm aware of. There's extends, uses with…

Audience member 4: Maybe it's something I….

Bart: There is.

Stephen: There is? OK, Bart knows. What's the secret, Bart?

Bart: PHP Doc.

Stephen: Oh, in PHP Doc?

Bart: Yeah.

Stephen: Oh, OK. If you do things like extend, it automatically generates those links with other classes or users.

Audience member 4: I guess the most common place I would use this tag is, for example, in the storage engine there's an interface, and classes that extend that interface, and I want to just point back to the interface where you could, rather than having the same definition repeated…

Stephen: I'll tell you how I do this. If I'm doing interfaces, since interfaces don't exist in JavaScript, I will actually just write a comment only file that contains interfaces. And I have done this before. I write comments that describe interfaces and use a formal i-this, i-that, class i-fu, and then I say that next one extends i-fu, and YUI Doc will automatically generate that link. So that will do the job.

Audience member 5: Are there any plans to make a standalone compiler version that doesn't require Python?

Stephen: No, but it's on GitHub and you can do it yourself. [laughs] I would say it's a minor hassle to install — once you have Python you can use Easy Install from Python to get the little things that are required. But anyone who is a Python hacker is free to do that, because that would be pretty cool. But no, not right now.

Further questions? No? Well, let me put back my information. Again, I'm Stephen Woods. The person who wrote this originally is Adam Moore, and he's the real super expert. He's available on the list and the IRC and all of that. Also, please, if you want to bug me — and I'm totally buggable — there's my bugging system.