Video: Isaac Schlueter and Matt Hackett — Server-Side JavaScript

Video published 2009-09-08.

• Download iPhone/iPod-compatible video file (320x240, .m4v)

Transcript:

Isaac Schlueter: Can you hear me? Excellent. Basically, the slides were whipped up kind of the way here. They're not like Nicholas' slides that are full of pictures and fun stuff; they're mostly just text. We're going to tell you a little bit about what we built, and then we're going to show it, and then answer some questions.

The basic point that got me, at least, starting in this, and that I think is really exciting about server-side JavaScript, is that context switching really sucks. OK, you're a Python guy, and you know JavaScript, and you have no problem at all going back and forth, and then one day you save some Python file and it doesn't work. You look at it and you don't understand why it's not working, and you're on a deadline, and you realize you put semi-colons all over the damn place. I'm not a Python guy, but I've made similar mistakes in JavaScript, going back and forth between JavaScript and PHP, and it's ugly.

Basically, for the typical LAMP stack, you've got JavaScript for the client, you've got PHP for the server, you've got some Apache Insanity for your URI rerouting, which has regular expressions - don't even get me started about Apache 1.3 versus 2.2, and how they handle those. You've got this other domain-specific language, either MySQL or some other SQL or something, for data. Basically, it comes down to at least four languages, probably a lot more.

One way to fix a lot of that is to ditch this whole concept of having the file server, and just go straight to using an application. This is something that Ruby and Python folks have been doing for a pretty long time.

We've got rack on Ruby, WSGI on Python. There's a couple of other implementations. I'm sure whatever your favorite language of choice is is probably coming pretty soon . The problem with that, with using Rack, or WSGI, or Django, or what have you, is that you still end up having to use JavaScript, you still end up having to go back and forth, because whatever's on the client is going to be running JavaScript.

There's a couple of ways around that. One of them is to compile your Ruby or Python code down to client-side JavaScript. That works. It adds a compilation step, which is arguably just as a bad as a context switch. Another possibility is to build for something other than the web browser, like Titanium, which supports client-side Python and Ruby. But then you have to bundle the interpreter with your application, and if you're not building for web browsers you're not really building a web app.

Also on the data side, you've got ORMs, which kind of tries to map your relational database table structure to something that is more native to your programming language. But most of the time you end up sharding and having this big table of a handful of records, and it's mostly one table that you're looking at most of the time, and it ends up being a total waste of all this relational power.

So if JavaScript is your favorite language, you actually get to use JavaScript from top to bottom here. You've got, obviously, JavaScript for the client, you've got a server running in JavaScript, we've got JavaScript for our URI rerouting, JavaScript for our data connection layer. Basically just one language end to end, which is wonderful.

The CommonJS group - there's a handful of them in the room, if you were taking advantage of those networking times, you probably met a few of them - basically just a bunch of programmers who think this is a good idea. There's an IRC channel, and a wiki, and message group, and lots of bike-shedding. The goal is to come up with a spec so that we can have programs that run on different implementations of a server-side JavaScript stack. So you can write your program, and it'll actually work in multiple JavaScript environments.

Now, of course, there's leaky abstractions – we know how well that works when you write JavaScript for one browser and expect it to run in all the browsers. But it's a little better than that. Narwhal is a specific implementation of CommonJS; it's the one that we use. I've been playing with it for awhile. It's pretty cool. There's a lot of pink text all over the web pages. That's it. [laughs] I'm sure there'll be questions – we'll probably say more about exactly what Narwhal is, and what CommonJS is, and what JSGI, and what Jack is later on - I'm just trying to give you a basic idea for how to understand this.

How do you guys pronounce JSGI? I've been saying "JSGI", but I know WSGI is usually pronounced "whisgee".

Audience member 1: It's just like that. It's a permutation of WSGI.

Isaac: Is it just like "sky"? Yeah, I guess "jay-sky" would be kind of like JSON, versus…

Matt Hackett: I heard "jay-gee".

[laugher]

Isaac: I don't know. I'm just trying to keep this G-rated.

Audience member 1: JSGI.

Isaac: Right. I'm going to keep calling it JSGI until somebody gives me a really cool pronunciation for it.

Jack is an implementation of JSGI. It was built by Tom Robinson, who's in the room; you can bug him later. It's an implementation of JSGI on Narwhal, and it's a collection of a bunch of middleware pieces that you can use to mix and match to build your application. It's kind of like Rack, but it's JavaScript instead of Ruby, so you change the 'R' to a 'J'.

Jic-Jack-Jo. Matt and I had this idea that we were going to write a game on server-side JS, to see what this stack can do, and actually use it for a proper project. It turns out, though, that I misinterpreted when exactly this conference was, thinking that it was in November. Don't ask me why. The email that I got, it clearly said September 8th, but then when I looked at it I was like oh, crap, that's a week. And that was when the announcement had gone up on MeetUp, and YUI Blog, so we were committed.

So we said: OK, well, what can we do in a week? We can't actually have a full-featured game - let's have a not-full-featured game. So it's a very quickly whipped up game that… It runs on Jack. It took about two solid days of programming, but it was pretty easy. And also, we're pretty awesome, so that helped.

[laughter]

Isaac: The code and the slides are both up on my GitHub repository.

[to Matt] Demo time: that's your cue.

Alright, there we go. We've got two different web browsers. When you're first connecting, it gives you a cookie if you don't already have one. It makes a call to a view sitting in the CouchDB database - I can show you guys the code for that in a minute - to find out what the newest game ID should be. And then it basically hits our server-side layer to create a game, rather than doing any complicated kind of in-memory caching stuff, which would be super, super smart, and you would definitely want to do before putting this on a public server. Instead, we just say CouchDB is our cache; that's our concurrency, that's our memory, everything. It's probably a good idea anyway, because in process memory can kind of get corrupted, or fall apart, or what have you, if you restart your server.

Yeah, so Matt whipped up some client-side stuff, and as you can see it's just pulling the server every time you make a move. And you don't know how to play Tic-Tac-Toe.

[laughter]

Isaac: Right. It's not as friendly. My error messages are so friendly, I swear. His, not so much.

[Inaudible audience comment]

Isaac: Well no, the loser message says: "Way to lose, loser. Play again or refresh the page, I'm not going to hold your damn hand."

[Inaudible audience comment]

Isaac: It does, it does. Well, you know, it's encouraging you to be competent and act on your own. You don't want to baby your users, because then they get weak.

[laughter]

Isaac: [to Matt] Oh, OK. Yeah, let's get in that. That's fine. That's why I keep you around. I know, I know. So we demoed.

OK, I'm going to tell you the ending before we get into the details and the nitty gritty, so you can see where we're going. If you need a professional level app – if your boss is getting on you to build this thing, and you haven't started it, and you've got investors who are banging down your door, and your start up is going to go belly up if you don't get this app out yesterday, you might not want to use server-side JavaScript. It's very exciting, and it's moving really, really fast, and it can do a lot of really impressive things right now, but it's not at the level of maturity of, say, Rack or Django.

But if you are going to be taking a while to build something, if you're getting started on a project now that you've got some flexibility on, you absolutely could get a lot out of using CommonJS. I have nothing but good stuff to say about Jack and Narwhal. The biggest thing that I can say about it is that it was really fun to build. When you remove that piece of having to change languages repeatedly throughout the course of your development, it really is just a very liberating experience.

What's also great about that: there's one class in particular where… Let's say you have a form validation code, right? At some point you have to express in some kind of a code what qualifies as a valid name – what qualifies as a valid username, what qualifies as a valid zip code, and phone number, and so on and so forth. It's different in different internationalizations, and different locales or what have you. You want to do that on the client side because that's nice, so you can give immediate feedback to the user – they don't have to wait until they submit the form, and then: oh, now you tell me I have a typo in my phone number, thanks! But at the same time, if they don't have JavaScript, you still have to be doing that same validation on your server side, because that's also, a lot of times, a security issue. If somebody put some XSS script in their username, now your whole site is compromised.

You end up having to write that code – at least, here at Yahoo!, we end up having to write that code a lot in both PHP and JavaScript, or both Java and JavaScript. Or you have to do all of those calls by Ajax, and you have to have some server side validate form, API. And then you write your username, and then you tab three fields down and now it's sending out the request, and it comes back asynchronously and says: oh, your username's invalid. So now you have to go back up to that one, and it's just a big pain. In this case, when you're using server-side JavaScript, there's a lot of cases where you can actually write that code once and use exactly the same file on the client, and on the server.

In fact, in our game, we have some logic around what's a valid move, and how do you know whether or not a move wins the game, and whose turn is it, and so on. Basically, the whole game of Tic-Tac-Toe. It doesn't sound like a lot of things, but there's a lot of little moving parts in that. We have one board.js that is exactly the same code on the client as it is on the server; that was fun.

You guys want to see some code?

Audience member 2: I'd like Matt to explain it.

[Isaac laughs]

Audience member 3: He's just a pretty face.

Isaac: Yeah, he's just here to make me look good. Or worse.

Matt: You've all written code twice before, where you've got your page validation and your [xx] validation. The really cool thing about this project was, I wrote the board class, which... Tic-Tac-Toe's not complicated, but having to write that once on the server and again on the client is kind of bitch work; nobody wants to do that. Having to write it once, and giving it off to Isaac, he called the exact same method that I am, which was really, really nice. It saved us a lot of time, and that's why it enabled us to write this in two days.

You've got some things in the client that you're not going to need at the server and vice versa. But when you have JavaScript, which is really trivial to make an event-based system, you just say, well, I'm going to listen to this event, so on the server, when you fire up a move, I've got to hit my CouchDB. And on the client, you say when I fire up my move, I need to show that on the screen, which, when you're rendering your board, that's irrelevant to the server, but you want to show the client. So you write the code once, you listen to events, and you're done on both sides. It makes less code, which, as you all know, is usually better code.

Isaac: None of that's on the video, I don't think.

Matt: Oh, my bad. You even told me that.

Isaac: [laughs] Yeah, you guys, that was exclusive. That's not going to be online, on the video.

Matt: You had to be here.

Isaac: Yeah, seriously.

So this is really the only bit that was kind of awkward, like wonky, crufty stuff that needed to be added in. And it's not even really necessary. I mean, we could've done that in some other client-side file. But basically, we just have this one thing that says: if the expert's objective is not defined, then define it, and then hang it off of exports.board.

And now on the server-side, if I look at some server-side code that calls that board class… I'm hoping that I don't have any horrible awful cursing in my comments here. [laughs] Alright, so here I have "board equals require tictactoe/board.board". That's a basic bit of how it works. It was very straightforward to understand. This is kind of what the whole CommonJS module's approach to writing code is: you define an exports object, and when you require your module, it returns that exports object.

A Jack application is basically a collection of these app functions. You can see here, I've got a URL map file that just maps /game and /move to various different things. I call the Jack.URL map plugin to turn that into an application. Then you can do some very clever and tricky things with small pieces of application logic like this, where you have a function that returns an application. Basically, what that's doing is, this is called a middle-ware, and it'll do something interesting to the resulting output. Then at the end, you just set exports.app to your app, and there it goes.

I'm trying to find a smaller… What an app looks like. Basically, an app kind of looks like this. You've got a function that takes an environment variable, an environment hash. You do something to it, and then you return a status headers and body type object, and Jack spits that back out to the browser.

Alright, CouchDB has started. Time to relax. Anybody here used CouchDB before? Have you used CouchDB before, anybody?

[Audience member makes inaudible comment]

Isaac: OK, a couple of people. Basically, CouchDB is just document store. Each of our games here is expressed as a document, where we have the history, and who's sitting at O, and who's sitting at X, and who the winner is, and whose turn it is.

That's a bug right there, I fixed that already. This is an old game.

You also have these things called views, so we can figure out which would the next seat be if I were to sit down at a new game. Those are expressed as small JavaScript functions as well. Really, what's very exciting about all of this is being able to use JavaScript as every point in your application stack. You can make the argument that some other language is better, but either you've got a compilation step that turns your client-side code into JavaScript, or you've got JavaScript end to end. I like JavaScript already anyway, so that was an easy choice. So does he.

And yeah, that's it. Any questions?

[applause]

Yes?

Audience member 4: [xx] the days of Netscape. So what's the performance [xx]?

Isaac: Well, the performance is not great.

[laughter]

Isaac: For Tic-Tac-Toe, it works really well, as you saw. It was pretty snappy. That's also running a local host, and it's [inaudible].

[laughter]

Isaac: But the nice thing about this, so you know seriously, part of the reason why it's not great performance is it's running on top of Rhino, and it's running an interpretive mode on Rhino. It's not recompiling, right? Yeah, so it's interpreting on Rhino, which is slow.

There is work being done to build a Narwhal interpreter layer for V8, which is much faster. The whole point of Narwhal, though, is once those V8 packers are done with the V8 Narwhal plug in, I can take my program and just put it on that and it'll keep working the same.

Yes?

Audience member 5: I just wanted to point out that there's another server-side library called Node.JS, which is written on top of V8. It doesn't even come close to conforming to the normal spec, but it does give us an idea of benchmarks, and it outperforms a lot of other competitors. Like, it outperforms Twisted, and Python, and Event Machine, and Ruby. It even outperforms some apps that are [xx], like Lighty and CGI calling it a C [?], which is pretty good. You can get concurrency up to tens of thousands of people. So it's not a limitation of JavaScript, it's just, as you said, you were running a random.

Isaac: Yeah. So the comment was Node.JS is wicked fast – I can definitely assert that as well. It seems to me like CommonJS is kind of the direction that has the most momentum on it. Node.JS is pretty exciting, and if you do need something that's production ready, and scales to a bajillion users, Node.JS is a pretty good option.

Yes?

Audience member 6: How does this compare to – I can't recall the name, but Haxe?

Isaac: HaXe? Usually the X is capitalized.

Audience member 6: Yeah, something like that.

Isaac: The difference is, with haXe you write in the haXe language, and then haXe can be compiled to client-side JavaScript, or I think there's like a server-side interpreter.

Audience member 6: But it's still JavaScript?

Isaac: It's not exactly JavaScript. It's JavaScript-ish.

Audience member 6: I mean, it's much more strict version of JavaScript for me.

Isaac: It's closer to the… Last time I looked at haXe, it's been awhile, but when I did look at it it was closer to the Harmony, or what was called ECMAScript before. And I think ActionScript 3 is similar to that as well. HaXe you can actually compile to ActionScript or PHP or haXe server-side script, or server-side JS. It's not a bad option, but it puts you in a [xx], writing your app and Ruby, and then compiling the client-side part. I'm not an haXe expert, and I don't want to come across as one, so take everything I'm saying with a grain of salt.

Audience member 6: OK, good.

Isaac: Anybody else? Any questions for Matt?

[laughter]

Audience member 5: Where'd you get that hat?

[laughter]

Matt: My girlfriend.

Audience member 4: We want company names.

Isaac: He got the hat from his girlfriend.

Matt: Alright, thanks guys.