Welcome, guest Sign In

Video: Douglas Crockford — The State and Future of JavaScript

Video published 2009-10-29.


Douglas Crockford: This is a basically a repeat of last night's talk. We're at an important moment now with JavaScript, and I think it's a good time to look at how we got to where we are and where we're going to go next, and how we're going to get there. I'm talking about JavaScript, but I really should be talking about ECMAScript. JavaScript is a trademark that's owned by Sun Microsystems, or possibly by Oracle — I'm not too sure what the status is today. The thing that describes what this language really is, the thing that's common that we all own and that we all share, is ECMAScript. We really should be calling it ECMAScript, which I find really difficult because it's such an awful name. But that's really what it's called, and that's what I'm going to be talking about tonight.

Last night, Brendan compared our journey to the 'Fellowship of the Ring' — you know, the movie — which was a very good movie and a very good metaphor. But from my experience, a better movie metaphor for what we did was '12 Angry Men'.


Douglas: '12 Angry Men' is a Sydney Lumet movie, brilliant murder mystery. You don't see the murder, you don't see the trial, you see the jurors and the jury room; the whole movie takes place in one room. It's an obvious open and shut case, 11 jurors are prepared to vote guilty, but there's one juror — number 7 there, Henry Fonda — who says 'I'm not sure'. Over the course of several angry hours, he manages to turn everybody's mind around and they acquit the guy. Brilliant movie. There's juror 3, played by Lee J. Cobb — that would be Brendan.


Douglas: The jury room is which we are working is called ECMA TC39, Technical Committee 39. At the time that I joined TC39 representing Yahoo!, there were no members of the committee who actually used the language. They all knew the language well because they were writing compilers and interpreters and doing all the runtime stuff, but none of them actually worked with the language and understood what the problems of the language were, and how to make them better. They all had ideas about what would make the language more fun to work on and things of that sort, but they really didn't have much sensitivity to the problems that you experience.

At that time it was a fairly small group. There were only four other active members: Adobe, Mozilla, Opera, and Microsoft. The Microsoft rep wasn't all that engaged either. Opera, at that time, wasn't actually a member of ECMA, so their participation there was in violation of the ECMA rules. In fact, the ECMA rules were not being followed very strictly at all at that time.

The committee had been working for several years pursuing a design that had begun at Netscape in 1999. That project had been abandoned, but was restarted again because of the big interest in Ajax. I came in skeptical about the whole thing. I thought that their design was way too big, way too complicated, and that it didn't solve any of our problems — particularly the security problems that we have which plague us with cross-site scripting attacks. Those attacks are enabled by the design of JavaScript and were standing in the way of our ability to mash-up, and mash-ups are going to be the important way of going forward. That got in the way of our allowing third party code on our pages, which is an important part of our strategy going forward. They weren't looking at any of those problems, and they were doing a lot of other things that I thought were going to undermine the stability of the language. I thought that they were making some really bad choices and bad trade-offs, and I wanted them to stop.

There was a strong interest in the web community to make the language suck less — apparently there's some people who think that ECMAScript, or JavaScript, sucks too much. I don't know if anybody here does. I was concerned that it was going to suck way more if they followed through on the things that they were planning on doing. We've seen lots of cases in web standards where good intentions have proven to be remarkably ineffective, and often counter-productive.

The committee did not want to hear any of that. I went to the first meeting with a speech prepared in which I outlined all that stuff, and Brendan kept interrupting me — you know, we're not going to do that, you're too late. They refused to argue about the necessity of individual features. They'd say, that doesn't solve a problem that we have, we can write code without it, there's nothing we can do with that that we can't do now. They weren't interested in those sorts of arguments, and they weren't interested in talking about the system as a whole, with the interaction between all the things. From their perspective, they'd been working on this for many years, they were almost done and they figured they had less than a year to go, so I was simply too late to have any input in this process. It was a done deal. Maybe I could have some influence on the next version, but this one was practically in the can. That was really discouraging, but I thought that was the state of things.

But it turned out that the Microsoft member had similar concerns — he also thought the language was getting too big and was out of control. He had not said anything prior to my joining the group because he was concerned that, if Microsoft tried to get in the way of this thing, it would be accused of anti-competitive behavior. Based on Microsoft's past performance, there were maybe some good reasons for them to be concerned about that — and it turned out, those concerns were well-founded, because that happened. But I convinced him that Microsoft should do the right thing, and to his credit, he decided that he should, and was able to convince Microsoft that it should. So Microsoft changed their position on ES4.

We had a big showdown meeting in Redmond. Everybody knew that this was going to come up — it was the best attended meeting in TC39 history. Mozilla brought lots of extra people, Adobe brought lots of extra people, and Microsoft filled the room with a lot of their people. I was still the only Yahoo! rep, I couldn't get anybody from Yahoo! to come with me on these things.


Douglas: There were a few occasions when I couldn't make a meeting so I'd send one of my associates from the Ajax strike force to represent me, and they'd come back and say wow, it was really interesting. On another occasion if I'd ask them if they'd go again and they'd say no.


Douglas: Will not do that again. Julian's here — you went once, right? Wouldn't go twice.

Some of the people at Microsoft wanted to play hardball on this thing, they wanted to start setting up paper trails, beginning grievance procedures, wanting to do these extra legal things. I didn't want any part of that. My disagreement with ES4 was strictly technical and I wanted to keep it strictly technical; I didn't want to make it nastier than it had to be. I just wanted to try to figure out what the right thing to do was, so I managed to moderate it a little bit. But Microsoft still took an extreme position, saying that they refused to accept any part of ES4. So the thing got polarized, but I think it was polarized as a consequence of the ES4 team refusing to consider any other opinions. At that moment the committee was not in consensus, which was a bad thing because a standards group needs to be in consensus. A standard should not be controversial.

I proposed an ES3.1 project, a minimalist alternative to ES4, something which solved specific problems that we had with the language but tried to be small in the approach about doing that, not adding new syntax, not trying to radically transform the language into something else, just making it work better for us.

Then, unfortunately, the argument went public. The squabbling that had been going on in the committee was now going on in the blogs, terrible accusations against people — it was all pretty nasty stuff. Partly that was because ES4 was positioned as the official new standard, that it was inevitable that it was the approved thing and that we were interfering in a natural process, which was not true at all. There were a lot of accusations, particularly against Microsoft, but a few against Yahoo! — for example, that I was conspiring with Microsoft in order to protect IE's dominance. If any of you know my opinions about IE, you know that's absolutely not the case.

I had one option that Henry Fonda didn't have: I could bring more jurors to the room. I went around talking to other companies asking: 'what side are you going to be on?' 'I don't know'. 'In any case, you should come to the meetings. We need to get more voices on this and figure out what the right thing is. We need to get to the middle, because this extremism is not working on either side'. There were many companies that chose to sit on the sideline and told me privately that they were glad that I was in there fighting for them, but they didn't want to get involved in the fight themselves because they didn't want to get smeared the way that Microsoft was getting smeared.'

So we continued to agree to disagree; that was about all we could agree on. We informally split into two committees but continued to have joint meetings, and they were really uncomfortable, and really unproductive. Eventually we were able to get other companies to finally join, so we got IBM, Google, Apple, Opera, Dojo, and Company 100, and that was good. Having more voices in the process definitely helped.

All this time ECMA was really worried, because ECMAScript is the most successful product at ECMA. If ECMAScript was going to die, that was going to be a really bad thing for them. We were pursuing two proposals, but there was no way that ECMA could tolerate that — we could not send two proposals to a General Assembly for one product and ask them to approve both. That was unthinkable, there was no way you could do that. ECMA demands consensus, and properly so. So the ECMA Secretary General and the ECMA President both began attending our meetings, which was really nice because with that adult supervision there, everybody started behaving more. Civility was restored and we were able to agree on more stuff.

For example, one thing we were able to agree on was a subset relationship, so that ES3.1 should be a proper subset of ES4. But we couldn't agree on what that meant. The ES4 group thought that that meant that ES3 could only pick from features in the feature set of ES4, that it was such a huge feature set that anything we wanted would have to be there already. From our perspective, the biggest thing we wanted to fix was security and that was not something that they were working on. So we were going to have to do things that they hadn't considered, which meant that they had to adopt anything that we adopted. So there was this power struggle going on: who gets to decide what's in or what's out? That was difficult.

Ultimately, ES4 was not converging. It had slipped a year, per year, since the project had started in 1999. It was not intended to be a ten year project, it was intended to be a one year project, but they were never able to get it finished because it kept growing and its complexity was always increasing. For example, ES4 was supposed to be finished well before Adobe AIR because it was going to be the language of Adobe AIR. Adobe AIR shipped without it, and at that time ES4 was predicted to be about a year away. It continued that way until it died. They recognized that they had gone way over their complexity budget, so they started jettisoning features. Finally in doing that, in reviewing the feature set that they had added, they started to realize that some of the things that they were proposing were, in fact, unsound. So they started pulling some stuff out — some of the stuff that we'd been complaining about from the beginning — and eventually they just gave up. Then Harmony happened and everything got better.

While that was going on, we were able to finish ES3.1 because it was a much simpler specification, much less ambitious, so it was complete-able. It is now a candidate to become the fifth edition of ECMAScript. We chose not to call it the fourth edition because there's an understanding in the world about what ES4 is, and what we were doing was nothing like that. So to avoid that confusion we skipped to the next whole number, so it's ES5. That goes before the General Assembly in December and I'm optimistic that it will be approved then. If so, it will become the next ECMAScript standard.

One company has stated its intention to vote against it, and that company is IBM. The issue is decimal arithmetic. Why would IBM care about decimal arithmetic? Well, it turns out that the most reported bug in ECMAScript is some variation on 0.1 + 0.2 is not exactly equal to 0.3. This is the most repeated bug, and since it's the biggest bug in the language you'd think it's something we'd want to fix, and it is. Where does the bug come from? It's a consequence of IEEE 754, the floating point number standard. 754 did a really good thing: it created a common representation for floating point numbers across all computer architectures. At the time that it was created, every computer manufacturer had several architectures, and each of those architectures had a completely different floating point number format with a different number of bits, different radix, different exponent size, different range, different sticky bits, different rounding techniques, different endnesses. All sorts of differences. So it was really difficult to write programs that would be portable across all of those computers. Today is doesn't much matter because it's all Intel, so who cares…


Douglas: But at that time it mattered.

This floating point works really well for applications which have very large numbers and very small numbers, and needs them to interact. That includes applications in astronomy, chemistry, and physics; they do really well on floating point. Unfortunately, floating point is poorly suited to most other applications, including virtually everything we do. The reason for that is that floating point cannot represent, exactly, decimal fractions. It can only represent binary fractions. It can't represent fractions that are based on a power of 10, so it can handle dollars and quarters, but it accumulates errors on pennies, nickels, and dimes. Anytime you type in a number in your program that's got a decimal point in it, you already have an error in your program because the value that gets put into your program is not the value that you typed in.

Given the applications of ECMAScript, IEEE 754 turns out to be a really bad choice. But you can't single out ECMAScript for that, because virtually every programming language designed in the last 20 years made the same mistake — they all are using 754, sometimes inappropriately. It's another instance of just because something is a standard doesn't mean it's the right choice for every application.

In order to fix that, IBM has gotten behind a thing called IEEE 754r, which is a revision to 754 which adds a decimal mode to the floating point format, so you can now have the accurate representation of decimals. IBM tried to get this into the third edition, and I don't know what the story was for why it didn't get in — maybe the proposal came too late, or whatever — but at that time they were putting out a new edition every year, so the expectation was that they would just put it in the next one. IBM's been waiting a long time for the next one, they think they're entitled to it, and they didn't get it in. It was rejected by ES4 and by ES3.1 — it was one of the few things that we could agree on. We all agreed that the IBM proposal should not go in.

Why was that? Well, this is the original 754. This is a float, or a double — it's a 64 bit number. This is the format that's used in JavaScript. We've got a 52 bit mantissa, or significand, which determines what digits you're going to get. We have an 11 bit exponent which has a sign, but the sign is biased. The reason for that is so that when the number is normalized, you can use an integer comparer and you don't have to do a floating point subtract to figure out if two things are equal or greater than each other. That's only of value if integer arithmetic on your machine is way, way faster than floating point arithmetic. But the problem with this is it can't do decimal numbers.

This is what IBM is proposing to replace it with: a 128 bit number — so now all your numbers are twice as big — and the mantissa is now broken into 11 10 bit fields, each containing a number between 0 and 999. If you want to add 1 to a number, say, you go to the first field, unpack it, add 1 to it, ask if it's bigger than 1000. If it is, you divide it by 1000, the remainder goes back into the slot, and you take the 1, and then take the next field and add it to that, and look to see if that overflowed, and so on, all the way up. As you can imagine, it's really slow. And it's even more complicated than that because the exponent field has more significant bits in it, and the encoding is kind of complicated. So it's a really cumbersome format, but it does produce results that are correct.

Although some applications are dependent on the incorrect results that they get. I mean, that's just how the web works. So if the browser's doing things wrong but your program works right, there's a good chance that if we fix the browser your program is going to break. That's one of the really big problems that we're sensitive to in the standard. We'll ask: if we fix this, will it repair more programs than it'll break? But that's really not good enough; we really don't want to break anything, so we can't just willy-nilly replace the old format with the new one.

So we looked at some ways of fixing this without breaking the old one. The first thing was, let's have a 'use decimal' mode. If we say 'use decimal' , that means we know we're going to get different results than we get with the binary format, and we want that, that would be good. That was rejected by the committee because the format is just so slow — it's literally hundreds of times slower than the current format. Right now we're in a race for who can get the fastest JavaScript engine, and none of the browser makers want to accept this hit. They're working too hard to make it go fast to want to do this. Now, it turns out that most of the arithmetic we do in JavaScript is integer, so we could have an integer optimization — if it's integers, do it the smart way, and if it's not then we do the decimal stuff. In doing that, I think most applications probably wouldn't feel the hit, but the browser makers have just decided no, don't want it. They want to wait until it gets into hardware, and even then they probably don't want to do it.

We considered adding a decimal library. You'd say new decimal, and give it a string, and the string would have some digits and maybe a decimal point in it. It would return an object and the object would have add methods and multiply methods and stuff like that. We have an implementation of it, but it's really ugly. I can't imagine people actually using it, or using it well. And it doesn't actually solve the problem, because most people won't want to do that; there's no reason why they should, so they're still getting incorrect results. So it's not really a solution.

We also considered adding another number type to the language, but that is doing violence to the language, because right now we've got a language which has a lot of simplicity due to the fact that there's just one number type, and that's one of the things that allows beginners to use this language well — you don't have to ask, is it an int? What size? It's just a number that works. So by avoiding having multiple kinds of numbers that interact strangely, the language is actually cleaner. Ultimately, we might want to create a scheme that would allow you to have multiple number types, and Ajax library makers could create them themselves — that's something Brendan was talking about — that appears to be very desirable. That's hard to do without damaging the language, so that's something we want to do very carefully, but it's something we're considering. TC39 is looking seriously at decimal for a later edition, but we just couldn't figure out how to get it into the fifth edition.

Now, through all of this there was no discussion about a better decimal format, one that doesn't have the big problems that 754 has. IBM's stance was 754 or nothing — well, not even nothing, just 754 period — but there are other approaches. I spent some time thinking about how else we could do floating point, and here's one proposal. I call it 9E6. The numbers are 64 bits scaled by 9 million, so if you want to turn an integer into one of these numbers, you multiply it by 9 million. The advantages to this are that addition and subtraction, which are what we mostly do with numbers in this language, run at integer speeds, so they go really, really fast. You get 6 exact decimal digits. For most of our applications 2 would be nice, 6 would be really good. And you get a 7th repeating digit, so that you can represent, exactly, any number which could be represented by an integer divided by 9 million. Which includes 1/3, for example. You could write 1/3 x 3 and you'll get exactly 1.


Douglas: That should always have worked, right? Well, it would work with this number system. And fractions of 60, and 360, and other things that we use a lot will work in this system, so that's really attractive.

There are disadvantages. 6 or 7 decimal digits are not enough for some applications. You might want more precision than that. Also, the maximum number you can get in this system is about a trillion, which is maybe about one million more than we need for most of the stuff that we do. But there are some applications that need more headroom than that — for example, if you have an application that wants to be talking about, say, the federal budget.


Douglas: A trillion's not going to get you there. So here's another suggestion: I call it DEC64. It's another 64 bit format. The mantissa is binary now, it's a signed 56 bit number, and the exponent is a signed byte, it's 8 bits long and it's sitting in the low end of the word. On Intel architectures you can get at that guy for free, that's just the way the instruction set works. You can get at the mantissa by doing an 8 bit shift, so that's really cheap too, and then the arithmetic with it is really simple. Again, for integers, you can add at integer speed, so it's really convenient.

Advantages are that it's way, way, hundreds of times faster 754r, and it's easy to implement in software. Disadvantages are that it's going to be a little bit slower than 96, but not much, and the range only gets you to 10 with 128 zeros. The current floating point format gets you 308 zeros. Now, I don't know how many zeros we need. I'm thinking 100 or so ought to be plenty, so this ought to be pretty good. Generally, if you're getting into the 300s, you've got a bug; there's no reasonable application that's getting you that high.

The reason we have the disagreement with IBM comes down to this: TC39's goal is to repair the language, not to add 754r. IBM's goal is to add 754r, not to repair the language. IBM has stated that they will vote against any language standard that does not include 754r, and that includes the ECMAScript standard.

So here's my appeal to IBM. It is irresponsible to inflict damage on a language and its community to solve an unrelated chicken and egg problem. If your nay vote fails then you have done nothing except to show contempt for the web development community, and if your nay vote succeeds then you will cause significant damage to the web and the open standards movement. So IBM, please vote yes. If any of you listening work for IBM, or are friends of IBM, please tell the company to get on the right side of this.


Douglas: Thank you.

I think they will fail; I think the other members of the General Assembly will not follow IBM, but we won't know until the vote actually happens.

The Fifth Edition defines two languages: the default language, which is just what you get if you don't do anything, and the strict language, which you get if you opt in to the strict mode. You should use the strict mode for reliability, or you should use the Third Edition for compatibility, so that you'll run on the old browsers. There's no excuse to write in the default language, and I highly recommend that you not use that language. It will be abandoned soon. It will not be supported going into future editions of the language.

The working title of the next edition is Harmony. We're not calling it ES6 because we want to avoid names which suggest that something is inevitable, or pre-approved, or has some status which it has not earned yet, such as HTML 5 or something like that.


Douglas: Until something is formally approved it's just a proposal, and at this point Harmony is not even a proposal, it's just an idea. Harmony will be built on top of the strict language of ES5, which is really, really good. Harmony will probably have incompatible syntax, which means that programs written for Harmony will fail in all existing browsers. That will be the opt in, that's the thing that gives us permission to break some of the features of the language, because it's already broken. Hopefully the IE6 problem will be solved before we get here. If not — if IE 6 still has some significant market share — then this will be a real short trip.

There's a lot of design pressure on this language. For example, there's a lot of pressure on us to suck less, which turns out to be really hard to do. There are so many wannabe ninjas out there who are taking the bad parts and the awful parts and showing how they can do all these clever things with them, and doing that means that we can't remove those awful features. Now, you could argue that they deserve to have their programs broken, or their legs broken…


Douglas: …but we're trying really hard not to break things. We're constrained in how much we can fix this language, given the way that it's used. There's a lot of pressure to make it more like the other languages, and that's something that we're getting better at resisting. If you want to be writing in Java, you should be writing in Java, you shouldn't be asking us to turn ECMAScript into Java for you. That really doesn't make sense.

There's some pressure to make it more expressive, so that you can get more done with less. Some people think that means getting more done with less typing, but typing is not where we spend most of our time so reducing keystrokes I don't think is a good win. But coming up with idioms which allow us to express powerful ideas more simply and more elegantly, we're very much looking at that kind of stuff.

There's pressure to make it a better compilation target. Now, this is a big surprise. Everybody thought that the Java VM was going to be the VM of the internet, but it turns out that JavaScript language is the VM of the internet. People are writing in Java, and Python, and lots of other languages, and then translating it into JavaScript because JavaScript, for all of its security problems, actually has a much better security model than everybody else. The CLR has a terrible security policy, Java has a terrible security policy — JavaScript is better than everybody else.

Audience member 1: Woo!


Douglas: Yeah! So that is where the world is starting to go to run their programs. If you're writing a compiler and you're trying to translate some language into JavaScript and your language has control structures that don't map onto JavaScript's control structures, then it gets hard to write good code. They've been asking us to do things like add 'goto' to the language. Now, most of you are too young to remember but the computer industry spend 20 years trying to figure out if 'goto' was a good thing or not, and we've finally figured out that it's not and managed to eliminate it. I don't want to be bringing it back, I think that would be a terrible thing. Even if it made it easier for them, the ninjas will jump all over it and go 'oh look, I think I can make my program go faster'. Maybe they will or maybe they won't, but they'll be writing really bad code and it'll be our fault, so we don't want to do that.

There's pressure to make the language more secure, and this is something that really resonates with me. We're already really good but we're not good enough. This year I was holding conferences on secure ECMAScript, trying to figure out what we have to do to the language in order to try and make it stronger. We reviewed Caja and AdSafe and FBJS and Microsoft Sandbox and Dojo Secure and a lot of other things, and tried to figure out what we can learn from that and put into the new language. I was thinking that we would probably have to fork ECMAScript, so we'd have the web language and the secure language. But since we've harmonized we've figured out that we don't have to do that, so we're actually going to do the secure language as the core language of the next one. Basically, we'll have Caja's strength native in the language itself, which would be a wonderful thing. We still have to fix the DOM, because the DOM is still crap, but at least there's a good chance we can fix the language now. The DOM, incidentally, is going in exactly the opposite direction of what we need in order to make it safe for all of us, so that's another problem.

Then we need to make the language better at math. We've got the decimal problem that we've been talking about. There's some people who think they want to be able to add general number types. How many of you are really wanting to have complex numbers in the language? Anybody? One. Two. Not a lot of demand for that, but the language architects really like that crap, so they want to get that in there. There's also a lot of demand to get big ints into the language. Big ints allow you to write numbers that can't be infinitely big, but can get really, really, really big. They're used primarily in cryptography. It's because they're so useful in cryptography that I don't want to put them in the language. One of the things we've learned is that if we allow people to roll their own crypto-routines, they invariably get them wrong. It's hard to write that stuff well. You want it vetted and you want it done by people who really know what they're doing, and it's unlikely any of that's going to happen in JavaScript. So I think putting big integers into the language will be another invitation for bad behavior.

There's a lot of interest in correcting the block scope problem. JavaScript has syntax which says it's got block scope, but in fact it doesn't, and that is an invitation for errors. Not having block scope is not actually a problem, but having syntax that says you do and then not delivering on that, that is a problem. We think we're going to add a 'let' statement and a 'const' statement, which are like 'var' except that they're block scoped, and when we do that you shouldn't use var anymore.

We'll have better support for functions with a variable number of parameters. Right now we do that with the arguments thing, and that's really awful, so we've got some new syntax which we think will do that in a much more pleasant way. We're looking at syntactic sugar, which will allow us to do something that looks more like classes, so we'll have more expressive support for writing reliable structures. We've talked about doing it with Macros, but the committee is skeptical about Macros.

We're looking at doing it in Functions using classes as closure patterns, but there's potentially a problem with this, which comes out of maybe a misreading of R.D. Tennents' 'The Principles of Correspondence' — a really good book from 1981, out of print but the principles are still relevant, and in particular the principle of correspondence. Here we have two pieces of JavaScript. In one, X and Y are variables, and in the other, X and Y are parameters, but these two programs do exactly the same thing. So the correspondence principle tells us that there's a correspondence between variables and parameters, so you can express one as the other, so that's an interesting thing. There are some people who believe that means that you should be able to take any arbitrary chunk of code, any expression, or statement, or block of statements, and wrap that in an immediate function, and not change the meaning of the program. If you can do that then some interesting things become possible.

But there's a problem: disruptive statements like break, return, throw, and implied parameters like this and argument cannot be put into a function because they're local to a function, so we don't have correspondence at that level. Now, Tennent doesn't say that you need correspondence at that level, but there are some people who read his book who think that he said you do. What Tennent actually said about programming language design is really interesting. He said: "many existing languages manage to combine countless features into a jumble that is neither easy to implement, nor a pleasure to use.” He might have been talking about ES4. He also said: "side effects are often confusing to program readers because they are unexpected. The familiar expressions of conventional arithmetic and algebra do not have side effects.” Tenant might be saying that we should not have put getters and setters into ES5, and he might be right.

It's been ten long years since the last edition of ECMAScript, and that's too long. We shouldn't have to wait that long for revisions. I think it was actually good that we waited some of those years. I think five of those years were really well spent. In 2000, everything that we needed to do Ajax was in the field and ready to go, but Ajax didn't happen until 2005. The reason was that the web wasn't stable; there were too many versions of too many browsers, and only two browser makers at that time. There were just too many bugs in it, so it was not stable enough that you could write applications. It took five years of nothing happening — specifically of Microsoft doing nothing — to make Ajax feasible. There are a lot of web developers who are angry at Microsoft for having done nothing for so long, but my view is that those five years of doing nothing were the thing Microsoft's ever done.


Douglas: Kudos Microsoft. Maybe ten years was too long, but five years, that was about right. Hopefully it won't take that long for the next one; we're hoping it'll be in two and a half years, but these things always take longer than we think so it'll probably be more. But hopefully not ten again.

There are some lessons that we can learn from the standards process that I think are important in going forward. First, we need a clear separation between research and standard setting. I believe it's possible for someone to be a researcher and working in a standards body, but you shouldn't be doing both at the same time. There should be nothing reckless or dangerous or speculative in a standard; it's all got to be absolutely easy to understand and easy to predict stuff. If you're doing science experiments in a standard, you're doing it way wrong. The standard is the last place in the world that you want to see innovation.

Don't promise what you can't deliver. For example, in ES4 they were proposing a type system and claiming that it was going to make the programs go faster. A lot of support they got in the community was: yeah, we want JavaScript to go faster, so we want that type system. It turned out that the type system caused programs to go slower. They had announced the benefits of a feature before they had actually done the work, and figured out if the feature was going to work.

Also, the standard itself is speculative. I anticipate that the fifth edition is going to pass in December, but I don't know yet. It could fail. It could fail because of IBM, it could fail because someone discovers there's a terrible stupid thing that we hadn't considered and is a disaster, so we shouldn't go forward. I don't think there's anything like that in there, but if there is, I really hope they find it in the next month, because I don't want the standard to go and get approved if it's got those sorts of problems in it. So until it actually gets voted on, it's just a proposal.

A change to a widely used standard is an act of violence. When ES4 started, ES3 was not a popular language — there wasn't a lot of use of JavaScript, nobody liked it. They thought they had a lot of license to make drastic changes to it in order to try to make it more popular. Today, JavaScript is the most popular programming language in the world — who would have thought that would happen? But it happened, and there are way too many people using it now so we can't afford to make drastic changes to the language. Any changes that we make to it had damned better be worth it. It's sort of like surgery: you really don't want to be cutting people open because you're wounding them, you're possibly killing them, and that's a bad thing to do to people generally. But sometimes there's a benefit to doing that which makes it worth the risk and the pain. We need to think about standard setting in exactly the same way.

If you have a great idea, don't tell it to a standards body. They are the last people in the world who should hear about it. What you should do instead is implement it and show it to the world, and if the world likes it then the world will say yeah, that should be a standard. I've seen too many cases where people try to do this in the reverse order, and you don't want to do it that way. Prove it first, prove the need, and then we should put it in the standard.

Standards are hard. They are really hard work and they require a huge amount of precision in order to scribe the stuff in a way which is accurate and understandable. It's really difficult, and fortunately we've got amazing people helping us on TS39 — Allen Wirfs-Brock of Microsoft was the editor on the thing, and he did amazing work on cleaning up the document and expressing the algorithms. He did a huge amount of work, really, really good stuff. We also have these amazing human computers, these Mentats, who were able to read that stuff, run it in their heads, and then figure out where the problems were. I can't do that, it's just way too hard, but they can, and we are really lucky to have them working with us on this. So those are really valuable skills to have in a standards body.

You can't please everybody. This should be obvious, but all too often it isn't. You've got expansionists who have features that they need to get in and feel that any cost is justified in order to accomplish that. Then you've got minimalists like me, who say no, you can't put anything in unless you prove its value. Ultimately we need consensus, so we have to find a way in the middle where both of those points of views can be satisfied. And ultimately that's not possible, so somehow we have to come to a consensus anyway; that's part of what's difficult about this process, but it's a really important aspect of this process.

You need to check your expectations. We have at least two companies who feel that they were betrayed — I've heard them both use the word 'betrayal' — in their feelings about how they were treated in the ES4 process. Adobe and IBM both felt that they were entitled to get particular outcomes from this process, they didn't get them, and they feel betrayed. I don't think they were betrayed. I can understand why they're disappointed, but they should not have felt any sense of entitlement that they were going to be able to get everything, or anything, that they wanted. That's just not what a standards process should do.

You also need to check your motives. A lot of people come to standards for wrong reasons — for example, to show off. You know, 'I'm a really smart guy, I thought of this stuff, I can get it in the standard and everyone will see how smart I am'. That's not actually a good way to become famous. I don't know if anybody can point to any feature in a standard and say yeah, I know the guy who did that. Fame doesn't come that way. We also see people on power trips — you know, 'if I can get this in, Microsoft will have to implement it and they will bend to my will!' That's not why we should be doing this stuff either.

The worst cases of this are people who are doing this for evil purposes — for example, there have been a number of cases where features have been proposed for standards and accepted, and then it turned out that the proposer owned a patent and they go now go after everybody who was using the standard, which is criminal, but it has happened. So standards bodies now have rules about intellectual property, which are complicated and ultimately not effective. You can have a very nice open standard which unintentionally infringes on something that's owned by someone who's not a member of the standards body — for example, the GIF format was snagged by a Unisys patent some years ago, so everybody who was trying to display GIFs had to pay tribute to Unisys. That was an awful thing. So one of the lessons is that patents and open systems are not compatible. I think the solution to that incompatibility is to close the Patent Office.


Douglas: The success of an enterprise should depend on the quality of its goods and services and its ability to execute efficiently, and not on a capricious Government office. The patent system made sense in the 18th and 19th century, it really did. They helped us to become a very innovative country, made us very good in manufacturing and process, but it has long outlived its usefulness and we need to change it. Patents can be attractive in that if you can get a patent and then snag a competitor with it, you can get some free money, and that's attractive. But you can get snagged on someone else's patent, and then you have to pay it back. There's a good chance that both of you are going to get snagged by a patent troll. So it's gambling, basically. It's a casino, and you hope that at the end of the day you win, but in the long term you lose. Nobody wins except the lawyers.

Another thing about standards is getting a lot of people to agree on anything- particularly something good — is a wonderful and valuable thing, and that's a great thing to be a part of. That brings us to ECMAScript: The Fifth Edition, the best web standard in the history of the world. There's still lots of room for improvement, so I'm optimistic about the future. I expect the future editions will be even better — there's a whole lot more to get right that isn't right yet, but we're trending in the right direction. Even if they suck a little bit.

That's all I have for you tonight. Thank you, and good night.


Douglas: I think we have time for a few questions. Yeah?

Audience member 2: Has there been any work or thought put into… you obviously need to keep events in the language, but in light of the fact that every condition has problems and implementations don't get updated as frequently, but it's kind of desirable to push new features out as quickly as possible, how do you reconcile that in the environment that we're working in? Is there any work or thought you've put into that at the standards level?

Douglas: Have we ever thought about the problems of how we advance the standards in the presence of bugs and implementations and lack of upgrading, and all of those sorts of things? Yeah, we think about it all the time. For a while there was an idea that versioning could work, but that turns out to impose way too big a burden on you, the developer. That was tried back in the Netscape era, failed badly, and we don't want to do that again.

We're now trying to figure out how to move the language in ways where we can be forward and backward compatible. One of the big goals in ES3.1 was to have no new syntax, because new syntax means you get a syntax error on the page if it runs on a current browser, and if we get a syntax error we don't get paid, so that's unacceptable. We tried to provide all of the functionality that you get with ES5 in a way that a program can sense if it's present or not, and in some cases can add it.

The Ajax libraries have proved to be really good at this, and I'll give you an example. IE 5 did not have 'try/catch', so for all of the years that IE 5 was out there — and it still had a significant share until about 2005 — you couldn't make any mistakes. There's no way you could catch anything, so you just had to be really, really careful. In 2005 their market share fell to the point where you could finally use 'try/catch', so we had to wait five years to use 'try/catch'. That was awful.

IE 5 also introduced some array method — so, splice, push, pop… No, IE 5 did not have those, it was lacking those., all the other browsers had them. But we could deal with that. An Ajax library could say, do we have 'array.splice'? No? OK, here it is. We'll write it in JavaScript — it won't be as fast, but at least our program won't fail. If we get worst performance in IE 5 we actually like that, because we hope maybe people will want to upgrade, but at least they won't fail. In ES3.1 we wanted to empower the Ajax library developers so they'd have more control over their environment and be able to adapt better to the shift in things. That's important as long as IE 6 is still in the market.

Now, in Harmony we're looking at adding syntax which will break again, so it's going to be more like 'try/catch' in IE 5, and I'm just hoping that in IE 6 that problem gets solved before that language comes out. If it doesn't, if IE 6 is still dominant or significant, then Harmony's going to fail, because people just won't be able to use it.

Audience member 2: With things like 'let' and 'const', how much of that is already implemented now that might not potentially be able to handle it when it comes out?

Douglas: 'Let' and 'const' are already available in Firefox but they're not available in IE 6, so as long as you care about IE 6 you cannot use them.

Audience Member 3: So 'var' is going to be around, right? Especially if 'let' is going to grade. [?]

Douglas: 'Var' is going to be around for a long time. At the time that we add 'let' and 'const', I think at that point we should deprecate 'var' and warn developers not to use that anymore. In fact, we might say you have to use one or the other; you can't use 'var' in a program in which you're also using 'let'. Because eventually we want to remove it from the language, but we have to allow it during the transition.

Audience member 3: Yeah, I see a lot of people misuse the scoping [xx].

Douglas: Yeah. Anybody else?

Audience member 4: You mentioned HTML 5 — tell us a little bit about your position on that.

Douglas: I think it is way too big. It has already declared itself as a standard, to the extent that they've criticized Microsoft for being incomplete in their implementation of that standard. It has not gone before W3C yet, it has not been approved by anybody, there has never been any kind of vote or any kind of approval, but they're just insisting that it is the inevitable winner and they'll probably get away with it because W3C is out of control. It's way too big. There's some good stuff in it, but ES4 had good stuff in it — just that there's good stuff is not justification for something that's that expansive.

I would like to roll it back and start over, perhaps trying to salvage some of the good stuff that was in it — I like the video tag, I like the canvas tag, but there's a whole lot of it that I think is way wrong — and start over with something simpler, and try to find some principles which allow us to grow it, and make it more extensible in a way which does not burden us with so much complexity.

The thing I'm particularly concerned about is that the DOM API is just awful. I mean, who's made miserable every day by the DOM API? The reason that YUI exists, mainly, is to deal with that; because it's such an awful API we need something more expressive and more reliable in order to write our applications. It's wonderful that JavaScript is such a powerfully expressive language that a remarkably small amount of it can implement YUI and turn the DOM into something good. I would like to do that, I would like to figure out what the DOM API should have been, what it needs to be to allow us to write applications, and also to think about it in terms of security — how can I put a third party component on a page and allow it to fully interact with everything else on the page, including advertising, and not compromise us anymore? From my view, W3C is moving in exactly the wrong direction for accomplishing that goal. If it were up to me we would stop that and we would start over with HTML 4.2, and get smarter about it. But I don't think that's going to happen. Damn.

Anybody else? OK, let's all go home. Goodnight, folks.


Recent Blog Articles

Copyright © 2016 Yahoo! Inc. All rights reserved. Copyright | Privacy Policy

Help us continue to improve the Yahoo! Developer Network: Send Your Suggestions