David Baron's Weblog

I've been pretty busy lately, mainly with work to get Firefox 3 out the door, so I haven't blogged yet about Acid3. For those who weren't paying attention, there was a recent race between Safari/WebKit and Opera to see which engine would pass first. (It's not actually clear to me who won, or what the rules of the race were.)

I think some people might be interpreting these Acid tests as a fair measure of standards support in browsers. I don't think they are. In fact, I don't think it's possible to construct a fair measure of standards support in browsers. That involves having somebody sit down and decide which standards count (which are in the test at all) and which are more important than others (how they're weighted in the test). And as shaver pointed out (among other points that I mostly agree with), the tests in Acid3 were required to be broken in either Firefox or Safari, which clearly doesn't help the fairness of the test or the scores of either of those browsers.

But let's suppose the acid tests are a good measure of standards support. After all, they are better than nothing. Are you more interested in knowing how browsers did before they knew which questions were on the test, or after? I think before, since it says more about the various browser teams' long-term committments to standards when they're not working in a high-publicity setting.

So the first evidence I can find of Acid3 bugs being pointed out to browser developers is this WebKit bug filed on April 22, 2007. If I look at the browsers shipping at that time, I find that Firefox 2.0.0.3 gets either a 52/100 or 53/100 (plus rendering bugs), Opera 9.20 gets a 46/100 (plus rendering bugs), Safari 3 (actually released slightly later, but it's easier to give Safari a few extra points and use Hixie's numbers since it's hard to install old versions of Safari) gets a 39/100 (plus rendering bugs), and IE7 gets a 12/100 (plus rendering bugs bad enough to make me unsure that I'm even looking at the right number to get the score).

Likewise, if I look at Acid2 behavior around when the Acid2 test was released, I can find screenshots of IE6, Opera 7.54, and Firefox 1.0.2 on David Naylor's blog entry from the time, and a screenshot from then-current Safari (at left) in this Macworld article. Firefox looks the closest there too.

That said, there is real value in fixing the bugs and implementing the features that would make Firefox pass Acid3, partly because Hixie's a pretty smart guy and has good ideas about what we ought to work on (although certainly not the only valid ones), and partly because other browsers are doing it, so if we do too, it will help add items to the set of features that authors can use reliably across browsers, whether or not they're really the most important features to put into that set. We're not planning to cram a bunch of Fixes into Firefox 3 since it's almost ready, and cramming features in at the last minute risks hurting other Web standards support or hurting some of the other things that make Firefox a great browser. That said, we are already working on a bunch of these bugs and features, and we welcome the help of others in doing so. (I'm not promising we'll fix all of them, and I think it's pretty unlikely we'll implement just the subset of a spec needed to pass Acid3.) As we consider what to work on for the next release of Firefox (and beyond), we'll weigh the bugs we need to fix and features we need to implement to pass Acid3 against other important bugs and features. I expect many of them will be found to be deserving of high priority, but there's also lots of other work that's important for Web authors competing for our attention.