Zoips, thanks, that helped a good bit. Even so, 1400 rows takes about 8-12 seconds to sort. The Perl code that generates these reports takes a fraction of a second to sort and print out (including all formatting) that many rows.
In a few years, most of the “webpages” will be displayed on handheld devices that don’t even seem to be running a browser (much like the iTunes store). The idea that the contest is between IE and FireFox is much like thinking that the battle for supremacy in the American luxury car market is between Lincoln and Cadillac.
Right now, Microsoft is having difficulties getting its act together. .NET was never really a rendering engine technology and Silverlight has the same problems as Flash: You need a proprietary client in order to see its content. That may be possible when you are only talking about x86 based chips and only two major operating systems with one having 90% of the market. When you have thousands of devices, each with its own chipset and maybe its own OS, you simply cannot depend upon proprietary clients.
Apple/Google are banking on WebKit and AJAX. WebKit is already in Mac OS X, the iPhone, and is on Android. Google is also banking on it for all of its stuff. KDE has now adopted WebKit over its own KHTML (which WebKit was based), and the GTK+ project is also adopting it too. That puts WebKit firmly into Linux which means it’ll be in all Linux based devices like setup boxes, phones, and PDAs.
I am not too sure what Microsoft is going to do. Next year will be very tumultuous for them. As the price of hardware drops, it gets harder and harder to convince PC makers to buy Windows licenses on all PCs they manufacture. Linux desktops are becoming more consumer oriented, and you can’t beat the cost of a Linux license. Combine that with OpenOffice, and you can offer a PC for about $100 less than a competitor. When you’re talking about $200 PCs, that’s quite a bit of money.
Microsoft also hasn’t done very well with Windows Mobile. Windows has been difficult to scale down to handheld devices, and they’ve pretty much lost the Setup box market. Scientific Atlanta, which has a near monopoly on the market, has gone solidly behind Linux. The iPhone, which hasn’t even been out for the whole year yet, has outsold all Windows Mobile smartphones combined.
Microsoft isn’t going to go away, and I am sure they have plenty of things they’re working on. They’re a smart bunch of people, and I’m sure they see the same things I mentioned. Bill Gates has always said that Microsoft is just an innovation away from irrelevancy which is why he always fought so hard for Windows’s monopoly. My guess is that the OM license price for Vista Home Basic will drop to less than a dollar by the end of next year. That will encourage PC manufactures to stick to Windows over Linux. I suspect that a Microsoft Home Basic Edition is also in the works too and will have a similar OM license price.
The big question is how Microsoft will handle the non-Desktop market. Right now, they’re way behind, but Microsoft has been in this position before with Netscape, and came out on top stronger than ever before. Can Microsoft do that again?
This is a fun excersize, but I dispute the usefulness of the results to the real world.
- Ansyncronous and syncronous HTML GET calls.
- DOM calls to the page structure for getElementByID() and setting style properties
- DOM updates like new Image().
- Responding to events like onClick() and onLoad()
@Jeff: thank you for the interesting analysis.
I think Tufte might like the first chart and the second… you are looking at two distinct angles on the same data set.
The first is the relative performance of each benchmark with the suite by each browser—you get to see where each team needs to spend more time to maximize performance… the Webkit team would do well to work on “access” and “bittops” while any effort “controlflow” would yield only a marginal impact on improving the overall score.
The second chart does a pretty darn good job at comparing which browser does the best/worst—the relative performance of a benchmark between browsers.
Nice work Jeff! I’d love to the first chart for my browser when I run SunSpider!
You are right that on many web sites, DOM/layout performance matters more than core JS. Maybe we need better DOM benchmarks, too. But it is useful to be able to analyze performance of different engine components separately.
Please take into account how long it takes to load the browser…
A cheap browser like Konqueror can load and display a page while firefox is still opening and loading as an app…
Little off the topic question. Can you tell me how you generate those graphs? Any specific tool or macros etc. I am a big fan of how you present information in your blog and would love some tips.
The second chart is horrible. I don’t think Tufte would approve. It makes it kind of hard to compare the different browsers.
prepare to get slashdotted
@Aditya Banerjee: did you try Firefox with a clean profile? It’s entirely possible that errant extensions that you don’t have in your Flock install would be causing a 10% drop.
While I commend you for taking the time to do these tests, the graphs at the top are a poor way to present the data. I don’t think people really care to compare how long one benchmark compares against another, they care how the various browsers compare.
I suggest you swap the browser and test data sources in your bar graphs so that people can more easily compare how well browsers fared against each other.
i’m looking forward to better js debugging tool. as of now, it’s seriously lacking.
Although Safari is based on KHTML could you run some benchmarks
with Konqueror to see how well it compares against its Apple raised cousin?
@Mat Hall: Opera 9.25 and 9.50 use different scripting engines, and the results are much better for 9.50:
Linear_b is used in Opera versions up to and including 9.25: http://en.wikipedia.org/wiki/Linear_b_(script_engine)
Futhark is used in Opera 9.50: http://en.wikipedia.org/wiki/Futhark_(script_engine)
Why hasn’t anyone complained about IE7 having one of it’s tests thrown out? Obviously it’s a test that the other browsers can handle and should not be thrown out for IE. Has anyone actually looked into why IE7 does so bad on the string test?
Scatter shot comments follow:
And I agree with some other posters: the second graph is pretty but remarkably bad, making it very easy to compare the runtime of different benchmarks (apples and oranges–useless!) as totaled across the browsers (double useless!) and very difficult to compare the browsers to each other.
Leaving off the strings (unfairly, since one does an awful lot of string manipulation in many apps), Firefox is about 30% slower than IE. That really ain’t much, and as soon as you do some string work that takes almost 9 times as long in IE, that 30% vanishes fast.