And even though most “real” laptop users will look down their nose at Chromebooks, these low-cost devices have consistently ranked on the top seller lists. In fact, we checked the Amazon top seller list for “laptops” and Chromebooks currently hold the top four out of five spots. Yes, we kid you not. So no matter what we think, it appears that Chromebooks are here to stay.Since I’ve been a Chromebook user since the original Google CR-48, and have used almost every Chromebook ever made, I decided to see how the modern Chromebooks actually stacked up in performance.
Testing Chromebooks isn’t exactly easy though. Unlike Windows, Android or iOS, most of your “apps” are confined to the browser environment so stand-alone tests aren’t available today. Another problem I’ve come to realize with Chromebooks is that as Google pushes out newer versions of Chrome, the performance can be quite variable. Fortunately, we have in hand the new HP Chromebook 11, Acer’s new C720 as well as Google’s gorgeous Pixel, CR-48. We also fortunately still happen to have the Acer C710 to make this performance roundup. This let us update each of the available Chromebooks to their latest browser/webkit/V8 versions before running tests. One thing to note: only the HP Chromebook 11 and Acer C720 are running Chrome browser 30.0.1599.101 here. Google hadn’t pushed out the latest stable builds of the Chrome OS to its older Chromebooks at press time. Older Chromebook users shouldn’t fret too much though. It’s not like an Android handset that is exiled to an old OS. Our experience says Google does eventually update all of the Chromebooks to the latest COS because even the original CR-48 is just one version back now. One issue we have seen though with Google pushing out updates at seemingly random times: They can greatly impact performance by as much as 10 to 15 percent—in both directions depending on the benchmark. That means any meaningful benchmark has to be done basically in one shot and hopefully before Google pushes out a new update.
Chromebooks are also fairly unique from the Windows world. Generally with Windows laptops, old models are flushed out to make room for new units so they rarely overlap. With this roundup, we get to see the impact of various microarchitectures from Haswell to Sandy Bridge and Ivy Bridge as well as ARM since two of the Chromebooks here run Samsung Exynos 5 parts at 1.7GHz. We’ve noticed that Chromebooks is a sector where OEMs don’t seem to mind selling old units alongside newer units too.
For a sanity check, we also included the original Microsoft Surface Pro hybrid device. It features a 1.7-2.4GHz Core i5-3317U on the Ivy Bridge microarchitecture. Why no Surface Pro 2? We weren’t able to get our hands on one or we would have compared it to these Chromebooks as well. Rather than induce even more variability into it, we used the latest version the Chrome browser on the Surface Pro rather than Internet Explorer.
Intel Specs (click here for the complete side-by-side specs)
We let Intel do our work for us so you could eyeball the major x86 contenders in this Chrome-book-off. The Atom N455 is in the original CR-48, the Celeron 847 powers the Acer C710, the Celeron 2955U is in the Acer C720, and the Core i5-3337U runs the Google Pixel. Oh, the last chip, the Core i5-3317U is inside the Microsoft Surface Pro. Amazingly, the Acer C720 pulls off its performance win with a chip that has no Turbo Boost.
Best-sellling Amazon laptops (click the image for the full list)
We are not frakking with you—Chromebooks hold four of the top five spots on Amazon’s best seller list according to this screen cap we took Oct. 28.
Google CR-48:Google’s original CR-48 prototype came out December of 2010 and ran a single core Intel 1.66GHz Atom part. The CR-48 never sold but Google gave away 60,000 of the devices. The original OS didn’t even feature a desktop. The CR-48 is way off the mark in performance today but in a testament to how thin ChromeOS is, it is usable for light duty browsing chores. Google still issues updates for the CR-48 as well.
Samsung Chromebook: With Samsung’s third Chromebook (the company offered a Celeron-based Chomebook) the company offered the first ARM-based Chromebook using the 1.7GHz Exynos 5 chip last October. The unit was light weight, thin and low cost. It also likely ticked off Intel too because if there’s one thing Intel doesn’t like, it’s ARM.
Acer C7 710: Acer also had been playing in the Chromebook arena and followed up its Atom-based AC700 with the Celeron 847-based C7 C710 in November of 2012. The Celeron offered slightly better performance than the ARM-based Samsung but its keyboard was criticized and its battery life just average.
Google Pixel Chromebook: With most Chromebooks under $300 (although the original Series 5 Samsung pushed $450) Google decided to get fancy with its Pixel. The $1,300 to $1,500 Pixel featured a Core i5, aluminum exterior and a gorgeous 2560x1700, touch-enabled IPS panel. The notebook also has perhaps one of the best track pads available today too. Sales figures of the Pixel haven’t been disclosed but when you’re Google—who cares?
Google Pixel Chromebook:The Pixel’s aluminum shell is exquisite and frankly makes all aluminum notebook competitors seem chincy by comparison. The only real mar to the ultra-luxury feel of the Pixel? The power cord still uses a simple round plug. Boo.
HP Chromebook 11: This is actually HP’s second Chromebook. Earlier this year the company introduced a Celeron 847 Chromebook Pavilion for $330. The new Chromebook 11 though features the same 1.7GHz Exynos 5 as the Samsung ARM Chromebook released last year. One key difference: It’s screen is a beautiful IPS panel and it has a 1990s-esque Macbook look too. HP also has a 14-inch version using a Haswell Celeron. Oh, it also charges via Micro-USB too.
Acer C720: We always thought Haswell would take on ARM in tablets but the battle is right here, right now. This Acer C720’s Haswell part destroys the ARM Chromebook’s we’ve seen in both performance and battery life. Yeah, you read that right—in run time too. At $330 on the street, this frankly, is the best bang for the buck in Chromebooks and would be perfect if it had a better screen and its looks weren’t stolen from five-year old Netbooks.
Chromebook Specs
Chromeoff! (Click for larger image)
It’s no joke: 18.5 percent of compiled spec charts on web sites contain 3.1415 mistakes. See if you find out what’s wrong here.Chromebook Benchmarks
SunSpider
SunSpider 0.9.1 is a popular JavaScript test that’s been around for sometime. It’s designed to compare browser-to-browser performance and different versions of a browser. Since we’re using just about the same browser in all of our tests, the comparison should show off actual hardware differences too in theory.
Among the Chromebooks, it’s no surprise that the Ivy Bridge Core i5-based Google Pixel comes in on top. It’s almost twice as fast as the Acer C710 which uses a Sandy Bridge-based Celeron locked down at 1.1GHz. It’s also more than twice as fast as the two ARM-based Chromebooks, the HP Chromebook 11 and the Samsung Chromebook (that’s the official name.) If you look at our chart, the respectable performance comes from the Acer C720. It features a Haswell-based Celeron at 1.4GHz. It’s damn near as fast as the Google Pixel which is priced from $1,300 to $1,500. That ain’t bad. For Wintel folks, the Surface Pro’s numbers are pretty impressive. It’s faster than the Google Pixel on balanced and performance settings, but once the Surface Pro is set for Power Savings (something you may do to squeeze out more battery life) performance is actually on par with the ARM-based Chromebooks. The ARM-based Chrome have long been criticized as being a little under powered and we agree but it could be worse. The original single-core CR-48 platform shows how slow that old Atom chip was.
Google Octane
Google has its own set of bake offs for browser performance. Originally called V8, the test is now refereed as Octane and includes 13 different browser tests that Google deems to be good real-world measurements of JavaScript performance. The test includes everything from Raytrace tests to Cryptography, Mandreel tests and 2D physics test among others. A higher score is better and once again that Tesla S of Chromebooks, the Pixel, comes out way in front. Unlike the SunSpider test which saw the 1.4GHz Hassy-based Celeron damn near as fast as the 1.8GHz-2.7GHz Core i5-3337U Ivy Bridge chip, the Pixel opens up a very nice lead here. Since both parts are dual-cores with Hyper-Threading (and our benchmarking experience of Chromebooks is still pretty limited), we don’t think it’s a function of the core count. Instead, our guess is the advantage the Pixel has over the Acer C720’s Haswell Celeron is in clock speed. There’s also the chance that the cache-sizes also play into it as well but with the tools we have, we can’t really tell. One thing we didn’t talk about with the SunSpider test that we should have is how the Acer C710 doesn’t exactly slay the pair of ARM Chromebooks. The same thing happens here with Acer C710 barely pushing out in front. Why? Again, if we had to hazard a guess, it’s the low clock speed of the Celeron which at 1.1GHz isn’t terribly high. The Celeron in the C710 is also based on the Sandy Bridge microarchitecture which is two generations old at this point. The Surface Pro on Power Savings mode has its clock speed locked to 800MHz. How does the Surface Pro do? That 800MHz Ivy Bridge part still manages to come out in front of the C710 and two ARM competitors. When the Surface Pro’s clock speeds are unshackled though, it’s easily the fastest here.
FutureMark PeaceKeeper
In FutureMark’s browser test we see an interesting result. The result is more graphics intensive as it tests HTML5 performance in graphics, physics and video among other tests. Even though the test seems to be quite different than Google’s Octane, the Chromebook performance results are almost identical to what we saw with Octane. The Pixel comes out on top, Surface Pro also represents well except on power savings mode where it’s as fast as the baseline Chromebooks. The one real change we see is the Acer C720 which turns in performance almost as fast as the Surface Pro and Pixel. Why? Our guess is the increase efficiency of the Haswell core vs. the older Ivy Bridge core powering both the Pixel and Surface Pro. We also think the graphics core in the Celeron Haswell could be responsible even though the Haswell Celeron’s features only “HD” graphics vs. the HD 4000 graphics in the Core i5 parts. We will note that the base clock of the Core i5 Ivy Bridge chips is 350MHz while the Celeron Haswell is 200MHz. One theory we’re floating is that the Haswell part may boost its graphics core for longer periods at higher clocks than the Core i5 chips can. The Core i5 chips can hit 1.05GHz and 1.1GHz while the Haswell Celeron can hit 1GHz. The older Sandy Bridge Celeron in the Acer C710 is so old it doesn’t even rate an “HD” from Intel. Its dynamic graphics frequency is also rated from 350MHz to 800MHz but we suspect it doesn’t run at 800MHz for very long.
Browsermark 2.0
Rightware’s Browsermark 2.0 has five test groups that tests CSS, DOM, how fast the browser will resize and load a page as well as WebGL, Canvas and JavaScript performance. It seems like a very good all-around browser test that doesn’t favor the faster hardware as much. At least, that’s what we’re seeing our results which match the other tests with the Surface Pro, Pixel in front. The Haswell Celeron is also damned close to the Surface Pro and Pixel too showing the test doesn’t seem to favor clock speed. The Sandy Bridge-based Celeron and pair of Exynos 5 Chromebooks also turn in a decent performance. Even that old dog, the Atom N455 is only half the speed of the latest and greatest CPUs with all the new bells and whistles. Yay! Everyone’s a winner. You may scoff, but in a way, Browsermark 2.0 is probably a pretty accurate representation of how these platforms will perform in light browsing loads. When you’re just checking email or reading a site, you’d be hard pressed to tell the difference between the Google Pixel and the HP Chromebook 11. If you push them harder though, you will feel the difference.
WebGL Planets
Since most of the tests here run multiple tests and then calculate an overall score, we wanted to run something that might give us an idea of how well the Chromebooks would handle pure WebGL thrown at them. For that we used SpaceGoo’s WebGL Solar System experience which is compute heavy as it computes the hundreds or thousands of planetoids orbiting a larger planet. We start out with 1,000 planetoids and then let it slowly spool down. We report the frame rate when the planetoid count hits 900. We thought this would give us a better feel for compute power but our results have us questioning its value. The Pixel, which had outperformed all others, was barely ahead of the Samsung Chromebook and the Samsung Chromebook which internally is nearly identical to the HP Chromebook 11 was far faster. Surface Pro also easily destroyed all of the Chromebooks here. This makes us wonder if it’s a driver issue even as we expected the Surface Pro to be just slightly faster than the Google Pixel as both are built around similar chips. One thing we’re not surprised by is the CR-48’s performance because we’ve always known that thing was slow.
Kraken 1.1
JavaScript is one of the key benchmarks all browsers strive to optimize for. This is the benchmark developed by Mozilla to be measure of realistic JavaScript use in applications. In our tests, we again see the Core i5 units come in front with the Acer C720 turning in fantastic performance for its price. The low clocked 1.1GHz Acer C710 again doesn’t exactly run away from the ARM Chromebooks and the single-core Atom easily setting the benchmark—for low performance.
Wirple
Wirple is a mostly HTML5 that measures browser performance in Canvas3D and WebGL and is heavily focused on 3D performance. There’s no surprises here as the Surface Pro and Pixel come out on top with the Acer C720 showing the best bang for the buck with it impressive performance given its $250 price tag. The ARM-based and older Sandy Bridge-based Celeron C710 offer pretty atrocious 3D performance comparatively.
RoboHornet Alpha
RoboHornet has been called the Holy Grail of browser benchmarks by Tom’s Hardware. Part of that may be because no less than Google, Facebook, Microsoft and Mozilla are behind the new benchmark as a way to accurately measure browser performance. The test measures CSS, DOM HTML and JavaScript performance. No surprise, the Ivy Bridge-based Core i5-based units place on top but again, the Acer C720 Haswell comes mighty close in overall performance. One thing we want to say about the RoboHornet test is that it purports to be a more realistic measurement of browser performance and we’d agree. While the ARM and older Sandy Bridge Celeron C710 are about half the speed of the Pixel, C720 and Surface Pro, you often don’t actually notice it unless you look for it. In other words, they’re still very usable. Even the CR-48’s scores are pretty decent here and in most light-duty browser tasks, we’d agree. Still, there’s no real change here. For the most part, if you want the best performing Chromebook, the Acer C720 and Pixel should be on your list.
Multi-tasking
Since few people have just one browser tab open, we decided to run the crowd through a multi-tasking environment consistent of an open tab to Google Music, Google Maps (set in satellite mode), Amazon.com as well as SpaceGoo’s WebGL planet and playing the 1080P version of the trailer for the movie Gravity on Youtube. With the trailer and WebGl actively running, we then ran the Kraken 1.1 benchmark. The results we saw showed that the slower Chromebooks had a far tougher time running so many tasks. We opted not to run it on the Samsung as its performance was pretty much the same as the Chromebook 11 and we also let the CR-48 go home early, because we know it’s already a dog. But to give you an idea how the Exynos 5 Chromebook 5 does, just peak back at the performance spread when it running just the Kraken benchmark along versus the multi-tasking load. One of the contributing factors besides clock speed and microarchitecture may simply be thread counts. The Surface Pro, Pixel and C720 all feature Hyper-Threading. The other Chromebooks were limited to but two threads and with so many workloads running, they were easily bogged down.
Battery rundown
Our final test was a battery run down test. Rather than loop a video or just leave them sitting there, we decided to use the battery test in Futuremark’s Peace Keeper as our run down. This kept the WiFi on the notebooks hot and the screens on. The result? All of that performance on the Google Pixel (in addition to its high-res screen) meant pretty average to mediocre run time. Of the Chromebooks, only the Acer 710 was worse pretty much tying the Pixel at 195 minutes of use. The Chromebook 11 also performed pretty average with 241 minutes of run time. Frankly, we expected more. Its near twin, the Samsung Chromebook turned in a very usable 283 minutes. We’re not sure why the HP performed worse but it could be the IPS panel it uses versus the TN panel in the Samsung ARM Chromebook or even a smaller battery—though both publically state they have 30 Wh batteries. The real stunner is the Acer C720. More than performance, Intel has been chasing ARM on battery life. People know x86 kicks ARM ass in performance, but in battery performance ARM has the perception of awesomeness. That’s not the case here as the Acer C720 turns in a stellar 360 minutes of run time. That’s enough to use it on a cross-continental flight (provided you pay for inflight WiFi). Yay for x86 right? Well, yes and no. First, you can’t just compare one Chromebook to another and proclaim it a victory for the microarchitecture at hand. You have to look at all of the hardware around it and, well, the entire platform. This isn’t about ARM vs. x86 – it’s really about HP Chrome 11 vs. Acer C720. The battery in the C720, for example, is beefier than the C710. The Acer is almost half a pound heavier too—much of which we think comes from the bigger battery. What we can say is if you want stellar Chromebook battery life (something the anemic CR-48 was always known for) buy the C720. One final note on battery life: the Surface Pro really doesn’t do too badly. With Windows 8.1 set to power savings mode, the run time is but 38 minutes shy of the Chromebook 11. That’s not really as horrible as people think the Surface Pro will run. It’s also clear to us that the Surface Pro 2 with its Haswell part, should offer a nice upgrade in battery life over Surface Pro.
We do declare the Acer C720 as the current best Chromebook!
SummaryWhat’s the take away from this? If you’re shopping for a Chromebook, our recommendation is Acer’s new C720 based on performance and battery life. We do love the Pixel, but unless you’re rolling in dough, you probably can’t afford it. The price/performance/battery life ratio on the Pixel is also disappointing next to the Acer C720. Our only real beef with the C720 is its screen. It’s a pretty mediocre TN panel especially next to the IPS in the Chromebook 11. But screen isn’t everything. We did a simple test where we loaded up five major news sites on the C720 and Chromebook 11. As we know, media web sites tend to be loaded down with Flash and enough ads to make you think you were Las Vegas at 2 a.m. and simply scrolling down the web page with the Chromebook 11 would choke its ARM processor. It was enough to choke the C720 on occasion but the scrolling was acceptable. In our opinion, based on its price/performance/battery life ratio that makes the C720 the best Chromebook available today that we’ve seen.
Source
0 comments:
Post a Comment