It's been a week since the BlackBerry Bold was released by Rogers in Canada. A lack of in-store availability on launch day meant I had to wait until day two to actually buy one, but buy one I did and I unboxed it on video with nearly uncontrollable excitement (can you blame me?!). I followed up soon after with another video showing off the device's media capabilities via the photos, movies, music and ringtones that come preloaded on the Bold.

With those two clips out of the way, it was time to move onto my next task - the BlackBerry Bold browser overview and Bold vs. iPhone 3G browser speed test showdown. I know some of you are getting sick of the Bold vs. iPhone which is better/comparison stuff, but there are a LOT of people out there who want to see this browser speed test happen (including the editors of some major tech sites that have been urging me to get this showdown done right). It was only a few minutes after I set out to begin the definitive Definitive DEFINITIVE Bold/iPhone browser speed test showdown that the frustrations that have consumed the past four days of my life began...

And four days later I am still unable to complete this master of all browser shootouts. My Bold's browser is fast at consuming pages over 3G with Javascript support turned off, but as soon as I enable Javascript support or try and browse over WiFi via the HotSpot browser, my Bolds (I have two of them now) fall apart. Read On for a whole lot of geek talk...

THE PLAN

So here was my master plan. Visit eight popular websites on a commercially purchased, full release BlackBerry Bold and iPhone 3G under the four basic browsing conditions used by most people...

Websites to be visited:

• CrackBerry.com
• TheiPhoneBlog.com
• Engadget.com
• Gizmodo.com
• BoyGeniusReport.com
• NYTimes.com
• WSJ.com
• NationalGeographic.com
  

Browser Conditions for Speed Test:

• Over 3G, Javascript Support Off
• Over 3G, Javascript Support On
• Over WiFi, Javascript Support Off
• Over WiFi, Javascript Support On

All told each phone would be timed over 32 website visits and compared under the same browser settings/conditions. The phones would not be "raced" side by side so as to eliminate variables (wouldn't want them competing for the same WiFi signals now would we?!). Rather, they would be individually time trialed. The timing device would be the stopwatch on my BlackBerry KickStart (err Pearl) 8220. Cache would be cleared, the appropriate settings would be picked per the four browsing states listed above, and then the eight sites would be visited in a row (timing each site individually, short break between each visit as the new URL is typed in). Once testing was complete for the BlackBerry Bold, I'd put it down, pick up the iPhone 3G, and visit the same eight sites under that same browser setting state. Once testing on a setting state was complete, cache would be cleared, settings would be changed, and the tests would be done under the next condition. And so it would go until each phone was timed visiting eight sites, four times over, each time under a different state. Also, the tests would be carried out late at night so that the sites being visited wouldn't be getting updated or change while the speed tests were taking place.

The other issue I wanted to be consistent and within regular operating norms on was signal strength. Though I own an iPhone 3G, I don't use it much or know that much about it to check those settings, but on the BlackBerry I used the ALT + NMLL shortcut to switch my signal strength to numbers. Throughout the 3G tests it registered at -70. For the WiFi tests, I checked the connection strength (Options > WiFi Connections > Menu Key > Network Details) prior to each test and and it was always between -35 and -40 dBm. So though I don't know the numbers for the iPhone 3G, at least it was always used in the exact same location. Lastly, I decided that the radios would be left on during the WiFi tests so as to duplicate real life use. I think most people still want the ability to receive calls even when browsing via WiFi.

All in all, about as fair a trial as I could think up or conduct within my work environment with the tools and smartphones at hand.

Special Note Regarding the BlackBerry Bold's Web Browsers

It should be noted that on the BlackBerry Bold additional setting changes need to be made in order to conduct the test as described. On the iPhone, the only settings required to be changed are Turn WiFi On or Off and Turn Javascript On or Off.

On the BlackBerry Bold, there are a couple more steps as there are actually two main browsers - the Internet Browser, used for browsing sites over 3G and also the HotSpot Browser, used for browsing sites when WiFi is enabled. It's kind of interesting to note that the default for the Bold's Internet Browser is to have Javascript Support disabled, while under the HotSpot Browser it is enabled (iPhone's default is to have Javascript support enabled). So when switching between 3G and WiFi tests on the Bold, the actual browsers need to be switched accordingly. Also, the Browser Identification settings needed to be changed. On the Browser Configuration page there are three options under the Browser Identification field: BlackBerry, Firefox, Internet Explorer. This setting determines how web servers see the device.

Seemingly Innocent Browser Settings...that for me lead to TROUBLE!

When set to BlackBerry, if you visit a site like NYTimes.com or WSJ.com, their servers will detect the BlackBerry setting and serve you up a mobile-optimized site instead of the full html site. A nice touch, especially if content is your only concern, but for this test since we want to compare HTML sites the default settings wouldn't work. So for all of the browser tests, the Bold would be set to "Internet Explorer" (which I later would find out from a Rogers Rep and RIM Support Rep was the recommended selection for this kind of test).

Preamble to the SUPER DUPER TEST

So it was after midnight on Sunday when the rest of the blogosphere was hopefully sleeping (didn't want anybody posting stories and changing the sites to be visited) that I got set up to do the testing. Video camera ready, digital camera ready, notebook and pen ready, stopwatch ready, iPhone 3G and a shiny new BlackBerry Bold ready.

Before getting to the official time trials per the rules above, I figured I'd do some general testing and browsing so I'd have a sense of what was to come. Both phones had their WiFi turned off, and their browsers set to factory defaults (minus Browser Identification on the Bold set to Internet Explorer). I was pleased to see the Bold beat the iPhone for pulling up the first three sites I visited - CrackBerry.com, Engadget.com and Gizmodo.com. It was only faster by a few seconds, but it was faster, so I took this as hopeful sign of good things to come. The Bold's browser doesn't do quite as a good of job at rendering pages as the iPhone 3G's browser, but it's pretty darn close (I'd argue 95% as good). There are some sites, like NationalGeographic.com, where the Bold really messes up the tables, and it also seems to have something against the header on CrackBerry.com. Regardless, the Bold pulled up these sites about a bajillion times faster than they would have on my 8320 Curve on Edge and looked about a gajilion times better than the browser on OS4.2.2. A HUGE leap forward for RIM.

Based on this initial testing with both phones set to their browser defaults, you could argue the Bold's browser is actually faster than the iPhone 3G's browser. But... BIG BUT... the Bold's Internet browser is set to Javascript Support disabled while the iPhone's default is to have it enabled. To tell you the honest truth, when it comes to just viewing websites on both the Bold and Safari I have not really noticed much difference visually between having JS on or off (the pages render about the same). In a world where speed is of the essence, I think most people could live with JS turned off. But for this to be a fair test and draw a fair conclusion of which smartphone is faster at browsing, the speed tests would have to be compared under the same browser settings. Takeaway Lesson: if you see any speed tests conducted between two smartphones, be sure to double check the settings. Enabling JS support on one and not the other can skew the results however you like. 

LET THE RACES & FRUSTRATIONS BEGIN

3G Browsing, Javascript Off
The Bold pulled up the majority of sites in the 29 to 40 second range. Not too bad I figured, considering some of the sites (like CrackBerry and theIphoneBlog are actually pretty beasty). The Bold seemed to be off to a good start.

iPhone 3G's turn. Wow. I was actually pretty shocked at how fast the iPhone 3G pulled up pages with Javascript disabled. In the time that I've had an iPhone 3G I have never thought to turn off Javascript (have just used the browser on its defaults), but apparently doing so is a pretty good idea - it seems to save about 10 to 20 seconds per site on downloading time.

When set on the same browser settings, in this round of speed trials the iPhone 3G definitely seemed to hold the upper hand. Round 1 goes to the iPhone 3G.

3G Browsing, Javascript On
And this is where my story takes a downhill turn and becomes a full-fledged nightmare. Up until now I was super happy with my BlackBerry Bold. Loving it. Though it was a bit slower in the first round of tests than the iPhone 3G I was still happy. Time for Round 2. I clear the cache on both phones and turn on Javascript support. I start with the Bold first - this would be the first time ever browsing a site over 3G on the Bold with Javascript support checked on. Time to visit CrackBerry.com.

2 minutes go by before the page finishes loading... WTF?!?! Over twice as long for Javascript support? Let's try the iPhoneBlog.com. 3 minutes?! Something can't be right here. Watching the pages load with Javascript support enabled was painful. The blue status bar was constantly going (requesting script, running script, loading images, requestin script, running script) and the hour glass began popping up on the screen. Ok. Let's try one more. Engadget.com. One minute, two minutes, three minutes... A problem occured while trying to render the page. WTF?! Ok Time to clear the cache and reboot this phone. Let's try this again. Over 3G, Javascript on, ok.. two minutes to load Engadget. Better. Let's try Gizmodo. Hmm... Four Minutes later it's still loading. I lose my patience. I try and hit Menu > Stop but the device is stuck thinking. Eventually it does respond and I manage to exit the browser. Alt + Caps + H brings up the Help Me! Screen and I look at the File Free number. Only 12 megs? But...but... I swear I had 43MB free just a few minutes ago. I mean, common on, I haven't even installed one app on this phone or done anything other than browse a couple of websites.

HOUSTON... errr.. WINNIPEG... we have a PROBLEM
There goes my well-planned, comprehensive and fair test. My Bold must buggered is all I can think. Let's give it a few minutes rest and I'll finish off the iPhone's tests. I whip off the eight sites with the Javascript on over 3G. Clear the cache, turn on WiFi and proceed to whip off the 16 site visits required for the test. You can check out the times on the tables. Kudos to Apple - these tests went fast and flawless. Maybe their new marketing slogan should be "The iPhone 3G's Web Browser can load NYTimes.com faster than Usain Bolt can run a 100 meters."

So with the iPhone 3G now done its share of the tests, I'm back to the Bold. Let's try visiting a few more sites over 3G with Javascript support on. No dice. Same issues. It'll pull up the first site I visit with Javascript on within a minute or two, then from there if I try to visit any other sites without first clearing the cache or restarting the phone it takes minutes to complete loading a page, if it loads at all. I found I was getting a whole bunch of errors - Cannot Render Page, Insufficient Memory, or my favorite, Communication Error. YET... if I close down the browser, restart it and disable Javascript once again, the site that just took 4 minutes to load now takes less than 30 seconds. And I can browse sites continuously. Conclusion - something is FRAK'd with the Bold's ability to support Javascript when browsing over 3G. Well, for ME anyway. Time to move on. I'm sure things have to go smoother with the WiFi tests.

I connect to my local network and the connection is STRONG... -35 dBm. I'm less than a hop and skip (no jump) away from the router for the duration of these tests to help ensure strong signal strength and hopefully quick WiFi browsing speeds. Let's start with Javascript off (where there doesn't seem to be an issue). I had CrackBerry.com loading in 40 seconds over 3G, so with the WiFi generating such a strong signal I'm thinking it should be at least 10 or even 20 seconds quicker, considering the iPhone loaded it in under 20 seconds. Nearly two minutes later the site finishes loading. WTF?! Shouldn't WiFi be faster than 3G? I'm not convinced. I pull the battery, clear the cache, and try it again. Problem rendering page?!!

After a while I got used to the Bold's Browser HATING me

At this point I'm thoroughly baffled. I try browsing websites over WiFi on the Bold for the better part of an hour. Unfortunately, that doesn't equal many sites. I eventually gave up. Something must be wrong. Images would begin to load, but then the blue status bar of death and hour glass would take over. It would take several tries of cache clearing and rebooting to get a site to load, and when it did, it was in the minutes. Either this phone is broken (yet everything else about the device seems to be in perfect working order) or the Bold hates me, or RIM released a device with some Gremlins living in the code, or maybe a bit of all three. 4am. Time for bed.

LET THE FRUSTRATIONS CONTINUE

Test Day 2 - Monday
I take care of my regular Monday morning CrackBerry duties then get back to the Bold. I do a few more tests. Same deal as the night before. I call Rogers for support. While the tech guy I got was super friendly and we examined the device and browser settings and options, everything looked good on my end. He thought maybe it was hardware problem and suggested returning the device since I just bought it a few days prior.

I call up RIM Technical Support. I give them my credit card number so they can charge me the $50 per tech issue fee. I just want to talk to somebody who has a Bold in there hand and hopefully several Bolds nearby (Rogers guy was on an 8700) so we can do some of these tests over the phone and we can get to the bottom of this issue. Unfortunately, the first rep I talked to didn't have a Bold on him and the guys in tech support who did were not around at the moment. But the rep assured me that turning on Javascript should not add minutes to the browsing experience nor mess up the phone the way I described it. We examined the settings and did a couple of quick tests. No luck. He then suggested I wipe the handheld (Options > Security Options > General Settings > Menu Key > Wipe Handheld), so we did that while on the phone. Still no luck. He suggested I download the OS from Rogers (they just put it up on their site that day) and do a clean install of the OS. He figured if that didn't help it may be a hardware issue and that I should take it back and return it for a new one.

So I reloaded the OS. No luck. I called up my Rogers guy to see if he had any Bolds left in stock. He had ONE. I make it to the store just before it closed for the night. Instead of returning it, I just decide to buy a second Bold. This way I'll now have a second device to test out my findings on and make sure my numbers are valid... because at this point I'm not really trusting the device. While at the store, I grabbed my sales guy's Bold and did some testing on his device. There was no WiFi to test on... but guess what? Same Javascript support issue over 3G. I browse a few sites with Javascript off and they load quick...turn on JS support, and I don't have the patience to watch event one site finish loading. I buy my second Bold, go home, put everything on the charger and go to bed.

Test Day 3 - Tuesday
Please Please Please just let my first Bold have been a dud. I love the device. I just want it to work properly the way it should.

Nope. Same EFF'n issues. Exactly the same. Web browser works nice and fast if it's over 3G and Javascript is turned off. I turn on Javascript and the device becomes a dud. And this one is HATING my WiFi connection just like the first Bold. It dawns on me that maybe for SOME reason the Bold has some sort of incompatibility issue with my router (a D-Link DIR-625). Seems weird, especially since the iPhone seems to love it, but to eliminate this possibility I go to the homes of a few of my friends and try doing some WiFi tests over their HotSpots. No dice. A page will start to load...and then it craps out. I have yet to see one page load fast on either of my Bolds over WiFi. Well - not totally true. One time after a reboot I managed to load NationalGeographic.com in 37.56 seconds. Not bad...though over 3G it only took 14.42. This has to be a curse. Two tainted Bolds?

I call back in to RIM Technical. I get a really helpful guy who has a lot patience with me and the situation at hand. Juggling both Bolds back forth, we go through different speed tests under different settings. He's taking notes. He gives me a task list of more tests to try and he gets started on doing some testing of his own. I email him my results. He emails me back an hour or so later that he's escalated the testing on to their Research department and somebody from that team will get back to me.

At this point I get an email from one of CrackBerry.com's forum moderators telling me to check out a forum post. It seems a few members are beginning to wonder where I'm at... nothing new on the Bold, not too many blog posts from me this week, no new podcast.. so I post where I'm at in the forums along with a little RANT!

I give up for now. Two new BlackBerry Bolds that have been three years in the making that I've been waiting months for to get released and I can't even get done the little browser speed test I've been trying to complete since Sunday night. I'm going to bed.

Test Day 4 - Wednesday
I email back RIM technical support first thing in the morning looking for an update. They respond and want more questions:

• make and model of the router I've been trying to use WiFi on
• the exact steps I have been going through in conducting my speed tests.

They're on it. I like that. I just wish they could tell me what was wrong and tell me how to fix it. Trust me, I do NOT want to be spending my time doing this (nor writing this blog post for that matter). I just want things to be perfect. And while I'm not opposed to going out and buying a third Bold, I just have a feeling it won't solve this nagging Javascript issue, nor my weird WiFi issue.. which seems strange.

But it seems my buds over at BlackBerry Cool were having a better time than me with the Bold. Simon posted up a video comparison between the Bold and iPhone 3G. The iPhone 3G wins a buy a bit, but it's close. But it also looks like the Bold's browser has Javascript support disabled for the tests. I couldn't help but WONDER if the boys at BBCool would run into the same issues as me if they turned Javascript support on.

I think it's time to get some outside help on this. My definitive Definitive DEFINITIVE browser test is obviously garbage at this point. I don't think it can considered a valid test because both my Bolds are messed up. So I call up Doug from BlackBerry Cool and bring him up to speed on my week of frustrations. BBCool's new Rogers Bold has been doing ok, but while we're on the phone Doug starts doing some tests. He's definitely not having the same WiFi browsers that I've been having. That's good. I have hope that there's a simple explanation to why my two Bolds hate WiFi. Doug switches to 3G with Javascript off and pulls up a couple sites. Still pretty fast. Now for the true test. Doug enables Javascript within the 3G/Internet Browser. I AM CURSED. It only took a minute for him to realize something was a mess. And when trying to visit a second or third site over the Internet Browser it slowed to a crawl and brought up the hour glass and rendered the device next to unusable.

That was enough to get BBCool onboard to try and figure out what's going on here. I emailed Doug my spreadsheet with sites to visit and my timing data (plus we added two more great sites to the test - blackberrycool.com and quicklybored.com) and he picked up the gauntlet. I have heard grumblings that there could be some "geographic" issues at play here with the issues I'm experiencing, and while I don't understand why or how that would be the case, with Doug and Simon being in Ottawa we could at least maybe validate or throw out my results data.

Test Day 5 - Thursday
RIM got back to me with some of their tests. They told me they had visited all eight websites I was testing over WiFi in 24 seconds or less with Javascript both Off and On. FAST. That's what I want. That's what I've been expecting! But in that email they made no mention of the 3G speeds (with JS on or off) or whether or not they were investigating that end of the equation further. They did however give me a new set of sites to visit and do some time trials on:

• TSN.ca
• WSJ.com
• CNN.com
• BBC.co.uk
• Drudge.com
• HuffingtonPost.com
• FoxNews.com
• CBC.ca
• Reuters.com
• Sportsnet.ca
  

You can see the results. The Bold handled the sites no problem over 3G with JS off, and I was able to run through them in succession, but I again ran into the same issues once I turned it Javascript support on or tried anything over WiFi. I actually had one site load fast over WiFi - TSN.ca. But once I tried to browse to WSJ.com the phone just locked up after 2 minutes of attempting to load the page. And really, should I have to reboot the phone and clear the cache just to browse one site? Something is obviously messed up here. I lost patience and sent them in my data from this round of tests for the sites I could do in reasonable time. I'm waiting to hear what's next.

In the meantime, Doug got back to me with the completion of his tests! The are POSTED HERE on BlackBerryCool.com. You'll want to check out the results. Pretty Interesting. At least I'm not totally crazy. He IS running into the same issues as me when he enables Javascript support over the 3G browser. Thank GOD his HotSpot browser is actually working and the test times look good. The WiFi strength he tested on isn't too strong (I think his times would be a lot faster if it was around -40 dBm) but obviously things are in check there and it seems like his HotSpot browser can handle Javascript. Even with things working better for him on the WiFi browser some of the results seem a bit out of whack... for some of the tests the pages actually loaded quicker with Javascript on.

WHAT ARE YOUR RESULTS?

You've seen what I'm trying to do. How do your test results compare? Post them in the blog comments or on this dedicated forum thread. Or email it in to Kevin at CrackBerry.com. Are you experiencing any of the same issues? Are you trouble free? Let me know! Follow the methodology in this post with the same sites being visited. I'll follow up with a Call for Bold Browser Data blog post shortly that lays out the exact steps.

Update: I've been receiving a TON of emails on this since posting, and it seems LOTS of you are experiencing the same issues. MANY with the Javascript enabled over 3G browsing, and SOME with the same WiFi issues. Ergggs. 

CONCLUSIONS FOR NOW

I'm sorry guys and gals. I failed you. I set out to put together an awesome test comparing the browsing speeds of the Bold and iPhone 3G and I couldn't get it done. The premise was there, but the equipment is not up to snuff yet to draw any definitive conclusions. I say that because I don't think this is a fair battle just yet. I may have bought the Rogers BlackBerry Bold (well, TWO of them) over the counter, but the Bolds I'm rocking are obviously still in training - not ready for the full-out browser speed test war just yet.

So here are my conclusions for NOW:

• the BlackBery Bold from Rogers has some serious issues when you attempt to enable Javascript support while browsing the web over 3G. Maybe not for everyone, but at least for four devices owned by a guy who works in a Rogers store and two BlackBerry Addicts who happen to be the editors of BlackBerry blogs.
• there has to be some sort of issue with the Bold and WiFi/HotSpot Browser that pops up under specific circumstances. I don't know what is yet, but I'm sure there's something up. I know I'm a CrackBerry Addict and not a BlackBerry Expert, but I don't *think* it's user error on my part in this case. 
• when the Bold browser is working it's pretty fast and works well. Not quite as fast as the iPhone's Safari browser, but fast. Word has it that RIM still has a lot of performance tweaking to do to extract more speed out of the browser and I believe it. There's no doubt the hardware is capable of sucking down the data quick, so this showdown may be pretty tight in a month or two as new code gets released. For now I'd be happy just having a native browser that worked as intended. I've got video footage of this whole debacle/massacre/mess or whatever you want to call, but I'm going to trash it in hopes a better day will be here soon. I'll turn the video recorder on once the Bolds' browsers are no longer buggered.
• if you're in the USA waiting for the Bold to come to AT&T, don't be mad, be GLAD. I'm sure hope by the time you get the device RIM will have addressed these issues and you won't have to be frustrated like ME.

If you've read this far you are hardcore and truely a smartphone geek. Be proud of it. I've spent the better part of a week in frustration dealing with this. I think me and the Bolds are officially now on a break. Just for a bit. At least until this issue gets resolved and I actually know which one of the Bolds I'm going to keep as my daily driver, my SIM is going back in my Curve.

No 3G or Awesome Display or Fast Processor, but my Curve is Still Sexy

Good thing I just did the Colorware thing and got it painted white. I'm still attracted to her and have a lot of love... call me a BlackBerry Polygamist, it's the truth!!