BBM Stress Test - The Aftermath

We invited. We accepted. BlackBerry Messenger on BlackBerry 10 conquered. 

By Kevin Michaluk on 3 Apr 2013 10:44 am EDT
0
loading...
0
loading...
89
loading...

The long awaited, much anticipated BlackBerry 10 BBM Stress Test is officially over.

Conclusion? BlackBerry 10 running on the BlackBerry Z10 passed the test with flying colors. We absolutely slammed our Z10 with BBM invites. From there, we accepted the requests until we hit BBM's max capacity of contacts, eventually capping out as high as 2072. 

Once the invite dust settled, we spent a day using the Z10 under "normal conditions" and the usability and performance of the device was unaffected. Everything worked 100% as it should. The only difference was that with over 2,000 eager to chat contacts now on our BBM list, our productivity dropped to zero as we spent the day talking and putting BBM Video Chat to use. :)

Compare BlackBerry 10's performance to the two times we tried the BBM Stress Test before on the BlackBerry Bold 9000 and 9650 and there is no comparison. In the past, this test rendered the phones useless - both during and after the test. 

Kudos to the BlackBerry software guys, for building a mobile computing platform that is stable even when under "attack". 

The Initial 'Attack'

You can almost think of the BBM Stress Test as an "attack" of sorts on BBM. By preannouncing the April 1st, 12 noon EDT start time on CrackBerry, we literally had thousands of awesome readers lined up and ready to take part in the experiment. 

It was 12:04pm when I publicly posted my Z10's barcode and PIN to this CrackBerry post. Within seconds, the BBM requests started pouring in. The BBM invite count immediately began counting up sequentially at a rapid pace. It was so fast the notification sound you would normally hear didn't sound (though on occasion we would hear a chime or two).

In reality, it appeared that we had literally thousands of people add us to BBM at the same moment, but to our Z10 the invites came through in metered fashion. In other words, we didn't jump from 0 invites to 2,000+ immediately, but they counted up one at a time. In just a few minutes we passed 1,000 invites. And from there they continued to roll in, passing the 2,000 mark.

We had previously heard that BBM on BB10 had a cap at 2,000 contacts, and that any requests beyond that would simply be ignored. That wasn't totally the case though. The invites continued to rack up. 

Kudos to the software guys for building a platform and BBM app that can finally handle a CrackBerry-style BBM Stress Test!

At that point, we began approving contacts one by one, starting with the first ones that came in at 12:04pm. This is a tedious process as there is no "Approve All Pending Contacts" option on BBM. In hindsight, we *could* have run things smarter to make the approval of contacts automatic. Had we only posted the our barcode on the web, forcing people to scan the barcode from their BB phones (vs. adding via pin) and had we left our Z10 on the BBM display barcode page, then all of the contacts would have been added automatically. We would have been simulating being together in the same room. We did take advantage of this option for a while and it worked as more invites continued to roll in, but after a while adding contacts via this manner didn't seem to be going as fast as approving them manually, so we went back to adding people from the start. 

We did a live webcast of the entire process, with Bla1ze and I taking turns approving contacts. It took a few hours to tap our way up to over 2,000 contacts.

During this initial "attack" phase, we chewed through the battery on the Z10. Keep in mind we also had in our hand the whole time, with the display on full brightness outputting to HDMI on a tv, but there's no doubt this kind of inbound attack keeps the radio cranking at full effort. The phone got pretty hot. We went from a fully charged battery to a dead one in under two hours. 

During this phase, the phone remained completely usable. We browsed web pages with ease. We even did a few BBM video chats while the BBM invites continued to pour in. 

We did experience a few glitches of course. At one point we hung up on a BBM Video Chat, but the phone thought the call was still ongoing - even with every app closed on the phone, the little phone call time indicator on the homescreen continued to count up, and other people trying to call us could not get through. We eventually had to reboot. 

The Hub also quit on us at one point and didn't want to restart after restarting with the magical five swipe gesture.

I sort of expected to see BBM crash and out close a few times, but it didn't

BBM remained surprisingly stable and fast throughout the process. Bla1ze and I were both surprised. I sort of expected to see BBM crash and out close a few times, but it didn't. Scrolling on the screen remained quite fast. There was definitely some "lag" in the experience, but this felt more like lag on the data transfer in and out of the phone as there was simply such a backlog of stuff happening vs. lag on the phone itself. For example, when changing our BBM status, once changing it and returning to the main contacts page, it could take 10 or 15 seconds before the new status showed up. 

Under 'Normal' Use

BBM Stress Test

Once we maxed out our contacts, the invites continued to roll in. Eventually the invites slowed down and we were left with thousands more invites that we couldn't accept. Well, that's not *quite* true. We can still open up invite requests and hit Accept, but nothing happens. New requests will come through too.

With the attack over, the phone settled into normal rhythm again and was completely usable. We created BBM groups. We sent some broadcast messages to hundreds of people. We used video chat, etc. BBM ran fine. And the rest of the phone continued to run fine too. 

Even battery life on our test Z10 has been OK, though in this case we really only have one email account connected to the phone and our maxed out BBM list. We're going to keep the phone rocking and using it as we normally would - installing apps, hooking up more accounts, etc. But overall, the Z10 is holding up strong.

The Verdict

Kudos to the software guys for building a platform and BBM app that can finally handle a CrackBerry-style BBM Stress Test!

With a ~2000 contacts, BBM runs just fine on BB10. 

If you have 2,000 people you want to add on your BBM list, our recommendation would be to not add them all at once as we did here. The "attack" is the hardest part of the Stress Test on the phone. If you add your contacts organically, which should be the normal case for 99.999999999% of users, you shouldn't run into any issues BBM'ing to your heart's content. 

Huge props and thank you go to everybody who participated in the test! The CrackBerry community never ceases to amaze me - you're all awesome. Seriously, we had a 1,000+ people tuning into the live stream for hours as we conducted the initial experiment. Just amazing. 

It was such a good time, I'm thinking we may just have to do it again when the Q10 comes out. Sound like a plan? :)

Reader comments

BBM Stress Test - The Aftermath

117 Comments

I didn't get accepted this time around. Mind you, I was quite late with my request. But I'm gonna try to do it on time next time around. When you test the Q10!

Posted via CB10

Good stuff Kevin to show that unlike the older BB's, this time the phone didnt go kaput on you guys. I did send an invite but it was quite a few hrs late...left it till the next day but didnt get an acceptance so i just left it.

But well done guys, shows that the new BB10 platform is far more efficient than the older BB devices.

I did state that the Z10 was fireproof and so it proved. No melted CPU or fried display in sight. Great test! The Z10 is an awesome beast! : )

Posted via CB10 using my awesome BB Z10

Hey Kev
The Event Log on the OS7 and below @ the home screen press ALT+LGLG. By any chance do you know how to access it on the Z10?

It was cool to be one of those 2072. Even better that the device held its own. I wonder if this same test was done a on an I phone how would it fair. Maybe one day the I more gang will put their beloved device to this test.

Posted via CB10 on my Z10

I second this motion. Let's see the other platforms perform a stress test. Ios, android and WP8, who wants to step in the ring with the champ?

I don't always comment on CrackBerry but when I do its on my Zader10 via CB10

Tried at 12:10 didn't get accepted but still awesome. Had to work so at least tried. Awesome guys. Always here if you need video help.

Need this thing for any further live events for users to call in live. Completely new show dynamic and engagement.

Posted via CB10

Gotta love the Z10 passing with flying colours and we just get news that iMessage can cause the iPhone to crash under similar stress.

It was a great experience and fun to watch and listen. I had to opportunity to grocery shop and listen to the live event on my Z10, then watched on my PB at home later. My daughter even BBM called (without me knowing until she was already taking with Kevin) and did it on her Curve. What a great test to the usability of the Z10's and hopefully great success for BB.

Great job on the test and thanks for letting me be apart of it! Thanks to you and the CrackBerry team for all that you guys do for TeamBlackBerry

Posted via CB10

Yes please do it on the Q10 because I did not get accepted when I deleted my request to try a refreshing add.

Great work Kevin and Blaze!

I was amazed to see an hour into approving requests, the"request time" was still 12:04! That was a huge number of people jumping in right as the test started!

This test says a lot about how well BB10 handles things.

Was fun participating in the stress test. I got approved. And you even responded to a message of mine that evening(or night your time), that in itself speaks to the usability of it.

Maybe by the time the Q10 roles out we can learn some tricks and plan it a little better. I think everybody was expecting it to just crash in the back of there minds. Nobody was prepared for the need to stress it out more then the previous tests. Auto approval, maybe we can all message the device at the same time or something.

Kudos to Dan Dodge and the QNX folks for building a RTOS that can handle a heavy load on small hardware.

Yes please so I can make a second attempt at getting in the list!

Posted via CB10 on my Zed10

I wondering if you delete an existing contact after hitting the max contacts will a previously approved contact automatically get added?

Just a thought Kevin don't know if you tried this.

I sent a request and was accepted!...and than I went and upgraded to a Z10 last night. No more BBm friends with Kevin. Bummer man.

Confirms my own experience with the Z10 in the past 8 weeks. OS has not crashed even once. Battery life keeps improving.

Posted via CB10 on BlackBerry Z10

I got accepted almost right away but none of my messages were ever delivered. I got the red X. In all I sent about 6 messages. A smiley here a smiley there. I asked if the Z10 blew up lol but as I said they were never delivered. I tried a resend on day 1 and 2 but no dice.

I would like to see the BlackBerry software guys surprise us and increase the max contacts limit before your test on the q10. It's probably a setting on the backend rather than on bb10 itself.

Posted via CB10

Still, battery life is a mess. No problem in the office with wifi, but traveling through a city or through the countryside (Hanoi and 4 h south which os largely covered by G3) depletes my battery within ca. 10 h. This is unacceptable particularly compared with the 9900 which still has 50% juice even though I keep a GPS tracker running all tje time. BB please improve battery life with prime priority, otherwise I can basically not use the Z10 during business and have to switch nack to the 9900.

Posted via BlackBerry Z10 (24C682D2)

Also, to Kevin's point, I do have a picture of my car as my avatar, it just didn't come across with the swath of invites hitting the phone.

Huge props to me for hanging out through the whole broadcast. Huge props to the Z10 for exceeding my outermost expectations. I hope the Q10 fares as well, but now my expectations are a lot higher.

So...whatever happened to that netflix facebook page 'attack' that kevin started during the live stream?

Posted via CB10

I'm thrilled to see the results of this stress test and simultaneously heartbroken that I did not get added so that Kevin and I could be best friends.

One of 2045 contacts with 3877 requests pending?! No wonder my message is stuck on D and not R!
Good effort Team CB! Looking forward to the seeing the Q10 handle the stress...

That was great man, it was a lot of fun to watch and participate. BB10 and the Z10 proved to be a hard berry to crack even for crackberry attack! I guess BB will be taking this and shouting it out, at least they should... not even Chad calling managed to get the Z to surrender lol! :D ( Go Chad!!! :D --- just kidding :D).

Great Stuff!

Next time should only be Z10, Q10 or newer phone so people actually buys a newer phone to join.

Posted via CB10

That was fun! If you do a Q10 test, make sure you keep the QR code screen open >.> lmao
First non-bricked BB! :D Keep this one around for call-ins or something :D I don't suggest doing support with screen share haha

@Kevin, Re: Lag when changing BBM status -- I actually got updates that looked like 'while you were typing' updates -- there would be some typo, then some backspacing, then a bit of correction, more backspacing -- got 5 or 6 updates with the 'BBM Stress Test - Dinner Break' in various stages of completion. Took a screen shot, tried to send from BBM during the test, if you can find it.

Is anyone else wondering where Kev got the cool BB graphics for his laptop, or is it just me?

it just shows you how incredible this new OS is. I bet Android and IOS Imessage wouldve crashed on immediately lol

I was too late for the BBM party, but I'm glad the Z10 proved to be a beast! I'll send my request early for the Q10 test. Good job, Kevin and Bla1ze! :-)

Pretty Crazy!

But with QNX real time operating system, I may have thought that the constant phone fluidity was predictable. Has I understand it, each process get their own separated allocation. So things like gesture control, hub management,... should remain mostly unaffected by any out of control running apps. So for this instance, "BBM attack" remain under BBM process allocations, with only few lagging due to remaining communication bandwith.
On previous legacy devices (and most of the other phones out there) this is normally "Queing" all yours requests which in that case may just monopolized the operating system, making other request simply not responding.
So thanks to QNX !

What should be the maximum number of contacts (not only BBM contact) that this OS 10 should be able to support ?
During my first days with my Z10 I had over 9000 contacts (containing multiple phone numbers, email address, and physical addess) all sync from my Outlook database. And it was just running as smooth as now (with my short contact list of around 1500 contacts).

Hey Kevin,

You should convince imore and android central teams to do this stress test with their platform's message service of choice.

It would be really interesting and a lot of fun.

You could even award a picture of a trophy to the winning platform site that the other sites would have to link to from their front pages. Let the spirit of competition live large!

Come on! You know you wanna! :-)

Posted via CB10

I was lucky, maybe it was the strength of the Rogers network? But I sent an invite right when his PIN went public and was accepted within a minute. And all my messages went through. Love the CrackBerry team!!!!

Posted via CB10

BB10 is quite remarkable. Dis test should be published on oda tech sites so more people can know abt it...

Posted via CB10

That's what I like to hear guys, I'm loving the new BlackBerry Z10 and can't wait for all the new apps to come along... let's start computing...

Posted via CB10

This test shows the power of QNX. Although there were a couple of hiccups, the OS functioned as designed: it kept threads separated and operating individually. It does look like the processor itself might have been stressed but the OS was not.

very cool,,, i'm big on my devices being able to perform on "the high end",,, atta boy to you guys & BB for testing/building a "runner",,, "I'm thinking we may just have to do it again when the Q10 comes out." = do it,,, put that bigger battery to the test...

I was a part of history. Along with #teamBlackBerry Thanks Kevin. ;)

Q10 shall be tested too ;)

Posted via CB10

I got accepted! It was about two hours later when I got the confirmation. Very happy to see the phone do extremely well. This phone truly rocks!
Keep Moving

Posted via CB10

That was a ton of fun and ill be there for the Q10 test too.

For anyone out there that doesn't realize this new platform truly is a complete revolution over the old one this test surely helps explain the massive progress!

Loving my Z10!

Posted via CB10

Great test. Was accept right away via the "auto-accept method".
Will be there for the Q10 test !!