Performance Benchmarks with AIR 2.7 for iOS

I’ve been working on this Benchmark based on Iain Lobb’s BunnyMark. Being a bit confused sometimes about what things speed things up or slow things down, I didn’t want to guess anymore, so I grabbed Iain’s code base (cause I’m lazy, and didn’t want to start from scratch), and added some tests for things I suspect are slowing things down (or speeding things up). I think this will also help shed some light on why some folks see a huge gain in AIR 2.7 CPU mode, while others do not.

Some caveats – this only tests instances of flash.display.Bitmap on the display list, at the size they are, moving the way they move. It’s on my list to add Blitting (I have some initial work on that done, thanks to Iain, but I need to add the rotation, and alpha settings to it), and I’d like to add a vector test, and maybe some extra sized Bitmaps (I’ve heard that makes a difference).

Enough! Here are some results – quality had no effect on GPU mode, so I included only one line:

Note: some are reporting they see a difference in GPU mode, but I still don’t. Update: It appears some users are confusing “Mobile Performance Tester” with BunnyMark, which explains the discrepancy. BunnyMark is not currently in any App Store, which is one key distinguishing feature. 😉

BunnyMark Results – 500 Bunnies
Alpha
Rotation
CaB
CaBM
iPhone 3GS – GPU
FPS 24 18 17 22 13 13 19 1 19
iPhone 3GS – CPU
FPS-L 28 21 19 9 19 5 7 5 5
FPS-M 28 21 19 4 18 3 7 5 3
FPS-H 28 21 19 3 18 2 7 5 2
FPS-B 28 21 19 3 18 2 7 5 2
iPhone 4 (Retina) – GPU
FPS 25 21 20 25 13 13 16 0.5 16
iPhone 4 (Retina) – CPU
FPS-L 32 23 20 10 21 6 8 6 7
FPS-M 32 23 20 5 20 3 8 6 3
FPS-H 32 23 19 4 20 2 8 6 2
FPS-B 32 23 19 4 20 2 8 6 2

Notes about the Benchmark:

  • In general, the CPU mode seems pretty consistent with the way you’d expect things to work on the desktop – the same optimizations you’d apply for the browser plugin, you’d also apply to mobile for CPU mode.
  • Rotation in this benchmark is not continuous – the Bunny graphics are only rotated at the edge of the stage, which is why cacheAsBitmap works to speed those up. If they were constantly updated, it would likely be much more expensive on CPU mode (probably more like rotation without CaB).
  • Alpha is continuous – the alpha value of each Bunny is based on the y position and is updated every frame. I would like to add a mode similar to the rotation, so see what effect CaB has on alpha transparent objects that don’t constantly change.
  • iPhone 4 and 3GS numbers aren’t directly comparable for practical purposes. The Bitmaps on the screen on 3GS take up much more real estate, since the 3GS screen res has 1/4 as many pixels as the iPhone 4. In a normal app, we’d probably resize things to look comparable between the two devices. I’ll try to add a mode that makes this more comparable (because I suspect we’ll find that 3GS can keep up with iPhone 4 with similar looking content).
  • Touching the screen seems to cost about 4 fps across the board.
  • I think there may be an issue with returning to rotation = 0 costing some performance in GPU mode. Still have to test that.
  • I’m definitely getting some variance on default speeds – basically, before any settings are messed with on some runs I get the faster numbers (the baseline numbers in the tables above). Other times it runs at default settings a couple of FPS slower (on start, or after resetting the switches). With any of the settings, everything is consistent across multiple runs.

 

It’d be nice to have more benchmarks for more devices, but I only have the above devices available. This should run just fine on Android, Blackberry Playbook, and iPads. If anyone wants to contribute a set of benchmarks, hit the comments. Here is the source. One of these days I’ll make another post, and try to draw some conclusions, maybe wrap the bullet points into a narrative, and edit some of this, but the tables are there, and the source code, and that’s the important stuff.

In the midst of playing with this benchmark, I found (or was pointed at) some great resources. Here are some of them:

Here is the Benchmark to see it in action:

unBrix Alpha in Android Marketplace!!

My first Android app is in the market place! Built with Adobe AIR, unBrix Alpha is a quick take on the classic breakout style game. This is more of a “lite” game at this point (hence the “Alpha” suffix), but it is already more complete than many of the other Arkanoid clones in iOS App Store. There also seems to be some last minute performance problems on the Android version. :-/ I guess that’s what I get for only testing on an iPhone for most of the development. I’ll have fixes to that soon. I’m pretty sure it’s related to the scaleMode I set in Flash – the problem is if I set that the faster mode – NO_SCALE – it’s way too small on most Android devices. I’ll probably need to add some manual sizing based on measurement. Of course non of this was needed on the iPhone version.

Download unBrix Alpha and let me know what you think! I’d provide a link, but I don’t know how.

Update: I Nerfed the framerate a bit to get it to run a little smoother. I think the problem will be solved better by setting NO_SCALE, but I’ll have to do that another time (probably when I get to iPad port!). I also fixed the red line, and the icon too (I don’t know that didn’t show up last time). There is a report of the paddle jumping to one side when some users remove their finger from the screen. I haven’t been able to reproduce, but please let me know if this happens to you! Here is a link to unBrix Alpha on appbrain (it isn’t showing the update yet).

Update 2: I switched to CPU rendering, because it seems as though GPU rendering is just slower on Android devices than CPU rendering – at least in this kind of game. Anyway, this solved a lot of problems, including missing text and missing affects. I also had to set a fullScreenRect to match the original intended size of the game (iPhone 3Gs size). Doing these two things cleaned up most of the performance issues and graphics glitches. I’ll work on getting the remainder of the basics in place, like proper shutdowns – so this doesn’t run in the background like it does now (didn’t have to worry about that for iOS!).

Flash iPhone Game at Silky 60FPS on 3GS

Well, it’s only a tech demo at the moment. I’ve been playing with this Breakout like game for a while, trying to learn the ins and outs of Flash mobile development – particularly as it relates to performance. I now have the unBrix demo running at close to 60FPS (59.1) – smooth as silk.

This won’t run at 60FPS on Android Flash Player plugin in the browser (or Firefox on Mac!) – this post is about the iPhone build – but here’s the web version to look at anyway.

Here is a blurry video of the thing running as a native iPhone app on a 3GS (I smoothed out the choppy splash transition in a later build by setting the BG element with cacheAsBitmapMatrix):

The most important thing was to make sure GPU acceleration was working, and to learn what things will impact performance in that area.

It turns out, there are some important differences with how GPU accelerated Flash woks compared with the traditional software renderer. In the software Flash renderer keeping your display list shallow, and sparse (using addChild/removeChild a lot) or avoiding the display list completely (by writing to BitmapData – as the Flixel game engine does) is a key optimization for performance. This is how the exploding bunny video demo is done, and why it’s so fast.

My current theory is that on GPU accelerated content (even on desktop) the reverse is true. You want to avoid CPU/system RAM to GPU/video RAM updates as much as possible – which means avoiding BitmapData updates which cause the player to upload a new texture to the GPU VRAM with every update. Because I don’t have access to the internals of the Flash Player architecture, I can’t be sure, but I think the bottle neck comes from clogging up the lanes between the CPU and GPU, and all stressing all three areas of the rendering pipeline (CPU, GPU and the bus) as they juggle around objects in memory. The key observation this conclusion is based on is the large performance impact addChild and removeChild has on the framerate. So I relentlessly avoid that in my iPhone Flash development – I precache everything, and don’t mess with the display list. This is also one reason why filters (which operate on a BitmapData representation of the DisplayObject you apply them to) are not recommended on mobile content.

Anyway, hopefully I can turn this into a full app for iPhone in a reasonable timeframe. 🙂

Frash shows Flash on iPhone can be Great (with Screenshots)

Warning: RANT ahead

Steve Jobs is full of crap. I could actually understand and respect a straightforward admission that the Flash Platform is a threat to Apple’s iOS business model – which is the real reason Jobs won’t let Flash on the iPhone and iPad. That’s not even a very good reason – the App Store has many compelling features on it’s own, even if Flash is in the browser, not the least of which is the easy to understand path to monitization. Performance is another issue – Flash is fast enough, faster than JS/Canvas by quite a bit, but it’s still not as fast as a native app and all it’s OpenGL goodness (among all the other great Apple APIs). Keeping Flash off iPhone (and especially CS5 iPhone apps) has nothing to do with performance, or compatibility. That’s just bunk! And nobody likes a liar.

/rant

Here are some screenshots to show how well many sites I’ve been involved with work in Flash on the iPhone (3GS using jailbreak and Frash).

A quit note on the technology: this is using Frash, which is a hack (a wrapper or compatibility layer) to get the Android version of Flash Player 10.1 running inside of iOS – it has not been optimized (or even completed at this point) to run well on iOS yet, and probably will never run as well as it can on Android. A recent test app I’ve been playing with runs 50% faster on a Droid Eris, vs. the iPhone 3GS – the Eris is slower hardware, running Android 2.1. It’s also crashy, and is missing features like streaming movie support (it does work with videos embedded in a swf) – and touch events are not quite as streamlined as they are on a real Android device (hover works better on Android for example, and hot spots are easier to hit). It’s also got all the quirks of the Wii and desktop Flash Player’s “noscale” feature (on Android there is a workaround that solves this, not implemented in Frash yet).

I just like to point that out, because some users are judging the viability of Flash on iOS/iPhone/iPad based on this hack (which is current at version 0.02), which is beyond silly.

For anyone interested, I followed these instructions to install it on my iPhone. Note: this uses SSH and dpkg to install. If you don’t know how to reverse that, you may want to find an apt repo and use Cydia to grab this, so you can uninstall Frash when you are done playing, as it can be quite unstable.

The Bunny (Video) Explodes. Explodes!

UPDATE: I intended to post the source for this a long while ago (after cleaning it up), but I never got around to it. Here it is in it’s current state.

I wanted to see how far I could push that exploding Actionscript 3.0 code – see if Flash could handle updating each animating pixel every frame, while playing a video, then blurring it. Sure enough, it can! It did take further optimization from the version I posted the other day – including swapping copyPixels with getPixel/setPixel, and removing an anonymous function call (wow that was expensive!). Here it is:

Note: This WILL run like slush on the debug player. I don’t know why. If anyone knows why, please let me know!