Forums C++ Framework A very long story about my optimization of Cocos2d-x Android apps in size and performance

Strawberry
Milkshake
A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 79
Added by Strawberry Milkshake over 2 years ago
The back story:
As I wrote before in the Chirp Chirp topic, the situation with internal memory on Android is bad. On most devices, it's not bad, it's
just VERY bad.
The HTC desire is a very good example here. It contains 512 MB of Read-Only Memory (ROM) and 576 MB of Random Access Memory
(RAM). Even though ROM means "read only", it is where the Android OS is installed. It's also the location where the user interface is
installed, and the apps.
Getting rid of Android to increase the available memory for new apps is obviously not an option. The user interface (the UI on top of
Android, like HTC Sense) can not be fully removed without rooting the device. So that leaves a lot less memory available of those
512 MB. But hold on, it gets even worse: very useful apps like GMail (4.46 MB), Google Maps (11.30 MB), Android Market (6.05 MB)
and the Youtube app (1.70 MB) are all installed right there. They cannot be moved to the SD card, so that's more memory that's not
available for additional apps, like Cocos2d-x games.
In the end it means that on the HTC Desire, there's about 50 MB available on average. But there's more: it's not possible to use all
of that, because it must be shared with stuff like personal data (sms, email, etc) and most importantly, at least 30 MB of free internal
memory is ALWAYS required when the vendor comes up with a ROM upgrade to install a more recent or otherwise updated version
of Android.
So, on the HTC Desire, there's only an average of 20 MB available for actual user apps. I really hope this speech gets the point
accross, because, well, 20 MB is not much My point is: if you're a developer looking into developing apps for Android, you'll have
to limit the installed file size of your apps, because otherwise the user will simply not install it, or they'll remove your app much
faster when they need space to install new apps, making you less money when you're dependent on advertising, which seems to
be one of the few methods to actually make some money on Android.
But.. then there's Cocos2d-x. Cocos2d-x is an awesome library. No doubt about that. But on my system, built using NDK-r4, it's
about 2.47 MB for just cocos2d.so alone. That's without CocosDenshion, Chipmunk or Box2D!
Finding a way out:
Many people would just say: hey, just enable the "Move to SD" option and let the user install your app onto the SD card. Well, you
could. But it's only useful for your assets (images, sounds, etc). The actual app will not shrink in size, so when I made a nice little
game using Cocos2d-x, I found it to be at least about 4 MB in size. That's without the optimized ARMv7 code. With it, the size would
literally double up, even though it's an absolute requirement for some apps/games because floating point calculations will be MUCH
faster with it. Box2D, for example, will get a huge speedup, which you cannot ignore as a developer looking into bringing high
performance apps to the user. Fortunately, I didn't use Box2D for that nice little game.
The biggest problem with "Move to SD" is that each version of Android up to 2.2 will install native libraries (like the Cocos2d-x
libraries) into /data/data/appname/lib/libxxx.so , so that's why the "Move to SD" option is useless: the native libraries will stay on
the internal memory, not the SD card. Only with Android 2.3 and up, native libraries can be installed to the SD card, according to the
following url (as told by Dianne Hackborn, who is an Android framework engineer): http://osdir.com/ml/android-ndk/2011-
04/msg00415.html
Is there a way out of distributing Cocos2d-x apps for Android without taking away at least 4 MB of internal memory or more, from
the limited amount of ~20 MB user app memory? Fortunately, yes, there is
The solutions:
- Move to SD
As mentioned before, this is only part of the solution as it will only install java code and assets to the SD card, leaving the rest of the
(native) code inside the internal memory. Yes, it should be enabled, but no, it's not the solution for fixing the issues with limited
internal memory on most Android devices and making your users happier with your small sized apps.
- "APP_OPTIM := release" @ Application.mk
Now we're getting somewhere. With this option put into Application.mk, your native libraries will lose debugging symbols and all
that, but it will be much smaller. cocos2d.so (2.47 MB when built straight from Git) turns into a smaller library of 1.70 MB with this
option enabled.
- Turning Cocos2d-x into a static library
This doesn't have as much of an effect as the APP_OPTIM option, but it's good for another 0.20 - 0.30 MB off the library size. That's
almost 1/3 of one Megabyte indeed. You can make Cocos2d-x go static by performing the following rituals:
1. remove 'cocos2d' and other modules that should be static from Application.mk
2. in Android.mk, add LOCAL_STATIC_LIBRARIES := libcocos2d (just before the include $(BUILD_SHARED_LIBRARY))
3. change $(BUILD_SHARED_LIBRARY) to $(BUILD_STATIC_LIBRARY)
4. in the Tests or Helloworld demo, or your own app, you should link with the following libraries:
libz, libpng, libxml2, libjpeg, libskia
Those additional libraries you'll have to link to are normally linked into the .so file, but that won't happen with a static library. Keep in
mind that you'll probably have to swizzle the inclusion order of these libraries to get rid of linkage errors.
The result for all of this is a static Cocos2d-x library that can be linked into your own app's native shared library, so you'll end up
with just one library that needs to be loaded from the Java side of the code.
- GCC Visibility feature
Last, but most definitely not least, is the GCC visibility attribute. It's a bit like __declspec for MSVC, but for GCC instead. Using this
feature you can improve loading times for your native libraries and cut down on the exported symbol table, which can get as large
as 5-6 MB when there's 30,000 symbols. The compiler can also optimize your code a little better when it's enabled.
Here's some more information on how to use the feature, it's quite easy:
http://gcc.gnu.org/wiki/Visibility
http://goldenhammersoftware.blogspot.com/2010/12/android-ndk-and-storage-size.html
(the person from the last article talks about stripping away debugging information, but as far as I know that's not necessary once
you're using the release optimization in Application.mk)
The conclusion:
With all of these options it's easily possible to make a Cocos2d-x library about 2.3 times smaller compared to what you'd get from
Git. I don't know if the 3rd party libs (the binary versions distributed with Cocos2d-x) were compiled without these options, but if
they were, the results could be even better.
So here's the part where I started testing these wonderful optimizations. I modified the makefiles for my app, included the static
Cocos2d-x libraries, all combined into one shared object, and there I was: expecting to reap the huge benefits...
..well, no. Far from it. The actual APK file (the one you'd offer for downloading on the Market) had decreased in file size with about
1/3 less of the original size. That's great, but my initial goal was to make the app use less internal memory on the actual device, in
which I did not succeed Why? Dunno! The installation size (and I did move it to the SD card) was actually just as big as before
which I did not succeed Why? Dunno! The installation size (and I did move it to the SD card) was actually just as big as before
without all the optimizations!
So I guess we'll have to wait for Android 2.3, which does install native libraries to the SD card.
Thank you for reading this topic, and please, please do not spend six hours on it like I did But if you do, please do post your
results. Maybe I didn't do things the right way? Who knows...
ps. The APP_OPTIM option is quite good though. You (the Cocos2d-x team) might want to consider adding it to Git. It does a nice job
on cutting at least part of the fat away!
Zhe Wang RE: A very long story about my failed attempt to optimize Cocos2d-x Android apps in size and performance
Posts 1676
Location Amoy,
China
Added by Zhe Wang over 2 years ago
Thanks for your share!
Maybe we must consider to change .so to .a static library as default, and add APP_OPTIM to Application.mk
Keep it simple and stupid.
Strawberry
Milkshake
RE: A very long story about my failed attempt to optimize Cocos2d-x Android apps in size and performance
Posts 79
Added by Strawberry Milkshake over 2 years ago
The static library approach might be bad for installation size (yeah, I'm really worried about that :p ), but APP_OPTIM is awesome
Also, when I tried the GCC visibility feature, I only tried to hide everything with the exception of all the JNI C-functions because
those functions should be visible to Java. What I didn't try, however, is to use the visibility attribute like __declspec as in the
example on the GCC wiki page. It might do less in terms of decreasing the file size without turning everything into one big happy
static library, but it also might decrease the size just enough to make things a little better for the shared library approach while
keeping the install size as small as possible.
Strawberry
Milkshake
RE: A very long story about my failed attempt to optimize Cocos2d-x Android apps in size and performance
Posts 79
Added by Strawberry Milkshake over 2 years ago
More!
All righty, I went ever deeper to figure out how to make cocos2d.so smaller in size :)
The following libraries are used by Cocos2d-x: libxml2, libpng, libz, libjpeg and libskia.
Skia
Skia isn't really big in what it's used for with Cocos2d-x, it compiles to only about 0,01 Megabyte. Not worth the time to optimize.
Leave it alone. It generates nice and sexy fonts.
JPEG library
This is an interesting one. libjpeg is quite large, so it's a great candidate to look at when optimizing for installation size. Also, there's
the knowledge of image quality. JPEG images are great for saving and loading photorealistic images, but it's horrible for cartoony
images, images with gradients, clean images without much detail or few colors, and so on.
Now, most Cocos2d-x games aren't using any real photo's or photorealistic images at all, so as a game developer you're most
probably using Portable Network Graphics because this format supports images without any compression artifacts.
I know I will never use JPEG for an app, so for me, it's a great choice to disable loading of JPEG images altogether. Fortunately,
that's very easy to do with Cocos2d-x, just a few lines in the texture classes. Add some comments and return false or null and
you're done.
The results: well, cutting libjpeg away makes cocos2d.so go from 1,70 MB (with 'APP_OPTIM := release' only) to 1,60 MB. Not even
that much, but still, well worth the very small effort if you aren't using the functionality anyway.
Z library
This one is needed for libpng. Don't remove.
PNG library
Again, this library is needed if you'd like to load images at all. Can't be removed.
XML library
Ahh, this is a great one. libxml2 from the Cocos2d-x distribution is about 3,02 MB in size, which is.. well, really big. If you're not afraid
to do some dirty work by downloading libxml2, spending some time to dig out a bunch of Android.mk magic (use the Android OS
version as an example) and carefully craft a well fitting xmlversion.h to disable support for stuff like DOM tree manipulation (DOM is
used by Cocos2d-x, but the manip functions aren't), push parsing interfaces, xmlPattern, ftp/http, validation, legacy stuff,
canonicalization, catalogs, docbook, xpath, xpointer, xinclude, iconv, icu, debugging, memory debugging, runtime debugging, unicode
(not sure if that's good for China, Japan, etc. Probably not, need to experiment with it), regexp, automata, formal expressions,
schemas, schematron, modules and zlib support. Well, if you disable all of that, you'll end up with a static libxml2 library that's about
2,32 MB.
Together with the removal of libjpeg, a cocos2d.so linked with this optimized libxml2.lib comes down to about 1,53 MB in size.
It would be possible to cut another 0,01 MB away by disabling the SAX1 functions for libxml, but Cocos2d-x is using the
xmlSAXUserParseMemory function so that's not possible (even though Cocos2d-x seems to do SAX2 instead of SAX1).
xmlCreateMemoryParserCtxt is probably the function to use here (as xmlSAXUserParseMemory does that as well), but like I said
before.. it's not worth the time to optimize for only 0,01 MB.
The end (?)
As it is now, my local version of cocos2d.so went down from 2,47 MB to 1,53 MB.
I'm really happy, because the APK for my little game also went down to 0,9 MB (libraries are zipped into the APK), meaning that the
physiological barrier to download my game on a capped connection to the web just went down as well.
More users, more happy users because of the smaller installation size = awesome!
I hope this information will be useful to other developers!
Zhe Wang RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 1676
Location Amoy,
China
Added by Zhe Wang over 2 years ago
Thanks for your awesome work! Finally you success on optimize it!
I modify the title a little, and add your article into sticky FAQ in the forum http://www.cocos2d-x.org/boards/6/topics/567 It will be
VERY helpful to other developers!
Keep it simple and stupid.
Carlos Sessa RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 7
Added by Carlos Sessa over 2 years ago
Excellent post!
Here the real link for Dianne Hackborn's post: http://groups.google.com/group/android-
ndk/browse_thread/thread/94986c853be60709/5bf27165b1482237
Thanks for taking the time to write this.
Laurens
Rodriguez
RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 26
Added by Laurens Rodriguez over 2 years ago
very very useful;, thanks!
Strawberry
Milkshake
RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 79
Added by Strawberry Milkshake over 2 years ago
Thanks
And oh, silly me... I forgot about the most obvious optimization of all
I used to use the Crystax version of the NDK (like said, r4). However, it does come with a slight overhead. Now if you're using
Crystax as well, please consider moving to the official NDK. I just did that, and what do you know.. the size of cocos2d.so went
down even more, to a nice size of about 0,99 MB (!).
With this new smaller cocos2d.so, the size of my little app's APK is... 0,75 MB. I guess I'm done with it now
Dan Zurz RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Added by Dan Zurz over 1 year ago
Posts 2
Added by Dan Zurz over 1 year ago
I`ve got some problem turning cocos2d-x into a static library as described above.
My test app builds and works well until I try to use some "bridge" features (touchscreen or keypad for example). In this case the
app crashes due to "No implementation found for native".
Something like that:
01-26 18:26:20.220: W/dalvikvm(22860): No implementation found for native
Lorg/cocos2dx/lib/Cocos2dxRenderer;.nativeTouchesBegin (IFF)V
01-26 18:26:20.260: W/dalvikvm(22860): threadid=11: thread exiting with uncaught exception (group=0x4001d648)
01-26 18:26:20.260: E/AndroidRuntime(22860): FATAL EXCEPTION: GLThread 12
01-26 18:26:20.260: E/AndroidRuntime(22860): java.lang.UnsatisfiedLinkError: nativeTouchesBegin
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.nativeTouchesBegin(Native Method)
01-26 18:26:20.260: E/AndroidRuntime(22860): at
org.cocos2dx.lib.Cocos2dxRenderer.handleActionDown(Cocos2dxRenderer.java:74)
01-26 18:26:20.260: E/AndroidRuntime(22860): at
org.cocos2dx.lib.Cocos2dxGLSurfaceView$7.run(Cocos2dxGLSurfaceView.java:302)
Any suggestions?
P.S. I`m really sorry for my English (
Michelle
Sangster
RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 6
Added by Michelle Sangster 10 months ago
Dan Zurz wrote:
I`ve got some problem turning cocos2d-x into a static library as described above.
My test app builds and works well until I try to use some "bridge" features (touchscreen or keypad for example). In this case
the app crashes due to "No implementation found for native".
Something like that:
01-26 18:26:20.220: W/dalvikvm(22860): No implementation found for native
Lorg/cocos2dx/lib/Cocos2dxRenderer;.nativeTouchesBegin (IFF)V
01-26 18:26:20.260: W/dalvikvm(22860): threadid=11: thread exiting with uncaught exception (group=0x4001d648)
01-26 18:26:20.260: E/AndroidRuntime(22860): FATAL EXCEPTION: GLThread 12
01-26 18:26:20.260: E/AndroidRuntime(22860): java.lang.UnsatisfiedLinkError: nativeTouchesBegin
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.nativeTouchesBegin(Native Method)
01-26 18:26:20.260: E/AndroidRuntime(22860): at
org.cocos2dx.lib.Cocos2dxRenderer.handleActionDown(Cocos2dxRenderer.java:74)
01-26 18:26:20.260: E/AndroidRuntime(22860): at
org.cocos2dx.lib.Cocos2dxGLSurfaceView$7.run(Cocos2dxGLSurfaceView.java:302)
Any suggestions?
P.S. I`m really sorry for my English (
I got this problem too, have you solved it?
and could you tell me how to solve it,thanks.
Michelle
Sangster
RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 6
Added by Michelle Sangster 10 months ago
Michelle Sangster wrote:
Dan Zurz wrote:
I`ve got some problem turning cocos2d-x into a static library as described above.
My test app builds and works well until I try to use some "bridge" features (touchscreen or keypad for example). In
this case the app crashes due to "No implementation found for native".
Posts 6 this case the app crashes due to "No implementation found for native".
Something like that:
01-26 18:26:20.220: W/dalvikvm(22860): No implementation found for native
Lorg/cocos2dx/lib/Cocos2dxRenderer;.nativeTouchesBegin (IFF)V
01-26 18:26:20.260: W/dalvikvm(22860): threadid=11: thread exiting with uncaught exception
(group=0x4001d648)
01-26 18:26:20.260: E/AndroidRuntime(22860): FATAL EXCEPTION: GLThread 12
01-26 18:26:20.260: E/AndroidRuntime(22860): java.lang.UnsatisfiedLinkError: nativeTouchesBegin
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.nativeTouchesBegin(Native
Method)
01-26 18:26:20.260: E/AndroidRuntime(22860): at
org.cocos2dx.lib.Cocos2dxRenderer.handleActionDown(Cocos2dxRenderer.java:74)
01-26 18:26:20.260: E/AndroidRuntime(22860): at
org.cocos2dx.lib.Cocos2dxGLSurfaceView$7.run(Cocos2dxGLSurfaceView.java:302)
Any suggestions?
P.S. I`m really sorry for my English (
I got this problem too, have you solved it?
and could you tell me how to solve it,thanks.
I got it solved.
It is because the native fuction you defined is call in the java side and because it is not accessed in the c++ side so the compiler
won't link it to the so.
you should pretend to access them at some place of the c++ side to force it link to so.
Minggo Zhang RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 1714
Added by Minggo Zhang 10 months ago
You can link the .a by using LOCAL_WHOLE_STATIC_LIBRARIES in Android.mk, then it will not remove the functions that are not used
in c++ codes.
santosh kumar RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Posts 4
Added by santosh kumar 9 months ago
very very useful
Sameer z RE: A very long story about my optimization of Cocos2d-x Android apps in size and performance
Added by Sameer z 8 months ago
Hi,
Thank for great info,can you please write code for disable libs.
What things need to write in Android.mk to make it work.
Because now iPhone app size is 60 MB and on Android apk is 107 MB.
If there is any other ways are then please tell so that we can also decrease the size of apk.
Posts 34
If there is any other ways are then please tell so that we can also decrease the size of apk.
Thank you.
(1-13/13)

Reply