Creating Killer Mobile Apps

The keys to developing, testing and launching mobile applications that your users love

A Free eBook from uTest

Creating Killer Mobile Apps

Introduction................2 The Mobile Boom...................4 Mobile User Growth Rate......................................4 Why Mobile? ...........................................5 Whats at Stake? ........................................6 Mobile App Strategy...............7 Mobile Web vs. Native Apps..............9 What the Users Say ..............11 Alternatives: WAC and Hybrid Apps .............12 Operating Systems, Devices and Carriers...13 Mobile Testing Types, Tools and Methods..................16 Functional Testing..............17 Usability Testing..............18 Performance Testing................19 In-House Testing .............20 Outsourced Testing ..............21 Emulators & Simulators...........22 Beta Testing............................23 Key Testing Challenges....................24 Crowdsourced Testing..................27 How It Works.......29 Challenges .............30 Benefits....33 Appendix: Key Players in the Mobile Space......35

Table of Contents

Page 1

The Challenges of Launching Great Mobile Apps

Launching a killer app is like catching lightning in a bottle and this is especially true of mobile applications. In an industry where there is fierce competition for media attention, VC investment and/or consumer dollars, even the smallest flaw can doom a mobile app (and the business behind it). With so much at stake, how can so many mobile app companies ignore the benefits of usability testing, functional testing and load testing? They want to launch apps that delight their users, right? This eBook will walk you through the challenges of creating a high-quality mobile application, and will show you how to overcome them. We will demonstrate how companies of all sizes from startups to enterprises can overcome these obstacles to build apps that their users love. In doing so, well also cover: The Mobile Boom: Learn whats at stake and also whats driving the worldwide mobile application frenzy. Mobile App Strategy: Well explain how building great apps begins long before code is written or tested. Mobile Testing Types, Tools & Methods: Emulators vs. real testers; the differences between mobile development platforms and other critical, strategic decisions that will need to be made. Key Testing Challenges: How to navigate through the peaks, valleys, twists and turns of testing your mobile application. Crowdsourced Testing: Discover how tapping into the crowd can help you launch a killer app, while getting you to market faster and on-budget. Key Players: If youre new to this mobile apps game, this appendix will highlight the companies, technologies and players that you need to be familiar with before getting started.

Mobile App Challenges

Page 2

THE MOBILE APP BOOM

Page 3

The Mobile Boom

To say that the use of mobile applications is booming would be an obvious understatement. In 2010, mobile app stores will recorded an estimated $6.2 billion in overall sales, including 4.5 billion app downloads. By 2013, some analysts expect mobile app revenues to exceed $21 billion. With charts like this, we wouldnt bet against them. As they say in the investing world, the trend is your friend. So if you think that mobile apps are for kids or that it wont affect your industry/company/team, think again!

The Mobile Boom

Page 4

Why Is The Mobile Market Booming?

The accelerating shift towards mobile applications (and the smart phones they run on) is really quite simple, but frequently overlooked. Here are just a few of the reasons why users are flocking toward this technology: Accessibility: With mobile devices, the web is always at your side. This includes email, social media, video, games, news, stocks, sports, shopping and other actions. The ability to stay connected 24/7 has perhaps the most widespread appeal of all, and is the number one reason why the use of mobile applications and devices show no signs of slowing down. Adaptability: Mobile applications are sometimes thought of as being used primarily by kids and consumers only, but this couldnt be further from the truth. Business professionals were, in fact, the earliest adopters of this technology, and continue to make up a large segment of the overall user base. This can be seen in the widespread use of business & productivity apps for CRM, ERP, documents, presentations, email and messaging tools. Of course, you can also play DoodleJump, too. Efficiency: Mobile apps (and to a lesser extent, the mobile web) have greatly enhanced the efficiency of our everyday online activities. As we will explain shortly, the goal of any mobile app should be to reduce each feature to its most basic component. For example, users who want to see the latest market news will generally download an app that focuses on this exclusively. They do not want to enter a search term and click through five pages to see this information. Mobile applications, in other words, cut out the middleman and give users direct access to the things they want most.

The Mobile Mantra

Much as we love freedom and choice, we also love things that just work, reliably and seamlessly. And if we have to pay for what we love, well, that increasingly seems OK. Have you looked at your cell phone or cable bill lately? - Chris Anderson, Wired

The Mobile Boom

Page 5

Whats At Stake?
Still think that mobile apps are for games, music and entertainment purposes only? Do you really think that mobile applications wont be relevant to your company, your team and your job a year from now? Think again. As the adoption rate increases, mobile applications will become ever more critical to the success of your business (and your career). Those who insist on waiting for further confirmation will have already been left behind. Companies that recognize this trend for what it is a technological revolution and take preemptive action to improve the quality and usability of their mobile applications, will find themselves with a tremendous advantage. Most companies will simply build a watered-down version of their website. But those who understand the implications of the mobile boom and who understand all of the feats this technology is capable of will build applications with enduring popularity. Which side will YOU be on? This mindset is essential. In order to take the steps required for developing and launching a quality mobile app, you will need to believe in what youre doing. If you think mobile apps are simply a passing fad, it will be reflected in your businesses and products. Your users are counting on you. Dont let them down.

The Mobile Future

According to a Morgan Stanley analyst, the world is currently in the midst of the fifth major technology cycle of the past half a century. The previous four were the mainframe era of the 1950s and 60s, the mini-computer era of the 1970s and the desktop Internet era of the 80s. The current cycle is the era of the mobile Internet, she says predicting that within the next five years more users will connect to the Internet over mobile devices than desktop PCs. - GigaOm

Mobile Testing Challenges

Page 6

STUFF YOU NEED TO KNOW FIRST

Page 7

Your Mobile App Strategy

One of the first and most critical decisions a mobile company will make is whether to develop a native application or spend their resources optimizing for mobile web browsers.

Mobile Web

Native Apps

vs.

Mobile Web vs. Native Apps

Page 8

The Case For (and Against) the Mobile Web

Pros:

Single platform (the web)

One app to build, launch and maintain All thats needed for some companies Cons: Less controlled user experience Slower user experience

No app store distribution

Lack of standards across mobile browsers

Mobile Web

Page 9

The Case For (and Against) Native Apps

Pros:

Rich media functionality

Controlled user experience Faster experience for users App store distribution Cons: Multi-platform

Must build, test & maintain multiple apps

Not necessary for some companies

Native Apps

Page 10

Native Apps or Mobile Web? What the Users Say

There are particular native apps that I use on a regular basis, mainly due to ease-of-use and simplicity. These include weather apps, travel planning apps (kayak.com), music apps (Pandora). However, if Im accessing news and searching for information, I use mobile sites that automatically render as mobile versions of their web sites (cnn.com, google.com)
I also foresee the future in which mobile web will be much easier to use, and displace most (not all) of the need for native apps. Instead of downloading the native app itself, there may be shortcuts on your mobile device for easy access. Think about the recent progression away from desktop apps and into purely web apps; mobile is simply a few years behind. ********** I prefer native apps as they are more focused on your information needs. I agree that the mobile web is more accessible and affordable, but it takes longer to wade through all the sites to get to what you want. Mobile web apps do solve the problem to certain extent, but theyre not on par with the native apps in my opinion.

vs.

User Testimonials

Page 11

Alternatives to Native Apps and Mobile Web

Of course, deciding what form of mobile app you plan to offer isnt an either/or decision. There are alternatives, including: WAC: The emergence of the Wholesale Applications Community (WAC) may end up providing software companies with their most flexible solution. WAC is a consortium of mobile leaders, which has come to include carriers like AT&T, Verizon, Sprint, China Mobile, Orange and others, as well as manufacturers like Sony, Samsung and LG. This community intends to enable developers to develop one mobile app that will work across ALL carriers, devices and operating systems. Hybrid Apps: Similar to the ideas behind developing for the mobile web, hybrid apps would allow developers to create mobile apps that share their backend with their web application. They would create a native app shell or wrapper which would utilize the same features, functionality, data and other characteristics. ***** However promising in terms of simplicity and cost, these two options are not without some major drawbacks, notably that of competing interests. Will these communities treat all devices, carriers, etc. as equals? So dont rule out these alternatives immediately, but proceed with caution.

Wholesale App Community

Though many doubted the longterm viability of the WAC, it looks as though the group is gaining the momentum it needs to make a difference.

Theres a long way to go, and the concepts the WAC are working towards are ambitious to say the least, but the steps theyve taken so far seem to be the right ones.
- MobileMarketing.com

Mobile Alternatives

Page 12

Other Considerations
Mobile devices are evolving into increasingly sophisticated general purpose computers. This has led to the development of a variety of platforms and operating systems. Today, the operating system inside your mobile device is as equally important as the device manufacturer (OEM). The various features, GUI, processing speed, and most importantly, the applications available for your device will depend greatly on the underlying OS, as well as the carrier and OEM.

Device Manufacturers

Operating Systems

Carriers

Mobile Decisions

Page 13

Mobile OS: Past Present and Future

The mobile wars are heating up. This recent study by Quantcast shows that Android is gaining on Apples web iOS, although the former still maintains a huge share of the overall market. Keep this trend in mind when deciding which operating systems you intend to build an app for first.

Mobile Marketshare

Page 14

Mobile Constants
No matter which route you choose native apps or the mobile web certain tasks and backend functionality will remain a constant, including: API / web services Architecture Database

Mobile Fact
Consumer conversion (from surf to purchase) on iPhone or Android native apps is 3X - 10X times that of mobile web sites Why? Because users are more comfortable with the device. In most cases, the usability of the native app, including page load speed, is much greater than the mobile web.

The key lesson here is that app quality begins before coding or testing. You should start by reviewing the makeup of your users. What features and functionality do they want? What types of information are they trying to access? And most importantly, how can you reach them on their terms?
Other key decisions that every mobile company will have to make include: Static content vs. rich content Commerce vs. functionality

Mobile Constants

Page 15

MOBILE TESTING TYPES AND METHODS

Page 16

Mobile Testing Types: Functional Testing

The most essential testing procedure will be to verify that the basic functionality of your application. Early on, much of this attention should focus on the download activities of the prospective user (i.e. that the download process is seamless and intuitive). From there, you can begin to verify more specific functionality. As well explain shortly, this can be done either via a mobile simulator (desktop browser) or by using real-world testers. Finding device specific bugs: If, for example, you discover links that are not functional on certain devices, then you should first determine whether it works on the desktop browser. If it is reproducible on the desktop browser, then it is obviously not a device-specific bug, but rather a bug in the basic application. If a link is functional in the desktop browser and it doesnt work in the device, then it should be identified as a device specific bug. Such functional testing should occur when the application is in the early development phase. At that point, testers should perform a quick test on the devices you deem as the having the highest priority, in order to find any rendering or any functional issues as soon as possible. This way, development can have enough time to fix them or arrive at a proper decision as to whether to support these devices or not. If fixing the bug involves high dev cost, they probably wont. If this sort of functional testing doesnt happen at the an early stage (and most of the time, it doesnt) there is the risk that the application might not work for certain devices. This can lead dev/PM to changes the design, which can further escalate costs.

Mobile Stats
51% of iPhone owners have five or more games installed on their devices

37% of iPhone owners play games at least once per day puzzle games are the most popular
46% of Blackberry owners have NO games on their device at all

- Source: Mobile Entertainment

Testing Types: Functional Testing

Page 17

Mobile Testing Types: Usability Testing

Usability testing plays an important role for launching a quality mobile application. Usability testing should occur as soon as the basic stable for the application is ready. This is also true in order to test whether the application UI is user-friendly or not, or whether some part/functionality of the application is easily discoverable or not. Also, usability studies should be done for the different types of devices, such as touch screen devices, since the user has different experience for the same application on these devices. A user may have a good experience on the iPhone, while another might have bad experience on a nontouch screen device. In this instance, changes to the UI should be considered to improve the experience on the non-touch screen device. For example, lets say there is a form with various fields and it is saved by clicking on the save button located at bottom of the screen. For this scenario, someone using iPhone will have no issues to scroll to the bottom to click on the save button, since the iPhone has a good scrolling mechanism. However, the same scenario will give the bad user experience for the non-touch devices users, where scrolling to the bottom of the page is not as easy. Needless to say, usability testing is an extremely critical step towards launching a high-quality mobile application.

User Loyalty
While most will tolerate a minor glitch, studies show that many wont hang around if they are disappointed with the performance of a mobile app or website. And the damage doesnt stop there. Word-of-mouth will ensure any bad experiences are soon broadcast. - Helen Leggatt, BizReport.com

Testing Types: Usability Testing

Page 18

Mobile Testing Types: Performance Testing

Performance testing is another critical step in the successful launch of a mobile application. The key factor to be aware of here is mobile carriers, which can effect the speed and ease-of-use of the application. If the application performance is poor for the end user, they will be likely to divert to other apps and services. The application must therefore be tested with different devices and carriers depending on the county and region. Most likely, you didnt build your application for automated users. You built it for real users. Why then do so many companies rely solely on automated testing results to verify functionality under stress? Until recently, it was because there was no such mechanism by which a company could utilize real testers while a system was under load. But as well outline shortly, that obstacle has largely been overcome. By leveraging real testers with automated tools, companies now have access to an entirely new layer of data. Better yet, they are able to identify and resolve issues that would have been reported later in the development process.

Mobile Testing Tip

If the application being developed will be dynamic or interactive, then the browser must support the following: - JavaScript support: Mobile browsers must support JavaScript in order build an interactive application UI. - XML HTTP Request Object: XML HTTP request object support is required to communicate with the back-end server and to update the page with new data without reloading the page. This will give the user a smooth browsing experience for the site. - CSS support: CSS defines how page elements are to be displayed and enables you to change appearance and layout of all of the pages by editing a single file.

Testing Types: Performance Testing

Page 19

Mobile Testing Methods: In-House Testing

Pros: In an ideal world, mobile testing could be performed under the watchful eye of product managers, developers and QA managers, who could observe firsthand how the application performs under every possible user scenario. The assembled testing team consisting of perhaps a dozen or so of the most talented testers on the market would have hundreds of handset models at their disposal: iPhone, Blackberry, Nokia, Samsung, Sony - you name it, they test it. It would be a testing lab that even the largest software company would envy. With this type of control over the testing process, what could possibly go wrong? Cons: The truth is, for mobile apps, building a comprehensive in-house testing lab is prohibitively time-consuming and expensive. This is true for almost every mobile app company, with very few exceptions. Imagine the expense of building an in-house team and lab capable of assuring the functionality for iPhone, Blackberry and Android handsets (of all makes and models) across wireless carriers in the U.S., U.K, Spain and France. Now imagine this with 3 apps 5 apps Will you send testers around the globe to assure quality? Of course not. Thus, its no surprise that mobile app companies from start-ups to large enterprises dont rely solely on in-house testing resources. To be fair, most firms dont intend to create an in-house team like the one described above (although some have tried). Worse, they spend their money trying to get whatever they can from this method. That is, they have 3-5 in-house testers, with a dozen devices (at most). But their testing coverage is confined to their own office, and the whole point of comprehensive testing is lost.

Just how important is global testing coverage?

By the end of this year, the mobile ad market is expected to grow to $913.5 million. And by 2013, Gartner expects mobile ad spending to surpass $13 billion, with the Asia-Pacific region bringing in the most revenue, followed by North America and Europe. - Meghan Keane, www.econsultancy.com

Testing Methods: In-House Teams

Page 20

Mobile Testing Methods: Outsourcing

Pros: Of course, when mobile application developers discovered that the inhouse route couldnt provide the testing they needed, they logically said to themselves: Ill just outsource it. This was not an isolated decision; it was made by thousands of software companies, and few could fault them for their logic at the time. If comprehensive mobile testing was to be achieved, then surely an offshore firm (with their low-cost labor) would be the only entity capable of getting the job done at a reasonable price.

Common User Complaints:

Failed or stalled downloads Credit card verification delays No screenshots App performance when there is slow connectivity Inconsistent buttons, fonts, etc. Missing or broken links App not supported by device Outdated versions; bugs still in app

Its this line of thinking that continues to define the way in which many mobile companies deal with their testing objectives.
Cons: While outsourcing has certainly proven to be less expensive than hiring an internal team, developers who contracted with such firms quickly ran into the same limits as that of their in-house teams while losing what little control they had over the QA process. And for what? With no insight into the day-today testing activities, how could these developers expect to improve their application in a timely manner? Developers must remain close to their application at all times, which is antithetical to the outsourcing mentality. And as it turns out, even the largest of traditional outsourcing firms dont have the resources needed to provide adequate testing coverage required by todays mobile environment. Just like their clients, they cannot easily (or affordably) test an app across dozens of handset makers, models, carriers, languages and locations. Plus, the testers employed by these offshore firms are unlikely to be among the target audience for a companys mobile apps.

Testing Methods: Outsourcing

Page 21

Mobile Testing Methods: Simulators/Emulators

Pros: In many ways, the use of mobile emulators and simulators is essential for developing and testing a quality application. Such technology enables developers to verify certain functionality that is not specific to any device, carrier or operating system. They are also very useful in terms of usability, and especially design, including data input, screen size, button use, etc. all from the convenience of their own laptop. Although many mobile app companies rely on these tools to indentify bugs at an early stage in development , they often fail to catch bugs at later stages. As weve seen, these type of bugs are always the most expensive to correct. Cons: One of the biggest challenges for mobile developers is that the testing of applications is occurring in an environment far removed from the real world, where actual users run and interact with those applications ON their devices. Said differently, the gap between in-the-lab simulation and in-the-wild usage is far too great to ignore. Consider, for instance, an application that is data entry intensive. To assume that it can be fully tested on a simulator (i.e. with full keyboard and mouse access) and be ready for release is a mistake mobile app developers have continually made for the past five years. The convenience of simulators and emulators has made it easier than ever to be lured into a false sense of security. But the advantages of such tools are limited in scope, and should never be considered a substitute for real-world, on-device testing.

Mobile App Purgatory

Bugs in your mobile app cause unintended consequences. Take this story for example: A mobile app developer is alerted to a iPhone bug, and instead of being able to fix it right away, he spends weeks pleading with Apple to allow him to proceed with the update. If bugs in your app appear, getting them fixed quickly can be very problematic. Of course, if the bug had been discovered BEFORE launch, this would be a non-issue.

Testing Methods: Simulators and Emulators

Page 22

Mobile Testing Methods: Beta Communities

Pros: Mobile companies that can attract a large beta community have a terrific advantage over their peers, as thousands of users submit bugs and feedback around the clockfor free. Needless to say, the allure of an active, engaged group of beta users is too great for many developers to pass up. Not only do they gauge your applications performance before the big release, they act as advocates for your product, adopting a strong sense of ownership and helping you to establish a customer base for your product. And so if youre at all uncomfortable with your apps performance prior to launch - and dont want to go to great lengths running professional usability tests beta programs seem like the easiest, safest route. Cons: In practice, it is rare for a software company to attract such a large group of beta testers. After all, not every company can be Google or Mozilla, with its various (and wildly popular) beta versions, where people line up to test drive their software. Chances are, your app either lacks enough mass appeal, or is relatively unknown (for now), and is hence not that attractive to the beta testing types. But even if you are able to assemble a sizeable beta group, the method still falls short of providing adequate testing coverage. And the reason for this is simple: beta testers are more often like users, in that they will try to get your app to function properly, whereas a real tester would attempt to break your app. In other words: Users use. Testers test.

The Future of Mobile

As mobile matures, more development tools and standards will emerge. And in time, testing tools and bulletproof practices will evolve. Until then, building and testing mobile apps for the iPhone, Blackberry, Android, Palm Pre and others will continue to feel a bit like the wild west. -Matt Johnston VP of Marketing, uTest

Testing Methods: Beta Communities

Page 23

KEY TESTING CHALLENGES

Page 24

The mobile app testing matrix is infinitely complex and getting worse

Features

Location Wireless Carriers Device Models OS & Browser

Mobile App Challenges Page 25

The Challenges of Mobile Application Testing

The true cost of bugs: According to a study highlighted on TechCrunch, the average shelf life of an iPhone app is roughly 30 days. It turns out that, for free apps, less than 20% of users return to an app even one day after downloading it. And by day 30, less than 5% of users are still utilizing the app. And for paid apps, the drop-off is even slightly steeper. Why the drop-off? For starters: bugs - especially ones that appear within minutes of downloading - are likely to cause users to abandon the application after a short period of use. With the variety of apps now available, users have an extremely low tolerance for buggy applications. In a never-ending attempt to appease their user base, developers must therefore test their products across: Handset Maker/Model Operating System Browser Wireless Carrier
100
80 60 40 20 0
Specification Design Coding Unit Test Integration Test Release Test Post-Release

(Relative cost of defect, by time of discovery)

Location
Language

No matter what type of mobile applications you develop whether theyre chat tools, social networking, games, business apps or others you face the same fundamental problems encountered by all mobile app developers. Lets now take a quick look at how mobile app developers have attempted to deal with these problems

Mobile Testing Challenges

Page 26

CROWDSOURCED TESTING FOR MOBILE

Page 27

Crowdsourced Mobile Testing

Combining the benefits of the aforementioned methods, the use of community-based testing or crowdsourcing has altered the way companies test their mobile applications. This shift has empowered progressive mobile app companies to make the best decisions, on a per-project basis, without sacrificing control, costs or accountability. By leveraging a global community of professional testers, QA teams are doing the impossible: maintaining app quality, achieving broad testing coverage, meeting launch dates and staying within tighter budgets. The net effect is that mobile app companies get instant access to a global community of testers via a platform that enables detailed management of the entire test cycle (or even connects to their in-house bug tracking system). This community can serve as a virtual testing team (for small companies and start-ups), or as an on-demand extension for in-house QA departments. While the thought of having a virtual QA team from around the globe might seem like a complicated free-for-all, the reality is just the opposite. With the right online platform, the process fits neatly into your existing teams and processes. Heres a quick look at how it works:

What is Crowdsourcing?
"Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to a large group of people in the form of an open call. - Jeff Howe Author of Crowdsourcing

Crowdsourced Mobile Testing

Page 28

Crowdsourced Testing: How It Works Customer Specify Testing Needs

- Handset Maker, Model & OS - Carrier, Location & Language

Tester Community Selected testers invited to project Testers report in real-time

Customer approves

Testers get paid

Crowdsourced Testing: How it Works

Page 29

Crowdsourcing Challenges: Quality and Control

Crowdsourcing has a reputation for being noisy and chaotic, without much actual productive work. Thus its critical to choose a reputable community that can point to its customers successes, and shares the past performance and reputation of community members. In short, make sure youre choosing a community and not a mob. Other common challenges include:
Communication: As with any successful project, communication is key. While crowdsourcing doesn't enable face-to-face conversations, it does enable decisionmakers to converse with their virtual QA team in real-time through an easy-to-use online platform. Also, make sure the testing company you choose provides a dedicated project manager to help you manage the community and the process. Intellectual Property: IP protection is sometimes a concern for customers who are new to crowdsourcing. While it might seem like your IP could be exposed to a community of thousands, top crowdsourcing companies allow customers to carefully select their testers and establish non-disclosure agreements (NDA) with their community members. Quality: With crowdsourcing, its easy to believe that anyone can test your application. Fortunately, the crowdsourcing model enables customers to be as selective as they want in terms of testers skill sets, experience, performance ratings in determining who gets invited to test their app. As with other methods, unexpected problems do arise, but what makes crowdsourcing different is its inherent ability to adapt to changes on-the-fly.

Mobile Challenges
Mobile applications come with their own unique set of challenges. Between Wi-Fi, 3G, Edge and all their different behaviors, there are so many angles you have to cover. Until recently, testing for this criteria would have been impossible, but crowdsourcing has changed all that.
-Travis McElfresh, -VP of Technology, 1Cast

Crowdsourcing Challenges

Page 30

Benefits of Crowdsourced Mobile App Testing

Although the advantages of such an approach are numerous, heres a quick look at what to expect when implementing crowdsourced QA for your mobile app: Cost Control: Crowdsourcing allows managers to utilize lower-cost outside support without being tied down by long-term commitments. The fundamental structure of online communities creates competition, making it more accountable, efficient and cost-effective than traditional outsourcing. Dealing with Increased Complexity: Significant quality improvement is achieved when development and testing is done across a wider set of handsets, carriers, locations and languages. Crowdsourced mobile app testers can be easily recruited based on a wide variety of criteria. Faster Time-to-Market: Crowdsourcing enables you to ramp up or down to meet your needs at any given time - eliminating the delays and staffing constraints that often stifle mobile app companies around peak release times. Higher Quality: Customers expect their mobile apps to be fully functional and bug-free from day one. Companies that believe good enough is good enough when it comes to quality and usability will learn a valuable lesson (the hard way). Retaining Users: If your product has problems, or doesnt meet expectations, its easier than ever for your customers (and prospective customers) to find your competition.

Why It Works
Because crowdsourcing is built from the open source template, any company looking to leverage computer professionals has the advantage of an audience already familiar with the basic idea of community production. -Jeff Howe Journalist, Wired.com

Benefits of Crowdsourcing

Page 31

Benefits of In the Wild Testing

Hands On Device Interaction On a simulator, you still use a mouse to touch the screen and simulate gestures. You also have a full-sized keyboard for data entry. Of course, this is very different from using a mobile device. First, a mobile device sits in your hand. Each of us likely has slightly different ways of holding and operating the device. For some, its done with one-hand using your thumb or a finger. For others, it might be two hands using both thumbs. Second, theres the act of touching various screen elements like buttons and controls. This is much easier to do with a mouse pointer than a pudgy finger. Based on the prior experience of many mobile testers, this difference is a critical one, and the biggest, for testing application design and function. Using a mouse with the simulator, you do not get the full effect of having to scroll through a large list view of items or having to play whack-a-mole on the screen with your thumb because button placement for navigating multiple screens is inconsistent. Mobile developers are strongly encouraged to ensure that application testing begins early, and happens often, on the mobile device itself rather than on a simulator. The same holds true for tablet devices.

Mobile Testing Tip

Even on the same mobile platform, screen sizes and resolutions can vary based on device type. For instance, the screen size and resolution on the HTC Incredible is different than that on the HTC EVO 4G. Consequently, for an application to have a consistent look and feel across both devices and across a variety of other devices, user interface elements and graphics need to be scalable. -Web Trends, Mobile Analytics

In-the-Wild Testing

Page 32

Benefits of In the Wild Testing

Screen Size and Viewing: Even though the simulator application mimics the size of the iPhone screen, its still a different experience viewing an application on the device rather than a monitor. A screen that is readable on a simulator may not be and usually isnt as readable on the device. Data Entry: No contest here. It would be a terrible mistake to develop an application thats data entry intensive and test it on a simulator, with full keyboard and mouse access, and believe that its ready to be released. Trying to type even a sentence is a challenge on a touch screen keyboard. Functionality: In the world of mobile, there are some things you just cant test on the simulator., including the use of accelerometers and location functions to name a few. Todays mobile devices are pushing the boundaries of traditional application design and testing methodologies. While fundamentally still the same, those methodologies need to be applied in new ways, especially with regard to testing application usability.

Mobile Testing Tips

Interrupts: Test how the device and application behaves when it is interrupted by another action, e.g. incoming/outgoing call, SMS, MMS, video-call, low battery, low/no connectivity. Device Settings: Change the device settings: Date and Time, Languages (Advanced regional settings), Connections, Shortcuts anything you have control over and learn how each and combined settings impact the device. - Bernard Lelchuk

Screen Size, Data Entry and Functionality

Page 33

Conclusion
Testing no longer has to be a neglected phase of mobile application development. There was a time when the testing matrix was indeed too complex, too burdensome to be completed using standard means. But thanks to the rapid evolution of crowdsourced testing, all that has now changed. The companies that are now leveraging crowdsourced testing - particularly in the mobile space - are gaining more of a competitive advantage than they perhaps even realize. As the mobile market doubles and triples in size over the next decade, those that have made testing coverage a priority will enjoy their ROI in terms of increased market share, profitability and above all, user loyalty. Those who continue to neglect the testing phase will struggle at best, and at worst, cease to exist. Its that simple. Either way, the future of mobile applications remains bright. Once reserved for a tech-savvy niche in the business world, the use of mobile applications is now firmly entrenched in the mainstream. Theres no turning back when it comes to mobile applications, and that includes methods for testing them. For more on how crowdsourcing can complete your mobile testing efforts, chat with one of our QA gurus by clicking here or by calling 800.445.3914.

Crowdsourced Capacity
Small development firms have limited means today to beta test and peer review their applications with a cross section of representative users. Given the hundreds of thousands of mobile apps, we believe that efficient (crowd-sourced) testing of apps in a global market of users is considerably under-utilized. - VisionMobile

Summary

Page 34

APPENDIX: MOBILE PLAYERS

Page 35

Mobile Operating Systems

iPhone OS from Apple The iPhone uses its well known operating system, the iPhone OS, which is derived from Mac OS X. The iPhone OS has four abstraction layers: the Core OS layer, the Core Services layer, the Media layer, and the Cocoa Touch layer. The operating system uses roughly 500 megabytes of the devices storage. The iPhone app store is the online market for downloading these applications. Third party applications were not officially supported until the release of iPhone OS 2.0 on July 11th 2008. Presently, XCode 3.1 is the development environment and the applications are written in Objective C (a variant of language C). There is no doubt that the iOs will continue to be among the preferred mobile operating systems worldwide. Android from Google Developed by Google, Android is an open source, Linux-derived platform backed by Google, along with major hardware and software developers (such as Intel, HTC, ARM, and eBay, to name a few). Together, they form the Open Handset Alliance. This OS, though very new, already has a cult following among programmers eager to develop apps for its flexible, Open Source, back end. Android promises to give developers access to every aspect of the phones operation. This lends many to foresee the promise of further growth for the Android platform. Like the iOS, Android figures to remain a prominent figure in the future of mobile application development.

Mobile Operating Systems: Apple and Android

Page 36

Mobile Operating Systems

RIM Blackberry OS The BlackBerry OS is a proprietary software platform, created by Research In Motion, for their BlackBerry phones. The OS is focused on easy operation and was originally designed for business professionals. The Blackberry operating system provides multi-tasking and supports specialized input devices that have been adopted by RIM for use in its handhelds, particularly the trackwheel, trackball, and most recently, the trackpad and touchscreen. The BlackBerry platform is perhaps best known for its native support for corporate email, through MIDP 1.0 and, more recently, a subset of MIDP 2.0, which and allows complete wireless activation and synchronization with Microsoft Exchange, Lotus Domino, or Novell GroupWise email, calendar, tasks, notes, and contacts, when used in conjunction with BlackBerry Enterprise Server. The operating system also supports WAP 1.2. Recently it has seen a surge in third-party applications and has been improved to offer full multimedia support. Linux Operating System The Linux operating system is strongest in China, where it is used by Motorola, and in Japan, where it is used by DoCoMo. Rather than being a platform in its own right, Linux is used as a basis for a number of different platforms developed by several vendors, including Android, LiMo, Maemo, Openmoko and Qt Extended, which are mostly incompatible. PalmSource (now Access) is moving towards an interface running on Linux. Another platform based on Linux is being developed by Motorola, NEC, NTT DoCoMo, Panasonic, Samsung, and Vodafone.

Mobile Operating Systems: RIM and Linux

Page 37

Mobile Operating Systems

HP/Palm webOS The HP/Palm webOS is Palms next generation operating system. PalmSource traditionally used its own platform developed by Palm Inc. Underneath the graphical user interface sits a software stack that has much in common with mainstream Linux distributions for desktop computers. webOS uses, among other components, the Linux kernel, Upstart, GStreamer as multimedia framework, libpurple for instant messaging, Simple DirectMedia Layer (SDL) for game programming, PulseAudio for audio management, and WebKit Applications for webOS are written in JavaScript, HTML, AJAX, and some other common web technologies. It supports streaming video in RTSP, H.263, and H.264 formats. Bada from Samsung The Bada operating system has a kernel configurable architecture, which allows the use of either the Linux kernel, or another Real-time operating system (RTOS) kernel. The Linux kernel is the preferred choice for smartphones, while RTOS is used for cheaper feature-phones, due to its smaller memory footprint. It has a web browser based on the open-source WebKit, and also features Adobe Flash. Both WebKit and Flash can be embedded inside native Bada applications. Bada offers interactive mapping with Point of interest (POI) features, which can also be embedded inside native applications. Bada supports various sensors, such as motion sensing, vibration control, face detection, accelerometer, magnetometer, tilt, and GPS, which can be incorporated into applications.

Mobile Operating Systems: Palm and Bada

Page 38

Mobile Operating Systems

Windows Phone 7 from Microsoft On February 15th, 2010 Microsoft unveiled its next-generation mobile OS, Windows Phone 7. The new mobile OS includes a completely new over-hauled UI called Metro. It includes full integration of Microsoft services such as Zune, Xbox Live and Bing. Windows mobile is based on the Windows CE 5.2 kernel, and features a suite of basic applications developed using the Microsoft Windows API. It is designed to be somewhat similar to desktop versions of Windows, feature-wise and aesthetically. Third-party software development is available for this OS, with several options for developers to use when deploying a mobile application. This includes writing native code with Visual C++, writing managed code that works with the .NET Compact Framework, or Server-side code that can be deployed using Internet Explorer Mobile or a mobile client on the users device. The .NET Compact Framework is actually a subset of the .NET Framework and hence shares many components with software development on desktop clients, application servers, and web servers which have the .NET Framework installed, thus integrating networked computing space called the cloud. It has been criticized for having a user interface which is not optimized for touch input by fingers; instead, it is more usable with a stylus. However, unlike iPhone OS, it does support both touch screen and physical keyboard configurations.

Mobile Operating Systems: Windows Mobile

Page 39

Mobile Development Platforms

BREW (Binary Runtime Environment for Wireless) from Qualcomm Brew is an application development platform that was originally developed for CDMA mobile phones, but it also supports GSM for some devices. It debuted in September 2001 as a software platform. It can download and run small programs for playing games, sending messages, sharing photos, and the like. The main advantage of BREW platforms is that the application developers can easily port their applications between all Qualcomm devices. BREW acts between the application and the wireless device on-chip operating system in order to allow programmers to develop applications without needing to code for system interface or understand wireless applications. J2me by Sun Java 2 Micro Edition (J2ME) is Suns version of Java aimed at machines with limited hardware resources such as PDAs, cell phones, and other consumer electronic and embedded devices. J2ME is aimed at machines with as little as 128KB of RAM and with processors a lot less powerful than those used on typical desktop and server machines. J2ME actually consists of a set of profiles. Each profile is defined for a particular type of device cell phones, PDAs, microwave ovens, etc. and consists of a minimum set of class libraries required for the particular type of device and a specification of a Java virtual machine required to support the device. There are now more than 2 billion Java ME enabled mobile phones and PDAs.

Mobile Development Platforms: BREW and J2ME

Page 40

Mobile Development Platforms

LiMo The LiMo Platform is developed by the LiMo Foundation (founded by a group of cellular handset makers and network operators) for handheld devices, and it uses Linux as its operating system. It has a modular plug-in architecture, and supports DRM. LiMo application developers will be able to use SDKs to write managed code running in a Java virtual machine, browser apps for WebKit, and native code. Qtopia Qtopia is a platform for embedded Linux devices that provides the end-user with a set of core applications, such as PIM applications (Address Book, Calendar and Todo) and a Media Player. Qtopia provides the developer with a clean object-oriented API to write applications for an embedded device. Most of the API presented to the developer is the same API offered by Trolltech for cross-platform development between Windows, Mac and the UNIX/Linux. The same code can be shared and reused on all of these desktop platforms and now Qtopia is just another platform to run Qt applications. The Qtopia platform encompasses both the software that runs on an embedded device and the software that runs on an end-user desktop. The software that runs on the desktop, called Qtopia Desktop, allows the user to synchronize their data between the embedded device and the desktop. Qtopia Desktop has built-in versions of the PIM application found in Qtopia. Qtopia Desktop allows developers to write plug-ins to synchronize their information.

Mobile Development Platforms: LiMo and Qtopia

Page 41

Acknowledgements: Special thanks to Bernard Lelchuk, Brad Sellick, Jigar Patel and Anuj
Gupta for their contributions to this eBook.

Whos Crowdsourcing?

About uTest

The number of mobile app companies using crowdsourced testing is growing everyday. Headquartered near Boston, uTest is the world's largest marketplace for software testing Heres a few companies ahead of the curve: services. The company provides real-world QA services through its community of 40,000+ professional testers from over 170 countries around the world. Thousands of companies from web start-ups to enterprise software firms - have signed up to get their apps tested by the uTest community. uTest enables companies to launch higher quality products; get their desktop, web and mobile applications to market faster; and control the cost of testing. Customers specify their testing requirements for location, language, OS, browser and more and uTest selects the testers who best match each project. And because uTest is on-demand, companies pay for completed test cycles. A brief online introduction is available at www.utest.com/demo. uTest can be contacted at: uTest, Inc. 153 Cordaville Road Southborough, MA 01772 p: 1.800.445.3914 e: info@utest.com w: www.utest.com

About uTest

Page 42