Historical and Technical Insights on Mobile App Testing

Facts

Since the early/mid-2010s, smartphones –and therefore, mobile apps– usage, literally exploded. Almost 4 billion users were registered in 2016, which grew up to more than 6 billion in 2021, and projected to scale up to at least 7,5 billion in 2026. This would be practically like having one smartphone per world inhabitant. That is a real lot of devices, constantly searching and retrieving information, downloading software from and uploading content to the internet. It’s a level of intensity and extent of usage never seen before for a technological device, which explains and justifies the need for rigorous QA in order to achieve the proper level of stability, usability and security any valuable product should have.

As is widely known, Android and iOS dominate the smartphone market in terms of operating systems; however, Android runs on a multiplicity of vendor hardware platforms, something which makes it both hugely more popular –over 70% market share– as complex to test mobile apps on.

iOS on the other hand, is rather more exclusive, since its availability is reduced to Apple devices, but that’s one of its strong points: it makes iOS apps easier to be tested, hence making more room for quality improvement.

In the mobile world, there are several types of apps, defined by their requirements and complexity. The three most common types are native, web and hybrid apps.

Native mobile apps are the ones built for a specific software or hardware platform, given their more direct –lower level– interaction with it. This application type does for faster and more versatile apps, since not only they can better interact with the platform but can also take advantage of features specific to their architecture. Given the use of native SDKs, higher consistency is achieved and so more stable UIs can be built. Also, native mobile apps are distributed via their respective platform stores, which centralize and facilitate the availability of curated and secure apps, as well as providing users the support they need from the app developers. As a deemable downside, native mobile apps require higher expertise and more time for their development, as well as more frequent updates, due to the intricacies and specificity implicit at platform and language level.

Web apps are served through a website which can be viewed on basically any web browser, hence being portable by definition. A responsive UI design makes web apps adapt to different devices and screen resolutions and therefore highly usable. Some of web apps advantages are their ease in deployment and maintenance, something which also makes them cost-effective due to reduced development and testing costs. Performance and security can be weak points, especially when the application scales in size and complexity.

Hybrid apps combine both the front end/UI of native applications with the underlying infrastructure which web apps implement. They’re essentially the wrapping of a web app in the shell of a native app. Like web apps, they have also the advantage of being cost-effective, due to native/web hybridization in combination with technologies like HTML5, JS and CSS, plus, as they’re usually built on a single code base, their code is highly reusable. Weak points with this type of apps will be less versatility, due to less harnessing of hardware features, and the added dependence on the browser’s performance.

Mobile apps, as software pieces, must go through basically all the common testing routines any software application does, but, given their specific nature, some aspects of their testing process must be emphasized.

Besides the rigorous functional, load/performance and security testing phases, which verify the correctness of basic functions and the general stability and safety of the system, the bulk of the effort must be generally put in aspects like usability, cross-browser/platform compatibility and responsiveness.

Usability is key when we’re talking about mobile apps. Given the intense use of smartphones today and the vast range of available hardware and software platforms, the user interface of a mobile app should adapt not only to the user needs but also to device specifications, making installation and device testing two important steps in the mobile app testing process. For the same reason, web and hybrid apps will require proper cross-browser compatibility and responsiveness testing. Apps should respond to events like screen rotation, split-screen or when viewed on other types of mobile devices like tablets or laptops with small screens, and they should do so regardless of the web browser being used.

Additionally, localization testing is important when mobile apps are oriented to the international/global market and hence need to be rendered and viewed in the native language of the user, displaying data according to their respective locales (date format, currency, measures, etc.).

Dos and Don’ts in Mobile App Testing

• Do –always!– have a plan in which the testing activities are circumscribed.
• Do use real devices as much as possible in order to better assess test results concerning the real world.
• Do perform beta testing, to have a better end-user perspective and assess its possible reception while moving towards UAT.
• Don’t test everything on emulators, –for the same reason we want to use more real devices, but rather keep them for the initial testing phases.
• Don’t overemphasize the UI; although it is a key part of our mobile app, testing the back end and APIs will be almost as important.

Automating Mobile App Testing

Automating our mobile app tests is the absolutely rational approach when thinking about the  market demands and the intensity of use mobile apps have today.

Performing automated regression tests will be paramount, especially for native apps, which are subject to more frequent changes. Recording GUIs and test scripts can serve us in this task.

Parallel testing is a great aid when we need to perform cross-browser/cross-platform tests. Besides being well suited for testing custom UIs, evolving software products, localization/internationalization or environment configuration, it is flexible, time saving, cost-effective offering high testing coverage, plus it can be integrated to our CI/CD pipeline.

Visual testing will be another strong arm in our body of testing strategies. While it addresses the graphically visible to the end user, it can capture exact changes a UI experiences and point out unexpected behaviors with utmost clarity, this being the reason why visual tests are generally performed as regression tests.

Emulated/Simulated Devices

There are basically two types of mobile virtualization: Android emulators and iOS simulators.

Android emulators allow for basic app functional and UI layer testing –the same goes for iOS emulators. Concerning emulation/simulation, emulators duplicate real device behaviors while simulators replicate them.

Testing is done faster on virtual than on real devices, no wonder why developers and testers choose them frequently, but there are some downsides to them.

Real conditions cannot be tested on virtual devices; e.g.: battery consumption, making performance assessment an impossible task. This can give way to the appearance of bugs.

Physical Devices

Testing on the real hardware platform will permit us environment related aspects, e.g.: carrier configuration, sensors, etc., and other physical characteristics like gestures.

Real device testing is helpful when performing locales/region testing, on several conditions, OS versions and apps that exist in the market.

Today’s web development requires tools that can cope with today’s world demands.

A SaaS delivery model, a no code platform –no coding required, a GUI to record test scenarios then play them back, test script maintenance by AI, the use of AI to learn the UI changes, adapt to them and alert the QA team, cross-browser compatibility, including mobile devices, integration with Slack, Jenkins, TestRail, etc, built-in reporting –no third party tools and an exceptional tech customer support, are just some of the values Autify has to offer.

Autify rids maintenance inconveniences by using AI to learn the changes and adapt to them; hence, no more juggling with script maintenance, as it features a no-code solution.

When it comes to customer support, you don’t end up in a maze of help pages and visual media, but talking to real humans who understand human needs.

You can see other client’s success stories here: https://nocode.autify.com/why-autify

We sincerely encourage you to request for our Free Trial and our Demo for both Web and Mobile products.

‍