Ecommerce Testing (Part IV): Diving Deep into Ecommerce Mobile App Testing

Introduction

Mobile apps, as software pieces, must go through basically all the common testing routines any software application does, but, given their specific nature, some aspects of their testing process must be emphasized.

Besides the rigorous functional, load/performance and security testing phases, which verify the correctness of basic functions and the general stability and safety of the system, the bulk of the effort must be generally put in aspects like usability, cross-browser/platform compatibility and responsiveness.

Overall Concepts

Usability is key when we’re talking about mobile apps. Given the intense use of smartphones today and the vast range of available hardware and software platforms, the user interface of a mobile app should adapt not only to the user needs but also to device specifications, making installation and device testing two important steps in the mobile app testing process. For the same reason, web and hybrid app types –more on that later– will require proper cross-browser compatibility and responsiveness testing. Apps should respond to events like screen rotation, split-screen or when viewed on other types of mobile devices like tablets or laptops with small screens, and they should do so regardless of the web browser being used.

Additionally, localization testing is important when mobile apps are oriented to the international/global market and hence need to be rendered and viewed in the native language of the user, displaying data according to their respective locales (date format, currency, measures, etc.).

Mobile App Types, briefly

The three most common types are native, web and hybrid apps.

Native mobile apps are the ones built for a specific software or hardware platform, given their more direct –lower level– interaction with it. This application type does for faster and more versatile apps, since not only they can better interact with the platform but can also take advantage of features specific to their architecture. Given the use of native SDKs, higher consistency is achieved and so more stable UIs can be built. Native mobile apps are also distributed via their respective platform stores, a thing which provides users the support they need. As a deemable downside, native mobile apps require higher expertise and more time for their development, as well as more frequent updates, due to the intricacies and specificity implicit at platform and language level.

Web apps are served through a website which can be viewed on basically any web browser, hence being portable by definition. A responsive UI design makes web apps adapt to different devices and screen resolutions and therefore highly usable. Some of web apps advantages are their ease in deployment and maintenance, something which also makes them cost-effective due to reduced development and testing costs. Performance and security can be weak points, especially when the application scales in size and complexity.

Hybrid apps combine both the front end/UI of native applications with the underlying infrastructure which web apps implement. They’re essentially the wrapping of a web app in the shell of a native app. Like web apps, they have also the advantage of being cost-effective, due to native/web hybridization in combination with technologies like HTML5, JS and CSS, plus, as they’re usually built on a single code base, their code is highly reusable. Weak points with this type of apps will be less versatility, due to less harnessing of hardware features, and the added dependence on the browser’s performance.

Automated Tests in Mobile Apps

Automating our mobile app tests is the absolutely rational approach when thinking about the  market demands and the intensity of use mobile apps have today.

Performing automated regression tests will be paramount, especially for native apps, which are subject to more frequent changes. Recording GUIs and test scripts can serve us in this task.

Parallel testing is a great aid when we need to perform cross-browser/cross-platform tests. Besides being well suited for testing custom UIs, evolving software products, localization/internationalization or environment configuration, it is flexible, time saving, cost-effective offering high testing coverage, plus it can be integrated to our CI/CD pipeline.

Visual testing will be another strong arm in our body of testing strategies. While it addresses the graphically visible to the end user, it can capture exact changes a UI experiences and point out unexpected behaviors with utmost clarity, this being the reason why visual tests are generally performed as regression tests.

Specific Testing for Ecommerce

We’ll share and explain some examples of certain critical scenarios from where to derive applicable test cases and what they should specifically check.

Functional Tests

• Home page, product page, special offers, related products, viewing history, sitemap pages, about us, etc, are correctly functional and visible.
• Search functionality works as expected, including when using filter criteria.
• The product page includes options such as product size, color, and type, and a sorting mechanism to filter items based on price, model, size, etc., as well as an Add to Cart or Add to Wishlist link/button in each element.
• Items can be removed from the product list, cash on delivery option, Select delivery option, card payment, pay now option, etc.
• Product prices are correct, shipping costs, VAT, discount codes are properly applied and the sum total is correctly calculated –even across different regions, including discount coupon codes.
• Payment gateway processes payments as expected, offering all set payment methods like online banking, credit/debit card, PayPal, etc.
• Invoice and emails are correctly generated and sent after the payment process is done.
• Refund process (refund receipts, emails, etc) works as expected.
• The app performs as expected on various mobile and OS versions.
• The app performance is stable when eventual external interruptions occur (SMS received, minimization during incoming phone call, etc).
• The app can be downloaded and installed without any issues.
• Device is able to multitask as expected when the app is in use or running in background.
• Other applications’ performance isn’t hindered after the app is installed.
• Social network options such as sharing, posting, etc. work as required.
• Mandatory fields work as required.
• App supports payment gateway transactions correctly.
• Performance is as expected in page scrolling scenarios.
• Navigation between various modules works as expected.
• Error messages are displayed correctly.

Performance Tests

• The app can handle the projected load volumes.
• Check what mobile app and infrastructure bottlenecks preventing the app from performing as expected are there, if any.
• Response time is as expected.
• Battery consumption, memory leaks, GPS, and camera performance work as per requirements.
• The current network coverage is able to support the app at peak, average, and minimum user levels.
• Check for possible performance issues if the network changes to/from WIFI and 2G/3G/4G.
• Check how the app behaves during intermittent phases of connectivity.
• Client-server configurations provide the optimum performance level.

Battery Usage Tests

• Energy consumption of the mobile app.
• User interface designs using intense graphics or result in unnecessarily high database queries.
• The battery life can support the application to perform under projected load volumes
• Low battery and high-performance demands.
• How the app performs if in use when the battery is removed.
• Battery and data usage leaks.
• New features and updates are not introducing new battery and data usage.

Usability Tests

• Buttons are a user-friendly size.
• Buttons location, style, etc are consistent within the app.
• Icons are consistent within the application.
• Zoom in and out facilities work as expected.
• The keyboard can be minimized and maximized easily.
• Going back, undoing an action, or on touching the wrong item can easily be undone.
• Contextual menus are not overloaded.
• Wording is simple, clear, and easily visible.
• The end-user can easily find the help menu or user manual should they need it.

Compatibility Tests

• The app passes tests on top notch mobile app testing technologies (emulators, simulators, parallel testing, etc) using the most popular operating systems.
• The app works properly with various mobile browsers like Chrome, Safari, Firefox, Microsoft Edge, etc.
• The app’s UI remains consistent, visible, and accessible on various screen sizes.
• UI text is clearly readable for a vast majority of users.
• The app works seamlessly across various configurations.

Security Tests

• The app can withstand any brute force attack to guess a person’s username, password, or credit card number.
• The app forbids any attacker to access sensitive content or functionality without proper authentication.
• The app packs a strong password protection system.
• Session management is properly implemented in order to prevent unauthorized users from accessing unsolicited information.
• Business logic implementations are secured and not vulnerable to any attack from outside.
• Protection against insecure data storage in the app’s keyboard cache works as expected.
• Memory corruption contingencies are correctly managed by the app.

Localization Tests

• Localized content is 100% accurate –including validation or error messages.
• Language locales are properly formatted. (e.g.: RTL, LTR, name/last name order, etc.).
• Terminology is consistent across UI.
• Time and date are properly formatted.
• Currency is the local equivalent.
• License and rules comply with local/regional laws and regulations.
• Hyperlinks and hotkey functionality work as expected.
• Entry fields support special characters and are validated as necessary (e.g.: postal codes).
• The localized user interface has the same type of elements and numbers as the source product.

Recoverability Tests

• The app resumes at the last operation on hard reboots or system crashes.
• Application recovery after an unexpected interruption or crash is effective enough to conserve critical data.
• The app can still handle a transaction during a power failure.

Regression Tests

A rigorous regression testing routine should be performed when:

• Any changes are added to the existing functionalities.
• New changes are implemented in the app code.
• New features are added to the app functionality.
• Any side effect which may show after changes are released.

Conclusion

We’ve covered testing for Ecommerce mobile apps, both in a general as in a more specific view of the possible scenarios from which to approach its testing process. There are many details and technical aspects to take into account when performing a quality assessment of a mobile application. There is a wildly competent market out there, ready to eat the world, and customers don’t settle as long as what is offered is the best product possible; they’re indeed the best bosses.