Application Security Testing: An Integral Part of DevOps
Anyone developing a software application today is going to run into some serious cross-platform challenges during testing. It may be expected to run on a host of different devices and hardware configurations, including smartphones, servers, laptops, smartwatches, and more. The variety of devices in the smartphone category alone is overwhelming.
You also must consider the underlying operating systems, the drivers, and the browsers. Fragmentation means that there are often multiple versions to contend with. Only 0.3% of users have the latest version of Google's Android (7.0 Nougat), for example, and even the previous version (6.0 Marshmallow) only accounts for 24% of users at the time of this writing.
The possible permutations when you put hardware and software together are staggering and that can cause some tricky issues.
1. User Interface Consistency
Whether you have an application running on multiple versions of multiple operating systems, or a Web app that runs on multiple versions of multiple browsers, or perhaps even both, the challenge is the same: How do you get a consistent look and user experience? You want the user to be able to pick up where they left off, regardless of the device they are using. It must work on devices with different screen sizes and resolutions.
Rendering fonts and managing style sheets to achieve consistency is far from easy. Thorough testing is going to include a wide variety of different hardware and software configurations. If you want to adopt a responsive design, it must be planned carefully and tested stringently.
2. Inputs and Interaction
A complex mixture of physical buttons, touchscreen controls, and voice controls further complicates the testing process. You may even have handwriting recognition to contend with. On the mobile platform itself, there's a host of third-party keyboards to be considered. It's vital to examine how users input data and interact with your software. Every possible method must be fully reviewed and tested appropriately.
There are also many differences in UI conventions from country to country. Some parts of the world prefer specific gestures like swipes, whereas others want a button. Regional preferences can create more variance to tests. Accessibility support is very important, too, but it's another set of possible interactions that must be fully debugged.
3. Data and Storage
In trying to solve the consistency issue, developers can run into trouble with data. If your application uses a lot of images or transactional data, there's a hit on cache memory that could impact the overall behaviour and performance of the application. Mobile and IoT devices may have severely limited storage space.
Testers need to make sure that the application is handling data properly. If budgets are being exceeded, the impact on performance can be drastic.
4. Security and Compliance
Performance and storage aren't the only concerns when handling data. You also need to consider security. How is data stored and transmitted? Can you track the data through your system? Is your app meeting security obligations and protecting users? Different browser modes, such as Chrome's incognito mode, may also have unexpected effects, changing the way that data is gathered and processed. There also are vulnerabilities and potential exploits to consider in the underlying operating systems.
Many apps may be subject to governmental or industry body regulations. If you fail to meet compliance, you could end up with a large fine. It's much cheaper to make sure that you test according to regulations and fix issues before release.
5. Meeting User Expectations
If your app doesn't meet expectations, people are not going to use it. Consider that 23% of people abandon an app after one use, according to Localytics research. If your feature set is not full and consistent across platforms, users will be left frustrated.
They also will have certain expectations based on the type of app it is. For example, a banking app should allow authentication through the fingerprint sensor where there is one available. Testers need to have a solid understanding of the intended audience to effectively test that it meets user expectations.
There's a lot to think about when testing cross-platform apps, but with careful consideration of the potential issues and proper planning, you can rise to the challenge.
About the Author
Mush Honda is Vice President of Testing for KMS Technology, a provider of IT services across the software development lifecycle. He was previously a tester at Ernst & Young, Nexidia, Colibrium Partners and Connecture.