This blog is a synopsis of Gartner’s recent report on mobile testing. Apexon has been mentioned as one of the vendors in the mobile testing market in their June 2012 report.
According to Gartner’s recent report on “a guide to mobile testing tools and services”, mobile testing is perhaps the ‘biggest elephant in the room’. There are increasing complexities and challenges due to the platform variants, emulator limitations and changing trends. Therefore, there is a combination of tools and services that an enterprise can use to manage this function better.
The hard world of mobile testing
I am sure you all have heard of these challenges over and over. And first among them being diversity of OS’s and devices and Android being the most fragmented. Second is the limitation in implementing manual testing. Some features such as NFC, GPS location, touch and audio sensors cannot be tested through automation, so the need for manual testing these stays on top of the charts. Third, network performance continues to produce bandwidth and latency issues. Fourth, applications are becoming more sophisticated with the onset of gaming and 3D graphics etc. There is also new technology like HTML5 etc. being used for developing applications and all this is not good news for the mobile testing club. Smartphones are acquiring a wide range of peripheral devices such as Bluetooth etc. that are not accessible to testing tools. And then there are contextual issues such as — can an app be accessible to someone wearing gloves? Is the on-screen text readable by all users.
The a la carte platter of mobile testing tools and services
Here are some of the useful components that can be used to formulate a mobile testing solution for your enterprise:
Cloud services – Cloud testing vendors perform modifications to devices and ensure that they can be accessed remotely over the internet. They build ‘device farms’ that allow access to a wide range of devices and offer scripting tools to enable automation. Advantage — access to real devices allowing to validate all aspects of application performance. Of course this comes with its own set of limitation such as access to limited range of devices over cloud and geographies.
Scripted testing tools – They work with devices connected to a PC or could be accessed via cloud service as discussed above. These tools allow developers to run automated scripts. Selenium is an example of such an open-source browser scripting tool. Again scripting tools cannot test all aspects of a device.
Monitoring and analytics – These tools are used for tracking application performance once the app or website has been deployed. The performance can be tracked from several locations across several network operators.
Emulators — There are native application based device emulators and emulators for web applications. Device emulators are included in all SDKs but have a few limitations — related to range of devices and testing features such as sensors. Browser emulators can be configured to behave like a wide range of smartphones and sophisticated emulators can emulate anything.
Outsourced, usability and crowdsourced testing — Enterprises can use offshore or onshore partners in conjunction with cloud services to access a wide range of devices. IT organizations that are not experts in areas of mobile development can engage consultants to perform special purpose testing. The other is the model of crowdsourced testing where the vendor manages a marketplace of testers. It has its limitations in terms of consistency of results as a more structured test environment and the security of the application IP.