IDC forecasts $1.2 trillion in IoT spending by 2022, reaching 13.6% CAGR by that time. In other words, the IoT industry is moving fast. Additionally, 90% of senior executives in technology industries see IoT as critical to all or some of their businesses. If your organization hasn’t yet touched IoT, that will likely change in the next 12-18 months.
IoT creates a whole new set of development and testing challenges. Due to its complexity, any testing methodology that falls short of the continuous iteration and delivery model will not work well for long. That’s why when we talk about IoT testing, we are implicitly talking about continuous testing.
Apexon has experience working on a wide range of IoT projects, from sport and fitness through to wearable biosensors. Having helped enterprises and technology visionaries accelerate their IoT initiatives through testing in real-life conditions, we can attest to what causes the biggest testing headaches….and how to cure them.
Challenge 1: Is Your Strategy Rock-Solid?
The initial discovery phase of a project may not yet be where organizations encounter actual test challenges, but it is where they can prevent future ones. Fully understanding the business requirements, the device’s usage and features may seem like “testing for beginners”. If that’s the case – great! It means your organization is on the right track. Have you also considered data extraction techniques from the device? What KPIs are you using to measure success? Have you determined your validation methodology?
Our take: Organizations need to define end-to-end strategy before starting testing because it will impact the test environment set-up, choice of software tools and other components used in the roll-out. Organizations can do this on their own provided they have the right skills in-house, or a strategic testing partner can be brought on board to check their strategic assumptions against industry best practices.
Challenge 2: Are You Automating Enough?
Multiple studies suggest that enterprises are automating less than a quarter of their testing. In the words of Apexon’s recent white paper on continuous testing, Digital Demands Continuous Improvement, “delivery paces are becoming faster, and that will mean more test cases. If organizations don’t figure out, and quickly, that they must automate their existing backlog of manual test cases, they will be stuck with more test cases, less time, and no feasible way to determine the correct gap analysis for future releases – leaving them in the digital dust with low quality product and inferior user experience.”
Our take: Are you able to test your system as often as you like? If the answer if is “no”, don’t despair. There are now plenty of great tools that can help with all aspects of automated testing, whether it’s test case migration, optimization, test environment management, acceptance, data management, prediction or prescriptive analytics. Our own AI-powered testing suite can do a lot of the heavy lifting when it comes to increasing automation. It’s proven to free up test resources by up to 35%.
Challenge 3: When Digital Becomes Physical
With IoT deployments, testing needs to account for the physical properties of the sensors involved and how practicable it is to upgrade these. They can be situated in hard-to-reach places, for example, either in remote locations, embedded in another device or even worn by a human. Connectivity issues and error tolerance are additional, related considerations. Weather conditions, geography and sudden impact all need to be accounted for in real-world testing scenarios.
Our take: Ground-truth validation is something we talk about a lot in IoT testing because these connected products and solutions require rigorous and reproducible development and testing of all the components under real-life conditions before going to market. A labs-based approach enables all the product’s interactions within its ecosystem to be tested together synchronously. By bringing together different test capabilities, a lab environment can reproduce almost any scenario. For example, in cases where delivering updates to the devices themselves isn’t practical, a labs-driven approach will define the extent to which an organization should deliver software-only upgrades to release cycles.
Challenge 4: Privacy and Risk in IoT Testing
The latest Software Fail Watch report from Tricentis identified 606 recorded software failures, affecting half the world’s population and $1.7 trillion in assets. Not to mention the negative headlines, tanking stock price or failed careers associated with a security breach. IoT exposes new attack surfaces and other security threats, so it’s easy to appreciate why we need to address security from inception to delivery. With IoT projects, the risk of customer data loss is not the only consideration. As we increasingly rely on IoT-enabled devices, digital vulnerabilities can have physical impacts, leaving people without power, or stranded, for instance.
Our take: Security is a foundational consideration in IoT development and testing. Continuous testing combined with a DevSecOps approach – where security considerations are “baked-in” from initial design onwards – is the best methodology. Apexon Security Testing Services work with the leading security solutions and tools to cover every aspect of the product lifecycle from requirements gather, through the dev and test cycle and onto production.
Whatever your experience of IoT project testing, we’d love to hear it. You can use the form below to get in touch.
