Our blog

Software QA for SaaS Firms – What is different?

SaaS (Software as a Service) delivers software over the Internet without the end users ever needing to install any software in premise, behind their own firewalls. On the surface doing software QA for a SaaS firm should be no different. After all, you are doing same old testing for an ISV (Independent Software Vendor), it is same old software, with a development life cycle, (well written, planned ☺ ) requirements, release plans, defects, triages, wrangling with the developers, resource crunches and so on; so what can be different?

There is a difference and it is significant. Worse, not knowing it, can impact the quality of your product and credibility to your customers. To understand it first lets see how SaaS is different (For a detailed account of how SaaS is different please read our “SaaS Operational Efficiency” white paper).

How is SaaS different?

  • SaaS is more of a service than a shrink-wrapped software
  • SaaS solution has more stringent performance and security implications
  • SaaS business model enforces an Agile methodology of software engineering
  • There are significant operational costs (infrastructure, software and deployment)
  • A new release (and its defects) is immediately available to your customers, with a very high risk of an error
  • Your customers have a lower risk, cost and impact to switch you and move to a different provider.

The above impacts SQA in more ways than what meets the eye. Following is a list of   key areas QA gets impacted:

  1. Agile Methodology: SaaS firms have so stringent time to market pressures that they are forced to adapt an Agile methodology. With a release happening every month or less, QA teams have to turbo charge their productivity, in order to ascertain good quality level.
  2. Performance and Scalability Requirements: SaaS firms deploy their software on their own hardware. Poor performance exponentially increases the operational costs and in worst situations becomes a death bed for the company. QA teams have to validate the principles of multi-tenancy and test each release for its scalability.
  3. Web Services and SOA: As SaaS software gets deployed behind its own firewall, for most enterprise applications, there are strong needs for data integration. Most SaaS products have a rich WS layer that enables them to integrate with third parties. This actually is a blessing in disguise as it provides an easy interface for QA to test the key business functionality through an easy to automate interface. QA teams however need to know how to best test the SOA platform. Please read about some specific work we have done around SOA Testing here, here.
  4. Test Automation: I believe SaaS QA teams (or for that matter, any agile lifecycle) can not survive without significant level of automation. This includes (but is not limited to) good unit testing, GUI automation, functional tests of key application interfaces (such as your main controllers), SOA interfaces, performance etc.   Having a good framework that extends itself to meet ongoing automation needs is critical. We have blogged about automation here and here.
  5. Security: As a SaaS company you are taking ownership of safekeeping of critical customer data both morally and legally. Any error with security can shut your company down. As a QA team, you have to know and test for major web exploits and plug them.
  6. Release Automation: Just like test automation, any agile methodology can not survive without automated builds, releases and deployments. QA teams have to understand and implement continuous integration process that builds regularly and reports failures in an automated way.
  7. Test Environments and clean Deployments: While the SaaS model is a boon for customer support (they are no longer required to support the unforeseen mess weird combination   of OS, server and software environments in customer IT, they just support what is in their production); it is yet another pain for QA. How do you know what you are testing will work in production? QA needs to have very tight control over deployment scripts and deploy the software in the test environment. The test environment itself needs to replicate production environment closely including all the load-balancing and clustering infrastructure.
  8. Testing for SLAs: So what if the code functionally works and passes performance criteria? Does it meet your SLAs? How do you guarantee your software will not breach a functional, performance or uptime SLA in production? Again it is QA’s extended role to validate for conformance with SLAs.

QA Managers working for SaaS firms have to keep the above points in mind when creating a test strategy for their SaaS product.

Interested in our Testing Services?

Contact Apexon +1 408-727-1100

By submitting this form, you agree that you have read and understand Apexon’s Terms and Conditions. You can opt-out of communications at any time. We respect your privacy.

Other stories you may enjoy...

Healthcare Apps and the Need for Security

One of the most exciting areas in tech right now promises to be “the most personal” ever. A key aspect of making wearable devices like the Apple Watch personal is through...

Developing an App for the 2020 General Election?

Here is a thought: With the UK General Election having just finished, could the next one in 2020 be the first to use a mobile app to allow people to vote? The polling...

Be honest. Describe the state of your test cases.

“There’s some dead wood in there.” “Hmmm…. Someone really needs to clean them up.” “A little outdated.” For those reading this in the northern hemisphere,...