Our blog

Quality Goals: Define and Measure Quality of your application

Often when we talk about quality of software,we define it in terms of number of testcases,bugs,requirement coverage but this numbers in itself are not enough to incrementaly measure the quality of the application as well as quality of testing.The quality goals are owned by QA but defined by entire team and should be as important artifact as a test plan.

What are Quality Goals?

It is collection of individual metrics that helps to quantify quality of application as well as quality of testing.

Why define and measure Quality Goals?

Agile process gives QA as well as management to change or prioritize tasks at end of each iteration.The quality goal metrics accomplishes two things

  • Establish protocol between product management and engineering what are the expectations from the product.
  • Empower QA to have product management/development prioritize between new feature development vs fixing old bugs

How to define Quality Goals

The answer to above question will largely rely on processes in your individual institution,but I will take the lowest common denominator which I believe should be present in any QA infrastructure

The first part is to define ways to measure if the testing is sufficient.If we are not doing the testing right,we lose the right to say that quality of application is sub-par.

Quality Goals for Testing

  • Written Testcase /Requirements ratio:This number should be greater than 1.Ofcourse this assumes that requirements are complete and sufficient.So for this discussion let’s assume that all the requirements that are known to entire team should qualify for this metrics.
  • Written Testcases/Executed testcases ratio:Again the ideal goal is this number should be 1,but remember quality goals are determined by prd management,developers,QA as a team,so you can estimate lower.
  • Bugs yet to be verified

Quality Goals for Application

  • Non-executed testcases:The number to achieve is 0.
  • High severity Open Bugs/Total Bugs
  • Medium severity Open Bugs/Total Bugs
  • Low severity Open Bugs/Total Bugs
  • Untargeted Bugs

The above metrics will vary depending on your own process and outputs that QA efforts produce.You can define it in terms of iteration,release milestones or combination of both.E.g it can look like following.The following figure shows

quality goals for measuring quality of testing

Written Testcase /Requirements ratio

Written testcase/Executed testcases




Soft Code Freeze



Hard Code Freeze






What to do with this numbers ?

The idea is to measure this against actual numbers and depending on that take corrective action to either achieve the goal or lower the goal(Ofcourse in practice,management will never like it if you say that we would be shipping with 50% of intended quality).The more often we measure this,we have the option to do something about quality of application.

Please add more metrics and ideas about how to quantify testing and application quality.

Next time a brief talk about integration testing.

PS:Just got a chance to view a presentation by Selenium author and WebDriver author.Glad to know that the Selenium 2.0 will merge the two.It is a huge step towards correcting some of the shortcomings of each tool.Presentation can be found here.Video is here

Interested in our Testing Services?

Contact Apexon +1 408-727-1100

By submitting this form, you agree that you have read and understand Apexon’s Terms and Conditions. You can opt-out of communications at any time. We respect your privacy.

Other stories you may enjoy...

Healthcare Apps and the Need for Security

One of the most exciting areas in tech right now promises to be “the most personal” ever. A key aspect of making wearable devices like the Apple Watch personal is through...

Developing an App for the 2020 General Election?

Here is a thought: With the UK General Election having just finished, could the next one in 2020 be the first to use a mobile app to allow people to vote? The polling...

Be honest. Describe the state of your test cases.

“There’s some dead wood in there.” “Hmmm…. Someone really needs to clean them up.” “A little outdated.” For those reading this in the northern hemisphere,...