Assuring product quality? Here’s the QA scorecard you need

We’ve consulted companies from across the industries for years and realize time and again that setting up an intensive testing process, reviewing it, and making improvements periodically is a sure-shot way to release products that resonate with the end-users and all the stakeholders involved.

One of the product quality owners at a firm, for instance, said they had been in the industry for a long time and had been quite adaptive to the trends such as test automation, setting up a continuous integration/continuous delivery model, etc., and yet they were still finding P0 P1 defects escaping into the production.

Though setting up a product QA looks reassuring, the end results were not what they were looking for.

We agree that no product or software is defect-free. But it can defect less.

And that’s just one example.

There are companies with different approaches to software quality, some doing so great, and others have distorted views of quality by measuring wrong metrics and resulting in bad decisions.

Big or small, incremental changes happen when you establish a basic software quality scorecard that measures the STLC process right from requirements till you hit the deploy button.

QA score card: What’s it?

A software QA scorecard is similar to your fit bit watches: Like the panel displays how many steps you have taken, sleep hours and other vital information, a product QA scorecard shows the KPI of your product quality – the metrics that will present how well your product quality is.

In short: Quickly identify areas of improvement, inefficient processes, or underperforming teams with QA scorecards.

Who uses it?

Organizations of any size, be it small or big, use QA Scorecards to improve the team’s performance and their QA strategies. An organization can use the QA Scorecard template with a list of checklists as the guideline for the team to meet the Quality and the standards as expected by their customers on a regular basis/cycle.

QA scorecards become helpful to the following people in general:

• QA analysts
• Team leaders
• Managers
• Supervisors

Engineering teams can prepare a QA scorecard by themselves or seek external help based on the need.

However, the QA team by themselves cannot assume the quality/performance of the team without evaluating against any checklists, guidelines that the team/organization is following.

QA score card benefits: 

Following are the benefits that the QA team can infer from QA score card: 

• Helps identify the areas of improvement in the quality roadmap 
• Clear understanding of the status or quality based on the feedback 
• Set up a benchmark and teams can learn best practices as well 
• Helps to monitor progress of the testing activities. 

Building a QA scorecard

As mentioned earlier, QA Score Card is a basically a template with the list of standard questions based on the project or the business. It consists of a limited set of questions, let’s say 10 to 20 to determine the quality.

Every question posed to the team comes with a briefing and detailed explanation of the expectations and standards to be met to maintain the quality, both functional & non-functional. Also, the weighing of the questions differs based on what the team wants to achieve.

For example, here’s an example of QA scorecard questionnaire you can use to measure your entire QA activities

The scoring is usually done in numbers from 0-5 or with basic Yes/No option to arrive at the score. The final score is provided in terms of percentage out of 100. 

Example of a QA scorecard:

As you can see, this team scores well on CI/CD processes, but they still lag in the Production readiness testing and setting up a defect analysis system. Assuring a product/software quality depends on all of these parameters put together.

Scoring in detail:

The objective of any product quality team is – continuous improvement and performance. A team can make use of this QA scorecard for two different types of projects.

• Existing project
• New project

Let’s take an example of an external QA vendor working with a customer who wants to arrive at the QA score for their product, the vendor’s team will have to come up with a clear goal before constructing the QA scorecard questionnaire.

In most cases, the customer wants to know the reason behind the issues in production, and missed bugs/defects. In such cases, the vendor will kickstart with an initial engagement, wherein they will coordinate closely with the customer’s team to get a grasp of their working model, test process, and work culture and arrive at the set of questions that will help meet the goals of the customer.

The benefit of doing this is to help understand if the bug leakage to production happens due to poor QA or all of their engineering practices combined right from the requirement to release.

Arriving at the final score involves weekly meetings with the team, studying their requirements, speaking with their engineering team, processes they follow, and going through test artifacts and defect tracking tools.

Analyzing and presenting the score usually spans 8 weeks followed by the sign-off process to the client.

For teams that are just starting out to build a product, the team can have the QA scorecard template as a checklist from the initial state to maintain the quality of the product.

Doing so would help them to bridge the process and knowledge gaps, and areas they need to work on to improve their performance and be successful in terms of delivering a seamless user experience.

Criteria for QA Score Card

What can QA teams infer from the scorecard?

QA teams infer the following from the  Scorecard:

• Based on the score, they identify areas of improvement viz., process, documentation
• Root cause of the issue
• Quality level of the product
• Evaluate themselves against the standard procedures/metrics

Ways to improve QA Scorecard

Just like any practice, quality involves continuous improvement. Arriving at the QA scorecard should not be viewed as a one-time process. Following are a few ways to put the QA scorecard in a continuous improvement mode.

• Monitor systematically
• Collaborate among teams (Dev, QA, Ops)
• Rewards and recognition within the team
• Transparency and continuous feedback system

Looking to improve your test automation coverage? Take a look at Zuci’s test automation services and see how you can leverage Zuci for your business needs.

The article is co-authored by Vidhya Shanmugham, Test Lead @Zuci Systems.