Downside of not having a formal defect analysis feedback system

Our team was getting ready to present a 56-pager gap consulting report that our Engineering team prepared for our client, a global e-commerce retailer

A little backstory:

This client had chanced upon our site, had taken the QA maturity quiz, and left the contact details for us to get back to them. From their response to the quiz, we got a high-level understanding of their product quality gaps in the system and the status of their geographically distributed teams’ testing processes.

After the initial meetings, knowledge transfer sessions on product scope, consultants at Zuci advocated taking a holistic approach to product quality. By holistic we mean – product quality is not restricted to QA teams. Quality is a shared responsibility and teams should approach product building, writing lines of code and other activities with a quality mindset.

When such a quality culture is set, engineering excellence then, becomes a habit and not an act.

So, in the first few weeks of the consultation, a 5-member team with mixed backgrounds reviewed the engineering practices, test engineering processes, and complications standing in their way to QA effectiveness and defect-less releases.

There were findings, reviews, and recommendations made in the software engineering processes, such as:

• Engineering agile process
• DevOps implementation & utilization
• Change management practices within the organization

A quick glance through the report, I found the “absence of formal defect analysis feedback system” in the observation section interesting. I wanted to pick this topic with our Testing lead, Madanika Rani, and see what she has to offer on the subject.

I asked her, “what does a defect analysis feedback system do?”

Madanika:The defect feedback analysis system helps test teams analyze defects, identify their root cause, and help us take measures to minimize or arrest these defects.

Defect leakage to production can happen due to any of the following reasons:

• Minimizing regression coverage
• Neglecting edge case scenarios due to some setup
• Stakeholders possessing a minimal or surface-level knowledge of the application
• Minimize test coverage to meet deadlines

On testers’ involvement in the STLC for defect identification, she adds, looking for bugs should be done right from the unit testing phase, with the agile methodology being at the helm of software development.

A snapshot of a defect analysis feedback system:

So, what are the downsides of not having a formal defect analysis feedback system?

1. Developers face more difficulty maintaining product quality as they don’t have explicit knowledge about the areas the defects will impact. There are chances that they will find a temporary solution for the issue which will come back to bite later.
2. Developers won’t have any reference to previous issues that occurred, and the resolution provided to resolve the same.
3. Developers won’t be able to prevent the other areas of the application from an issue or its impact in the absence of the issue’s root cause/history being fed back. This happens when a new team member or someone without in-depth knowledge about the product joins.
4. It becomes challenging to reduce the re-occurrence of an issue. Issue repetition happens only when it was not analyzed properly earlier and not having a permanent fix to the issue.
5. Increases rework required to resolve any issue. When a defect is raised, we won’thave access to thebackground of the issue, giving rise to analyzing the issue from the start.
6. Rework requires more time to analyze and resolve the defect, also, it takes huge resource consumption.
7. Miscommunication is bound to happen among development teams, testing teams and managers when feedback analysis systems are not in place.
8. Overall Software testing life cycle (STLC) time is lengthenedas a lot of time is required to resolve defects and their recurrence
9. Developers and testers face difficulty in finding bugs asearly as possible in the development cycle.
10. Teams use multiple tools for defect tracking. Without a proper defect feedback system, migrating platforms and integrating with each other becomes a huge challenge. Thus, leading to a loss in description/workaround of issues from the developer/tester perspective/business View/user comments.
11. It’s not quite easy to track non-technical issues/environmental issues withoutproper feedback loop systems.
12. Bugs can’t be fixed easily without a system that capturesdetails on the previous history of team dependencies etc
13. In the absence of a proper defect feedback system, we’ll slip through some of the details of the activities that are stored in the brains of the teams
14. Teams run into the risk of reduced traceability and not being able to provide valuable metrics
15. Defect feedback system acts as a repository for team members to troubleshoot issues. When deprived of it, developers/testers struggle to resolve tedious issues.
16. Without a proper history or the knowledge of an issue, reporting becomes questionable
17. Learning lags when the developer/tester doesn’t get a chance to review the feedback of resolved defects.
18. Delivering the project within budget is difficult as we have redundant bugs and high resolution time.
19. Complaints from end-users will be high and CX will go down
20. The manual process of collecting data gets tiresome in addition to the daily tasks
21. Feedback/ Ideas from automation are scattered, and teams won’t be able to easily fix the redundant issues and more priority problems.
22. Impossible to track a defect background and if someone has worked on it, unless we have the defect feedback analysis system.
23. Getting closer to the zero-defect or defect less goal is hard
24. It’s tricky to categorize defectsor make it traceable across revisions
25. Increased RISK when the developer/tester handles an issue without any previous issue history or knowledge about it.
26. Reduced visibility to others working in a team.
27. Slows down developers’ decision to sign off as they are not very confident about the issue resolution they provided, that happened earlier until the customer accepts the solution
28. Delays product recall recovery process as testers don’t have a dump of defect history.
29. Lack of information circulation among teams/management.
30. Minimizes measures taken to control defects that affect significant functionality.

An efficient defect analysis feedback system is more important for teams with significant production defects despite fair testing practices.

Developers/Testers can’t store so much information about bugs in their minds, nor does it help to have all this information in siloes.

So, having a checklist for defect analysis which includes ‘Steps to reproduce included,’ ‘Screenshots included,’ ‘Software version added,’ ‘Memory dump added,’ ‘Log added,’ ‘Detected in Release,’ Source Release,’ ‘Release Version,’ etc. helps in defect management and reporting.

Like to read the gap consulting report? Send us a ‘hi’ at sales@zucisystems.com