Building a robust regression suite is a challenge in today's competitive and fast-paced environment. It demands reduced timelines. Risk Based Testing (RBT) is an approach for building such a suite, for successful regression testing. RBT prioritizes the features and functions to be tested, based on their levels of risk.
Despite these benefits, it finds limited use across organizations.
The key reasons for this are the lack of:
- Understanding and implementation of RBT concepts by the testers
- Well-defined methods to measure the success of RBT
- Stakeholder involvement for RBT planning
Mindtree’s RBT implementation approach
Identification and analysis of risks by testing teams:
Testing and risks are closely associated with each other. As testing uncovers more defects, more risks are exposed. Hence, building a robust regression suite is vital and testers play a key role in it. Testers need to:
- Ensure that the highest risks are addressed by testing first
- Schedule tests based on the priority of the risk
RBT lifecycle process
The proposed RBT approach is planned to be implemented for both testing new requirements and regression testing.
The business impact has to be provided along with the requirements identified for the release, by the business. Business Process Blue Print (BBP), is the document where the core business process is mapped to the functional and sub functional areas, along with the business impact. New requirements can be prioritized based on their business impact and development complexity. This enables an effective test planning and effort distribution.
For the regression testing to be targeted and non-redundant, identifying the regression suite should be concentrated around the impacted functionalities. This helps in successfully building a robust regression suite, making RBT implementation easier.
New requirements
- Identify probability of failure
- Evaluate business impact
- Identify risk areas
- Prioritize test execution
- Prioritize test design
- Update test management tool
Business Process Blueprint (BPP)
- Identify risk areas
- Facilitate risk-based effort prioritization
- Provide targeted & CORE regression test cases for new requirement
- Do feasibility study on prioritized regression suite
- Automate high priority regression suite
Monitoring and measuring risk-based testing
It is imperative to measure and quantify the effectiveness of implementing the risk-based testing approach. The following listed metrics should be collected and analyzed, before and after implementation of RBT. This will enable the teams to evaluate its effectiveness. This includes:
- Test execution productivity
- Defect removal effectiveness
- Post release defect density
- Percentage of re-opened defects
- Rate of defect detection
Case study
The customer is a major domestic airline in the USA, providing short haul, high-frequency, point-to-point, low-fare service. Their applications go through constant changes to meet their business requirements.
Business challenge
- Testing a few major applications within the scheduled project timelines
- Lack of required experience in offshore project execution
- Limited availability of SMEs who can bolster new team members
Mindtree’s RBT framework for the customer, aims to evolve the testing process into a more structured one. Our testing team, in partnership with the customer’s team, has implemented a risk-based testing approach. It strives to reduce and optimize the testing, with risk evaluation methodology as one of the solutions.
Customer benefits
- Significant improvement in product quality and on time delivery
- Quantitative measurement of testing effectiveness through the metrics
- 25% reduction in test cycle execution
- Achieved process standardization for both up-stream and down-stream application
Key benefits
- Improves detection of critical defects
- Reduces test effort by focusing and prioritizing test case execution with higher risk
- Enables better risk-informed decision-making
- Reduces the need for a full regression testing through pre-calculated risk analysis
