The challenge
The Microsoft Assessment and Planning (MAP) toolkit is an agentless inventory, assessment and reporting tool that securely assesses IT environments for various platform migrations. This includes Windows 8, Windows 7, Office 2010 and Office 365, Windows Server 2012 and Windows 2008 R2, SQL Server 2012, Hyper-V, Microsoft Private Cloud Fast Track and Windows Azure.
The key challenge is to ensure two major MAP releases every year, along with the release of a couple of target customer-focused ones. It is a huge challenge to manage resources under tight timelines and schedules, for these parallel releases. It is also a big task to have a varied environment to effectively test and add new features to every release, without affecting the legacy features.
Our solution
We have established a robust delivery model, with our in-depth understanding of the customer’s business needs. Exhaustive training has been given to teams, version after version, to keep them updated with new features and technologies. The teams have been divided into mini feature teams and they work closely together to stay abreast of any new change. It also helps to have onshore Mindtree minds. They can help offshore teams understand new features and resolve issues / challenges quickly.
Our approach
- Provide end-to-end ownership and accountability of testing
- Ensure senior members own different modules, with junior members assisting different modules
- Implement BVT automation execution on daily builds, to catch severe issues
- Facilitate regular performance test runs, to help keep the performance in check
- Maintain an offshore lab to help reduce the dependence on the onshore lab, enabling complicated setup scenarios
- Enable C# and SQL code coverage via manual testing and automation testing, to keep the test suite up to date
Business impact
- Ensured consistent and measurable quality, through adaptation and implementation of SLA-based service delivery
- 90% on-time delivery
- >= 95% test coverage
- Developed one click automation and schedule based Build Verification Test (BVT) run, which can run without manual intervention. This saved 15% of trivial manual effort
- Implemented regular performance test automation that helps to simulate MAP inventory, against huge environment / machines. This saved a lot of time and labor
Customer accolades
Thanks a lot for making the best effort to
- Find an alternative solution
- Understand the impact and its limitation and working around it to unblock RTM test execution, thus reducing the impact
Given the limited lab resources yesterday, the offshore team was still able to make good progress on legacy test execution. Thanks. I really appreciate it.
-Ying Liu (Test Lead – Microsoft)
Team, awesome job of wrapping up the RTM test pass! The overall quality of this release is far and away, the highest that we’ve ever achieved.
– Product Manager – Microsoft
