Client
A globally operating investment company, which provides an offering of unique and powerful B2B Behavioral Marketing solutions. With a focus on converting intent data into measurable ROI, the client provides accurate and targeted findings, collects and analyzes data on buyer behavior, for various industry sectors.
Challenge
The client wanted to have a unified and scalable solution in the form of a sharable analytics platform that could cater to both data scientists as well as business teams. The data handled would include performance data, financial data (like stock estimates/pricing, market data, etc.) as well as marketing data (including sources such as social media, website).
One of the primary issues for the client was that their specialized resources were spending too much time in consolidating and cleaning source data. Moreover, information was getting distributed across disparate systems, with a variability in the levels of confidentiality. This led to many challenges in terms of data aggregation as well as access management. In addition, multiple teams were using various sets of softwares to get insight into organizational data - and this had additional cost as well as maintenance overheads.
This solidified the requirement for a modern, cost effective, scalable and shared analytical platform.
Solution
As the first step, Mindtree created a solution where data was ingested from other data warehouses (SQL DW), Medm Platform (Fund Benchmarking Software), Factset, Reuters, Google Analytics, etc. Then, we architected and designed a Cloud-based platform that was cost-effective, scalable, and could be shared across teams. This solution was deployed on Azure HDInsight, driven through ARM templates, where the data was ingested via scheduled Data Factory pipelines. A CI/CD setup was also implemented for deploying scripts, data factory pipelines and configuration tables. Moreover, there was also an automated cluster scaling via runbooks on HDInsights.
Benefits
As a result of creating a shared analytical platform, Mindtree was able to deliver the following benefits:
- Increased the user base from 5 to 50+ in a year - and it’s still consistently increasing
- Optimal management of 5 clusters with 65TB of data in data lake
- Optimized cluster usage via automated cluster scaling for compute. where upto 10 worker nodes are reduced when demand is reduced for a cluster. 16 to 4.
- Supported the model scaling and productionalizing along with customer data scientist
- Enabled the reutilization of data, factors and techniques that have proven value for better collaboration amongst the data scientists
- Enabled users to deliver advanced performance analytics and insights, eg. page views, session views, campaign effectiveness, and social media analytics
- Empowered users to provide predictions and recommendations to support decisions
