Machine learning and natural language processing bring the power of data science to help businesses cut the costs, time and complexity of managing reports.
By Prem Swarup
I like taking pictures and have been capturing different scenes and memories since the ’90s. Not surprisingly, I have a lot. Some have been printed and are kept in albums, some are in boxes while others live on various thumb drives and even an old laptop. More recently, I have been storing my pictures in the cloud courtesy of Google Photos. And this summer I decided to get all my photos and pictures in order. It was an enlightening process, and the similarity to business intelligence (BI) organizations beginning their own report rationalization process was striking.
If you are like most BI and data leaders today, you receive almost daily requests for new reports and analysis. Frequent user requests to test or acquire exciting new BI platforms is also common. Over time, your company ends up with a lot of reports in different formats and locations. And just like all those old pictures of mine, it becomes very challenging to manage this diverse ecosystem of reports — not to mention expensive, because reports need to be stored and maintained. Unfortunately, you can’t just keep them in an old shoebox.
This summer, I decided to 'rationalize' all of my pictures. Why did I have so many? Were they all for keeps? And why did I take almost identical pictures of the same scene with only slight variations? I went through a process remarkably similar to what we tell our clients. I discarded all the no longer needed photos, pictures and duplicates. I got everything converted to the same format (digital) and consolidated my pictures in the cloud.
Even better, by modernizing my entire collection on Google Photos, I can now take advantage of Google’s powerful AI techniques to very quickly analyze what I have and find what I am looking for as they are all indexed and labelled automatically. I can even search by date, location or a specific person.
For many of the same reasons, business organizations need to rationalize their many reports on a regular basis — either as a way to reduce costs, improve performance from a smaller report set, or as a precursor to migrating to a modern BI system that could bring data transformation in business intelligence. Unfortunately, most report rationalization processes are still primarily manual in nature, typically taking about 10 hours per report. That’s very time consuming and cost prohibitive even for a relatively small set of 100 reports.
Fortunately, a new approach has emerged that utilizes the promise of data science, specifically machine learning (ML) and natural language processing (NLP), to reduce the cost, time and complexity of managing and consolidating reports. NLP can be used to extract and read report headers or data labels — not the data itself — in any document type and from any platform. Data analytics service providers could design a roadmap and implement such efforts.
Similarly, ML can be used to provide a comprehensive review of all the existing reports and documents and automatically identify duplicates (reports providing data on the same report fields) as well as similar reports (reports with an overlap of report fields). This way only the ones that are required by the business and used regularly will be consolidated.
Report rationalization driven by data science has consistently delivered more than a 30% reduction in the total volume of reports. In my experience, reports have been reduced by over 70% and costs by up to 40% in some cases. In addition, BI consolidation can provide better insights from a smaller, more efficient report set and support a BI modernization initiative.
For organizations about to embark on their own report rationalization process — especially if they leverage data science to do the heavy lifting — they will get to enjoy many of the same benefits as I did after getting three decades of my photos and pictures in order. And that should make any manager, especially those with BI responsibilities, smile.
Click here for a more in-depth look at report rationalization using data science.
30% Reduction in volume of reports
25% - 40% Reduction in costs
About the author:
Prem Swarup is Vice President, Data & Analytics, Iris Software, Inc.