80% of Data Is Generally Considered Unstructured Data and Is Left Unused for Decision Making.
Find Out What 4 Insights Are You Missing Out on by Not Utilising That 80% of Data
By Michael Benz and Moritz Müller of Synpulse, a Squirro Full Service Gold Partner. The post originally appeared on the Synpulse Blog.
Banks and financial services providers have embarked on a journey of digitalization with massive investments into banking systems and digital transformation. As owners of these increasingly digitalized platforms, they have come to realize the potential of the highly valuable but untapped information that their systems have been accumulating over time. In the following article, the authors illustrate why it is essential for banks to start using unstructured data and where areas of potential applications can be found.
It is common practice in banks (and other industries) that business analytics are carried out using conventional tools and systems (i.e. relational databases) to handle any form of structured data. However, this leaves the vast majority of data which is collected unutilized and incomputable: because the data is unstructured and it is scattered across many locations. It is estimated that more than 80% of data is consequently meaningless — unable to act as a foundation for business insights and decision making.1 However, the attitude towards unstructured data has started to change. Banks are considering how they combine their digital transformation and data analytics to reap the full benefits of data driven business insights. This can only happen by including the data with most insights in the analysis. News, earnings calls transcripts, social media, multiple CRM systems, financial filings, call notes, email exchanges and many other sources are all part of the untapped unstructured data pools.
By integrating them, conventional decision-making will be moved up on a higher level of ultra-informed decision-making.
From Traditional Keyword Based Searches Context Driven Real-Time Insights
Until not long ago, the worlds of structured and unstructured data have been kept more or less separate. In fact, only the former has been directly useful for automated or semi- automated analytics.
Accordingly, the latter has traditionally not been part of the business decision-making process. Structured data relies on defined fields with data types, input restrictions and other assigned parameters. It is easy to enter, request and analyse them. The technology and systems to work with structured data go back several decades to the dawn of IT.
Unstructured data by contrary has its origin in a much wider range of sources like Word documents, email, PowerPoint presentations, survey responses, transcripts of call center interactions, and posts from blogs and social media sites, etc.2 It is typically text-heavy but may also contain data such as numbers. This leads to irregularities and ambiguities that make it more difficult to understand using traditional programs as compared to data stored as fielded form in databases.
When companies try to include unstructured data in their quest for insights, they typically face a series of challenges:
Unstructured data is not included in traditional enterprise search engines.
- Traditional keyword-based search methodologies are used. Such technologies are of limited value as they rely on the user knowing exactly what they are looking for and using the right keywords to find it.
Too much time is spent on inefficient and inaccurate searches.
A number of different CRM platforms and other enterprise systems are used, meaning both structured and unstructured data is stored in multiple locations making enterprise search even more challenging.
Coupling big (unstructured) data with Fintech capabilities is becoming increasingly popular as API services that let users hand off data to a cognitive system are becoming available. This means that a typical financial institution does not necessarily need to have its own deep learning expertise nor that all the systems have to be neatly integrated and consistent in order to generate insights.
Cognitive search represents a new generation of enterprise search that uses sophisticated algorithms to increase the relevance of returned results. It essentially moves the nature of searches from basing relevance on keyword hits to understanding user intent, observing behaviours and applying pattern detection to correctly assert the most relevant pieces of information.3 Structuring data and finding relations within it can bring tremendous additional business value. Even more, value can be created by employing smart analytic tools in combination with machine learning. Training these algorithms with the valuable expertise of analysts can be a game changer that allows a bank to differentiate itself and lead to even more educated investment decisions. The tools to harvest the full potential of data are here today.
Yet banks and financial services firms are not known for their ability to embrace innovative new technologies. There is an inhe- rent conservatism in financial institutions that means change can be hard to facilitate and organisations are often tied into their existing infrastructures, making new investment in technology even harder. All this has all led to many banks trying to do a challenging job without the insights and knowledge that could be made available if they adopted some of the newest and most innovative tools available.
What they are missing is significant. For instance, filtering and dashboarding solutions allows the generation of summarized information from various sources, in turn enabling users to dive deep into the data for efficient and insightful exploration. Users gain 360-degree views on what is happening, why it is happening, what to do next and who should be involved in the process.
Let us zoom in now on some use cases to see whether the solutions are ripe enough and ready for the banks to step into the richness of unstructured data.
Finding the Business Cases
Embracing a data-driven approach can revolutionise a wide range of business areas and functions from client-facing sales units, to the ideation and origination of financial market transactions, to IT service management and the way how risks are managed in credit portfolios or how background checks of clients are conducted as part of KYC.
The main challenge for CRM solutions is to obtain a centralized view of a customer based on all accessible data sources. Yet most CRM systems work only with structured data. New Fintech tools which build on cognitive search techniques are capable of integrating unstructured data as well as 3rd party content with in-house data to condense the client specific information in real- time. They add structure to that data to provide users with a comprehensive understanding of what their clients are doing and alert them to any potential new deals or leads. They may present the info in dashboards or even embed it in a CRM system. This enables the user — the sales person or relationship manager — to get an intimate understanding of markets and customers in real- time and to automate the digital research so that she or he can spend more time engaging customers with bespoke and relevant opportunities.
In any relationship oriented business, for a relationship manager to appear aware and well read into the client’s issues is certainly advantageous. They will be seen as competent about understanding a client’s business, its potential future requirements and the wider market that the client operates in. Not only that, they must also be able to use that knowledge and insight to recommend the best deals and opportunities for each client. A client will feel truly valued if she or he is being told about the best deals ahead of the competition.
A second area of opportunities for banks to use unstructured data lies in the ideation and execution of financial market transactions. These can be the early identification of new financing opportu- nities in real estate transactions, the identification of off-setting capital market needs among corporate clients or simply a timely retrieval of all clients who did certain types of global market trans- actions in the past. Such information can be extracted from sales notes, call reports or any other source of text document within the existing systems of a bank or from external sources.
A third area of application is IT service management (ITSM). There is a significant potential for employing more sophisticated tools to handle the unstructured information in IT service tickets. Finding correlations between similar tickets phrased differently but relating back to the same cause is an old and well-known challenge in ITSM. Innovative cognitive search tools are capable of structuring IT ticket data and identifying the right resolution. This does not only reduce the time to resolution, and thereby the costs caused by the downtime of the affected IT system, but also opens the opportunity to automate the ticket assignment process. Furthermore, the improved structuring of IT service tickets can be exploited to predict trends and identify anomalies for capacity planning and more efficient resource management. Eventually, multichannel self-service options (e.g. chatbots) can be combined with these tools to automatically provide the users with the right manuals and handbooks to resolve more common issues that have already been addressed before such as changing email signatures or dealing with network issues.
Last but not least, from a risk management perspective, the connection of external, unstructured news data with in-house risk analysis data enables risk managers to obtain real-time insights and early warnings for any portfolio of assets, corporate entities or individuals.
The insights gained from systematically screening and exploring internal and external sources help a risk manager obtain close to real-time insights of risk relevant events, from new policy announcements of financial regulators to debtor specific news. Requirements on the understanding of risk in financial institutions are constantly growing, and having adaptive tools to deal with new requirements will limit future business spending.
Real-time monitoring of news sources is also crucial for adverse media screening as part of the KYC process. Combining proactive compliance monitoring with standardised risk indicators gives an organisation a speed advantage that might be decisive to make highly informed, timely decisions ahead of critical events. Beyond improving the timing and quality of decisions in risk management, compliance or KYC tasks, these new unstructured data analysis tools also provide significant potential to increase process efficiencies and cost savings in those functions. Tasks like adverse media screening for individual high-risk clients can be automated to a large extent.
Conclusion
As has been shown in this article, the breadth of potential opportunities for extracting value from unstructured data is enormous for financial institutions. The main sources of unstructured data are news feeds, social media, earnings call transcripts, multiple CRM platforms, email, call notes and much more. The appeal and added value comes from a significantly improved information base by including unstructured data for decision making in terms of relevance and timeliness.
The use cases profiting from such enhancements are widespread, including building comprehensive customer insights, the ideation of and matching of deals in investment banking and global trading, adverse media screening for KYC and client risk management in general, and the optimization and proactive problem identification of IT services and increasing business efficiency.
In the experience of the authors, while many times some initial hurdles of more technical nature have to be overcome before an organisation can launch its first use case of working with unstructured data, once it is live, it is astonishing to see how quickly and widespread further applications pop up up and how fast the implemented solutions are adopted and appreciated by the end user.
Given the amount of unstructured and unutilized data which banks are collecting, proper data management might be the key for banks to transform their businesses to a digital world. Significant value can be unlocked by offering to customers and other stakeholders what they already expect as a baseline from non-bank digital service providers.
