5 data challenges facing financial services firms

From disparate data sources to a lack of integration, financial services must overcome many challenges as they move from dark data to insight

The financial sector generates a truly enormous amount of data, with large businesses facing data sets in the petabyte scale. The vast majority of this data remains unused for analytics, instead tucked away in legacy systems and data silos where it is not readily accessible for driving informed decision-making.

With billions of dollars shifting across global markets every day, financial analysts face a huge challenge when it comes to monitoring and understanding data at scale to untap its true value. However, by garnering an understanding of the challenges and embracing newer business models, leaders can create a strong competitive advantage.

Here are some of the most pressing challenges that they need to overcome first:

#1. Regulatory requirements

The success and sustainability of the financial services sector revolves around trust. Without trust, they have nothing. To achieve and maintain that trust, financial services firms must work hard to adhere to external regulatory requirements and internal policies like corporate social responsibility (CSR).

The rapid proliferation of data, combined with the evolving regulatory landscape, has placed firms under enormous pressure to modernize their systems. After all, it is much harder to meet the demands of things like data-retention and privacy rules when you have information spread across disparate systems without a centralized governance model.

Given the maze of regulations around financial data access and use, businesses must strive to consolidate their data streams to scale risk-management in a way that is cost-effective but flexible enough to leverage that data for analytics.

#2. Data integrity and quality

Having lots of data is often considered a good thing, but it is also important to remember that not all data is useful. When data is coming from many different sources, it is only a matter of time before conflicts start to arise. When that happens, poor data quality becomes a serious challenge that leads to defective analysis.

When data integrity is compromised, it is usually a result of human error. That said, a machine learning algorithm used to analyze data at scale will only ever be as effective as the data used to train it. If the data sources themselves are burdened with outdated or conflicting information, then the results will likely be problematic too.

To overcome challenges around data integrity and quality, financial services firms must work towards pruning and integrating their data streams. The end goal is to have a single source of truth (SSOT), instead of conflicting data streams all result in different insights.

#3. Information security and privacy

Most of the world’s data is so-called dark data, which refers to any data that goes unused for analytics and is not comprehensively managed. In the case of finance, dark data can present a serious risk, simply because it is impossible to properly protect what you do not know about. Dark data often falls outside organization-wide security, privacy, and compliance controls too.

The financial services sector is, unsurprisingly, a favorite target for attackers. Moreover, threat actors almost invariably go after the easier targets. Dark data is inherently more vulnerable to attack than data collected and stored by systems in production, yet it can still hold a wealth of value for attackers to misappropriate.

Information security and privacy cannot simply be tacked on later. Instead, they must be baked in by design in any data governance strategy and the tools used to facilitate it.

#4. Data and organizational silos

The three aforementioned challenges are exacerbated by the fact that many companies have data widely distributed across legacy systems. This is especially likely to be the case in well-established financial services firms, many of which have been around for decades.

Banks often hold huge quantities of data in legacy systems that do not interface well, if at all, with newer cloud-based systems. This is neither an easy nor cheap challenge to tackle, as it requires time and investment to extract data from old systems and translate it into a format that is ready for modern analytics.

While modernizing data systems should be the primary goal, financial services firms also need to take a realistic approach by iteratively overhauling and replacing their systems to integrate their data streams.

#5. Lack of talent and experience

Big data analytics is a relatively new and emerging area born of the rapid and unprecedented proliferation of digital assets. Indeed, technology has evolved at such a pace that companies have struggled to keep up with the developments. As such, there is a growing scarcity of those with the required skills and experience to tackle big data effectively.

While the coronavirus pandemic has seen a surge of interest in high-tech skills, including big data analytics and information security, many companies still have a long way to go to fill all the skills gaps.

To mitigate these issues, they must adopt the optimal blend of employee training and career development, outsourcing, and automation.