Go Back

Decoding the AI Dilemma: Transforming compliance with regulatory guidance in mind

Vivek Shankar

September 10, 2024

Artificial intelligence (AI) is deeply embedded in our lives and communications surveillance in financial institutions is gradually following suit. Regulators have signaled how important AI is to policy—from the US Department of Justice appointing a Chief AI Officer and the European Union’s (EU) AI act to FINRA’s AI guidance.

In a recent webinar on AI’s impact on compliance, Alex de Lucena, Shield’s Director of Product Strategy, laid out the tension at the center of firms’ interest in AI’s compliance monitoring capabilities.

“There’s a contradiction that sits at the heart of interest and adoption of AI in the compliance space,” he said. “On the 1 hand, interest is elevated. Interest in compliance with AI is equally elevated. But there is some tension between the 2 because a lot of firms aren’t sure how to vet these capabilities and adopt them.”

The panel, moderated by de Lucena, included Lore Aguilar, Director of Surveillance Design, Research, and Analytics at Citi, Alvin Huang, Business Development at AWS, and Shlomit Labin, VP of Data Science at Shield.

These experts delved into how AI is dominating conversations in surveillance and how firms should proceed to implement it the right way.

Key takeaways:

Generative AI can potentially revamp how firms implement communications surveillance. But it isn’t a solution to every problem.
As regulators increase AI guidance, firms must ground GenAI use in the value it delivers and implement use cases accordingly.
Firms must also avoid blindly trusting AI outputs and validate them through multiple layers of technology.
Ultimately, ROI must drive GenAI implementation, given the complexity of scaling it across an organization.

From rules to reasoning: The evolution of AI in compliance

AI in compliance has evolved from simple rule-based systems to more sophisticated machine learning and natural language processing (NLP) technologies. The AI breakthrough in compliance began with firms using Large Language Models (LLMs) to execute research and information retrieval use cases. However, as LLMs grow in sophistication, its potential for compliance is increasing exponentially.

Labin noted that recent developments in AI have profound implications for communication monitoring and surveillance. 

“Even if we’re looking at a simple detection task, the understanding of the full nuances that occur in a communication can be done by a single model,” she said. “We can theoretically talk to it and perform multiple steps in our assignment with a kind of assistant that follows us around.”

Financial institutions have noted these developments and are increasingly leveraging AI to enhance their surveillance capabilities and streamline compliance processes, as Aguilar highlighted.

“At Citi, we have been using the traditional AI in terms of machine learning for our alert generation,” he said. “So it’s been useful in complementing our workhorse of communication surveillance: Our lexicons of keywords and rules.”

This traditional approach, however, has its limitations. For instance, LLMs cannot fully analyze the context behind a series of messages—something crucial to deciphering message intent. Firms are investing heavily in NLP and supervised learning approaches to overcome this hurdle.

Interestingly, and perhaps unsurprisingly, alert generation is the next use case under exploration. With more sophisticated AI tools, particularly in the realm of natural language processing and LLMs, finding real risk in alerts becomes a true possibility.

But this shift towards more context-aware AI is not limited to alert generation. The use cases continue to develop based on real need. At AWS, Huang sees a variety of financial organizations finding ways to use the technology for them. 

“A global exchange has integrated GenAI into their market surveillance technology platform,” he said. “Instead of an analyst having to go through and manually search for and collect all necessary data, this GenAI application will automate and produce for you in a single pane of glass things like a consolidated table of the company’s regulatory filings, news summaries, links to companies, sentiment analysis, and more.”

The impact of these advancements is significant. Huang noted that this implementation “saw a reduction of around 30% in the time it takes to close those alerts.” As exciting as these developments are, they merely scratch the surface of what AI can do for surveillance monitoring teams.

AI’s potential for new tools and use cases

Recent developments in generative AI are generating significant enthusiasm among industry observers. While earlier comparisons to comic book fantasies like Ironman suits may have seemed hyperbolic, they’re becoming apt. 

GenAI is turning into a superpowered suit for the compliance professional, sourcing information, summarizing it, offering context, and plugging gaps at lightning speed. However, AI tools are not replacing human expertise–just like Ironman didn’t replace Robert Downey, Jr.–but rather augmenting it.

 ”If we venture outside of surveillance and into investment banking,” Huang said, “tasks like summarizing historical research, generating new ideas, are being done.”

But we can’t explore AI without talking about what everyone’s wondering: What are the regulators thinking? 

These use cases come at a moment when regulators are recognizing AI’s potential for improving compliance. The EU AI act and recent FINRA guidance strike positive notes for AI adoption. Regulators are encouraging firms to explore more AI use cases and seem satisfied with existing controls. 

Some regulators, like the FCA, have even gone as far as explaining how AI could be used to improve compliance.

Summarization particularly stands out as a highly promising use case. Aguilar offers an example. 

“Let’s say you have an alert that goes on for multiple exchanges in a chat,” he says. “If we’re able to prompt it in the right way, that’s going to assist the surveillance analysts.”

This capability could dramatically reduce the time and effort required to process and analyze large volumes of communication data. Aguilar said that significant work is currently underway on building LLM classifiers to cater to more advanced use cases.

“We’re putting pieces together,” he said. “You can have classifiers that have certain indications of behavior. Like secrecy.” This shift towards contextual understanding could represent a breakthrough in compliance monitoring.

de Lucena posed the question where AI adoption can really be as easy as it sounds. The answer is no, challenges come with every revolution and they’re ever present in the compliance industry. 

The reality check: Challenges in AI adoption for compliance

Every technological leap brings unique challenges—often from surprising quarters. For instance, Johannes Gutenberg found his printed Bible in 1559’s Index of Forbidden Books, thanks to the Catholic Church fearing the mass production of unauthorized Biblical interpretations. 

This move unintentionally criminalized the very book the Church wanted to distribute.

AI doesn’t quite face such stern opposition, but it is experiencing challenges from unexpected quarters. Firms adopting AI have become aware of a friction between the functions that use GenAI. 

de Lucena explained. “We have the model risk management [MRM] function, which doesn’t sit in compliance necessarily, but is used to vet models across a firm,” he said. “Then we have compliance. What we’re seeing is the MRM functions are quite mature but haven’t adjusted to how they should vet LLMs.”

Has this led to hesitation within firms that are blocking AI adoption? Hesitation is probably the wrong word here. The approach financial institutions are currently adopting is one of careful consideration. After all, GenAI does come with risks.

 ”There are risks in generative AI, like hallucination, privacy, potential fairness and bias in the data it was trained on, especially if you’re not on the corpus it was trained on,” Aguilar said.

This begs the question, how can firms mitigate these risks? Labin recommended installing output validation methods such as:

Asking the LLM multiple questions – This helps you double-check the LLM’s assumptions and mitigate hallucination risk.
Using multiple agents – One agent’s process might differ from another’s. Using multiple agents helps you spot potential problems in an LLM’s  sequential reasoning.
Framing the same question in different ways – This helps you uncover context and discover biases in the LLM’s training. For instance, removing information or reframing it might result in a different output, indicating risk in the reasoning process.

The good news is most of these techniques are well-known in the data science world.

Huang suggested a less technical and more overarching mental model to approach the question of hallucinations. 

“Treat that LLM as a person,” he said. “Number one, just let the LLM know that the answer ‘I don’t know’ is acceptable. If you don’t know the answer, just say ‘I don’t know’ instead of making something up.”

Huang also expanded on the RAG—Retrieval Augmented Generation—approach. “It’s an open book test for the large language model,” he explained. “You provide a data repository for the large language models to reach into to find the answer to your question.”

By fixing the size of the repository like this, verifying the LLM’s output mirrors the approach one would take with a human researcher. The LLM would point to a place in the repository, just as a human researcher would, offering firms an easy way of verifying output accuracy.

Best practices when implementing generative AI

LLMs are currently great for retrieving information from a dataset and acting as Q&A assistants. However, the future is different. AI development will create products that collect information, risks, decisions, and summarize action steps in a single output. 

Labin said future AI products will initiate these steps proactively. Given this direction, what are some of the best practices financial firms must follow?

For starters, firms must think about using AI the right way. AI is not a forecasting tool that can predict the future. Instead, think of it as a tool that gets you to insights faster. Some other best practices the webinar panel recommended were:

Define use cases in detail before diving into implementation – This prevents mass implementation where AI might be a poor fit.
Avoid assuming GenAI is the default choice – Existing technology might handle the task faster.
Always keep ROI in mind – Scaling AI is expensive. Grounding implementation in ROI will help you prioritize the highest-impact use cases.
Install a governing body to guide AI adoption – Staff this body with functions from different parts of the business.

Aguilar offered an example of the final point. “We have a center of excellence that has principles for the ethical use of AI,” he said. “I certainly don’t want to be restricted to the knowledge of just surveillance. There are other folks who’ve been using this in other domains who can inform me of the way to use it.”

The appetite for AI is growing

GenAI implementation is rapidly increasing and is poised to change the way firms execute communication surveillance. Given its state as a relatively new and unexplored technology, current statements and encouragement from regulators is a huge positive. This guidance gives firms the license to explore new use cases.

However, there is complexity akin to building a vehicle while driving it. GenAI comes with massive potential but poses risks too. Implementing it per the best practices the panel highlighted is the best way of mitigating them.

Ultimately, GenAI is not a silver bullet for every issue. Existing technology can potentially do a quicker job with several use cases. Firms are best off considering the use cases where GenAI will deliver the most value. 