Efficient Deployment of Compliance Models in Financial Services

A leading financial services institution (henceforth referred to as simply The Firm) places strong emphasis on securing and frequently monitoring the sensitive data, confidential records, and frequent trade transactions of its more than eight million clients. The Firm is also committed to strong AI Governance. In addition to adhering to data governance standards and promoting AI explainability, The Firm conducts daily compliance checks on all data assets to prevent market manipulation, assess risk, and prevent clients from potential financial exploitation.

With millions of daily financial transactions, The Firm’s compliance teams rely on the deployment of numerous machine learning (ML) models in production to identify and predict indicators of potential account-level risk. These models support more than 125 field supervision agents with their hourly, daily, or monthly compliance checks.

Roadblocks to Efficiently Deploying Compliance Models

Prior to implementing Dataiku in 2022, The Firm leveraged several desktop-based, open-source solutions. In addition to a general lack of governance plus difficulty with centralized deployment and performance monitoring, the team faced a slew of challenges that made the maintenance of these models cumbersome and inefficient:

• Exorbitant time-to-production (eight months, on average): The time to optimize the model code and deploy complex, open-source models leveraging data from various source systems required nearly 1,440 standard deployment hours from tech support engineers.
• Low return on investment (ROI): Given the amount of time it took to deploy ML models, generating continuous value from compliance ML models was an uphill battle.
• Lack of rigor: Compliance models residing within desktop open-source solutions meant MLOps teams could not fully embrace agile and DevOps methodologies for version control and for building end-to-end continuous integration and continuous deployment (CI/CD) automated ML pipelines.
• Lack of collaboration: Models built on desktop solutions made it difficult to encourage cross-functional project collaboration between data scientists and ML engineers.

In addition, there were certain types of models The Firm simply could not implement working only with models residing within desktop solutions. For example, conditional pipelines designed to perform specific actions based on the threshold levels and percent change results detected within the model output.

Agile Model Deployment & Elevated AI Governance With Dataiku

In the spring of 2022, The Firm developed an MLOps team to collaborate with data scientists and work with various cross-functional departments to support the end-to-end deployment processes of operationalizing ML models. 

Dataiku has played a tremendous role in scaling our MLOps practice. Leveraging Dataiku allowed us to stand up a new MLOps team within a few short months and hire staff ready to deploy ML models after just a few days of researching technical documentation and Dataiku knowledge tutorials.

The MLOps team chose Dataiku to help scale their processes and improve their day-to-day ability to execute. ML engineers were able to quickly migrate compliance models from prior desktop solutions into Dataiku to ensure proper versioning and project governance.

More holistically, the MLOps team uses Dataiku to provide the business community (more than 125+ stakeholders) with an opportunity to deliver mission-critical workloads at scale. These models not only help detect potential risk, but they also have informed decision making and reduced the amount of manual effort required by field supervision agents.

At a granular level, the use of Dataiku for MLOps and AI Governance by The Firm has shown:

• An increase in The Firms’ overall efficiency rate for deploying compliance models into production by more than 900%.
• A 90% reduction in time-to-deployment compared to prior desktop solutions. 
• ML engineers write 75% less pipeline production code thanks to Dataiku visual recipes and graphical user interface (GUI), which contributes to the overall reduction in time-to-deployment. 
• An 86% reduction in the amount of time spent optimizing and refactoring model code for production by leveraging Dataiku visual ML capabilities, again contributing to the overall reduction in time-to-deployment.
• The ability to manage a portfolio of several dozen machine learning models in production  while encouraging and enforcing peer reviews and stakeholder sign off, thanks to implementing the Dataiku Govern node.

In addition, the team was also able to develop conditional pipelines to support compliance — a use case that was impossible with their previous setup. Using Python recipes and the Dataiku API, ML engineers can customize conditional pipelines and control the flow of data between managed folders. This allows them to satisfy the specifications for comparing daily batch scoring results against prior batch scoring results to adhere to compliance standards.

Dataiku is a quintessential analytics solution that encourages cross-functional team collaboration between analysts, scientists, engineers, and other technical professionals. The solution places a strong emphasis on the scaling of analytics or data science workloads quickly and efficiently, from inception throughout production.