The use of machine learning (ML) models by financial institutions has grown steadily in recent years given their enhanced capabilities and widespread potential application. However, machine learning model risk management remains a challenge given the difficulty of building, adopting, and regulating these models.
Increasing Use of ML Models
Banks collect huge amounts of data to build models for decision making, risk prediction, capital calculations and more. Machine learning algorithms are being used more and more in this modelling process as they enhance analytical and forecasting capability, allow users to optimize feature selection, and facilitate automation. Moreover, these models can be used widely across functions such as credit underwriting, fraud detection, compliance, and risk management.
Evolving Regulation
Regulators are actively exploring the usage of ML algorithms in regulatory models. Last year, a group of U.S. regulatory bodies requested information and comments on how financial institutions are using artificial intelligence (AI) and machine learning. The European Banking Authority also recently published their own discussion paper on using machine learning for internal ratings-based models to calculate regulatory capital for credit risk.
While banks are currently using existing guidelines such as SR 11-7 for the risk management of machine learning models, new regulation specific to ML is likely to develop in the coming years. Banks will need to be prepared to adjust to meet these new requirements as they emerge.
Challenges with Model Risk Management
Under SR 11-7 guidelines for model risk management, model methodologies, mathematical specifications, and numerical techniques used must be explained in detail. This can be difficult with machine learning models due to the intricate development methodologies, difficult model traceability, and complications in explaining the interaction between variables.
Evaluating the conceptual soundness of ML models can also be complicated as developers and validators need to have a deep understanding of ML algorithms and the underlying assumptions. Additionally, ML models are constantly updating, which could create instability in model performances. Modelers and validators must stay informed of any changes in the algorithms by putting in place periodic checkpoints as the altered model result might be different than expected.
Furthermore, machine learning models typically ingest Big Data, or large sets of data from various sources, creating a challenge around data traceability and data quality.
Conclusion
While machine learning models have become pivotal for financial industries because of their potential scope and enhanced predictive capabilities, building them can be tedious and the end-to-end process can be quite complicated.
Although such models are still new to the industry, given their potential and the huge possibility that they will replace traditional regulatory models, it is a good time for financial institutions to start planning for machine learning pipelines and model risk management. Evalueserve is prepared to assist companies with the end-to-end process of these models and to ensure the models comply with conceptual soundness requirements.
