Publication Date: 2023-05-17 | Regulator: Bank of England (BOE)
SS1/23 outlines the principles and expectations for managing model risk within the banking sector effectively, setting out a framework which ensures transparency, accountability, and sound governance in the development, implementation, and validation of models.
The Regulation emphasizes the need for banks to have robust processes and controls in place and highlights the importance of considering expert judgment in making model adjustments to address limitations and ensuring that such adjustments are adequately justified and recorded in the model inventory.
The document also emphasizes the significance of comprehensive and up-to-date documentation on the design, theory, and logic underlying the development of models. It stresses the need for clear documentation on data sources, methodology, performance testing, and model limitations. The documentation should be detailed enough to allow an independent third party to understand the model’s operation and replicate its results.
Furthermore, the regulation mandates that supporting systems that are thoroughly tested, subject to rigorous quality control and change control processes, and periodically reassessed for suitability.
Independent model validation is needed to provide ongoing, effective challenge to model development and use. The validation function should assess the suitability and soundness of models, the design and integration of supporting systems, and the accuracy and relevance of model results.
What are the core principles of model risk management outlined in this document?
The document outlines five core principles of model risk management (MRM) that banks should adhere to. These principles provide a framework for managing model risk effectively across all model and risk types. Here are the core principles of model risk management outlined in the document:
Principle 1: Model Identification and Model Risk Classification
- Firms should have a clear definition of a model that sets the scope for MRM.
- A model inventory and risk-based tiering approach should be established to categorize models and manage model risk.
- The tiering approach should consider factors such as materiality, complexity, and the purpose of the model.
Principle 2: Governance and Oversight
- Firms should have a comprehensive governance and oversight framework for MRM.
- Senior management and boards should be actively engaged and participate in MRM processes.
- Responsibility for the MRM framework should be allocated to the relevant Senior Management Function (SMF).
Principle 3: Model Development, Implementation, and Use
- Firms should have a robust model development process with standards for design, implementation, and performance measurement.
- Regular testing of data, model construct, assumptions, and outcomes should be conducted to identify and remediate model limitations and weaknesses.
Principle 4: Independent Model Validation
- Firms should have a validation process that provides ongoing, independent, and effective challenge to model development and use.
- Validation recommendations for remediation or redevelopment should be acted upon to ensure models are suitable for their intended purpose.
Principle 5: Model Risk Mitigants
- Firms should have established policies and procedures for using model risk mitigants when models are underperforming.
- Procedures for the independent review of post-model adjustments should be in place.
What are the key requirements mentioned in the document?
- Documentation of Model Owners, Users, and Developers: Model owners, users, and developers should be documented for all models. Model owners are responsible for monitoring the model’s performance, ensuring appropriate validation, and maintaining accurate model information. Model users are responsible for using the model consistent with its intended purpose and considering known limitations. Model developers are responsible for conducting research, development, evaluation, and testing of the model according to firm standards.
- Internal Audit Assessment: Internal Audit (IA) should periodically assess the effectiveness of the Model Risk Management (MRM) framework and compliance with internal policies. IA should assess the effectiveness of each component of the model lifecycle and the overall effectiveness of the MRM framework.
- Suitability and Soundness of Data: The model development process should demonstrate that the data used to develop the model are suitable for the intended use. The data should be consistent with the chosen theory and methodology and representative of the underlying portfolios, products, assets, or customer base. Any adjustments made to the data or use of proxies should be documented and subject to validation.
- Model Development Testing: Model development testing should demonstrate that a model works as intended. Clear criteria should be established as a basis to measure a model’s quality and select between candidate models. The monitoring pack, consisting of tests or criteria, should be used to monitor a model’s ongoing performance during use.
- Roles and Responsibilities: Roles and responsibilities should be clearly documented for each stage of the model lifecycle. The requisite skills, experience, and expertise required for each role should be specified. Responsibility for model performance monitoring and reassessment should be clearly defined.
- Model Documentation: Models should have comprehensive documentation that includes information on data sources, methodology, performance testing, and model limitations. The documentation should be sufficient to validate the firm’s use of the model.
- Supporting Systems: Models should be implemented in information systems or environments that have been thoroughly tested for the intended model purposes. The systems should be subject to rigorous quality control and change control processes. Regular reassessment of system suitability is recommended.
- Independent Model Validation: Firms should have a validation function that provides an objective, unbiased, and critical opinion on the suitability and soundness of models. Validation recommendations for remediation or redevelopment should be acted upon to ensure models are suitable for their intended purpose.
What specific governance structures are required for model risk management (MRM)?
The document emphasizes the importance of strong governance structures for effective model risk management (MRM). These governance structures ensure that MRM receives appropriate attention and oversight from senior management and the board. They promote a culture of effective model risk management and provide the necessary accountability and responsibility for implementing and maintaining a robust MRM framework:
- Board of Directors: The board of directors plays a crucial role in MRM. They should provide leadership and oversight by setting clear model risk appetite and approving the MRM policy. The board should ensure that the MRM framework is effectively designed and proportionate to the firm’s size, complexity of models, and extent of model usage. They should promote an understanding of model risk and its management as a distinct risk discipline.
- Accountable Individual: The board should appoint an accountable individual who assumes responsibility for implementing a sound MRM framework. This individual should have the necessary authority, expertise, and resources to establish and maintain effective MRM practices. They should ensure that the MRM framework aligns with the board’s defined model risk appetite and that appropriate controls are in place.
- Senior Management Function (SMF): The responsibility for the MRM framework should be allocated to the relevant SMF(s) within the firm’s organizational structure. The SMF(s) should have the appropriate skills, experience, and expertise to ensure the MRM framework operates effectively. The SMF(s) should update their Statement of Responsibilities to reflect their role in MRM.
- Model Risk Function: Large firms often establish a designated Model Risk Function (MRM function) within their risk management or compliance departments. The MRM function is responsible for creating and maintaining the MRM framework and risk controls. They may have distinct responsibilities from the model validation function. The stature and authority of the MRM function are important for effective MRM practices.
- Model Risk Committees: Firms may establish model risk committees to oversee and provide guidance on model risk management. These committees can include representatives from relevant business units, risk management, compliance, and other control functions. The committees should have the authority to restrict the use of models, recommend conditional approval, or grant exceptions to model validation or approval.
What should be included in the model documentation?
Model documentation should be comprehensive, up-to-date, and sufficiently detailed to enable an independent third party with relevant expertise to understand the model’s operation, identify key assumptions and limitations, and replicate any parameter estimation and model results. The documentation should be subject to proper governance, including ongoing independent validation. It should contain:
- Data Description: A comprehensive description of the data sources used in the model development process. This should include information on whether any data proxies were used and the rationale behind their selection. Additionally, the documentation should provide the results of data quality, accuracy, and relevance tests conducted to ensure the suitability of the data for the intended use of the model.
- Modelling Techniques and Assumptions: A detailed explanation of the modelling techniques adopted in the development of the model. This should include information on the mathematical specifications, numerical and statistical techniques employed, and any assumptions or approximations made during the modelling process. It is important to clearly outline the theory and logic underlying the model’s design.
- Performance Monitoring: Details of the tests or criteria that will be used to monitor the model’s ongoing performance during use. This includes the key indicators or metrics that will be tracked to assess the model’s performance against predetermined thresholds. The rationale for the selection of these tests or criteria should also be provided.
- Model Limitations: A thorough discussion of the nature and extent of model limitations. This involves identifying and documenting any inherent limitations or constraints that may impact the accuracy or reliability of the model’s outputs. It is crucial to highlight the potential impact of these limitations on the model’s performance and communicate them to model users and owners.
How should a firm define and document its model risk appetite?
Defining and documenting the model risk appetite is an important step in model risk management (MRM) to ensure clarity and consistency across the firm. Key considerations include:
- Scope and Coverage: The model risk appetite should clearly define the scope and coverage of models to which it applies. It should specify the types of models included, such as quantitative models, statistical models, economic models, or any other relevant categories. The scope should also consider the level of model complexity and materiality.
- Risk Tolerance and Limits: The model risk appetite should articulate the firm’s risk tolerance and limits concerning model risk. It should establish thresholds for acceptable model performance and tolerance for errors. These thresholds can be based on various factors, such as accuracy, reliability, stability, and predictive power of the models. The risk tolerance should be aligned with the firm’s overall risk appetite and take into account the potential impact of model risk on the firm’s objectives.
- Compliance and Control: The model risk appetite should address compliance with internal policies and applicable regulatory requirements. It should emphasize the importance of adherence to the MRM framework, including policies, procedures, and control mechanisms. The risk appetite should also highlight the need for effective controls and testing of model outputs to support robust MRM practices.
- Model Selection and Approval: The model risk appetite should include measures for identifying models and approving their use for decision-making purposes. It should outline the criteria and processes for model selection, including considerations such as data quality, model performance, documentation standards, and the suitability of the model for its intended purpose. The risk appetite should ensure that models undergo appropriate validation and approval processes before their implementation.
- Expert Judgment and Mitigants: The risk appetite should address the use of expert judgment in model development and validation. It should emphasize the importance of expert input and the proper documentation of expert judgment. Additionally, the risk appetite should highlight the need for effective mitigants to manage model risk, such as alternative models, stress testing, scenario analysis, or other risk management techniques.
- Reporting and Monitoring: The risk appetite should establish reporting requirements to monitor the firm’s model risk profile. It should specify the frequency and content of reports on model risk, including qualitative and quantitative measures. The risk appetite should ensure that the board of directors receives regular reports on the firm’s model risk profile against the established risk appetite.
- Review and Updates: The risk appetite should be reviewed periodically to ensure its continued relevance. It should be updated as needed to reflect changes in the firm’s risk profile, business environment, regulatory landscape, and advancements in technology. The risk appetite should be a living document that evolves with the firm’s evolving model risk landscape.
How should firms address and remediate identified model risks and limitations?
Firms should address and remediate identified model risks and limitations by performing:
- Independent Review: All identified model risks and limitations should be subject to an independent review. The intensity of the review should be commensurate with the materiality of the risks and limitations. The review should assess the continued relevance of the risks and limitations to the underlying portfolio, the soundness of underlying assumptions, the integrity of data used, and the plausibility of model outputs.
- Root Cause Analysis: Firms should conduct a root cause analysis to gain a clear understanding of the underlying model limitations. This analysis helps determine whether the limitations are due to significant model deficiencies that require remediation. It is important to identify the factors contributing to the limitations and assess their impact on the model’s performance.
- Documentation and Justification: Remediation efforts should be supported by appropriate documentation. Firms should provide a clear justification for applying post-model adjustments (PMAs) to compensate for model limitations. The criteria for calculating PMAs and determining when they should be reduced or removed should be documented. The documentation should also include the rationale for using any model adjustments and how they should be calculated over time.
- Trend Analysis: Firms should analyse the materiality of PMAs and the trend of recurring PMAs for the same model limitations. This analysis helps identify whether there are flaws in the model design or misspecifications in the model construct. If recurring PMAs indicate significant limitations, firms should consider remedial actions such as model recalibration or redevelopment to address the underlying issues and reduce reliance on PMAs.
- Governance and Control Framework: Firms should establish a governance and control framework for reviewing and supporting the use of PMAs. This framework should include processes for reviewing and approving PMAs, implementing decisions related to their calculation, assessing their completeness, and determining when they should be reduced or removed. The impact of applying PMAs should be clearly communicated when reporting model results for decision-making purposes.
What are the key standards and practices for model development and testing?
By adhering to the following standards and practices, firms can enhance the reliability and effectiveness of their models, mitigate model risk, and ensure that models are suitable for their intended purpose:
- Model Purpose and Design: Models should have a clear statement of purpose and design objectives that guide the model development process. The design of the model should be suitable for its intended use, with conceptually sound variables and parameters that support the design objectives. The choice of modelling technique should be conceptually sound and supported by published research or industry practices. The model’s limitations and sensitivities to changes in inputs should be communicated to stakeholders.
- Data Quality and Suitability: The model development process should ensure that the data used are suitable for the intended use and consistent with the chosen theory and methodology. The data should be representative of the underlying portfolios, products, assets, or customer base the model is intended to be used for. Any adjustments made to the data or use of proxies should be clearly documented and subject to validation. Data privacy and other relevant data regulations should be complied with.
- Model Development Testing: Model development testing should be conducted to demonstrate that the model works as intended. It should include clear criteria and tests to measure the model’s quality and select between candidate models. Backward-looking performance tests should be conducted using actual observations across various economic and market conditions. Forward-looking performance tests should assess the model’s ability to consider changes in conditions without deteriorating performance. Performance tests should also include comparisons with challenger models or alternative theories and assumptions.
- Model Adjustments and Expert Judgment: Risks relating to model limitations and uncertainties should be understood, monitored, and managed. Model adjustments and expert judgment should be appropriately documented and subject to validation. The assumptions made, factors used for adjustments, and rationale should be independently validated, monitored, reported, and recorded. The use of expert judgment should be transparent and supported by appropriate documentation.
- Independent Model Validation: Firms should have a validation process that provides ongoing, independent, and effective challenge to model development and use. The validation process should be commensurate with the materiality and risk profile of the model. Validation recommendations for remediation or redevelopment should be actioned to ensure models are suitable for their intended purpose. The validation process should be independent from the model development process.
- Model Performance Monitoring: Ongoing model performance monitoring should be conducted to assess the model’s performance against thresholds for acceptable performance. It should ensure that parameter estimates and model constructs are appropriate and valid, assumptions are applicable, and changes in products, exposures, activities, clients, or market conditions are appropriately addressed. Model performance monitoring should be performed regularly to identify and address any deterioration in model performance.
What are the reporting requirements for MRM to the PRA and internal stakeholders?
It is important for firms to ensure that reporting to both the PRA and internal stakeholders is timely, accurate, and comprehensive. The reports should provide a clear understanding of the firm’s model risk profile, control environment, and any actions taken to address identified risks or weaknesses. Reporting requirements for Model Risk Management (MRM) vary for different stakeholders, but include:
Reporting to the PRA:
- Model Risk Inventory: Firms are expected to maintain a comprehensive model inventory that identifies the sources of model risk. This information should be available for reporting to the PRA.
- Model Risk Reporting: Firms should provide management information on model risk to the PRA. This may include reports on the overall model risk profile, model inter-dependencies, and any material control exceptions or inappropriate model use.
- IA Findings: Findings from Internal Audit (IA) assessments related to MRM should be documented and reported to the board and relevant committees on a timely basis. These findings should also be made available for reporting to the PRA.
Reporting to Internal Stakeholders:
- Board Reporting: Reports on model risk should be provided to the board of directors on a regular basis. These reports should include an assessment of the firm’s model risk profile, any identified weaknesses or limitations, and progress on remediation plans.
- Management Reporting: Regular reports on model risk should be provided to senior management and relevant committees within the firm. These reports should cover areas such as model performance, validation results, control exceptions, and any emerging risks or issues.
- Audit Committee Reporting: Reports on the effectiveness of MRM for financial reporting should be made available to the audit committee. These reports should provide insights into the firm’s system of internal controls and control activities related to model risk.
Self-Assessment and Remediation Reporting:
- Self-Assessment Findings: Firms are expected to conduct self-assessments of their MRM frameworks and document the findings. These findings should be shared with the board and relevant stakeholders, highlighting any identified shortcomings and areas for improvement.
- Remediation Plans: If any deficiencies or weaknesses are identified through self-assessment or other reviews, firms should develop remediation plans. These plans should outline the necessary actions, timelines, and responsible parties. Regular updates on the progress of remediation plans should be provided to the board and relevant stakeholders.
How should firms conduct their initial and annual self-assessments for MRM?
Firms should conduct both initial and annual self-assessments for Model Risk Management (MRM) to evaluate the effectiveness of their MRM frameworks and identify any areas for improvement. The key steps are as follows:
Initial Self-Assessment:
- Familiarize with the MRM Principles: Review the MRM principles set out by the Prudential Regulation Authority (PRA) and ensure a clear understanding of the expectations and requirements.
- Evaluate Current MRM Framework: Assess the existing MRM framework against the MRM principles. Identify any gaps or areas where the framework may not meet the expectations.
- Identify Shortcomings: Document any shortcomings or deficiencies in the current MRM framework. This may include gaps in policies, procedures, controls, or governance structures.
- Prepare Remediation Plans: Develop remediation plans to address the identified shortcomings. These plans should outline the necessary actions, responsible parties, and timelines for implementation.
- Document Findings: Document the findings of the self-assessment, including both strengths and weaknesses. Share the findings with the board and relevant stakeholders.
Annual Self-Assessment:
- Review MRM Framework: Conduct a comprehensive review of the MRM framework to assess its effectiveness and alignment with the MRM principles.
- Evaluate Implementation: Evaluate the implementation of the MRM framework, including the effectiveness of policies, procedures, controls, and governance structures.
- Assess Compliance: Ensure compliance with regulatory requirements and internal policies related to MRM.
- Identify Areas for Improvement: Identify any areas for improvement or emerging risks in the MRM framework. This may include changes in business activities, models, or regulatory expectations.
- Update Remediation Plans: Review and update the remediation plans developed during the initial self-assessment. Ensure progress is made on addressing the identified shortcomings.
- Document Findings: Document the findings of the annual self-assessment, including any progress made on remediation plans. Share the findings with the board and relevant stakeholders.
Throughout both the initial and annual self-assessments, it is important for firms to maintain transparency, accuracy, and documentation of the assessment process and findings. Regular communication with the board and relevant stakeholders is crucial to ensure awareness and support for the MRM framework and any necessary remediation efforts.
Suggest a control framework for a firm to comply with this regulation?
It is important for firms to tailor the control framework to their specific organizational structure, risk profile, and business activities. Regular monitoring, testing, and review of controls should be conducted to ensure their effectiveness and ongoing compliance with regulatory requirements.
- Policy and Procedure Controls:
- Develop and maintain clearly documented policies and procedures that formalize the MRM framework.
- Ensure policies and procedures are approved by the board and reviewed regularly to align with regulatory requirements and changing business environments.
- Cross-reference and align MRM policies with broader risk management policies.
- Implement controls to ensure compliance with internal policies and applicable regulatory requirements.
- Model Identification and Inventory Controls:
- Establish a formal definition of a model and maintain a model inventory.
- Implement controls to ensure accurate recording and maintenance of model tiering in the model inventory.
- Regularly validate and review the firm-wide model tiering approach to ensure its relevance and accuracy.
- Model Development Controls:
- Implement standards for model development, including model testing procedures, documentation standards, and model performance assessment criteria.
- Establish controls for data quality management, including rules and standards for data quality, accuracy, and relevance.
- Implement controls to ensure adherence to model selection criteria and proper documentation of model assumptions and limitations.
- Model Validation Controls:
- Establish clear roles and responsibilities for the validation function.
- Implement validation procedures, including determining the scope, frequency, and prioritization of model validation.
- Implement controls for effective challenge and monitoring of the validation process.
- Establish reporting mechanisms for validation results and any required remedial actions.
- Model Performance Monitoring Controls:
- Implement standards for measuring and monitoring model performance.
- Set thresholds for acceptable model performance and establish criteria for model recalibration or redevelopment.
- Conduct root cause analyses to identify model limitations and causes of performance deterioration.
- Implement controls for performing and using back-testing.
- Governance and Oversight Controls:
- Ensure strong governance oversight with a board that promotes an MRM culture and sets clear model risk appetite.
- Appoint an accountable individual to implement and oversee the MRM framework.
- Regularly assess and report MRM practices to the board of directors.
- Establish controls for effective engagement and participation of senior management and boards in MRM practices.
- Internal Audit Controls:
- Maintain an Internal Audit function that assesses the effectiveness of the MRM framework.
- Conduct periodic audits of the MRM framework and compliance with internal policies.
- Ensure IA reports findings and recommendations to relevant stakeholders.
Write me a policy for model risk management based on the requirements in this document?
Model Risk Management Policy
- Introduction
1.1 This Model Risk Management Policy (“the Policy”) outlines the framework and expectations for managing model risk within our organization. It is based on the principles and requirements set forth in the Prudential Regulation Authority’s (PRA) supervisory statement on Model Risk Management Principles for Banks.
1.2 The purpose of this Policy is to ensure that our organization has robust processes, controls, and documentation in place to effectively identify, assess, mitigate, and monitor model risk throughout the model lifecycle.
2. Scope
2.1 This Policy applies to all regulated United Kingdom (UK)-incorporated banks, building societies, and PRA-designated investment firms within our organization that have internal model approval to calculate regulatory capital requirements.
2.2 The Policy covers all models used within our organization, including those developed in-house, externally sourced models, and models used for financial reporting purposes.
3. Model Documentation
3.1 Model documentation is a critical component of effective model risk management. All models used within our organization must have comprehensive and up-to-date documentation that includes the following:
3.1.1 Data Description: A description of the data sources used, including any data proxies, and the results of data quality, accuracy, and relevance tests.
3.1.2 Modelling Techniques and Assumptions: An explanation of the modelling techniques adopted, including mathematical specifications, numerical and statistical techniques, and any assumptions or approximations made.
3.1.3 Performance Monitoring: Details of the tests or criteria used to monitor the model’s ongoing performance during use, including key indicators or metrics and predetermined thresholds.
3.1.4 Model Limitations: A discussion of the nature and extent of model limitations, including inherent constraints that may impact the accuracy or reliability of the model’s outputs.
4. Policies and Procedures
4.1 Our organization shall establish comprehensive policies and procedures that formalize the Model Risk Management (MRM) framework and ensure its effective implementation. These policies and procedures shall include, but not be limited to:
4.1.1 Model Definitions and Tiering: Clear definitions of a model and model risk, and a tiering approach to determine the scope, intensity, and frequency of model validation.
4.1.2 Model Development Standards: Standards for model development, including model testing procedures, selection criteria, documentation standards, and performance assessment criteria.
4.1.3 Data Quality Management: Procedures for managing data quality, including rules and standards for data accuracy, relevance, and the management of alternative or unstructured data sources.
4.1.4 Model Validation Standards: Standards for model validation, including clear roles and responsibilities, validation procedures, prioritization, scope, frequency of re-validation, and reporting of validation results.
4.1.5 Model Performance Monitoring: Standards for measuring and monitoring model performance, including criteria for acceptable performance, thresholds for recalibration or redevelopment, and processes for root cause analysis and back-testing.
5. Roles and Responsibilities
5.1 Clear roles and responsibilities shall be allocated to staff with appropriate skills and experience to ensure the effective operation of the MRM framework. This includes:
5.1.1 Documenting roles and responsibilities for each stage of the model lifecycle, specifying the requisite skills, experience, and expertise required for each role.
5.1.2 Defining responsibility for model performance monitoring and reassessment, which may be undertaken by model owners, users, or developers.
5.1.3 Ensuring that the relevant Senior Management Function(s) (SMF) have overall responsibility for MRM, including establishing policies and procedures, ensuring effective challenge and independent validation, and taking prompt remedial action when necessary.
6. Self-Assessment and Remediation
6.1 Prior to the implementation of this Policy, an initial self-assessment of our implemented MRM framework shall be conducted. Any identified shortcomings shall be addressed through remediation plans.
6.2 Self-assessments shall be updated at least annually thereafter, and remediation plans shall be reviewed and updated on a regular basis. Findings and progress on remediation shall be documented and shared with the board of directors.
7. Compliance and Reporting
7.1 Compliance with this Policy, internal policies, and applicable regulatory requirements shall be ensured. Regular reporting on model risk management activities, including self-assessment findings and remediation progress, shall be provided to the relevant SMF(s) and the board of directors.
8. Review and Updates
8.1 This Policy shall be reviewed on a regular basis to ensure its continued relevance and effectiveness. Updates shall be made to align with changes in the regulatory landscape, advancements in technology, and the evolving risk environment.
8.2 Any updates to this Policy shall be communicated to all relevant stakeholders and incorporated into the MRM framework.
Note: This policy is based on the requirements outlined in the provided document and should be tailored to the specific needs and circumstances of our organization. It is subject to review and approval by the appropriate governance bodies within our organization.
Empower your compliance team with FinregE’s state-of-the-art software and watch as regulatory challenges transform into opportunities for growth and innovation. Contact us today to see how FinregE can revolutionize your approach to regulatory compliance.
