Evaluation Framework - Evaluation of Legal Risk Management in the Department of Justice
| Issues/questions |
Indicators |
Data sources |
| Rationale of the LRM Initiative |
| 1. How has the context within which LRM is operating changed since 2003? What are the implications of any changes? |
- Trends in number and complexity of cases involving the federal government
- Changes in administration/management structure of LRM (devolution)
- Opinion on effects of any changes
- Opinion on continued need for LRM
|
- Document review/iCase
- Key informant interviews
|
| 2. Is LRM consistent with the principles of Integrated Risk Management and the achievement of DOJ's strategic objectives? |
- Alignment of LRM with Integrated Risk Management and the DOJ's strategic objectives
|
|
| Implementation of the LRM Initiative |
| 3. Does the LRM Accountability Framework and Governance Structure (AFGS) provide an effective framework for systematically managing legal risk? |
- Opinion on continued relevance of LRM AFGS
- Existence of shared understanding of roles and responsibilities for LRM
- Existence of shared understanding of LRM
|
- Document review
- Key informant interviews
- Legal counsel survey
- Case studies
|
| 4. What structures and processes have been put in place in the DOJ to support the identification, assessment, mitigation, and management of legal risk? |
- Monitoring activities to identify risks
- Standards to assess risks
- Risk mitigation and management strategies
- Communication and consultation strategies within DOJ and with client departments/other affected departments/central agencies
- Appropriateness/adequacy of LRM structures and processes to support effective LRM practices
- Adequacy of communications strategy to support LRM
- Role of Law Practice Management Directorate
|
- Document review
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
|
5. Is LRM integrated into the work (policy/advisory/
legislative/litigation) of the DOJ? |
- Level of integration of LRM into work of the DOJ
- Level of integration of LRM in risk practices of government departments and agencies
|
- Document review/iCase
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
|
| 6. How consistent are LRM practices? |
- Consistency of assessing, managing and reporting risk (particularly files with risk rating of 6)
- Variations by type of legal activity (advisory, legislative drafting, litigation)
- Variations by LSU, region, portfolio
- Consistency/comparability of tools
- Consistency of use of tools including iCase
|
- Document review/iCase
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
|
| 7. What are the advantages and disadvantages of having different LRM practices across the DOJ? |
- Opinion on advantages/disadvantages of different LRM practices across the DOJ
|
- Key informant interviews
- Legal counsel survey
- Case studies
- Focus groups
|
| 8. Are the necessary tools in place to support LRM? |
- Types of tools available
- Adequacy of tools to support LRM
- Adequacy of legal risk reporting
- iCase data quality integrity
|
- Document review/iCase
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
- Focus groups
|
| 9. To what extent are counsel aware of these tools and use them? |
- Awareness of availability of tools to support LRM
- Use of current tools by counsel
- Satisfaction with tools
|
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
|
| 10. Are there adequate resources (human, financial, information, training, technological) to support the management of legal risk in the DOJ? |
- Opinion on adequacy of resources to support LRM decision-making
- Costs associated with different LRM models
- Capacity to measure and report on results
|
- Key informant interviews
- Legal counsel survey
- Case studies
- Focus groups
|
| 11. Are risk assessments used in resource allocation and planning in advisory/legislation/litigation files? |
- Opinion on whether risk assessments are used to allocate resources and plan
- Evidence in files that risk assessments are used to allocate resources and plan
|
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
|
| 12. Are there elements of LRM that have not been implemented? Are there any gaps that should be addressed? |
- Components of Accountability Framework and Governance Structure not implemented
- Gaps in processes and practices for the mitigation/management of legal risk
|
- Document review
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
|
| Results of the LRM Initiative |
| 13. Are current DOJ practices making an impact in terms of mitigating/managing legal risks across the federal government? |
- Government stakeholders are routinely informed about legal risks
- DOJ understands clients' main legal risks
- Level of understanding of legal risk management in government departments and agencies
- Consideration of legal risks in the development and implementation of government policies, programs, and legislation
- Helpfulness of legal advice on legal risk
- Effective LRM partnerships with client departments
- Level of success in avoidance/mitigation of legal risks due to LRM (timely responses, effective strategies)
- Capacity to measure and report on results
|
- Document review
- Key informant interviews
- Legal counsel survey
- Case studies
- Justice Services Satisfaction Survey
- File review
- Focus groups
|
| 14. Does the DOJ approach to LRM reflect effective risk management practices? What are the commonalities/ divergences? |
- Lessons learned and best practices in risk management from other jurisdictions
- Opinion on whether LRM reflects best practices in risk management
|
- Document/literature review
- Focus groups
|
| 15. What are the key lessons learned and best practices with LRM? Are they being effectively communicated and shared within the DOJ? |
- Lessons learned and best practices
- Process to share best practices
|
- Document review
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
- Focus groups
|
| 16. How could LRM be improved? |
- Identified areas for improvement
|
- Document review/iCase
- Key informant interviews
- Legal counsel survey
- Case studies
- File review
- Focus groups
|
