Public Safety, Defence, and Immigration Portfolio Evaluation
Appendix A: Evaluation Matrix
| PSDI Evaluation Issue | Evaluation Questions | Performance Indicator | Data Sources/Methods |
|---|---|---|---|
| Relevance | |||
Continued Need for the PSDI Portfolio |
1. To what extent does PSDI address the needs of the Government of Canada and federal client departments and agencies? |
Extent, scope and nature of demand for PSDI legal services (litigation, legal advisory services, law management and practice), related to PSDI Portfolio cases and issues |
PSDI ICase reports/analysis |
Trends (past, current, forecasts) in public safety, national security, defence and immigration issues requiring |
Trends reports/analysis |
||
Alignment with Government Priorities |
2. To what extent are the activities of PSDI aligned with (1) The Government of Canada priorities and (2) The strategic priorities of the Department of Justice? |
Extent to which the work of PSDI is aligned to the needs of the Government of Canada |
PSDI staff/interviews |
Extent to which the activities of PSDI are aligned with (1) The Government of Canada priorities and (2) The strategic priorities of the Department of Justice |
Speech from the Throne, Federal Budget/analysis |
||
Alignment with Federal Roles and Responsibilities |
3. To what extent do the activities of PSDI align with federal roles and responsibilities? |
Extent to which the activities of PSDI align with: 1. Federal roles and responsibilities and 2. Government of Canada’s legal obligations and framework (national and international) |
Constitutional/Statutory Authorities/analysis |
| Portfolio Design | |||
PSDI Design |
4. Are the mandate and objectives of PSDI clear? |
Evidence, scope and clarity of PSDI mandate and objectives |
PSDI historic development/contextual analysis Business Plans/analysis |
|
5. Is the PSDI DLSU composition appropriate given the legal service needs of the Government of Canada? |
Appropriateness of PSDI DLSU composition in relation to Government of Canada legal service needs, and to the PSDI mandate and objectives |
PSDI historic development contextual analysis |
|
6. Are PSDI governance structures appropriate, clearly understood and being implemented as intended? |
Appropriateness of PSDI governance structures as planned |
PSDI administrative files/analysis |
|
7. Are PSDI processes (e.g. coordination and communication) structured in such a way that the Portfolio is likely to achieve its planned results? |
Appropriateness of PSDI processes as planned to support the achievement of results |
PSDI administrative files/analysis |
PSDI Performance Monitoring and Evaluation Capacity |
8. Is there sufficient capacity within PSDI to support ongoing quality improvement, performance measurement and evaluation? |
Sufficiency of capacity and resources, including methods/systems to support ongoing quality improvement, performance measurement and evaluation |
PSDI administrative files/analysis |
| Performance | |||
| Achievement of Expected Outcomes | |||
| Direct Outcomes | |||
|
9. To what extent has PSDI provided high-quality, cost-effective legal services? |
Nature and scope of PSDI legal services and activities undertaken in relation to Government of Canada needs/requirements |
PSDI administrative files/analysis |
|
|
Nature, number of strategies and tools that PSDI utilizes to assess, plan and manage costs in delivery of legal services |
PSDI administrative files/analysis |
|
|
Usefulness of tools |
PSDI administrative files/analysis |
|
|
Number and percentage of files being litigated by PSDI by nature and scope of complexity and legal risk |
PSDI legal files/analysis ICase reports/analysis Litigation files/analysis |
|
|
Level of effort dedicated to the provision of PSDI litigation services (by nature and scope of complexity and legal risk of files, type/nature of issues) |
PSDI administrative files/analysis |
|
|
Proportion of PSDI files where more than one client is involved |
PSDI administrative files/analysis |
|
|
Level of effort dedicated to the provision of PSDI legal advisory services (by files involving one client and more than one client, and type/nature of issues) |
PSDI administrative files/analysis |
|
|
Number/type/nature of PSDI’s law management and practice efforts (e.g. tools, resources) towards quality assurance to deliver legal services (planned/undertaken) |
PSDI administrative files/analysis |
|
|
Number/type/nature of professional development (e.g. practice groups) and training sessions, presentations delivered and results achieved |
PSDI administrative files/analysis Professional |
|
|
Extent to which PSDI clients are satisfied with the timeliness, responsiveness, usefulness and overall quality of PSDI legal services provided relative to costs of service provided. |
Justice Client Feedback survey/analysis |
|
10. To what extent has awareness/understanding within the federal government of legal issues and options for mitigating legal risk been enhanced? |
Strategies and tools that PSDI utilizes to enhance awareness and understanding of legal issues and options for mitigating risk |
PSDI administrative files/review |
|
|
Nature/level of awareness/understanding within client federal departments and agencies of legal issues and options for mitigating legal risk |
Case studies/analysis |
| Intermediate Outcomes | |||
|
11. To what extent is PSDI contributing to the achievement of the Government of Canada’s policy and operational objectives, in accordance with the rule of law? |
Evidence that PSDI approach is effective in achieving a coordinated, consistent and responsive approach that contributes to achievement of the Government of Canada’s policy objectives |
PSDI administrative files/analysis |
|
|
Evidence that PSDI approach is effective in achieving a coordinated, consistent and responsive approach that contributes to achievement of the Government of Canada’s operational objectives |
PSDI administrative files/analysis |
|
12. To what extent is PSDI contributing to government departments and agencies being better able to fulfill their mandates and manage their legal risks? |
Trends in Crown results for litigation files |
Performance Reports/Analysis |
|
|
Number and percentage of files where legal risk is reported as high, and nature of strategies that PSDI identifies and advises to manage/mitigate risk |
PSDI litigation files/analysis |
|
|
Nature/effectiveness of structures and processes to manage and mitigate risks at strategic and operational levels |
PSDI client interviews/analysis |
|
|
Extent to which PSDI and clients are effectively engaged in working together to manage/mitigate risk (informally and formally) |
PSDI client interviews/analysis |
|
|
Extent to which clients assess that PSDI is contributing to 1. their ability to fulfill their mandates and 2. the identification and assessment of legal risks related to client issues17 |
PSDI litigation files/analysis |
|
|
Extent to which government decision- makers perceive that PSDI contributes to the decision-making and legal risk management process |
PSDI client opinion/focus groups |
|
|
Extent/nature of PSDI legal advice that contributes to client policy and legislative development |
PSDI client opinion/focus groups |
| Ultimate Outcome | |||
|
13. To what extent is PSDI contributing to a federal government supported by high quality legal services? |
Cumulative |
Cumulative/contribution analysis |
| Efficiency and Economy | |||
Demonstration of Efficiency and Economy |
14. Has the PSDI resource utilization been appropriate and cost- effective? |
Budget and work plan priorities and activities in relation to PSDI outputs |
PSDI administrative files/analysis |
|
|
Appropriateness of management structures, processes and tools to support efficiency and effectiveness of service delivery (e.g. mix of FTEs, hours of work in relation to complexity and level of legal risk of file) |
PSDI governance structures and processes/analysis |
|
|
Extent to which roles and responsibilities to support cost effective-delivery are communicated and understood by Justice counsel |
PSDI governance structures and processes/analysis |
|
|
Type/nature of challenges (e.g. staffing, internal support, including information technologies) that influence ability to achieve outcomes cost-effectively |
PSDI administrative files/analysis |
|
|
Nature/extent of PSDI efforts, including outreach to clients and joint management strategies, to deliver legal services cost-effectively (past, current and planned efforts) |
PSDI administrative files/analysis |
|
|
PSDI human resource plans/requirements versus actual profile |
PSDI human resources plans/analysis |
|
15. Is the PSDI capacity to deliver legal services, and to provide effective national strategic coordination, appropriate and sufficient, given the current and forecasted demand? |
Level, scope and nature of PSDI and client activities in relation to demand for legal services, including national strategic coordination |
PSDI administrative files/analysis |
|
16. Are PSDI activities sufficiently integrated and coordinated to support the achievement of intended results? |
Levels, scope and nature of integration and cooperation within (1) PSDI (2) Justice (3) Client departments and agencies and externally (i.e. nationally, internationally) |
PSDI administrative files/analysis |
|
17. Are there alternative ways to deliver the same services? |
Trends in law practice/service delivery models (within Justice, private sector, international) in delivering legal services 1. Generally, and 2. On public safety, national security, defence, and immigration issues specifically? |
Environmental Scan/analysis PSDI staff and client focus groups/analysis |
- Date modified: