Language selection

Français

Search

Phone Scam Warning

Fraudsters are pretending to be from the Department of Justice

Central Agencies Portfolio Evaluation

Appendix B: Evaluation Matrix

Evaluation Issue	Question	Performance Indicator	Measures	Data Sources/Methods
1. Relevance
Continued need for the CAP	1.1 To what extent do CAP services address the legal needs of its clients? Is there a continued need for the CAP?	Extent, scope and nature of demand for: Legal advisory services Legislative drafting services Litigation services CAP advisory, drafting and litigation services meet client needs.	Number of actively managed files over the evaluation period^{Footnote 47} Trends in the types of files actively managed (advisory, litigation, legislative drafting) Number of files closed Perceived relevance of CAP services to the needs of clients	Document review and analysis CAP clients/survey Interviews CAP staff^{Footnote 48}interviews, survey iCase/analysis CAP client files/analysis
Continued need for the CAP	1.2 Has the demand for CAP services changed and if so, how?	Nature of and extent to which demand for CAP services has changed over the past five years Nature and extent to which demand for CAP services is expected to change over the next five years	Trends in client requests for legal services by type of service (advisory, litigation, legislative drafting) Changes in types of requests, including new or special requests Changes in the complexity of requests for services Trends in risk levels Expected trends/emergent issues	CAP clients/survey, interviews Justice officials/interviews CAP clients/survey, interviews CAP staff/interviews, survey CAP administrative records/analysis CAP client files/analysis
Alignment with Government Priorities and Departmental Strategic Outcomes	1.3 To what extent are the activities of the CAP aligned with Government of Canada priorities? and The second strategic outcome of Justice?	Extent to which the CAP mandate and activities are aligned with Government of Canada priorities	Consistency between CAP mandate and activities with the priorities of the federal government Contribution to drafting legislation and regulation required to support Government of Canada priorities Perceived relevance of CAP to the needs/ priorities of the Government of Canada	Documentation/analysis Speech from the Throne, federal budget/analysis CAP staff/interviews Clients/interviews
		Extent to which CAP activities are aligned with the second strategic outcome of Justice	Consistency between CAP activities and the Justice strategic outcome	Justice strategic priorities/analysis CAP Business Plan/analysis Documentation/analysis CAP staff/interviews Justice officials/interviews
Alignment with Federal Roles and Responsibilities	1.4 To what extent do the activities of the CAP align with federal roles and responsibilities?	Extent to which the CAP mandate and activities are aligned with the federal government’s role and responsibilities	Alignment of CAP services with federal government’s roles and responsibilities Alignment of CAP services with clients’ mandates	Documentation/analysis Constitutional & Statutory Authorities/analysis PAA/analysis
2. Portfolio Design
CAP Design	2.1 Are the mandate and objectives of the CAP clear?	Scope and clarity of CAP mandate and objectives	Clarity of stated mandate and objectives of CAP Awareness of the stated mandate and objectives of CAP among client agencies	Constitutional & Statutory Authorities/analysis Documentation/analysis PAA/analysis CAP staff/interviews CAP clients/interviews
	2.2 Is the CAP governance structure appropriate?	Clarity of CAP roles and responsibilities	Clarity of the roles and responsibilities of CAP counsel Adherence to the roles and responsibilities of the CAP by counsel Awareness of CAP counsel role/ responsibilities among client departments/agencies	Documentation/analysis CAP staff/interviews CAP legal counsel/survey CAP clients/interviews Justice officials/interviews
	2.3 Do the program/services duplicate or overlap with programs/services delivered by other legal service areas of Justice?	Areas of overlap with other legal services provided by Justice	Evidence of duplicative services	Document review and analysis CAP legal staff and management/interviews Justice legal staff/interviews
	2.4 Is the CAP structured in such a way that it is likely to achieve its objectives?	Organizational structure and delivery strategies support achievement of CAP objectives	Organizational factors and activities that contribute to/detract from achieving CAP objectives	CAP governance structure/analysis CAP documentation/analysis CAP staff and client interviews CAP legal counsel survey
CAP Performance Monitoring	2.5 Is there sufficient capacity within CAP to support ongoing quality improvement, and performance measurement?	Sufficiency of capacity and resources, including methods/systems to support ongoing quality improvement, performance measurement		CAP administrative files/analysis CAP staff/interviews
3. Performance
Achievement of Expected Outcomes
3.1. Direct Outcomes
Effectiveness	3.1.1 To what extent has the CAP enhanced the awareness and understanding of legal risk management of clients?	Change in level of awareness/ understanding by clients of the legal risks relevant to their agency or business line	Client decision-makers are aware of legal risks Counsel identify or assess legal risks Number of files where legal risks have been identified and assessed Client awareness of potential legal risk	CAP clients/interviews, survey CAP counsel/interviews, survey Case files/analysis iCase data/analysis
	3.1.2 To what extent has the CAP produced high quality legal advice and support in response to client requests?	Extent to which legal services provided by CAP responds to client needs	Legal advice addresses client needs Legal options are provided to the client Usefulness of the legal advice and services provided	Document review and analysis iCase data/analysis CAP clients/interviews, survey CAP counsel/interviews
		Extent to which CAP staff has effective tools and resources necessary to support the provision of legal services	Processes, tools and resources available to support the delivery of legal services that are consistent across Justice Evidence of quality assurance mechanisms in place in a regular use Evidence of consultations to ensure consistency of law and the Government of Canada position CAP counsel levels of satisfaction with tools and resources to support delivery of legal services	Documentation/review CAP staff/interviews, survey Other Justice counsel/interviews
		Responsiveness of the CAP to meeting client requests	CAP meets or exceeds established departmental services standards for timeliness, usefulness, responsiveness	Clients/interviews, survey CAP staff/interviews, survey iCase data/analysis CAP client files/analysis Case studies
	3.1.3 To what extent has the CAP contributed to the creation of regulations and statutory instruments that support government related legislative needs?	Regulations and statutory instruments drafted by CAP contribute to the making of legislation, regulations and statutory instruments that support government legislative needs	Usage of drafting protocols/ procedures, tool and resources available Legislative drafting services and products meet federal government needs (e.g., enacting federal budget)	Document review and analysis CAP counsel/interviews, survey iCase data/analysis CAP client files/analysis CAP client/interviews Case studies
	3.1.4 To what extent does CAP provide timely and responsive litigation services	Responsiveness of the CAP to clients’/Government of Canada litigation needs	Evidence of consultations to ensure consistency of law and the Government of Canada position in litigation cases Evidence of options considered Number of files where DR was used Appropriate dispute resolution (DR) options are identified to address client needs Client is kept informed of the status of the litigation	Clients/interviews, survey CAP counsel/interviews, survey DoJ counsel/interviews iCase data/analysis CAP client files/analysis Case studies
3.2 Intermediate Outcomes
Effectiveness (continued)	3.2.1 To what extent do the legal services provided by CAP counsel contribute to program and policy development, and to decision making of client departments and agencies?	Extent to which legal advice provided by CAP counsel is considered by decision-makers	Legal advice provided responds to needs of the client Usefulness of the advice to support clients’ decision making Client satisfaction with the legal advice	CAP case files/analysis Clients/interviews, survey CAP staff/interviews, survey Case studies
	3.2.2 To what extent has the CAP contributed to client departments and agencies being better able to manage their legal risks?	Nature and extent of CAP contribution to legal risk management	Number/% of files where legal risk is assessed/reassessed Number/% of files where legal risk mitigation/management options are provided to clients Number/% of high-risk or high impact files where contingency plans have been developed Clients understand/consider mitigation options	CAP client files/analysis Clients/interviews, survey Case studies
	3.2.3 To what extent has the CAP contributed to the creation of federal legislation that respects the Constitution and other legal requirements?	Extent/nature of measures available and used to ensure that legislative drafts respect the Constitution and other legal requirements	Nature of quality assurance mechanisms in place in regular use Evidence of collaboration with LSB drafting experts Among CAP counsel, level of satisfaction with tools to support legal services	CAP client files/analysis CAP staff/interviews Other Justice counsel Interviews Case studies
	3.2.4 To what extent is there appropriate resolution of litigation cases?	Extent to which litigation cases are effectively resolved/litigated	Evidence that alternatives to litigation were explored Extent that DR is used in litigation cases where feasible Number/% of potential litigation cases resolved through different options (e.g., withdrawn by party, court, DR, etc.) Level of client satisfaction with CAP management of litigation files	iCase CAP client files/analysis CAP counsel interviews CAP clients/interviews Case studies
	3.2.5 In what ways do CAP services protect the interests of the Crown?	Legal issues/risks are addressed so that the Crown’s interest is protected	Protocols, processes and resources available to assist in developing strategies to protect Crown’s interests Legal strategies developed with a whole-of-government perspective Extent to which the Government of Canada’s position is consistently communicated and represented by CAP counsel	CAP client files/analysis CAP counsel/interviews CAP clients/interviews Case studies
3.3 Ultimate Outcome
Effectiveness (continued)	3.3.1 To what extent has the CAP contributed to a federal government that is supported by high-quality legal services?	Cumulative	Achievement of direct and intermediate outcomes	Cumulative/contribution analysis CAP staff/interviews Justice officials/interviews
4. Demonstration of Efficiency and Economy
Efficiency and Economy	4.1 Has the CAP’s resource utilization been appropriate in relation to the resources allocated, activities and outputs produced and demand for services?	Budget and work plan priorities and activities in relation to outputs delivered	Cost of legal inputs for each type of service (e.g., advisory, litigation) Change in level of effort for each type of services Analysis of planned versus actual inputs/resources used	CAP financial data/analysis iCase data/analysis CAP staff/interviews, survey
	4.2 Is the CAP’s capacity to deliver services sufficient given current and future demand for services?	Level and scope of services available in relation to demand for services (past, current, projected) by area of service delivery and LSU Appropriateness of Labour Law Continuing Education to support efficient and economical service to clients in the area of labour law.	Volume of requests per type of service and LSU Capacity to complete requests (within reasonable timelines) Capacity to cover all court/tribunal proceedings Capacity to assign counsel to all court/tribunal proceedings Impact of the Law Practice Model on human resource needs Evidence of initiatives to improve efficiency Evidence that the level of CAP counsel assigned to a file is appropriate given its level of risk and complexity	iCase/analysis CAP staff/survey, interviews Clients/interviews, survey Case studies
	4.3 Does the financial model utilized by the CAP provide appropriate and sufficient funding to meet current and future demand for services?	Sufficiency of A-base funding to discharge the Minister’s responsibilities Level of satisfaction that CAP and its clients have with the CAP funding model, and the specific agreements/MOUs Trends in financial models for supporting/funding legal services	Gap between financial inputs and resource requirements for outputs Examples from other jurisdictions that provide similar types of legal services within government	CAP staff/interviews Financial documentation/ review Clients/interviews Literature /environmental scan Financial documentation/ review CAP managers/interviews FIN staff/interviews
	4.4 Are there alternative ways of delivering the same services to CAP clients?	Trends in practices/service delivery models	Examples from other jurisdictions/private sector Alternative models to deliver similar types of services in a government context.	Literature/environmental scan CAP managers/interviews Justice officials/interviews

Date modified:: 2022-05-13