Special Advocates Program Evaluation

Appendix A: Evaluation Matrix

Questions Indicators Sources
Relevance (TB core issues 1–3)

Do the activities of the SAP align with the Department of Justice’s strategic objectives?

Assessment of whether the activities of the SAP align with the Department of Justice’s strategic objectives and core expected results

Key informant interviews

Document review

Do the activities of the SAP align with federal priorities?

Assessment of whether the activities of the SAP align with federal priorities

Key informant interviews

Document review

What need is the SAP intended to address? Is there evidence of an ongoing need?

Assessment of the continued need for the SAP

Key informant interviews

Legal trends and emerging issues (identified needs based on case law, media scan, consultations)

Key informant interviews

Document review

To what extent are the activities carried out through the SAP appropriate to the federal government and a core federal role?

Constitutional and statutory authority for federal involvement

Document review

Constitutional and statutory authority for Department of Justice involvement

Document review
Achievement of Expected Outcomes (TB core issue 4)
To what extent has the SAP achieved its expected outcomes?

 

Immediate Outcome (IRPA Division 9)

Fair representation of subjects through a Charter-compliant process

Number of contribution agreements with special advocates (by fiscal year)

Document review

Number of special advocates on roster (by fiscal year)

Document review

Adequacy/appropriateness of roster (e.g. language profiles, conflicts of interests, turnover rate)

Key informant interviews

Document review

Extent to which the roster is published in a manner that is considered appropriate to facilitate public access to it

Key informant interviews

Document review

Number of special advocates trained (by fiscal year)

Document review

Adequacy/appropriateness of training and support to special advocates

Key informant interviews

Document review

Number and type of complaints from special advocates pertaining to obstacles in the performance of their duties (by fiscal year)

Document review

Number of cases where special advocates appointed and number of hours billed (by fiscal year)

Document review

Assessment of whether IRPA Division 9 subjects’ interests are protected

Key informant interviews
Achievement of Expected Outcomes (TB core issue 4)

Intermediate Outcome (IRPA Division 9)

Foreign national and permanent residents inadmissible on serious grounds are denied status in Canada in a manner that respects international human rights and Charter obligations

The SAP’s contribution to a process that respects international human rights and Charter obligations

Key informant interviews

Ultimate Outcome (Justice)

A fair, relevant and accessible Canadian justice system

The SAP’s contribution to a fair, relevant and accessible Canadian justice system

Key informant interviews
Efficiency and Economy (TB core issue 5)

Could the work of the SAP be undertaken more efficiently and economically?

Evidence of strategies to achieve outputs and outcomes in the most cost-effective way

Key informant interviews

Document review

Measures in place to manage efficiency

Key informant interviews

Document review

Suggestions for improvement in efficiency of performance

Key informant interviews

Program administration ratio (total costs to administer the Program versus total contributions paid under the Program)

Document review

Alternative approaches to service delivery

Key informant interviews

Document review
Are there any best practices or lessons learned in the delivery of the SAP? Evidence of best practices and lessons learned

Key informant interviews

Document review