Evaluation of the Contraventions Act Program

3 Evaluation Methodology

To guide the evaluation, a methodology and evaluation matrix were developed, based on the guidance provided by the TB Policy on Results (2016). The scope of the evaluation focussed on relevance, effectiveness and efficiency. The evaluation included eight main questions, as summarized below.

Issue #1: Relevance

  • 1.1 How does the Contraventions Act Program align with current federal priorities? How does it align with the Action Plan for Official Languages?
  • 1.2 How is the current range of federal offences designated as contraventions meeting the needs of enforcement authorities?

Issue #2: Effectiveness

  • 2.1 To what extent are enforcement authorities using the contraventions regime to support the achievement of their mandate?
  • 2.2 What impact does the Contraventions Act Fund have on the ability of provincial/municipal authorities to deliver judicial services in both official languages?
  • 2.3 To what extent is the contraventions regime, including the Contraventions Act Fund, achieving its expected benefits for Canadians?
  • 2.4 What is the strategy of Justice to pursue the implementation of the contraventions regime in jurisdictions where it is not yet operational?

Issue #3: Efficiency

  • 3.1 How efficient is the current process for allocating the financial support provided to participating jurisdictions through the Contraventions Act Fund?
  • 3.2 What measures are in place to manage the Fund efficiently?

An Evaluation Working Group was established at the outset of the evaluation to support this process by providing inputs, advice and suggestions regarding the design and conduct of the evaluation. It included Program officials and representatives from the Evaluation Branch.

Three lines of evidence were used to address these evaluation questions, including a document and data review, key informant interviews, and one case study. Each of these methods is briefly described in the following sub-sections.

3.1 Document and Data Review

The document and data review informed all evaluation issues and questions. It provided updated information on the use of the contraventions regime in all participating provinces and municipalities, and on the range of measures that the Contraventions Act Fund has been supporting. This review was ongoing throughout the life of the project and included the following types of documents:

  • Contraventions Act Program files, including previous evaluations conducted of the Program
  • Reports from participating jurisdictions
  • Data from the Grants and Contribution Information Management System (GCIMS) on the financial support provided to participating provinces
  • Data from the Integrated Finance and Material System (IFMS) on all resources invested (e.g. FTEs) in the Program, to support the analysis of the efficiency of the Program

3.2 Key Informant Interviews

Key informant interviews constitute the primary source of data for this evaluation. Much of the experience of enforcement authorities, court representatives, and provincial and municipal governments with the contraventions regime is not systematically documented. As a result, interviews become pivotal to gather a meaningful understanding of the strengths and limitations of the Program as it currently stands.

A total of 53 in-depth interviews were conducted with 59 individuals representing the following groups:

  • Justice Canada representatives, including those involved in managing and implementing the Program, as well as legal counsel from legal services units serving federal departments and agencies that are using the contraventions regime.
  • Enforcement authorities (municipal, provincial and federal, as applicable), including both enforcement officers and their managers.
  • Representatives from federal departments and agencies that manage programs or legislative initiatives that rely on the contraventions regime.
  • Prosecutors, including those working with the Public Prosecution Service of Canada, and provincial prosecutors (as applicable).
  • Managers from municipal and provincial courts that receive financial support from the Contraventions Act Fund.
  • Representatives from provincial governments and municipal authorities that manage the Contraventions Act agreements signed with Justice Canada.

3.3 Case Study

Initially, an in-person site visit in Newfoundland and Labrador was scheduled. The primary purpose of this site visit was to adequately assess and document the experience of this province where the contraventions regime became operational during the 2018-19 fiscal year. Examining the experience of the most recent province to have joined the contraventions regime was expected to support the efforts of Justice to expand the regime in all provinces, as recommended in previous evaluations of the Program.

The COVID-19 pandemic did not allow for this in-person visit to occur. As a result, more detailed interviews were conducted via phone and videoconference, in an attempt to mitigate the impact of conducting the site visit remotely. The list of individuals interviewed included provincial representatives, court manager and personnel, enforcement officers, and prosecutors.

3.4 Consideration of Gender-based Analysis Plus (GBA+)

The evaluation of the Contraventions Act Program provided an opportunity to explore whether a GBA+ could strengthen our understanding of the extent to which the Program’s reach and benefits include diverse groups of women, men and non-binary individuals. Given the nature of the Program, the assessment indicated that a GBA+ would not add significant perspectives and insights, since the Program relates to the implementation of a ticketing system to better support a fair and efficient enforcement of federal offences.

3.5 Constraints, Limitations and Mitigation

The constraints and limitations related to this evaluation include the challenges of reaching stakeholders during the COVID-19 pandemic, limited insights that can be gathered from jurisdictions not participating in the Program, and the limited data on direct interactions with Canadians who are involved in the contraventions process.

These limitations did not significantly affect the capacity of the evaluation team to adequately address each evaluation question. While the pandemic made it somewhat more challenging to reach stakeholders, the targeted number of interviews was met. Some information was also gathered on the two provinces where the regime is not yet operational, particularly through interviews conducted with enforcement authorities operating in all regions of the country.

Date modified: