Appendices – Charter Statements

Appendix A – Methodology

The evaluation questions identified for the purpose of this evaluation were addressed through the following six methods.

Document review

The document review covered both documents internal to Justice, such as the Performance Information Profile for Advisory Services, and documents produced by HRLS to support the development of Charter Statements, along with other relevant sources, such as peer-reviewed publications and grey literature addressing Charter Statements. This task also included a review of court decisions that referred to Charter Statements. A thematic analysis was completed to summarize relevant findings.

Administrative data review

The administrative data review included data obtained from Justice’s Departmental Business Analytics System (Explore). Data was extracted from Explore’s Data Warehouse via Tableau, which included data from iCase, LEX, the Integrated Financial and Material System (IFMS), and the Human Resources Management System (HRMS). iCase and LEX data were extracted as of September 2023. Additional administrative data included the Charter Statement tracker in Excel used by HRLS, Justice Canada website analytics and other relevant data as available.

Content analysis

A content analysis was undertaken, and it included a review of parliamentary debates and testimonies in front of parliamentary committees studying proposed bills to identify references to Charter Statements. With the support of Justice’s Business Analytics Centre, a total of 669 mentions were gathered. These were analyzed and categorized based on predetermined criteria, including purely descriptive statements (e.g., “we are tabling the Charter Statement for Bill X”), statements about the process (e.g., inquiring about the expected date for the tabling of a Charter Statements, or asking for a new Charter Statement), and discussions about the content of a Charter Statement (e.g., supporting or challenging the conclusions of a Charter Statement).

Key informant interviews

A total of 33 semi-structured interviews were conducted. They included 8 interviews with representatives from HRLS (managers, legal counsel, paralegal), 13 interviews with legal counsel assigned to LSUs, 8 interviews with representatives from the Library of Parliament, and 5 interviews with other Justice groups (e.g., centres of expertise). A thematic analysis was performed on the findings from the interview, using the evaluation questions and indicators as the overall analytical framework. The analytical process was performed using the NVivo software.

Electronic survey

A survey of non-parliamentary stakeholders was conducted. The list of respondents included individuals who appeared in front of a parliamentary committee studying a new bill within the previous five FYs. This included a range of stakeholders such as representatives from non-profit organizations, corporate organizations, expert witnesses and unions. The survey was administered electronically in both English and French. It included mainly close-ended questions, with one open ended-question. A total of 111 individuals completed the survey questionnaires, for a response rate of 18%. Results from the survey were analyzed based on the categories of key informants. The main purpose of the survey was to obtain insights from the public on how Charter Statements are used when they engage in the parliamentary process.

Case studies

A total of three case studies were conducted, each addressing a specific legislative initiative. The goal was to select cases that had identified potential effects on Charter rights and freedoms, and where Charter Statements attracted higher levels of attention. The three initiatives selected were medical assistance in dying (bills C-14, C-7, and C-39), administrative segregation (Bill C-83), and changes to the Broadcasting Act, (bills C-10 and C-11). In each case, relevant findings from the interviews, the document review, the content analysis, and other publicly available information were used to provide a more detailed understanding of Charter Statements and the impact they had.

Limitations and mitigation strategies

Administrative data

The administrative data provided important information on the level of effort that Justice is dedicating to Charter Statements. However, it did not include the contribution of all Justice groups, nor did it provide a file-specific data (e.g., level of effort per Charter Statement). This limitation was mitigated through the evidence collected through other lines of evidence, particularly through interviews.

Insights from key stakeholders

Charter Statements are expected to enhance the awareness related to human rights and freedoms among key stakeholders involved in parliamentary proceedings related to legislative initiatives, including members of parliament, non-parliamentary stakeholders, and the public. The evaluation gathered perceptions and opinions of a number of these stakeholders, but not all. In particular, no direct consultations were held with members of parliament, and a portion of non-parliamentarian stakeholders were consulted. This limitation was largely mitigated by the document review and the content analysis.