Justice Partnership and Innovation Program Evaluation, Final Report

• Previous Page
• Table of Contents
• Next Page

2. FINDINGS

2.1 Continued Need for the Program

Based on the program objectives, the evaluation findings are organized around the following four overriding JPIP aims:

• the promotion of greater knowledge among the public respecting justice issues,
• the promotion of greater public access to the justice system,
• the promotion of dialogue and understanding among justice stakeholders respecting justice issues, and,
• the identification of new justice issues.

The evaluation found that there is a continuing need to promote public understanding of, and access to, the justice system. Indeed, these needs appear to be growing. The PLEI organization survey identified the Canadian public’s need to understand the law in general, so that they can participate knowledgeably in a democratic society and be aware of the laws that govern Canadians. The survey found that the general public lacks knowledge and thus misunderstands the functioning of the justice system. The increasing number of self-represented litigants, aging individuals, immigrants, minority groups, Aboriginal persons, persons with disabilities, and persons with aging parents need to better understand the Canadian justice system.

As part of the applicant survey, respondents were asked to rate the extent to which stated JPIP objectives represented existing needs. Some 80% of survey respondents rated “promoting innovations in the justice system to ensure greater access to the justice system” as a great need (as opposed to not needed, a slight need, or a moderate need). At least 60% of survey respondents also rated “informing Canadians about access to justice issues and the justice system in order to contribute to increased public understanding, participation, confidence and trust in the justice system”, “building knowledge, awareness, understanding and informed dialogue among justice stakeholders on justice issues”, and “building knowledge, awareness, understanding and informed dialogue among the public on justice issues” as great needs.

When asked to describe what they considered to be emerging trends, issues and gaps, the most common responses among survey respondents related to access to justice, including difficulties experienced by certain populations, the general volume in the system, and a corresponding trend toward self-representation.

These findings speak to the continuing need to promote awareness and understanding among Canadians about access to justice issues and the justice system. By extension, these findings also support the continued relevance of the objective, “to assist the ten designated public legal education and information organizations in promoting greater access to justice through various means” because the primary purpose of PLEI organizations is to promote public understanding of, and access to, the Canadian justice system.

The JPIP core objectives relating to the promotion of greater understanding and dialogue among justice stakeholders respecting justice issues were supported by key informants, who noted that initiatives that promote networking, partnerships, knowledge creation and sharing among members of the justice community strengthened the justice community. Findings from two of the case studies supported these views. Interviewees related to the CACP and ULCC case studies cited numerous examples of conferences and other networking events during which new and innovative justice concepts were introduced, discussed and moved forward. Survey respondents and the majority of the key informants consulted in the PLEI organization survey also agreed that engaging legal professionals in the justice system was important.

The core objective respecting the identification of new justice issues was, generally speaking, not noted among survey respondents or case study interviewees as important. This is not surprising given that this objective is not one that is typically achieved by any single project, nor would it commonly be listed by project proponents as an objective of their project. However, some key informants cited this objective as among the most important of the JPIP objectives. The identification of new justice issues is achieved by the Program as a whole, through its very nature. By seeking project proposals from a broad and relatively unrestricted range of organizations on an equally wide range of ideas, JPIP is expected to foster the identification of new issues. Key informants describe this as an extremely important function of the Program, and a strong continuing need.

Evidence indicates that the provision of funding in support of organizations listed in the Main Estimates (i.e., Named Grants), and the ICCLR also meets a need in Canada. Case studies focused on two recipients of Named Grants (CACP and ULCC) and the ICCLR. In the case of Named Grants, recipients’ JPIP-funded projects can, and do (as in the case of both of CACP and ULCC, as noted above), support the achievement of core JPIP objectives. Funding of the ICCLR supports the promotion of effective international cooperation, protecting victims of crime (in particular, children), helping to develop more effective and fair justice systems in Canada and abroad, and helping Canadians understand the importance of international cooperation and mutual assistance in the fight against serious crimes. Interviewees associated with this case study, as well as key informants, reported that the JPIP-funded work of the ICCLR continued to meet an important need.

The evaluation also confirmed the need for the provision of funding to support LSAP. Research indicates that there is an underrepresentation of Aboriginals in the legal profession. In 2005, approximately 1000 Aboriginal persons were practicing lawyers in Canada, representing a little less than one third of their proportional representation.^[19] Further, the overrepresentation of Aboriginal persons in the corrections system^[20] suggests an even greater need to increase the number of Aboriginal persons in the legal profession.

Findings respecting the continued relevance of the assessed contributions are limited to selected key informant interviews. The evidence suggests a continued need for both contributions; the work is seen by these sources as valuable and important in helping support the achievement of the goals of the funded organizations and in continuing to meet Canada's international obligations.

2.2 Alignment with Government Priorities, and Alignment with Federal Roles and Responsibilities

According to its 2010-11 Report on Plans and Priorities, the Department of Justice’s mission is “to ensure that Canada is a just and law-abiding society with an accessible, efficient and fair system of justice”. The related strategic outcome is to provide “a fair, relevant and accessible justice system that reflects Canadian values”.

An analysis of the JPIP objectives reveals a high level of correspondence with these aims. Accessibility is a central objective of JPIP, with funding provided to support PLEI organizations and other projects to increase public understanding and accessibility to the justice system.

Greater efficiency is one of the long-term goals of, in particular, dialogue among justice stakeholders. As described in detail in Subsection 2.3, one of the fundamental outcomes of many of the conferences and other events put on by such JPIP-funded organizations as CACP and ULCC is the development of resolutions aimed at improving the coordination of different levels of players in the justice system and across provinces and territories. Duplications and overlaps are eliminated or reduced, thus increasing the efficiency of the system as a whole.

Greater fairness is achieved, again, through the JPIP objective of enhanced public access to the justice system, and in particular through JPIP’s emphasis on projects focused on various marginalized or otherwise underrepresented populations. To the extent that JPIP-funded projects and activities increase the access to and effective utilization of the system by these populations, the general fairness of the Canadian justice system is improved.

JPIP’s emphasis on innovation, or the identification and addressing of new and emerging justice issues, also promotes fairness. Populations and/or justice issues previously not addressed or underrepresented in the system are brought to light such that they may take a more effective place in the system. The emphasis on innovation also relates to the departmental aim of relevance. To the extent that JPIP-funded projects and activities help identify emerging issues, the system is aided in maintaining its relevance to matters of current concern.

Objectives related to the funding of the ICCLR and LSAP were found to have a lesser degree of direct alignment with departmental priorities. However, there is alignment with government priorities. In the case of the ICCLR, the federal government has interests in various international justice initiatives related to protecting persons from crime and promoting respect for human rights. The mandate of the ICCLR aligns with these interests. LSAP’s mandate aligns with the statement made in the 2011 Speech from the Throne: “Concerted action is needed to address the barriers to social and economic participation that many Aboriginal Canadians face.”

The JPIP program has demonstrated a significant ability to adapt to changing needs or issues as indicated by the large number of different projects in different organizations funded over the years. Another example of the Program’s adaptation is seen in the fact that in 2007, there were nine organizations that were eligible to receive a Named Grant. However, in 2010-11, only five organizations remained eligible, reflecting changing priorities over the years. As needs and issues have changed, some initiatives have ceased to be funded while others, such as the Missing and Murdered Aboriginal Women Initiative, have been introduced.

The PLEI organization survey found that the Department is responsive to government priorities through the development of policies, laws and programs in areas such as Aboriginal justice, criminal justice, youth justice, family justice, and international public and private law. Recent projects undertaken by the PLEI organizations involve activities in these areas.

Key informants thought that JPIP met policy and program priorities of the Department of Justice and the federal government. For example, policy-makers have used annual reports from JPIP-funded organizations in developing policies, e.g., on the topic of family violence and honour killings. In some cases the policy dialogue is reciprocal, with Department of Justice staff working closely with organizations to help shape their policy development work. The work undertaken with ULCC is an example of this. A key informant stated that JPIP was an excellent mechanism – due in part to the flexible nature of its terms and conditions – to allow the Department to quickly respond to the missing and murdered Aboriginal women issue, as well as issues faced by other marginalized populations. In addition to domestic issues, JPIP also addresses changing federal government priority areas in international fora, including a number of Hague Conventions (e.g., Treaty on International Parental Child Adoption).

2.3 Achievement of Expected Outcomes

2.3.1 Greater Access and Knowledge among the Public Respecting Legal Information and Justice Issues

Evidence from multiple sources supports the finding that greater knowledge among the public respecting justice issues has been achieved, primarily, although not exclusively, through the activities of PLEI organizations.

Surveyed successful JPIP applicants describe benefits stemming from their projects. Concerning public legal information, respondents mentioned an extensive variety of benefits related to increasing public knowledge including enhanced family law guides, a Virtual Family Law Centre to support self-represented family law litigants, public legal education documents on the Internet, booklets, teachers’ manuals (used in schools programs), news articles, presentations, speakers’ bureau, and websites.

PLEI organization key informants also reported that their JPIP-funded activities had built knowledge and informed dialogue among the public on justice issues, and gave many examples of such knowledge-building activities including conferences and other training sessions, brochures, and phone-in service. A PLEI organization representative stated that there are 3,000 pages on the PLEI organization’s website along with hundreds of pamphlets. In addition, this organization produces videos, conducts training in plain language and workshops for schools, and promotes services through event booths/kiosks. Another PLEI organization has produced two training courses, offers a speakers’ bureau, runs conferences, and has produced 53 publications.

The review of project files, particularly those of PLEI organizations, provided substantial additional evidence of outputs and outcomes related to increased public knowledge, including:

Publications

The annual figure for dissemination of PLEI organization publications (including brochures, booklets, audio materials) ranged from 5,060 to 1,979,237 (based on a review of seven PLEI organization files). The number of newsletters that were disseminated annually ranged from 10 to 2,258 (based on four files).

Learning

Numerous conferences, workshops and legal information sessions were organized by PLEI organizations according to reviewed files. These were attended by subject matter experts, practitioners, students, teachers, immigrants and the general public.

Website Access

The number of annual website hits reported by PLEI organizations ranged from 44,044 to 1,300,000 (based on five PLEI organization reports).

Telephone and In-person Response/Referral Services

The number of telephone calls that were received annually ranged from 434 to 7551 (based on three PLEI organization reports). The number of referrals made annually ranged from 5 to 2,706 (based on five reports).

PLEI organization public knowledge-related outputs and outcomes may be considered incremental benefits. That is, many of these benefits would not have occurred in the absence of JPIP funding. This observation applies to the ten provincial PLEI organizations to varying extents, generally depending upon the size of the organization. Smaller PLEI organizations rely more heavily on JPIP funding than do larger organizations. Several interviewees stated that without JPIP funding, no projects or activities would have taken place in their organization. One key informant stated that his/her organization would not have existed without JPIP. Another informant stated that core funding was absolutely necessary. Other PLEI organization interviewees stated that JPIP funding enabled their organizations to conduct activities with greater scope (than without JPIP funding). As explained by one of the PLEI partners, “Core funding is essential to the long-term effectiveness and sustainability of an organization. Although project funding is a useful tool, it is the core funding which allows an organization to create a stable pool of expertise within the organization so that it is able to bid on contracts.” Others indicated that without Justice core funding from JPIP, some PLEI organizations would no longer be able to continue operations, noting that for some PLEI organizations, JPIP provides 100% of their funding.

These findings were echoed in the PLEI organization survey findings which suggested that the absence of core funding provided by JPIP would have a significant negative impact on PLEI organizations. The report also indicates that, in some cases, funds from other partners could not be secured without JPIP funding. This would be the case for all core-funded organizations that receive funding from the Law Foundation or the Law Society. The leveraging impact of JPIP funding is thus critical to the sustainability of several PLEI organizations.

It is generally understood that knowledge can lead to behaviour. With respect to the justice system, it follows that, to one extent or another, increased public understanding leads to increased access to the justice system. It stands to reason that a better informed populace will be able to more effectively engage with the justice system. This is supported by the views of key informants.

2.3.2 Dialogue and Understanding among Justice Stakeholders Respecting Justice Issues

Evidence from all sources strongly supports the finding that dialogue and understanding among justice stakeholders respecting justice issues are promoted by JPIP-funded projects and activities. Dialogue occurred in a variety of ways among a range of players. Four broad categories of dialogue were conferences and related events for members of the justice community, events that targeted or included the public, consultations or discussions between members of the justice community (including JPIP project proponents) and Department of Justice policy staff, and partnerships between JPIP project proponents such as PLEI organizations and justice community players. In all cases, positive results were reported in the form of increased understanding and innovation respecting issues and trends of current concern.

The review of files revealed many examples of justice and legal-related knowledge building and sharing activities such as conferences, symposiums, meetings, etc. This information largely came from the reports of the proponents. In some cases, the results of beneficiary satisfaction surveys or other beneficiary research (e.g., questions regarding the extent to which beneficiaries’ knowledge was enhanced), were included in the files. Results reported from these exercises were generally positive. Conference and symposium topics included dispute resolution in the courts, mental health and the justice system, human rights and intellectual disabilities, children’s rights, criminal justice system issues, and court technology. Other dialogue/outreach activities include consultations, information sessions and kiosks, presentations, and working groups. Funded organizations also conducted training activities, such as an Aboriginal training session focusing on providing Aboriginal Courtworkers with information on various issues, and family violence prevention training for front-line workers.

An average of 262 participants attended each of the reported conferences/symposiums in the reviewed files. Among participant views on satisfaction with dialogue events or training activities, an overall increase in understanding and knowledge was reported in most cases. One organization conducted a participant satisfaction survey for a 2010-11 project showing that 90% of participants were “very satisfied” or “somewhat satisfied” with the speakers and resource people, and 81% were “very satisfied” or “somewhat satisfied” with the quality of the content provided.

The survey of conference participants surveyed attendees – mostly government officials and legal professionals – after the conferences to determine the level and increase in knowledge and understanding of justice-related issues as a result of the three conferences. Results indicate that many respondents already had a moderate to high level of knowledge of the conference subject matter(s) prior to attending the conference. However, respondents indicated a significant increase in their knowledge and understanding of justice-related issue(s), as 83% of participants overall had a high to very high increase in their knowledge and understanding after the conference. Half of the respondents (51%) reported that their knowledge had increased moderately and 35% indicated that their knowledge and understanding had increased a great deal. Results also show that the conferences constituted good opportunities for participants to generate best practices, and to network and connect with other people at the conference. The events were also perceived as useful for the work they do.

The PLEI organization survey identified various ways in which PLEI organizations enter into partnerships, from a simple agreement for distribution of publications to a formal contractual relationship. Key informants interviewed as part of the PLEI organization survey reported positive outcomes resulting from their partnerships, including expanding their reach in the community, ensuring their clients have access to referrals and quality information, increasing their level of expertise in plain language, and being part of larger projects targeted at larger audiences through the pooling of human and financial resources.

PLEI organizations’ role is not primarily to contribute to policy development, but their activities have led to greater awareness of policy issues within the legal community. Key informants representing PLEI organizations felt that participants in their events had increased their knowledge of, and confidence with, justice issues to a great extent. One stated that the credibility of the PLEI organization led to other justice stakeholders referring people to them with “full confidence”, that the people being referred find useful and reliable justice information. Project respondents thought that participants’ level of satisfaction with partnerships/networking was very high. One project respondent had been approached by a departmental policy-maker and believed that his/her PLEI organization’s project activities had been used to inform policy development in the Department.

Two of the case studies examined organizations heavily involved in conferences, i.e. the CACP and the ULCC, both Named Grants. The ULCC, in fact, focuses primarily on an annual conference bringing together federal, provincial and territorial government representatives as well as representatives from the legal sector and academics to discuss and recommend changes to federal criminal legislation based on identified deficiencies, gaps or duplications. The CACP is involved in a range of activities. It has for the past four years organized an annual conference on criminal legislative policy developments. Both of these conferences are similar in that they bring together key players from the justice system who would otherwise not have an opportunity to interact. The benefits are reportedly great in terms of idea exchange and generation. Interviewees for the ULCC case study indicated that JPIP funding has enabled ULCC to identify important justice/legal trends in Canada, such as the need for uniform laws in the areas of powers of attorney, advanced health care directives, reverse mortgages (particularly for seniors), and assisted human reproduction. The element of separation between ULCC and the federal government was seen to enable discussions that do not bind the provincial and federal governments, and thus open up dialogue and consultation among justice stakeholders. The uniform laws that are developed by ULCC were seen to have provided the federal (and provincial/territorial) governments with high-quality products that have enabled the Canadian justice system to address emerging issues, innovate and evolve. ULCC’s annual conference was seen to build knowledge and understanding among judicial and academic participants, and allowed them to provide their time and expertise to meet, network and liaise on issues of common interest. Relationships and dialogue continued throughout the year through working groups. JPIP funding also allowed for a number of ways for disseminating information about the work of ULCC, e.g., a website database, a commercial law newsletter, updates of the commercial binder, and publication of annual conference proceedings.

The CACP conferences explored ways of making more effective use of information technologies, e.g., web-based disclosure and voice-activated transcripts, many of which, when properly implemented, could decrease costs, for example, by eliminating unnecessary court appearances. The conferences also addressed new approaches to supporting active judicial case management and more effective judicial pre-trial procedures, and they encouraged the use of administrative solutions versus legislative solutions. Many similar initiatives for reform across Canada were seen to have their birthplace at CACP conferences, as participants often have the desire to go further with the ideas and best practices that they have learned. In both cases, according to CACP officials, the conferences would not have been put on in the absence of JPIP funding.

Survey respondents also mentioned a number of benefits related to dialogue among justice community members including student law chapters at law schools, the establishment of a wellness cohort for First Nations communities, and exposure to leading-edge researchers and thinkers with divergent views, resulting in beneficiaries being more knowledgeable after conferences.

All key informants representing projects thought that their JPIP-funded activities had built knowledge, awareness, understanding, confidence and trust, and informed dialogue among justice stakeholders and the public on justice issues at least to some extent, and gave many examples of these activities such as conferences and other training sessions. The large majority of project respondents thought that participants in sponsored events had increased their understanding of, and confidence with, justice issues to a great extent. All project respondents thought that participants had increased their understanding and confidence to at least some extent. The large majority of project respondents thought that participants’ level of satisfaction with partnerships/networking was very high.

Department of Justice key informants said that JPIP-funded activities were communicated to the Department and used in policy development. One respondent pointed to conferences as a medium for such communication, while another noted a flow of information and policy development on emerging justice issues in general. All project key informants had been approached by a Department of Justice policy area, and believed that their JPIP-funded activities had been used to inform policy development in the Department.

2.3.3 Identification of New Justice Issues

The evaluation evidence supports the finding that the identification of new justice issues has been achieved by JPIP in two ways. First, and most importantly, by its very nature, including its stated objectives and the criteria used to select and fund projects, JPIP constitutes a gathering place of new ideas. The terms and conditions of JPIP allow the Program to fund a wide range of projects. This can, and does, include projects related to subjects that may not have been previously focused upon by the Department of Justice. As noted earlier, the Program's flexibility allowed the Department to respond to such important new issues as the missing and murdered Aboriginal issue and issues faced by other marginalized populations, such as persons with mental health problems or disabilities. Department of Justice key informants agreed strongly that JPIP-funded activities promoted the identification and/or improved understanding of emergent issues in the justice system in this way. One respondent noted that “JPIP promotes innovation not on a project-by-project basis, but as a program that offers a pool of money that will attract projects related to emerging issues as they arise in the field.”

If the preceding describes the collective impact of the Program, the second way it promotes the identification of new justice issues is through the innovative ideas of individual projects. PLEI organizations surveyed identified such innovative activities as bringing together groups who have never, or have rarely, worked together on PLEI projects, sharing information that had never been shared before with particular populations, and using new technologies for improved communication and cooperation between organizations. The file review revealed additional innovations including a project combining English as a Second Language and PLEI in British Columbia, a project to develop an online tool for agencies to share their PLEI materials and strategies in Ontario, the Legal Interpreter’s Project in Newfoundland bringing together lawyers, court officials and interpreters to develop a training curriculum for Aboriginal legal interpreters in both criminal and family court law, and a partnership between a PLEI organization and a consumer magazine to add legal issues to the topics covered by the magazine. One PLEI key informant stated that JPIP-funded activities had promoted innovations in the justice system to a “great extent”, for example through promoting alternative resolution or participative justice.

2.3.4 Named Grants, NAAF/LSAP, ICCLR and Assessed Contributions

Program impacts derived through funded projects and activities, as described above, pertain to the achievement of the core JPIP objectives. Recipients of Named Grants, LSAP and ICCLR, alongside other funding recipients, also contribute to the achievement of the core objectives. The main difference between regular grants and contributions recipients and recipients of Named Grants, LSAP and ICCLR, is that the latter have their own, specific JPIP objectives^[21] in addition to the core objectives. In each of these cases, the objective is essentially to fund the target organization(s) in pursuing its aims that are in line with the Department priorities.

The evidence supports the finding that JPIP-funded activities undertaken by recipients of Named Grants and the ICCLR promoted dialogue and understanding among justice stakeholders respecting justice issues. As part of the file review, data were collected and summarized on outputs, and results were produced by Named Grant projects from four organizations. All these projects primarily consisted of dialogue activities such as conferences and symposiums; workshops, e.g., on new breath and blood alcohol testing equipment; and meetings, e.g., on the prevention of economic fraud and identity-related crime, drug evaluation and classification program. Projects also included training and other educational activities such as courses; seminars, e.g., Judging through Public Emergencies; Innovative Criminal Trial Management; The Art and Craft of Judging; Neuroscience in the Courtroom; Judging and Social Inclusion: Disability; and training programs, such as the annual International Human Rights Training Programme (to improve understanding of human rights and the role of human rights education in empowerment and social change). A few organizations reported that they published conference materials/proceedings and educational materials such as toolkits and binders as part of their projects’ activities.

Three organizations conducted projects whose files contained information on feedback from participants on dialogue activities or training sessions/courses. Overall, participants reported increased knowledge and understanding of justice-related issues discussed during workshops, conferences or courses, and demonstrated a willingness to attend more events on similar topics. As part of a project providing the Human Rights Training Program, a participant satisfaction survey was conducted and showed that 91% of participants reported that their satisfaction with the training program was “definitely high” or “high”, and 79% of participants indicated they felt “definitely able” or “very probably able” to use a framework based on internationally accepted human rights standards and principles to analyze the issues encountered in the work of their organizations.

Key informants thought that JPIP’s Named Grants were significant in assisting NGOs listed in the Main or Supplementary Estimates, including ULCC. One respondent thought that JPIP’s Named Grant funding provided stability for the NGOs listed in the Main or Supplementary Estimates, allowing these organizations to focus on creating different projects, leverage other funds, and innovate. Another key informant (who was only able to speak to one of the Names Grants) indicated that that particular grant assisted one NGO to undertake extensive work complementing the Department’s mandate. This respondent also pointed out the element of separation from both the provincial and federal governments that this NGO enjoyed, allowing it to enable discussions that do not bind either level of government, but rather open up avenues for consultation and discussion.

Between 2007 and 2011, the ICCLR organized several fora and symposia, including, for example, “Criminal justice responses to violence against women: Linking local and international efforts” and “National and international perspectives on identity theft and fraud”. The Centre participated in several expert round tables and panels on a number of criminal justice topics, e.g., human trafficking, violence against women, international human rights. Over 2,000 justice stakeholders and/or members of the public attended these events. ICCLR developed several manuals, e.g., “Responding to victims of identity crime: A manual for law enforcement agents, prosecutors and policy-makers”.

JPIP funding provided to ICCLR and Named Grant recipients was found to promote incremental benefits; i.e., the evidence from key informants and the file review supports the observation that these organizations were able to fulfill their mandates at least in part, as a direct result of JPIP funding.

Evidence of objectives achievement is not as strong with respect to funding for LSAP. The NAAF reported that in 2009-10, 49 applications for assistance were received, 47 applications were approved (average award: $1,277), and students who received a bursary included 11 Non-Status First Nations members and 32 Métis. In 2010-11, 31 applications were approved (average award: $3,397). Evidence was insufficient to assess unequivocally whether JPIP support (via NAAF) made the difference between students entering or not entering law programs. Key informants, however, stated that although it was difficult to track whether LSAP promoted equitable representation in the legal system of Métis and non-status Indians, LSAP was critical for pre-law students. This applies primarily to the Program of Legal Studies for Native People at the University of Saskatchewan. This eight-week course is offered to Aboriginal students from across Canada and focuses on teaching its students the skills they will need to succeed in law school. Students also participate in a customary law component which incorporates Aboriginal traditions such as talking circles and other ceremonies.

The evaluation also examined assessed contributions to the Hague Conference on Private International Law and the International Institute for the Unification of Private Law (UNIDROIT). Evidence was limited to the file review and key informant interviews. The objective of paying these annual assessed contributions is to enable Canada to meet international financial obligations to which it has agreed. There are no reporting requirements associated with these contributions; essentially, payment of the assessed amounts constitutes fulfillment of the objective. In the case of the Hague Conference, the project file contained annual reports. From these reports it would appear that activities undertaken by this recipient are in line with other JPIP-funded activities. Hague Conference activities in 2008-09 included the adoption of a new Convention on the International Recovery of Child Support and other Forms of Family Maintenance, and the Protocol on the Law Applicable to Maintenance Obligations.

2.4 Demonstration of Efficiency and Economy

Evidence from all sources was used to address the performance issue respecting the demonstration of efficiency and economy. Findings are presented under three headings: Administration, Program Operational Efficiency, and Leveraging.

2.4.1 Administration

Program materials such as the application form, application guidelines, reporting templates, and the JPIP performance summary sheet appear to be clear, comprehensive and user-friendly. Those who are familiar with the Program – for example, PLEI organization proponents – state that the Program webpage clearly communicates its features and application requirements. Other sources indicate, however, that the Program is not well publicized, i.e., if an organization does not already know about the Program, it may have difficulty learning about it. A Department of Justice key informant explained that the Program does not actively publicize its existence since organizations which are involved with the justice community – in areas such as missing and murdered Aboriginal women, access to justice for marginalized persons, and family violence – already know about the JPIP program and its staff. However, several project key informants said that it might be difficult for individuals who are new to the justice sector and in need of funding to find information about the JPIP program. A key informant from outside the Department stated that although the information on the website is good, one has to look for it in order to find it. Another Department of Justice key informant thought that Justice needs make it is easier for potential applicants to understand the Program and thus more easily apply for funding; the interviewer also noted some repetition in application questions.

The application procedure was examined through the file review focusing on the time taken between key points in the process. In the majority of cases, time lags were found to be short and response times were prompt. Correspondence between proponents and JPIP administrators appeared in some files. In general, analysis of correspondence revealed what appeared to be effective service, including explanations and assistance to help proponents improve their proposals where appropriate.

Long delays, either in funding decisions or in payments, occurred in a few cases. The files did not always contain explanations of these delays, particularly respecting delays in funding decisions. Where payments were delayed, it tended to relate to a failure on the part of the organization to submit required documentation.

Files also contained contribution agreements and project reports submitted in compliance with the requirements stated in the contribution agreements. Many of these agreements contain specific reporting requirements. In virtually all cases, it appeared that proponents were able to meet these requirements.

Exhibit 8 summarizes administrative data obtained from the most recent project file of 28 of the 30 organizations examined and 3 of the 5 case study files reviewed. Average and median processing times were computed, from application submission to issue of final payment, for grants, contributions (including under the Aboriginal Missing and Murdered Women Initiative) and core funding (including PLEI core funding and ICCLR) files. Because the average was sometimes significantly affected by one or two outliers, the median is also provided. LSAP, ICCLR and Named Grants were not treated as a separate stream, but rather included under the Contributions, Core Funding and Grants categories respectively, as it was expected that the application process would be slightly different between grants, contributions and core funding files.

The median time between application submission and receipt of a formal approval letter was 60 days for Contributions, 62 days for Grants, and 85 days for Core Funding files. Overall, the median time between application submission and formal approval for all files was 77 days. These time periods seem reasonable and comply with the Programs Branch service standards stipulating that the review and assessment of a funding proposal “may take up to 4 months depending on its complexity”.^[23]

It is also worth noting that 9 organizations out of 31 across three groups received acknowledgement of receipt of their project application on the same day it was sent (usually by email). The other projects’ times ranged from 2 to 25 days between application submission and acknowledgement of receipt.

Comparison across three groups of the average and median number of days between when a final claim is received by Justice and a final payment is issued indicates that contribution payments required more time to process than grants and core funding files. However, the number of contribution files reviewed providing this information is too small to draw conclusions from these results (5 out of a total sample of 13). The time between receipt of an interim claim and issue of interim payment (for 9 out of 13 contribution files) averaged 9 days.

Surveyed applicants were asked about their satisfaction with various elements of the JPIP administrative process. Most applicants were satisfied with most of the aspects of the application process.

• 100% of respondents reported being satisfied or very satisfied with the ease of interaction between themselves and Justice Canada.

• All successful applicants were satisfied with the appropriateness of the selection criteria (unsuccessful applicants were less satisfied with this aspect of the application process).

• 89% of successful applicants were satisfied or very satisfied and 71% of unsuccessful applicants were satisfied or very satisfied with the timeliness of responses to their communication with Justice Canada.

• 89% of successful applicants were satisfied or very satisfied with the clarity of the application process.

• 69% of successful applicants and 57% of unsuccessful applicants were satisfied or very satisfied with the timeliness of the information about their application status.

• 71% of successful applicants and 57% of unsuccessful applicants were satisfied or very satisfied with the type of information required in the application process.

• 83% of survey respondents said that the current application forms and processes allowed their organizations to provide enough information so that Justice Canada can make good decisions.

• 83% of successful applicants and 63% of unsuccessful applicants were satisfied or very satisfied with the ease of the application process.

• 89% of successful applicants were satisfied or very satisfied with reporting requirements.

• 83% of successful applicants were satisfied or very satisfied with the time period for which the funding was awarded.

• 72% of successful applicants were satisfied or very satisfied with size of the amount awarded.

• 67% of successful applicants were satisfied or very satisfied with the timeliness of payments.

These findings were echoed by the responses of project key informants. All of them were satisfied with JPIP application forms, criteria and process. They also indicated satisfaction with communications with JPIP program staff. Interviews conducted as part of the PLEI organization survey also indicated that there is a high level of satisfaction with application forms and the timeliness of the approval and payment processes. Interviewed ICCLR, NAAF and ULCC project respondents were also very satisfied with communications with JPIP program staff.

Dissatisfaction was expressed with respect to one area, corresponding to observations noted above. Survey respondents indicated a low level of familiarity with JPIP. Overall, successful applicants were more familiar with JPIP (50% reported being somewhat or very familiar with JPIP) compared to unsuccessful applicants (13% reported being somewhat familiar with the program and none were very familiar). Some respondents explained that although they were familiar with certain elements or objectives of the Program, they did not have a sense of the whole Program. When asked how familiar other organizations were with JPIP, 46% of successful applicants and 100% of unsuccessful applicants thought that these other organizations would be “slightly familiar” or “not at all familiar”.

Monitoring and reporting requirements regarding outputs appear to be in place for all JPIP-funding recipients, including recipients of regular grants and contributions and PLEI organizations receiving core funding as well as LSAP, Named Grant recipients and ICCLR (which provided a tailored report to JPIP at the end of each funding cycle). The file review revealed what appeared to be good monitoring of PLEI organization activities and outputs. If anything, it appears that PLEI organization reporting requirements are more comprehensive and strict than those associated with other grants and contributions. Overall, almost all projects reviewed (the exception being eight ongoing projects) complied with reporting requirements outlined in their agreements.

Department of Justice key informants stated that applicants generally comply well with project performance measurement requirements and processes. One respondent said that organizations comply well, although descriptions of outcomes are not always clear. Another respondent stated that s/he was satisfied that applicants are complying with project performance measures and that these are improving.

Project key informants generally thought that project performance measurement requirements were appropriate. They referenced various reports that they provide to the Department of Justice, e.g., annual reports (both narrative and financial), interim and final reports, and specific performance indicators for which they regularly collect data. In addition to collecting, using and reporting on the various performance indicators referred to earlier in this report (e.g., number and type of PLEI products developed), one PLEI organization distributes and collects evaluation forms with each publication order and each lawyer referral. This organization also distributes and collects participant surveys at each conference and training course. It then uses the results of such evaluation tools to improve its various information services, e.g., edit publications, adjust conference topic offerings, adjust the list of lawyers to whom PLEI organization staff refer clients.

The JPIP expenditures from 2007-08 to 2010-11, as depicted in the Introduction section, have been less than the available budget for each of the four years, resulting in some lapsing of funds. Although some of this limited take-up may be a result of a low level of program familiarity, instances of administrative delays are likely also a contributing factor. It should be noted, however, that the lapsing has been reduced dramatically each year over the course of the period covered by the evaluation.

2.4.2 Program Operational Efficiency

At the time of the evaluation, there were 2.3 full-time staff attached to the Program. This compares with eight FTEs identified by the 2006 JPIP evaluation study as being directly associated with the Program. JPIP’s umbrella terms and conditions have allowed the Department to address issues respecting missing and murdered Aboriginal women, family violence, racism and other emergent issues without extra administration. One Department of Justice respondent thought that JPIP’s efficiency could be improved by creating summary reporting templates which are focused on outcomes, as well as changing application forms to make them easier to fill out (e.g., the evaluation section of the form).

2.4.3 Leveraging

Survey respondents attributed the cost effectiveness of their JPIP-funded projects to three main factors. First, most JPIP funding recipients also received funding from other funding organizations in addition to their JPIP funding. In many cases, JPIP funding helped the proponent organization to secure additional funding by either supporting infrastructure that enabled the organization to seek funding from other sources, or by lending legitimacy to their projects, thus attracting additional partners. For example, one respondent indicated that JPIP provided $10,000 to his/her organization, which obtained an additional $90,000 from other sources. Some survey respondents noted that departmental managers were very helpful in assisting the respondents in securing support from other government departments and/or funders. The PLEI organization survey findings indicated that all organizations were able to leverage funds with JPIP core funding. Both ICCLR (as a recipient of core funding) and ULCC (as a recipient of Named Grants) indicated they have leveraged further monies from other funding sources.

The second factor contributing to project cost effectiveness was said to be in-kind contributions. Many JPIP-funded organizations reported receiving significant in-kind donations. These often took the form of volunteer time from lawyers and other experts. The third factor relates to the reach of project benefits. According to respondents in the PLEI organization survey, JPIP funds are being used to achieve expected results. In some cases, these funds are used to support staffing, rent, printing and distribution of materials, while other organizations will use them for their operating costs including travel, developing partnerships, professional duties and bookkeeping, and specific programs. Surveyed applicants reported what they considered to be substantial benefits for relatively modest outlays. One respondent stated, “[benefits are] big. We support 12 chapters and universities across Canada, generating huge activity.” Another stated, “[benefits are] extensive. [Our activities] save host cities millions and millions [of dollars] each year.” The running of ICCLR’s core-funded project was seen as being less costly than if it had been run by the Department, and one of ICCLR’s initiatives alone could represent millions of dollars in savings for the federal and provincial governments. It was considered that without JPIP core funding, no ICCLR projects or activities would have taken place. Key informants indicated that without JPIP Named Grant funding, there would not have been any work done by ULCC.

• ^[19]  University of Saskatchewan’s Native Law Centre of Canada website for the Program of Legal Studies for Native People (PLSNP): Native Law Centre of Canada. (2007). Legal Studies for Native People (PLSNP). Retrieved from http://www.usask.ca/nativelaw/programs/plsnp.php
• ^[20]  Statistics Canada, The Incarceration of Aboriginal People in Adult Correctional Services, Juristat 29(3), July 2009.
• ^[21]  Objective 5: To assist selected NGOs, as listed in the Main or Supplementary Estimates, whose mandate and/or activities complement the Department’s mandate, objectives and legal and socio-legal priorities by providing a grant. Objective 6: To promote equitable representation in the legal system of Métis and Non-Status Indians by encouraging them to pursue their studies in law. Objective 8: To assist the International Centre for Criminal Law Reform and Criminal Justice Policy in promoting human rights, the rule of law, democracy and good governance.
• ^[22]  As processing-time information was missing in some files, the number of files used to calculate the average and median times does not always equal the total number of files reviewed.
• ^[23]  Department of Justice Canada Website. (2010). Funding Programs. Retrieved on August 9, 2011 from: https://canada.justice.gc.ca/eng/pi/pb-dgp/index.html.

• Previous Page
• Table of Contents
• Next Page

Date modified:

2018-09-26