Evaluation of the Justice Partnership and Innovation Program

4. Findings

This section presents the findings of the evaluation according to relevance and performance issues.

4.1. Relevance

The core issues with respect to relevance include the following:

4.1.1. Continued need for the JPIP

The evaluation found a continued need for the types of initiatives funded through the JPIP. All key informants believe that the JPIP is still necessary and relevant. The JPIP addresses what are viewed as important ongoing issues — such as family violence and violence against Indigenous women — and can assist in identifying emerging trends and gaps. The JPIP can also be used to address emerging issues. Interviewed recipients were all able to identify objectives of the JPIP that aligned with the objectives of their own projects. The file review confirms these observations, with all projects assessed as aligning with one or more JPIP objectives. Just over half of survey respondents (including approved and unapproved applicants) believe that the JPIP objectives are completely or very much aligned with the justice needs of the general public (52%; n=24) as well as with the justice needs of the legal community in general (54%; n=25), while most others believe they were somewhat aligned (33%, n=15; and 22%, n=10, respectively).

This strong alignment with JPIP objectives indicates project applicants’ perceived need for and desire to take part in initiatives in these areas. As further evidence that stakeholders view the JPIP objectives as important for meeting the needs of Canadians, and also that support is needed to achieve these objectives, applicant survey respondents reported a high level of need for support to their organization to assist them in working toward achieving JPIP objectives.

In particular, over 80% of respondents identified a moderate to great need for support to help their organization achieve the following JPIP objectives:

As well, over two-thirds of respondents identified a moderate to great need for support to help their organization achieve the following JPIP objective:

The continued need for JPIP funding to support initiatives that strive to achieve specific JPIP objectives is discussed below.

Building knowledge, awareness, and understanding around justice issues.

Key informants spoke widely of the need for building knowledge and awareness of the justice system and justice issues in a wide range of areas. Funded PLEI activities are viewed as needed to help the public understand the law, the Canadian justice system, their rights, and how to deal with their legal matters. PLEI activities are particularly of importance to assist the increasing number of self-represented litigants. Furthermore, PLEI materials need to be provided in a user-friendly manner and in a number of formats to ensure the materials are accessible and understandable to a wide range of audiences. A few key informants observed that PLEI activities contribute to the Nine-Point Access to Justice Roadmap outlined by Justice Cromwell in the report on access to civil and family justice. This report, conducted by the Action Committee on Access to Justice in Civil and Family Matters, identifies a serious access to justice issue in Canada due to the civil and family justice system being complex, slow, and expensive. One of the main beginning elements of the roadmap laid out in the report for achieving access to justice is for a front-end set of services that can facilitate early resolution of legal matters to avoid the involvement of the formal justice system (e.g., tribunals and court). The report identifies public legal information and education as one of the foundational components of these early resolution services (Action Committee on Access to Justice in Civil and Family Matters, 2013).

Key informants spoke of the need to build awareness, knowledge, and understanding for the public as well as for justice stakeholders with respect to specific issues that are addressed through the JPIP, such as family violence, violence against Indigenous women and girls, and the harmonization of laws.

Identifying emerging trends, issues, and/or gaps and possible responses with respect to the justice system.
Key informants spoke of how the funded initiatives can contribute toward identifying emerging trends and gaps, and the importance of this work to inform and improve the justice system. For example, a key role of the ICCLR is to inform and support the federal government in key priority areas with respect to criminal law and to do so in both a national and international context. It does so through the early identification of trends occurring at the national and international level, and supporting policy-oriented research with respect to responding to trends. The work of the ULCC is also viewed as important for identifying aand contributing to justice reform to address trends, issues, and gaps in the justice system. One way the ULCC does this is in bringing attention to issues identified by stakeholders working in the justice system, such as judges and other practitioners, or by provinces and territories.
Participating in efforts to harmonize international private law.
Canada’s participation in efforts to harmonize international private law is viewed as needed to assist Canadian individuals, families, and commercial enterprises in resolving their legal matters that have an international context. Furthermore, harmonization has become increasingly important, given the global nature of commercial entities as well as the rising tendency of Canadians to have personal and business ties to other countries.
Increasing awareness of and strengthening responses to family violence.
Family violence is a pervasive problem in Canada and globally, as evidenced by a 2014 report by Canada’s Chief Public Health Officer (Public Health Agency of Canada, 2016). According to the report, while acknowledging that complete and reliable statistics on family violence are difficult to obtain, available data indicate the following: on average, 172 family-related homicides are committed in Canada annually; those at greatest risk of family violence include women, children, Indigenous peoples, people with disabilities, and people who identify as lesbian, gay, bisexual, trans, or questioning; women are more likely to be victims of intimate partner abuse (physical and sexual) and to be killed by an intimate partner; and family violence is often unreported, due to the reluctance of victims to speak out. All key informants spoke of the need to increase awareness of and strengthen the response to family violence, with many interviewed recipients providing examples of how their projects contribute to addressing this need. An example of highlighting this need was provided through case study interviews that revealed relationship violence as an increasing form of violence, while a culture of shame still exists in reporting this violence. Empowering young women to acknowledge relationship violence and seek assistance is viewed as an important unmet need that can be addressed through JPIP funding. Furthermore, a few key informants working in the area of family violence indicated that there is a need for more work to ensure that victims and their families receive equitable and fair access to justice, or that the issue of forced marriages is an emerging issue in Canada requiring further attention with respect to protecting and supporting victims of forced marriages.
Reducing the vulnerability of young Indigenous women to violence.

A 2009 Statistics Canada study based on General Social Survey data found that Indigenous women self-reported rates of violent victimization three times higher than that of non-Indigenous women (Brennan, 2011). The 2013 study initiated by the Commissioner of the RCMP on MMIWG found that not only are Indigenous women over-represented with respect to missing and murdered women, but that the numbers exceeded previous estimates. The study reported police records of 1,017 homicide victims and 164 missing Indigenous women and girls (Royal Canadian Mounted Police, 2014). Further evidence of the recognition of the need for initiatives to reduce the vulnerability of Indigenous women to violence is the Parliament-established Special Committee of the House of Commons that conducted hearings regarding MMIWG. The Special Committee’s report Invisible Women: A Call to Action looked at the extent of MMIWG in Canada and considered the root causes of violence against Indigenous women (Special Committee on Violence, 2014). Key informants also spoke of the need for programming to assist vulnerable Indigenous women to gain an understanding of how to recognize violence and of the supports available to address the violence.

Promoting the equitable representation of Métis and Non-Status Indians in the legal system by encouraging them to pursue their studies in law. The financial support from the JPIP is viewed as needed to financially assist Indigenous students enrolled in a pre-law or law program and to increase their access to becoming members of the legal profession. In recognition of the need to address the educational disparities that exist for Indigenous students, the Truth and Reconciliation Call to Action made a series of recommendations around education, including for the development of a joint strategy between the federal government and Indigenous groups “to eliminate educational and employment gaps between Aboriginal and non-Aboriginal Canadians,” as well as for the “federal government to provide adequate funding to end the backlog of First Nations students seeking a post-secondary education” (Truth and Reconciliation Commission of Canada, 2015). In the December 2015 Speech from the Throne, the Government of Canada committed to work to implement the Truth and Reconciliation’s recommendations and to “work with First Nations so that every First Nations child receives a quality education” (Government of Canada, 2015c).

Responsiveness of the JPIP to the needs of Canadians

The JPIP’s scope is necessarily broad given its wide range of objectives. JPIP activities and funding investments include providing funding for the following purposes, and, as was discussed above, stakeholders believe there is a need for this support:

Similarly, recipients’ area of focus for their JPIP-funded initiative is broad, as revealed through both the file review and the applicant survey (Table 3). The majority of survey respondents (approved applicants) reported that their initiative’s area of focus was related to family violence and access to justice, and a majority of the files reviewed had access to justice as an area of focus. Over 40% of both the survey respondents (approved applicants) and file reviews had an area of focus related to PLEI; over 30% for each had victims of crime as an area of focus and, for files reviewed, family violence.

Table 3: Areas of focus for JPIP-funded initiative – survey of applicants (approved applicants) and file review
Areas of focus Survey approved applications (n=38) File review (n=31)
# % # %
Family violence 25 66% 11 35%
Access to justice 21 55% 17 55%
PLEI 18 47% 13 42%
Victims of crime 14 37% 10 32%
Violence against Indigenous women and girls 9 24% 6 19%
Justice policy 5 13% 5 16%
Missing or murdered Indigenous women 3 8% 5 16%
International law (justice-related issues in the international fora) 2 5% - -
Criminal law reform 1 3% 2 6%
Forced marriage - - 4 13%
Other 7 18% 6 19%

Note: Respondents could provide more than one answer; totals may sum to more than 100%.

Caution: Small sample size; response from a total of 114 applicants surveyed.

Initiatives funded through the JPIP also take part in a variety of activities to achieve their objectives, as demonstrated in Table 4.

Table 4: JPIP-funded initiatives’ activities – survey of applicants (approved applicants) and file review
Activities that recipients take part in Survey approved applications (n=38) File review (n=31)
Offering workshops, conferences, symposia 61% 52%
PLEI projects 53% 39%
Offering training sessions 45% 39%
Conducting pilot, demonstration, or research projects 40% 16%
Promote increased harmonization of legislation 11% 3%
Promote international cooperation on justice issues 8% 3%
Research related to access to justice 5% 29%
Support-type services (counselling, legal advice) - 32%
Hosted events to raise awareness - 6%
Other 15% 16%

Note: Respondents could provide more than one answer; totals may sum to more than 100%. Some recipients may have been included in both survey and file review findings.

Caution: Small sample size; response from a total of 114 applicants surveyed.

All lines of evidence identified that JPIP initiatives had a wide reach with respect to the types of target audiences and beneficiaries, which included Indigenous women and girls, or Indigenous people in general; women or women and girls; youth, including youth at risk and youth involved in the justice system; seniors; victims of crime; visible minority groups; justice professionals and other professional groups; government; northern and/or remote communities; urban communities; persons with disabilities and/or mental health issues; and the general public, amongst other groups. Table 5 illustrates the findings from the applicant survey and file review.

Table 5: Most commonly identified intended direct beneficiaries or target populations for project(s) – survey of applicants (approved applicants) and file review
Most commonly identified direct beneficiaries/target populationsTable note i Survey approved applications (n=38) File review (n=31)
# % # %
Indigenous peoples 21 55% 6 19%
Youth (12 to 18 years old) 21 55% 8 26%
Victims of crime 20 53% 6 19%
Justice-related professionals 17 45% 16 52%
Ethno-cultural or visible minority groups 15 40% 4 13%
Urban communities 15 40% - -
Remote communities 14 37% - -
Persons undergoing separation or divorce 12 32% - -
Persons with mental health issues (including FASD) 12 32% - -
Seniors 12 32% 2 6%
Youth at risk 12 32% 2 6%
General public - - 9 29%
Table note i

Includes only direct beneficiary groups identified by 25% or more of survey respondents and/or files reviewed.

Return to table note i referrer

Note: Respondents could provide more than one answer; totals may sum to more than 100%.

Caution: Small sample size; response from a total of 114 applicants surveyed.

The intended reach of a project is indicated by the numbers and types of activities offered, as well as by who the target audience is. Actual reach is indicated by the numbers of the target population that avail themselves of the offered activities and materials. Participation in JPIP-funded activities and the use of materials offered were primarily available to the evaluation through examples provided by key informants and reviews of project reporting.

Some examples of reach for PLEI activities reported by PLEI organizations as identified through the file review include the following:

The case study of the Community Legal Education Association Incorporated revealed detailed information on reach, such as a law phone-in line staffed by full-time lawyers and taking an average of 6,800 calls per year, as well as responding to email queries (1,215 emails in 2015); a lawyer referral program, with 1,262 referrals provided in 2015; an inventory of 120 titles as of 2015 (including their own materials and those from other organizations); 8,500 items distributed per year on average; and approximately 135,000 visitors to the PLEI website over the past three years. The Community Legal Education Association Incorporated also offered community activities, such as various PLEI community events to provide free legal information sessions and community legal intermediary training to provide training on the legal system and various areas of law to other service providers who may interact with individuals in need of legal information.

Examples of reach for other types of JPIP-funded initiatives include the following:

Needs not being met or emerging needs

Many key informants also identified areas where justice needs of Canadians are not being sufficiently met, or emerging needs. Examples given for unmet needs include continuing gaps in services and resources for self-represented litigants, or for services for specific types of vulnerable groups (e.g., victims of crime, people new to Canada, low-income people, youth, the growing senior population, people living in remote communities). Others indicated it was not so much that there were specific areas of unmet need, it was more that the JPIP did not have sufficient funds to meet the level of needs. As a result, the program is not always able to fund initiatives of interest for emerging justice issues.

Key informants identified continued needs with respect to PLEI, such as for plain-language materials, the provision of PLEI services and PLEI resources in non-official languages, paralegal assistance in completing forms, and practical and procedural step-by-step guides, with much of this information targeted to assisting self-represented litigants. Identified areas of law need include those related to consumer, debt, accidents, and injuries.

Some key informants provided suggestions on emerging areas of need, which included implementing trauma-informed approaches where service providers are trained to be more aware of the impact of trauma on people; considering the interconnectedness of issues such as drugs, addiction, mental health, violence, and justice, particularly in light of the federal government’s intention to legalize and regulate marijuana; restorative justice in Indigenous communities; human trafficking in the context of domestic violence; general legal education in relation to issues of violence and violence prevention at schools, the workplace, and online (i.e., cyberbullying). A few key informants had differing opinions on whether forced marriage is an area of emerging need.

4.1.2. Alignment with Government Priorities

The JPIP’s alignment with government priorities can be demonstrated through the program’s logic model and the linkages of outcomes to priority outcomes of the Department. From the logic model (see Appendix A), each of the sub-activities, outputs, and immediate and intermediate outcomes are expected to lead to the ultimate outcomes of “increased access to the Canadian justice system” and a “strengthened Canadian legal framework.” These ultimate outcomes contribute to the Department’s strategic outcome of “a fair, relevant, and accessible Canadian justice system,” as defined in the Department’s Program Alignment Architecture (PAA) (Department of Justice Canada, 2016a). As well, JPIP falls under Department Program 1.1: Stewardship of the Canadian Legal Framework, which supports the Government of Canada’s priority outcome for “a safe and secure Canada” (Department of Justice Canada, 2016b).

The JPIP’s specific objectives are also aligned with government priorities. The JPIP objective of reducing the vulnerability of young Indigenous women to violence and empowering Indigenous women aligns with the federal government’s priority areas related to MMIWG (Government of Canada, 2015b; Office of the Prime Minister, n.d.).

The Action Plan to Address Family Violence and Violent Crimes Against Aboriginal Women and Girls further articulates the federal government’s priority areas for addressing violence against Indigenous women and girls. The action plan was in response to recommendations made by the Special Committee on Violence Against Indigenous Women. The plan outlines actions that were to be made by the federal government over five years to prevent violence, support victims, and protect Indigenous women and girls. Actions defined in the plan align well with the JPIP objectives related to family violence and reducing the vulnerability of young Indigenous women to violence including:

The Mandate Letter for the Minister of Justice and Attorney General of Canada also outlined the Minister’s areas of responsibility, including “policy in such areas as criminal law, family law, human rights law, public law and private international law, constitutional law and Aboriginal justice,” most of which align well with the needs targeted by the JPIP (Office of the Prime Minister, n.d.). Key informants who could speak on alignment with government priorities further confirmed this alignment, observing that funded initiatives align with departmental priorities for improving access to justice, enhancing the justice system’s response to family violence, promoting a fair and accessible Canadian justice system, and developing approaches to strengthen the Canadian legal framework. In particular, key informants mentioned that the JPIP aligns well with current governmental priorities with respect to family violence and violence against Indigenous women and girls. As well, given that the Department of Justice has the responsibility of ensuring Canada’s membership in the HCCH and UNIDROIT, the JPIP aligns well with meeting this responsibility.

Key informants also view the JPIP as complementing initiatives taking place in other federal departments/agencies and other jurisdictions. For example the Department of Justice, Status of Women Canada, Public Safety Canada, and Indigenous and Northern Affairs Canada all have distinct and complementary roles in the Action Plan to Address Family Violence and Violent Crimes Against Aboriginal Women and Girls. PLEI organizations receiving PLEI funding through the JPIP are organizations designated by their province to provide PLEI services.

4.1.3. Alignment with Federal Roles and Responsibilities

The JPIP appears to align well with federal roles and responsibilities as outlined in legislation and articulated in the Department’s roles and responsibilities with respect to the administration of justice. Section 4 of the Department of Justice Act (R.S.C., 1985) outlines the powers, duties, and functions of the Minister, including to:

“(b) have the superintendence of all matters connected with the administration of justice in Canada, not within the jurisdiction of the governments of the provinces;

(c) generally advise the Crown on all matters of law referred to the Minister by the Crown; and

(d) carry out such other duties as are assigned by the Governor in Council to the Minister” (Government of Canada, n.d.).”

Within these responsibilities, the Department of Justice has three defined roles, all of which are related to the objectives of the JPIP, including acting as “a policy department with broad responsibilities for overseeing all matters relating to the administration of justice that fall within the federal domain — in this capacity, it strives to ensure a fair, relevant, and accessible Canadian justice system for all Canadians” (Department of Justice Canada, 2016b).

The JPIP’s overall goal of supporting the policy directions of the Department of Justice directly assists the Department in its role of acting as a policy department. Each of the JPIP’s specific objectives can be viewed as contributing to this overall JPIP goal and, therefore, support the policy directions of the Department. The JPIP objectives related to the identification of justice-related emerging trends, issues, and/gaps, and of building knowledge, awareness, and understanding can be viewed as contributing not only the first role, but to all three of the Department’s roles defined above. Furthermore, the funding provided by the JPIP supports the work of various organizations that have the potential to assist the Department in each of the above roles. This includes each of the named grants recipients, the ICCLR, the ULCC, the HCCH, and UNIDROIT. For example, key informants reported that the five organizations receiving named grant funds will advise and inform the Department and other stakeholders on legislative matters. Key informants also observed that the funded initiatives contribute toward the federal government’s ability to fulfill its roles in such areas as promoting access to justice, and in the provision of accessible justice-related information, as well as with assisting the federal government in its shared jurisdiction with provinces/territories for the administration of justice. PLEI organizations that receive funding from both their provincial governments and the JPIP are examples of the latter.

4.2. Performance — Achievement of Immediate Outcomes

4.2.1. Increased Capacity to Build Knowledge, Awareness, and Understanding

Extent that stakeholders can continue ongoing activities

The JPIP is important to recipients to assist them in undertaking the various activities, services, and supports that contribute to building knowledge, awareness, and understanding of justice issues. While JPIP funding may not constitute all or even a majority of some recipients’ resources, key informants reported the funding is still critical to organizations for assisting with such things as core operations, offering programming, expanding their client base, and leveraging resources. For some recipients, however, the JPIP funds are the primary or sole source of cash funds for their initiative, with any partners primarily providing in-kind resources. Most key informants believe JPIP recipients’ services or projects would either have to discontinue or be curtailed without JPIP funding.

This belief was further confirmed by the applicant survey, where almost all (95%; n=36) of the respondents with an approved application indicated that their project would not have been able to proceed without JPIP funding. Similarly, close to three-quarters (72%; n=13) of the respondents with an unapproved application indicated that their project was unable to proceed without JPIP funding. As well, the majority of survey respondents indicated their organization had a great to moderate need for support to help them conduct various JPIP-funded activities, such as the following:

And, in particular, almost three-quarters of respondents (72%) indicated they had a great need to assist them in developing and/or offering resources, tools, and supports related to family violence, and close to two-thirds (63%) indicated they had a great need to assist them in conducting pilot, demonstration, and research projects and in conducting PLEI projects.

Some specific examples of how the JPIP allows organizations to continue their ongoing activities include the following:

Extent that the JPIP contributes to increased capacity

Separating out the extent that recipients’ capacity has increased due to the JPIP funding from the extent that recipients are able to continue ongoing activities is challenging, particularly as there is no baseline upon which to measure ongoing activities versus increased capacity. Evidence of increased capacity is primarily from recipient specific examples, as provided through key informants, case studies, and the file review. Some of these examples are provided below:

Key informants also identified that language and other barriers can affect clients’ (or potential clients) abilities to access and make use of needed information and services. Examples were provided of how some funding recipients make efforts to address the language needs of clients, which contributes to capacity for building knowledge and understanding. Such efforts could be in the form of offering PLEI materials in both official languages, or through translating materials into other languages, including Mandarin, Hindi, Spanish, and Arabic. A few key informants who provide services to Indigenous communities reported more material should be produced using Indigenous languages, while also ensuring the material is culturally relevant. However, it was noted that it can be difficult to find translators for some Indigenous languages. Others have used Indigenous interpreters in programming or services.

For those initiatives receiving JPIP funding on an annual basis — such as for core funding or named grants — this ongoing funding provides a certain level of security that assists organizations with planning for both ongoing and new activities. However, organizations receiving short-term funds — such as through special initiatives funding for one year — can find it challenging or not possible to offer the services once JPIP funding has ended. Alternatively, there may be an interruption in services while waiting for program decisions on continued funding, or the organization may not have the capacity to implement or conduct ongoing communications or training on tools and resources developed through the JPIP-funded initiative.

Contribution of partnerships to capacity

Partnerships are an important aspect of JPIP-funded initiatives for contributing to capacity; in fact, partnerships are encouraged by the Program and multiple partners are not uncommon. Just over three-quarters (76%, n=29) of the survey respondents with an approved application indicated they had a partner for their JPIP-funded initiative. Almost all (90%, n=28) of the reviewed files identified partners, with the number of partners ranging from one to over 40. Just over one-third (35%, n=11) of the reviewed projects identified one to five partners, and 29% (n=9) identified 10 to 20 partners.

While somewhere under half of the partners (45%, n=13 of approved applicants responding to the survey; 36%, n=10 of files reviewed) provide cash funds, the majority provide in-kind contributions (79%, n=23 in the survey; 88%, n=23 in the file review). In-kind contributions are primarily in the form of human resources, expertise, and advice.

JPIP recipients have a high level of satisfaction with their partnerships; 76% (n=22) of survey respondents were very satisfied, and 24% (n=7) were satisfied with their partnerships. Partnerships are highly valued not only for the cash resources they provide but also for the expertise and knowledge they contribute to the initiative. Key informants identified that partners often have expertise that assists the initiative in developing and delivering resources, tools, and programming. As well, partners often come with their own networks and associations, which can contribute to increasing the reach of the initiative. Partners will often take tools, resources, and information developed through initiatives, and implement and use them in their own organization, as well as share and endorse them with other organizations. Other identified helpful contributions by partners include volunteering their time toward the initiative, or providing space, supplies, and other infrastructure.

As revealed in interviews, funded initiatives have a wide range of partnerships. Initiatives operating at the community level, for example, will have partnerships and draw on the knowledge and skills of other community organizations or service providers in the areas of justice, social services, health, and education. Participants in the work of the ULCC include representatives of federal, provincial, and territorial governments; the Canadian Bar Association; academic institutions; and others. The ICCLR has partnerships not only with other justice stakeholders within Canada, but also at the international level, such as the United Nations (UN).

For several JPIP-funded initiatives — specifically the intergovernmental organizations HCCH and UNIDROIT — partnerships are critical to the work and ongoing operations of the organizations. The 80 member states of the HCCH and the 63 member states of UNIDROIT fund the organizations, make decisions, contribute to their work, reach agreements on conventions and other instruments, and subsequently adopt conventions for the purpose of harmonizing laws. Canada also makes contributions to the HCCH and UNIDROIT beyond the JPIP funding in the form of expertise and support from the Department of Justice, other federal departments, provinces and territories, and the private sector.

4.3. Performance — Achievement of Intermediate Outcomes

4.3.1. Increased Awareness and Knowledge

JPIP-funded activities that contribute toward increased awareness and knowledge

Most, if not all, of the activities that JPIP recipients initiate can be viewed as knowledge and awareness-building activities. In particular, as demonstrated in Table 4, a majority of recipients take part in activities that are conducted specifically to build awareness, knowledge, and understanding. For example, 61% (n=23) of survey respondents who had approved applications conducted activities such as workshops, conferences, and symposia. Just over half (53%, n= 20) of respondents indicated they took part in PLEI-related projects, and a substantial number were offering training sessions, or conducting pilot, demonstration, or research projects.

PLEI organization initiatives focus on assisting the public to better understand the Canadian justice system and the law, and offer a variety of activities in a number of formats to ensure that a wide range of topics are covered in a range of accessible formats. Examples gathered through the evaluation include call-in lines and responding to email queries: PLEI materials distributed online, as well as print forms distributed through the PLEI organization and through other service organizations; legal information sessions and presentations; and training to other organizations to assist them in providing legal information to their clients. The information provided covers a range of areas including criminal, family, and civil law, as well as general information on understanding and navigating the justice system.

Other examples were provided of activities conducted to increase awareness and knowledge in vulnerable populations. Some examples are provided below:

Some activities are intended to contribute to the increased awareness and knowledge of justice system stakeholders such as legislators, judges, the courts, federal and provincial justice departments, and students. A few specific examples include the following:

The evaluation was not able to obtain any overall estimates or assessments of numbers and types of activities offered. While reporting templates ask recipients to report on the reach of their project, such as the number of participants or number of materials distributed, the information is not collected in a manner that is easily aggregated. Additionally, the file review involved reviewing only a sample of JPIP files. However, a number of specific examples were identified through the file review and other lines of evidence, particularly for PLEI activities, and are described at the end of Section 4.1.1 of this report.

Promotion of awareness and knowledge, and extent of increased knowledge and understanding

The evaluation considered how the JPIP funded activities promoted awareness and knowledge, and the extent to which they contributed to increased understanding. All lines of evidence for the evaluation indicate that JPIP-funded initiatives do promote and contribute to increased awareness and knowledge of justice-related issues. As was demonstrated in Table 4, the majority of recipient activities involves knowledge-building, such as offering workshops, conferences, and symposia (61% of survey respondents with approved applications; 52% of files reviewed); conducting PLEI projects (53% of survey respondents with approved applications; 39% of files reviewed); offering training sessions (45% of survey respondents with approved applications; 39% of files reviewed); and conducting pilot, demonstration, or research projects (40% of survey respondents with approved applications; 16% of files reviewed).

Most survey respondents believe that their JPIP-funded initiative contributes to the JPIP objectives that are related to awareness and knowledge; moreover, the majority believes their JPIP-funded initiative does so to a great or very great extent:

Furthermore, from the file review, four of five named grant recipients reported their initiatives resulted in new information, and 77% (n=20) of other projects reviewed reported new skills, new information, and changed awareness.

The previous evaluation of the JPIP, published in 2012, also conducted an applicant survey. The current survey was developed using similar questions in order to facilitate a comparison of results. Compared to the previous survey, respondents participating in the survey for this evaluation report somewhat more positively on the extent to which their project contributes to the JPIP objective to build knowledge, awareness, and understanding on justice issues.

Examples of how JPIP initiatives are viewed as contributing to increased awareness, knowledge, and understanding are provided below:

The communication of project results and successes to other relevant stakeholders is an important aspect of increasing knowledge and awareness. Based on the file review, funding recipients are making efforts to share their project results, with 73% (n=19) of reviewed projects reporting that they communicated their results.Footnote 5 As shown in Table 6, meetings, conferences, websites, and workshops are popular methods of informing other stakeholders of project results.

Table 6: How project results were communicated – file review (n=19)
Type of communication Number of projectsTable note ii %
Meetings 17 89%
Conferences 14 74%
Website 14 74%
Workshops 13 68%
Local media 9 47%
Newsletters 7 37%
Reports 6 32%
Service providing agencies 5 26%
National media 4 21%
Other 6 32%
Table note ii

Not applicable to named grants.

Return to table note ii referrer

Note: Projects may be applicable to more than one category; totals may sum to more than 100%.

4.3.2. Promotion of Canadian Legal Interests Internationally

JPIP-funded involvement in international fora is primarily through the ICCLR, the HCCH, and UNIDROIT. As mentioned, one aspect of the ICCLR’s work is with respect to international cooperation in areas of criminal law reform and criminal justice. Both the HCCH and UNIDROIT are intergovernmental organizations involved in some aspect of developing tools to harmonize international private law. The work of the ULCC can also have an influence beyond Canada when its work and initiatives originally developed for a Canadian context are introduced in international fora, such as through the HCCH. The ULCC can also facilitate the implementation of internationally developed instruments in Canada. In that respect, the ULCC, and Canada’s involvement in the HCCH and UNIDROIT work in a complementary fashion.

The ICCLR participates in numerous international initiatives, including some with the UN. For example, the ICCLR is one of 18 global institutes that are part of the UN’s Crime Prevention and Criminal Justice Programme, which involves conferences, information sharing, and collaborations. As part of that, the ICCLR participates in the UN’s major congress on crime prevention and criminal justice, which occurs every five years. The Centre will also be working on behalf of the Government of Canada and the UN to organize a meeting of experts on using restorative justice principles in criminal matters, which includes developing background information. Other examples of international initiatives that the ICCLR has participated in include collaboration with the Thailand Institute of Justice on a reference tool related to justice indicators and criminal justice reform, and participation in a pan-African consultation on HIV/AIDS in prisons.

Through such participation, the ICCLR is able to represent Canada’s interests at the UN and to other international bodies and other countries, and influence work being done at the international level. While the JPIP funding contributes to the operational aspects of the ICCLR, this support makes it possible for the organization to participate internationally and to attract skilled and experienced associates who take part in national and international efforts on behalf of the ICCLR. Several examples where the ICCLR has influenced international work include a UN model strategy for the protection of all forms of violence against children, the aforementioned reference tool on justice indicators and criminal justice reform that has been distributed worldwide, and a tool that is currently in development for the UN, related to anti-corruption for law enforcement officials. The ICCLR has also supported Canada in introducing principles on the use of restorative justice in criminal matters at the UN Commission on Crime Prevention and Criminal Justice.

With respect to the HCCH and UNIDROIT, Canada’s assessed contribution for its membership in these organizations is provided through the JPIP. However, as described earlier, Canada makes other contributions to these organizations beyond JPIP funding. The Department of Justice’s CAILS is responsible for meeting the federal government’s work requirements as part of their membership; Canadian delegates and representatives participate directly in this work.

Canada is considered an influential member of both the HCCH and UNIDROIT. Canada contributes expertise to the development of new and the operation of existing conventions, and also participates in governance aspects, thus contributing to the direction of the organizations. For example, a Canadian representative was elected as a member of the Governing Council of UNIDROIT in six consecutive elections from 1984 to 2013. As further illustration of how Canada has influenced international work, Canada will also propose new conventions. For example, Canada is credited with making the initial proposal for the work that led to the HCCH Convention of 25 October 1980 on the Civil Aspects of International Child Abduction. In some cases, Canadian legal policy has been able to affect the development of the HCCH and UNIDROIT conventions, resulting in uniformity and predictability between international instruments and existing Canadian legal policies. As noted by key informants, one example of an HCCH convention originating from Canada is the Convention of 29 May 1993 on the Protection of Children and Co-operation in Respect of Intercountry Adoption. An example for UNIDROIT is the 2001 Cape Town Convention on International Interests in Mobile Equipment. Out of that convention came a 2001 Aircraft Protocol that assists international financiers of equipment to recover funds in the event of a loan default. A more recent example is a current first preliminary draft of a fourth Protocol to the Cape Town Convention on International Interests in Mobile Equipment on Matters Specific to Agricultural, Construction and Mining Equipment.

The work of the ULCC also has an international influence. The UNIDROIT convention on mobile equipment, for example, began with the ULCC’s work on personal property security legislation, which was taken forward by Canada and introduced internationally through UNIDROIT. Another example is the ULCC’s work on civil protection orders which Canada brought forward to the HCCH Conference, and where work is now currently being conducted on cross-border civil protection orders at the international level.

While most of these examples of instruments that Canada influenced occurred outside of the evaluation period, they serve to demonstrate how the assessed contributions provided through the JPIP — along with the work conducted by Canadian representatives — can be viewed together as contributing to the promotion of Canadian legal interests internationally. As well, it would be expected that the ongoing work of Canadian delegates in both of these organizations would involve the promotion of Canadian legal interests through participating in the development of new instruments and the operation of existing instruments.

4.4. Performance — Achievement of Ultimate Outcomes

4.4.1. Increased Access to Justice

Prior to making any assessments on the extent to which the JPIP has contributed to increased access to justice, it is useful to first consider what “access to justice” means. The Canadian Forum on Civil Justice states that “access to justice refers broadly to the access that citizens have to dispute resolution tools of justice including but not limited to courts. Effective access to justice does not only refer to reductions in costs, access to lawyers and access to courts; but rather, it is a broad term that refers more generally to the efficaciousness of a justice system in meeting the dispute resolution needs of its citizens” (Canadian Forum on Civil Justice, n.d.). In that sense, one measure of the efficaciousness of Canada’s justice system in the context of the stated objectives of the JPIP is the extent to which information is available about the Canadian justice system. A more informed public is better able to make the appropriate decisions and take the necessary steps for resolving their legal matter, and a more informed justice system is better able to support the public with respect to their justice issues through the appropriate tools, legislation, and decision-making skills.

Concrete measures on how the JPIP directly affects access to justice are not available. As well, some key informants observed that assessing the impact of the funded initiatives on access to justice is difficult, or could be assessed through the information and supports generated. Stakeholders believe that the JPIP-funded initiatives contribute to informing about aspects of the justice system or providing supports to victims and vulnerable populations, thereby contributing to access to justice. Most key informants provided examples of how specific initiatives achieved their objectives, thereby contributing to access to justice. As well, from the applicant survey, almost all approved applicants (95%; n=36) noted that their completed JPIP project(s) were able to almost or fully achieve their objectives, with two-thirds (66%; n=25) of respondents indicating that their project(s) fully achieved their objectives.

PLEI activities, for example, inform the public on a wide range of law and justice issues including criminal, family, and civil law. Key informants believe that providing PLEI materials, tools, and resources assists the public in becoming more informed on justice issues, and their legal matter, and that this assists them in achieving access to justice. In particular, this information is important for newcomers to Canada, who are unfamiliar with the Canadian legal system, and to those individuals who plan or need to self-represent on a litigation matter. The survey found that 70% (n=19) of applicants who reported an awareness of PLEI activities believe that the general public finds PLEI materials and activities useful for addressing their legal needs and concerns to either a great or very great extent; 16% (n=4) indicated the public finds this information useful to some extent. PLEI organizations also work to consult and collaborate with government and other justice service providers to inform them of access to justice issues experienced by their clients and to provide a community perspective on such issues.

Projects that provide tools and resources to address the needs of women experiencing family or relationship violence and Indigenous women and girls vulnerable to violence help these women and girls gain increased knowledge of their rights and how the justice system can assist them, thereby increasing their access to justice.

The work of the ULCC assists in the development of uniform legislation between provinces and territories, which creates increased consistency in how individuals are treated between jurisdictions. More consistency creates a fairer justice system, thereby contributing to increased access to justice.

The conventions and tools developed through the HCCH and UNIDROIT contribute toward informing not only the Canadian justice system but also the justice system on a global level. The application of the conventions and tools developed through these organizations provides residents and companies of member states, including Canada, with increased access to justice in matters with a cross-border legal component. The HCCH, for example, has developed several conventions that assist parents and their children obtain access to justice, such as the Child Abduction Convention. Under this convention, if a child is removed to another country, such as through parental abduction, jurisdiction is provided to the courts of the child’s habitual residence. If that were Canada, the child would be returned to Canada for the Canadian courts to determine what should happen to the child. Several of the HCCH’s conventions work in a complementary fashion. For example, in the case of a parental abduction to another country, the determination of who holds the authority to take protection measures for the child can be achieved through the Child Protection Convention. An example for UNIDROIT is the aforementioned 2001 Aircraft Protocol that facilitates Canadian airlines in arranging foreign financing for aircraft acquisition by providing these foreign investors with protection in the event of a loan default. Another convention that is considered important to many Canadians with ties to other countries is the Convention on International Wills that ensures wills made in one country are recognized in countries party to the convention.

While data are not available to assess if a more equitable representation of Indigenous people in the legal profession exists due to the JPIP, 158 students received funding from 2012–13 to 2015–16 to assist them in their pursuit of an education in law. No data are available on whether these students would or would not have been able to pursue their education in the absence of JPIP funding. From the Indspire case study, the overall body of students who receive funding from the Building Brighter Futures: Bursaries, Scholarships and Awards program (of which the JPIP-funded LSAP Program is a component) has a 93% graduation rate and an 82% employment rate. These figures suggest that JPIP funding is assisting in contributing toward a more equitable representation of Indigenous people in the legal profession, although no statement can be made on the extent to which this is occurring.

PLEI activities offered through the AJA

The evaluation also considered PLEI funding to the territories that is provided through the AJAs with the territories. The PLEI funding provided through these agreements is an important component of access to justice to residents in the territories, as these are usually the only available source of PLEI activities in these locations. Below is a summary of how these PLEI activities have contributed to increased information about the Canadian justice system and, therefore, increased access to justice in the territories.

PLEI services and activities delivered
Examples of how PLEI is provided in the territories include a toll-free lawline, a main office in one of the territory’s larger communities where drop-in services are available, in-person clinics and outreach through visits to communities, the provision of PLEI materials, and, in some cases, duty counsel services for family and child protection related law matters. Clinics or presentations outside the main office centre are primarily by request from communities, or where court workers have been able to identify a need or interest in a community on a specific topic. Outreach or requests for information are often in the areas of child protection, family law, housing, income support, powers of attorney matters, and wills. While some PLEI is provided for criminal matters, much of this type of information and assistance is available through criminal lawyers. Court workers are considered valuable components to the delivery of PLEI. The territories are very large sparsely populated areas with remote and isolated communities. Court workers live in the communities and have established relationships with community residents. Court workers can assist residents with accessing PLEI materials, informing PLEI staff of justice needs of the communities, and providing interpreter services.
Challenges in delivering PLEI services and gaps in services

The large geographical area of the territories populated by small, remote, and poorly serviced communities is one of the main challenges in delivering PLEI services in the territories. The distance to communities that mainly have to be accessed by air transportation makes it difficult and costly to provide in-person services. While one service mentioned striving to ensure a visit to each community to become acquainted with the communities, visits for specific services are largely through community request. For example, elders or community leaders will request a workshop or presentation on an area of law for which there is a perceived need. Alternatively, as mentioned, court workers will identify the interest or need in a community and inform PLEI staff.

Language and cultural barriers are another main challenge. In-person sessions that give communities an opportunity to become acquainted with the PLEI staff are considered helpful for overcoming cultural barriers and assisting residents to increase their comfort level to request information. As well, court workers are considered important for assisting in bridging the cultural gap and providing interpreter services. However, there are limited numbers of court workers available, with each worker having responsibility for a region of the territory. While one key informant noted that some effort has been made to provide materials in Indigenous languages, their territory has many official languages. Furthermore, internet services are poor in many of these communities, and the costs of owning a computer and accessing the internet are prohibitive; as a result, many residents cannot access forms and information online. In addition, many of the communities have no courthouse and few other support services that could provide materials.

Challenges to residents in accessing justice are that there is a lack of lawyers and particularly a lack of lawyers available to take on family and civil law matters. As a result, many individuals must self-represent in such matters, making assistance from PLEI services critical.

4.4.2. Strengthened Canadian Legal Framework

The JPIP contributes to a strengthened legal framework through the positive influence that funded initiatives have on informational, policy, procedural, or legislative changes that provide Canadians with greater access to justice. Each of the various types of JPIP initiatives can be viewed as contributing to this strengthened legal framework in one or more ways. This could be through providing information and supports to clients that inform them of their rights and gain a greater comfort level with the legal framework. Through collaborations and communications, the various funded initiatives share information with other justice stakeholders, including with the Department of Justice, that help to identify gaps, emerging trends, and unmet needs, and that can assist in informing and influencing programming, policy, and legislation, including the greater harmonization of legislation. These actions all lead to a strengthened Canadian legal framework.

Key informants provided several examples of how JPIP-funded projects have contributed to the legal framework:

The work of the ULCC, the HCCH, and UNIDROIT contribute toward harmonized legislation, which further strengthens the legal framework. The work of the ULCC assists in the development of uniform legislation between provinces and territories, which creates increased consistency regarding how individuals are treated between jurisdictions. For example, the Uniform Child Status Act provides basic rules for determining the parentage of children, and which encompasses assisted reproduction. From key informant interviews, Parliament requested that the ULCC work toward uniform legislation on this matter that was not within federal legislative authority, and a number of jurisdictions have adopted the legislation.

The work of both the HCCH and UNIDROIT focusses on the development of conventions and instruments for the unification and harmonization of international private law. Such harmonization enhances the ability of Canadians and Canadian companies affected by cross-border legal matters to achieve access to justice. Furthermore, for both the HCCH and UNIDROIT, Canada has been able to propose and affect development of new international instruments that align well with existing Canadian policy. The HCCH has adopted 38 conventions; Canada is party to four, including the following:

UNIDROIT has conducted close to 70 studies and projects, which have resulted in a series of international instruments, including 13 international conventions and model laws. UNIDROIT instruments that are viewed as benefitting from Canada’s involvement include the following:

When the subject matter of the international instruments falls under provincial authority, the ULCC assists in the adoption and implementation of international instruments in Canada by working towards the adoption of uniform legislation at the provincial level.

4.4.3. Contributing/Constraining Factors to Achieving Outcomes and Unexpected Outcomes

All lines of evidence used for the evaluation show that partnerships and collaborations are viewed as one of the main contributing factors to the success of JPIP-funded initiatives. Key informants mentioned the value of having partnerships and relationships with other organizations, in that these provide access to expertise and other in-kind resources, as well as avenues for networking, disseminating and sharing information, and expanding the reach of the initiative in terms of accessing clients and other relevant stakeholders. Other identified success factors at the initiative level include the following:

Several program-level attributes were also considered to facilitate the achievement of outcomes, including the following:

Any of the identified constraints to achieving outcomes were mainly related to resources. The $1 million reduction of JPIP funding, due to the 2012 DRAP measures, required a reduced scope for the program, as well as a reduction in the number of projects that can be funded. As a result, the Program may not be able to address some emerging priorities. Long-term recipient organizations also observed that their funding levels have not changed for many years, despite increasing demands for service or a greater demand than they had anticipated. Results of the survey of applicants indicates that 63% of respondents with an approved application were satisfied or very satisfied with the amount awarded, 8% were neither satisfied nor unsatisfied and 24% were unsatisfied or very unsatisfied.

The evaluation revealed few unexpected outcomes other than several identified by a few funded recipient key informants, such as a higher than expected level of demand for the funded initiative’s services or outputs, or that the initiative was able to tap into unexpected resources or networks. No unexpected outcomes were reported for the JPIP itself.

4.5. Performance — Efficiency and Economy

The Treasury Board Secretariat’s 2009 Policy on Evaluation defines economy as minimizing the use of resources, which is achieved when the “cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes” (Treasury Board Secretariat, 2009).Footnote 7 The Policy defines efficiency as the use of resources such that the same level of input produces more output or the same level of output is produced by fewer inputs. Efficiency and economy are considered below using available financial information for the JPIP, as well as survey and qualitative data collected through the evaluation process.

4.5.1. Economy

One aspect of economy, and the extent to which expected outcomes are achieved while using the minimum amount of resources required, is how well actual spending matches budgeted amounts. Table 7 shows the annual 2012–13 to 2016–17 budgeted and actual expenditures for JPIP grants and contributions by categories (excluding the assessed contributions to HCCH and UNIDROIT), as well as the overall variance between budgeted and actual expenditures per fiscal year. The largest variance ($808,330) was for Family Violence and MMIWG contribution projects in 2012–13, which was the beginning of a new five-year program cycle. The program had placed a hold on approving new contribution projects while awaiting confirmation that the program would be continued pending the findings of the previous evaluation. According to program staff, unspent JPIP funds in 2013–14 were due to the limited number and quality of proposals and to the timeliness of approvals. Unspent funds in 2014–15 and 2016–17 were largely due to a lengthy approval process for Family Violence projects, which have apparently improved since then.

Lapsed JPIP amounts in subsequent years, as well as some allocated JPIP funds, were shifted to budgets for HCCH and UNIDROIT, which exceed the allocated amounts ($330,000 per year, see Table 2 in each year of the evaluation. These shifts were to account for conversion rates from the Euro to Canadian dollars, as well as a one-time requirement paid in 2013–14 to contribute toward Canada’s share of the HCCH staff pension costs. Budgeted and actual expenditures were very similar each fiscal year for HCCH and UNIDROIT (Table 8). Overall variances between budgeted and actual expenditures over the evaluation period amounted to $1,121,773 for JPIP and $3,550 for HCCH and UNIDROIT, with the former largely due to the $808,330 for family violence and MMIWG projects in 2012–13, as discussed above.

Table 7: JPIP budgeted and actual expenditures, 2012–13 to 2016–17 (in $000)Table note iii
Category 2012–13 2013–14 2014–15 2015–16 2016–17
Budget Actual Budget Actual Budget Actual Budget Actual Budget Actual
Grants
JPIP general - - - - 1,564 1,524 1,524 1,524 1,524 1,524
Family violence - - - - - - 225 40 - -
Named grantsTable note iv 493 493 493 493 - - - - - -
Total grants 493 493 493 493 1,564 1,524 1,749 1,565 1,524 1,524
Contributions
JPIP 283 253 304 304 389 373 376 376 443 437
PLEI2 1,031 1,031 1,031 1,031 - - - - - -
Family violence 314 162 542 542 589 589 454 454 528 513
Family violence – PLEITable note iv 284 - - - - - - - - -
MMIWG 350 8 321 272 393 393 220 216 296 296
ICCLRTable note iv 215 215 215 215 - - - - - -
Total contributions 2,476 1,668 2,413 2,364 1,372 1,356 1,050 1,046 1,267 1,246
Grand total $2,970 $2,161 $2,906 $2,858 $2,936 $2,880 $2,799 $2,611 $2,791 $2,770
Difference ($ not spent)   $808   $49   $55   $188   $21Table note v

Source: Data provided by program.

Table note iii

Not including assessed contributions.

Return to table note iii referrer

Table note iv

Incorporated into JPIP grants after 2013–14.

Return to table note iv referrer

Table note v

There is a pending project, which, if approved, will use most of the remaining amount not spent in 2016–17.

Return to table note v referrer

Table 8: Assessed contributions budgeted and actual expenditures, 2012–13 to 2016–17 (in $000)
Category 2012–13 2013–14 2014–15 2015–16 2016–17
Budget Actual Budget Actual Budget Actual Budget Actual Budget Actual
HCCH 260 260 732 732 284 284 295 295 296 296
UNIDROIT 163 162 193 193 172 172 194 194 181 178
Total $422 $422 $925 $925 $456 $456 $489 $489 $477 $474

Source: Data provided by program.

Note: Main estimates provide $250,000 for HCCH and $80,000 for UNIDROIT. Additional funds to cover the yearly assessment are taken from JPIP resources.

4.5.2. Efficiency

Results achievement

One indicator for efficient resource utilization is the extent to which the resources support the achievement of results which, for JPIP projects, is the extent to which project objectives are being achieved and target audiences are being reached. Almost all survey respondents (95%, n=36) who had an approved JPIP initiative indicated their project had either fully achieved or achieved almost all their objectives, or were expected to do so on completion. As well, most (84%, n=32) respondents reported they were able to reach their intended target audience to a great or very great extent. Furthermore, almost all (84%, n=26) projects reviewed for the file review reported achieving some category of results (e.g., new information or skills, increased awareness, best practices), although there was no measure available on the extent to which they achieved their expected results.

Leveraging resources

This reported level of achievement, coupled with the relatively modest amount of funding available to allocate to a wide range of topics and objectives, suggests that JPIP funding has been able to make effective use of the available resources to achieve results. Some key informants attribute this success to the initiatives’ ability to draw on other sources of cash and in-kind funding that augment and build on the funds available through the JPIP. In particular, as noted previously, the in-kind resources in the form of expertise and knowledge are viewed as making substantial contributions to initiatives that assist them in achieving positive results. Some key informants commented specifically on the JPIP initiatives’ ability to make significant accomplishments with a relatively small amount of funding, due to their ability to leverage funding and in-kind contributions from other sources. Key informants noted that the JPIP funding adds credibility to the initiative and assists in leveraging other resources. For example, one PLEI organization estimated that they are able to leverage $1.56 in cash funds for every JPIP dollar, and observed that some of these funds are contingent on receiving JPIP funding. A few department key informants indicated that, while JPIP funding represents a relatively small amount of federal money, it is used to bolster or add to the budgets and activities of well-placed and carefully screened initiatives that align with the priorities of the program. As mentioned previously, under half of the partners involved in funded projects provide cash funds (45%, n=13 of approved applicants responding to the survey; and 36%, n=10 of files reviewed), but the majority also provide in-kind contributions (79%, n=23 in the survey; 88%, n=23 in the file review). In-kind contributions are primarily in the form of human resources, expertise, and advice.

Program management

Several measures have been put in place for the efficient management of the JPIP, some of which have been discussed in previous sections. Most key informants had positive comments about the application process and believed the process has improved over the past several years. Department stakeholders view the use of standardized funding applications, along with a formalized method and set of criteria for reviewing proposals (that includes using application reviewers who have content experience), as an effective and efficient approach for allocating available JPIP funds. Recipient key informants reported that the funding application was user-friendly, straightforward, “very manageable,” and asked good questions. They also appreciated that applications could be tailored to address the specific program objective that the project is intending to address. Survey respondents, including applicants with approved and/or unapproved applications, also showed a high level of satisfaction with the application process, with close to three-quarters and more being satisfied or very satisfied with most elements of the application process, as follows:

Several recipient key informants commented on the responsiveness of JPIP program staff, for example, in addressing questions. They also appreciated that they were able to deal with the same program staff person on an ongoing basis. Moreover, 72% of survey respondents were also satisfied or very satisfied with the ease of interaction with JPIP representatives.

A few key informants believe the application process is still too cumbersome or that there is still a level of uncertainty within the current process. For example, delays in the application process leave organizations uncertain about whether their projects will go forward; alternatively, if their project is approved, they are uncertain when the funding will be released. This situation makes it difficult for organizations to plan for staffing needs, and it can leave non-profit organizations — which operate with little surplus — cash strapped. From the survey of applicants, the timeliness of receiving approval decisions is the element of applications that had the lowest proportion of applicants who were satisfied/very satisfied, although this still represented 65% of respondents. Meanwhile, 26% indicated they were neither satisfied nor dissatisfied, or that they were dissatisfied/very dissatisfied.

The evaluation found a high level of satisfaction also exists with the management of JPIP projects, which can be viewed as another indicator of efficient and effective management. From the survey of applicants, most respondents with an approved application were satisfied with various elements of program management related to reporting requirements and funding period and/or payments, as follows:

Looking at the performance against departmental service standards with respect to applications and payments of funds, the JPIP showed a high level of meeting departmental service standards, with 80% to 100% of files meeting the standard from 2011-12 to 2015-16 for each of the standards related to acknowledgement, funding decision, and payments (Table 9).Footnote 8 One exception is that only 67% of the files sampled met the payment standard in 2014–15.

Table 9: JPIP performance against departmental service standards for G&C, by type of service standard 2011–12 to 2015–16
Fiscal year Acknowledgement Funding decision PaymentTable note vi
Files meeting standard % meeting standard Files meeting standard % meeting standard Files meeting standard % meeting standard
2011–12 41 (of 50) 82% 43 (of 50) 86% n/a n/a
2012–13 57 (of 63) 90% 53 (of 63) 84% 13 (of 13) 100%
2013–14 30 (of 36) 83% 29 (of 36) 81% 4 (of 4) 100%
2014–15 29 (of 35) 83% 28 (of 35) 80% 10 (of 15) 67%
2015–16 35 (of 37) 95% 33 (of 37) 89% 9 (of 10) 90%

Source: Data provided by program.

Table note vi

Based on a random sample of payments (numbers in brackets for the payment column refer to number sampled).

Return to table note vi referrer

n/a – not available.

A few recipient organizations commented that the reporting process has improved over the last several years. The program uses standard project summary reporting templates, with one template for named grants and one for other recipient types. The file review also considered the extent that recipient reporting reflects the requirements of the reporting templates. Consistent means of reporting contributes to a more efficient and consistent process for assessing reports for the achievement of results. Almost all (n=27; 87%) of the funding recipients followed the JPIP reporting templates with the exception of named grant projects; while most recipients of named grants did submit final reports, they were very brief and did not include the required information to address the questions in the template. As a result, the information collected on named grants is very limited. Other funding recipient types provided very detailed summary reports that followed the JPIP template. Although the template appeared to evolve over the course of the evaluation period, the information gathered was very similar.

Key informants identified several practices that they believed contributed to more efficient and effective reporting, including providing recipients with reminders when reporting is due, reducing the number of required reports (a few recipient organizations reported that they were only required to report annually), and having the ability to use reporting that had already been prepared for other funding sources. A few other key informants reported challenges with the reporting process, such as having insufficient time to submit reports, and not being granted an extension to provide additional time.

With respect to program management, another aspect of efficiency is the proportion of actual total expenditures that are comprised of administrative costs, where administrative costs consist of salaries and operations and maintenance (O&M) expenditures. Table 10 shows actual expenditures for salaries, O&M, and grants and contributions (G&C), as well as the proportion of total expenditures represented by administrative costs (salaries plus O&M). JPIP administrative expenditures are low relative to total expenditures, ranging from 1% to 5% over the evaluation period, with the increase mainly being due to an annual increase in salary expenditures. JPIP staff resources are required for developing application and reporting templates, accepting and screening applications, making funding decisions, dispersing funds, monitoring recipients’ adherence to their reporting requirements, reviewing reports, and interacting with applicants and funding recipients, among other JPIP program management responsibilities. Key informants were not able to identify more cost-effective approaches to achieving the program objectives.

Table 10: Percent of total JPIP actual expenditures as administrative costs, 2012–13 to 2016–17
Expenditures 2012–13 2013–14 2014–15 2015–16 2016–17
Salaries $26,803 $51,209 $68,657 $126,249 $167,250
O&M - $3,384 $4,846 $2,223 $1,850
G&C $2,582,998 $3,782,400 $3,336,776 $3,099,795 $3,244,195
Total expenditures $2,609,801 $3,836,993 $3,410,279 $3,228,267 $3,413,138
Admin = salaries + O&M $26,803 $54,593 $73,503 $128,472 $169,100
% Administrative costs (5/4) 1% 1% 2% 4% 5%

Source: Data provided by program.