Aboriginal Courtwork Program, Formative Evaluation

2. Findings – Implementation and Delivery

This chapter presents the findings on the implementation and delivery of the program, including how the program has been implemented, the effectiveness of the federal contribution agreements, the Tripartite Working Group (TWG) and the TWG Steering Committee, some of the more immediate outcomes of the program, and strengths and weaknesses of the program.

2.1. Program Delivery

This section presents the different models used to deliver the ACW program to provide contextual information for this evaluation. As each P/T government is responsible for determining the most appropriate service delivery model, there are several distinct service delivery models:

Several jurisdictions have additional oversight mechanisms, such as an advisory or steering committee. 

There are several features common to courtworkers, according to the survey results. Most courtworkers are experienced Aboriginal persons employed full-time working out of an office and travelling regularly as part of the job. Specific courtworker characteristics include:

2.2. Federal ACW Contribution Agreements

The latest federal contribution agreements with the provinces are in effect April 1, 2004 to March 31, 2008 and reflect the changes in Treasury Board's Policy on Transfer Payments. The agreements outline:

The most recent Access to Justice Services Agreements between Canada and the territories extended the provisions of the 2003-2006 agreements for one year, ending March 31, 2007 and new agreements are being negotiated. The Agreements integrate federal funding for Aboriginal Courtwork as well as Legal Aid and Public Legal Education services under one contribution agreement with each territory.  In return for this funding, the territories agree to maintain certain minimum levels of service in providing these justice-related services to eligible persons.  The Agreements accommodate an integrated service delivery model by giving the territories the flexibility to allocate the federal funds among the eligible services as they see fit.

The agreements define the minimum levels of service for each component and the territories undertake to maintain a service delivery system that meets or exceeds these levels. Each territorial government agrees to provide the federal Department of Justice with information such as:

DOJ interviewees stated that the requirements were adequate and necessary to meet Treasury Board's accountability requirements, but had concerns with the P/T capacity to meet the accountability requirements.

Almost all P/T officials felt that the P/T contribution agreements would help them meet their objectives; two indicated the impacts were neutral to negative.  Half the P/T officials expressed concern that meeting accountability requirements could take resources away from services.

Conclusions:  The contribution agreements with the provinces and the AJAs with the territories are flexible enough to meet the jurisdictions' objectives and priorities.  Half of the P/T officials are concerned that meeting accountability requirements may take away from the resources available for services.

2.3. Timeliness of Federal Payments

The contribution agreements with the provinces state that federal payments are made to the provinces twice yearly, on September 30 and March 31. The first interim payment to the provinces, due on September 30, is for up to 50 percent of the maximum federal allocation.  The second interim DOJ payment is due on March 31, and is for up to 90 percent of the federal total contribution based on: i) the budget component of the workplan referred to in section 7 of the contribution agreement or ii) the interim financial report referred to in section 8 of the contribution agreement or iii) the amended budget submitted under section 9, whichever is the lesser and the first interim payment. As per section 27 of the contribution agreements, a 10 percent holdback is released once the financial claim statement for the previous year and any necessary financial adjustments have been made. In the territories, the Department provides quarterly progress payments.

DOJ interviewees felt that federal resources were flowing reasonably well to the P/Ts and that delays result from not receiving specific claim information from the P/Ts. However, they acknowledged that new staff and new reporting requirements may have contributed to delays. P/T officials had mixed views. Although four P/T officials felt the federal resources were timely, three said they were not timely but acknowledged the P/T was at least partly responsible.  One province felt that resources were not timely and that the federal government was the cause of the delays.

At least one province waits until it has received the federal contribution before funding the SDAs, meaning a potentially significant impact on that province's SDAs if federal resources are delayed.  At least two provinces provide cash advances to their SDAs so that the SDAs are not affected by delays in the federal funds.

Conclusions: There were mixed views on whether federal resources flow to the P/Ts in a reasonably timely fashion. Changes to the reporting requirements and the arrival of new DOJ staff may have contributed to delays.  Further delays may be the result of the Department holding back 10 percent of the final payment until after a final review of the claim, as per section 27 of the federal contribution agreements with the provinces.

2.4. Federal Contribution Agreements Reporting Requirements

2.4.1. Federal, Provincial Contribution Agreements

Under the Federal, Provincial Contribution Agreements, the provinces must file a number of reports, including a workplan, interim financial reports, financial claims statement and performance report.

The workplan, a new requirement of the contribution agreements, is due every July 1, and must include:

In accordance with the provincial contribution agreements, each SDA submits a workplan to the province. The province submits a provincial workplan to the Department of Justice. SDAs in Alberta provide five separate workplans to the province who in turn submit them to the Department of Justice.

As of the July 1, 2006 deadline, three of the eight provincial workplans had been received. Six of the seven provincial officials indicated problems meeting the July 1 deadline, citing lack of capacity and conflicting priorities in the SDAs.  Another province noted that the July 1 deadline does not allow enough time for the reflection and analysis that is needed for effective planning. Although DOJ program officials recognize that the July 1 deadline is not ideal for smaller organizations, it says it needs this information to be able to release the first payment by September 30.

The interim financial report, due by December 31 of each year, provides the total eligible expenditures to September 30, and forecasts expenditures to March 31.  The Department must receive the interim financial report to release the second interim payment by March 31. The document review indicated that the interim financial reports are often not completed on time.  Of the two provinces that indicated problems with the deadline, one cited limited capacity in the SDAs and the other that accurate forecasting could not be done until February.

Both the financial claim statement and the performance report cover the preceding fiscal year.  The claim statement enables the Department to verify the full amount of eligible expenditures claimed and verify that the province complies with all provisions of the contribution agreements.  The annual performance report provides information against the workplan and includes:

The Department phased in performance reporting requirements for the provinces. The provinces were required to implement core performance measures 1 to 6 and 10 to 12 in 2005-06 and submit the information by December 31, 2006, and to implement 7 to 9 starting in 2006-07.[2]

DOJ interviewees indicated that it is unusual for the provinces to submit all required financial information and that the financial claim statements are sometimes late -- sometimes up to a year late.

On the performance report requirements, DOJ program officials indicated that it was too early to tell whether P/Ts would be able to meet the December 31 requirement for core performance measures. This was consistent with the views expressed by provincial respondents. In some provinces, SDAs were either in the process of conducting pilots or awaiting the results of pilots. Some jurisdictions were awaiting the outcome of discussions on outstanding performance measurement issues that were to take place at the November 7, 2006 meeting of the TWG. Most provinces foresee problems reporting on core performance measures 7 to 12, which are related to the client and judiciary surveys.[3] More detailed findings relating to the performance measurement strategy are presented in the following chapter.

Conclusions: Provinces' capacity challenges and other priorities delay completing their workplans, which in turn delay the first federal payments (up to 50 percent of the total) that would normally flow on September 30.  Although most provinces indicated no problems submitting interim financial reports and financial claims statements by December 31, the document review indicated that most interim financial reports are late. 

2.4.2. Territorial Access to Justice Services Agreements

Under the Access to Justice Services Agreements, several reports must be filed by October 31 each year, including financial claims statements, Program Reports including certain statistics and a narrative on activities. In addition, in 2006-07, DOJ program officials asked the territories to start collecting data on core performance measures 1 to 6.

Financial claims statements detail how quarterly progress payments were used the preceding fiscal year. A document review revealed that only one territory met the deadline, the other one provided partial information and the third one was delayed.

The Program Report provides detail on the number of Aboriginal adults or young persons served, the types of services provided, the courtwork areas, the names of SDAs, and the number of courtworkers in each area or clinic. Two territories indicated that providing this information has been challenging because of the limited capacity of the SDAs.

The Program Report also discusses the activities undertaken and provides SDA statistics on services related to federal contributions. 

Two territorial officials are comfortable with the narrative but did not know whether there would be problems collecting the required performance information.  The third territory is not providing the narrative information or the detailed data for the Delivery of Services Report .

Federal program officials are aware of the capacity limitations that affect the territories' ability to meet the reporting requirements and therefore try to be as flexible as possible. At the same time, they believe that performance information is necessary to demonstrate the value of the program.

Conclusions: The territories have experienced problems meeting the reporting requirements, particularly the deadline for the financial claims statement, and the requirement to provide statistical information. The limited capacity of SDAs in the North makes it challenging to collect the required statistical information.

2.4.3. Audit Requirements

The Risk Based Audit Framework (RBAF) requires that a federal audit be completed with each province at least once between 2003 and 2008 and with at least one territory during the term of the Access to Justice Services Agreements, which is 2003 to 2007.

DOJ program officials indicated that the Department is on track to meet the federal audit requirement. Since 2003, audits have been completed in Manitoba, Quebec, Newfoundland and Labrador, Nova Scotia and NWT. The remaining provinces will be audited by March 31, 2008.

Conclusion: The audit requirements are being implemented by the Department and are on track.