The Effectiveness of Restorative Justice Practices: A Meta-Analysis
4. Results
Twenty-two unique studies that examined the effectiveness of 35 individual restorative justice programs generated 66 effect sizes. A summary of specific study characteristics is presented in Table 2. The frequencies presented in Table 2 are based upon the 35 programs with the exception of the type of outcome measure and study source, which are based upon 66 effect sizes and 22 unique studies respectively.
| VARIABLE | FREQUENCY (%) |
|---|---|
| Restorative Justice Model | |
| Conferencing | 8 (22.9) |
| Victim−offender mediation | 27 (77.1) |
| Entry Point | |
| Pre-charge | 7 (20.0) |
| Post-charge | 6 (17.1) |
| Pre-sentence | 1 (2.9) |
| Post-sentence | 1 (2.9) |
| Mixed | 20 (57.1) |
| Outcome Measure | |
| Victim satisfaction | 13 (19.7) |
| Offender satisfaction | 13 (19.7) |
| Restitution compliance | 8 (12.1) |
| Recidivism | 32 (48.5) |
| Gender | |
| Predominantly male (>70%) | 33 (94.3) |
| Mixed | 2 (5.7) |
| Ethnicity | |
| Predominantly Caucasian (>70%) | 14 (40.0) |
| Other | 2 (5.7) |
| Mixed/unspecified | 19 (54.3) |
| Age Group | |
| Adult | 9 (25.7) |
| Youth | 26 (74.3) |
| Study Source | |
| Published | 10 (45.5) |
| Unpublished | 12 (54.5) |
The vast majority of the effect sizes were derived from programs that targeted predominantly male (94%), young (74%) offenders. Interestingly, a large proportion of the effect sizes were drawn from studies that were not published in peer-reviewed academic journals (55%), which, as discussed previously, is typically not the case in meta-analytic work.
As shown in Table 2, studies commonly included one or more of the following outcome measures: victim satisfaction, offender satisfaction, restitution compliance and recidivism reduction. Each of these issues will be discussed accordingly in the following subsections.
4.1 Victim Satisfaction
The overall mean effect size for the 13 tests of treatment that explored the impact of restorative justice programming on victim satisfaction was +0.19 (SD=.18) with a 95 percent confidence of +0.30 to +0.08 (see Figure 1). Although the effect sizes ranged from +0.44 to -0.19, the latter was the only negative value found in the distribution. In other words, participation in a restorative justice program resulted in higher victim satisfaction ratings when compared to a comparison group in all but one of the 13 programs examined.
Figure 1. Distribution of Effect Size Estimates (VICTIM SATISFACTION)
It should be noted that the one negative result was found in the only program that operated at the post-sentence (or corrections) entry point. Compared to victims who participated in the traditional justice system, victims who participated in restorative processes were significantly more satisfied (t (12) = 3.89, p < 0.01).
Given the relatively wide range of effect sizes, additional analyses were conducted to explore whether characteristics of the study sample or methodological considerations could explain this variability. Initially, we had hoped to explore a relatively large number of potential moderators, such as gender, ethnicity, criminal history, offence type, etc. The relative homogeneity of the offenders used in the studies, however, as well as the large amount of missing data, rendered many of these analyses untenable. On the other hand, this homogeneity increases our confidence in the generalizability of the findings to this population. Therefore, the analyses reported below focus on six factors: random assignment, offender age, publication source, restorative justice model, entry point and control/comparison group type.
As indicated previously, we used two coding methods for capturing information on the control/comparison group used in the studies. First, we combined multiple control/comparison groups from the same study to calculate a single effect size; second, we calculated individual effect sizes for each control/comparison group. For the comparison of control groups in Table 3 (and subsequent Tables), we used the latter coding technique. This allowed us to compare restorative justice programs with individuals who were referred to a restorative justice program but refused participation versus all other control group types (i.e. probation, court, prison).
The mean effect sizes for each value of the moderator variable, along with their corresponding significance tests, are presented in Table 3. Although these variables did not yield significant between-group differences, studies using non-randomized comparisons and studies published in academic journals displayed a higher mean effect size than their counterparts. In addition, VOM models tended to display higher victim satisfaction rates than conferencing models when compared to the non-restorative approaches. The lack of significance between moderating variables might be due to the low number of effect sizes.
| VARIABLE | N | EFFECT SIZE Unweighted | T value (p) | |
|---|---|---|---|---|
| AGE | Youth | 8 | .20 | -.09 |
| Adult | 5 | .19 | (ns) | |
| RANDOM ASSIGNMENT | Yes | 3 | .14 | .94 |
| No | 10 | .21 | (ns) | |
| STUDY SOURCE | Published | 3 | .30 | -1.42 |
| Unpublished | 10 | .16 | (ns) | |
| ENTRY POINT (earliest) | Pre-charge Other entry points | 6 6 | .16 .18 | -.31 (ns) |
| ENTRY POINT (latest) | Pre-charge Other entry points | 3 9 | .15 .18 | -.32 (ns) |
| MODEL CONTROL GROUP | Conferencing | 4 | .14 | .94 |
| V−O mediation | 9 | .21 | (ns) | |
| Non-participation Other control | 9 6 | .22 .18 | .62 (ns) | |
v–o = victim–offender
ns = not significant
4.2 Offender Satisfaction
The overall mean effect size for the 13 tests of the impact of restorative justice programming on offender satisfaction was +0.10 (SD=.28), while the effect sizes ranged from +0.31 to -0.71 (see Figure 2). While offenders who participated in restorative justice programs displayed higher satisfaction with the process than their comparisons, the one-sample t-test indicated that this difference was not statistically significant. Since the 95 percent confidence interval included zero, this further decreased our confidence that these programs have had any discernible impact on offender satisfaction.
This conclusion is mitigated, however, by the finding that although there were two negative effect sizes contributing to this result, the -0.71 was a clear outlier. Moreover, given that the sample size used in this outlier study was extremely small (n=7), we removed the study from the analysis. This increased the mean effect size to +0.17 and substantially reduced the standard deviation (SD=.13). Furthermore, and more importantly, removal of this study resulted in the confidence interval not including zero, thus suggesting that these programs have a moderate to weak positive impact on offender satisfaction. The difference in offender satisfaction between restorative and non-restorative participation also becomes significant (t (11) = 4.52, p < 0.01). Interestingly, the -.71 effect size was drawn from the same post-sentence entry point program as the only negative victim satisfaction effect size.
Figure 2. Distribution of Effect Size Estimates (OFFENDER SATISFACTION*)
* Does not include -.71 outlier
To account for this substantial discrepancy, we presented the results both with the outlier (Table 4.1) and without the outlier (Table 4.2). Given this extreme outlier, interpreting these results was inappropriate as the conclusions would be drastically different in each case based upon the inclusion or exclusion of one value.
| VARIABLE | N | EFFECT SIZE Unweighted | T Value (p) | |
|---|---|---|---|---|
| AGE | Youth | 8 | .15 | -.47 |
| Adult | 5 | .05 | (ns) | |
| RANDOM ASSIGNMENT | Yes | 4 | .09 | .17 |
| No | 9 | .12 | (ns) | |
| STUDY SOURCE | Published | 1 | .08 | .00 |
| Unpublished | 12 | .11 | (ns) | |
| ENTRY POINT (earliest) | Pre-charge Other entry points | 8 5 | .15 .03 | .17 (ns) |
| ENTRY POINT (latest) | Pre-charge Other entry points | 5 8 | .09 .11 | -.13 (ns) |
| MODEL CONTROL GROUP | Conferencing | 6 | .11 | .12 |
| V−O mediation | 7 | .09 | (ns) | |
| Non-participation Other control | 8 7 | .10 .10 | .02 (ns) | |
v–o = victim–offender
ns = not significant
| VARIABLE | N | EFFECT SIZE Unweighted | T Value (p) | |
|---|---|---|---|---|
| AGE | Youth | 8 | .15 | 1.71 |
| Adult | 4 | .22 | (ns) | |
| RANDOM ASSIGNMENT | Yes | 4 | .09 | 1.33 |
| No | 9 | .21 | (ns) | |
| STUDY SOURCE | Published | 1 | .08 | .00 |
| Unpublished | 11 | .18 | (ns) | |
| ENTRY POINT (earliest) | Pre-charge Other entry points | 8 4 | .15 .22 | -1.2 (ns) |
| ENTRY POINT (latest) | Pre-charge Other entry points | 5 7 | .09 .23 | -1.79 (ns) |
| MODEL CONTROL GROUP | Conferencing | 6 | .11 | .12 |
| V−O mediation | 6 | .23 | (ns) | |
| Non-participation Other control | 7 7 | .21 .10 | 1.93 (ns) | |
v–o = victim–offender
ns = not significant
4.3 Restitution Compliance
One of the potential advantages of a restorative justice approach is that it could be more effective in ensuring offender compliance with restitution agreements. This would be a significant contribution as the victims would have a greater likelihood of receiving compensation for the harm caused by the criminal activity and the offenders would be actively accepting responsibility. The results of the studies that included a measure of restitution compliance are reported below.
Only eight studies examined the impact of restorative justice programming on restitution compliance. Although this number may seem small, it may have been, in part, due to the inclusion criteria for this meta-analysis (i.e. the study used a comparison group). Overall, the mean effect size of +0.33 (SD=.24) was quite high, indicating that offenders who participated in restorative justice programs tended to have substantially higher compliance rates than offenders exposed to other arrangements. Furthermore, there was a great deal of variability in the effect sizes found in these studies, with values ranging from +0.63 to -0.02 (see Figure 3). Compared to the comparison/control groups not participating in a restorative justice program, offenders in the treatment groups were significantly more likely to complete restitution agreements (t (7) = 3.87, p < 0.01).
The small number of effect sizes (k=8) made conducting the moderator analyses inappropriate.
Figure 3. Distribution of Effect Size Estimates (RESTITUTION COMPLIANCE)
4.4 Recidivism
Arguably, one of the most important outcome variables for any form of criminal justice intervention is recidivism. Considerable public and institutional support for correctional programming rests on its ability to reduce future criminal activity. Therefore, the ability of restorative justice programs to reduce recidivism was felt to be particularly important for the present meta-analysis.
The overall mean effect size for the 32 tests that examined the effectiveness of restorative justice programming in reducing offender recidivism was +0.07 (SD=.13) with a 95 percent confidence interval of +0.12 to +0.02. Although the effect sizes ranged from +0.38 to -0.23, more than two thirds of the effect sizes were positive (72%). In other words, restorative justice programs, on average, yielded reductions in recidivism compared to non-restorative approaches to criminal behaviour. In fact, compared to the comparison/control groups that did not participate in a restorative justice program, offenders in the treatment groups were significantly more successful during the follow-up periods (t (31) = 2.88, p < 0.01).
One of the major areas of debate in the correctional treatment literature is the impact of different methodological and demographic characteristics on program effectiveness. Subsequently, we conducted moderator analyses to explore the impacts of several
variables on recidivism reduction. The results of these analyses are presented in Table 5 and are discussed below.
Figure 4. Distribution of Effect Size Estimates (RECIDIVISM)
| VARIABLE | N | EFFECT SIZE Unweighted | T value (p) | |
|---|---|---|---|---|
| AGE | Youth | 24 | .06 | .60 |
| Adult | 8 | .10 | (ns) | |
| RANDOM ASSIGNMENT | Yes | 8 | .06 | .33 |
| No | 24 | .07 | (ns) | |
| STUDY SOURCE | Published | 12 | .12 | -1.73 |
| Unpublished | 20 | .04 | (ns) | |
| ENTRY POINT (earliest) ENTRY POINT (latest) | Pre-charge Other entry points | 16 16 | .07 .06 | .17 (ns) |
| Pre-charge Other entry points | 8 24 | .06 .07 | -.16 (ns) | |
| MODEL | Conferencing | 8 | .06 | .22 |
| V−O mediation | 24 | .07 | (ns) | |
| CONTROL TYPE | Non-participation Other control | 9 31 | .02 .12 | -1.73 (ns) |
v–o= victim–offender
ns = not significant
As stated previously, one of the primary criticisms lodged against meta-analysis is its predominant reliance on published studies and the subsequent problem of potential publication bias. This issue has been addressed in the present meta-analysis by conducting searches of governmental and non-governmental reports, graduate theses and dissertations and by directly contacting researchers active in the field for unpublished research. Nevertheless, we directly tested the impact of publication source on effect size. Inspection of Table 5 reveals that the mean effect size for studies from published sources was somewhat higher than the mean effect size found in unpublished sources. This, in combination with the reported difference above in victim satisfaction rates, lends support to the “file-drawer” problem in meta-analytic work.
- Date modified: