Department of Justice Canada Client Feedback Survey

Interpreting Results

The survey results represent estimates of client population perceptions of service delivery. Prior to the launch of the survey, the Department established a target of 8.0 on the 10-point scale for each of the items for which client feedback was sought. These targets were identified in the absence of any pre-existing benchmarks, but were developed with the goal of setting realistic and attainable targets that were not too easy to attain.

Throughout the remainder of this report a colour coding scheme / legend for the presentation of results has been adopted (see table below). This provides a visual means for portraying the extent to which departmental targets have been met.

Legend

Strong
– surpassed targets (mean ratings of 8.4 to 10)

Positive
– met targets (mean ratings of 7.9 to 8.3)

Moderate
– slightly below targets (mean ratings of 7.3 to 7.8)

Opportunities for Improvement
– targets not met (mean ratings of 6.5 to 7.2)

Attention Required
– significantly below targets (mean ratings less than 6.5)

In reviewing the results presented throughout the remainder of this report, there is an important caveat to bear in mind, namely the calculated margins of error. The magnitude of the margin of error is generally affected by the extent of variability in respondent feedback and by the overall size of the respondent group.

There are two key elements to calculating the margins of error from survey findings. First, there is the confidence level which, in the most simplistic terms, refers to the extent to which we believe the same results would be obtained if the survey were administered repeatedly. For the purposes of the Department of Justice Client Feedback Survey, a 95% confidence level was adopted for calculating results.

Second and more importantly there is the confidence interval, which refers to the range in which the results will fall if the measurements are repeatedly taken. For the purposes of this project, we recommend caution in interpreting any results that have a calculated margin of error greater than ±0.4. Note that large margins of error may also represent wide variation in the opinions of respondents, indicating a large disparity between the satisfied and the unsatisfied groups^{Footnote 3}.

Footnotes

Footnote 3

While confidence intervals traditionally reflect the use of sampling methodology, the DoJ Feedback Survey used a census approach in which invitations to participate in the survey were sent to all potential users of services. In this case, confidence intervals account for variability related to nonresponse. Had all service users responded to the survey, there would be no variability, as all opinions would be accounted for. In the calculation of the confidence interval, we are assuming that nonresponse is independent of respondent characteristics but is affected by use of legal services (i.e. actual service users are more likely to answer the questionnaire). It is a reasonable assumption that a relatively large proportion of non-respondents are non-users. The Finite Population Correction Factor (FPCF) has been applied in the calculation of the margin of error in order to take the size of the total number of potential users into account; otherwise the margins of error would be overstated.

Return to footnote 3 referrer

Date modified:: 2022-05-11

Language selection

Search

Phone Scam Warning

Department of Justice Canada Client Feedback Survey

Interpreting Results

Legend