Legislative Services Branch Evaluation
Appendix A: Evaluation Matrix
| Evaluation Issue | Question | Performance Indicator | Measures | Data Sources/ Methods |
|---|---|---|---|---|
| Continued Need for the LSB | 1.1 To what extent is the LSB responsive to the legislative drafting needs of the Government of Canada? | Extent to which the LSB is relevant to the legislative drafting needs of the Government of Canada | Comparison of ongoing Government of Canada (GoC) legislative drafting requirements to the types of services offered by the LSB Perceived relevance of LSB services to the needs of the GoC |
Document review LSB departmental clients/ survey, interviews Justice legal staff/ interviews PCO, TBS policy makers/interviews LSB staff/interviews, survey LSB client case files/analysis |
| 1.2 Has the demand for legislative services changed and how? | Nature of/extent to which demand for legislative services has changed | Trends in client requests for legislative services by type of service Changes in types of requests New or special requests |
LSB departmental clients/ survey, interviews Justice legal staff/ interviews PCO, TBS policy makers/interviews LSB staff/interviews, survey LSB client case files/analysis LSB administrative records/analysis |
|
| Alignment with Government Priorities | 1.3 To what extent are the activities of the LSB aligned with: 1. GoC priorities? and 2. strategic outcomes of Justice? | Extent to which LSB activities are aligned with GoC priorities | Consistency between LSB activities and legislated responsibilities Changes made to LSB activities to maintain alignment with changes to GoC priorities |
Document review LSB managers/ interviews Justice legal staff/ interviews PCO, TBS policy makers/interviews |
| Extent to which LSB activities are aligned with the strategic outcomes of Justice | Consistency between LSB activities and Justice strategic outcomes | Document review LSB managers/ interviews Justice legal staff/ interviews |
||
| Alignment with Federal Roles and Responsibilities | 1.4 To what extent do the activities of the LSB align with federal roles and responsibilities? | Extent to which LSB activities are aligned with the federal government’s role and responsibilities. | Alignment of the responsibilities with the legislated authority of the GoC (and outside of P/T legislated authority) Alignment of staff roles/responsibilities with the responsibilities of the LSB |
Document review |
Performance
Achievement of Expected Outcomes
| Evaluation Issue | Question | Performance Indicator | Measures | Data Sources/ Methods |
|---|---|---|---|---|
Effectiveness |
2.1 To what extent has the LSB contributed to the creation of legislation that is responsive to federal government policy directions? | Responsiveness of the LSB to current GoC needs and policy directions | Ability to respond to needs of the GoC Harmonization files completed Policy developed related to legislative and regulatory processes (with PCO and TBS) Streamlined regulations Services provided to the PCO |
LSB staff/interviews PCO, TBS policy makers/interviews LSB administrative records LSB client case files/analysis Case studies |
| Responsiveness of the LSB to current client needs | Ability to respond to requests/needs of Justice Ability to respond to requests/needs of clients Drafted legislation completed Drafted regulations completed |
LSB departmental clients/ interviews, survey PCO, TBS policy makers/ interviews Justice legal staff/ interviews LSB staff/interviews, survey LSB administrative records LSB client case files/analysis Case studies |
||
| Provision of legislative services, by type of services provided | Number/type/nature of legislative products created in response to federal government policy directions by type of service provided
|
LSB administrative records/ analysis PCO, TBS policy makers/interviews |
||
| Extent to which LSB clients are satisfied with the responsiveness of LSB services, by type of service provided | Client satisfaction with responsiveness of LSB staff by type of service
|
LSB departmental clients/ interviews, survey PCO, TBS policy makers/interviews Case studies |
||
| Extent to which the LSB has contributed to the creation of legislative products that express intended federal government policy directions | Client experience working with LSB Alignment of requests with services provided and results Contribution of legislative products created to policy directions |
LSB departmental clients/ interviews LSB client case files/analysis Case studies |
||
| 2.2 To what extent has the LSB contributed to the creation of federal legislation that is accessible to Canadians? | Extent to which federal legislation drafted is clear and consistent in both official languages, while respecting the civil law and common law in private matters | Type and adequacy of quality assurance protocols/procedures, etc. in place to ensure clarity and consistency of language, and civil and common law traditions Capability of staff (e.g., credentials, training received) to draft legislation to meet bijural and bilingual requirements Clarity of language and consistency between French and English |
LSB staff/interviews, survey LSB staff/PSES Documentation/review LSB client case files/analysis LSB departmental clients/ interviews, survey Other legal professionals/ interviews |
|
| Nature/extent of use of Justice Laws website | Justice Laws website usage
|
Website statistics/ analysis | ||
| Extent to which legislation is available/accessible in appropriate formats | Format in which federal legislative products are provided Ease of accessing legislation in the formats available Satisfaction with formats available |
Publication/dissemination schedules/analysis LSB departmental clients/ interviews, survey Other legal professionals/interviews |
||
| 2.3 To what extent has the LSB enhanced awareness/ understanding of federal legislation, legislative principles, processes and options by federal departments and agencies? | Level of awareness/ understanding within the federal government of federal legislation, legislative principles, processes and options by target audience Footnote 66 | Client awareness of relevant federal legislation, legislative principles, processes and options Satisfaction with training received (content, clarity and relevance) |
Training files/feedback results/analysis LSB departmental clients/ interviews, survey |
|
| 2.4 To what extent has the LSB enhanced its capacity (skills, tools, information) to deliver consistent federal legal and legislative products? | Evidence of activities undertaken to enhance capacity in the LSB to deliver consistent products to its clients and to the GoC | Number/type/nature of quality improvement and capacity building efforts planned/undertaken by the LSB Number/type/nature of capacity enhancing tools developed/applied (e.g., Charter check list) |
Document review LSB staff/interviews |
|
| Extent to which LSB staff perceives Branch capacity has been enhanced through internal training and knowledge management tools | Staff satisfaction with training received (content, relevance) Staff satisfaction and use of materials designed to enhance consistency Staff satisfaction with quality control processes/protocols within the LSB |
Training files/feedback results/analysis LSB staff/interviews, survey PSES results (Q20) |
||
| Extent to which other recipients of LSB capacity building efforts perceive capacity is enhanced | Client satisfaction with training received (content, relevance) | Training files/feedback results/analysis Justice legal staff/ interviews LSB departmental clients/interviews, survey |
| Evaluation Issue | Question | Performance Indicator | Measures | Data Sources/ Methods |
|---|---|---|---|---|
| Effectiveness | 3.1 To what extent has the LSB contributed to the creation of federal legislation that respects the Constitution and other legal requirements? | Nature of/extent of LSB measures to ensure that legislative drafts respect the Constitution and other legal constraints | Number/type/nature of Charter challenges Footnote 67 Type and adequacy of quality assurance measures in place |
LSB client case files/analysis LSB staff/interviews, survey Case Studies |
| 3.2 To what extent has the LSB contributed to a bilingual and bijural federal legislative framework? | Extent to which the bijuralism website is used | Justice Laws, Bijuralex.ca usage by type of user
|
Web site statistics/analysis | |
| Extent to which LSB has contributed to a bilingual and bijural federal legislative framework | Experiences/perceptions of committee members and federal government departments representatives | Document Footnote 68 review LSB departmental clients/ interviews PCO, TBS policy makers/interviews Justice legal staff/ interviews Case studies |
||
| 3.3 To what extent has the LSB contributed to government departments and agencies being better able to manage their legal risks? | Extent of LSB contribution to legal risk management | Number/percentage of files where legal risk is reported as high and nature of strategies used to manage/mitigate risk Avoidance of legal risks Perceptions of government departments and agencies on LSB contribution to legal risk management Perceptions of Justice officials (e.g. DLSU) of LSB contribution to legal risk management |
LSB administrative records/analysis LSB client case files/analysis LSB departmental clients/ interviews, survey Justice legal staff/ interviews LRM evaluation/analysis Case studies |
| Evaluation Issue | Question | Performance Indicator | Measures | Data Sources/ Methods |
|---|---|---|---|---|
| Effectiveness | 4.1 To what extent has the LSB contributed to a federal government that is supported by effective and responsive legal services? | Cumulative | Achievement of direct and intermediate outcomes | Cumulative/contribution analysis LSB staff/interviews PCO, TBS policy makers/interviews Justice legal staff/ interviews |
| 4.2 To what extent has the LSB contributed to a fair, relevant and accessible justice system that reflects Canadian values? | Cumulative | Achievement of direct and intermediate outcomes | Cumulative/contribution analysis LSB staff/interviews PCO, TBS policy makers/interviews Justice legal staff/ interviews |
| Evaluation Issue | Question | Performance Indicator | Measures | Data Sources/ Methods |
|---|---|---|---|---|
| Efficiency and Economy | 5.1 Has the LSB’s resource utilization been appropriate, in relation to the resources allocated, activities and outputs produced and demands for services, by area of service delivery? | Budget and work plan priorities and activities in relation to outputs delivered by service area | Cost of legal inputs for each service line/type of service Resources used to produce request by type of request and by service line Sufficiency (i.e., quality and quantity) of planned/allocated resources to support tasks required Redundancy, duplication, surplus of resources Analysis of planned versus actual inputs/resources used |
Document review LSB financial database/ analysis LSB administrative records/ analysis LSB staff/interviews, survey Case studies |
| 5.2 Is the LSB’s capacity to deliver services sufficient given current and future demand for services, by area of service delivery? | Level of service in relation to demand for services (past, current, projected) | Volume of requests per staff Volume of requests per service line Ability to complete requests (within reasonable timelines) Over or under-commitment of resources that compromised the quality, quantity, appropriateness or timeliness of required outputs Future human resource needs (churn/vacancies) Changes that have been made to increase the level of outputs produced with resources allocated |
LSB administrative records/analysis Documentation/review LSB staff/survey, interviews LSB staff/PSES LSB departmental clients/ interviews, survey |
|
| 5.3 Are the LSB’s activities sufficiently integrated and coordinated to support the achievement of results? | Extent to which integration of LSB activities supports the achievement of expected results | Level of coordination/integration of activities within LSB, Justice and LSB client departments and agencies Satisfaction with the level of coordination |
LSB staff/ interviews/survey Justice staff/interviews |
|
| 5.4 To what extent has the LSB been able to use IT support and technology to improve efficiency and effectiveness of service delivery, by area of service delivery? | Extent to and nature in which IT innovation has been developed/applied by service area | IT supports in place Effects of IT on administrative practices/service delivery |
LSB files/analysis LSB staff/ interviews/survey |
|
| Extent to which LSB use of IT support and technology is consistent with or comparable to other jurisdictions | Comparison of LSB IT support and technology to that in other jurisdictions Trends in use of IT support and technology in legislative services/ analysis |
LSB staff/interviews LSB files/analysis Literature/environmental scan |
||
| 5.5 Does the financial model Footnote 69 utilized by the LSB provide appropriate and sustainable funding for the LSB to meet current and future demand for legislative services? | Sufficiency of A-base funding to discharge the Minister’s responsibilities Footnote 70 | Resource coverage by A-base funding Proportion of LSB work completed that is not cost recoverable (relative to amount of A-base funding) Gap between financial inputs and resource requirements for outputs |
LSB managers/interviews LSB files/data Financial documentation/ review |
|
| Trends in financial models for supporting/ funding legislative services | Examples from other Justice branches/services (e.g., Public Law Sector) Examples from other jurisdictions that provide similar types of legal services within government Comparison of alternative models with the LSB model (with respect to federal Canadian needs) |
Literature/environmental scan Financial documentation/review LSB managers/ interviews Finance staff/interviews |
||
| 5.6 Are there alternative ways of delivering the LSB’s services by areas of service delivery? | Trends in practices/ service delivery models | Examples from other jurisdictions Comparison of alternative models with federal Canadian needs |
Literature/environmental scan LSB staff/interviews |
- Date modified: