Legal Risk Management in the Department of Justice
Appendix C: Data Collection Instruments
Evaluation of Legal Risk Management
Key Informant Interview Guide
Since 1999, the federal government has managed potential and realized legal risks within the framework of the Legal Risk Management (LRM) Initiative. Wide in scope, the Initiative reaches into organizational structures and processes, work and management tools, and expected skills and behaviors of legal counsel and managers within the Department of Justice and Government of Canada departments and agencies.
In 2003, the LRM Initiative evolved from a time-limited initiative, with a project office and dedicated human and financial resources, to an ongoing initiative that was integrated into the overall work of the Department. As a result, the LRM Accountability Framework and Governance Structure (AFGS) was put into effect (see Annex A). The AFGS stipulates that it is "the responsibility of all employees and managers across the DOJ to know and apply LRM principles and methods appropriate to their particular positions and areas of responsibility"
. It also establishes defined roles and responsibilities for senior managers.
The Department's Evaluation Division is now evaluating the implementation of the LRM Initiative. As part of the data collection process, interviews will be conducted with a range of key informants within the Department.
All information that you provide is confidential and will be used only for research purposes. No individual participant will be identified in any of the reports submitted to Justice Canada. The information that you provide will be reported in aggregate form, and individual responses will not be shared outside of PRA Inc. and the Department's LRM Evaluation Team, which includes representatives of the Evaluation Division and the Law Practice Management Directorate.
The interview should take about 60 minutes and, with your permission, will be tape-recorded to ensure the accuracy of information reported. All tape recordings will be erased at the end of the study. Finally, we would like to remind you that your participation in this interview is completely voluntary.
Introduction
1. Please describe your current role and responsibilities. How do they relate to LRM?
Rationale for the LRM Initiative
-
2. How has the context within which LRM is operating changed since 2003? In particular, how is decentralization of LRM under the 2003 LRM Accountability Framework and Governance Structure working? Has this had any positive or negative effects on the DOJ's ability to systematically manage legal risks?
-
3. What is the need, if any, for a systematic approach to managing legal risks?
Implementation of LRM
-
4. In your opinion, is there a common or shared understanding of LRM roles, responsibilities, and accountabilities across the Department?
-
5. How consistent is the DOJ in its approach to identifying and assessing legal risks? In your answer, please consider issues such as the following:
- Consistency in rating risk levels (particularly levels 6, 7, 8, and 9)
- Standards used to assess legal risk and possible overuse of risk ratings 5 and/or 6
- Assessing legal risk in all DOJ activities (litigation, advisory, legislative services, policy)
- Approach to reassessing legal risk
-
6. How consistent is the DOJ in its approach to mitigating and managing legal risks? In your answer, please consider issues such as the following:
- Use of contingency and communication plan models
- Managing and reporting on legal risk in files with comparable risk ratings (particularly levels 6, 7, 8, and 9)
- Managing legal risk in all DOJ activities (litigation, advisory, legislative services, policy).
-
7. What initiatives, if any, has your sector, portfolio, region, legal services unit/client department or agency implemented to assess or manage legal risks more systematically?
-
8. What are the advantages and disadvantages of having different LRM practices across the DOJ? Are there areas where LRM practices should be better coordinated or where national practices/standards should be used?
-
9. Are the tools that the DOJ has put in place adequate to support LRM? (Probe: Does the risk matrix accurately capture risk that should be actively managed? Does the matrix work for advisory/legislative services/policy files?) To what extent are counsel aware of these tools? Are they routinely used when circumstances warrant?
-
10. Which tools are most effective? Which are least effective and why? What other tools are needed?
-
11. Are the current practices for communicating or reporting legal risk within the DOJ effective? Why or why not? (Probe: Are DOJ senior managers receiving adequate and timely information? Does the assessment grid ensure that all high risk files are communicated to senior management? Does the current communication strategy appropriately consider advisor, legislative services, and policy files?)
-
12. How do client departments/agencies understand their role in managing legal risks? To what extent is legal risk integrated into their corporate decision making?
-
13. Has your sector, portfolio, region, legal services unit undertaken any activities to engage client departments/agencies and enhance their understanding of LRM? (Probe: Does your sector, portfolio, region, legal services unit have any joint LRM-related committees?) Have these activities resulted in effective LRM partnerships? (Probe: Are client department/agency senior managers receiving adequate and timely information? Are they actively engaged in LRM?) How could partnerships be improved?
-
14. Are there adequate resources (human, financial, information, training, technological) to support the management of legal risk in the DOJ? What additional resources, if any, are needed? (Probe: Dedicated/additional staff, training opportunities, resources to overcome any technological challenges, etc.)
-
15. Are risk assessments used in resource allocation and planning? Please explain how they are used. (Probe: what measures might be taken, such as additional counsel, reassignment of senior counsel to file, etc., and are there criteria for when these steps might occur?)
-
16. What have been the main challenges in implementing LRM? (Probe: Are there elements of LRM that have not been implemented? Are there elements that are too difficult/time-consuming/not helpful? Are counsel having difficulties integrating LRM into their daily work?)
-
17. Are there any gaps in processes and practices for the identification, assessment, and management of legal risk? Should LRM processes extend to monitor and manage level 6 files?
-
18. What do you see as the role of the Law Practice Management Directorate in LRM?
Results
-
19. How have DOJ practices had an impact, if any, on the mitigation or management of legal risks in the federal government? For example,
- Has the level of awareness of legal risks changed among client departments/agencies?
- Has the understanding of legal risk management improved among client departments/agencies?
- Do client departments/agencies routinely consider legal risks in the development and implementation of policies, programs, legislation, and regulations?
- Has LRM been successful in avoiding or mitigating legal risks?
- Has LRM reduced government liabilities?
- Others? Please explain.
-
20. What measurement of success would you need to determine if LRM has been effective in identifying, managing, and mitigating legal risk? Do you think that the DOJ measures the results of LRM in a way that assists managers in planning? (Probe: iCase LRM reports) What other measures or tools would assist you?
-
21. What are the key lessons learned and best practices from the various models of LRM used by sectors/portfolios/regions/legal services units/sectors? Are they being effectively communicated and shared across the DOJ? If not, how could this be improved?
-
22. What suggestions do you have for improving LRM?
-
23. Do you have any other comments?
Thank you. We greatly appreciate your participation.
Legal Counsel Survey
Si vous préférez y répondre dans l'autre langue officielle, veuillez changer de texte maintenant vous ne pourrez pas le faire une fois que vous aurez commencé à répondre aux questions.
Since 1999, the federal government has managed potential and realized legal risks within the framework of the Legal Risk Management (LRM) Initiative. Wide in scope, the Initiative reaches into organizational structures and processes, work and management tools, and expected skills and behaviours of legal counsel and managers within the Department of Justice (DOJ) and Government of Canada departments and agencies.
The DOJ Evaluation Division is conducting an evaluation of LRM in the Department. The purpose of the evaluation is to examine how LRM has been implemented in the Department and to assess whether the structures and processes put in place to support LRM are contributing to the effective management of legal risk in the federal government. The evaluation is being conducted by PRA Inc., an independent research firm working in collaboration with the Justice Evaluation Division, the Law Practice Management Directorate and the Research and Statistics Division. This survey of legal counsel is being conducted as part of the evaluation. Your response is very important to us. We want to hear from all DOJ counsel, even if legal risk management is not part of your daily work.
The survey is easy and quick to answer. Most questions only ask you to click on the appropriate response. The questionnaire should take about 15 minutes to complete. You can only access the survey site once, so please complete the survey in one sitting. (Please note that if you click on your browser's "back" button while completing the survey, you will need to respond again to the questions that follow the one you returned to.)
All information you offer is confidential and will be used only to create aggregate results to be included in the evaluation report. No individual's responses will be identified.
We would appreciate receiving your completed questionnaire by March 18, 2008.
If you have questions about the survey, please contact Susan Kelly, Evaluation Division at (613) 957-7657. For technical questions, please contact Jo-Anne Chrétien at (613) 957-9610.
Background
The following questions will be used to establish a profile of survey respondents.
-
1. When did you first join the Department of Justice?
- Less than a year ago
- Between 1 and 5 years ago
- Between 6 and 10 years ago
- More than 10 years ago
-
2. Where do you currently work?
- Headquarters
- Regional Office
- Legal Services Unit
- Satellite Office
-
3. What is the classification level of the position you currently occupy?
- LA-1
- LA-2A
- LA-2B
- LA-3A
- LA-3B
- LA-3C
- Are you in a management position?
Shared Understanding
The following questions ask about your experience with legal risk management.
A legal risk is a risk arising out of an issue or event giving rise to legal considerations. A legal risk may arise from a government action or decision (or proposed action or decision), or from exterior actions or decisions that have legal implications and that require a government response or action of a legal, communication-related, organizational or political nature.
-
4. How would you rate your overall understanding of legal risk management (LRM)?
- Very good
- Good
- Limited
- Poor
-
5. Please indicate your level of agreement with the following statements:
- LRM is well defined by DOJ.
- Strongly agree
- Agree
- Disagree
- Strongly disagree
- Don't know
- My role and responsibilities in relation to LRM are clearly defined.
- Strongly agree
- Agree
- Disagree
- Strongly disagree
- Don't know
- Client departments and agencies understand their roles and responsibilities in relation to LRM.
- Strongly agree
- Agree
- Disagree
- Strongly disagree
- Don't know
- Managing legal risk is part of my practice of law.
- Strongly agree
- Agree
- Disagree
- Strongly disagree
- Don't know
-
6. Which kind of files do you work on most often that require the management of legal risk?
- Litigation
- Litigation support
- Advisory
- Legislative services (legislation and regulation drafting)
- Policy and program development
Implementation of the LRM - Litigation Files
-
7. Taking into consideration the litigation files you have been involved with in the last two years, how often were legal risks identified and assessed?
- Frequently (75%-100% of files)
- Regularly (50%-74% of files)
- Occasionally (25%-49% of files)
- Rarely (1%-24% of files)
- Never (0% of files)
- Don't know
- Not applicable to my work
-
8. In assessing legal risk, how often during the last two years have you or a member of the litigation team…
- consulted the client's Legal Services Unit (LSU)? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted the client department? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted with specialized units within DOJ (e.g.,units within the Public Law Sector or Policy Sector, etc.)? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted other potentially affected LSUs? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reassessed risk after the initial assessment? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- inputted risk assessments in iCase? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
-
9. Please rate how useful you have found the LRM assessment grid/matrix through iCase or otherwise.
- Very useful
- Useful
- Not very useful
- Not at all useful
- Not applicable / have not used this tool
-
10. Why do you find the LRM assessment grid/matrix not useful? (Check all that apply)
- The scale for the level of impact on the government (significant, moderate, minor) is not well defined.
- The scale for the likelihood of an adverse outcome (low - less than 30%; medium - 30%-70%; high - over 70%) does not have enough categories.
- The grid results in under-reporting the number of files that you believe are actually high risk.
- The grid results in over-reporting the number of files that you believe are actually high risk.
- The grid results in inconsistent risk level ratings across DOJ.
- The grid does not take into account urgency or time (e.g. high risk but nothing will happen for several years).
- The grid does not capture other relevant aspects of risk (please specify)
- Other, please specify
-
11. Why have you not used the LRM assessment grid/matrix? (Check all that apply)
- I am not aware of the LRM assessment grid.
- I do not know how to use the grid.
- The grid will not accurately capture risk.
- Other, please specify
-
12. Please rate how useful you have found the iCase risk evaluation screen.
- Very useful
- Useful
- Not very useful
- Not at all useful
- Not applicable / have not used this tool
-
13. Have you used another risk assessment tool?
a) Please rate the other risk assessment tool that you have used.
- Very useful
- Useful
- Not very useful
- Not at all useful
-
14. In developing legal risk management strategies, how often during the last two years have you or a member of the litigation team…
- consulted the client department? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted the client's LSU? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted your manager? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted litigation or risk-related committees? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted LRM contact person for portfolio/ region/LSU? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted specialized units within DOJ (e.g., units within the Public Law Sector or Policy Sector, etc.)? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- consulted other potentially affected LSUs? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
-
15. Please estimate the combined number of high risk files (risk levels 7, 8, and 9) that you have been involved with in the last two years.
Estimated number of files: (enter 0 if none)
-
16. Considering only your high risk files (risk levels 7, 8, and 9), how often during the last two years have you or a member of the litigation team…
- reported these files to senior managers in DOJ headquarters? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to regional/legal unit managers? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to litigation or risk-related committees? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to the client's LSU? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to client officials? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to portfolio managers? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- prepared Briefing Notes for these files? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- prepared Early Warning Notes for these files? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- assisted in the development of a contingency plan for these files? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- Have you ever used dispute resolution for your high risk files (risk levels 7, 8, and 9)?
- In your risk level 7, 8, and 9 files, at what stage do you most often use dispute resolution options?
- Post-Pleading
- Post-production of documents
- Post-discovery
- Just prior to trial or hearing
- Other, please specify
- Don't know
-
17. Please estimate the number of risk level 6 files you have been involved with in the last two years.
Estimated number of files: (enter 0 if none)
-
18. Considering only your risk level 6 files, how often during the last two years have you or a member of the litigation team…
- reported these files to senior managers in DOJ headquarters? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to regional/legal unit managers? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to litigation or risk-related committees? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to the client's LSU? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to client officials? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- reported these files to portfolio managers? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- prepared Briefing Notes for these files? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- prepared Early Warning Notes for the files? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- assisted in the development of a contingency plan for these files? Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- Have you ever used dispute resolution for risk level 6 files?
- In your risk level 6 files, at what stage do you most often use dispute resolution options?
- Post-Pleading
- Post-production of documents
- Post-discovery
- Just prior to trial or hearing
- Other, please specify
- Don't know
-
19. Please estimate the number of files that you have been involved with in the last two years, where the level of risk was reassessed from medium to high.
Estimated number of files: (enter 0 if none)
-
20. Considering only files that you have been involved with that have been reassessed from medium to high risk in the last two years, how often have the following occurred after the risk level was raised?
- The number of counsel increased. Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- Senior counsel were assigned. Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- Increased consultations. Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work
- Increased reporting. Percentage of files:
- Frequently (75%-100%)
- Regularly (50%-74%)
- Occasionally (25%-49%)
- Rarely (1%-24%)
- Never (0%)
- Don't know
- Not applicable to my work