CAA Level 3 Applied Avalanche Risk Management Course Evaluation Process and Guidelines
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines
CAA Level 3 Applied Avalanche Risk Management Course
Evaluation Process and Guidelines
Course Evaluation for Written Report and Oral Presentation
Goal
The goal of the evaluation process is for participants to demonstrate a comprehensive understanding of
the application of Level 3 (L3) curriculum in real world environments. Participants must be able to both
write and orally present a well-reasoned year-end report, integrating and applying L3 content and
principles. The ability to present effectively, both orally and in written format, is an important skill for
those working in supervisory roles in avalanche operations.
Structure
The course evaluation will be based on a case report of avalanche risk management in your work place
during the winter following the classroom portion of the course. The evaluation will consist of both a
written report and oral presentation components.
1. A written report must be submitted to the Canadian Avalanche Association (CAA) office as an
electronic PDF as well as two mailed hardcopies. All copies of your written report need to be received
by May 20th. The electronic PDF copy can be emailed to alustenberger@avalancheassociation.ca or, for
larger files, uploaded to the link provided in the spring CAA exam registration e-mail. Your hardcopies
should be mailed to the following address: Canadian Avalanche Association, Attention: ITP Manager,
P.O. Box 2759, Revelstoke, BC, V0E 2S0.
Note: Lakeside Printing (250.837.0161) in Revelstoke does a professional job of printing and binding
papers. For students struggling with the deadline, or mailing hard copies from afar, this is a good
alternative. Contact them directly to have the printed papers ready for pick-up May 20th. Also contact
the ITP Manager that your papers will need to be picked up. Only Lakeside Printing in Revelstoke can
be used for this pick-up option as it is ½ a block from the CAA office.
Students wishing to have their hard copies returned to them with marking comments must also
submit two self-addressed pre-paid courier envelopes.
2. An oral presentation and evaluation of your work will occur in the period of June 10th – 17th.
Evaluations will occur as a Skype conference, not in person. The CAA office will contact you
regarding your exact day and time.
Both evaluation components must apply the concepts and language taught on the L3 course to describe
how avalanche risk was evaluated and managed in day to day operations. Refer to the Guidelines for
Written Reports and Guidelines for Oral Presentations sections below for more details on the structure,
content, and expectations. Appendix B - Marking Scheme for Written and Oral Reports includes more
details on how the written and oral presentations are evaluated.
Late Reports: Reports received, including the hard copies, after May 20th will receive a 2% penalty
for every day past the due date. No exceptions.
©Canadian Avalanche Association 2020 Page 1CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines
Deferments: Participants may defer their written and oral reports for one year but must provide
notification of the intent to defer to the CAA Industry Training Program (ITP) Manager no later than
May 1st following their classroom course.
Appeals/Disputes: Participants who wish to appeal/dispute their marks must do so within three weeks
of the release of the marks from the CAA office.
Guidelines for Written Reports
The first component for the evaluation process is a written report. Participants are to create a written case
report approximately 15-20 pages long, 12-point font with 1.5 line spacing (about 5000 words). The
content of the report describes your role and experiences in your operation(s) in the season following the
classroom portion of the course. The intent of this assignment is to give you the opportunity to
demonstrate your application of the course concepts to your work environment by creating a well written,
clear report. Think of your report as a year-end summary. The audience you are writing for is familiar
with standard avalanche terminology and L3 concepts but is not familiar with your position, your
responsibilities, or your work context. The evaluators do not judge individual operations nor do they
second guess operational decisions made during the season. The reports are confidential amongst the
participants and evaluators. Note: For students unfamiliar with technical writing, see Appendix A -
Guidelines and Suggestions on Technical Writing.
The eight general headers below illustrate the basic structure of the report. The bullet points highlight the
general content each header may contain. Do not feel restricted to only the bulleted points and be aware
that each point may not be relevant to every operation. This is a guideline only. The written report may
contain figures and images as images may be essential to convey key information. A picture can be worth
1000 words! However, choose all figures and images wisely and selectively.
1. Introduction:
• Introduce your topic and establish your key messages.
• Describe the nature of the operation.
• Describe your roles and responsibilities within the operation.
• What are the elements at risk and how, and to what hazards, are they vulnerable?
• Describe the geographic setting and general snow climate.
• Describe the temporal and spatial scales relevant to avalanche forecasting in your operation.
• Provide any historical information that puts this operation into context.
• Describe any unique challenges or unique elements of this operation.
2. Winter Summary:
• Summarize the significant weather events and general pattern of the winter.
• Describe the significant snow layers of interest through the season.
• Explain the related avalanche cycles and character.
• Describe what was unusual or notable about this winter, if anything.
3. Data and Evidence:
• Describe the sources of data and evidence used in hazard analysis in your operation.
• Provide examples of strength and weight of evidence at play using real examples.
• Provide examples of how confidence and uncertainty were managed.
• Provide examples of how bias influenced the interpretation of data and evidence.
©Canadian Avalanche Association 2020 Page 2CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines
4. Hazard Evaluation:
• Describe the operation’s process for evaluating hazard.
• Describe how you evaluate/forecast likelihood of triggering and expected avalanche size.
• As appendices, provide three examples of hazard evaluations you have produced that demonstrate the
use of the Conceptual Model. If your operation uses a format similar to the CAA hazard worksheet or
InfoEx you may use that. The examples should usually link to the notable event described in the
body of the report. If the examples do not link to your notable event, provide enough additional
background that the examples are meaningful and add value to the reader. For example, you could
link the hazard evaluations to risk treatment. Make sure to demonstrate your capacity to analyze risk
factors and do not simply recapitulate basic weather/snowpack data; provide analysis of why these
data are significant. These examples can be scanned paper copies, digital forms, or the web based
system developed by Pascal Haegeli with Avisualanche Consulting at www.avisualanche.ca.
5. Risk Treatment:
• Summarize the risk treatment and control methods used – both field and office.
• Describe how you evaluate and manage exposure and vulnerability.
• Describe how the effectiveness of the risk treatment is measured and the actual risk reduction
achieved.
• Describe how risk is communicated to your target audience. How is communication structured to
ensure the message is received by the target audience? What are the key messages?
• Identify changes to your operations that will improve risk control in the future.
6. Notable Events:
• Describe a notable event(s) from the winter. Make links to the hazard form examples in the
appendices.
• Why was this event significant or unusual? Was it expected or a surprise?
• How do you benchmark the event(s) from a historical perspective?
• Using the language of risk, describe what biases may have influenced these events.
• If your operation had a “notable free” winter, describe the reasons/factors that lead to this.
Note: Events that are contentious or events that may result in liability issues should not be included
in the report.
7. Lessons Learned:
• What key lessons were learned from this winter?
• What changes are anticipated for the future in light of lessons learned?
• What successes did the operation have this season? What challenges were experienced?
8. Summary and Conclusion
• Summarize and conclude the key messages from your report and link back to your
introduction.
• Identify what the L3 course has meant to you. Reflect on L3 curriculum material that were/
were not pertinent to you.
Guidelines for Oral Presentations
The second component to the evaluation provides the participant with the opportunity to demonstrate their
learning orally using a different but equally important presentation skill. Oral presentations will take place
©Canadian Avalanche Association 2020 Page 3CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines
via Skype (video conference tool) unless a convenient meeting place can be arranged between the student and
all of the evaluators. Students will need to download a free version of Skype at www.skype.com. You will
need to “invite” your evaluators to join you on Skype, so it is important to have a “Skype name” which will
be recognized by the evaluators. Invitations should precede your interview by a minimum of 48 hours. As
mentioned earlier, your presentation will occur in the period from June 10th to 17th. The CAA office will
contact you regarding your exact day and time. You will need a voice/video capable computer and a fast
internet connection. Be sure to test drive your system at least a week before the evaluation.
Though not required, visual aids will likely enhance your presentation and make some communications
more effective. Audio-visual materials should be in a PowerPoint format and e-mailed to the evaluators. You
must forward the PowerPoint via e-mail at least 48 hours in advance of your presentation and confirm that it
was received. Evaluators will run this presentation on a separate computer in their locations and advance the
images on your cue. This method has worked in the past with good success.
The oral evaluation will be kept to one hour duration. Fifteen minutes is allotted for your presentation; 45
minutes for the question and answer (Q & A) session. The timelines need to be strictly adhered to.
Content for the oral presentation is largely up to you. Think of it as your opportunity to take two or three key
messages from your written report and expand on them. We suggest that you focus your presentation on one
or two interesting elements. For example, you could include unique or notable events, significant lessons
learned, new knowledge gained, or how certain challenges were dealt with. Allow yourself lots of time to
prepare and rehearse if you want to deliver a polished and professional presentation. The following basic
outline of topics to consider will help you structure your presentation.
The presentation is to be 15 minutes. Longer presentations will not allow the evaluators sufficient time
for the Q & A period. Be prepared and rehearsed.
1. Introduction:
• Set the context. What is the nature of the operation and why is it unique?
• Describe the elements at risk.
• Include the geographic location, snow climate, and avalanche regime from this past winter.
• Describe the key messages of what you will talk about.
2. Main Body of Content:
• Choose a limited number of key points or issues from your written report.
• Provide specific details to describe the targeted issue(s).
• Demonstrate why these were important issues to discuss.
• Describe the conclusions that were drawn or the lessons learned.
3. Summary and Conclusion:
• Summarize what was discussed.
• Re-enforce key messages or conclusions.
Following your 15 minute presentation, a maximum of 45 minutes is available for the question and answer
session. The Q & A session is your opportunity to defend your presentation; answer questions the evaluators
may ask that demonstrate your understanding of L3 terms and concepts; clarify elements that were unclear in
either the written or oral presentations; or pursue an interesting topic. Good familiarity with the language,
©Canadian Avalanche Association 2020 Page 4CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines
terms, and concepts of the L3 program coupled with a strong understanding of your operation is essential for
a successful oral presentation.
Marking
Marking begins directly after the written reports are submitted on May 20th. Two evaluators review each
written report and usually the same two evaluators grade the oral presentation. In some cases, a third
evaluator will review a paper to ensure consistency in marking. The grading rubrics found in Appendix B will
provide more details to the marking process.
Furthermore,
• Each component, the written report and oral presentation, is evaluated separately;
• Both components must be passed for the L3 Certificate;
• An overall mark of 71% or greater is needed to pass each component;
• If either the written or oral component is not passed, revise that element based on the evaluator
feedback and resubmit before the end of the following October*;
o If this retest is also failed, then the student must retake the L3 course in its entirety at
the student’s expense.
• If both the written and oral components are not passed, then there is the option to challenge the
evaluation process again the following spring. Submission of a new written report is due by
May 20th the following year*;
o In this case, where both components are being retested, if one of these components is
failed during the retest, then the student must retake the L3 course in its entirety at the
student’s expense.
• In case of disagreement, the student must submit his/her appeal to the CAA ITP Program
Manager within three weeks of the release of the marks from the CAA office.
*retest fees apply as outlined in the Retest Policy.
(Found under CAA Policies and Retest Fees: www.avalancheassociation.ca/?page=Student_Resources)
Confidentiality
Content of the written report and the oral presentation is confidential between the evaluators and the
participants. Written reports are destroyed securely after the marking process is complete. All electronic
material is also securely disposed of. If the ITP Manager becomes aware of report content through the
distribution and collection of written papers or through marking discussions they will keep all information
confidential.
Frequently Asked Questions:
What do I do if I am not currently making field decisions on a day to day basis in my current job as a
program manager? Good question, particularly if your program encompasses a large geographic region.
Describe the region in general and then focus the bulk of your report on one or two regions for a specific time
period. This is a great opportunity to contrast how policies and procedures support or inhibit the field based
practitioner. How do bias and heuristics affect your operation? How do you manage confidence and
uncertainty with operations when you, as the manager, may be removed from field operations? Make
sure that your roles and responsibilities are clearly outlined in the report.
The report due date of May 20th is inconvenient. I can be really busy that time of year. How do I deal with
this timeline? There is no easy answer to this question other than good time management. However, here are
a few tips. Don’t leave the whole report to the last minute. Set aside periods throughout your winter season to
©Canadian Avalanche Association 2020 Page 5CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines tackle sections of the report. Much of the report can be written any time. For example, you could create an outline of your thoughts and start the introduction right after you finish the theory portion of the classroom course. Keeping track of the winter summary on an ongoing basis will make that section easier to finish rather than waiting until the end of the season. If a notable event occurs mid-season take notes and write up that section while it is fresh in your mind. Some past participants finished their “report season” mid-March and added a post script for interesting things that happened after that. My current job does not require me to do much writing. Can I get help with the written portions? Yes, the guidelines for technical writing provided in these guidelines are a good place to start. There are also 1-2 day courses at colleges/universities/community centers that focus completely on technical writing. Past participants have found these courses extremely helpful. You could also ask a past Level 3 graduate to help guide and mentor you with your report. At the very least, have an objective third party edit your report for spelling, grammar, and clarity. I am a guide or consultant that works in a variety of locations through the winter for short periods. How do I handle this situation in my report? Good point. Describe what you do in the introduction and the unique challenges of your work. Then focus your report on a time or location that you feel is interesting to discuss. Keep in mind that the primary goal of the report is to demonstrate your understanding and application of L3 concepts and a good capacity for written and oral presentations. A time frame of one to two weeks of the winter should be sufficient to achieve this. A complete winter summary is still required. Is there a process to receive mentorship and guidance producing my report? In a formal sense, no. However, there are opportunities. Contact a L3 graduate or someone else in a senior avalanche position to mentor you through the report process. Mentors can be a sounding board for ideas and help edit the writing. An interested but unbiased opinion on the writing and oral presentation will help you prepare. If you are stuck for such a person to be a mentor contact the ITP Manager for help. Neither my current job (or life situation) has me completing any field work this winter nor am I managing an avalanche program. How do I develop the material for my report? Well, your situation presents a problem. The best option is to defer your report until the following year if you are doing active field work then. After all, the course is Applied Avalanche Risk Management! If you are not doing field work this year or the next please talk to the ITP Manager. ©Canadian Avalanche Association 2020 Page 6
CAA Avalanche Operations Level 3 – Course Evaluation Process and Guidelines
Appendix A – Guidelines and Suggestions for Technical Writing
Adapted with permission from Bruce Jamieson, September 2011
• Assume the reader is intelligent but unfamiliar with your topic. The reader is conversant with
“avalanche and risk” terminology but is not familiar with what it is you do or your operation.
• Use a publication manual or style guide. There are many good technical writing guides. Patrick
McDonagh and Jack Borden (2000) at Concordia University created a classic reference for
writing technical documents
https://www.concordia.ca/content/dam/ginacody/ces/docs/2014FormandStyleGuide.pdf
• Allow ample time for writing your report. Plan on proofreading and revising your report many
times. Reading aloud may help you catch typos and grammatical errors. The language needs to
be clear and unambiguous. Your message needs to be understood by the reader. Spelling,
grammar, and sentence structure are all important.
• Have a title page, table of contents, and number the pages.
• Place tables, diagrams, and photos with the text, not at the end of the report.
• The title for a table is placed at the top of the table.
• Place the figure title (caption) at the bottom of the figure. Captions should be sufficiently
descriptive so that a figure can be understood without referring to the text. An explanatory
sentence can be included in the caption so the figure and the caption can be interpreted separately
from the text.
• Use clear labels with sufficiently large font for axes and other attributes of graphs.
• Appendices follow the main report. Appendices should have a number and title as this one:
Appendix A.
• Use International System of Units (SI), except where the use of other units helps the reader. Use
the CAA OGRS to provide guidance for standard units of measure in avalanche work.
• While technical avalanche terminology will be understood by the reader, terminology from other
fields may not be and should have an accompanying definition.
• Repetitive use of lengthy technical terms may be abbreviated. Therefore, “persistent weak layer”
becomes PWL. This convention should be evident the first time the word is used. For example,
“The operation was plagued with persistent weak layers (PWLs) throughout the season.” Keep in
mind that overuse of abbreviations may make it more difficult for the reader and not enhance the
writing.
• If you use diagrams or photos which are not your own credit the source.
• Although the word “data” is often used with a singular verb in common speech, it is plural in
scientific writing. “The data are…”. Datum is the singular noun.
• The word “significant” implies that a statistical test has been applied to reach the finding. If this is
not the case, consider alternatives like “substantial” or “important”.
• In scientific and technical writing there is a space between the number and the unit for all units
except inclination and temperature. Therefore, three meters per second is 3 m/s, not 3m/s. Three
degrees inclination is: 3°. Three degrees Celsius is: 3° C.
• At the end of the day, the exercise is all about communication. Plain and concise language may
convey your message more effectively than a document filled with unnecessary jargon and
convoluted sentences.
©Canadian Avalanche Association 2020 Page 7CAA Avalanche Operations Level 3 – Course Evaluation
Appendix B: Marking Scheme for the Written Report and Oral Presentation
CAA Avalanche Operations Level 3 Course Grading Rubrics for Written Reports
Student’s Name:
Content Categories
Excellent Satisfactory Requires Does not meet Mark
(8.6-10.0) (7.1-8.5) Improvement Requirements
(5.0-7.0) (CAA Avalanche Operations Level 3 – Course Evaluation
Excellent Satisfactory Requires Does not meet Mark
(8.6-10.0) (7.1-8.5) Improvement Requirements
(5.0-7.0) (CAA Avalanche Operations Level 3 – Course Evaluation
Excellent Satisfactory Requires Does not meet Mark
(8.6-10.0) (7.1-8.5) Improvement Requirements
(5.0-7.0) (CAA Avalanche Operations Level 3 – Course Evaluation
Literary Quality Category
Excellent Satisfactory Requires Does not meet Mark
(8.6-10.0) (7.1-8.5) Improvement Requirements
(5.0-7.0) (CAA Avalanche Operations Level 3 – Course Evaluation
CAA Avalanche Operations Level 3 Course Grading Rubrics for Oral
Presentations
Student’s Name:
Content Category
Excellent Clear introduction of objectives; remains on topic; provides sufficient information
(21.5 – 25) and credible evidence; and highlights key content. Picks a useful but limited range
of topics from the written report to explore in detail. The oral content clearly
complements the written report providing further insight and value. The speaker
consistently applies L3 concepts and language.
Satisfactory Mostly clear introduction of objectives; usually remains on topic; usually provides
(17.5 – 21.25) sufficient information and credible evidence. May attempt to discuss too many topics
for a 15 minute presentation. Not all of the oral content clearly complements the
written report, but the listener has gained some further insights and value. The
speaker generally applies L3 concepts and language accurately but has a few minor
errors or omissions.
Requires Introduction needs some clarification or detail. Deviates from the topic somewhat.
Improvement Does not provide sufficient information or credible evidence in important sections.
(12.5 – 17.25) Oral content is simply a re-cap of the written report and neither complements the
written report nor provides further insights or value. Includes too many topics for a
15 minute presentation. Inconsistently applies L3 concepts and language with
significant errors and omissions.
Does not meet Introduction is vague and unclear. Does not remain on or complete a topic.
Requirements Provides insufficient information, and evidence is questionable or lacking. Oral
(˂12.5) content does not relate well to the written report or covers in detail material already
provided. Oral content does not add value or insights to the written report or to the
listener. Does not apply L3 concepts and language correctly.
Mark / 25
Comments
©Canadian Avalanche Association 2015 Page 12CAA Avalanche Operations Level 3 – Course Evaluation
Organization Category
Excellent Well prepared and rehearsed. Presentation has clear structure followed by a thorough
(17.5-20) summary of main points and a clear closure.
Satisfactory Prepared and rehearsed with some rough sections. The presentation structure is
(14-17) usually clear and most of the main points are summarized followed by the closure of
the presentation.
Requires The presentation is not well prepared or rehearsed. The structure is vague, and main
Improvement points are not summarized well. The audience is left hanging due to a weak or absent
(10-13.5) closure.
Does not meet Preparation and rehearsal are not apparent or absent. There is no structure to the
Requirements presentation. Main points are not summarized. The presentation simply ends
with no closure.
(CAA Avalanche Operations Level 3 – Course Evaluation
Question and Answer Category
Excellent Answers questions in the context of class materials and readings. Demonstrates a
(21.5 – 25) consistent understanding of all L3 concepts and language. Can provide definitions
of terms or examples of concepts when asked.
Good Answers most questions in the context of class materials and readings.
(17.5 – 21.25) Demonstrates a good understanding of most L3 concepts and language with minor
errors or omissions. Can usually define terms and provide examples of concepts
when asked.
Requires Inconsistent in answering questions and may not frame answers in the context of
Improvement class materials or readings. Occasional but significant errors in the use of L3
(12.5 – 17.25) concepts or language. Struggles defining terms or providing examples of concepts
when asked.
Does not meet Does not answer questions in the context of class materials and readings.
Requirements Demonstrates a poor understanding of L3 concepts and language with frequent
(You can also read
