The Sensemaking Model for Airline Pilot Training: TCC Online Conference Master's Project Presentation Barth Baron "JR" April 15, 2021
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
The Sensemaking Model for Airline Pilot Training:
Building and Maintaining Expert Flight Path Management
TCC Online Conference
Master’s Project Presentation
Barth Baron “JR”
April 15, 2021
10 years old: First flight
Wind tunnel project
12,000 hours ~ 1.4 years as a pilot
Graduating Master’s Project:
Asynchronous
Six interactive videos
20 participants
How pilots learn!
Distraction
Task saturation
Automation mode confusionWhat usually causes accidents? 1: Technology 2: Humans What usually prevents accidents? 1: Technology 2: Humans
Solo à Crew Analog à Digital Training does not support this transition
Dorrit Billman
Randy Mumaw
Michael Feary
A Model of Monitoring as Sensemaking: Application to
Flight Path Management and Pilot Training
(2020)Why is this so hard?
Image by Chris Wilson www.starnow.co.uk/christopherw33618Purpose Statement The purpose of this learning assessment is to evaluate how design for motivation impacts perceived relevance on current airline pilots of an online module introducing a new framework of flight path monitoring based on current research, and how successful task-based learning is at integrating this monitoring framework into the pilot’s experience. Additionally, this assessment will evaluate pilot acceptance of this model for accomplishing flight path monitoring in actual flight operations.
Literature Review: Current state of monitoring training Effectiveness of typical monitoring behaviors Target audience affective qualities: (Helmreich & Merritt, 2001) Invulnerability…”Lack of regard for personal limitations” Demonstrates a cultural resistance to receive outside input
Well, that sounds like fun!
How to do it? How to measure it?
Relevance Instructional Materials
(Keller, 2010) Motivation Survey “IMMS”
Integration (Keller, 2010)
Task-centered
Teaching and Learning
Generalizable skill
Quality “TALQ”
(Merrill 2009) (Frick et al. ,2009).
Multimedia principles
Voice/Redundancy/ Loorbach et al., (2015).
Personalization (Mayer 2009)RQ 1) How does design for motivation in an online
module impact perceived relevance of a
“sensemaking model of monitoring” for pilots?
Keller ARCS, design for motivation (Relevance):
1) Align with learner goals and motivations
2) Highlight the gap between the current state and our goals
3) Concrete examples that are familiar to the learnerRQ 2) How successful is a task-centered instructional module
at integrating the “sensemaking model of monitoring”
into the actual operating experience of pilots?
Merrill Task-centered learning: Integration principle
1) Apply with real-world tasks
2) Reflect on how the skill is used in learner’s worldRQ 3) How successful is the module at promoting pilot acceptance
of the “sensemaking model of monitoring” into their
operating habits?
Exploration of hybrid design: Keller + Merril
Relevance + Integration = Acceptance?Content Analysis
Learner can synthesize the situation model, task and attention
management, and crew communication to perform “expert flight path
management
Perform
Build and Perform task and communication
maintain the attention behaviors that
“Situation Model” management support the
situation modelModule and Development
How to convey the content in an interactive
asynchronous module that ensures:
Relevance Integration
Authentic tasks Popup interactives
Real scenarios Reflection
Familiar imagesSensemaking and the Situation Model
VIDEO
Task and Attention Management
Activate Prior KnowledgeCrew Communication
Instruments
Five point Likert
5 Point Likert
Google Forms 1 = Strongly Disagree 5 = Strongly Agree
Demographics: Age, rank, gender
RQ1: Keller Relevance questions from Keller’s IMMS
RQ2: Merrill Integration questions from Frick’s TALQ
RQ3: Pre-then-post retrospective questionsResults: RQ1 Perceived Relevance
Mid Survey 1: The Situation Model Mean s The concepts presented in this section align with my goals 4.55 .47 The skill learned in this section will be useful to me 4.30 .78 Mid Survey 2: Crew Communication The concepts presented in this section align with my goals 4.80 .40 The skill learned in this section will be useful to me 4.75 .43
RQ 1) Exit survey, open ended question “The first module included a diagram. I find this type of diagram very abstract and academic.” (“Dennis,” male A330 captain, > 50) “I frankly can not stand models. in this case, you were able to lay out the situational model in a very practical manner that allowed me to actually get something from it. It also was obviously very important.” (“Harrison,” male A321 captain, 40-49)
Post-survey: Relevance
Mean s
4.58 .49
I’m glad I completed this module 4.85 .36
Captains: 4.0 First Officers: 4.36 4.20 .50
I’d like more opportunities to learn this 4.75 .43
4.65 .48
“This module was NOT relevant to my needs because I already have
monitoring skills.” Reverse codedRQ 1) Post survey, open ended question “I thought it was excellent subject material, incredibly helpful with a number of highly relevant examples.” (“Marlon,” A330 first officer) “I really liked that you used Seattle in the scenarios. It was very relatable.” (“Colleen,” A330 first officer). “I specifically liked the examples as they were challenging, practical, and pertinent…Overall, I’d love to see this emphasized in class training as well as in a sim-based environment.” (“Harrison,” A321 captain)
Results: RQ2 Integration
Mid 1&2: I see how I can apply what I learned to my line flying
Situation Model Mean = 4.5 s = .59
Communication Mean = 4.85 s = .36
Post:
Mean = 4.4 s = .58
Mean = 4.1 s = .62
E.g: “I was able to demonstrate what I used in the module”“My favorite and most poignant line - the one about how "under-communication" is often a cause.... Love this. This point needs to be emphasized in our training program!!!” (“Lance,” male A321 captain, 40-49) (A321 CA M 40-49)
RQ 2) Exit survey, open ended question The questions made me consider how much I'd actually been taught about monitoring (“Laurence,” A330 first officer) I loved there were interactions throughout the video to keep me engaged and motivated to pay attention (“Colleen,” A330 first officer) Following the course I felt curious/open to learning more. (“Martin”A321 first officer)
Results: RQ3 Acceptance
n = 20
RQ 3) 14
Pre-then-post 39%
12
retrospective survey
10
Participants
Specific monitoring 8
behaviors 6
4
Mean BEFORE = 2.6
Mean AFTER = 4.55 2
0
Definitely not Probably not Possibly Probably Definitely
PRIOR to watching the module AFTER watching moduleFollow up participant interviews with NASA group
Five interviews conducted after IRB approval (4-6 weeks after study)
Q: Has this changed how you fly? “Absolutely influenced my monitoring”
“I absolutely incorporated the [framework] “
Q: What most stands out? Crew CommunicationSummary
“Refreshing” “Relatable” “Valuable”
Engagement/Relevance?
“Relevant” “Concise” “Important”
Integrated? “More examples”
Accepted? Need to work on those older captains!
Negatives? LMS login and navigation issuesDiscussion Affective/Cognitive: Cognitive “companion module”
Let’s go flying!
Thank you! Dr. Fulford
CF Elle Nakamura
Image courtesy of Evan BaronYou can also read
