Towards Diversity, Equity and Inclusion: "Right Resident, Right Program, Ready Day One" Project Update - March 3, 2021
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Towards Diversity, Equity and Inclusion:
"Right Resident, Right Program, Ready
Day One" Project Update
March 3, 2021
@Maya_Michigan
Disclosures
• Dr. Dalrymple: Current APGO Board member
• Dr. Hammoud:
• Paid consultant for the American Medical Association
• Current Board member and the immediate Past President of APGO
• PI on the AMA Reimagining Residency Grant “Transforming the UME
to GME Transition: Right Resident, Right Program, Ready Day One”
• Dr. Katz: Current APGO President
@APGOnews #ChangeResEd
Agenda
Update on the Reimagining Residency grant “Right Resident,
Right Program, Ready Day One”
• 2020 Recap
• Debrief on workgroups accomplishments and goals
Planning for 2021 application cycle
Open dialogue
@APGOnews #ChangeResEd
Goals
Improve the continuum of education between medical school
and residency to ensure that the right resident matches at the
right program and is ready for day one of residency.
1. Optimizing the alignment and fit between interested applicants
and potential programs through the application and Match
processes
2. Providing incoming (matched) PGY-1 residents with professional
development to ensure they have the medical knowledge, skills
and support necessary to successfully begin residency
@APGOnews #ChangeResEd
RRR Grant Leadership Team
• Project Members: Banks, Bienstock, Chiang, Cox, Dalrymple, Forstein,
George, Hammoud, Katz, Major, Marzano, Morgan, Ogburn,
Ollendorff, Strand, Winkel, Woodland
• District Representatives: Franz, Karjane, Kilpatrick, Ratan, Tache
• Staff: Georgakis, Wetmore
@APGOnews #ChangeResEd
Workgroups Meetings: Feb 9, 2021
• 7 Students
• 5 Residents
• 38 Program Directors/1 • AAMC GSA/COSA (6)
DIO • 4 AMA/FREIDA
• 7 Program Managers • 1 NBME
• 9 Clerkship Directors • 3 APGO/CREOG/RRR Staff
• 1 Chair • RRR Grant Leadership Team
• 8 Deans
• 5 AAMC/ERAS
@APGOnews #ChangeResEd
How Might We…?
Develop Standards to
Form Residency
the OBGYN Application
Learning Communities
and Interview Processes
Learner
s
Develop a Ready for Patients Develop an Applicant
Intern Year Curriculum Progra Compatibility Index
and Assessments m App
Director
s
Create an Optional Develop additional
Early Result Application Application Review
Program Metrics
Big & Bold Ideas
If not now,
when? If not us,
who?
Standards to the OBGYN Application and
Interview Process (SOAIP)
• Task Leads: Eric Strand, Helen Morgan
• Co-Chairs: Nicole Karjane, David Forstein
• Group Members: Timothy Burns, Tanya Dorman, Natasha Johnson, Erica
Oberman, Sarah Page-Ramsey, Erin Panarelli, Laura Pliska, Chadburn
Ray, Brandi Ring, Ryan Spencer, James Benson, Christle Nwora, Luiz
Gustavo Brito, and Vanessa Rogers
@APGOnews #ChangeResEd
SOAIP: Accomplishments
• Revision of dates/standards to accommodate COVID pandemic
• Updated list of programs participating in the standards
• Published guidelines for students applying into Ob/Gyn
• Publications:
• Stakeholder Perspectives on Standardizing the Residency Application
and Interview Processes
• Can standardized dates for interview offers mitigate excessive
interviewing?
• The Case for Capping Residency Interviews
@APGOnews #ChangeResEdSOAIP: Goals for 2021
• Develop updated standard dates for 2021-22
• Update PD and student surveys
• Post-match review
• Models of interviewing: best practices
• Things to keep
• Things to scrap
• Importance of uniformity
• Feasibility of an early match
• Scholarly activities
@APGOnews #ChangeResEdSOAIP: Breakout Session Recap
Action Items:
1. Finalize standard dates
• Will depend on ERAS/NRMP
timeline Potential Data/ Resources
• Continued need for two offer Needed:
dates?
• Final notification for programs
1. ERAS/NRMP Timeline
2. Update PD and student surveys 2. Increased Student/Resident
3. Specialty interview planning Participation
guidelines 3. Deadline for specialty for in
• Convene PD workgroup (or person vs virtual interviews
SOAIP?) after match day
• Desire for official recommendations
for the specialty
@APGOnews #ChangeResEdSOAIP: DEI Impact
• Standard expectations/communication levels the field
• Virtual interviews(?): relieves financial burdens
• Interview caps(?):
• Shuttles extra interviews from the “haves” (who are very likely to
match) to the “have nots” (who need opportunity)
@APGOnews #ChangeResEdApplication Review Metrics (ARM)
• Task Leads: John Dalrymple, David Marzano
• Co-Chairs: Carol Major, Arthur Ollendorff
• Group Members: Anitra Beasley, Susan Goldsmith, Beverly Gray, Lisa
Pompeo, Sherly Previl, Meaghan Shanahan, Véronique Tache, Sara
Whetstone, Ribal Haddad, Halley Staples, Dayna Burrell
@APGOnews #ChangeResEdARM: Accomplishments
• Defined goals/needs for ARM:
• Developed list of possible Alternative Metrics
• Developed framework:
• Metrics-Attributes-Experiences-Alignment
• Explored options for collecting/displaying applicant data:
• Thalamus/Casper/Freida
• Collaborated with ACI group on manuscript:
• A Model for Exploring Compatibility Between Applicants and Residency
Programs Right Resident, Right Program (Obstet Gynecol 2020). Winkel
AF, Morgan HK, Burk-Rafel J, Dalrymple JL, Chiang S, Marzano D, Major C, Katz
NT, Ollendorff AT, Hammoud, MM
@APGOnews #ChangeResEdARM: Goals for 2021
• Highlighting applicants’ attributes and experiences:
• Determine best way to ID from current application in ERAS/MSPE
• Develop new recommendations for ERAS
• Determine most meaningful/efficient way to display (dashboard)
• Role of Standardized Letter of Recommendation
• Aligning metrics, attributes, experiences with program goals
What applicants want in a
program
What programs want in an
applicant @APGOnews #ChangeResEdARM: Breakout Session Recap
Action Items:
1. SLOE – yes! move forward (?timing for next cycle)
2. Engage Stakeholders: PDs, applicants, letter writers
3. Survey cohort of PDs; LAG; faculty/deans/chairs
4. Vetting process critical; Share a mock-up during process
5. Creation of SLOE + companion guide; platform (centrally administered)
6. Consider pros/cons of pilot
7. ERAS/MSPE (seen as low utility; red flags; doesn’t add much)
8. List top 3 most meaningful experiences and why (list others after that)
9. Tag experiences with key word characteristics (e.g. grit, leadership)
@APGOnews #ChangeResEdARM: DEI Impact
• Strive for greater equity by considering WHAT we assess (e.g. patient
advocacy, leadership, teamwork)
• Address the unconscious bias that exists with current assessments and
grades
• Promote holistic review
@APGOnews #ChangeResEdApplicant Compatibility Index (ACI)
• Task Leads: Seine Chiang, David Marzano
• Co-Chairs: Nadine Katz
• Group Members: Jessica Arluck, Betty Chou, Camille Clare, Melody Hou,
Laura Jacques, Kathy Jorda, Martha Morris, Rini Ratan, Neda Etezadi,
Kavitha Ram, Peter Schnatz, Jesse Rafel, Patricia Franz, Elizabeth
Southworth, Paul Lemen
@APGOnews #ChangeResEdACI: Accomplishments
• Developed a conceptual framework with specific domains to
break down the concept of “compatibility” between OBGYN residency
applicants and programs to improve the application AND selection
process
• Reviewed current tools and data sources and met with
stakeholders: AAMC Residency Explorer, AAMC ERAS, NRMP, AMA
FREIDA, APGO Residency Director, Thalamus, CASPer Situational
awareness summarized on spreadsheet.
• Green Journal publication with ARM group: A Model for
Exploring Compatibility Between Applicants and Residency
Programs. Obstetrics & Gynecology: January 2021 - Volume 137 -
Issue 1 - p 164-169. Winkel, AF; Morgan HK; Burk-Rafel J; Dalrymple, JL;
Chiang S; Marzano D; Major C; Katz NT; Ollendorff AT; Hammoud MM
@APGOnews #ChangeResEdACI: Goals for 2021
• Finalize the compatibility framework and key elements under each
domain for applicants and programs
• Computerized modeling to generate “compatibility index” for
applicants (ACI) which aligns applicants’ and program’s priorities.
• ACI would need both to prioritize/rank attributes, experiences, values
• Likelihood of applicant interviewing AND matching in a specific program using
outcomes data from prior NRMP results
• Develop guidelines on use of ACI data for career counseling:
• Casting a more realistic rather than just a wide “net”
• Reducing number of residency applications, allowing for holistic review
Define and prospectively track the elusive “success” and “compatibility” in
residency
@APGOnews #ChangeResEdACI: Breakout Session Recap
• Input by programs: Increase transparency as to program values (rubric),
standardize, and have one source of information
• Input by students: Students enter their data and prioritize attributes,
experiences, and values they seek in a program
• Output of the ACI:
• For the students (applicant facing): Very compatible, Compatible, Not compatible based
on algorithm driven by annual entry by programs (rank the applicant attributes they are
seeking) and program attributes the applicant is seeking (prioritize but modifiable)
• Convene a half-day retreat to boil down attributes into something
quantifiable.
@APGOnews #ChangeResEdACI: DEI Impact
• ACI tool may help level the field for applicants who may not have access
to robust OBGYN advising.
• Usefulness as a self-reflection tool where data would be individualized
• Program data/recruiting priorities would be available to all applicants
• Less reliance in visiting programs and associated costs
• De-emphasize academics as primary criteria and shift towards diversity, individual
attributes/strengths
• Allows applicants to “cast” a more realistic rather than just a wide “net”,
as applicants can see what programs value and align their selection of
programs with their own values.
• Ultimately allowing those who “have not” to have more (particularly if
there is an interview cap)
@APGOnews #ChangeResEdEarly Result Application Program (ERAP)
• Task Leads: Abigail Winkel, Maya Hammoud
• Co-Chairs: Jessica Bienstock, Mark Woodland
• Group Members: Erika Banks, Mary Jane Mangan, Erin Nelson, Kimberly
Pilkinton, Jody Stonehocker, Nanci Wieneke, Katherine Chen, Isaiah
Cochran, Susan Alkasab, Bri Tristan, Elaine Hart, Ali Bokhari, Andrea
Bodine
@APGOnews #ChangeResEdERAP: Accomplishments
• Background: ongoing efforts in other specialties/program to work around
issues with application process
• NRMP ERAP Retreat 1 (Dec 2020) and Retreat 2 (Feb 2021)
• Socialization of ERAP concept and stakeholder discussions
• Consensus emerging for ERAP:
• Early decision
• Limited applications and virtual interviews
• Results before regular applications open
@APGOnews #ChangeResEdERAP: Goals for 2021
• Understand stakeholder perspectives re: ERAP
• Define what an early application looks like
• Clarify timelines, # of applications/interviews/offers for ERAP
• Consider simulated early match (timeline/parameters)
@APGOnews #ChangeResEdERAP: Breakout Session Recap
Action Items:
1. Define ERAP application: SLOE,
standardized personal Potential Data/ Resources
statements, standardized Needed:
assessments like situational
judgment tests 1. Stakeholder survey data,
match data
2. Parameters: 1-3 applications, up
to 50% of slots, limited
interviews 2. Work group to meet next in
April to flesh out further and
3. Define Timeline: August - Early propose simulation
September
@APGOnews #ChangeResEdERAP: DEI Impact
Opportunities:
1. Less reliance on Sub-Is which
threaten equity Threats:
2. Opportunities to standardize
1. Increased reliance on metrics
application process
like AOA, clerkship grades,
3. Fewer applications encourages NBME scores
holistic review
@APGOnews #ChangeResEdReady for Intern-Year Curriculum and
Assessments (RICA)
• Task Leads: Karen George, Abigail Winkel
• Co-Chairs: Susan Cox
• Group Members: Kimberly Gecsi, Charles Johnson, Carey Mcdonald,
Kristiina Parviainen, Colleen Stockdale, Greg Ward, Natalie Bowersox,
Kindzierski John, Trisha Pederson, Sarah Wagner, Anita Malone, Baillie
Bronner, Abigail Wolf, John Kindzierski, Etoi Garrison, Sarah Shaffer,
Anna White
@APGOnews #ChangeResEdRICA: Accomplishments
• Thoughtful contemplation ☺
• Pilot “Wellness Milestone 0”
• Decision to Expand “Wellness Milestone 0” à “Readiness for
Residency” to include:
• Resilience and Gratitude
• Time management and priorities
• Mission/values/of residency
• Evolving your learning skills
• Collaboration with the Resident Learning Communities Group (RLC)
@APGOnews #ChangeResEdRICA: Goals for 2021
• Readiness for Residency curriculum
• Complete 2 new modules
• Pilot new version with incoming intern cohort (March thru July)
• Align with RLC to “coach the coaches/facilitators”
• IRB exemption for survey
• Evaluate and report
• Needs assessment of incoming class, PDs, senior residents
@APGOnews #ChangeResEdRICA: Breakout Session Recap
Action Items:
1. Resurvey interns who went through original
wellness pilot
2. FB from work group on the “readiness
curriculum”
Potential Data/Resources
3. Create survey for ”readiness curriculum” Needed:
PDs/PMs/interns
4. Create self-assessment checklist (based on 1. Partner with AMA on #4
milestone 0) for matched students (strength finder
and learning goals) 2. IRB exemption on surveys
in #3
5. Create toolkit with readiness resources and best
practices for delivery for matched students &
faculty
6. Faculty Development Webinar for PDs who trial
updated curriculum with incoming residents
@APGOnews #ChangeResEdRICA: DEI Impact
• Self assessment tool designed to point out potential gaps to allow for
post-match self-directed learning prior to start of residency
• Initial curricular materials are designed to build confidence and
readiness for internship and build communities of trust and connection
with new cohort
@APGOnews #ChangeResEdResident Learning Communities (RLC)
• Task Leads: Tony Ogburn, Helen Morgan
• Group Members: Sharon Byun, Julia Cron, Nicole Donnellan, Tracy
Foote, Charles Kilpatrick, Christine Kim, Kelli Krase, Natalie Moore, Sara
Petruska, Sloane York, William Gist, Rodrigo Munoz
@APGOnews #ChangeResEdRLC: Accomplishments
• Needs assessment of learners for Learning Communities
• IRB exemption obtained/Survey developed/participants
recruited/group sessions scheduled à Attendance was minimal
• Regrouped: Now planning 1:1 interview sessions with LAG members
and survey of PDs
• Coaches as integral component of Learning Communities
• Planning underway for pilot project
• Members of RICA/RLC workgroups
• Formal coaching training
@APGOnews #ChangeResEdRLC: Goals for 2021
1. Complete needs assessment from stakeholders – learners, PDs
2. Develop foundations for implementation of coaches
• Identify incentives
• Training
3. Initiate pilot project for Learning Communities/Coaches
4. Address issues of equity in development of RLCs
@APGOnews #ChangeResEdRLC: Breakout Session Recap
Action Items:
1. Learner survey – 1:1 interviews – completed by
June 2021
Potential Data/Resources
2. PD and Program Manager Survey – July 2021 Needed:
3. Intro to Coaching training – RLC/RICA groups –
Consider this Spring for RICA pilot this year, or at 1. Time at CREOG retreat for
CREOG retreat 2021 PD “survey”
4. Develop pilot for program learning communities –
implement Spring 2022 2. Support for Intro to
Coaching training
5. Consider learning communities for affinity groups
outside of home program
@APGOnews #ChangeResEdRLC: DEI Impact
• Ease transition for all residents by building a community among
incoming interns prior to immersion in clinical work
• Identify challenges specific to URM students and implement strategies
to address these challenges to facilitate transition for these students
• Consider development of internal (department/institutional) and
external affinity group communities to address diverse needs
@APGOnews #ChangeResEdLearner Advisory Group (LAG)
• Task Leads: Taylor Standiford
• Group Members: Christle Nwora, Hailey Staples, Isaiah Cochran, Jesse
Rafel, Luiz Gustavo Brito, Ribal Haddad, Rodrigo Munoz, Sally Salari,
Baillie Bronner, Sarah Frank, Mark Mullen, Oluwabukola Akingbola
@APGOnews #ChangeResEdLAG: Accomplishments
• 2 new members added
• 5 publications in high impact journals
• Panelist at the NRMP Transition to Residency Conference
• AMA funded $20,000 planning grant on developing a national platform
for preference signaling
• Meaningful learner insight on all RRR grant initiatives
@APGOnews #ChangeResEdLAG: Goals for 2021
Integrate and orient new LAG
members to goals of organization
Diversify learner stakeholder
perspective
Design and implement innovative
initiatives aimed to improve the
residency selection process
@APGOnews #ChangeResEdLAG: DEI Impact
• At the core of the LAG is to represent a diverse body of learner
perspectives to advocate all types of learners:
• Underrepresented minority students
• International medical graduates
• Osteopathic students
• Non-OBGYN learners
• Learners from all tiers of medical schools and residency programs
• Representation from the AAMC, AMSA, AMA MSS
@APGOnews #ChangeResEdAdditional Accomplishments
• Stakeholders Surveys
• Response to COVID Guidelines
• Webinar Series (4 for applicants, 3 for PDs
and PMs)
• RRR Program Directors Listserv
• RRR Website
• RRR App
• Conversations with other specialties
@APGOnews #ChangeResEdAdditional Accomplishments
National Presentations: Numerous Publications
• CAAM Feb 2020
• APDS April 2020 • 2 JAMA
• CREOG Educational Retreat July • 3 Academic Medicine
2020
• 3 Obstetrics and Gynecology
• AMA GME Innovations Summit
• 4 Journal of Surgical
October 2020
Education
• NRMP Transition into Residency
• 1 Journal of Graduate
October 2020
Medical Education
• AADPRT coming March 2021
@APGOnews #ChangeResEd2021 Recipients of the RRR Mini-Grants for Diversity, Equity and Inclusion • Bridging the Advising Gap by Utilizing a National Network of Faculty to Enhance Equity in the Residency Selection Process in Obstetrics and Gynecology. Jasmin Capellan, MS1, Kristen Morrill, MS1, Sheila Panez, MS2, Jared Trimarchi, MS1, Archana Pradhan, Neda Etezadi-Amoli, Christine Kim, and James Cook. Rutgers Robert Wood Johnson Medical School, University of Nevada, University of California, University of South Carolina-Columbia. • Optimizing the UME to GME Liminal Space: Identification of Diversity, Equity,and Inclusion Competencies as Core Entrustable Professional Activities. Kavya Davuluri, Samantha Lyons, Brandon Ellsworth, Aliya Moreira, Erica Jaiyeola Odukoya, MPH, Keli Santos-Parker, PhD, Keerthi Gondy, Marcia Perry, MD, Michael Gisondi, MD, Dowin Boatright, MD, MBA, and Laura Hopson, MD. The University of Michigan Medical School, Stanford University School of Medicine, Yale University School of Medicine. • Developing a Values Matching Program in the Residency Application Process. Jonathan Hunt, MD, MBA and Diane Young, MD. The Cleveland Clinic Foundation. @APGOnews #ChangeResEd
Action Item Timeline
March
2022
CAAM
July
2021
CREOG
PD
Retreat
Toda
yObjectiv Implement changes to the residency application and processes
e1 to decrease the number of residency applications needed for a
successful Match
Standards to the 1 Application Review
OBGYN Application Metrics to
and Interview Encourage Holistic
Processes Review
Standardizat
Alignment
ion
@APGOnews #ChangeResEdObjectiv Implement changes to the residency application and processes
e1 to decrease the number of residency applications needed for a
successful Match
Applicant 1 Optional Early
Compatibility Index Result Acceptance
Program
Disruptive
Compatibility
Change
@APGOnews #ChangeResEdDevelop a national curriculum to prepare learners for PGY-1
Objectiv
e2 OBGYN milestones and engage them in learning communities
to ensure readiness for residency
Ready for
Intern Year Residency 1
Curriculum Ready Day
Learning
and One
Communities
Assessments
@APGOnews #ChangeResEdAnalysis of current practices, Development of standards, Stakeholder input
Pass/
Pilot implementation
Fail
Implementation of SOAIP
Establish and implement FRIEDA priorities
Pilot implementation with medical student opt-in
Presentation at 7/20 CREOG PD Retreat
Develop additional metrics Implementation of ARM ERAS
Review ERAS data, Survey PDs and students, App development
Presentation at 3/21 CAAM**
Pilot within grant team programs*
Implementation in 6/22 ERAS
Presentation at 3/21 CAAM** Presentation at 3/22 CAAM**
Development of ERAP Guidelines
ERAP Pilot
Implementation in 6/22 ERAS
Needs assessment and Curriculum Development
Available to all incoming PGY1s
Strongly encouraged of all incoming PGY1s
Development of Coaching Competencies, Recruitment and Training of Pilot coaches
Pilot of coaching and RLCs within grant team residency programs*
Coaches School 3/22 CAAM**
Pilot Coaches School 3/21 CAAM**
Implementation
Revised Timeline 4/2020 Development
Phases
Pilot Full implementationWe choose to do things not
because they are easy, but
because they are hard.
- JFKThank you!
You can also read
