COMMON ACGME CITATIONS AND HOW TO AVOID THEM WORKSHOP - MARIANNE CHALOUPEK, MHA TASHA GRAHAM, MBA LYNNE MEYER, PHD, MPH AUGUST 13, 2020

Common ACGME Citations
  and How to Avoid Them
              Workshop
           Marianne Chaloupek, MHA
                Tasha Graham, MBA
             Lynne Meyer, PhD, MPH

                    August 13, 2020

Disclosures

• None to report

Workshop Objectives

Goal: Gain understanding of common AGME citation areas and properly design your
program to avoid citations.

Objectives:
•   Identify common citations given by the ACGME
•   Identify common “types” of citations
•   Discuss, identify and implement strategies to prevent common citations

What is a Citation?

•   A citation is determined when the ACGME Review Committee (RC) finds that a
    Sponsoring Institution or program has failed to comply substantially with a
    particular accreditation requirement.

•   If no citations were identified, the Areas Not in Substantial Compliance Section will
    include a statement of commendation to the program and Sponsoring Institution
    for demonstrating substantial compliance with the accreditation requirements.

Non‐Compliance versus Improvement
       Citations                           VS      AFI = “Areas for Improvement”
•   Each citation requires a separate, specific   • Address AFIs in “Major Changes and
    response in ADS (use the PDSA cycle
    format in citation response)                    Other Updates” section in ADS .
•   Programs must submit response to the          • RCs assume concerns will be
    GME office for review prior to submitting       reviewed and addressed for
    response in ADS.
                                                    improvement
                                                  • AFI trends may draw further scrutiny
                                                    or become citations.

Common UF Citations* by ACGME Categories
Categories with none to few       • Evaluations
Citations                           •   Program*
• Institutional Support             •   Faculty*

• Resident Appointment              •   Resident*

• Experimentation and             • The Education Program
 Innovation                         •   Patient Care Experience*
                                    •   Procedural Experience*
Categories with Common
Citations                           •   Service to Education Imbalance*
                                    •   Scholarly Activities*
• Program Personnel & Resources     •   Supervision*
  •   PD Responsibilities*
                                    •   80 Hours per Week*
  •   Faculty Responsibilities*

Unofficial “Types” of Citations by Root Cause
• Unclear (e.g. vague and/or conflicting information, response
 did not address question asked or partially addressed, citations
 may start with….“it is not apparent that…..”)
• Administrative (e.g. expired licenses, outdated publications,
 missing data, etc.)
• Educational Design/Components/Setting (data is
 clear however ‐‐‐ issues with areas such as survey data, duty
 hours, ability to raise concerns, evaluations, resources, etc.)

What are Program Director Responsibilities?

Can be found in Common Program Requirements

It is the Program Directors responsibility to communicate
with the residents, faculty members, DIO, GMEC and
ACGME, including accuracy of ADS updates
     • PDs are responsible for ADS content and assuring
        compliance with program requirements
     • PDs provide the Coordinators with information to
        enter into ADS.

Responsibilities of Program Director Citations

     Program Requirement: II.A.4.a).(8) …accurate and complete information…..

Program 1: ………Specifically, Dr. XXXXX Florida license is listed incorrectly as expired
and needs to be updated.
Program 2: ……….The application as submitted to the Review Committee contained
many errors including length of non‐surgical rotations, publications of Dr. XXXXX,
certification errors on the Faculty Roster, length of the night float rotation and
identification of other learners. The Review Committee expects the program to correct the
errors in ADS in a timely fashion.

           “Administrative” citation examples……

Qualifications of Faculty Citations

   Program 1
   Faculty Certification [Program Requirement II.B.3]
   The physician faculty must have current certification in the subspecialty by the American Board
   of XXXXXXXX, or possess qualifications judged acceptable to the Review Committee. (Core)

   …Specifically, on review of the physician faculty roster, the Committee noted that 2 of the 6 core
   faculty are not ABXX certified.

   Program 2
   Subspecialty Faculty Program Requirement II.B.9.a).(1)
   Subspecialty faculty should include a faculty member in the xxxxx subspecialty area of
   xxxxxxxxxxx medicine. (Core)

   The Faculty Roster did not identify a faculty member who is board‐certified in XXXXXXX
   medicine.

                       Was the information not entered ? i.e. “administrative”
                                                  OR
             Do you need an action plan to recruit needed faculty? “program component”

Mini Activity

• 1 minute: Type at least one idea to prevent
 “administrative” citations using the “Chat”
 function
• 4 minutes: large group discussion on
 examples/lessons learned

Follow‐Up Discussion

• 1 minute: Type at least one idea to prevent
 “administrative” citations using the “Chat”
 function
• 4 minutes: large group discussion on
 examples/lessons learned

TIPS to avoid the “We should have caught that” or
             “administrative” citations:

Print your ADS draft.

Some answers prepopulate –check for accuracy.
TIP: Where is the section I need to edit? (ADS: Summary Tab: Edit
Program Information ‐‐ will take you directly to the correct section)

If information on the same topic is asked in more than one place,
ensure responses are consistent.

Use reviewers who are detail‐oriented and familiar with the program.

Responsibilities of Program Director Citations

Program 1
[Program Requirement II.A.3]
……administer and maintain an educational environment
conducive to educating the fellows…..

……..Specifically, on review of the program policies and
procedures provided in the application, the Committee noted
that none were provided for patient safety, professionalism,
well‐being, fatigue mitigation or care transitions.
     Educational Design/Components/Setting example….

Responsibilities of Program Director Citations

Program 2
Program Requirement: II.A.4.a).(10)
…….provide….environment in which residents have the opportunity to
raise concerns and provide feedback in a confidential manner…..
without fear……

It is not apparent that the program director provides an environment in
which residents can raise concerns and provide feedback without fear and
intimidation. The ACGME Resident Survey indicated that XX percent of
residents were not satisfied with the process to deal with problems and
concerns, and XX percent of residents did not feel they could raise
concerns without fear.

 Educational Design/Components/Setting example….

TIPS to avoid a citation in “The program director must provide a learning
  and working environment in which residents have the opportunity to
    raise concerns and provide feedback in a confidential manner as
        appropriate, without fear of intimidation or retaliation.”

  Policy
      1. Include algorithm/flow chart of communication channels

     2. Explain yearly to residents and faculty
         a. Ensure residents understand policy and resources available
         b. Review more often if breakdown with the established policy
            occurs.

     3. Solicit feedback on the policy

     4. Review policy with PEC annually

Responsibilities of Faculty Citations

           Program 1
           Personnel‐Faculty/Interest, Sufficient Time PR: II.B.2.c‐e)
           …demonstrate a strong interest in the education of residents.
           Faculty …must… fulfill their supervisory and teaching
           responsibilities.
           Faculty…. maintain an educational environment….

           The Resident Survey indicated that XX percent of residents did
           not feel they had sufficient supervision or the appropriate level
           of supervision, and XX percent felt they did not receive
           sufficient instruction, nor that faculty and staff were interested
           in residency education or created an environment of inquiry.

           Educational Design/Components/Setting example….

TIPS to avoid a citation in “Personnel‐Faculty/Interest, Sufficient Time,
   and Educational Environment Program Requirements: II.B.2.c‐e”
 Faculty members must demonstrate a strong interest in the education of residents. (Core)
 Faculty members must devote sufficient time to the educational program to fulfill their supervisory and
  teaching responsibilities. (Core)
 Faculty members must administer and maintain an educational environment conducive to educating
  residents. (Core).”

1. Internal evaluations
    1. Resident evaluation of Faculty
    2. Resident evaluation of Program
    3. Faculty evaluation of Program
    4. Special topic surveys can be distributed through Qualtrics
2. Annual Faculty reviews
    1. Faculty expectations

    ****Identifying negative trends through internal evaluations/surveys will allow programs/PEC to rectify
    problems prior to ACGME surveys or site visits.****

UF GME websites: Types and Timing of GME Training Program Evaluation Forms & Curriculum Basics

Responsibilities of Faculty Citations

Program 2
Responsibilities of Faculty II.B.5.
…establish and maintain an environment of inquiry and scholarship with
an active research component. (Core)

….Specifically, there is limited faculty scholarship. Limited faculty scholarship
results in inadequate opportunities for resident scholarly activities as
demonstrated by no resident publications in peer‐reviewed journals or books,
and only one presentation.

       Was the information entered correctly? “administrative”
                          OR
   Do you need an action plan to increase scholarly activity? “program component”

Qualifications of Program Director

Program 1
Scholarly Activity by the Program Director Program Requirement II.A.3.d)
Qualifications of the program director must include a record of ongoing involvement in
scholarly activities, including peer‐review publications and mentoring (i.e., guiding fellows
in the acquisition of competence in the clinical, teaching, research, and advocacy skills
pertinent to the discipline). (Core)

Although the program director is required to have a record of scholarship, according to the
information in the program’s application, the program director has only two presentations and
two peer review publications and very limited evidence of other scholarship in the last five
years

               Was the information entered correctly? “administrative”
                                   OR
           Do you need an action plan to increase scholarly activity? “program component”

Mini Activity

• 5 minutes: Discuss at least one idea to prevent
 “Educational Design/Components/Setting” citations –
 e.g. tracking and/or increasing scholarly activity using
 the assigned Breakout Room
• 5 minutes: report out from 1‐2 groups on best
 practices or lessons learned

Follow‐Up Discussion

• 5 minutes: Discuss at least one idea to prevent
 “Educational Design/Components/Setting” citations –
 e.g. tracking and/or increasing scholarly activity using
 the assigned Breakout Room
• 5 minutes: report out from 1‐2 groups on best practices
 or lessons learned

TIPS to avoid a citation in “The faculty must establish and maintain
 an environment of inquiry and scholarship with an active research
                             component.”
 Discuss scholarly activity with faculty and residents/fellows

      1. Define what is considered scholarly activity (research, grants, QI, curricula, etc.
          • Download Templates for Scholarly Activity in ADS for both faculty and residents

      2. Create a succinct method to report scholarly activity

      3. Require specific types/amounts of scholarly activity in faculty expectations and in
         resident curriculum

      4. Ask Faculty and residents to update CV’s annually to include scholarly and QI activity

      5. Request faculty and residents to review their ADS reporting
         of scholarly activity prior to submission

Scholarly Activity
www.statpearls.com
StatPearls includes a growing list of free, online, peer‐reviewed and PubMed‐indexed reviewed
activities and affordable adaptive learning questions with explanations.
The goal of StatPearls to provide health professionals with the educational tools needed to identify
weaknesses and stay current.

Residents/Fellows meet requirements to be Authors.
Minimum Time Requirement: Articles completed in 3 to 20 hours depending on the topic selected.

Authors write a review article summary of 1,000 words or more and create 5 to 10 second or third order
questions with explanations. Articles are published and indexed in PubMed in about 30 days.

NI Home Page: directions on how to track
           scholarly activity

Resource Citations

 Program 1
 Resources/Patients [Program Requirement II.D.1]
 …sufficient number of new and follow‐up patients must be
 available…..

 …... Specifically, on review of the specialty‐specific application the
 Committee noted that a very old and outdated form was used
 (dated 2013). The form contains the institutional operative data
 table that is also in the newest application form (dated 2017) but
 does not contain the defined case category table. Therefore, the
 clinical resource data was incomplete.

TIPS to avoid a citation in “The program must provide a sufficient volume and
variety of patients to ensure that resident gain experience in the full range of
examinations, procedures, interpretations, outpatient clinic visits, and
inpatient consultations on.”

    1. Ensure number provided in ADS are factual and consistent throughout the
       document.

    2. Review Case log numbers for accuracy.
        • Discuss numbers with residents if numbers are much higher or much lower than
           expected.

Was the information asked for in different sections and with discordant data provided?
Was an outdated form used? “administrative”
                        OR
Do you need an action plan for patient volume & diversity? “program component”

Summary
• AFIs: Keep tabs and address so they don’t become citations
• Identify root cause of the citation
   •   Unclear citations are a result of vague answers or discordant information
   •   Administrative citations are easily addressed.
   •   Educational Design/Components/Setting citations usually need a plan
• PDs are responsible for content of ADS
• Citation responses address each component of the citation and
  demonstrate compliance with requirements
• Pay attention to ACGME survey results
• Details, details, details…..

Questions?

Find these slides here:
https://gme.med.ufl.edu/graduate‐
medical‐education/gme‐office‐
faculty‐and‐staff‐
development/gme‐cme‐series/