SUMMARY OF CHANGES TO THE 2018 LEAPFROG HOSPITAL SURVEY & RESPONSES TO PUBLIC COMMENTS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
SUMMARY OF CHANGES TO THE
2018 LEAPFROG HOSPITAL SURVEY &
RESPONSES TO PUBLIC COMMENTS
PUBLISHED MARCH 23, 2018
Each year, The Leapfrog Group’s team of researchers, in conjunction with the Armstrong Institute for Patient Safety and
Quality at Johns Hopkins Medicine, review literature and convene national expert panels to ensure the Leapfrog Hospital
Survey aligns with the latest science as well the public reporting needs of purchasers and consumers. We assemble a list of
proposed changes for the next year’s Survey and release those changes for a 30-day public comment period. Comments are
reviewed carefully and used to further refine the Survey. The Survey is then pilot tested with a diverse group of hospitals
across the country. Following the pilot test, Survey content and scoring are finalized for launch on April 1.
Leapfrog received over 150 public comments in response to its proposed changes for the 2018 Leapfrog Hospital Survey.
Those comments, as well as the results from the pilot test, were incorporated into the final content and scoring algorithms
for the Survey. We have summarized the changes in this document, and included summaries and responses to public
comments in Appendix I.
We offer our sincere gratitude to all commenters for the time and thought they gave to the 2018 Leapfrog Hospital
Survey. The submitted comments were invaluable to the development of a high-quality Survey that serves our many
constituents, including purchasers and payers, as well as hospitals and the public at large.
The 2018 Survey will open on April 1, 2018. Leapfrog has already scheduled a number of informative Town Hall Calls.
Hospitals and other stakeholders can register on the Town Hall Calls webpage.
1CONTENT AND SCORING CHANGES SECTION 1: BASIC HOSPITAL INFORMATION To ensure accurate reporting of pediatric admissions, Leapfrog updated the endnote describing the criteria for pediatric admissions to include pediatric admissions (i.e.
SECTION 3A: HOSPITAL AND SURGEON VOLUME STANDARD
Based on minimum hospital and surgeon volume standards first published in 2015 by researchers at Dartmouth-Hitchcock
Medical Center, Michigan Medicine (University of Michigan) and Johns Hopkins Medicine, as well as information collected
in 2017 from the Leapfrog Hospital Survey, peer-reviewed literature, and consultation with national experts, Leapfrog is
implementing the following hospital and surgeon volume standards for 2018:
Procedure Hospital Volume Surgeon Volume
(minimum per 12-months or 24- (minimum per 12-months or 24-
month annual average) month annual average)
Bariatric surgery for weight loss 50 20
Esophageal resection for cancer 20 7
Lung resection for cancer 40 15
Pancreatic resection for cancer 20 10
Rectal cancer surgery 16 6
Carotid endarterectomy 20 10
Open abdominal aortic aneurysm repair 15 10
Mitral valve repair and replacement 40 20
Hospitals will be asked to report on their total hospital volume over a 12-month period or their annual average over a 24-
month period based on updated procedures and diagnosis codes that include several recommendations from participating
hospitals and health systems. Hospitals can download an Excel Workbook which includes the list of procedure and diagnosis
codes on the Survey and CPOE Materials webpage beginning April 1.
In addition, the list of high-risk procedures has been reduced from ten to eight. Leapfrog has eliminated total hip and total
knee replacement for the 2018 Survey to allow for additional time in finalizing the recommended hospital and surgeon
volume standards.
We are removing the individual surgeon volume questions due to the challenges hospitals reported in obtaining accurate
volume data on surgeons that perform that surgery at multiple facilities. Instead, hospitals will be asked whether their
process for privileging surgeons requires that the surgeon meet or exceed the minimum surgeon volume standards listed in
the table above. See the updated questions below:
1) Check all procedures that your hospital performs as defined Carotid endarterectomy
in the Inpatient Surgery Reference Information. Mitral valve repair and replacement
Open abdominal aortic aneurysm repair
If your hospital does not perform the procedure, or ONLY does Lung resection for cancer
so when a patient is too unstable for safe transfer, OR ONLY Esophageal resection for cancer
when a procedure is urgent or emergent, do not check the box Pancreatic resection for cancer
next to that procedure. Rectal cancer surgery
Bariatric surgery for weight loss
If “None of the above,” please skip remaining questions in None of the above
Section 3A and 3B, and go to the Affirmation of Accuracy.
Hospitals will only respond to questions #2 and #3 based on the procedures selected in question #1.
2) Total hospital volume for each selected procedure during the reporting period:
3Number of Procedures Performed
Procedure
(12-month count or 24-month annual average)
Carotid endarterectomy
Mitral valve repair and replacement
Open abdominal aortic aneurysm repair
Lung resection for cancer
Esophageal resection for cancer
Pancreatic resection for cancer
Rectal cancer surgery
Bariatric surgery for weight loss
3) Does your hospital’s process for privileging surgeons include the surgeon meeting or exceeding the minimum
surgeon volume standard listed below?
Procedure Surgeon Volume Standard
Yes
Carotid endarterectomy 10 No
Plan to implement within 12 months
Yes
Mitral valve repair and replacement 20 No
Plan to implement within 12 months
Yes
Open abdominal aortic aneurysm repair 10 No
Plan to implement within 12 months
Yes
Lung resection for cancer 15 No
Plan to implement within 12 months
Yes
Esophageal resection for cancer 7 No
Plan to implement within 12 months
Yes
Pancreatic resection for cancer 10 No
Plan to implement within 12 months
Yes
Rectal cancer surgery 6 No
Plan to implement within 12 months
Yes
Bariatric surgery for weight loss 20 No
Plan to implement within 12 months
Scoring Algorithm for the Minimum Hospital and Surgeon Volume Standard
Hospital and Surgeon
Volume Standard Score For each of the surgeries performed by the hospital…
(Performance Category)
Fully Meets the Standard The hospital met the minimum hospital volume standard for the
(four-filled bars) surgery; and
4 The hospital’s process for privileging surgeons includes meeting or
exceeding the minimum surgeon volume standard.
Substantial Progress The hospital met the minimum hospital volume standard for the
(three-filled bars) surgery; and
The hospital’s process for privileging surgeons does not include meeting
or exceeding the minimum surgeon volume standard, but the hospital is
committed to doing so within the next 12 months.
Some Progress The hospital did not meet the minimum hospital volume standard for
(two-filled bars) the surgery, but the hospital’s process for privileging surgeons includes
meeting or exceeding the minimum surgeon volume standard; OR
The hospital met the minimum hospital volume standard for the
surgery, but the hospital’s process for privileging surgeons does not
include the minimum surgeon volume standard, and the hospital is not
committed to doing so within the next 12 months.
Willing to Report The hospital did not meet the minimum hospital volume standard for
(one-filled bar) the surgery; and
The hospital does not include the minimum surgeon volume standard in
its privileging policy, whether or not they are committed to doing so in
the next 12 months.
Does Not Apply The hospital does not perform the surgery.
Declined to Respond The hospital did not respond to the questions in this section of the survey or did
not submit a survey.
SECTION 3B: SURGICAL APPROPRIATENESS
Questions in this section focus on the hospital’s progress in developing appropriateness criteria based on published
guidelines and input from local surgeons, supporting and monitoring adherence, as well as communicating with surgeons,
hospital leaders, and board members about adherence to the criteria.
In 2018, responses to this subsection will be publicly reported but not scored by Leapfrog. When visitors to Leapfrog’s
public reporting website click the score icon for the surgical volume standard (i.e. four filled bars, three filled bars, etc.),
they will see a statement indicating whether the hospital has processes and protocols in place to ensure surgical
appropriateness. Hospitals that respond “Yes” to all five questions specific to that surgery will be reported as “Yes” and
hospitals that respond “No” to one or more of the five questions will be reported as “Not Yet.” This approach aims to
encourage hospitals to continue implementing processes and protocols to ensure surgical appropriateness while giving
them additional time before the responses are used in scoring, planned for 2019.
SECTION 4: MATERNITY CARE
There are no changes to the questions in this section.
However, there are important updates to the data specifications. For hospitals that do not submit data to The Joint
Commission (TJC) and need to retrospectively collect data using the TJC specifications provided, two of the three TJC
measures included in Section 4, Early Elective Deliveries (PC-01) and NTSV C-sections (PC-02), will use multiple TJC measure
specifications based on the discharge dates of included cases due to updates between each version:
5 v2016B1: Discharges between 01/01/2017 – 06/30/2017
v2017A1: Discharges between 07/01/2017 – 12/31/2017
v2017B1: Discharges between 01/01/2018 – 06/30/2018
Please be sure to refer to the correct specification manual for the discharge date if using TJC measure specifications as
there have been updates to the ICD-10 tables and data elements between releases, including the addition of “history of
stillbirth” in v2017A1 and v2017B1 as an exclusion for Early Elective Deliveries (PC-01).
The Antenatal Steroids (PC-03) measure will only use one set of TJC measure specifications (v2017A1) for both reporting
periods since there were no updates between the releases.
SECTION 5: ICU PHYSICIAN STAFFING (IPS)
Leapfrog has made minor updates to the wording of some of the questions and response options in Section 5 ICU Physician
Staffing to better understand hospitals’ use of tele-intensivists.
In addition, in 2018, Questions #7 and #8 will not be included in the scoring criteria for the “Some Progress” (i.e. two-filled
bars) performance category. This change will ensure consistency between responses to Question #3, which asks if all
patients in these ICUs are at any time managed or co-managed by one or more physicians certified in critical care medicine
either on-site or via telemedicine, and Questions #7 and/or #8, which asks how much time patients are managed or co-
managed by intensivists either on-site or via telemedicine.
Please see Appendix III for the 2018 IPS Scoring Algorithm.
SECTION 6: NQF SAFE PRACTICES SCORE
Due to the absence of national training/educational opportunities to support managers in integrating risk and hazard
information, Leapfrog is removing Safe Practice element 4.3.c, which is listed below, from Safe Practice 4 Risks and Hazards.
No updates to the scoring or weight for Safe Practice 4 Risks and Hazards are proposed.
In regard to developing the ability to appropriately assess risk and hazards to patients, the organization has done the
following or had in place during the last 12 months:
Senior managers have received training in the integration of risk and hazard information across the organization.
Training was documented. (pp. 107-108)
Instructions for reporting on Safe Practices in Section 6 have been updated to include information about collecting key
documentation to support each answer, as Leapfrog’s verification process includes a review of safe practices
documentation from a random selection of hospitals every month during the survey cycle (April 1 to December 31).
Leapfrog would like to thank those organizations that provided feedback on the addition of two unscored, fact-finding
questions in Safe Practice 19 Hand Hygiene related to the use of electronic hand hygiene and/or video monitoring systems.
Leapfrog will not be adding these two unscored questions to Safe Practice 19 in 2018. We are convening a national expert
panel to develop an evidence-based structural measure that represents the best practices in hand hygiene monitoring and
compliance and envision this new measure replacing Safe Practice 19 in the future.
6SECTION 7: MANAGING SERIOUS ERRORS
SECTION 7A: NEVER EVENTS POLICY STATEMENT
Based on comments received during the public comment period, Leapfrog has made two minor updates to Section 7A
which were not originally proposed. First, we will update the wording of each practice statement to indicate fact rather
than intent. For example, the current policy states that “we will apologize to the patient and/or family effected by the never
event” and will be updated to state, “we apologize to the patient and/or family effected by the never event.” Next, we have
updated the number of days that hospitals are required to report the never event from 10 days to 15 business days. This is
consistent with the Minnesota Department of Health’s Adverse Event Reporting Systems, which has been cited by experts
as a national model for reporting adverse events. More information can be found here.
Leapfrog will implement the following scoring algorithm for Section 7A Never Events which includes the four (4) new
principles added in 2017. These results will be publicly reported in 2018.
Never Events Score Description
(Performance Category)
The hospital has implemented a policy that adheres to all 9
Fully Meets the Standard
principles of The Leapfrog Group’s Policy Statement on Serious
(four-filled bars)
Reportable Events/“Never Events.”
The hospital has implemented a policy that adheres to all of the
Substantial Progress original 5 principles of The Leapfrog Group’s Policy Statement on
(three-filled bars) Serious Reportable Events/“Never Events,” as well as at least 2
additional principles.
The hospital has implemented a policy that adheres to all of the
Some Progress
original 5 principles of The Leapfrog Group’s Policy Statement on
(two-filled bars)
Serious Reportable Events/“Never Events.”
The hospital responded to the Leapfrog Hospital Survey questions
Willing to Report
pertaining to adoption of this policy, but does not yet meet the
(one-filled bar)
criteria for “Some Progress.”
The hospital did not respond to the questions in this section of
Declined to Respond
the Survey or did not submit a Survey.
SECTION 7B: HEALTHCARE-ASSOCIATED INFECTIONS
There are no changes to this subsection.
We are pleased that hospital feedback regarding Leapfrog’s removal of burdensome questions related to healthcare-
associated infections has been overwhelmingly positive. Therefore, in 2018, Leapfrog will again draw infections data
directly from the National Healthcare Safety Network (NHSN). To do so, hospitals must join Leapfrog’s NHSN Group,
provide a valid NHSN ID in their Leapfrog Hospital Survey Profile, and submit Section 7 of the Leapfrog Hospital Survey by
the designated deadlines. This is all that is necessary to be scored and publicly reported on the five infection measures:
CLABSI, CAUTI, MRSA, C. Diff. and SSI Colon.
There are no changes to the scoring algorithm for this section.
7SECTION 7C: HOSPITAL-ACQUIRED CONDITIONS – PRESSURE ULCERS AND INJURIES
Due to feedback Leapfrog received from hospitals in 2017 regarding the feasibility of using the updated ICD-10 measure
specifications to report on the hospital-acquired pressure ulcers and injuries measures, Leapfrog is removing Section 7C
Hospital-Acquired Conditions – Pressure Ulcers and Injuries from the 2018 Survey.
SECTION 7D: ANTIBIOTIC STEWARDSHIP PRACTICES
We are pleased that hospitals will no longer need to devote time to responding to questions in Section 7D. Instead,
Leapfrog will obtain information regarding the hospital’s implantation of antibiotic stewardship practices directly from
CDC’s National Healthcare Safety Network (NHSN).
Hospitals will be required to join Leapfrog’s NHSN Group, provide a valid NHSN ID in their Leapfrog Hospital Survey Profile,
and submit Section 7 of the Leapfrog Hospital Survey by the designated deadlines in order to be scored and publicly
reported on the Antibiotic Stewardship Practices measure.
There are no changes to the scoring algorithm for this section.
SECTION 8: MEDICATIO N SAFETY
PROPOSED CHANGES TO SECTION 8A BAR CODE MEDICATION ADMINISTRATION
In Section 8A BCMA, Question #15a – 15e, hospitals currently report if they have any of five Processes and Structures to
Prevent Workarounds:
a. Has a formal committee that meets routinely to review data reports on BCMA system use
b. Has back-up systems for BCMA hardware failures
c. Has a Help Desk that provides timely responses to urgent BCMA issues in real-time
d. Conducts real-time observations of users using the BCMA system
e. Engages nursing leadership at the unit level on BCMA use
Leapfrog has added three additional processes to this question in the 2018 Survey:
Which of the following has your hospital done with the data and information identified through items a-e above:
f. In the past 12 months, used the data and information obtained through items a-e to implement quality
improvement projects that have focused on improving the hospital’s BCMA performance
OR
In the past 12 months, used the data and information obtained through items a-e to monitor a previously
implemented quality improvement project focused on improving the hospital’s BCMA performance
g. In the past 12 months, evaluated the results of the quality improvement projects (from f) and demonstrated that
these projects have resulted in higher adherence to our hospital’s standard medication administration process
OR
In the past 12 months, evaluated the results of the quality improvement projects (from f) and demonstrated
continued adherence to your hospital’s standard medication administration process
8h. Communicated back to end users the resolution of system deficiencies and/or problems that may have
contributed to the workaround.
To meet the Processes and Structures to Prevent Workarounds component of the BCMA standard, hospitals would need to
respond “yes” to 6 out of 8 questions above. No other updates are proposed for Section 8A BCMA.
Please see Appendix IV for the 2018 IPS Scoring Algorithm.
PROPOSED CHANGES TO SECTION 8B MEDICATION RECONCILIATION (APPLICABLE TO ADULT/GENERAL
HOSPITALS ONLY)
In 2017, Leapfrog added a new NQF-endorsed medication reconciliation measure: Number of Unintentional Medication
Discrepancies per Patient (NQF 2456). The measure focuses on the quality and accuracy of the hospital’s medication
reconciliation process. The measure is developed for adult inpatients only, but Leapfrog continues to advocate for an
adaptation for pediatric patients. The data collection protocols in the measure are effective for hospitals’ quality
improvement and the measure stewards provide significant free resources and information for hospitals to implement
these protocols. Information about the significant impact of poor medication reconciliation is available on Leapfrog’s
website.
Hospitals were not scored or publicly reported on this measure in 2017. In 2018, Leapfrog will address feedback received
from hospitals in 2017. First, Leapfrog will give hospitals two options for meeting the data collection requirements for this
measure:
Hospitals that started and have continued to sample 10 patients on a quarterly basis using the 2017 Leapfrog
Hospital Survey measure specifications can use those data when reporting on this section of the Survey.
Hospitals that did not start sampling patients in 2017, can sample in real-time (i.e. after April 1) and start data
collection anytime during the survey cycle by sampling 15 patients.
Next, we will limit sampling for the measure to patients admitted to medical/surgical units only. Lastly, we made the
following updates to the data collection instructions:
Standard language that pharmacists can use to inform patients selected for the measure that the pharmacist is not
normally part of the patient’s care team, but interviewing the patient to ensure that the hospital’s medication
reconciliation process is accurate.
Instructions on how to record unintentional discrepancies that have been corrected prior to the patient’s
discharge. Pharmacists are expected to intervene upon identification of an error on the admission or discharge
orders (i.e. discrepancy between the Gold Standard Medication History the pharmacist obtained from the patient
and the admission or discharge orders), and these discrepancies should still be recorded in the measure.
Hospitals will continue to report the number of unintentional medication discrepancies identified between the Gold
Standard Medication History obtained by a trained pharmacist and the admission and discharge orders, including the
number of additional unintentional medications.
Hospitals who submit this section of the 2018 Leapfrog Hospital Survey, and whose responses are not flagged during
Leapfrog’s monthly data review, will be scored as “Fully Meets the Standard” for having a protocol in place to collect data
9on the accuracy of the hospital’s medication reconciliation process. Hospitals that submit this section, but whose responses are flagged during Leapfrog’s monthly data review for potential data entry errors, will be scored as “Willing to Report” for beginning to put a protocol in place to collect data on the accuracy of the hospital’s medication reconciliation process. Hospitals that do not submit this section will be scored as “Declined to Respond.” In 2018, Leapfrog does not intend to publicly report a hospital’s rate of unintentional medication discrepancies per patient. SECTION 9: PEDIATRIC CARE SECTION 9A: CAHPS CHILD HOSPITAL SURVEY Hospitals with at least 500 annual pediatric inpatient admissions (patients
Hospital responses will be scored and publicly reported for head scans and abdomen/pelvis scans separately. Leapfrog will
score this section of the Survey by comparing the hospital’s median dose for each anatomic region and age stratum to two
benchmarks. The first benchmark is the Median Benchmark, which will be the median of the median doses reported across
all Leapfrog-reporting hospitals as of June 30, 2018. The second benchmark is the median of the 75th percentile doses
reported across all Leapfrog-reporting hospitals as of June 30, 2018.
Hospitals will receive points based on their reported median dose compared to the benchmarks. If the hospital’s reported
median dose is less than the Median Benchmark, it will receive 2 points. If the hospital’s reported median dose is greater
than or equal to the Median Benchmark and less than the 75 th Percentile Benchmark, it will receive 1 point. Otherwise, if
the hospital’s reported median dose is greater than or equal to the 75 th Percentile Benchmark, it will not receive points for
that category.
Therefore, for each anatomic region, there are at most 10 possible points. If a hospital had less than 10 CT scans for an age
stratum, that age stratum is not included in scoring. For each anatomic region, the percentage of points awarded is
calculated by summing the points earned and dividing by the total number of possible points (i.e. 2 times the number of age
strata with at least 10 CT scans). This percentage of points earned will be used to assign a performance category according
to the table below:
Pediatric CT Dose Score Head Scans Abdomen/Pelvis Scans
(Performance Category)
Fully Meets the Standard
>= 75% of total possible points >= 75% of total possible points
(four-filled bars)
Substantial Progress
>= 50% and < 75% of total possible points >= 50% and < 75% of total possible points
(three-filled bars)
Some Progress
>=25% and < 50% of total possible points >=25% and < 50% of total possible points
(two-filled bars)
Willing to Report
< 25% of total possible points < 25% of total possible points
(one-filled bar)
Unable to Calculate Score Fewer than 10 CT scans for all age ranges Fewer than 10 CT scans for all age ranges
Does Not Apply The hospital does not perform CT scans on pediatric patients.
Declined to Respond The hospital did not measure pediatric scan doses or did not submit a survey.
11APPENDIX I: RESPONSES TO PUBLIC COMMENTS
Leapfrog received over 150 public comments in response to the proposed changes to the 2018 Leapfrog Hospital Survey.
Comments were submitted from health care organizations, as well as health care experts, patient advocates, and
purchasers.
Responses to the public comments are organized by survey section below. If you submitted a comment, and do not see a
response, or if you have additional questions, please contact the Help Desk at https://leapfroghosptialsurvey.zendesk.com.
SECTION 1 BASIC HOSPITAL INFORMATION
Several commenters expressed agreement with Leapfrog’s decision to obtain teaching status from the NHSN Patient
Safety Component.
Leapfrog is committed to aligning measures and designations with other national organizations as appropriate. Find a
complete national crosswalk of 2018 Leapfrog Hospital Survey measures here.
Several comments expressed agreement with Leapfrog’s decision to update its definition for “pediatric admissions” to
include admission to any inpatient unit (i.e. adult units).
This change was in direct response to several hospitals that submitted feedback during the 2017 Survey cycle. We
appreciate continued feedback on refining the new definition.
Several commenters had questions about which patients to include when reporting on the new pediatric admissions
question.
The definition of “Total Pediatric Acute-Care Admissions” has been updated to include acute-care medical and surgical
pediatric (aged 17 years or younger) admissions to any inpatient unit. Include transfers from other hospitals as admissions
to your hospital. Include any admissions directly to an ICU or NICU (any level NICU) in your hospital, even if counted in
question #9. Exclude normal newborn admissions to the nursery and pediatric patients admitted for maternity care,
behavioral health, or discharged to hospice.
SECTION 2 COMPUTERIZED PHYSICIAN ORDER ENTRY (CPOE)
Some commenters expressed concern regarding the increased CPOE utilization target due to low utilization by non-
employed physician groups.
While contracting with external physician groups may present unique challenges in engaging physicians to use CPOE,
hospitals’ first and foremost responsibility is the safety of patients. Leapfrog expects hospital leadership to use maximum
leverage to ensure all physicians utilize CPOE for medication ordering.
Some commenters felt that requiring physicians to use CPOE to enter all orders reduces the speed and efficiency of care.
While entering orders through CPOE can add some time to the ordering process, significant evidence suggests the decision
support embedded in CPOE systems provides protection to patients from potentially life-threatening medication errors.
Leapfrog’s expert panel concluded that these safety benefits outweigh the possible inefficiencies introduced by ordering
through CPOE.
12Some commenters expressed concern that some of their physicians refuse to use CPOE due to the perceived steep
learning curve.
While learning to use CPOE may require a time commitment by physicians, the benefits of CPOE are well established in the
literature as important to patient safety. The adoption of CPOE has been shown to significantly reduce the risk of adverse
drug events, the most common error made in hospitals.
Some commenters noted that community physicians’ CPOE system interfaces may not sync with the hospital’s CPOE
interface due to a lack of financial and personnel resources.
While aligning community physician CPOE system interfaces with hospital CPOE interfaces may require resources, the
added safety that the CPOE provides patients should be a top priority. To the extent this problem threatens patient safety,
hospitals should find immediate solutions, possibly to include assisting community physicians with the human and financial
resources to help implement the needed interfaces.
Some commenters expressed concern over hospitals’ ability to meet the increased targets for both utilization (change
from 75% to 85%) and efficacy (change from 50% to 60%) and suggested we give hospitals additional time to achieve
these thresholds.
While the higher standards may prove difficult for some hospitals to meet initially, results from previous Leapfrog Hospital
Surveys show that most hospitals are already meeting these standards. Leapfrog also believes it is important to continue
encouraging hospitals to improve the safety of their care delivery systems. For these reasons, Leapfrog has decided to move
forward with the proposed changes of increasing the targets for both utilization and efficacy.
Several commenters expressed agreement with the changes to the CPOE Scoring criteria.
Based on an analysis of 2017 Leapfrog Hospital Survey Results, almost 50% of hospitals are already meeting the targets of
the new, higher criteria.
Some commenters requested that the Sample CPOE Test (via the CPOE Evaluation Tool) be expanded to include
additional test scenarios.
The Sample Test is designed to give hospitals, including the licensed prescriber, an opportunity to practice completing the
test process (e.g., time limits and steps, browser compatibility, lab and ADT links, etc.). The Sample Test is an abridged
version of the Adult Inpatient Test, and not fully representative of the Test Patients and Test Orders that will appear in the
Adult Inpatient Test. It is not meant to predict a hospital’s score on the Adult Inpatient Test. Sample Test materials consist
of only two Test Patients and four Test Orders. Leapfrog will publish a guidance document developed by the CPOE
Evaluation Tool developers to provide hospitals with information on how to improve their clinical decision support
capability on the Survey and CPOE Materials webpage on April 1.
A commenter asked how the 10 order-checking categories included in the CPOE Evaluation Tool were selected
The ten order-checking categories included in the CPOE Evaluation Tool were identified by the CPOE Evaluation Tool
developers using several sources: data from the ISMP Medication Error Reporting System, peer reviewed literature, and
data from several large health systems including the Veteran’s Health Administration, Duke Health, Boston University, and
Partners Healthcare. A complete list and description of the order-checking categories is available in Appendix II.
Several commenters expressed frustration with their inability to customize their CPOE system’s clinical decision support
functions and asked if there is a role that the CPOE Evaluation Tool developers can play in communicating safety
requirements to the major vendors.
The CPOE Evaluation Tool developers have developed a guidance document for hospitals that participate in the CPOE
13Evaluation Tool, which includes information to help hospitals update and improve their clinical decision support system.
Hospitals can share this document with their CPOE vendors. Leapfrog will be publishing this document on the Survey and
CPOE Materials webpage on April 1.
SECTION 3 INPATIENT SURGERY
Some hospitals had questions about how Leapfrog’s new surgeon volume question would affect surgeons who practice
at multiple hospitals.
Based on hospital feedback, in lieu of asking hospitals to report on the exact volume for each surgeon, in the 2018 Leapfrog
Survey Leapfrog will ask hospitals if they require their surgeons to have minimum experience with a procedure to be
privileged at that hospital to perform that procedure. Leapfrog’s expert panel has recommended the appropriate minimum
level of surgeon volume for each procedure. We expect that hospitals will consider total experience in the privileging
process, not just experience gained at a single facility.
Some hospitals expressed confusion regarding how to determine which surgeons are privileged at the hospital to
perform the eight procedures.
Leapfrog recommends checking with surgical leadership at your hospital on the process your hospital uses for privileging
surgeons. Every surgeon must have “privileges” to perform specific procedures.
Several commenters noted that they do not support minimum hospital and/or surgeon volume standards as outcomes
measures are most important to patients.
While Leapfrog recognizes that volume is not a perfect substitute for outcomes, for the eight surgeries being measured in
the 2018 Leapfrog Survey, there is a strong body of evidence linking hospital and surgeon experience and patient outcomes.
Leapfrog continues to explore opportunities to incorporate evidence-based, endorsed outcome measures (e.g., mortality,
morbidity) into the survey.
A commenter expressed concern about Leapfrog implementing the surgical volume standards in 2018 because hospitals
may not have Leapfrog’s minimum volume standards as part of their current privileging process, and putting that in place
will take time.
As described in Section 3A of this document, hospitals will be able to earn partial credit for committing to include Leapfrog’s
Minimum Surgeon Volume Standards in their privileging process within the next 12 months.
Two commenters noted that procedures may be done by a primary and assisting surgeon, and wanted to ensure the
assistant surgeon received credit for the case.
For determining surgeon volume, if a surgeon assists another surgeon during a procedure, the procedure should count for
both surgeons’ procedure totals. This would apply when both surgeons are experienced, practicing surgeons. If one or more
of the surgeons is a resident, fellow, or being proctored by an experienced surgeon, only the experienced surgeon should
receive credit toward her/his procedure total. Surgeons who have just finished their training should receive a 24-month
grace period to build up their experience. After that point, his/her volume should be tracked for the surgeon volume. The
procedures performed by this surgeon during the reporting period should still be counted towards the hospital’s volume
total, as the broader staff still had the experience with the surgery.
Several commenters noted that the bariatric surgery cases should be limited to those undergoing the procedure for
weight loss.
Leapfrog found this comment very helpful and has worked to clarify for the 2018 Survey that bariatric surgery needs to be
14done explicitly for weight loss. Hospitals will be given two sets of instructions for identifying these cases: (a) use the listed
ICD-10 diagnosis codes to identify those patients who underwent the procedure where obesity was the primary diagnosis
or (b) use the listed diagnosis codes to identify those patients who underwent the procedures, where obesity was not the
primary diagnosis, but chart review indicated the procedure was performed for the purpose of weight loss.
One commenter suggested that Leapfrog give additional credit to hospitals that participate in national clinical registries
(i.e. NSQIP) and/or allow hospitals to share their NSQIP outcomes in place of volume.
Leapfrog appreciates the importance of registry participation, and continues to work with its surgical expert panel to
explore opportunities to add additional outcomes measures into the survey, including clinical outcome measures used in
registries. We welcome feedback from hospitals on potential barriers to participating in a national registry.
Some commenters questioned why Leapfrog is asking hospitals to develop surgical appropriateness criteria only for
these particular eight procedures, even though there may be other procedures with more documented evidence of
overuse.
Given that Leapfrog is measuring hospital performance based on volume for these eight procedures, Leapfrog seeks to
ensure that appropriate safeguards are in place for these eight procedures to ensure there isn’t a perverse incentive for
hospitals to operate when not necessary. We recognize the importance of overuse as a more widespread problem, and
over time, Leapfrog plans to explore additional opportunities to address it.
A commenter expressed concern about the burden of implementing appropriateness criteria without sophisticated EHR
capabilities.
While the implementation of appropriateness criteria can be streamlined and more easily tracked with EHR capabilities,
Leapfrog’s research and discussions with hospitals have indicated that implementation and monitoring of appropriateness
criteria can be achieved without the use of EHR capabilities. Leapfrog has convened an Advisory Workgroup who will be
helping to identify resources for hospitals to begin the work of developing, implementing, and monitoring appropriateness
criteria. We are planning to host a webinar later this spring.
SECTION 4 MATERNITY CARE
No comments were submitted.
SECTION 5 ICU PHYSICIAN STAFFING
Commenters expressed support for Leapfrog’s minor update to the scoring algorithm.
Leapfrog continues to look for opportunities to clarify the questions and scoring algorithm for Section 5. Additional
feedback on the reformatting of the questions in Section 5, as well as the re-formatting of the scoring algorithm are
welcomed.
SECTION 6 NQF SAFE PRACTICES
With regard to Leapfrog’s request for feedback on adding unscored questions to Safe Practice 19 Hand Hygiene on the
use of electronic hand hygiene monitoring, commenters expressed mixed perspectives. Some commenters noted that
electronic monitoring is potentially cost prohibitive for small hospitals while others expressed strong support for
electronic monitoring as a best practice.
Leapfrog shared the comments with its technical experts and determined that further consultation and fact-finding is
15necessary before adding this to the Survey. Moreover, given the importance of hand hygiene and statistics that point to
poor compliance, Leapfrog will convene a national expert panel to review the entirety of Safe Practice 19 Hand Hygiene,
including reviewing the latest literature and evidence-based best practices for monitoring compliance with hand hygiene
protocols.
SECTION 7 MANAGING SERIOUS ERRORS
Several commenters expressed concern over Leapfrog’s Never Evens Policy Statement that requires hospital to notify an
external agency within 10 days.
Leapfrog’s technical experts have reviewed the comments and recommended that Leapfrog align with the Minnesota
Adverse Event Reporting requirements in 2018. Therefore, Leapfrog has updated the requirement to report to an external
agency on the 2018 Survey from 10 days to 15 business days.
Several commenters supported Leapfrog’s decision to remove the Hospital-Acquired Pressure Ulcers and Injuries
measures from the 2018 Survey, but some commenters noted that these are important safety measures and should not
be dropped from the Survey.
Leapfrog agrees that measures of hospital-acquired conditions are critical to patients, families, and to healthcare
purchasers. However, given the burdens and potential for error reported by hospitals in 2017 in identifying over 3,650 ICD-
10 diagnosis codes for hospital-acquired injuries, Leapfrog must remove these measures for 2018. We will continue to
include the DRA HAC falls with trauma measure and the PSI 3 Pressure Ulcer Rate in the Hospital Safety Grade.
SECTION 8 MEDICATION SAFETY
With regard to Bar Code Medication Administration (BCMA), several commenters expressed concern about Leapfrog
using the three new process/structural measures to monitor and prevent workarounds in scoring in 2018.
Leapfrog has reviewed the comments as well as the proposed scoring algorithm for 2018. In 2018, hospitals will be able to
meet the Leapfrog standard with 6 out of 8 processes or structures in place to prevent workarounds. The updated scoring
algorithm is detailed in Appendix IV.
With regard to Medication Reconciliation, several commenters expressed concern that the sample sizes are too small,
and therefore, not valid.
Leapfrog is taking a phased approach to rolling out this measure on the Survey. In 2017, hospitals were asked to sample 10
patients, and responses were not scored or publicly reported. For 2018, hospitals will be asked to sample 15 patients and
Leapfrog will publicly report whether or not the hospital has a process in place to protocol in place to collect data on the
accuracy of the hospital’s medication reconciliation process. We will not score and publicly report a hospital’s rate of
unintentional medication discrepancies. We will monitor and study the results to determine future reporting and standards
for this endorsed measure, which is the only endorsed measure of its kind.
With regard to Medication Reconciliation, some commenters questioned why pediatric patients are excluded?
The medication reconciliation measure on the Survey is endorsed by the National Quality Forum (NQF 2456: Number of
Unintentional Medication Discrepancies per Patient), and unfortunately this measure is currently specified for adult
patients only. We will advocate with NQF and measure developers to adapt the measure for pediatric patients.
With regard to Medication Reconciliation, some commenters expressed concern that limiting the sampling to patients
admitted to medical/surgical units would not result in a representative sample from the hospital and may exclude
16patients with multiple chronic conditions and polypharmacy who are most at risk for medication reconciliation errors.
In order to respond to hospital’s feedback regarding the challenges of scheduling the interview with the pharmacist and the
patient to collect the Gold Standard Medication History, Leapfrog is allowing hospitals to restrict sampling to
medical/surgical units in 2018. However, hospitals are welcome to sample for additional units if they choose. Sampling from
medical/surgical units is the minimum requirement.
Two health systems that participated in the national Pilot of the 2018 Leapfrog Hospital Survey noted that they will be
monitoring the time spent on data collection for the Medication Reconciliation measure in 2018.
Any feedback hospitals collect regarding the time commitment to complete data collection would be greatly appreciated. In
addition, hospitals may want to refer to the information from the Medication Reconciliation Technical Assistance Calls
hosted in 2017 where three different hospitals experienced with the measure shared ways to reduce the time burden of
data collection. Materials are posted on the Town Hall Calls webpage.
SECTION 9 PEDIATRIC CARE
Regarding the CAHPS Child Hospital Survey, several commenters expressed concern with administering the survey for
NICU discharges.
The Child CAHPS Hospital Survey was designed to be administered to pediatric discharges including NICU discharges.
Additional details on fielding the CAHPS Child Hospital Survey can be found here. In 2018, hospitals that have been
administering the CAHPS survey without including NICU discharges in their sample can report those results to Leapfrog,
provided they meet the minimum sample size and timing requirements in the Leapfrog Hospital Survey. However, we are
urging those hospitals to begin including NICU discharges-- per the manual guidelines-- immediately, as CAHPS is designed
to include those patients. Hospitals that are just starting to administer the survey in 2018 should include NICU discharges in
their sample per the sampling framework detailed in the manual.
Regarding the CAHPS Child Hospital Survey, some commenters expressed the cost of administering the survey and low
response rates.
Some hospitals have asked about the use of alternative, lower cost modes of survey administration, such as administering
paper surveys as discharge that can then be batched and mailed to a vendor to calculate results. This approach is
potentially an opportunity both lower the cost of administration and increase response rates.
Leapfrog’s Pediatric Expert Panel, has noted that while administering the CAHPS Child Hospital Survey using paper forms at
discharge on the list of AHRQ-approved modes, hospitals that are trying to find ways to administer the survey and increase
response rate should be able to submit results to the 2018 Leapfrog Hospital Survey. That said, the Pediatric Expert Panel
has expressed a desire for these different modes to be tested, and so we cannot guarantee that you will be able to submit
these results for future Leapfrog Hospital Surveys.
Regarding Pediatric CT Radiation Dose, several commenters expressed concern that Leapfrog’s scoring algorithm for the
Pediatric CT Radiation Dose Measures, which gives hospitals credit for having a median average DLP value at or below
the 50th percentile, would drive down doses to ranges that would results in less than optimal scan quality.
The measure developer has found in a large randomized trial that routine review and sharing of dosage by hospitals leads
to dose reductions without compromise to clinical efficacy. However, Leapfrog has decided to continue requiring hospitals
to report on their average 25% DLP and we will carefully monitor the prevalence of low doses. We will work with our
national expert panel to make changes to the scoring algorithm as appropriate in the future.
17APPENDIX II: CPOE SCORING ALGORITHM FOR ADULT/GENERAL HOSPITALS
Score on Adult Inpatient Test via the CPOE Evaluation Tool
Incomplete
Full
Substantial Some Evaluation
Demonstration Insufficient
Implementation Demonstration Demonstration Completed (Failed
of National Evaluation
Status of National of National The deception
Safety Standard (Hospital
(from Leapfrog Safety Standard Safety Standard Evaluation analysis or
for Decision was not able
Hospital Survey for Decision for Decision (Less than timed out)
Support to test at
Questions Support Support 40% of test -or-
(60% or greater least 50% of
#3-4) (50-59% of test (40-49% of test orders Did not
of test orders test orders)
orders correct) orders correct) correct) complete an
correct)
evaluation
85% or greater of Substantial Some
all inpatient Progress Progress Unable to
Fully Meets the Substantial Willing to
medication orders (Previously (Previously Calculate
Standard Progress Report
entered through “Fully Meets the “Substantial Score
CPOE System Standard”) Progress”)
75-84% of all Fully Meets
Some Progress
inpatient Standard Unable to
Substantial (Previously Some Willing to
medication orders (Previously Calculate
Progress “Substantial Progress Report
entered through “Substantial Score
Progress”)
CPOE System Progress”)
50-74% of all Substantial Willing to
inpatient Progress Report Unable to
Substantial Willing to
medication orders (Previously Some Progress (Previously Calculate
Progress Report
entered through “Some “Some Score
CPOE System Progress”) Progress”)
CPOE implemented
in at least one
inpatient unit but Substantial Some Progress
Unable toOrder Checking Category Description Example
Therapeutic Duplication Medication combinations overlap therapeutically Using clonazepam and
(same agent or same class) lorazepam together
Drug-Dose (Single) Specified dose of medication exceeds safe range for Tenfold overdose of digoxin
single dose
Drug-Dose (Daily) Specified frequency of administration results in Ordering ibuprofen regular
daily dose that exceeds safe range for daily dose dose every three hours
Drug-Allergy Medication (or medication class) is one for which Penicillin prescribed for
patient allergy has been documented patient with documented
penicillin allergy
Drug-Route Specified route of administration is inappropriate Use of vitamin K
and potentially harmful intramuscular injection
Drug-Drug Medications in pair of orders result in known Concurrent linezolid and
harmful interaction when used in combination sumatriptan
Drug-Diagnosis Medication contraindicated based on documented Nonspecific beta-blocker in
problem/diagnosis patient with asthma
Drug-Age Medication contraindicated based on patient age Prescribing diazepam for a
patient over 65 years old
Drug-Lab Medication contraindicated based on documented Use of enalapril in patient
laboratory test results (includes renal status) with severe renal failure
Drug Monitoring Medication for which the standard of care includes Prompt to order drug levels
subsequent monitoring to avoid harm when ordering
aminoglycoside.
The Tool also includes an “Alert Fatigue” test category, which checks if prescribers are receiving alerts or
information for inconsequential medication interactions that clinicians typically ignore. An example would be
alerting on the concurrent use of hydrochlorothiazide and captopril. This test category is not included in scoring.
The Tool also includes a “Deception Analysis” test category, which checks for “false positives” (e.g., orders that
should not have generated any warning in the hospital’s CPOE system). Hospital’s that “fail” the Deception
Analysis are scored as “incomplete evaluation” and will not be able to re-take an Adult Inpatient Test for 120
days.
19APPENDIX III: ICU PHYSICIAN STAFFING SCORING ALGORITHM
IPS Score
(Performance Meaning that:
Category)
The hospital responded “Yes” or “Not applicable, intensivists present 24/7” to all of the
following questions:
Question #3:
All critical care patients in adult and pediatric general medical and/or surgical ICU(s) and
neuro ICUs are managed or co-managed by one or more physicians who are certified in
critical care medicine (i.e. “intensivists”) when these physicians are present (on-site or
via telemedicine)
Question #4 or #5:
o One or more intensivist(s) is/are present via telemedicine 24 hours per day, 7
days per week, with some on-site intensivist time;
o One or more intensivist(s) is/are present in each ICU during daytime hours for
Fully Meets the at least 8 hours per day, 7 days per week, providing care exclusively in each ICU
during these hours
Standard
Question #6:
(four-filled bars)
When intensivists are not present (on-site or via telemedicine) in these ICUs, one of
them returns more than 95% of calls/pages/texts from these units within five minutes
Question #7:
When intensivists are not present (on-site or via telemedicine) in the ICU or not able to
physically reach an ICU patient within 5 minutes, another physician, physician assistant,
nurse practitioner or FCCS-certified nurse “effector” is on-site at the hospital and able
to reach ICU patients within five minutes in more than 95% of the cases
Note: When telemedicine is employed as a substitute for on-site intensivist coverage, it
must meet all ten requirements detailed in endnotes in the hard copy of the Survey, which
includes some on-site intensivist time to manage the ICU patients’ admission, discharge, and
care planning.
The hospital responded “Yes” to all of the following questions:
Question #3:
All critical care patients in adult and pediatric medical and/or surgical ICU(s) and neuro
ICUs are managed or co-managed by one or more physicians who are certified in critical
Substantial care medicine (i.e. “intensivists”), when these physicians are present (on-site or via
Progress telemedicine)
(three-filled bars) Question #8 or #12:
One or more intensivist(s) is/are present in each ICU during daytime hours for
at least 8 hours per day, 4 days per week or 4 hours per day, 7 days per week;
Clinical pharmacists make daily rounds on all critical care patients in adult and
pediatric medical and/or surgical and neuro ICUs 7 days per week
Question #13 or #14:
20 An intensivist leads daily, multi-disciplinary team rounds on-site on all critical
care patients in adult and pediatric medical and/or surgical and neuro ICUs 7
days per week;
When intensivists are on-site in adult and pediatric medical and/or surgical and
neuro ICUs, they make all admission and discharge decisions
The hospital responded “Yes” to all of the following questions:
Question #3:
All critical care patients in adult and pediatric medical and/or surgical ICU(s) and neuro
ICUs are managed or co-managed by one or more physicians who are certified in critical
Substantial care medicine (i.e. “intensivists”), when these physicians are present (on-site or via
Progress telemedicine)
Question #9:
(alternative for
One or more intensivist(s) is/are present via telemedicine 24 hours per day, 7 days per
hospitals) week, with on-site care planning done by an intensivist, hospitalist, anesthesiologist, or
a physician trained in emergency medicine
Note: When telemedicine is employed as a substitute for on-site intensivist coverage, it
must meet all nine requirements detailed in endnote in the hard copy of the Survey.
The hospital responded “Yes” to all of the following questions:
Question #3:
All critical care patients in adult and pediatric medical and/or surgical ICU(s) and neuro
ICUs are managed or co-managed by one or more physicians who are certified in critical
care medicine (i.e. “intensivists”), when these physicians are present (on-site or via
telemedicine)
Question #10:
One or more intensivist(s) is/are present on-site at least 4 days per week to establish or
revise daily care plans for all critical care patients
Question #13 or #14:
An intensivist leads daily, multi-disciplinary team rounds on-site on all critical
care patients in adult and pediatric medical and/or surgical and neuro ICUs 7
Some Progress days per week;
When intensivists are on-site in adult and pediatric medical and/or surgical and
(two-filled bars)
neuro ICUs, they make all admission and discharge decisions
Or the hospital responded “Yes” to all of the following questions:
Question #11:
If not all, at least some critical care patients are managed or co-managed by physicians
who are certified in critical care medicine (i.e. “intensivists”), either on-site or via
telemedicine
Question #13 or #14:
An intensivist leads daily, multi-disciplinary team rounds on-site on all critical
care patients in adult and pediatric medical and/or surgical and neuro ICUs 7
days per week;
When intensivists are on-site in adult and pediatric medical and/or surgical and
neuro ICUs, they make all admission and discharge decisions
21Note: When telemedicine is employed as a substitute for on-site intensivist coverage, it
must meet all nine requirements detailed in endnote in the hard copy of the survey.
Willing to Report The hospital responded to all the Leapfrog survey questions, but it does not yet meet the
(one-filled bar) criteria for Some Progress.
The hospital does not operate an adult or pediatric general medical or surgical intensive care
Does Not Apply
unit or a neuro intensive care unit.
Declined to
The hospital did not respond to this section of the survey, or has not submitted a survey.
Respond
22APPENDIX IV: BAR CODE MEDICATION ADMINISTRATION (BCMA) SCORING ALGORITHM
Processes &
BCMA Score Structures to
% Units % Compliance Decision Support
(Performance Category) Prevent
Workarounds
Fully Meets the Standard
100% 95% 7 out of 7 6 out of 8
(four-filled bars)
Substantial Progress
The hospital meets 3 of the 4 standards
(three-filled bars)
Some Progress
The hospital meets 2 of the 4 standards
(two-filled bars)
Willing to Report
The hospital meets 1 or 0 of the 4 standards
(one-filled bar)
The hospital did not respond to the questions in this section of the survey or did not
Declined to Respond
submit a survey.
The hospital does not operate an ICU, medical/surgical unit, or labor and delivery
Does Not Apply
unit.
23You can also read
