Watson Health 15 Top Health Systems Study, 2018 - 10th edition | April 23, 2018 - Truven Health

 
Watson Health
15 Top Health Systems
Study, 2018
10th edition | April 23, 2018
IBM Watson Health™
75 Binney Street
Cambridge, MA 02142
800-525-9083
ibm.com/watsonhealth

Watson Health 15 Top Health Systems Study, 2018; 10th edition

© Copyright IBM Corporation 2018. All rights reserved. IBM, the IBM logo, ibm.com,
Watson Health, and 100 Top Hospitals are trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide. Other product and service
names might be trademarks of IBM or other companies.

Printed and bound in the United States of America.

The information contained in this publication is intended to serve as a guide for general
comparisons and evaluations, but not as the sole basis upon which any specific
conduct is to be recommended or undertaken.

The reader bears sole risk and responsibility for any analysis, interpretation, or
conclusion based on the information contained in this publication, and IBM shall not
be responsible for any errors, misstatements, inaccuracies, or omissions contained
herein. No part of this publication may be reproduced or transmitted in any form or
by any means, electronic or mechanical, including photocopying, recording, or by
any information storage and retrieval system, without permission in writing from
IBM Watson Health.

ISBN: 978-1-57372-475-3
Introduction
Contents
                                           Welcome to the 10th edition of the
03  Introduction                           Watson Health 15 Top Health Systems study.
09	2018 15 Top Health Systems
    award winners                          This 2018 study from IBM Watson Health™
11  Findings                               marks another milestone in the 100 Top Hospitals®
21  Methodology                            program’s rich history: a full decade of publishing
37	Appendix A:                            an annual quantitative study designed to shine
    Health system winners and              a light on the nation’s highest-performing
    their member hospitals                 health systems.
41	Appendix B:
    The top quintile: Highest-             Our research of US health system performance
    performing health systems              began with the same goal that has driven each
43	Appendix C:                            study since the beginning of the 100 Top Hospitals
    Methodology details                    program: To identify top performers, and also
57	Appendix D:                            deliver insights that may help healthcare systems
    All health systems in study            better focus their improvement initiatives on
                                           achieving consistent, balanced, and sustainable
                                           high performance.

Truven Health Analytics® was acquired by   Health systems do not apply for our 15 Top Health
IBM in 2016 to help form a new business,   Systems selection process, and winners do not pay
Watson Health.                             to market their honor.

                                           Illuminating achievement for a
                                           value-based world
                                           Our research is based on clinical, operational,
                                           financial, and patient perception-of-care measures
                                           that form a balanced scorecard. For 10 years,
                                           the health systems achieving excellence on our
                                           scorecard inherently set attainable benchmarks for
                                           others in the industry to aspire to over time.

                                           Providing these measures of success may
                                           be especially important today as we see the
                                           healthcare landscape continuing to evolve from
                                           fee-for-service toward value-based care models,
                                           with many health systems paying closer attention
                                           to population health management and system-
                                           wide alignment of performance.

                                                                                                 3
The Watson Health                                                                              By finding ways to take balanced performance to
 15 Top Health Systems                                                                          the next level, the winners of our 15 Top Health
 scorecard results are divided                                                                  Systems award are identifying opportunities to
 into two separate sections                                                                     deliver healthcare value to patients, communities,
 that graphically illustrate:                                                                   and payers. The performance levels achieved by
 ––A health system’s                                                                            these systems may motivate their peers to use
   performance and                                                                              data, analytics, and benchmarks to close their
   improvement versus peer                                                                      performance gaps.
   health systems
 ––Cross-system performance
   alignment of member                                                                          Delivering a robust and transparent assessment
   hospitals
                                                                                                We have designed this study to provide a view
                                                                                                of health system performance across multiple
                                                                                                dimensions: how they stand compared to peers
                                                                                                and high performers (whole-system performance),
                                                                                                where they stand in the evolution of their own
                                                                                                cultures of performance improvement (relative
                                                                                                long-term improvement and rate of improvement),
                                                                                                and the achievement of cross-system performance
                                                                                                alignment (member hospital performance).

                                                                                                These collective insights may be used by health
                                                                                                system leaders to adjust continuous improvement
                                                                                                targets, enrich the collaboration of member
                                                                                                hospitals, and track system-wide alignment toward
                                                                                                common performance goals.

                                                                                                To maintain the 15 Top Health Systems study’s
                                                                                                integrity and avoid bias, we use public data sources
                                                                                                and explain the methodologies we use to calculate
                                                                                                outcome metrics. This approach supports inclusion
                                                                                                of systems across the country and facilitates
                                                                                                consistency of definitions and data.

                                                                                                Our national balanced scorecard, based on Kaplan
                                                                                                and Norton’s concept1, is the foundation of our
                                                                                                research. It is comprised of key measures of
                                                                                                performance: inpatient and extended care quality,
                                                                                                operational efficiency, financial health*, and
                                                                                                patient experience. The composite score derived
                                                                                                from these measures reflects top performance in
                                                                                                hospital-based care, management, and leadership.

 * In this study, measures of financial health are reported for information-only purposes because public, audited financial statements are not available for all US health systems.

4 IBM Watson Health
In addition, to support consideration of the                                           Comparing the performance of our
   differences in scale among systems, the study                                          2018 winners to nonwinners
   categorizes the nation’s systems into three groups:                                    Using the measures in our national balanced
   large, medium, and small health systems. This                                          scorecard, this year’s 15 Top Health Systems study
   produces benchmarks that are comparable and                                            revealed important differences between award
   action-driving across similar systems.                                                 winners and their nonwinning peers.

                                                                                          Our study’s highest-performing systems:
   Yielding a measure of leadership excellence
                                                                                            –– Had lower inpatient mortality and fewer patient
   Since 1993, the 100 Top Hospitals program has
                                                                                               complications, considering patient severity
   sought to shed light on the efficacy of innovative
   leaders. The methodology is aimed at identifying                                         –– Delivered care that resulted in fewer HAIs
   leaders who can transform an organization by
                                                                                            –– Had lower 30-day readmission rates*
   pinpointing improvement opportunities and
   adjusting goals for key performance domains.                                             –– Sent patients home sooner
   We believe that higher composite scores on
                                                                                            –– Provided faster emergency care
   the balanced scorecard typically indicate more
   effective leadership and a consistent delivery                                           –– Kept episode-of-care expenses low, both
   of value.                                                                                   in-hospital and through the aftercare process
                                                                                            –– Scored higher on patient ratings of their overall
  The leadership of today’s health systems, including
                                                                                               hospital experience
  the board, executive team, and medical staff
  leadership, is responsible for ensuring all of their
                                                                                          Our study projections also indicate that if the
  system hospitals are performing at similarly high
                                                                                          benchmarks of performance established by our
  levels in both the short and long term. The 15 Top
                                                                                          2018 winners were achieved by all US health
  Health Systems study and analytics provide a view
                                                                                          systems we studied, the following would be true:
  of that enterprise performance alignment.
                                                                                            –– Over 60,000 additional lives could be saved
                                                                                               in-hospital
   Providing new insight into clinical quality
                                                                                            –– Over 31,000 additional patients could be
   For this 2018 15 Top Health Systems study, we                                               complication-free
   added a newly available measure of clinical quality:
   healthcare-associated infections (HAIs). Since                                           –– 16% fewer infections would be acquired by
   there is public interest in tracking and preventing                                         hospital patients
   hospital-acquired infections, we used the HAI data                                       ––The typical patient could be released from the
   reported by the Centers for Medicare & Medicaid                                            hospital almost a half day sooner and would
   Services to analyze hospital performance and                                               have 5.6% fewer expenses related to the
   provide national benchmarks in this area.                                                  complete episode of care than the median
                                                                                              patient in the US
                                                                                            –– Patients could spend 40 minutes less time in
                                                                                               hospital emergency rooms per visit

* 30-day mortality was slightly higher for winning health systems in this year’s study.

                                                                                                                                                   5
Understanding the similarities    The analysis on the previous page is based on
 and differences between           applying the difference between study winners
 high and low performers can       and nonwinners to Medicare patient counts. If the
 help provide benchmarks for       same standards were applied to all inpatients, the
 the industry. The findings we     impact would be even greater.
 assemble for this study provide
 examples of excellence, as        For more details about this study’s findings and
 evidenced in several additional   the achievements of the 15 Top Health Systems,
 published studies2 - 24.          please see the Findings section of this document.

                                   Welcoming your input
                                   The 100 Top Hospitals program works to ensure
                                   that the measures and methodologies used in our
                                   studies are fair, consistent, and meaningful. We
                                   continually test the validity of our performance
                                   measures and data sources. In addition, as part of
                                   our internal performance improvement process, we
                                   welcome comments about our study from health
                                   system, hospital, and physician executives. To
                                   submit comments, visit 100tophospitals.com.

                                   Showcasing the versatility of the
                                   100 Top Hospitals program
                                   The 15 Top Health Systems research is one of three
                                   major annual studies of the Watson Health 100 Top
                                   Hospitals program. To increase understanding of
                                   trends in specific areas of the healthcare industry,
                                   the program includes:
                                     –– 100 Top Hospitals and Everest Award studies
                                        Research that annually recognizes the
                                        100 top-rated hospitals in the nation based
                                        on a proprietary, balanced scorecard of
                                        overall organizational performance, and
                                        identifies those hospitals that also excel at
                                        long-term rates of improvement in addition
                                        to performance
                                     –– 50 Top Cardiovascular Hospitals study
                                        An annual study introduced in 1999 that
                                        identifies hospitals demonstrating the highest
                                        performance in hospital cardiovascular
                                        services for four important patient groups:
                                        heart attack, heart failure, coronary artery
                                        bypass graft, and percutaneous coronary
                                        intervention

6 IBM Watson Health
–– 15 Top Health Systems study
     An annual study introduced in 2009 that
                                                       The 2018 study
     provides an objective measure of health           analyzed 338
     system performance overall and offers insight
     into the ability of a system’s member hospitals
                                                       health systems and
     to deliver consistent top performance across      2,422 hospitals that
     the communities they serve, all based on our
     national health system scorecard                  are members of
                                                       health systems.
In addition to the major studies, customized
analyses are also available from the 100 Top
Hospitals program, including custom benchmark
reports. Our reports are designed to help
healthcare executives understand how their
organizational performance compares to peers
within health systems, states, and markets.

100 Top Hospitals program reports offer a
two-dimensional view of both performance
improvement over time, applying the most
current methodologies across all years of data to
identify trends, as well as the most current year
performance.

You can read more about these studies, order
customized reports, and view lists of all winners by
visiting 100tophospitals.com.

About IBM Watson Health
Each day, professionals throughout the health
ecosystem make powerful progress toward a
healthier future. At IBM Watson Health, we help
them remove obstacles, optimize efforts, and
reveal new insights to support the people they
serve. Working across the landscape, from payers
and providers to governments and life sciences,
we bring together deep health expertise; proven
innovation; and the power of artificial intelligence
to enable our customers to uncover, connect, and
act as they work to solve health challenges for
people everywhere.

For more information, visit ibm.com/watsonhealth.

                                                                              7
Note that the order of health systems in the
2018                                                                                      following tables does not reflect performance
                                                                                          rating. Systems are ordered alphabetically.
15 Top Health                                                                             For full details on these peer groups and the
                                                                                          process we used to select the winning benchmark
Systems award                                                                             health systems*, see the Methodology section of
                                                                                          this document.
winners
The Watson Health 100 Top Hospitals® program
is pleased to present the 2018 Watson Health
15 Top Health Systems.

 15 Top Health Systems award winners
 Large health systems (> $1.85 billion)                                     Location
 Mayo Foundation                                                            Rochester, MN
 Mercy                                                                      Chesterfield, MO
 Sentara Healthcare                                                         Norfolk, VA
 St. Luke’s Health System                                                   Boise, ID
 UCHealth                                                                   Aurora, CO

 Medium health systems ($800 million - $1.85 billion)                       Location
 Aspirus                                                                    Wausau, WI
 HealthPartners                                                             Bloomington, MN
 Mercy Health - Cincinnati                                                  Cincinnati, OH
 Mission Health                                                             Asheville, NC
 TriHealth                                                                  Cincinnati, OH

 Small health systems (< $800 million)                                      Location
 Asante                                                                     Medford, OR
 CHI St. Joseph Health                                                      Bryan, TX
 Maury Regional Health                                                      Columbia, TN
 Roper St. Francis Healthcare                                               Charleston, SC
 UPMC Susquehanna Health System                                             Williamsport, PA

* To see a full list of Winners Through the Years, visit truvenhealth.com/Products/100-Top-Hospitals/Program-Info/15-Top-Health-Systems/Winners-Through-the-Years.

                                                                                                                                                                      9
Note: In Tables 1 through 4, data for the
Findings                                                                                       15 Top Health Systems award winners is labeled
                                                                                              “Benchmark,” and data for all health systems,
The Watson Health 15 Top Health Systems study                                                  excluding award winners, is labeled “Peer group.”
profiles the top-performing health systems* in the                                             In columns labeled “Benchmark compared
country. According to publicly available data and                                              with peer group,” we calculated the actual and
our transparent methodologies, these industry                                                  percentage difference between the benchmark
leaders appear to be successfully addressing the                                               hospital scores and the peer group scores.
challenge of deploying innovative clinical and
operational approaches to multiple hospital sites                                              15 Top Health Systems had better survival rates**
to achieve consistent top performance.                                                             ––The winners had 14.6% fewer in-hospital
                                                                                                     deaths than their nonwinning peers,
For 10 years, the 15 Top Health Systems study                                                        considering patient severity (Table 1)
has followed the results achieved by leading                                                       –– Mortality results for medium health systems
health systems and published numerous                                                                 showed the greatest difference between
examples of the benchmark systems’ clinical                                                           winners and nonwinners, with 15.9% fewer
and operational excellence. The study is more                                                         deaths among benchmark health systems
than a list of accomplishments; it is a tool US                                                       (Tables 2 - 4)
health system leaders can use to help guide their
own performance improvement initiatives. By                                                    15 Top Health Systems had fewer
highlighting what the highest-performing leaders                                               patient complications**
around the country are doing well, we create
aspirational benchmarks for the rest of the industry.                                              –– Patients treated at the winning systems’
                                                                                                      member hospitals had significantly fewer
                                                                                                      complications, with rates 17.3% lower than
How the winning systems compared                                                                      at nonwinning system hospitals, considering
to their peers                                                                                        patient severity (Table 1)
In this section, we show how the 15 Top Health                                                     –– Large health systems had the greatest
Systems performed within their comparison groups                                                      difference between winners and nonwinners,
(large, medium, and small systems), compared                                                          with 20.9% fewer complications (Tables 2 - 4)
to nonwinning peers. In addition, we identify
some key findings among comparison groups. For
performance measure details and definitions of
each comparison group, see the Methodology
section of this document.

* To be defined as a health system in this study, an organization must have at least two short-term, general, acute care hospitals with separate Medicare provider identification
  numbers. Systems with multiple hospital facilities reporting under one provider ID are profiled as a single hospital in the Watson Health 100 Top Hospitals® study.
** Mortality and complications index values cannot be compared among the three different comparison groups because they are normalized by comparison group.

                                                                                                                                                                                     11
15 Top Health Systems had fewer healthcare-                                                 15 Top Health Systems had mixed results on
 associated infections*                                                                      longer-term outcomes
 A new ranked measure in the 2018 study,                                                     Several patient groups are included in the 30-
 healthcare-associated infections (HAIs)**,                                                  day mortality and readmission extended care
 captures information about the quality of inpatient                                         composite metrics. The mean 30-day mortality
 care. Based on nationwide data availability, we                                             rate includes heart attack (AMI), heart failure (HF),
 built a composite measure of HAI performance                                                pneumonia, chronic obstructive pulmonary disease
 at the system level, considering up to six HAIs,                                            (COPD), and stroke patient groups. The mean 30-
 depending on assigned comparison group. The                                                 day readmission rate includes AMI, HF, pneumonia,
 six reported HAIs are: methicillin-resistant                                                total hip arthroplasty and/or total knee arthroplasty
 staphylococcus aureus (MRSA-bloodstream),                                                   (THA/TKA), COPD, and stroke patient groups.
 central line-associated blood stream infections,
 catheter-associated urinary tract infections,                                                30-day mortality results
 clostridium difficile (C. diff), surgical site infections
                                                                                                 –– In this year’s study, the winning systems had
 (SSIs) following colon surgery, and SSIs following
                                                                                                    a higher mean 30-day mortality rate than the
 an abdominal hysterectomy.
                                                                                                    nonwinning systems, due to slightly higher
    –– Among all types of systems, winners overall                                                  rates among large and small systems
       had a more favorable composite median HAI                                                    (Tables 1 - 4)
       index value than nonwinner peers, at 0.77
                                                                                                 –– Small health systems displayed the largest
       versus 0.92, respectively; this reflects 16.2%
                                                                                                    gap between winners and nonwinning peers
       fewer infections occurring at the 15 Top Health
                                                                                                    on 30-day mortality (13.4% versus 12.9%),
       Systems compared to other peer systems
                                                                                                    while medium health system winners had
       (Table 1)
                                                                                                    performance the same as nonwinning peers,
    –– Small health system winners and nonwinners                                                   at 12.7%, which was also the lowest median
       showed the most dramatic difference on                                                       30-day mortality value observed in this year’s
       HAI performance: winners had a median HAI                                                    study (Tables 1, 3, and 4)
       composite index value of 0.64, which was
       26.5% lower than the median HAI index score                                            30-day readmission results
       at nonwinning systems (0.87) (Tables 2 - 4)
                                                                                                 –– Winning health systems had lower 30-day
                                                                                                    readmission rates than their nonwinning peers
                                                                                                    nationally (0.6 percentage points lower) and for
                                                                                                    all comparison groups (Table 1)
                                                                                                 –– Small winning systems had the best mean
                                                                                                    30-day readmission rate (14.4%) among all
                                                                                                    comparison groups and outperformed their
                                                                                                    nonwinning peers by the greatest margin,
                                                                                                    0.9 percentage points (Tables 2 - 4)

 * HAI index values cannot be compared among the three different comparison groups because they are normalized by comparison group.
 ** As developed by a unit of the Centers for Disease Control and Prevention, the National Healthcare Safety Network, and reported by the Centers for Medicare & Medicaid Services
    (CMS) in the public Hospital Compare data set.

12 IBM Watson Health
Patients treated in 15 Top Health Systems                                              15 Top Health Systems hospitals had lower
hospitals returned home sooner*                                                        Medicare spending per beneficiary episode costs
   –– Winning systems had a median average length                                          –– Overall, winning systems had a 5.6% lower
      of stay (ALOS) of 4.4 days, nearly a half day                                           Medicare spending per beneficiary (MSPB)
      shorter than the nonwinner median of 4.9 days                                           index than nonwinners (Table 1)
      (Table 1)
                                                                                           –– Medium health systems showed the greatest
   ––The ALOS difference between winners                                                      difference between winners and nonwinners
     and nonwinners was consistent across all                                                 with an 11.8% lower MSPB index (Table 2)
     comparison groups, with benchmark systems
                                                                                           –– Medium winning systems also had the lowest
     discharging patients 0.4 days sooner
                                                                                              average MSPB index (0.88) among the
     (Tables 2 - 4)
                                                                                              comparison groups (Table 2)
Patients spent less time in 15 Top Health Systems
                                                                                        Patients rated 15 Top Health Systems hospitals
emergency departments
                                                                                        higher than peer system hospitals
The mean emergency department (ED) throughput
composite metric measures the amount of time                                               –– Winning systems had a 2.3% higher overall
spent in the ED. It includes median time from ED                                              score on the Hospital Consumer Assessment of
arrival to ED departure for admitted ED patients,                                             Healthcare Providers and Systems (HCAHPS),
and median time from ED arrival to ED departure                                               which tells us that patients treated by
for non-admitted ED patients.                                                                 members of the top health systems reported a
                                                                                              better overall hospital experience than those
   –– Overall, winning systems had, on average, 40
                                                                                              treated in nonwinning peer hospitals (Table 1)
      minutes shorter ED wait times per patient than
      nonwinners (Table 1)                                                                 –– Small health system winners had the best
                                                                                              HCAHPS score (270.3) among the comparison
   ––The greatest difference between winning
                                                                                              groups (Tables 2 - 4)
     systems and their peers was in the medium
     health systems comparison group, with                                                 ––The small winning systems also had the biggest
     medium system winners averaging 40.2                                                    lead over nonwinning peers, with an HCAHPS
     minutes less time spent in the ED per patient                                           score that was 3.1% higher (Tables 2 - 4)
     visit than nonwinners; the range of time
     saved was between 34.8 and 40.2 minutes
     (Tables 2 - 4)

* ALOS cannot be compared among the three different comparison groups because values are normalized by comparison group.

                                                                                                                                           13
Table 1. National health system performance comparisons (all systems)
  Performance measure                                             Medians                             Benchmark compared with peer group
                                                                  Winning           Nonwinning        Difference     Percent          Comments
                                                                  benchmark         peer group                       difference
                                                                  health            of US Health
                                                                  systems           systems

  Inpatient mortality index1                                      0.88              1.03              -0.15          -14.6%           lower mortality
  Complications index1                                            0.85              1.02              -0.18          -17.3%           fewer complications
  Healthcare-associated infection (HAI) index         2
                                                                  0.77              0.92              -0.15          -16.2%           fewer infections
  30-day mortality rate (%)       3
                                                                  12.9              12.7              0.1            n/a4
                                                                                                                                      higher 30-day mortality
  30-day readmission rate (%)         3
                                                                  14.7              15.3              -0.6           n/a4
                                                                                                                                      fewer 30-day readmissions
  Average length of stay (ALOS) (days)1                           4.4               4.9               -0.4           -8.8%            shorter stays
  Emergency department (ED) throughput measure2                   180.3             220.3             -40.0          -18.2%           less time to service
  Medicare spending per beneficiary (MSPB) index           2
                                                                  0.94              0.99              -0.06          -5.6%            lower episode cost
  Hospital Consumer Assessment of Healthcare                      270.1             264.0             6.1            2.3%             better patient experience
  Providers and Systems (HCAHPS) score2

  1. Mortality, complications and average length of stay based on Present on Admission (POA)-enabled risk models applied to MedPAR 2015 and 2016 data (ALOS 2016 only).
  2. HAI, ED measure, MSPB and HCAHPS data from CMS Hospital Compare Jan 1, 2016 - Dec 31, 2016 data set.
  3. 30-day rates from CMS Hospital Compare July 1, 2013-June 30, 2016 data set.
  4. We do not calculate percent difference for this measure because it is already a percent value.
  NNote: Measure values are rounded for reporting, which may cause calculated differences to appear off.

  Table 2. Large health system performance comparisons
  Performance measure                                             Medians                             Benchmark compared with peer group
                                                                  Benchmark         Peer group        Difference     Percent          Comments
                                                                  health            of US health                     difference
                                                                  systems           systems

  Inpatient mortality index1                                      0.89              1.00              -0.11          -11.0%           Lower mortality
  Complications index     1
                                                                  0.82              1.04              -0.22          -20.9%           Fewer complications
  HAI index  2
                                                                  0.79              0.93              -0.14          -14.9            Fewer infections
  30-day mortality rate (%)3                                      12.8              12.7              0.1            n/a4             Higher 30-day mortality
  30-day readmission rate (%)3                                    14.6              15.3              -0.7           n/a4             Fewer 30-day readmissions
  ALOS (days)    1
                                                                  4.4               4.9               -0.4           -8.9%            Shorter stays
  ED throughput measure       2
                                                                  189.9             229.6             -39.7          -17.3%           Less time to service
  MSPB index     2
                                                                  0.94              0.99              -0.05          -5.5%            Lower episode cost
  HCAHPS score2                                                   270.1             264.0             6.0            2.3%             Better patient experience

  1. Mortality, complications, and ALOS based on POA-enabled risk models applied to MEDPAR 2015 and 2016 data (ALOS 2016 only).
  2. HAI, ED measure, MSPB, and HCAHPS data from CMS Hospital Compare Jan. 1, 2016 - Dec. 31, 2016 data set.
  3. 30-day rates from CMS Hospital Compare July 1, 2013 - June 30, 2016 data set.
  4. We do not calculate percent difference for this measure because it is already a percent value.
  Note: Measure values are rounded for reporting, which may cause calculated differences to appear off.

14 IBM Watson Health
Table 3. Medium health system performance comparisons
Performance measure                                           Medians                           Benchmark compared with peer group
                                                              Benchmark        Peer group       Difference       Percent        Comments
                                                              health           of US health                      difference
                                                              systems          systems

Inpatient mortality index1                                    0.85             1.01             -0.16            -15.9%         Lower mortality
Complications index    1
                                                              0.85             1.03             -0.18            -17.0%         Fewer complications
HAI index  2
                                                              0.77             0.92             -0.15            -16.5%         Fewer infections
30-day mortality rate (%)3                                    12.7             12.7             0.0              n/a4           No difference in 30-day mortality
30-day readmission rate (%)3                                  14.7             15.2             -0.5             n/a4           Fewer 30-day readmissions
ALOS (days)    1
                                                              4.5              4.9              -0.4             -7.6%          Shorter stays
ED throughput measure      2
                                                              181.0            221.2            -40.2            -18.2%         Less time to service
MSPB index     2
                                                              0.88             1.00             -0.12            -11.8%         Lower episode cost
HCAHPS score2                                                 269.2            265.0            4.2              1.6%           Better patient experience

1. Mortality, complications, and ALOS based on POA-enabled risk models applied to MEDPAR 2015 and 2016 data (ALOS 2016 only).
2. HAI, ED measure, MSPB, and HCAHPS data from CMS Hospital Compare Jan. 1, 2016 - Dec. 31, 2016 data set.
3. 30-day rates from CMS Hospital Compare July 1, 2013 - June 30, 2016 data set.
4. We do not calculate percent difference for this measure because it is already a percent value.
Note: Measure values are rounded for reporting, which may cause calculated differences to appear off.

Table 4. Small health system performance comparisons
Performance measure                                           Medians                           Benchmark compared with peer group
                                                              Benchmark        Peer group       Difference       Percent        Comments
                                                              health           of US health                      difference
                                                              systems          systems

Inpatient mortality index1                                    0.99             1.03             -0.04            -3.4%          Lower mortality
Complications index    1
                                                              0.83             1.00             -0.17            -17.1%         Fewer complications
HAI index  2
                                                              0.64             0.87             -0.23            -26.5%         Fewer infections
30-day mortality rate (%)3                                    13.4             12.9             0.5              n/a4           Higher 30-day mortality
30-day readmission rate (%)3                                  14.4             15.3             -0.9             n/a4           Fewer 30-day readmissions
ALOS (days)    1
                                                              4.5              4.9              -0.4             -8.7%          Shorter stays
ED throughput measure      2
                                                              177.3            212.2            -34.8            -16.4%         Less time to service
MSPB index     2
                                                              0.95             0.99             -0.04            -4.0%          Lower episode cost
HCAHPS score2                                                 270.3            262.2            8.1              3.1%           Better patient experience

1. Mortality, complications, and ALOS based on POA-enabled risk models applied to MEDPAR 2015 and 2016 data (ALOS 2016 only).
2. HAI, ED measure, MSPB, and HCAHPS data from CMS Hospital Compare Jan. 1, 2016 - Dec. 31, 2016 data set.
3. 30-day rates from CMS Hospital Compare July 1, 2013 - June 30, 2016 data set.
4. We do not calculate percent difference for this measure because it is already a percent value.
Note: Measure values are rounded for reporting, which may cause calculated differences to appear off.

                                                                                                                                                                    15
Appendix A.

16 IBM Watson Health
                                                                                                                                                                                                                                      Winning health system results

                                                                                                                                                                                               For a list of all hospitals included
                                                                                                                                                                                               In Table 5, we provide the 15 Top
                                                                                                                                                                                               Health Systems values for each of

                                                                                                                                                                                               in each winning health system, see
                                                                                                                                                                                               the study’s performance measures.

                       Table 5. Winning health system performance measure results
                                           Winning system name                         Mortality        Complications HAI index2               30-day           30-day      ALOS1   ED            MSPB                HCAHPS
                                                                                       index1           index1                                 mortality        readmission         measure       index4              score4
                                                                                                                                               rate3            rate3               mean
                                                                                                                                                                                    minutes4
                       Large health        Mayo Foundation                             0.82             1.23                  0.7              11.9             14.6        4.5     166.6         0.92                278.9
                       systems
                                           Mercy                                       0.89             0.72                  0.8              12.8             14.9        4.4     174.6         0.96                269.9
                                           Sentara Healthcare                          0.87             0.82                  0.9              12.9             15.2        4.7     235.8         0.94                270.1
                                           St. Luke's Health System                    0.96             0.74                  0.8              12.5             14.1        4.1     206.3         0.88                268.9
                                           UCHealth                                    0.91             0.88                  0.9              12.9             14.1        4.1     189.9         0.98                271.5
                       Medium health       Aspirus                                     0.98             0.71                  0.6              13.3             14.8        3.9     154.6         0.69                269.2
                       systems
                                           HealthPartners                              0.87             1.08                  0.8              12.7             14.7        4.3     174.8         0.88                268.7
                                           Mercy Health - Cincinnati                   0.68             0.85                  0.5              12.0             15.4        4.6     210.0         1.00                271.6
                                           Mission Health                              0.82             1.02                  1.0              13.0             13.2        4.5     195.1         0.87                275.2
                                           TriHealth                                   0.85             0.83                  0.8              12.3             14.7        4.5     181.0         0.99                267.3
                       Small health        Asante                                      0.99             0.53                  0.6              13.1             14.3        4.5     185.0         0.91                270.3
                       systems
                                           CHI St. Joseph Health                       0.73             0.88                  0.5              14.1             15.3        4.6     166.6         0.98                273.0
                                           Maury Regional Health                       1.05             0.51                  0.5              13.6             15.5        3.9     177.3         0.95                269.0
                                           Roper St. Francis Healthcare                1.06             0.83                  1.1              12.0             13.9        4.8     159.2         0.99                276.9
                                           UPMC Susquehanna Health System              0.74             1.09                  0.7              13.4             14.4        4.3     180.3         0.79                267.0

                       1. Mortality, complications, and ALOS based on POA-enabled risk models applied to MEDPAR 2015 and 2016 data (ALOS 2016 only).
                       2. HAI, ED measure, MSPB, and HCAHPS data from CMS Hospital Compare Jan. 1, 2016 - Dec. 31, 2016 data set.
                       3. 30-day rates from CMS Hospital Compare July 1, 2013 - June 30, 2016 data set.
                       Note: Mortality, complications, HAI and ALOS measures cannot be compared across comparison groups because they are normalized by comparison group.
Top and bottom quintile results                                                           ––They had somewhat lower mean 30-day
We divided all the health systems in this study into                                        mortality rates (0.4 percentage points lower;
performance quintiles, by comparison group, based                                           includes AMI, HF, pneumonia, COPD, and
on their performance on the study’s measures.                                               stroke patients)
In Table 6, we have highlighted differences                                               ––They had lower mean 30-day readmission
between the highest- and lowest-performing                                                  rates (14.8% versus 15.5%; includes AMI,
quintiles by providing their median scores on the                                           HF, pneumonia, THA/TKA, COPD, and
study performance measures. (See Appendix B                                                 stroke patients)
for a list of the health systems included in the
top-performance quintile and Appendix D for all                                           ––They had dramatically lower mean ED wait
systems included in the study.)                                                             times, with an average difference of 62
                                                                                            minutes per ED patient (24.1% less than the
The top quintile systems outperformed their lowest                                          bottom quintile)
quintile peers in the following ways:                                                     ––They were more efficient, releasing patients
  ––They had much better patient outcomes:                                                  almost one day (0.9) sooner than the lowest
    19.7% lower mortality and 5.7% lower                                                    performers and at a 5.3% lower MSPB index
    complications                                                                         ––They scored 11.3 points higher on the HCAHPS
  ––They had fewer occurrences of HAIs in their                                             overall patient rating of care
    facilities: a 25.8% smaller median HAI index
    value (0.8) in the top performance quintile

Table 6. Comparison of health systems in the top and bottom quintiles of performance1
Performance measure                      Top quintile           Bottom quintile        Difference              Percent          Top versus bottom
                                         median                 median                                         difference       quintile
Inpatient mortality index2               0.89                   1.11                   -0.22                   -19.7%           Lower mortality
Complications index2                     0.99                   1.05                   -0.06                   -5.7%            Fewer complications
HAI index  3
                                         0.8                    1.0                    -0.3                    -25.8%           Fewer infections
30-day mortality rate (%)    4
                                         12.6                   12.9                   -0.4                    n/a 5
                                                                                                                                Lower 30-day mortality
30-day readmission rate (%)      4
                                         14.8                   15.5                   -0.8                    n/a 5
                                                                                                                                Fewer 30-day readmissions
ALOS  2
                                         4.5                    5.4                    -0.9                    -16.1%           Shorter stays
ED measure mean minutes3                 195.0                  256.9                  -62.0                   -24.1%           Less time to service
MSPB index3                              0.96                   1.01                   -0.05                   -5.3%            Lower episode cost
HCAHPS score    3
                                         269.0                  257.7                  11.3                    4.4%             Better patient experience

1. Top and bottom performance quintiles were determined by comparison group and aggregated to calculate medians.
2. Mortality, complications, and ALOS based on POA-enabled risk models applied to MEDPAR 2015 and 2016 data (ALOS 2016 only).
3. HAI, ED measure, MSPB, and HCAHPS data from CMS Hospital Compare Jan. 1, 2016 - Dec. 31, 2016 data set
4. 30-day rates from CMS Hospital Compare July 1, 2013 - June 30, 2016, data set.
5. We do not calculate percent difference for this measure because it is already a percent value.
Note: Measure values are rounded for reporting, which may cause calculated differences to appear off.

                                                                                                                                                            17
Test metrics: Reported for information only            Excess days in acute care measures
 Every year, we evaluate the 15 Top Health Systems      The newest set of measures available from CMS
 study and explore whether new measures would           in the Hospital Compare data set are the excess
 enhance the value of the analysis we provide.          days in acute care (EDAC) measures for AMI and
 For this 2018 study, we are testing several            HF. CMS defines “excess days” as the difference
 new performance measures that update basic             between a hospital’s average days in acute
 standards of inpatient care and expand the             care and expected days, based on an average
 balanced scorecard across the continuum of care.       hospital nationally. Days in acute care include
 These metrics were not used in the ranking and         days spent in an ED, a hospital observation unit,
 selection of winning health systems.                   or a hospital inpatient unit for 30 days following
                                                        a hospitalization. The data period in our study for
 If you would like to provide feedback on               these measures is the same as for the other 30-day
 the following proposed measures, email                 metrics for specific patient conditions: three years,
 100tophospitals@us.ibm.com.                            combined (July 1, 2013 - June 30, 2016).

 30-day all-cause, hospital-wide                        90-day episode-of-care payment measure
 readmission measure                                    Another measure recently made available in the
 We are continuing to publish the hospital-wide,        Hospital Compare data set is the 90-day episode-
 30-day readmission measure, which CMS is               of-care payment metric for primary, elective
 publicly reporting in the Hospital Compare data set    THA/TKA. Like the other 30-day episode-of-
 to provide an overall readmission comparison, for      care payment measures, CMS calculates risk-
 information only. However, we rank on a composite      standardized payments associated with a 90-day
 score based on the publicly available individual       episode of care, compared to an average hospital
 patient groups. The data period for the hospital-      nationally. The measure summarizes payments
 wide readmission measure is July 1, 2015 -             for patients across multiple care settings, services,
 June 30, 2016.                                         and supplies during the 90-day period, which
                                                        starts on the day of admission. The data period for
 30-day episode-of-care payment measures                this measure combines three years, April 1, 2013 -
 We are continuing to publish risk-standardized         March 31, 2016.
 payments associated with 30-day episode-of-care
 measures for three patient groups that are now         90-day complication measure
 being published by CMS in the Hospital Compare         Along with the THA/TKA 90-day payment
 data set. These measures capture differences in        measure recently made available in Hospital
 services and supplies provided to patients who         Compare data, CMS is also publishing a THA/
 have been diagnosed with AMI, HF, or pneumonia.        TKA 90-day complication measure. This measure
 According to the CMS definition of these measures,     calculates a risk-standardized complication rate for
 they are the sum of payments made for care and         THA/TKA procedures using the occurrence of one
 supplies starting the day the patient enters the       or more of the subsequent complications within
 hospital and for the next 30 days. In our study, the   the specified timeframes. The data period for this
 data period for these measures is the same as          measure combines three years, April 1, 2013 -
 for the other 30-day metrics for specific patient      March 31, 2016 (complications are listed on the
 conditions: three years, combined (July 1, 2013 -      following page).
 June 30, 2016).

18 IBM Watson Health
–– AMI, pneumonia, or sepsis/septicemia/shock                                            –– On all reported CMS episode measures of cost,
     during or within seven days of index admission                                           winning systems consistently outperformed
                                                                                              their nonwinner peers; the difference was
  –– Surgical site bleeding, pulmonary embolism,
                                                                                              greatest for THA/TKA 90-day episode payment
     or death during or within 30 days of index
                                                                                              (5.8%)
     admission
                                                                                           –– Winning health systems performed better
  –– Mechanical complication or periprosthetic joint
                                                                                              on the AMI and HF 30-day EDAC measures,
     infection/wound infection during or within 90
                                                                                              showing that patients spent fewer days than
     days of index admission
                                                                                              expected in the ED, in observation, or back in
                                                                                              the hospital after an initial index acute care
See the CMS website for measure methodology25.
                                                                                              stay (5.2 days under the expected amount
                                                                                              for AMI EDAC and 8.5 under expected for HF
Table 7 shows the national performance of
                                                                                              EDAC); whereas nonwinning peers averaged
benchmark and peer health systems on the
                                                                                              7.6 and 8.1 days more than expected (EDAC
test metrics.
                                                                                              values are reported as excess days per 100
                                                                                              discharges)
This year, the 15 Top Health Systems winners
outperformed nonwinning peers on all test                                                  –– On the 90-day THA/TKA complications rate,
measures, which is an interesting finding given that                                          another CMS measure of outcomes extended
these are independent variables not used in the                                               outside the hospital stay, winning systems had
selection of the winners.                                                                     0.3 percentage points lower complication rate
                                                                                              than peers
  –– Benchmark systems had stronger performance
     on 30-day, hospital-wide readmissions (14.8%
     versus 15.4% for peers)

 Table 7. Information-only measures – Health system performance comparisons (all classes)
 Performance measure                                              Medians                              Benchmark compared with peer group
                                                                  Benchmark         Peer group         Difference      Percent       Comments
                                                                  health systems    of US health                       difference
                                                                                    systems

 30-day, hospital-wide readmission rate1                          14.8              15.4               -0.63           n/a3          Fewer 30-day readmissions
 AMI 30-day episode payment        1
                                                                  $22,798           $23,204            -$406           -1.8%         Lower episode cost
 HF 30-day episode payment1                                       $16,023           $16,376            -$353           -2.2%         Lower episode cost
 Pneumonia 30-day episode payment            1
                                                                  $17,117           $17,318            -$201           -1.2%         Lower episode cost
 AMI 30-day excess days in acute care         1
                                                                  -5.2              7.6                -12.81          n/a 3
                                                                                                                                     Fewer excess days
 HF 30-day excess days in acute care      1
                                                                  -8.5              8.1                -16.62          n/a 3
                                                                                                                                     Fewer excess days
 THA/TKA* 90-day episode payment         2
                                                                  $20,653           $21,931            -$1,278         -5.8%         Lower episode cost
 THA/TKA* 90-day complications rate2                              2.4               2.7                -0.30           n/a3          Fewer complications

 1. 30-day measures from CMS Hospital Compare July 1, 2013 - June 30, 2016 data set.
 2. 90-day measures from CMS Hospital Compare April 1, 2013 - March 31, 2016 data set.
 3. We do not calculate percent difference for these measures because it can be a negative number or is already a percent value.
 * Primary, elective total hip arthroplasty and total knee arthroplasty.

                                                                                                                                                                 19
Financial metrics                                                                             –– Overall, benchmark health system
 We continue to publish the financial measures                                                     performance is better than nonwinning peers,
 each year for information only, as audited financial                                              both on operating margin (1.2 percentage
 statements are not available for all systems                                                      points higher among winners) and long-term
 included in the study*. These measures are not                                                    debt-to-capitalization ratio (LTD/cap)
 included in the ranking and selection of benchmark                                                (0.1 lower ratio among winners)
 health systems.                                                                                –– Notably, medium health system winners had
                                                                                                   a much higher operating profit margin than
 Results for included systems are found in                                                         nonwinners (8.7% versus 2.4%) and showed
 Table 8 below.                                                                                    a greater difference on the LTD/cap (0.2
                                                                                                   versus 0.4)

  Table 8. Information-only – Financial performance
  Performance measure                Health system                       Medians                                                                Difference
                                     comparison group
                                                                         Benchmark health systems           Peer group of US                    Benchmark compared
                                                                                                            health systems                      with peer group

  Operating margin                   All systems                         4.2                                3.0                                 1.2
  (percentage)
                                     Large                               4.3                                3.6                                 0.7
                                     Medium                              8.7                                2.4                                 6.3
                                     Small                               4.0                                2.2                                 1.8
  Long-term debt-to-                 All systems                         0.3                                0.3                                 -0.1
  capitalization ratio
  (LTD/cap)                          Large                               0.3                                0.3                                 0.0
                                     Medium                              0.2                                0.4                                 -0.1
                                     Small                               0.3                                0.3                                 -0.1

  Note: Data sourced from audited 2016 financial reports via dacbond.com, emma.msrb.org, yahoo.brand.edgar-online.com, and sec.gov.

 * A total of 84.2% of parent and independent systems published audited financial statements for 2016. Subsystems that are members of a larger “parent” health system generally
   do not have separate audited financial statements. This translated into 65.1% of all in-study health systems with available financial reports.

20 IBM Watson Health
–– Ranking systems on each of the performance
Methodology                                                                                        measures by comparison group
                                                                                                –– Determining the 15 top performers (five in
Watson Health 15 Top Health Systems is a                                                           each comparison group) from the health
quantitative study that annually identifies 15                                                     systems’ overall rankings, based on their
US health systems with the highest overall                                                         aggregate performance (sum of individual
achievement on a balanced scorecard.                                                               weighted measure ranks)

The health system scorecard is based on the                                                 The following section is intended to be an overview
100 Top Hospitals® national balanced scorecard                                              of these steps. To request more detailed information
methodologies and focuses on four performance                                               on any of the study methodologies outlined here,
domains: inpatient outcomes, extended outcomes,                                             email us at 100tophospitals@us.ibm.com or call
operational efficiency, and patient experience.                                             800-525-9083.

This 2018 health systems study includes nine
measures that provide an objective comparison                                               Building the database of hospitals
of health system performance using publicly                                                 Like all the 100 Top Hospitals studies, the 15 Top
available data. The health systems with the highest                                         Health Systems study uses only publicly available
achievement are those with the highest ranking on                                           data. The data for this study primarily came from:
a composite score based on these nine measures.
                                                                                                –– Medicare Provider Analysis and Review
To analyze health system performance, we include                                                   (MEDPAR) data set*
data for short-term, acute care, nonfederal US                                                  –– Centers for Medicare & Medicaid Services
hospitals, as well as cardiac, orthopedic, women’s,                                                (CMS) Hospital Compare data set
and critical access hospitals (CAHs) that are
members of the health systems.                                                              We use MEDPAR patient-level demographic,
                                                                                            diagnosis, and procedure information to calculate
The main steps we take in selecting the top 15                                              mortality, complications, and length of stay (LOS)
health systems are:                                                                         by aggregating member hospital data to the health
   –– Building the database of health systems,                                              system level. The MEDPAR data set contains
      including special selection and exclusion criteria                                    information on the approximately 15 million
                                                                                            Medicare patients discharged annually from US
   –– Identifying which hospitals are members of                                            acute care hospitals. In this year’s study, we used
      health systems                                                                        the most recent two federal fiscal years (FFYs) of
   –– Aggregating the patient-level and hospital-                                           MEDPAR data available (2015 and 2016), which
      level data from member hospitals and                                                  included Medicare Advantage health maintenance
      calculating a set of performance measures at                                          organization encounters. The 100 Top Hospitals
      the system level                                                                      program has used the MEDPAR database for
                                                                                            many years. We believe it to be an accurate and
   –– Classifying health systems into comparison                                            reliable source for the types of high-level analyses
      groups based on total operating expense                                               performed in this study.

* The MEDPAR data years quoted in 100 Top Hospitals research are based on an FFY, a year that begins on October 1 of each calendar year and ends on September 30 of the
  following calendar year. FFYs are identified by the year in which they end (for example, FFY 2016 begins October 1, 2015, and ends September 30, 2016). Data for all CMS Hospital
  Compare measures is provided in calendar years, except the 30-day rates. CMS publishes the 30-day rates as three-year combined data values. We label these data points based
  on the end date of each data set. For example, July 1, 2013 - June 30, 2016, is named “2016.”

                                                                                                                                                                                  21
We used the CMS Hospital Compare data                 The recalibrated models were used in producing
 set published in the third quarter of 2017 for        the risk-adjusted inpatient mortality and
 healthcare-associated infection (HAI) measures,       complications indexes, based on two years of
 30-day mortality rates, 30-day readmission rates,     MEDPAR data (2015 and 2016). The severity-
 Medicare spending per beneficiary (MSPB) index,       adjusted LOS was produced based on MEDPAR
 and Hospital Consumer Assessment of Healthcare        2016 data.
 Providers and Systems (HCAHPS) patient
 perception-of-care data26.                            Present-on-admission coding adjustments
                                                       From 2010 through 2016, we have observed a
 We also used the 2016 Medicare cost reports,          significant rise in the number of principal diagnosis
 published in the federal Hospital Cost Report         and secondary diagnosis codes that do not have
 Information System (HCRIS) third-quarter 2017         a valid POA indicator code in the MEDPAR data
 data set, to create our proprietary database for      files. Since 2011, an invalid code of “0” has been
 determining system membership based on “home          appearing. This phenomenon has led to an artificial
 office” or “related organization” relationships       rise in the number of complications that appear to
 reported by hospitals. The cost reports were also     be occurring during the hospital stay. See Appendix
 used to aggregate member hospital total operating     C for details.
 expense to the system level. This data was
 used to classify health systems into three            To correct for this bias, we adjust MEDPAR record
 comparison groups.                                    processing through our inpatient mortality and
                                                       complications risk models and LOS severity-
 Risk- and severity-adjustment models                  adjustment model as follows:
 The IBM Watson Health™ proprietary risk- and
                                                         1. Original, valid (Y, N, U, W, or 1) POA codes
 severity-adjustment models for inpatient mortality,
                                                            assigned to diagnoses were retained
 complications, and LOS have been recalibrated for
 this study release using FFY 2015 data available        2. Where a POA code of “0” appeared, we took
 in the all-payer Watson Health Projected Inpatient         the next four steps
 Database (PIDB). The PIDB is one of the largest
                                                            a. We treated all diagnosis codes on the
 US inpatient, all-payer databases of its kind,
                                                               CMS exempt list as “exempt,” regardless
 containing approximately 23 million inpatient
                                                                of POA coding
 discharges annually, obtained from approximately
 5,000 hospitals, which comprise more than 65%              b. We treated all principal diagnoses as
 of the nonfederal US market. Watson Health risk-             “present on admission”
 and severity-adjustment models take advantage
                                                            c. We treated secondary diagnoses where the
 of available present-on-admission (POA) coding
                                                                POA code “Y” or “W” appeared more than
 that is reported in all-payer data. Only patient
                                                               50% of the time in the Watson Health all-
 conditions that are present on admission are
                                                                payer database as “present on admission”
 used to determine the probability of death,
 complications, or the expected LOS.                        d. All others were treated as “not present”

22 IBM Watson Health
Hospital exclusions                                                                         Health system exclusions
After building the database, we exclude hospitals                                           Health systems are excluded if:
that would have skewed the study results.
                                                                                                –– One or more required measures are missing*
Excluded from the study were:
                                                                                                –– Fewer than 50% of member hospitals have valid
   –– Certain specialty hospitals (children’s,
                                                                                                   POA coding
      psychiatric, substance abuse, rehabilitation,
      cancer, and long-term acute care)                                                         –– Fewer than 50% of member hospitals have valid
                                                                                                   data for any one or more required measures**
   –– Federally owned hospitals
   –– Hospitals not located within the 50 states                                            After all system exclusions were applied, 338
      (such as those in Puerto Rico, Guam, and the                                          individual health systems were included in the
      US Virgin Islands)                                                                    2018 study.
   –– Hospitals with Medicare average LOS longer
                                                                                            NOTE: CMS does not publish MSPB measures for
      than 30 days in FFY 2016
                                                                                            Maryland hospitals due to a separate payment
   –– Hospitals with no reported Medicare patient                                           agreement. For this reason, we substituted the
      deaths in FFY 2016                                                                    comparison group median, and winner-excluded
                                                                                            Maryland health systems that had no reported
   –– Hospitals that had fewer than 60% of patient
                                                                                            MSPB measure to allow Maryland health systems
      records with valid POA codes
                                                                                            to remain in the study. If a Maryland health system
                                                                                            included hospitals in other states, we winner-
Cardiac, orthopedic, women’s hospitals, and CAHs
                                                                                            excluded them when more than 50% of their
are included in the study, if they are not excluded
                                                                                            member hospitals had no reported MSPB measure.
for any other criteria listed above.

In addition, specific patient records are
also excluded:
   –– Patients who were discharged to another
      short-term facility (this is done to avoid
      double-counting)
   –– Patients who were not at least 65 years old
   –– Rehabilitation, psychiatric, and substance
      abuse patients
   –– Patients with stays shorter than one day

After all exclusions were applied, 2,422 individual
hospitals were included in the 2018 study.

* For composite measures (HAI, 30-day mortality, 30-day readmissions), the exclusion is applied ONLY if all individual measure comprising the composite are missing.
    - For HAI, different numbers of individual measures were required depending on the comparison group (five for large and medium systems; three for small systems). A system not
      meeting the minimum was excluded. See Appendix C for details.
    - In systems where one or more individual 30-day mortality or 30-day readmission rates were missing, BUT NOT ALL, we calculated a median value for each, by comparison
      group, and substituted the median for the missing value.
** This rule was not applied to the HAI composite, which followed different exclusion logic. See Appendix C for details.

                                                                                                                                                                                  23
Identifying health systems                                                                To analyze health system performance, we
 To be included in the study, a health system must                                         aggregate data from all of a system’s included
  have at least two short-term, general, acute                                             hospitals. In the methodology summary tables in
  care hospitals with separate Medicare provider                                           this section, we provide specific details about the
  identification numbers. The minimum of two                                               calculations used for each performance measure
  hospitals must be met after hospital exclusions                                          and how these measures are aggregated to
  have been applied. In addition, we also include                                          determine system performance.
  any cardiac, orthopedic, women’s hospitals, and
  CAHs that passed the hospital exclusion rules                                            After all exclusions were applied and parent
  cited on the previous page. For the 2018 study,                                          systems identified, the final 2018 study group
  we identified the “parent” system by finding the                                         included 338 health systems with the profiles
 “home office” or “related organization”, as reported                                      outlined in Table 9.
  on the hospitals’ 2016 (or 2015) Medicare
  cost report.

  We identify health systems that have subsystems
  with their own reported home offices or related
  organization relationships. Both the parent system
  and any identified subsystems are treated as
 “health systems” for purposes of this study and are
  independently profiled. Hospitals that belong to a
  parent health system and a subsystem are included
  in both for analysis of system performance.

  Table 9. 2018 health systems study group
  System category              Systems                      Member hospitals             Medicare patient              Average hospitals            Average discharges
                                                                                         discharges, FFY               per system                   per system
                                                                                         2015
  Winning systems              15                           121                          330,760                       8.1                          22,051
  Nonwinning systems           323                          2,798                        9,368,880                     8.7                          29,006
  Total systems                338                          2,919                        9,699,640                     8.6                          28,697

  Note: A hospital can be a member of both a parent system and a subsystem of that parent. They will be included in both parent and subsystem member hospital counts.
  The total unduplicated hospital count in this study was 2,422 hospitals.

24 IBM Watson Health
Classifying health systems into                              The nine measures included in the 2018 study, by
comparison groups                                            performance domain, are:
Health system comparison groups
We refine the analysis of health systems by dividing         Inpatient outcomes
them into three comparison groups based on                     1. Risk-adjusted inpatient mortality index
total operating expense of the identified member
hospitals. This is done to develop more action-                2. Risk-adjusted complications index
driving benchmarks for like systems. For the 2018              3. Mean HAI index
study, the three comparison groups we used are
listed in Table 10.                                          Extended outcomes
                                                               4. Mean 30-day risk-adjusted mortality rate
Table 10. Health system comparison groups, defined
                                                                  (includes acute myocardial infarction [AMI],
Health system   Total operating     Number of    Number of
comparison      expense             systems in   winners
                                                                  heart failure [HF], pneumonia, chronic
group                               study                         obstructive pulmonary disease [COPD],
Large           > $1.85 billion     113          5                and stroke)
Medium          $800 million -      116          5             5. Mean 30-day risk-adjusted readmission
                $1.85 billion
                                                                  rate (includes AMI, HF, pneumonia, total hip
Small           < $800 million      109          5
                                                                  arthroplasty and/or total knee arthroplasty
Total systems                       338          15               [THA/TKA], COPD, and stroke)

Scoring health systems on weighted                           Operational efficiency
performance measures                                           6. Severity-adjusted average LOS
Evolution of performance measures
                                                               7. Mean emergency department (ED) throughput
We use a balanced scorecard approach, based on
                                                                  (wait time minutes)
public data, to select the measures we believe to
be most useful for boards, CEOs, and other leaders             8. MSPB index
in the current health system operating environment.
In addition, we continually review trends in the             Patient experience
healthcare market, to identify the need for, and
                                                               9. HCAHPS score (overall hospital performance)
availability of, new performance measurement
approaches. We welcome feedback from hospital
                                                             The data sources for these measures are listed in
and system executives on the usefulness of our
                                                             Table 11.
measures and our approach.

As the healthcare industry has changed, our
methods have evolved. Our current measures
are centered on four main components of system
performance: inpatient outcomes, extended
outcomes, operational efficiency, and patient
experience. Measures of financial performance are
also included for information only, as not all health
systems have publicly reported, audited financial
statements.

                                                                                                                 25
Table 11. Summary of measure data sources and data periods
  Performance measure                                    Current performance                                 Five-year trend performance
                                                         (15 Top Health Systems award selection)
  Risk-adjusted inpatient mortality index                MEDPAR federal fiscal year (FFY) 2015 and 2016*     MEDPAR FFY 2011 - 2016*
  Risk-adjusted complications index                      MEDPAR FFY 2015 and 2016*                           MEDPAR FFY 2011 - 2016*
  Mean HAI index                                         CMS Hospital Compare Calendar Year (CY) 2016        Trend not available
  Mean 30-day mortality rate                             CMS Hospital Compare July 1, 2013 - June 30, 2016   CMS Hospital Compare: Three-year data sets
                                                                                                             ending June 30 in 2013, 2014, 2015, 2016
  Mean 30-day readmission rate (AMI, HF,                 CMS Hospital Compare July 1, 2013 - June 30, 2016   CMS Hospital Compare: Three-year data sets
  pneumonia, THA/TKA**, COPD, stroke)                                                                        ending June 30 in 2013, 2014, 2015, 2016
  Severity-adjusted ALOS                                 MEDPAR FFY 2016                                     MEDPAR FFY 2012 - 2016
  Mean ED throughput measure                             CMS Hospital Compare CY 2016                        CMS Hospital Compare 2012 - 2016
  MSPB index                                             CMS Hospital Compare CY 2016                        CMS Hospital Compare 2012 - 2016
  HCAHPS score (overall hospital rating)                 CMS Hospital Compare CY 2016                        CMS Hospital Compare 2012 - 2016

  * Two years of data are combined for each study year data point.
  ** Primary, elective total hip arthroplasty and total knee arthroplasty.

 Following is the rationale for the selection of our                                  Extended outcomes
 balanced scorecard domains and the measures                                          The extended outcomes measures (30-day
 used for each.                                                                       mortality rates for AMI, HF, pneumonia, COPD, and
                                                                                      stroke patients; and 30-day readmission rates for
 Inpatient outcomes                                                                   AMI, HF, pneumonia, THA/TKA, COPD, and stroke
 Our measures of inpatient outcomes include                                           patients) help us understand how the system’s
 three measures: risk-adjusted mortality index,                                       patients are faring over a longer period27. These
 risk-adjusted complications index, and mean HAI                                      measures are part of the CMS Hospital Value-
 index. These measures show us how the system                                         Based Purchasing Program and are widely reported
 is performing on what we consider to be the most                                     on in the industry. Hospitals with lower values
 basic and essential care standards (survival, error-                                 appear to be providing, or coordinating the care
 free care, and infection prevention) while treating                                  continuum with better medium-term results for
 patients in their hospitals.                                                         these conditions.

                                                                                      As systems become more interested in contracting
                                                                                      for population health management, we believe that
                                                                                      understanding outcomes beyond the walls of the
                                                                                      acute care setting is imperative. We are committed
                                                                                      to adding new metrics that assess performance
                                                                                      along the continuum of care as they become
                                                                                      publicly available.

26 IBM Watson Health
You can also read
NEXT SLIDES ... Cancel