Protocol Expanding Cost-Effectiveness Analysis to All of Health Care: Comparisons between CEAs on Pharmaceuticals and Medical/Surgical Procedures ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
11100 Santa Monica Boulevard, Suite 500
Los Angeles, CA 90025
2 Bethesda Metro Center, Suite 850
Bethesda, MD 20814
Phone: 310 984 7793
Fax: 310 982 6311
Protocol
Expanding Cost-Effectiveness Analysis to All of
Health Care:
Comparisons between CEAs on
Pharmaceuticals and Medical/Surgical
Procedures
Version: 3.0
June 8, 2017
Project Team:
James Baumgardner, PhD
Senior Research Economist, Precision Health Economics
Michelle Brauer, BS
Associate Research Scientist, Precision Health Economics
Katie Everson, MS
Research Programmer, Precision Health Economics
Michelle Skornicki, MPH
Senior Research Scientist, Precision Health Economics
Peter Neumann, ScD
Director, Center for the Evaluation of Value and Risk in Health at the Institute for Clinical Research and
Health Policy Studies at Tufts Medical Center; Professor of Medicine, Tufts University School of
MedicineThis work is licensed under the Creative Commons Attribution 4.0
International License. To view a copy of this license, visit
http://creativecommons.org/licenses/by/4.0/.
Version: [3.0] -1-Funding
This study was funded by the Innovation and Value Initiative, a collaboration among thought
leaders in academia, patient advocacy organizations, payers, pharmaceutical companies,
providers, delivery systems and other organizations dedicated to preserving innovation, value,
and choice in the healthcare system.
Disclosures
James Baumgardner, Michelle Brauer, Katie Everson, and Michelle Skornicki are employees of
and Peter Neumann is a principal scientific advisor to Precision Health Economics, a health
economics consultancy providing services to the life sciences industry.
About the Innovation and Value Initiative
The Innovation and Value Initiative (IVI) is a multi-stakeholder initiative that seeks to improve
the way value is measured and rewarded in the healthcare system to promote the development
and use of high value interventions that advance human health. To achieve this, IVI pursues the
following goals:
Establish best practices for measuring the real-world value of healthcare technologies
using both existing and innovative scientific methods;
Provide a range of marketplace stakeholders – including patients, consumers, providers,
healthcare systems, and payers – with salient, accurate, and actionable information about
value in healthcare;
Develop and test innovative approaches to link healthcare spending to value.
The IVI is hosted by Precision Health Economics, a health economics consultancy. IVI’s
direction and research agenda are determined in collaboration with its Panel of Health Advisors,
which includes representatives from patient advocacy organizations, pharmaceutical firms,
academia, insurers, and health systems. All funding supports IVI’s overall activities, with no
funding or funder tied to specific activities or research projects.
Acknowledgements
Anirban Basu of the University of Washington served as an external reviewer and provided
helpful comments on an earlier version. Mark Linthicum and Ahva Shahabi of Precision Health
Economics provided advice and assistance in earlier phases of the project.
Version: [3.0] -2-Table of Contents
Funding ....................................................................................................................................... 2
Disclosures .................................................................................................................................. 2
About the Innovation and Value Initiative ....................................................................................... 2
Acknowledgements ...................................................................................................................... 2
Table of Contents ......................................................................................................................... 3
List of Abbreviations .................................................................................................................... 5
Synopsis ...................................................................................................................................... 6
1. Introduction ....................................................................................................................... 7
1.1. Objectives ........................................................................................................................................ 7
2. Study Design ...................................................................................................................... 8
2.1. Overview of study design .................................................................................................................. 8
2.2. Data source/data collection process ..................................................................................................... 9
2.2.1. Included Variables ..................................................................................................................... 9
3. Statistical Analysis............................................................................................................ 2
3.1. Simple Comparisons of Pharmaceutical and Medical/Surgical CEAs ........................................... 2
3.2. Logistic Regressions ...................................................................................................................... 3
3.3. Evidence On The Supply of Clinical Trial Data .............................................................................. 8
4. Study Limitations/Strengths ................................................................................................. 8
5. References ......................................................................................................................... 9
6. Appendix. .........................................................................................................................10
6.1. Examples of Assignment of the Property Rights Variable ............................................................ 10
6.2. Background Bibliography.............................................................................................................. 10
6.2.1. Methodology and Trends in CEA .......................................................................................... 10
6.2.2. Medical/Surgical Studies ....................................................................................................... 12
6.2.3. Pharmaceutical Studies ......................................................................................................... 17
Version: [3.0] -3-List of Tables Table 1. Variables Existing in the Tufts Database* ........................................................................... 1 Table 2. Additional Variables Collected by our Study Team* ............................................................ 2 Table 3. Variables Re-Named ........................................................................................................ 1 Table 4. Variables Created ............................................................................................................. 1 Table 5. Logistic Regressions Part 1 (RCT data) .............................................................................. 5 Table 6. Logistic Regressions Part 2 (Special Statistical Techniques) .................................................. 7 List of Figures Figure 1. Potential Differences in Pharmaceutical vs. Medical and Surgical Procedure CEAs if Hypothesized Relationships Hold ................................................................................................... 3 Version: [3.0] -4-
List of Abbreviations Abbreviation Description CEA Cost effectiveness analysis NMA Network meta-analysis QALY Quality adjusted life-year RCT Randomized controlled trial Version: [3.0] -5-
Synopsis
Study Title: Expanding Cost-Effectiveness Analysis to All of Health Care:
Comparisons between CEAs on Pharmaceuticals and Medical/Surgical
Procedures
Objective(s): Our main goal is to find potentially important differences in the
comparison of characteristics of pharmaceutical CEAs with those of
medical/surgical procedures. In particular, we are interested in the
following areas: (1) Source of data used to measure effectiveness, (2)
Sponsorship of the CEA and underlying effectiveness studies, (3)
Statistical techniques used to measure effectiveness, (4) Property rights for
the intervention being studied, and (5) Other issues of interest –time
horizon,, whether costs outside of the health care sector and other CEA
elements are considered in the analysis, and degree of adherence to current
CEA guidelines.
Data Source: Tufts CEA Registry
Data Collection We will randomly select 100 pharmaceutical and 100 medical and surgical
Methods: procedure studies in the Tufts Medical Center CEA Registry conducted in
2015. For each CEA we will identify relevant study characteristics in the
Tufts database and randomly select up to 3 effectiveness studies (source
studies for effectiveness data, if different from CEA study selected from
Tufts database) for each CEA. For each effectiveness study, we will
extract relevant study characteristics pertaining to: (1) data source, (2)
study sponsorship, (3) statistical technique implemented, (4) property
rights of treatment, as well as other characteristics of interest.
Data Analyses: Simple comparisons and logistic regressions.
Strengths and Strengths: Though other studies of cost effectiveness analyses have been
Limitations: conducted, this is the first to our knowledge that examines characteristics
of underlying studies of treatment effectiveness.
Limitations: Data collection limited to studies in 2015 that were
considered high impact by the Tufts Medical Center Registry team.
Version: [3.0] -6-1. Introduction
For several decades, cost effectiveness analysis has been used by policymakers and payers to
determine the amount of value delivered by a good or service at a given price. Interest and
utilization of cost effectiveness analyses (CEAs) by health care policymakers and payers has
piqued in recent years as prices have risen and the industry has experienced a movement toward
value based pricing. Though pharmaceutical spending contributes less than 15% to total
healthcare spending, [1] pharmaceutical CEAs represent a disproportionately large fraction of
total CEAs (46%). [2] Only 27% of CEAs have examined medical/surgical procedures although
calculations performed on Medicare data indicate that spending on such procedures exceeds
spending on pharmaceuticals.
Our goal is to compare the attributes of existing CEAs of pharmaceuticals with those conducted
on medical/surgical procedures. The analysis may shed light on the underlying reasons for the
disproportionate differences in the types of healthcare intervention undergoing CEAs and
suggest policy changes that could move toward greater balance.
1.1. Objectives
Our main goal is to find potentially important differences in the comparison of characteristics of
pharmaceutical CEAs with those of medical/surgical procedures as well as the effectiveness
studies used in those CEAs. Our study is exploratory in nature. We have collected data in order
to allow us to measure the characteristics of CEAs and the effectiveness studies that underlie
them and to see whether the data are consistent with several general hypotheses. In particular,
we are interested in exploring the following:
1) Source of Data Used to Measure Effectiveness – in particular, does the source data
come from a randomized clinical trial (RCT) or observational data? One hypothesis is
that one reason more CEAs are done on pharmaceuticals is because RCT data are more
prevalent for drugs. We are also interested in seeing whether medical/surgical
procedures are less likely to use RCT data given less regulation / lack of FDA-approval
process for procedures.
2) Sponsorship of CEA and Effectiveness Studies – in particular, is one type of entity
more likely to sponsor a pharmaceutical versus medical/surgical procedure study? One
hypothesis is that industry is more likely to sponsor a pharmaceutical CEA.
3) Statistical Techniques used to Measure Effectiveness – in particular, did the
effectiveness analysis use more sophisticated techniques associated with inference from
observational data -- such as instrumental variables, regression discontinuity, or
differences-in-differences. Or, was propensity score matching used – to compensate for
differential characteristics in the study population? One hypothesis is that these
sophisticated techniques will be used more often with medical/surgical procedures
because they are often needed to infer effectiveness from observational data. The
hypothesis may fail to be supported by the data because: analyses of medical/surgical
Version: [3.0] -7-procedures have gravitated to situations where RCT data are available, and/or analysts or
editors are not comfortable with the sophisticated techniques.
4) Property Rights for the Intervention being Studied – were clear property rights for the
intervention likely to exist? One hypothesis is that property rights are less likely to exist
for procedures and that may at least partially explain the disproportionate number of
CEAs conducted on pharmaceuticals. This cannot be tested directly because we can only
observe situations in which a CEA was actually performed. The data may provide
suggestive evidence. It will also be interesting to see whether the existence of property
rights impacts use of an advanced statistical technique or affects the type of data used, as
there may be less incentive to obtain RCT data where property rights do not exist.
We are curious whether existence of property rights and the same entity sponsoring both
the CEA and its underlying effectiveness studies occurs more frequently for
pharmaceutical versus medical/surgical procedure studies.
We may also examine whether property rights impact QALY threshold cited as the
appropriate benchmark, or the breadth of costs or time horizon considered in the analysis.
5) Other issues of interest – We may also explore related issues of interest. One hypothesis
is that government sponsorship is needed (or relatively helpful) in getting an RCT done
when there are not property rights to the intervention(s) being analyzed.
We are interested in how expansive or comprehensive pharmaceutical CEAs are in
comparison to medical/surgical procedure CEAs; to evaluate this we will consider studies
more expansive to the extent they include time horizon, time burden, non-health costs or
report a cost/QALY threshold. We will also identify the proportion of pharmaceutical
versus medical / surgical CEAs that conduct a probabilistic sensitivity analysis.
Additionally, we are interested in examining characteristics of studies where the entity
has property rights and sponsors both the CEA and effectiveness study. Specifically, we
are interested to identify the proportion sponsored by industry versus government versus
another type of entity to determine if one sponsorship type is more likely.
,We will also examine whether study type or possibly sponsorship affects the degree to
which a CEA would already have been compliant with the new guidelines put forth by
the Second Panel on Cost-Effectiveness in Health and Medicine (Second Panel).[3]
2. Study Design
2.1. Overview of study design
The study will compute the proportion of CEAs and their source effectiveness studies that have
various characteristics and will compare CEAs of pharmaceuticals with those on
medical/surgical procedures. In this context, “effectiveness” is defined as the CEA parameter for
specific health gains or QALY gains, and “effectiveness sources” are the RCTs, analyses, or
referenced studies that provide information to populate effectiveness parameters. To collect the
necessary data on inputs to effectiveness in CEA studies, we will examine articles in the CEA
literature, identify sources of effectiveness parameters, and extract and collect information on the
Version: [3.0] -8-characteristics of those articles and studies that are the sources of the evidence on effectiveness. These data will then be combined with existing data in the Tufts Medical Center Registry for analysis. 2.2. Data source/data collection process Our goal is to collect the characteristics of 100 recent pharmaceutical CEAs and 100 recent medical/surgical procedure CEAs. We will focus on articles from 2015 in the Tufts Medical Center CEA Registry (www.cearegistry.org, a database with detailed information on cost- effectiveness and cost-utility analyses published in the peer-reviewed literature) because that is the most recent year for which additional descriptive variables are available. We will limit to the subset of articles for which Tufts researchers have collected certain descriptive variables that we plan to use in the analysis (see the first block of variables in Section 2.2.1 for the list of variables already in the Tufts database – for 2015 CEAs, Tufts collected those variables for what they deemed to be ‘high impact’ studies). We will randomly choose 100 of the 213 existing pharmaceutical CEAs from the 2015 high- impact category in the Tufts database. We will aim for an even balance of medical procedure versus surgical studies for the 100 that will represent the combined medical/surgical procedure category of CEAs. It is often the case that a CEA gets its measures of effectiveness from other studies. Sometimes the CEA obtains its effectiveness evidence from an intermediate article that may be a meta- analysis or other summary article, which in turn references studies that are the underlying sources of the effectiveness information. For each CEA we will record variable information for up to three studies that are the underlying source of the effectiveness evidence. If a CEA has more than 3 studies as its ultimate sources of effectiveness, we will randomly choose 3 studies for our data collection. Our data collection will be restricted to articles in English, or those that have an English translation available. In most cases, the notion of whether an intervention has a property right associated with it will be based on the judgment of the data collector. Generally, if a specific drug is mentioned, we will assume that it is associated with a property right. If a specific medical device or equipment is required to complete the medical/surgical procedure in the effectiveness study, we will assume that there are property rights on that device/equipment. In cases where specific manufacturers are mentioned, we will assume that property rights applied. (See part 1 of the Appendix for illustrative examples.) 2.2.1. Included Variables The following variables will be included in the study. Some variables already exist in the Tufts Registry (Table 1) and will not be modified, and additional variables will be collected by our study team (Table 2). Other variables will be re-named for clarity or consistency (Table 3), and several variables will be created by grouping existing variables or performing calculations (Table 4). Version: [3.0] -9-
Table 1. Variables Existing in the Tufts Database*
Variable Name Description
TimeHorizonStatedID Was time horizon stated clearly
The unit of time horizon (where 0=Lifetime, 1=Weeks, 2=Months,
TimeHorizonUnitsID
3=Years)
TimeHorizonMagnitude Time horizon (positive value)
Costs PatientTime Study included costs related to patient time
Costs CaregiverTime Study included costs related to caregiver time
CostperQALY Study reports cost per QALY estimate
ΔCosts/ΔQALYs re-calculated by reader, based on the costs and
$/QALY from Reader QALYs value reported in the article and adjusted for inflation,
currency, errors
Perspective for evaluation of costs and outcomes, where
ReaderPerspectiveID 1=Societal, 2=Health care payer, 3=Limited societal, 4=Health
care sector, 254=Not stated/could not determine, 250=Other
Sponsorship
Study sponsored / funded by governmental organization
PharmOrDevice
Sponsorship Government Study sponsored / funded by governmental organization
Sponshorship
Study sponsored / funded by healthcare organization
HealthCare
Study sponsored / funded by professional membership
Sponshorship
organization, membership organization of professional persons
ProfMembOrg
formed for the advancement of the interests of their profession
Sponsorship Other Study sponsored / funded by Other
*(binary where 0=False, 1=True, unless otherwise specified)
Version: [3.0] -1-Table 2. Additional Variables Collected by our Study Team*
Variable Name Description
Flag to indicate whether study is a pharmaceutical or
pharm_CEA _indicator
medical/surgical CEA
prob_SA The CEA conducted a sensitivity analysis
Effectiveness Study Data Source
Underlying effectiveness data or effectiveness parameters
data_rct
sourced in an RCT
Underlying effectiveness data or effectiveness parameters
data_qrct
sourced in an quasi-randomized controlled trial
Data or effectiveness parameters sourced in observational
data_obs_not_claims
data-but not claims data
data_registry Data or effectiveness parameters sourced from a registry
Data or effectiveness parameters sourced from
data_obs_claims_pri
observational claims data from a private company
Data or effectiveness parameters sourced from
data_obs_claims_gov observational claims data such as Medicare or another
government program
data_other_article Effectiveness evidence sourced from another article
data_source_not_specified Effectiveness data source was not specified
Sponsorship of Effectiveness Study
Sponsor of effectiveness study was a government institution
effect_sponsor_govt
or grant
Sponsor of effectiveness study was a pharmaceutical or
effect_sponsor_industry
device company
Sponsor of effectiveness study was an academic institution,
effect_sponsor_other
non-profit, etc.
effect_sponsor_no_info No information is available about study sponsorship
effect_sponsor_no_sponsor Study specifies no sponsor
Same entity sponsored both effectiveness and CEA studies
effect_sponsor_same_as_cea
(Yes (1) /No (0) ) [same exact entity; not same kind of entity]
effect_sponsor_same_as_cea_
Both the CEA and effectiveness study have no sponsor.
na
Statistical Technique Used in Effectiveness Study
Effectiveness determined using instrumental variables
stats_iv
technique
Version: [3.0] -2-Effectiveness determined using difference-in-differences
stats_diff_in_diff
technique
Effectiveness determined using regression discontinuity
stats_reg_discont
technique
Effectiveness determined using propensity score matching
stats_psm
technique
Effectiveness determined using multivariate regression
stats_multi_reg_treat
technique
Not an advanced technique (e.g. Student-t test, Wilcoxon
stats_simple_method
rank sum, etc.)
stats_simple_method_notes Denote which type of simple statistical technique
Treatment in Effectiveness Study has Property Rights
prop_rights Property rights exist for main intervention being analyzed
Denote which intervention may have a property right (e.g.
prop_rights_notes
"drug" "lab test" "equipment" etc.)
*binary where 0=False, 1=True, unless otherwise specified
Version: [3.0] -3-Table 3. Variables Re-Named
Tufts Registry Variable Name Re-named for analysis
TimeHorizonStatedID time_horizon
$/QALY from Reader cost_per_qaly
Costs Nonhealthcare non_health_costs
Sponsorship PharmOrDevice cea_sponsor_industry
Sponsorship Government cea_sponsor_gov
Table 4. Variables Created
Variable Name Description
Inclusion of costs related to patient or caregiver time (equals
time_burden
1 if Costs PatientTime=1 or Costs CaregiverTime=1)
Set equal to 1 if any of the following=1 (Sponsorship
cea_sponsor_other Foundation, Sponsorship HealthCare, Sponshorship
ProfMembOrg, Sponsorship Other)
Level of adherence to Second Panel recommendations
panel_adh_payer
(percent between 0 and 100%) for the payer perspective
Level of adherence to Second Panel recommendations
panel_adh_soc
(percent between 0 and 100%) for the societal perspective
Set equal to 1 if any of the following=1: stats_iv,
stats_special_any stats_diff_in_diff, stats_reg_discont, stats_psm,
stats_multi_reg_treat
Set equal to 1 if any of the following=1: stats_iv,
stats_special_a
stats_diff_in_diff, stats_reg_discont, or stats_psm
Set equal to 1 if any of the following=1: stats_iv,
stats_special_b
stats_diff_in_diff, or stats_reg_discont
pers_payer Set equal to 1 if ReaderPerspectiveID=2 (Health care payer)
Set equal to 1 if ReaderPerspectiveID =1 (Societal) [or 3
pers_societal
(Limited societal)]
Version: [3.0] -1-3. Statistical Analysis
3.1. Simple Comparisons of Pharmaceutical and Medical/Surgical CEAs
To explore our hypotheses, we will examine particular contrasts between the pharmaceutical and
the medical/surgical procedure CEAs. In particular, we plan to
1. Calculate the proportion of Pharmaceutical and Medical/Surgical CEAs that have any of
the following categorical (dichotomous) variables =1
a. Industry sponsor (cea_sponsor_industry)
b. Government sponsor CEA level (cea_sponsor_gov)
2. Calculate proportion of Pharmaceutical and Medical/Surgical CEAs underlying
effectiveness studies that have any of the following categorical (dichotomous) variables
=1
a. RCT as source (data_rct=1)
b. RCT or Quasi RCT as source (data_rct=1 or data_qrct=1)
c. Observational data as a source (data_obs)
d. Use of a special statistical technique (separately examine stats_special_any=1,
stats_special_b=1, or stats_multi_reg_treat=1)
e. Use of a special statistical technique (separately examine stats_special_any=1,
stats_special_a=1, stats_psm=1 or stats_multi_reg_treat=1) only among studies
with observational data as the source
f. Property rights (prop_rights=1)
g. Government sponsor (effect_sponsor_gov=1)
h. Probabilistic sensitivity analysis (prob_SA=1)
i. Prop_rights=1 and effect_sponsor_same_as_cea=1
i. When effect_sponsor_same_as_cea=1, we will also calculate the
proportion that have cea_sponsor_industry=1, cea_sponsor_gov =1 or
cea_sponsor_other=1.
We also plan to evaluate the CEAs’ expansiveness by calculating the proportion of
Pharmaceutical and Medical/Surgical CEAs that include
a. Time horizon (time_horizon)
b. Time burden (time_burden)
c. Non health costs (non_health_costs)
d. Cost/QALY threshold stated (cost_per_qaly=1)
All calculations of proportions of studies with categorical dichotomous variables equal to 0 or 1
will be calculated using the Proc freq SAS procedure with the Pearson chi-square test option. If
number of observations is not large enough, we’ll employ Fisher’s exact test.
Version: [3.0] -2-Mock Figure
A potential figure would look like the following:
Figure 1. Potential Differences in Pharmaceutical vs. Medical and Surgical Procedure
CEAs if Hypothesized Relationships Hold
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Effectiveness Data from RCT Government Sponsorship Property Rights to Intervention
Pharmaceutical Procedures
Next we will compare means for the continuous time horizon variable (time_horizon), ICER
value (cost_per_qaly), and adherence to Second Panel recommendations (panel_adh_payer for
all studies where pers_payer=1 For andIllustrative Purposes
panel_adh_soc for Only
all studies where pers_soc=1) for
pharmaceutical versus medical/surgical procedure CEAs. Prior to running statistical tests of
significance, we will test the normality of each variable. If the normality assumption is satisfied,
we will run two-sample-t-tests. If the normality assumption is violated, we will use the Mann
Whitney U test. We will also superimpose plots of the distributions of ICERs for both
pharmaceutical and medical/surgical CEAs.
3.2. Logistic Regressions
The following analyses are set up to test hypotheses about variables that could reasonably be
thought to affect whether or not a study has particular attributes, independent of whether the
CEA was looking at a pharmaceutical or non-pharmaceutical intervention. The idea is to first
run simple logits to see whether the hypothesized relationship exists in our pooled dataset that
includes all of the CEAs (both pharmaceutical and medical/surgical procedure). Then, we will
see whether the addition of other variables, among them the pharmaceutical indicator variable,
adds additional explanatory power.
Following are a set of proposed relationships to test with simple logistic regression along with
additional variables to subsequently add for a multivariate logistic regression. For all of the
following regressions, we will employ the Proc Logistic SAS procedure.
The first set of regressions (
Version: [3.0] -3-Table 5) will examine factors (property rights, sponsorship) that may impact a study’s likelihood of having RCT data as a source. Each observation will represent an individual effectiveness study that informed the sample of CEAs. We will incorporate an interaction term to assess whether the relationship between property rights and RCT data source differs based on sponsorship, with the goal of determining the impact of industry versus non-industry sponsorship on employing an RCT data source. We expect some combinations of interaction between sponsorship, property rights, and the pharm_CEA_indicator to be very highly correlated, which may preclude us from identifying the separate effects of those three characteristics on the likelihood of using RCT data. In the last step of the sequence of regressions we will add the pharm_CEA_indicator to determine whether adding the study type adds any additional explanatory power to the model. We will test the impact of pharm_CEA_indicator as an individual covariate and in interactions with property rights and sponsorship. We suspect that an industry-sponsored study where property rights exist for the intervention, may be more likely to use RCT data because the incentives exist to generate data that will be viewed as meeting a gold standard for testing effectiveness or to make use of that high standard of data. Version: [3.0] -4-
Table 5. Logistic Regressions Part 1 (RCT data)
Outcome variable Independent variables
Step 1:
RCT-data (data_rct)
prop_rights OR effect_sponsor_industry
(separately)
Step 2:
data_rct prop_rights AND
effect_sponsor_industry
Step 3:
prop_rights AND
data_rct
effect_sponsor_industry AND
effect_sponsor_industry*prop_rights
Step 4:
prop_rights AND
effect_sponsor_industry AND
effect_sponsor_industry*prop_rights AND
data_rct
pharm_CEA_indicator AND
effect_sponsor_industry*pharm_CEA AND
effect_sponsor_industry*prop_rights*
pharm_CEA_indicator
The second set of regressions (
Version: [3.0] -5-Table 6) will examine factors that may impact the likelihood of using a special statistical technique. First, we will examine the use of techniques that are designed for use with observational data – instrumental variables, difference-in-differences, regression discontinuity, and propensity score matching. For that set of analyses, we will limit to the subset of relevant effectiveness studies – namely, those that use observational data. Second, we will look at the use of multiple regression analysis, which is potentially useful with both RCT and observational data. We will include covariates related to property rights and sponsorship type (and their interactions) independent of whether the CEA was looking at a pharmaceutical or non-pharmaceutical intervention. Lastly we will include study type (pharm_CEA_indicator), both as an individual covariate and in interaction terms to evaluate whether some effects on the outcome are amplified. The goal will be to determine whether type of CEA adds additional explanatory power on the use of special statistical techniques beyond the effects of property rights, sponsorship, and their interactions. For all of the following regressions, we will employ the Proc Logistic SAS procedure and the unit of observation will be the subset of effectiveness studies that use observational data (data_obs=1). Version: [3.0] -6-
Table 6. Logistic Regressions Part 2 (Special Statistical Techniques)
Outcome variable Independent variables
Use of a special
Step 1:
statistical technique
prop_rights OR effect_sponsor_industry (separately)
(stats_special_a)
Step 2:
stats_special_a
prop_rights AND
effect_sponsor_industry
Step 3:
stats_special_a prop_rights AND
effect_sponsor_industry AND
effect_sponsor_industry*prop_rights
Step 4:
prop_rights AND
effect_sponsor_industry AND
stats_special_a effect_sponsor_industry*prop_rights AND
pharm_CEA_indicator AND
effect_sponsor_industry*pharma CEA AND
effect_sponsor_industry* prop_rights *
pharm_CEA_indicator
In step 3 of the previous two tables, we may substitute pharm_CEA for prop_rights to see if there
is much of a distinction in the data between the two concepts.
We plan to also repeat the above set of regressions with the outcome variable: Multiple
regression (multi_reg_treat), which unlike the other special statistical techniques, may be useful
in both RCT- and observational data-based studies. Here we will include covariates for data type
(data_rct) as both an individual factor and in interactions with other covariates. We will explore
whether industry or government sponsorship makes it more likely that multiple regression is
used and whether such effects are amplified by the existence of property rights and/or in
applications used for pharmaceutical CEAs.
Depending on the adequacy of the data, in addition to looking at the effect of industry versus
non-industry-sponsorship, we may adjust for government sponsorship specifically, as not all
non-industry sponsored studies are government sponsored. We are interested in whether
government sponsored studies are more likely to employ advanced statistical techniques than
other types of studies.
We may further explore the relationships above by evaluating the impact of CEA level
sponsorships with the above dependent and independent variables.
Version: [3.0] -7-What we may learn from these regressions is whether some underlying structural variable seems to drive the use of particular types of data or certain statistical techniques, separate from the identity of a CEA as being in the pharmaceutical arena or not. We may run further exploratory logistic regressions to examine other outcome variables of interest. 3.3. Evidence On The Supply of Clinical Trial Data A study by Califf et al. may help to interpret our findings. That study measured the number of clinical trials for pharmaceutical treatments, medical procedures and surgeries over a period spanning October 2007 to September 2010.[5] We plan to use their evidence as indicative of the relative supply of clinical trial data and, presumably, randomized clinical trial data as well, with the caveat that the authors do not identify a break down in specific years nor separate RCTs from non-randomized clinical trials, and our study is focused on 100 studies published in 2015. Their evidence may turn out to be useful in understanding some of our results. For example they found that six times as many clinical trials were done on drugs versus procedures. Thus, if we find that pharmaceutical effectiveness studies are more likely to use RCT data than are those that look at the effectiveness of medical or surgical procedures, we know that kind of data is in greater supply for pharmaceuticals. Lack of RCTs for procedures may mean that more medical/surgical CEAs are forced to rely on observational data studies. Of course, the relative supply of RCT data may reflect the demand for such data, which in turn may owe to the characteristics we will study such as property rights or sponsorship. The supply may also reflect the regulatory approval process for pharmaceutical versus procedures. 4. Study Limitations/Strengths In interpreting our results, we should keep in mind that our own analysis is based on observational data. The actual data that exists in an observational setting reveals the outcomes of a real world where various parties working under various constraints interact with each other. For example, one might expect that RCT data will more commonly be seen in pharmaceutical CEAs because RCTs are generally required as part of government-regulated approval processes, while the same is not true of medical/surgical procedures. But if it is sufficiently difficult to apply the special techniques needed to properly analyze observational data or if journal editors are extremely skeptical of the use of non-RCT data, then that could lead to an observational data set like ours finding no difference across pharmaceutical and medical/surgical CEAs in the use of RCT and non-RCT data. Under that latter scenario, almost all studies of either type would use RCT data, but the frequency of pharmaceutical CEAs would be disproportionately high to reflect the relatively easier access to the kind of data required to produce publishable analyses. Because of resource constraints, we limited our data collection efforts to a sample of CEAs from 2015 and if there were more than 3 studies that were the source of effectiveness evidence, we randomly chose 3 for our sample. Nonetheless, our choices were random and should be representative. Because of the limited availability of certain variables of interest from the Tufts Registry to studies that they rated as more impactful from our sample year (2015), our analysis is Version: [3.0] -8-
representative of studies that met the more impactful criteria as judged by Tufts. (See CEA
Registry Team (2016) for details.)[6]
To our knowledge, our study is the first to collect data on and to examine the characteristics of
the underlying effectiveness studies that provide source material for CEAs. Our focus on the
particular statistical techniques used to measure effectiveness is also original.
5. References
1. Centers for Medicare and Medicaid, Table 02 National Health Expenditures; Aggregate
and Per Capita Amounts, by Type of Expenditure. 2016: Baltimore, MD, USA.
2. Neumann, P.J., C.-H. Fang, and J.T. Cohen, 30 years of pharmaceutical cost-utility
analyses. Pharmacoeconomics, 2009. 27(10): p. 861-872.
3. Sanders, G.D., et al., Recommendations for conduct, methodological practices, and
reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and
medicine. Jama, 2016. 316(10): p. 1093-1103.
4. Amgen, Trends in Biosimilars Report: Navegating the Path to Biosimilars. 2017, Amgen
Inc.: Thousand Oaks, CA, USA.
5. Califf, R.M., et al., Characteristics of clinical trials registered in ClinicalTrials. gov,
2007-2010. Jama, 2012. 307(17): p. 1838-1847.
6. CEA Registry Team, A new era for the Cost-Effectiveness Analysis (CEA) Registry: The
Registry starts triaging published cost-utility studies based on impact. 2016.
Version: [3.0] -9-6. Appendix.
6.1. Examples of Assignment of the Property Rights Variable
Illustrative Examples:
A CEA comparing different procedures for identifying a certain type of tumor/cells
pointed to an effectiveness study that used a special dye/staining technique. Though
neither funder nor manufacturer was mentioned, we will assume that the dye likely has a
property right associated with it.
A medical/surgical CEA directed us to an effectiveness study that specifies
manufacturers of medical equipment used in the procedure, but does not report funding.
We will assume that property rights exist for the equipment used in the procedure, and
are owned by the stated manufacturer.
In an effectiveness study examining two different methods of breast reconstruction, we
will assume that no property rights exist as no specific companies were mentioned in
association with equipment and no special equipment was needed.
6.2. Background Bibliography
Our main analysis will consider the attributes of 200 CEAs from the Tufts Medical Center CEA
Registry and up to 3 additional articles that are the sources of the effectiveness evidence used in
the CEA.
This appendix contains a short bibliography of background information on characteristics of
CEAs. The two objectives of this targeted review were to (1) identify literature that describes
characteristics of CEAs, ideally with some mention of pharmaceutical and medical/surgical
procedures, and (2) to collect a sample of CEAs to inform which variables may be of interest in
the data collection phase of this study to supplement existing data in the Tufts Medical Center
Registry.
6.2.1. Methodology and Trends in CEA
Chandra, A., A.B. Jena, and J.S. Skinner, The pragmatist's guide to comparative effectiveness
research. The Journal of Economic Perspectives, 2011. 25(2): p. 27-46.
In this publication, Chandra et al describe the use, methods and challenges of
comparative effectiveness research. The paper cites problems such as treatment effect
heterogeneity, imprecisely measuring value by collapsing outcomes into a single index (page
7/19), heterogeneity in provider skill (economies of scale, learning by doing, spillovers to/from
other therapies). Approaches mentioned include observational studies and RCTs. Observational
studies use the standard “as treated” approach at the individual patient level with either
propensity-score matching or regression analysis with covariates; there is no randomization and
the researcher interprets the “treat/nontreat” coefficient as the treatment effect. Observational
Version: [3.0] - 10 -studies cannot control for confounding variables such as adherence, social and family support,
and health literacy. In some cases, IV are used (e.g. distance from treatment center) or
discontinuity designs (e.g. when birthweight cutoffs determine admission to an intensive care
unit).
Neumann, P.J., C.-H. Fang, and J.T. Cohen, 30 years of pharmaceutical cost-utility analyses.
Pharmacoeconomics, 2009. 27(10): p. 861-872.
This paper reviews and evaluates published cost-utility analyses from 1976 to 2006
pertaining to pharmaceuticals. Neumann et al. examined data from the Tufts Medical Center
Cost-Effectiveness Analysis Registry which contains detailed information on English language
CUAs and their ratios published in peer-reviewed journals. Using descriptive statistics, they
summarized key study features for these articles. Two sets of analyses were conducted: the first
analyzed characteristics of articles published from 1976 to 2006, and the second evaluated ratios
from articles published between 2000 and 2006. They found that 45.9% of CUA included in the
registry were pharmaceutical interventions (640 in total). Surgical and medical procedures
combined make up 26.6% of all studies (200 surgical, 170 medical). Other types of interventions
in the database included screening, care delivery, diagnostic, health education/behavior, medical
device, immunization and other. Over time, the number of CUA focused on pharmaceuticals
have also increase substantially from 34% for the 1990-1995 time period, to 47% for the 2001-
2005 time period.
Neumann, P.J., et al., The changing face of the cost-utility literature, 1990–2012. Value in
health, 2015. 18(2): p. 271-277.
In this study, Neumann et al examine the growth and changing characteristics of cost-
utility analyses over time. Data were collected from more than 3700 analyses published between
1990 and 2012 in the Tufts Medical Center Cost-Effectiveness Analysis Registry. They
summarized key study characteristics including intervention type, funding source, journal of
publication, and methodological practice (i.e. use of probabilistic sensitivity analysis). CUAs
were also examined by country, region and the degree to which diseases studied correlated with
disease burden. Results indicate that the number of CUAs has risen from 34 per year in the 1990-
1994 time period, to 431 per year from 2010 to 2012. Furthermore, the proportion of studies
focused on the United States declined from 61% during the 1990s to 35% from 2010 to 2012.
Lastly, close to half (47%) of the most recent studies from 2010 to 2012 were focused on the
pharmaceutical industry.
Sanders, G.D., et al., Recommendations for conduct, methodological practices, and reporting of
cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA,
2016. 316(10): p. 1093-1103
The purpose of this publication is to update recommendations established by the First
Panel on Cost-Effectiveness in Health and Medicine intended to guide the design conduct and
use of CEA by researchers, government policy makers, public health officials, health care
Version: [3.0] - 11 -administrators, payers, businesses, clinicians, patients and consumers. The Second Panel
describes the use of a “reference case” and a set of standard methodological practices that all
cost-effectiveness analyses should follow to improve quality and comparability. Specifically,
they recommend reporting two reference cases, one from the healthcare perspective, and one
from the societal perspective. Analyses from the healthcare perspective should capture health
outcomes such as longevity effects, health-related quality of life effects, and other health effects
(e.g. adverse events and secondary transmission of infections) as well as medical costs such as
those paid for by third-party payers, those paid for by patients out-of-pocket, and future related
and unrelated medical costs paid for by both payers and patients. According to the Second Panel,
the societal perspective should encompass patient time costs, unpaid caregiver-time costs,
transportation costs, as well as costs to non-healthcare sectors (e.g., production of toxic waste
pollution by intervention, cost of intervention on home improvements, etc.)
Sox, H.C. and S.N. Goodman, The methods of comparative effectiveness research. Annual
review of public health, 2012. 33: p. 425-445.
This source provides a review of methods used in comparative effectiveness research.
They mention that baseline characteristics (e.g. being older and sicker) may confound the
relationship between treatment and outcome because it affects the choice of treatment and it
affects the treatment outcome. Data quality may also be a problem with observational studies. To
cope with confounding variables, the paper suggests using propensity score analysis, sensitivity
analyses, or instrumental variables, however each method has limitations. Although randomized
confounding is usually of minor concern in randomized trials, there are additional considerations
to using this kind of data that often require larger trials. Treatment response heterogeneity,
variation in the direction and/or magnitude of response to the same treatment, is highlighted as
an additional consideration when examining studies. The paper also outlines decision analysis
and highlights ways to model long term outcomes (e.g., declining exponential approximation to
life expectancy (DEALE), Markov models) and methodology to conduct cost effectiveness
analyses.
6.2.2. Medical/Surgical Studies
Cummins, J. S., et al. (2009). "Cost-effectiveness of antibiotic-impregnated bone cement used in
primary total hip arthroplasty." J Bone Joint Surg Am 91(3): 634-641.
The purpose of this study was to model the use of antibiotic-impregnated bone cement in
primary total hip arthroplasty for the treatment of osteoarthritis to determine whether use of the
cement is cost-effective when compared with the use of cement without antibiotics. To evaluate
the cost-effectiveness of each strategy, Cummins et al used a Markov decision model to tabulate
costs and quality-adjusted life years (QALYs) accumulated by each patient. Rates of revision due
to infection and aseptic loosening were estimated from data in the Norwegian Arthroplasty
Register and were used to determine the probability of undergoing a revision arthroplasty
because of either infection or aseptic loosening. Data from randomized controlled trials were not
available in this study. As a result, estimates of revision rates used in this study were obtained
from registers. The primary outcome measure was either all revisions or revision due to
infection. Perioperative mortality rates, utilities, and disutilities were estimated from data in the
Version: [3.0] - 12 -arthroplasty literature. Costs for primary arthroplasty were estimated from data on in-hospital
resource use in the literature. The additional cost of using antibiotic-impregnated bone cement
($600) was then added to the average cost of the initial procedure ($21,654). The study found
that off-label use of antibiotic-impregnated bone cement for primary total hip arthroplasty with
cement appears to be a cost-effective strategy if the patient population is young and the cost of
the cement is relatively low.
Del Rizzo, D.F., et al., Safety and cost-effectiveness of MIDCABG in high-risk CABG patients.
The Annals of thoracic surgery, 1998. 66(3): p. 1002-1007.
Myocardial revascularization without cardiopulmonary bypass has been proposed as a
potential therapeutic alternative in high risk patients undergoing coronary artery bypass grafting.
To evaluate this possibility, the study compared high risk patients in whom minimally invasive
direct coronary artery bypass grafting was used as the method of revascularization with patients
who underwent conventional coronary artery bypass grafting during 1 month. They found that
the average and expected intensive care unit stays were lower in high risk patients versus their
low risk counterparts. They also found that the expected costs for the high risk cohort would
have been $11,997 whereas the average cost in the low-risk cohort was $5,977 per patient.
Divisi, D., et al. (2013). "Surgical management of traumatic isolated sternal fracture and
manubriosternal dislocation." Journal of Trauma and Acute Care Surgery 75(5): 824-829.
The aim of this study was to evaluate the surgical procedures of osteosynthesis and the
respective costs in isolated sternal fractures and manubriosternal dislocations. From January
2006 to July 2011, 47 patients with an isolated fracture and 18 patients with a dislocation of the
sternum were treated with surgery. QALYs were obtained using the EQ-5D questionnaire.
Average QALYs were used in the calculation of the ICER. Data were analyzed by two-way
analysis of variance tests of the time of intervention, the duration of drainage, and the length of
hospitalization for each surgical method in isolated sternal fractures and manubriosternal
dislocations. In the sternal fractures, titanium plate showed a decreased time of intervention,
length of stay, and total cost, compared with steel plate and steel wire procedures. This had a
large impact on QALYs. Steel wire technique highlighted a poor QALY, although this approach
was less expensive than the titanium plate with demineralized bone matrix. Incremental cost-
effectiveness ratios revealed that the titanium plate costs were €623.55 more than the steel wire
per QALYs gained. From this study, the authors concluded that the rigid titanium plate
application ensured a safe and easy management of traumatic sternal lesions with a good
prognosis of patients, justified by the improved QALYs compared with other methods.
Espigares, J. L. N. and E. H. Torres (2008). "Cost-outcome analysis of joint replacement:
evidence from a Spanish public hospital." Gaceta Sanitaria 22(4): 337-343.
This study performed a prospective cohort pre-test/post-test study of patients undergoing
primary hip or knee arthroplasty. Sociodemographic, clinical and healthcare characteristics were
obtained from hospital records. Health related quality of life was assessed using the SF-36,
Version: [3.0] - 13 -QALYs gained were estimated using the EQ-5D instrument. A standard statistical analysis was
carried out, by means of both univariate (descriptive analysis of variables) and bivariate (analysis
of statistical significant differences between pre-test and post-test scores in the evaluated
characteristics) analysis. A total of 80 patients, 40 from each intervention were included in the
study. Both functional and perceived health status improved after the intervention. The number
of QALYs gained in the knee cohort was 4.64, while that in the hip cohort was 0.86. Total cost
of knee replacement (€6,865.52) was lower than that of hip replacement (€7,891.21). The cost
per QALY gained was €1,275.84 and €7,936.12 for knee and hip interventions, respectively. The
authors concluded that the costs of both knee and hip replacement were lower than the threshold
of €30,000 considered acceptable in Spain.
Fehlings, M. G., et al. (2012). "Is surgery for cervical spondylotic myelopathy cost-effective? A
cost-utility analysis based on data from the AOSpine North America prospective CSM study."
Journal of Neurosurgery: Spine 17(Suppl1): 89-93.
This study evaluates the cost-effectiveness of cervical spondylotic myelopathy (CSM).
As part of a larger prospective multicenter study, the direct costs of medical treatment for 70
patients undergoing surgery for CSM at a single institution in Canada were retrospectively
obtained from the hospital expenses database and physician reimbursement data. Utilities were
estimated on the entire sample of 278 subjects enrolled in the multicenter study using SF-6D–
derived utilities from 12- and 24-month SF-36v2 follow-up information. Costs were analyzed
from the payer perspective. A 10-year horizon with 3% discounting was applied to health-
utilities estimates. Sensitivity analysis was performed by varying utility gain by 20%. Dividing
the mean cost of treatment by the mean number of QALYs gained provides an estimate of cost
utility measured in cost per QALY. The SF-6D utility gain was 0.0734 at 12 months and
remained unchanged at 24 months. The 10-year discounted QALY gain was 0.64. Direct costs of
medical treatment were estimated at an average of $21,066 (Canadian dollars). The estimated
cost-utility ratio was $32,916 (Canadian dollars) per QALY gained. The sensitivity analysis
showed a range of $27,326–$40,988 (Canadian dollars) per QALY gained. Thus, the authors
concluded that the direct cost of medical treatment per QALY gained places this form of
treatment within the category deemed by payers to be cost-effective.
Hadley, J., et al., An exploratory instrumental variable analysis of the outcomes of localized
breast cancer treatments in a Medicare population. Health economics, 2003. 12(3): p. 171-186.
This study examines the potential problem of using observational data to draw inferences
about treatment outcomes when experimental data are not available. Two approaches were
compared, ordinary least squares (OLS) and instrumental variables (IV) regression analysis, to
estimate the outcomes (three-year post treatment survival) of three treatments for early stage
breast cancer in elderly women. Treatment options included mastectomy, breast conserving
surgery with radiation therapy, and breast conserving surgery only. Contrary to randomized
clinical trial results, analysis with observational data found highly significant differences in
survival among the three treatment options. Using OLS to control for observable characteristics
narrowed the estimated survival rate differences but they were still statistically significant.
Version: [3.0] - 14 -However, when IV analysis was used, treatment effects were larger than OLS, unstable and not
statistically different from OLS estimates. The study concludes that unadjusted observational
data on health outcomes of alternative treatments for localized breast cancer should not be used
for cost-effectiveness studies. Whether one places greater confidence in the OLS or the IV
results depends on at least three factors: (1) the extent of observable health information that can
be used as controls in OLS estimation, (2) the outcomes of statistical tests of the validity of the
instrumental variable method and (3) the similarity of the OLS and IV estimates.
Kaplan, R. I., et al. (2015). "Comparative cost-effectiveness of the Baerveldt implant,
trabeculectomy with mitomycin, and medical treatment." JAMA ophthalmology 133(5): 560-
567.
This study examines the cost effectiveness of the baerveldt implant, tabeculectomy with
Mitomycin and medical treatment. A Markov model was created that incorporated clinical trial
data on rates of surgical success, risks of short-and long-term surgical complications, need for
supplemental medication, rates of visual progression, associated medical and surgical costs, and
quality of life based on visual preservation and adverse effects of surgical intervention. Annual
field progression and other estimates needed for long term modeling, but not reported in the
clinical trial were obtained from the literature. Costs were identified from Medicare Current
Procedural Terminology and Ambulatory Payment Classification reimbursement codes and Red
Book medication costs. The QALYs were based on visual field and visual acuity outcomes. The
hypothetical societal limit to resources was included using a willingness-to-pay threshold of $50,
000 per QALY. Costs and utilities were discounted at 3% per year. Uncertainty was assessed
using deterministic sensitivity analyses. In this study, the incremental cost-effectiveness ratio
was $8289 per QALY for trabeculectomy vs medical treatment, $13 896 per QALY for tube
insertion vs medical treatment, and $29 055 per QALY for tube insertion vs trabeculectomy. The
cost-effectiveness of each surgical procedure was most sensitive to early and late surgical failure
rates and was minimally affected by adverse events, rates of visual field progression, or
medication costs. Assuming a willingness to pay of $50,000 per QALY, trabeculectomy and tube
insertion are cost-effective compared with medical treatment alone. Trabeculectomy, however, is
cost-effective at a substantially lower cost per QALY compared with tube insertion.
McClellan, M., B.J. McNeil, and J.P. Newhouse, Does more intensive treatment of acute
myocardial infarction in the elderly reduce mortality?: analysis using instrumental variables.
Jama, 1994. 272(11): p. 859-866.
This study serves as an example in numerous publications, as it is the first to apply
analysis using instrumental variables in a healthcare setting. The goal was to determine the effect
of more intensive treatments on mortality in elderly patients with acute myocardial infarction
(AMI). The analysis of incremental treatment effects was performed using differential distances
as instrumental variables to account for unobserved case-mix variation (selection bias) in
observational Medicare claims data (1987 through 1991). The main outcome was survival to 4
years after AMI. Results indicate that patients who receive different treatments differ in
observable and unobservable health characteristics, biasing estimates of treatment effects based
Version: [3.0] - 15 -You can also read
