Do Cost-Effectiveness Models Need a Reality Check?

In a thoughtful commentary published in the British Medical Journal, clinical researchers from Europe question the claims of cost-effectiveness made for many commonly used pharmacological treatments.    The authors argue that “…although there are claims that important preventive drugs such as statins, antihypertensives, and bisphosphonates are cost effective,^{6 7 8 9} there are no valid data on the effectiveness, and particularly the cost effectiveness, in usual clinical care. Despite this dearth of data, the majority of clinical guidelines and recommendations for preventive drugs rest on these claims.”

The authors cite a 2009 study, which examined a cost-effectiveness model of selective cyclo-oxygenase-2 (COX 2) inhibitors, as evidence of the weak external validity of claims of cost-effectiveness.  The COX-2 evaluation, which was based on a clinical trial, found that the cost of avoiding one adverse gastrointestinal event by switching patients from conventional non-steroidal anti-inflammatory drugs to COX-2 inhibitors was approximately $20,000.  In contrast, when the same analysis was conducted using the UK’s General Practice Research Database, which includes patients’ medical records in routine care, the cost of preventing one bleed was over $100,000.²

These findings are similar to work myself and others conducted several years ago on the COX-2s.  While the original cost-effectiveness model for the U.S. reported a cost per year of life saved (YLS) of about $19,000 for COX-2s when compared to non-selective NSAIDs, our revised model, which was based on actual practice, found a cost per YLS of $107,000. 

In a different study, we examined the external validity of a cost-effectiveness model of treatment options for eradication of h. pylori.  The original decision-analytic model found that the lowest cost per effectively treated patient was for the dual combination of PPI and clarithromycin ($980), whereas we found that the lowest cost per treatment was for the triple combination of bismuth, metronidazole, and tetracycline at a cost of $852.  Why the disconnect?  In the original h. pylori model, the authors had made assumptions about medication compliance and the cost of recurrence that simply did not hold up in the real-world. 

In the case of the COX-2s, the recent commentary concluded that the published cost-effectiveness analyses of COX 2 inhibitors neither had external validity nor represented the patients treated in clinical practice. They emphasized that external validity should be an explicit requirement for cost effectiveness analyses that are used to guide treatment policies and practices.  At least one academic modeler vehemently disagrees with the requirement of external validity, arguing that “it is wrong to insist that models be ‘validated’ by events that have not yet occurred; after all, the modeler cannot anticipate advances in technology, or changes in human behavior or biology. All that can be expected is that the model reflects the current state of knowledge in a reasonable way, and that it is free of logical errors.”

It is true that right when a drug comes to market, the only available data will likely be from the original clinical trials used to seek FDA approval, and the modeler will be forced to make numerous assumptions about compliance, costs, concomitant medication use, etc.  The problem is that the extent to which these assumptions are made without bias is unclear.  Research has shown that sponsorship by the pharmaceutical industry affects the results of economic models.  In a review published in 2010, researchers found that 95 percent of studies sponsored by pharmaceutical manufacturers reported favorable conclusions compared to only 50 percent of nonsponsored studies.  While it could be argued that this reflects a publication bias, the validation studies that I have described above suggest otherwise.  In each of these cases, there were key assumptions that drove model outcomes which, from a plan sponsor perspective, we found highly questionable at the time the model was first published.

Surprisingly, the issue of model validity receives relatively little attention given the central role that these models play in the field of pharmacoeconomics, as for example, in the AMCP dossier process.  The commentary authors argue that real-world comparative studies are the key to producing cost-effectiveness models that possess external validity.  This certainly will help with the quality of models post-FDA approval.  However, for models used at the time a drug is launched, ultimately, I expect that plan sponsors will have to develop their own models to ensure systematic bias is removed.

Does Makena Actually Have a Favorable Pharmacoeconomic Profile?

While there has been no shortage of press about the recent FDA approval of Makena^TM, made by K-V Pharmaceutical Company, I think the recent recently published letter in the New England Journal of Medicine about the pharmacoeconomics of the drug is quite useful.

Separate from the business and ethical questions of the $30,000 plus price tag for Makena (a drug that previously cost around $300 as a generic), one could argue that the pharmacoeconomics of the drug at the new price may still render the medication a cost-effective therapy.  A physician at Aetna examined just this question, evaluating the pharmacoeconomics of Makena based on the likely rate of treatment, the efficacy of the drug versus placebo, and the subsequent costs avoided due to treatment.

Based on published literature, Aetna estimated that about 139,000 women are candidates for Makena, of which about 22% (30,500) are likely to have a recurrent preterm birth absent medication.  With treatment, 33% (10,000) preterm births could be prevented, saving $334 million in direct medical costs and $519 million in indirect medical costs. The indirect medical costs include maternal care, special education, early intervention costs, etc.  The cost of treating the 139,900 women, at $29,000 per course of treatment (price before distribution mark-ups) would be $4.0 billion.  Accordingly, including both direct and indirect medical costs, Makena will cost $8 for every $1 saved, a strongly negative ROI.

However, one could argue that demonstration of net cost-savings should not be the criteria for coverage but rather payers should examine the drug’s cost-effectiveness, measured as cost per life-year saved (LYS).  To meet the threshold of $100,000 per LYS (inflation adjusted for the more commonly used $50,000 per quality-adjusted life year), the drug would need to prevent nearly 35,000 life years or 435 deaths, assuming an average lifespan of 80 years.  Previous studies suggest this is unlikely as the drug has shown a small (and not statistically significant) effect on reducing mortality.

Another coverage option is to identify opportunities to further target use of the drug to patients at the greatest risk for preterm birth.  However, even with an ability to target only patients with a 100% risk of preterm birth (highly implausible), the drug would still cost nearly double the amount it saves in direct and indirect medical costs given its effectiveness. Of course, these figures represent back-of-the-envelope pharmacoeconomic calculations for Makena; but absent very significant unmeasured benefits, the analysis highlights the difficulty this drug will have in demonstrating a favorable pharmacoeconomic profile.

Can You Reduce Risk Through Outcomes-Based Contracting for Pharmaceuticals?

I recently attended the Pharmacy Benefit Management Institute’s (PBMI) Annual Drug Benefit Conference where one of the keynote speakers, Ed Schoonveld of ZS Associates, shared his point of view on the merits of outcomes-based contracting.  As you can read in his 2010 article on the subject, Ed advocates to pharmaceutical manufacturers for judicious use of these contracts given their potential complexity and potential risk for being nothing more than a thinly veiled price reduction. 

While I found his article and presentation insightfuI, I was struck by Ed’s comment questioning the wisdom of pharmaceutical companies providing insurance to insurance companies via outcomes-based contracts.   From a literal perspective, a large percentage of employers purchase pharmacy benefits on a self-insured basis through their health plan or PBM so the organization bearing the risk for the pharmaceuticals is often the employer and not an insurance company.  Second, while the price paid to the manufacturer is certain, the uncertainty and accordingly, risks to the employer and health plan are many, including:    

• Uncertainty about effectiveness under real-world conditions of use that often lacks close monitoring and quick dose adjustment;
• Uncertainty about the nature and extent of side effects in the broader population and over the long-term;
• Uncertainty about the extent of use for off-label conditions lacking scientific support;
• Uncertainty about the extent to which the new product will replace older, equally efficacious alternatives
• Unknown medication compliance rates under real-world conditions; and
• Ultimately, unknown cost-effectiveness in the real-world.

There is no shortage of examples of the impact that this uncertainty can have on clinical and economic outcomes for patients and payers; and while employers can mitigate some of these risks through benefit design choices, most are beyond their control.   Certainly outcomes-based contracts can be messy.  However,  the much higher unit cost of specialty medications and even greater uncertainty on these various dimensions relative to traditional medications makes outcomes-based contracting a logical alternative for sophisticated plan sponsors who want to mitigate uncertainty.

Are Statins Cost-Effective for Primary Prevention?

There has been no shortage of headlines about the cost-effectiveness of statins in recent weeks, and another study was reported today in Journal of the American College of Cardiology. In a pharmacoeconomic model developed by Choudry and other researchers at Brigham and Women’s Hospital/Harvard Medical School using the JUPITER (Justification for the Use of statins in Prevention: An Intervention Trial Evaluating Rosuvastatin) trial, the authors project that the average JUPITER patient treated with rosuvastastin will have $7,900 higher life costs and an additional 0.31 quality-adjusted life years (QALYs), providing a cost-effectiveness ratio of $25,000/QALY. Against the widely used benchmark of $50,000/QALY, the authors conclude that statins appear to be cost-effective for primary prevention.

These results are somewhat contradictory to the recent Cochrane Collaboration review of statins in primary prevention, which recommended that statins be prescribed with caution to those at low risk of cardiovascular disease. The researchers reviewed data from 14 trials and nearly 35,000 patients. Although clinical benefits were found, the authors concluded that “ there was evidence of selective reporting of outcomes, failure to report adverse events and inclusion of people with cardiovascular disease. Only limited evidence showed that primary prevention with statins may be cost effective and improve patient quality of life.” The authors pointed out that all but one of the trials they reviewed were industry-sponsored and that you cannot simply extrapolate results from studies of people with history of heart disease to those without.

So why the differing conclusions about statin cost-effectiveness? In a commentary accompanying the latest study, Mark Hlatky, a physician and researcher at Stanford, provides some insights on the study. Hlatky points out that the model was only based on JUPITER and not the full breadth of evidence, which has NOT found large reductions in risk from the use of statins in primary prevention. Furthermore, the longer term risk reduction is simply unknown because clinical trials rarely last longer than 5 years. Choudhry assumed that rosuvastatin would reduce the risk of cardiac events by more than 50% for 15 years. If the same effect does not extend beyond 5 years, the cost-effectiveness grows to $62,100/QALY. The assumption of proportional risk reductions across levels of severity has been a limitation of many of the more recent cost-effectiveness analyses of statins.

There are additional questions about this and other recent studies of statins for primary prevention. Choudhry did not have data on long-term adverse effects, to which their model was quite sensitive. As Hlatky pointed out, if patients taking rosuvastatin had a 2% decrease in their well-being, the cost-effectiveness ratio grew to more than $62,000 per QALY. Also unclear is whether the model adjusted for medication persistency rates over the longer-term. Studies have shown that even after the first year of therapy, 50% of patients discontinue their statin medication, leading to increased short-term costs with little or no clinical benefit.

In hearing this latest information, many providers and plan sponsors are likely to point to diet and exercise changes as the solution to reducing the long-term risk of cardiovascular disease. Unfortunately, a recent Cochrane Collaboration review also found that education and counseling to encourage people to change their diets and stop smoking had little or no impact on deaths or disease caused by cardiovascular disease. An accompanying editorial pointed out that “Although various multiple prevention strategies exist, the most effective and cost-effective intervention for primary prevention in adults at low risk currently remains unclear.”

Do you know what percent of your current statin use is for primary prevention? Depending on the population, it could be as high as 50% of your statin users and may be growing.  If you are funding a wellness program to reduce the risk of cardiovascular disease, do you have any rigorous evidence that the program is leading to sustained changes in behavior?

Do Drugs Save Money? A Plausibility Test

In a previous post, I commented on a study that examined the correlation between medication compliance and use of other healthcare services over three years for a large population of commercially insured members with a diabetes or cardiovascular-related diagnosis.  The study found a strong association between greater medication adherence (defined as a medication possession ratio of 0.80 or higher) and lower utilization of other medical services, primarily hospitalizations.  Considering all healthcare costs, not just disease-related, they reported an ROI of 3:1 for medication adherence in dyslipidemia and 10:1 for hypertension.

While my previous post highlighted a key methodological concern about the study, one of the most powerful tools for quickly stress-testing a study’s findings is to conduct a plausibility test to see if the results match up with what other rigorous research would suggest.   For this plausibility comparison, I selected a meta-analysis of data from 90,056 individuals in 14 randomized trials of statins.  There are other meta-analyses and randomized controlled trials that I could have chosen, which would have led to similar conclusions.

The adherence study included all patients with dyslipidemia as evidenced by a diagnosis in the medical claims.  To be conservative, I selected patients from the meta-analysis who were taking statins for secondary prevention and looked at 5-year effectiveness, as shown in the table below.  Given the absolute risk reductions observed for hospitalizations for MI, revascularization, and strokes, the estimated number of hospital days avoided across all the patients was 0.33.   In contrast, the adherence study reported an average of 1.18 fewer hospital days for adherent patients versus non-adherent patients with dyslipidemia.

It is not plausible that the nearly 4-fold greater hospital reduction reported in the adherence study (1.18 versus 0.33 days) was due to greater medication compliance.  The implausibility is compounded by the fact that I included more severe patients, followed them for a much longer period of time, and examined the full effect of statins versus placebo rather than the effect of differences in adherence, all of which inflated our hospital days avoided.

Patients with Previous MI or CAD
Event	Statin	No Statin	Absolute Reduction	Length of Stay	Hospital days avoided
MI	12.00%	15.00%	3.00%	5	0.15
Revascularization	11.00%	13.70%	2.70%	5	0.14
Stroke	3.20%	4.00%	0.80%	5	0.04
				Total hospital days avoided	0.33

Plausibility tests are quick and powerful and can be used to test the ROI claims from disease management vendors, medication compliance programs, and many other healthcare services.  Recognizing the need for such tools and plan sponsors’ limited time to examine vendors’ savings claims, we designed plausibility calculators specific to disease management and value-based insurance to help plan sponsors discern fact from fiction.  They are free so the next time you are listening to a vendor’s sales pitch, you can do a quick reality check.

Disease Management: Why Doesn’t It Save Money?

The honeymoon for disease management (DM) has clearly passed.  Health plans and employers are frustrated with the lack of value provided by their DM vendors but they often struggle to understand why their DM program isn’t working as intended or what to do to correct it.  The combination of frustration and confusion has resulted in a wave of market experimentation fueled by vendors who claim to have discovered the allusive secret ingredient that makes DM work.

Published today in the American Journal of Managed Care is a paper I wrote to help plan sponsors better understand what is really known about savings from DM, why telephone-based DM does not save money and what it takes to generate real savings.  This work is based on my market experiences, an extensive review of the literature, both in DM and other related disciplines, and a detailed assessment of what is known about cost-saving healthcare services and treatments.

The answer to why DM does not provide short-term savings lies partly in the myriad of cost-effectiveness assessments that have been conducted on the treatment for chronic disease over the last 30 years. Cohen and Neumann reported that less than 20 percent of preventive measures or treatments for chronic conditions are cost-saving, even for a 30-year time horizon.  Kahn found that aggressive implementation of nationally recommended medical activities would increase costs over a 30-year period for all activities except smoking cessation.  Looking more specifically at the individual components of DM programs, the evidence is equally compelling. At the level of the individual program activity, cost-savings has not been shown for treatment of these chronic conditions, the exception being heart failure.  Accordingly, one should not necessarily expect cost savings for the program as a whole, absent a belief that DM can independently improve the outcomes of patients with chronic disease without affecting the key clinical goals for each disease.

Cost-Effectiveness of Common Disease Management Activities/Goals

Disease/Condition	Treatment/Service	Cost-Savings
Diabetes	A1C < 7%	No
	LDL Cholesterol < 100 mg/dl	No
	Blood pressure < 130/80 mmHG	No
	Feet examination	No
	Retinal screening	No
	Microalbuminuria screening	No
CAD	Antihyperlipidemics (LDL < 100 mg/dl)	No
	Beta blocker	No
	Lifestyle changes	No/Unknown
Asthma	Inhaled anti-inflammatory use	No
	Asthma education on symptom monitoring and/or trigger avoidance	Unknown
HF	ACE inhibitor use	Yes
	Beta blocker use	Yes
	Structured remote monitoring (weight, blood pressure, etc.)	Yes
	Daily exercise	No/Unknown

While the cost-effectiveness literature does not bode well for the future of telephone-based DM as currently designed, cost-effectiveness research suggests that better targeting of treatment activities and patients may provide opportunity for cost-savings for some disease states.  Although a targeted approach is both intuitive and supported by the evidence, it has two practical problems that will prevent it from being widely adopted in the marketplace.

• First, a more targeted approach is not a revenue-optimizing model for DM vendors.  For example, only about 5% of asthmatics would be targeted for intervention if the criteria were real cost-savings, hardly a desirable approach for DM vendors whose revenue model is based on volume due to their large fixed cost structure.
• A second barrier to adoption is the marketability of the more realistic return-on-investment (ROI).  An employer is unlikely to select a vendor that is offering a 1.85 ROI over a vendor with an inflated ROI (that also includes a much larger group of patients since it is not targeted).  The same problem holds true for health plans—even though they often understand the methodological problems of the vendors’ ROIs, ultimately they too must market their program to the employers who sometimes believe all ROIs are created equal.

To learn more about proven strategies for short-term savings, take a look at the full published article.

---

Do Drugs Save Money? Check The Assumptions

Econometrics has finally discovered the hidden truth about pharmaceuticals—they save money, a lot of money.  A recent study found that compliance with pharmaceuticals demonstrated a 8.4:1 ROI for congestive heart failure, 10.1:1 for hypertension, 6.7:1 for diabetes, and 3.1:1 for dyslipidemia.  This is certainly a surprising result given the contradictory findings from other credible research on the cost-effectiveness of pharmaceuticals.  However, it’s likely to capture some headlines given its intuitive appeal to many so I took a closer look.

This study, published by researchers at one of the large PBMs, examined the correlation between medication compliance and use of other healthcare services over several years for a large population of commercially insured members with a diabetes or cardiovascular-related diagnosis.  It is well-known in the published literature that making causal conclusions from the cross-sectional examination of medication compliance against total medical spend is highly problematic due to the Healthy Adherer Effect, which is the tendency of people who are adherent to their medications to also engage in other healthy behaviors, such as exercising regularly and eating a healthy diet.  Reason being, it is difficult, if not impossible, to control for differences in patient behavior that one cannot measure in the claims data. In this particular study, the Healthy Adherer Effect was likely compounded by assigning a compliance score of zero to patients having a diagnosis but no prescription claim for the condition.

The study authors stated that they overcame the systematic selection bias created by the Healthy Adherer Effect through the use of an econometrics technique called fixed-effect regression—basically an alternative form of the more commonly used ordinary least squares regression (OLS).  As is often the case with econometric models, to believe in the superiority of fixed-effect modeling, you have to believe a key underlying assumption that the model makes, which is that confounders, such as eating healthy and exercise, do not vary over time in conjunction with medication compliance.  As the authors acknowledge in the references, “fixed-effects modeling does not allow for the control of confounders that vary over time. Thus, for example, if patients who become adherent simultaneously start exercising regularly (assuming that both of these behavioral changes reduce health services use and spending), the estimated impact of adherence would remain biased.”  The authors make no case for why this assumption would hold true and it hardly seems plausible given what is already known on the subject.

While such fixed-effects estimators may be an improvement on basic cross-sectional methods, they are still quite limited when it comes to uncovering a true causal effect when  the confounder(s) varies over time; and like OLS, will tend to overestimate the causal effect of pharmaceuticals on medical spending in the presence of the Healthy Adherer Effect.  Some evidence to that effect lies in the manuscript’s appendix—for most of the conditions, the difference between the estimate for the OLS regression, which the authors acknowledge does not address the selection bias problem, and the fixed-effect model was minimal (e.g., marginal effects estimate for inpatient days for heart failure of 5.731 for OLS vs. 5.715 for fixed-effects).

However, setting aside the more technical discussion, one of the most informative and practical ways to test the strength of these results is to conduct a plausibility test of the ROI, which basically means to compare these findings against credible randomized controlled trials of medication efficacy.  I’ll show you those results in an upcoming post.  Of course, none of this discussion is meant to minimize the importance of improved medication compliance.  It continues to be a critically important gap to address but employers and other plan sponsors should be provided realistic expectations about the economic value of improved compliance.