Do Cost-Effectiveness Models Need a Reality Check?

In a thoughtful commentary published in the British Medical Journal, clinical researchers from Europe question the claims of cost-effectiveness made for many commonly used pharmacological treatments.    The authors argue that “…although there are claims that important preventive drugs such as statins, antihypertensives, and bisphosphonates are cost effective,^{6 7 8 9} there are no valid data on the effectiveness, and particularly the cost effectiveness, in usual clinical care. Despite this dearth of data, the majority of clinical guidelines and recommendations for preventive drugs rest on these claims.”

The authors cite a 2009 study, which examined a cost-effectiveness model of selective cyclo-oxygenase-2 (COX 2) inhibitors, as evidence of the weak external validity of claims of cost-effectiveness.  The COX-2 evaluation, which was based on a clinical trial, found that the cost of avoiding one adverse gastrointestinal event by switching patients from conventional non-steroidal anti-inflammatory drugs to COX-2 inhibitors was approximately $20,000.  In contrast, when the same analysis was conducted using the UK’s General Practice Research Database, which includes patients’ medical records in routine care, the cost of preventing one bleed was over $100,000.²

These findings are similar to work myself and others conducted several years ago on the COX-2s.  While the original cost-effectiveness model for the U.S. reported a cost per year of life saved (YLS) of about $19,000 for COX-2s when compared to non-selective NSAIDs, our revised model, which was based on actual practice, found a cost per YLS of $107,000. 

In a different study, we examined the external validity of a cost-effectiveness model of treatment options for eradication of h. pylori.  The original decision-analytic model found that the lowest cost per effectively treated patient was for the dual combination of PPI and clarithromycin ($980), whereas we found that the lowest cost per treatment was for the triple combination of bismuth, metronidazole, and tetracycline at a cost of $852.  Why the disconnect?  In the original h. pylori model, the authors had made assumptions about medication compliance and the cost of recurrence that simply did not hold up in the real-world. 

In the case of the COX-2s, the recent commentary concluded that the published cost-effectiveness analyses of COX 2 inhibitors neither had external validity nor represented the patients treated in clinical practice. They emphasized that external validity should be an explicit requirement for cost effectiveness analyses that are used to guide treatment policies and practices.  At least one academic modeler vehemently disagrees with the requirement of external validity, arguing that “it is wrong to insist that models be ‘validated’ by events that have not yet occurred; after all, the modeler cannot anticipate advances in technology, or changes in human behavior or biology. All that can be expected is that the model reflects the current state of knowledge in a reasonable way, and that it is free of logical errors.”

It is true that right when a drug comes to market, the only available data will likely be from the original clinical trials used to seek FDA approval, and the modeler will be forced to make numerous assumptions about compliance, costs, concomitant medication use, etc.  The problem is that the extent to which these assumptions are made without bias is unclear.  Research has shown that sponsorship by the pharmaceutical industry affects the results of economic models.  In a review published in 2010, researchers found that 95 percent of studies sponsored by pharmaceutical manufacturers reported favorable conclusions compared to only 50 percent of nonsponsored studies.  While it could be argued that this reflects a publication bias, the validation studies that I have described above suggest otherwise.  In each of these cases, there were key assumptions that drove model outcomes which, from a plan sponsor perspective, we found highly questionable at the time the model was first published.

Surprisingly, the issue of model validity receives relatively little attention given the central role that these models play in the field of pharmacoeconomics, as for example, in the AMCP dossier process.  The commentary authors argue that real-world comparative studies are the key to producing cost-effectiveness models that possess external validity.  This certainly will help with the quality of models post-FDA approval.  However, for models used at the time a drug is launched, ultimately, I expect that plan sponsors will have to develop their own models to ensure systematic bias is removed.

The Evidence Base for Step Therapy for Pharmaceuticals

While step therapy has been around for a decade or more, it still represents one of the lowest hanging fruit for plan sponsors who want to improve the value received from their pharmacy benefit with minimal member disruption.  Today more than 50 therapy classes have the opportunity for step therapy, including several specialty medication classes.

When I recently returned to pharmaceutical policy work, I was surprised to see how few evaluations of step therapy had been conducted in the last decade, particularly considering step therapy’s high degree of popularity among plan sponsors in both private and public settings.  I was also surprised to see that most of the evaluations of step therapy that had actually examined clinical and economic outcomes were funded by pharmaceutical manufacturers rather than managed care organizations that can provide thought leadership on this benefit tool.

In a paper published today in the Journal of Managed Care Pharmacy , I review the literature on step therapy and highlight important areas for future research.  Clearly, evaluations of step therapy are needed for numerous therapy classes of clinical significance, such as statins and specialty medications.  This is important because one would fully expect patient response and the clinical implications of patient choices to vary by therapy class and by the underlying indication.  The lack of evaluations of step therapy in the Medicare Part D population is a particularly notable gap.

Most of the research to date has focused on the drug cost savings of step therapy, a necessary condition for step therapy’s uptake but certainly not the only outcome of interest.  While savings from step therapy are widely known within healthcare organizations, a better understanding the clinical profile of patients who receive prior authorization for brand medications or receive no medication following a step edit is an important area of inquiry as it has the potential to affect not only economic outcomes, but clinical outcomes and member satisfaction. Perhaps the second most notable gap is the lack of evaluation of alternative forms that are growing in popularity, such as removal of grandfathering and integration of medical claims into the real-time step edits.

 In the paper, I also discuss some of the key methodological concerns with the publications to date and highlight examples of potential bias.  Based on this review, here are a few methodological tactics you should watch for when considering step therapy evaluations:

• Reporting of non-significant findings as if they were statistically significant
• Evaluation of all-cause medical expenses rather than disease-related expenses (all-cause medical costs are highly variable are have a greater chance of showing random differences across groups for reasons that have nothing to do with the program being evaluated)
• Inclusion of patients who were unaffected by the program to calculate drug savings, which will reduce the magnitude of apparent savings
• Examination of a small subpopulation of patients affected with extrapolations to the entire program

 As I point out in the paper, the popularity of step therapy among commercial, Medicaid, and Medicare plans is no doubt due to the wide availability of generic alternatives that offer significant savings, the strong clinical evidence that typically underlies these programs, and their ability to affect only new users, thereby minimizing member disruption. It is important that evaluations of step therapy keep pace with their growing use in order to optimize program design and patient outcomes.

Are Statins Cost-Effective for Primary Prevention?

There has been no shortage of headlines about the cost-effectiveness of statins in recent weeks, and another study was reported today in Journal of the American College of Cardiology. In a pharmacoeconomic model developed by Choudry and other researchers at Brigham and Women’s Hospital/Harvard Medical School using the JUPITER (Justification for the Use of statins in Prevention: An Intervention Trial Evaluating Rosuvastatin) trial, the authors project that the average JUPITER patient treated with rosuvastastin will have $7,900 higher life costs and an additional 0.31 quality-adjusted life years (QALYs), providing a cost-effectiveness ratio of $25,000/QALY. Against the widely used benchmark of $50,000/QALY, the authors conclude that statins appear to be cost-effective for primary prevention.

These results are somewhat contradictory to the recent Cochrane Collaboration review of statins in primary prevention, which recommended that statins be prescribed with caution to those at low risk of cardiovascular disease. The researchers reviewed data from 14 trials and nearly 35,000 patients. Although clinical benefits were found, the authors concluded that “ there was evidence of selective reporting of outcomes, failure to report adverse events and inclusion of people with cardiovascular disease. Only limited evidence showed that primary prevention with statins may be cost effective and improve patient quality of life.” The authors pointed out that all but one of the trials they reviewed were industry-sponsored and that you cannot simply extrapolate results from studies of people with history of heart disease to those without.

So why the differing conclusions about statin cost-effectiveness? In a commentary accompanying the latest study, Mark Hlatky, a physician and researcher at Stanford, provides some insights on the study. Hlatky points out that the model was only based on JUPITER and not the full breadth of evidence, which has NOT found large reductions in risk from the use of statins in primary prevention. Furthermore, the longer term risk reduction is simply unknown because clinical trials rarely last longer than 5 years. Choudhry assumed that rosuvastatin would reduce the risk of cardiac events by more than 50% for 15 years. If the same effect does not extend beyond 5 years, the cost-effectiveness grows to $62,100/QALY. The assumption of proportional risk reductions across levels of severity has been a limitation of many of the more recent cost-effectiveness analyses of statins.

There are additional questions about this and other recent studies of statins for primary prevention. Choudhry did not have data on long-term adverse effects, to which their model was quite sensitive. As Hlatky pointed out, if patients taking rosuvastatin had a 2% decrease in their well-being, the cost-effectiveness ratio grew to more than $62,000 per QALY. Also unclear is whether the model adjusted for medication persistency rates over the longer-term. Studies have shown that even after the first year of therapy, 50% of patients discontinue their statin medication, leading to increased short-term costs with little or no clinical benefit.

In hearing this latest information, many providers and plan sponsors are likely to point to diet and exercise changes as the solution to reducing the long-term risk of cardiovascular disease. Unfortunately, a recent Cochrane Collaboration review also found that education and counseling to encourage people to change their diets and stop smoking had little or no impact on deaths or disease caused by cardiovascular disease. An accompanying editorial pointed out that “Although various multiple prevention strategies exist, the most effective and cost-effective intervention for primary prevention in adults at low risk currently remains unclear.”

Do you know what percent of your current statin use is for primary prevention? Depending on the population, it could be as high as 50% of your statin users and may be growing.  If you are funding a wellness program to reduce the risk of cardiovascular disease, do you have any rigorous evidence that the program is leading to sustained changes in behavior?