Agostino Colli, Mirella Fraquelli , Daniele Prati, Alessia Riva, Alessandra Berzuini, Dario Conte, Alessio Aghemo, Massimo Colombo, Giovanni Casazza Published: October 10, 2016
http://dx.doi.org/10.1371/journal.pone.0164452
Abstract
Introduction
In a perspective of economic constraints the prioritizing of patients to IFN-free regimens is mainly based on the determination of liver stiffness by transient elastography (TE). Being a continuous variable the interpretation of TE results requires the identification of cut-off values, to date set to maximize diagnostic accuracy even if such values should be better based on more helpful outcome prediction endpoints.
Aim
To define the TE cut-off values in different clinical scenarios, including new IFN-free regimens, and to balance the clinical benefits versus harms in treated and untreated patients.
Methods
We assessed the accuracy of TE in staging 728 consecutive HCV patients and the distribution of TE values in 1,001 blood donors. Ten experts quantified the expected harm/benefit ratio for 6 scenarios resulting from 2 stages of liver disease (F≥2 or F≥3) and 3 treatment regimens: PEGIFN+ribavirin, PEGIFN+RBV+first-generation protease inhibitor, and IFN-free regimens. The optimal TE cut-off values were identified using the Metz equation.
We assessed the accuracy of TE in staging 728 consecutive HCV patients and the distribution of TE values in 1,001 blood donors. Ten experts quantified the expected harm/benefit ratio for 6 scenarios resulting from 2 stages of liver disease (F≥2 or F≥3) and 3 treatment regimens: PEGIFN+ribavirin, PEGIFN+RBV+first-generation protease inhibitor, and IFN-free regimens. The optimal TE cut-off values were identified using the Metz equation.
The estimated mean expected harm/benefit ratio for IFN-free regimens was 1/8.3 in patients with F≥2 and 1/10 in those with F≥3. The resulting optimal cut-off values were respectively 4.5 kPa with sensitivity at 99% and specificity at 12%, and 6.8 kPa with sensitivity at 94% and specificity at 41%. These cut-off values are lower than those maximizing accuracy and allow to reduce the number of false negative results.
Conclusions
The optimal TE cut-off values to prioritize patients for IFN-free regimens, are sensibly lower than those used to maximize diagnostic accuracy.
Discussion Only
The access of patients to antiviral treatment, especially after the introduction of the highly effective but extremely expensive IFN-free regimens, is mainly based on the assessment of liver fibrosis stage.The consideration that a good level of cost effectiveness is obtained treating patients at an early stage of disease [13] and that chronic HCV infection also causes significant extra-hepatic morbidly [14], leads to the requirement of redefining the TE cut-off, switching from merely diagnostic criteria to disease-outcome-predicting endpoints, this implying an extensive hepatologic clinical judgment.
TE cut-off values actually serve the purpose of maximizing diagnostic accuracy according to Youden’s index, which is based on the sum of true positive and true negative results [12]. In this scenario, the downstream clinical consequences of testing are not explicitly evaluated, and the effect of the false positive and false negative results is implicitly considered equivalent. However, when there is the aim of setting a cut-off value in order to support clinical decision-making, the consequences of false results (i.e. from treating subjects who do not have the target condition and or from not treating subjects who have it) should be weighed, and the cut-off value should vary depending on the treatment options to avail.
Using chronic hepatitis C as an example, we have showed that the diagnostic test cut-off values can be defined with the aim of maximizing the benefit of treatment rather than overall diagnostic accuracy. The decisions concerning the treatment of patients with chronic hepatitis C are driven by the stage of fibrosis and liver disease severity [3, 13, 15–19] which can be accurately and safely assessed by measuring LS as an alternative to percutaneous liver biopsy. Furthermore, the results can be expressed as continuous variables, thus enabling the categorization of multiple cut-off values, which can be optimized for different clinical scenarios [8]. We have obtained two LS ROC curves for diagnosing significant and severe fibrosis/liver disease in patients with chronic hepatitis C, using liver histology as the reference standard. We have then identified the cut-off values maximizing treatment benefit in 6 therapeutic scenarios (Fig 1) by taking into account the harm/benefit ratio of the treatment defined by a group of expert clinicians and the prevalence of the target condition [6].
In the scenarios of IFN-based treatment, which is less effective and more harmful than IFN-free regimens, we have identified cut-off values that are close to those currently in use as calculated by maximizing diagnostic accuracy (7.65 kPa for significant fibrosis and 9.6 kPa for severe fibrosis) [4]. However, when the scenarios changed to those of IFN-free regimens, the optimal cut-off value decreased to 6.8 kPa when the aim was to treat patients with severe fibrosis, this figure being close to the upper reference limit obtained on the basis of the distribution of LS values in healthy blood donors (7.6 kPa) (Table 2). This indicates that, when highly effective and safe treatment options are available, clinicians favor sensitivity over specificity and end up electing for treatment patients with any degree of fibrosis. Furthermore, when the threshold was lowered in order to include patients with significant fibrosis, the optimal cut-off value (4.5 kPa) was close to the 50th percentile of the healthy donors, thus indicating that all people are potentially treated regardless of their LS values because the possibility of treating carriers without any degree of fibrosis is clinically acceptable.
As expected, our data shows that, in the case of a very low H/B ratio such as that of IFN-free regimens, the penalty from being false positive is more acceptable than that from being false negative. Furthermore, if the H/B ratio is close to one (as in the case of the two obsolete treatments), the same weight is attributed to false positive or negative test results, and the same cut-off value can maximize both accuracy and treatment benefit.
The application of the new cut-off values in a virtual cohort of 1,000 patients with chronic hepatitis C and a 30% prevalence of severe fibrosis increases the number of true positive results to 35 (with the consequent benefit of IFN-free treatment) at the expense of 310 more false positive results: i.e. the penalty from being false negative clearly outweighs the penalty from being false positive (Table 4). Similarly, in the case of significant fibrosis, the new cut-off value would lead to IFN-free treatment for further 313 patients (67 true and 246 false positive).
The estimated cut-off values obtained using Metz’s equation [11] has not only depended on the H/B ratio, but also on the prevalence of the target condition (severe or significant fibrosis) as a different prevalence leads to a different number of false results: in the case of a higher prevalence, there are usually more false negative results, and in the case of a lower prevalence, there are likely to be more false positive results. However, the prevalence of severe and significant fibrosis among the patients with chronic hepatitis C of this study appears to be quite similar to the rates observed also by other centers [9–12], and therefore the LS cut-off estimates are mainly due to the H/B ratios of the treatment options.
One major advantage of the clinically oriented approach described in this study is that any center can set its own cut-off value to reflect the prevalence of the target condition and balance the expected rates of false positive and negative results against treatment efficacy; the values could be even adjusted to an individual patient’s characteristics or preferences [20]. For example, a patient’s age which has not been taken into account in the 6 scenarios presented here for the sake of simplicity, is an important factor related to life expectancy and benefit estimation, and can be included in the model. Furthermore, the average of the H/B ratios estimated by different centers can help define homogenous rules against resource constraints, as in the case of very expensive IFN-free regimens. This approach has been suggested over 15 years ago for tuberculosis testing, mammography and palpation for breast cancer, Shotz tonometry for glaucoma, etc., but it has been very rarely applied in practice [10, 11]. There is a widespread perception that there needs to be a link between diagnosis and treatment, and it is generally accepted that tests should not be used unless their results change patient management [21]. The measurement of LS in patients with chronic hepatitis C seems to satisfy these requirements, and the same method of defining cut-off values can also be applied to other validated non-invasive tests for staging hepatitis C as FibroTest®, platelet ratio index (APRI) and FIB-4 or other liver diseases such as chronic hepatitis B.
A possible limitation of our study is that as healthy controls we have used a population of blood donors in whom the presence of latent liver disease cannot be absolutely excluded even against their repeated normal blood tests.
Another limitation is that we have compared the TE LS values with the histologic findings of liver biopsies, which are currently considered an imperfect reference standard as they lead to a false negative rate of 20% in the case of a diagnosis of severe fibrosis [23–27]. It is possible that some of the LS results were classified as false positive whereas they may be in fact true positive and false negative at biopsy histology, meaning that the specificity of LS measurements has been possibly underestimated despite our efforts to reduce biopsy sampling errors by accepting adequately sized specimens only. However, in order to counteract the possible underestimation of this specificity, the recommended cut-off values would have been even lower. Of note, we have used inclusion criteria requiring ALT elevation >1.5 ULN, which is no more required.
A further possible shortcoming of this study is that the estimated H/B ratios (Table 3) were obtained on the basis of the holistic clinical judgement of the specialists working in the participating centers, and are therefore a potential source of heterogeneity. However, it is unlikely that the alternative approach of building a more formal cost/benefit analysis would have addressed such an issue because there is little data concerning the natural history of hepatitis C or the extent of benefit or harm arising from treatment [22]. Interestingly, in the literature there are several examples supporting clinical judgment as a valid approach to complex clinical decisions [28, 29].
In conclusion, we suggest that a “one cut-off value fits all” approach is not appropriate, especially when the results of a single test are the major driver of a clinical decision, and that cut-off values should vary depending on the H/B profile of the proposed/available treatments. Given that for the IFN-free regimens of chronic hepatitis C treatment the benefits largely exceed the harms, the LS values justifying treatment initiation are lower than those computed on the basis of diagnostic accuracy, independently of the liver fibrosis score.
Specifically, our study emphasizes that in the setting of IFN-free regimens the values of LS indicating the need to start antiviral treatment are sensibly lower than those used only to maximize diagnostic accuracy as usually incorporated in the clinical guidelines. Thus, in the context of economic constraints the decision to treat or not to treat should be based on clinical considerations and the assessment of LS should be re-interpreted. From a practical viewpoint, when limited resources preclude the treatment of all patients, independently from the severity of the disease, an appropriate prioritization strategy is needed. Compared to the current cut-off set at 10 kPa chosen in most European countries, a new cut-off value defined on the basis of the expected harm/benefit ratio would reduce the number of inappropriately untreated patients, i.e. the false negative results.
On the other hand, in those national health system enjoying a more favorable level of resources and where the access to IFN-free regimens is broader, including also patients with mild to moderate, or absent fibrosis, LS determination by TE is not helpful or justified.
Continue reading...
Introduction
Materials and Methods
Results
Discussion
Acknowledgments
Author Contributions
References
No comments:
Post a Comment