LlV-4: A novel model for predicting transplant-free survival in critically ill cirrhotics

Christina C Lindenmeyer,Gianina Flocco,Neal A Mehta,William D Carey,Carlos Romero-Marrero,Department of Gastroenterology,Hepatology and Nutrition,Cleveland Clinic,Cleveland,OH 44195,United States

Vedha Sanghi,Ahyoung J Kim,Fadi Niyazi,Department of Internal Medicine,Cleveland Clinic,Cleveland,OH 44195,United States

Rocio Lopez,Department of Quantitative Health Sciences,Cleveland Clinic,Cleveland,OH 44106,United States

Aanchal Kapoor,Eduardo Mireles-Cabodevila,Department of Critical Care Medicine,Cleveland Clinic,Cleveland,OH 44195,United States

Abstract

Key words:Risk stratification;Resource allocation;Intensive care unit;Acute-on-chronic liver failure;Modeling;Mortality

INTRODUCTION

Patients with cirrhosis,particularly those with acute decompensation necessitating intensive care unit (ICU) admission,are at elevated risk for short-term mortality[1-3].Acute-on-chronic liver failure,as defined by sequential organ failure in patients with cirrhosis,portends a poorer prognosis,with 28-day mortality approaching 80% in patients with 3 or more organ failures[1,4-7]. The most recent data from the nationwide inpatient sample in the United States estimates that more than 26000 patients with cirrhosis are admitted to ICUs annually,of which less than half (about 47%) survive hospitalization[7,8]. Critical care for patients with cirrhosis is estimated to cost upwards of United States $3 billion annually,with each admission totaling on average United States $116200[8]. Survival analysis tools aid in the early identification of critically ill patients,which,when applied as part of therapeutic decision-making,can help guide goals of critical care discussions with patients and their families,and may contribute to cost-effective healthcare resource utilization[9].

To this end,the Acute Physiology and Chronic Health Evaluation (APACHE)methodology[10]and the Simplified Acute Physiology Score (APS)[11]are widely applied to estimate the risk of inpatient mortality based on values collected from within the first 24 hours of critical care admission. Similarly,the Sequential (or Sepsis-Related) Organ Failure Assessment (SOFA) is commonly used to describe,compare,and track a patient's clinical course in the ICU[9]. On the other hand,liver-specific scores,such as the Child-Pugh Score,Model for End-Stage Liver Disease (MELD),MELD-Sodium (MELD-Na) and Chronic Liver Failure-SOFA (CLIF-SOFA) are broadly applied in patients with liver disease to predict 90-day mortality,allocate donor organs for liver transplantation,and to define hepatic decompensation as well as acute-on-chronic liver failure[1,12-16]. Critical care scoring systems that include the assessment of organ dysfunction have generally performed as well,or better,in patients with cirrhosis than these liver-specific models for short-term mortality[1,17-28].However,none of these prognostic models have been widely accepted for use in clinical practice,perhaps due to complexity of calculation,or lack of universal variables readily available for cirrhotic patients in the ICU. We aimed to identify markers that best identify likelihood of transplant-free survival in critically ill patients with cirrhosis and to determine the performance of existing survival models.

MATERIALS AND METHODS

Study aims

The aims of this study are to: (1) Identify clinical and laboratory markers universally available at the time of ICU admission that best identify the likelihood of survival;and (2) To compare this model to existing survival models.

Study design

Patients over the age of 18 with a diagnosis of cirrhosis admitted between 2008-2014 to an ICU at a major quaternary referral and liver transplantation center in the United States comprised the training cohort. Patients from the APACHE IVb database (a prospective database of consecutive patients admitted to the ICU) were identified retrospectively by searching the database for the APACHE chronic health items (1)hepatic failure and (2) cirrhosis. The diagnosis of cirrhosis was subsequently confirmed either (1) radiographically,based on imaging evidence of cirrhosis or portal hypertension;(2) histologically by liver biopsy,if performed,and/or (3) by evidence of hepatic decompensation,including hepatic encephalopathy,variceal bleeding,or ascites. Patients with acute liver failure,history of liver transplantation,or who underwent liver transplantation during the contemporaneous hospital admission were excluded from the analysis.

Patient population

Demographic patient data consisting of age,gender,co-morbidities,etiology of chronic liver disease,and vital signs on admission to ICU were recorded from the electronic medical record. Clinical laboratory tests collected on admission to ICU included platelet count,prothrombin time (PT),International normalized ratio,lactate,arterial blood gas,pH,partial arterial pressure of carbon dioxide and oxygen,inspired oxygen concentration (FiO2),oxygen/FiO2,alveolar-arterial partial pressure oxygen gradient (A-a gradient),hematocrit,white blood cell count (WBC),potassium,blood urea nitrogen,albumin,sodium (Na),creatinine,bilirubin,bicarbonate,and glucose. Additional clinical parameters,including 24-hour urine output,need for mechanical ventilation,need for dialysis,variceal hemorrhage,Glascow coma scale,vasopressor dose,and degree of ascites and encephalopathy were recorded. This information was used to grade the severity of liver disease and prognosticate ICU mortality based on the calculation of previously validated liver-specific and ICU prognostic scores,including the MELD,MELD-Na,Child-Pugh,SOFA,CLIF-SOFA,Royal Free Hospital (RFH),APS and APACHE III scores. Subjects were followed from admission to hospital discharge or death.

The internal validation cohort was comprised of prospectively enrolled patients over the age of 18 with a diagnosis of cirrhosis admitted to the same institution as the training cohort between 2015-2017 and were subject to identical exclusion criteria. All patients that met the inclusion criteria were included in the analysis;no formal sample size calculations were done. The Institutional Review Board of the Cleveland Clinic Foundation reviewed and approved this study. On behalf of all authors,the corresponding author states that there is no conflict of interest.

Statistical analysis

A univariate and then multivariate analysis was performed to assess factors associated with in-hospital mortality. Data are presented as mean ± standard deviation,median (25th,75thpercentiles) orn(%). Analysis of variance or the nonparametric Kruskal-Wallis tests were used for continuous or ordinal variables and Pearson’s chi-square tests were used for categorical factors. In addition,Spearman correlations coefficients were used to assess correlation between length of stay and the different scores.

Receiver Operating Characteristics (ROC) analysis was performed to assess the value of various scores in predicting in-hospital mortality;areas under the ROC curves (AUROC) and corresponding 95% confidence intervals are presented.

A predictive model was developed using logistic regression analysis. An automated stepwise variable selection method performed on 1000 bootstrap samples was used to choose the final model. All variables known at time of ICU admission were considered for inclusion. Variables with inclusion rates of at least 50% were further assessed and the most parsimonious model with highest AUROC is reported.Variable transformations were assessed to account for any possible non-linearity.Observations with missing values were not included when building models.

After choosing the final model,the method described by Harrell[29]was used to compute the validation metric with over-fitting bias correction through bootstrap resampling. A thousand bootstrap samples (B = 1000) were drawn from the original data set and a new model with the same model settings was built on each bootstrap resample. Prediction on patients that were not chosen in the resample was calculated.An optimism factor was calculated over the 1000 new models and the bias-corrected validation metric was obtained by subtracting this optimism value from the AUROC directly measured from the original model. In addition,the Hosmer-Lemeshow goodness-of-fitχ2test and calibration plots were used to assess calibration of the models. DeLong’s method was used to compare predictive ability of LIV-4 to that of the various scores by comparing AUROCs[30]. A univariable analysis was performed to assess differences between the training and validation cohorts. SAS (version 9.4,The SAS Institute,Cary,NC,United States) was used for all analyses and aP＜ 0.05 was considered statistically significant. The statistical review was performed by a biomedical statistician.

RESULTS

Training cohort

Patient characteristics:Training Cohort. In total,436 patients cirrhotic patients,aged 57 ± 10.6 years,65.4% males,mostly with alcohol-related liver disease -(45.2%),Hepatitis C Virus -(33.7%) and Non-alcoholic steatohepatitis -(22%) related cirrhosis were included in the training cohort (Table 1). The majority of patients presented with severely decompensated liver disease,evidenced by the presence of moderate/severe encephalopathy (47.5%),moderate/severe ascites (44.3%),or variceal bleeding (25.7%)on admission,with median MELD score of 23.3 and Child-Pugh Score of 10.2 (C). 119 patients (27.3%) died in the hospital. The median ICU length of stay was 2.6 (25th,75thpercentiles: 1.4,5.2) d and the median hospital length of stay was 8.7 (4.7,16.8) d.

Factors associated with in-hospital mortality

Table1 summarizes univariable comparisons of subjects who died and those who were discharged alive. There was no significant difference in patient age,gender,ethnicity,etiology of liver disease or co-morbidities between survivors and nonsurvivors. Survivors had lower MELD (20.3vs31.1) and Child-Pugh (10.3vs11.9)scores. Variceal hemorrhage (P= 0.26),presence/grade of hepatic encephalopathy (P= 0.43),and presence/degree of ascites (P= 0.85) were not predictive of in-hospital mortality.

Patients who died in the hospital were more likely to require mechanical ventilation (49.6%vs35%,P= 0.005) and dialysis (12.6%vs6.3%,P= 0.031) on admission to the ICU than patients who survived. Patients who did not survive hospitalization had significantly lower mean arterial pressure (MAP),temperature and Glasgow coma scale (P＜ 0.001). Additionally,non-survivors were more likely to have lower hematocrit and bicarbonate,as well as higher WBC,A-a gradient,lactate,PT/International normalized ratio,potassium,blood urea nitrogen,creatinine,and bilirubin (P＜ 0.001). There was no significant difference in serum sodium or albumin levels between survivors and non-survivors (P= 0.81 and 0.57,respectively).

Table1 Training cohort: Patient characteristics and univariate analysis of factors associated with ln-hospital mortality

Values presented as Mean ± SD,Median (P25,P75) or n (column %). P values:1ANOVA.2Kruskal-Wallis test.3Pearson’s χ2 test.4Fisher’s Exact test. ICU: Intensive care unit;COPD: Chronic obstructive pulmonary disease;A1AT: Alpha 1 anti-trypsin deficiency;AIH: Autoimmune hepatitis;ALD: Alcoholic liver disease;HCV: Hepatitis C virus;HBV: Hepatitis B virus;NASH: Non-alcoholic steatohepatitis;PBC: Primary biliary cholangitis;PSC: Primary sclerosing cholangitis;INR: International normalized ratio;MAP: Mean arterial pressure;ABG: Arterial blood gas;PaCO2: Partial arterial pressure of carbon dioxide;PaO2: Oxygen;FiO2: Inspired oxygen concentration;A-a gradient: Alveolar arterial partial pressure oxygen gradient;FiO2/PaO2: Oxygenation index;GCS: Glascow coma scale;rpm: Respirations per minute;bpm: Beats per minute;WBC: White blood cell count;BUN:Blood urea nitrogen;MELD: Model for end-stage liver disease;Na: Sodium;CPS: Child-Pugh score;SOFA: Sequential (or sepsis-related) organ failure assessment;CLIF-SOFA: Chronic liver failure-sequential organ failure assessment;RFH: Royal free hospital;APS: Acute physiology score;APACHE: Acute physiology and chronic health evaluation.

In multivariate analysis,a combination of the natural logarithm (ln) of the bilirubin,PT,WBC,and MAP was found to most accurately predict in-hospital mortality. Based on the regression coefficients of the independent variables (Table 2),a novel model to predict inpatient mortality was established. The final proposed model was defined as:z= 1.19330 + [0.6137 × ln (bilirubin)] - (47.203/PT) + (0.0715 ×WBC) - (0.0198 ×MAP).Thezvalue is subsequently converted into a risk score to calculate probability of mortality utilizing the formula: LIV-4 = Probability of death (%) = [ez/(1 +ez)] × 100.Percentage values range from 0 to 100.

The Hosmer-Lemeshow goodness-of-fitX2test was 5.4 (P= 0.72) and the AUROC for this model was 0.86 (95%CI: 0.82-0.90). Using bootstrap resampling,internal validation of the model was undertaken and produced an AUROC of 0.85. Based on Youden's index[31]and using a cutoff of 26.5,the new score performed with a sensitivity of 81%,specificity of 76%,Positive Predictive Value of 58%,and Negative Predictive Value of 92%. Alternatively,a cutoff of 45.8 yields a sensitivity of 61% and specificity of 90%.

Comparison of prognostic models

Several scores demonstrated excellent accuracy for prediction of in-hospital mortality.The CLIF-SOFA and MELD scores,both liver-specific models,performed the best in the cohort with AUROCs of 0.81. The RFH score performed with an AUROC of 0.77.By comparison,ICU-specific scores,including the SOFA,APS and APACHE IIIperformed with AUROCs of 0.79,0.76,and 0.76,respectively. The liver intensive care unit variable-4 score (the LIV-4 score) performed higher than all other models,with an AUROC of 0.86. Figure 1 displays AUROCs of the top-performing scores. DeLonget al[30]s method was employed to compare the predictive ability of the new model to that of the other scores. The LIV-4 score performed significantly better than the MELD,MELD-Na,Child-Pugh Score,RFH and APACHE III scores (Table 3).

Table2 Training cohort: Factors associated with in-hospital mortality: multivariate logistic regression with variable transformations

Validation cohort

Table4 presents a comparison of the training and validation cohort characteristics. A total of 336 cirrhotic patients were admitted between 2015-2017,of whom 107 (31.8%)died. Patients in the internal validation cohort were substantially sicker,as evidenced by higher MELD,MELD-Na,APACHE III,SOFA and LIV-4 scores. Despite differences between the cohorts,the LIV-4 score remained significantly higher in subjects who expired during the hospital stay (Figure 2) and exhibited good prognostic values in the validation cohort with an AUROC of 0.80 (Figure 3). There was no statistically significant difference between the LIV-4 score's AUROC from the training cohort and the validation cohort (P= 0.11). In the validation cohort,the SOFA score performed with an AUROC of 0.78,the APACHE III with an AUROC of 0.74,the MELD score with an AUROC of 0.80,the MELD-Na with an AUROC of 0.79,the CLIF-SOFA with an AUROC of 0.83,and the RFH with an AUROC of 0.64. The LIV-4 model performed with a significantly higher AUROC than the RFH [AUROC: 0.64(0.56,0.72)],and was non-inferior to other ICU- and liver-specific scores (Table 5).Using a cutoff of 26.5,LIV-4 continued to perform with a high negative predictive value of 89.1 (84.6,93.6) (Table 6).

DISCUSSION

Our new model,the LIV-4 score,is calculated based on objective variables typically available at the time of ICU admission in patients with liver disease: The MAP,WBC,bilirubin,and PT. This combination of variables reflects hepatic and extra-hepatic(circulatory and immune) dysfunction,which are validated risk factors for mortality in patients with cirrhosis[32-34]. This score performed better in our training cohort as a predictor for short-term mortality than other ICU- and liver-specific models,including the SOFA,CLIF-SOFA,and RFH scores,with excellent discriminative ability and calibration. In our validation cohort,it performed better than the RFH and was non-inferior to all others. In addition,the LIV-4 provides a survival probability score. This survival probability calculation may be useful for critical care,hepatology and surgical specialists when addressing goals and expectations of critical care with patients and their families. The APACHE methodology,APS,and SOFA were developed to assess the clinical course and predict survival of all-comers admitted to the ICU[9-11]. Liver-specific scores,such as the Child-Pugh Score,MELD,MELD-Na and CLIF-SOFA are used to grade severity of liver disease,predict 90-day mortality,allocate organs for transplantation,and define acute-on-chronic liver failure[1,12-15].Liver-specific scores have been extrapolated for use as predictive models for mortality in the ICU,but have not performed better than ICU-specific scores[17-25]. In our study,the MELD and the CLIF-SOFA scores (both liver-specific scores and both with AUROCs of 0.81),performed better than ICU-specific scores,including the SOFA,APACHE III,and APS scores (AUROCs of 0.79,0.76,0.76,respectively). We postulate that the differences in our observations relate to critical care trends over time,with associated improved survival and lower event-deaths in more recent years. Our model was formulated in a more contemporary cohort than previous models and was subsequently prospectively validated,with objective variables that more accurately reflect current critical care challenges in the approach to the cirrhotic patient-most notably circulatory/ adrenocortical dysfunction and infection/inflammation[32-35].Variable mortality trends with time were also observed in the development of the updated RFH score[19]. Our mortality rates of 27.3% between 2008-2014 and 31.8%between 2015-2017 are similar to that of a comparable cohort from the Royal Free Hospital (2009-2012;35.4%)[19],as well as the cohort of patients described in the development of the CLIF-SOFA score (2011;29.7% in patients with acute-on-chronic liver failure)[1].

Figure1 Training cohort: Predictive scores for in-hospital mortality in cirrhotic patients.

The RFH score is a liver-specific ICU score that has been previously externally validated in several centers in Scotland[26]. Our study is the first in the United States to validate the updated RFH score,which performed in our cohort with an AUROC of 0.77. However,we found that the RFH score was limited in its generalizability as lactate and A-a gradient were not universally available on admission in our cohort.Lactate has been shown to be an independent predictor of mortality in cirrhotic patients[18,19,26,27,36,37]and in patients with acute liver failure[38]admitted to the ICU.However,lactate clearance has also been shown to be impaired by liver and extrahepatic organ dysfunction,as evidenced by decreased clearance with increasing LSOFA score[39],which suggests that lactate levels may not be reliable in cirrhotic patients. Similarly,arterial blood gas analysis and calculation of the A-a gradient is more likely to be collected in patients with respiratory failure necessitating mechanical ventilation,and is not universally available in patients admitted to the ICU as the precise FiO2is often unknown. Finally,variceal hemorrhage as a reason for admission to the ICU was not an independent predictor of in-hospital mortality in our cohort. For these reasons,in an effort to create a widely applicable score for all cirrhotic patients admitted to the ICU,we did not include lactate,arterial blood gas analysis,or variceal hemorrhage in our new prognostic model. The LIV-4 performs with better discrimination and calibration in all patients with cirrhosis admitted to our ICU,independent of variceal hemorrhage,presence/grade of encephalopathy,and presence/degree of ascites.

In terms of limitations,patients were identified from the prospectively developed ICU APACHE IVb database and data was collected retrospectively. It is possible that all consecutive patients with cirrhosis were not captured with our retrospective methodology as a consequence of coding error,or if cirrhosis was not recognized as a pre-existing chronic health condition on admission to ICU. While internal prospective validation at our center suggests that the LIV-4 score will be widely applicable,we advocate for external,prospective analyses to be undertaken across diverse ICU settings in an effort to validate the clinical applicability of the score. Finally,it isimportant to recognize that,much as the APACHE scoring system has evolved to reflect progressive trends in the practice of critical care medicine,temporal study for re-calibration of LIV-4 will be necessary.

Table3 Training cohort: Predictive abilities of critical care and liver-specific scores compared to the LlV-4 score

Patients with cirrhosis admitted to the ICU present unique clinical challenges for the clinician,and are best managed by a multidisciplinary team,comprised of specialists in both critical care and hepatology[8]. Prognostication of short-term survival is important in order to identify patients at highest risk for mortality in terms of allocation of resources,studies and interventions. We report the development and prospective validation of a new prognostic model for the prediction of inpatient transplant-free survival in a contemporary cohort of cirrhotic patients admitted to the ICU. This tool can be easily accessed online at http://riskcalc.org:3838/LIV-4/. If external validation is undertaken,the LIV-4 score could become a standard clinical tool in the ICU and maybe used as a means of stratifying critically ill patients with cirrhosis in clinical and translational research studies.

Table4 Validation cohort characteristics

Table5 Validation cohort: Comparison of the various scores and LlV-4

Table6 Validity measures for LlV-4

Figure2 LlV-4 score is higher in subjects who expired during the hospital admission.

Figure3 Validation cohort: Predictive scores for in-hospital mortality in cirrhotic patients.

ARTICLE HIGHLIGHTS

Research background

Critically ill patients with cirrhosis have higher mortality rates in the intensive care unit (ICU)than patients without chronic liver disease. Prognostication of short-term mortality is important in order to identify patients at highest risk of death. None of the currently available prognostic models have been widely accepted for use in cirrhotic patients in the ICU,perhaps due to complexity of calculation,or lack of universal variables readily available for these patients.

Research motivation

We believe a simple and widely applicable survival model can be developed,to guide therapeutic decision-making and contribute to cost-effective healthcare resource utilization.

Research objectives

To identify clinical and laboratory markers universally available at the time of ICU admission that best identify the likelihood of transplant-free survival in critically ill patients with cirrhosis.

Research methods

A new predictive model (the LIV-4 score) was developed retrospectively using logistic regression analysis from a large cohort of critically ill patients with cirrhosis admitted to a quaternary care liver transplant center ICU and was prospectively validated in a cohort of patients admitted to the same institution.

Research results

Our validated model for predicting mortality in cirrhotic patients on admission to the ICU performs better than previously published liver and ICU-specific scores.

Research conclusions

LIV-4 could become a standard clinical tool for patients with advanced liver disease in the ICU and could be used as a means of stratifying critically ill cirrhotic patients in clinical research studies.

Research perspectives

Survival modeling is an important tool for therapeutic decision-making as well as for research study design. The LIV-4 score was designed and validated prospectively in a single-center cohort. External,prospective validation is needed to determine widespread applicability and utility of the model.